Let's say I have two HBase cells:
x:y
x:z
How do I do the equivalent of this SQL:
SELECT * FROM some_table WHERE x_y = ? AND x_z = ?
This is the (Groovy) code I have for generating the basic filters:
static SingleColumnValueFilter makeColumnFilter(String family, String qualifier, String expectedValue) {
new SingleColumnValueFilter (
Bytes.toBytes(family),
Bytes.toBytes(qualifier),
CompareFilter.CompareOp.valueOf('EQUAL'),
new SubstringComparator(expectedValue))
}
def filterz = filters.collect {
makeColumnFilter(it.family, it.qualifier, it.expectedValue)
}
def fl = new FilterList(filterz)
def scan = new Scan()
scan.setFilter(fl)
def family = 'x'.bytes
t.getScanner(scan).each {
println "${Bytes.toString(it.getValue(family, 'y'.bytes))}"
count++
}
The print statement shows nothing but nulls even though I'm passing in x for the family value and y/z for the qualifiers. It appears to not be filtering the values. What am I doing wrong?
You need to filter the rows if the column is not found using setFilterIfMissing.
Change the makeColumnFilter to :
static SingleColumnValueFilter makeColumnFilter(String family, String qualifier, String expectedValue) {
def colFilter = new SingleColumnValueFilter (
Bytes.toBytes(family),
Bytes.toBytes(qualifier),
CompareFilter.CompareOp.valueOf('EQUAL'),
new SubstringComparator(expectedValue))
colFilter.setFilterIfMissing(true)
colFilter
}
I think SingleColumnValueExcludeFilter should do the job.
Scan scan = new Scan();
SingleColumnValueExcludeFilter singleColumnValueFilterY = new SingleColumnValueExcludeFilter("x".getBytes(),
"y".getBytes(), CompareOp.EQUAL, new BinaryComparator("valueY".getBytes()), true, true);
SingleColumnValueExcludeFilter singleColumnValueFilterZ = new SingleColumnValueExcludeFilter("x".getBytes(),
"z".getBytes(), CompareOp.EQUAL, new BinaryComparator("valueZ".getBytes()), true, true);
FilterList filterList = new FilterList(Operator.MUST_PASS_ALL);
filterList.addFilter(singleColumnValueFilterX);
filterList.addFilter(singleColumnValueFilterY);
scan.setFilter(filterList);
More detailed documentation on HBase filters could be found here.
Hope this will help.
Related
I need to get the list of column qualifiers available in a HBase table.
Suppose I have a table 'Customers' with column 'info:Age' and 'contact:PhoneNo'. To get the list of column families I see there's a method 'getColumnDescriptors' which returns the value 'info' and 'contact'.
But how to retrieve the full column names 'info:Age' and 'contact:PhoneNo' from the table or atleast the qualifiers 'Age' and 'PhoneNo' alone.
With the method 'getRowWithColumns' I am able to get the list of column names where I have to pass the row key value.
Will I be able to achieve this through any other convenient way?
Here is a working solution in Java as you asked, you just have to change a few things to translate it to C++.
Basically this method will scan the table and retrieves the column qualifiers, then I add them into a list if this list does not already contains it.
Here, I look at all the rows, but if all your rows always have the same columns, you can just scan the first row, using a Get for example (look at the HBase documentation, I've written several examples there).
public ArrayList<String> getColumnName(String tablename) {
ArrayList<String> properties = new ArrayList<String>();
try {
Table table = this.connection.getTable(TableName.valueOf(tablename));
Scan scan = new Scan();
ResultScanner rs = table.getScanner(scan);
try {
for (Result r = rs.next(); r != null; r = rs.next()) {
for (Cell c : r.rawCells()) {
String family = new String(CellUtil.cloneFamily(c));
String qualifier = new String(CellUtil.cloneQualifier(c));
System.out.println("Column Family : "+ family);
System.out.println("Column Qualifier : " + qualifier);
if (!properties.contains(qualifier))
properties.add(new String(CellUtil.cloneQualifier(c)));
}
}
} finally {
rs.close(); // always close the ResultScanner!
}
} catch (IOException e) {
e.printStackTrace();
}
return properties;
}
I have an Access database which I need to retrieve all fields except the first and last and display it in a JTable. Everything works perfectly fine when I create my Object[][] but when i return it, i get a NullPointerException. I tried to find where there could be a null value in the database by printing the whole object out but that works fine and no values are null. Why would returning the Object[][] give me a NullPointerException and how can i fix it?
the stack trace is:
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
public Object [] [] SetTrainingLogTable() throws SQLException
{
DatabaseConnection connection = new DatabaseConnection();
//Retrieves all the data from the TrainingLog table
ResultSet resultset = connection.SelectStatements("SELECT * FROM TrainingLog");
//Retrieves the number of entries
ResultSet numberofworkouts = connection.SelectStatements("SELECT COUNT(*) FROM TrainingLog");
int count = numberofworkouts.getInt(1);
number = count;
String[][] table = new String [count] [6];
//Number to incriment for while loops
int row = 0;
String date = "";
while(row<count)
{
date = resultset.getString(2);
table [row][0] = calculate.RefineDate(date);
table [row][1] = resultset.getString(3);
table [row][2] = resultset.getString(4);
table [row][3] = resultset.getString(5);
table [row][4] = resultset.getString(6);
table [row][5] = resultset.getString(7);
resultset.next();
row++;
}
Object[][] data = table;
connection.close();
return data;
}
I ran a debugger and it only gives the error when the return line is run.
It's best to post the stack trace and tell which line is raising the error. However, the typical way of writing such code is:
Connection con = ...;
Statement st = ...;
ResultSet rs = ...;
while (rs.next()) {
// ...
}
The result set starts out pointing before the first row. rs.next() returns whether there is a next row, and advances to it if it exists. Can you rewrite it in that style?
Other suggestions:
Can you create an actual object type instead of using Object[] to store the data from each row? Call it Workout.
Can you use a List<Workout> instead of your Object[][]?
Is the date stored in the database as a SQL DATE or TIMESTAMP? Then, don't convert it to a Java String: use java.sql.Date or java.util.Date. At work, I have a large old program that uses strings for dates, and it uses different formats to convert the values at different times. It's pretty miserable.
Don't use SELECT *. Give the names of the columns to return. Use the rs.getString("column_name") syntax.
There's no need to set one variable to the returned table and immediately set another variable to it.
Closing the connection or statement should be done in a finally block, or by try-with-resources.
I want to use Groovy with JDBC to load some data from a table. I then want to copy the properties across where the property names match. How can I do this in Groovy?
Something like this:
sql.eachRow("select * from temp_table") {
def e = new MyJavaClass()
// copy matching fields from it to e
}
In addition to topchef's answer, you might be able to use some groovy map magic
If you limit your sql to the properties in your Java Class (and assuming you can hold the entire result in memory at once), then you should be able to use the rows method to get a List of GroovyRowResult objects (one per row). As this class is an instance of Map, groovy will use it to construct your java object automatically for you:
class MyJavaClass {
String property1, property2, property3
}
sql.rows("select property1, property2, property3 from temp_table").each { row ->
MyJavaClass e = row
}
Some groovy magic helps:
def filtered = ['property1', 'property2', ....]
sql.eachRow("select * from temp_table") {
def e = new MyJavaClass(it.properties.findAll{filtered.contains(it.key)})
}
Given that
list filtered contains all property names that you intend to copy;
your table column names correspond to MyJavaClass declared property names (the same as in the list filtered);
MyJavaClass has both default (empty) constructor and constructor
that takes all properties as parameters;
MyJavaClass declares public setters for properties;
E.g:
public MyJavaClass() {}
public MyJavaClass(String property1, String property2, ....) {
this.property1 = property1;
this.property2 = property2;
.... }
public void setProperty1(String property1) {this.property1 = property1;}
public void setProperty2(String property2) {this.property2 = property2;}
....
You may use filtered as a list of undesired properties like this:
def filtered = ['class', 'metaClass', ....]
then:
def e = new MyJavaClass(it.properties.findAll{!filtered.contains(it.key)})
I am Unit Testing one of my functions. Here is my code:
public void TestLabels()
{
//Step 1: Creating a mock table with columns exactly like in the real table.
DataTable table = new DataTable();
DataRow mydatarow;
mydatarow = table.NewRow();
//Step 2: Adding the row as same as the Real Data!
mydatarow["Name"] = "Test";
mydatarow["Address"] = "00000 ST.";
mydatarow["ZipCode"] = "77665";
mydatarow["Tracking#"] = "";
table.Rows.Add(mydatarow);
foreach (DataColumn column in table.Columns)
Console.WriteLine(column.ColumnName);
//Step 3: Call method we are testing.
var updateTable = IceTechUPSClient.Instance.CreateLabels(table);
foreach (DataRow row in updateTable.Rows)
{
var trackingNumber = row["Tracking#"].ToString();
Assert.IsFalse(String.IsNullOrWhiteSpace(trackingNumber), "Expecting tracking number generated for every row!");
Assert.IsTrue(File.Exists(trackingNumber + ".gif"));
}
}
Now I am getting an error: Column 'Name' does not belong to table. As you can see I have specified column name "Name" here and also added that particular row. Then why I am getting this error? Any help?
Thanks!
You haven't set up your columns (unless you've missed out some code in your example).
You need to create the columns with the required names before you can access them like this:
var columnSpec = new DataColumn
{
DataType = typeof(string),
ColumnName = "Name"
};
this.table.Columns.Add(columnSpec);
When you read data from the database if you've set AutoGenerateColumns to true (the default) you don't need to do this explicitly as it's done for you behind the scenes.
I need help creating a LINQ SQL with subsonic. First the basics, this works fine:
var query = (from o in bd.concelhos
orderby o.descricao
select o);
var results = query.ToList<concelhos>();
However, I want to filter out some columns and I have created the following code:
var query = (from o in bd.concelhos
orderby o.descricao
select new FilteredConcelhos { id = o.idDistrito + "/" + o.idConcelho, descricao = o.descricao });
var results = query.ToList<FilteredConcelhos>();
which errors out in the ToList method with the description "Sequence contains no matching element"
Any help would be great with this...
update:
Turns out I was missing get set attributes in the newly declared class...
Like so
public class FilteredConcelhos
{
public string id { get; set; }
public string descricao { get; set; }
}
This clears the exception, but the resulting List is still all wrong (FilteredConcelhos.id contains nothing and FilteredConcelhos.descricao contains numbers)
Can you try to first execute the ToList and the select afterwards - then the select is performed via linq 2 objects!
Have you tried to work with an anonymous type?
var query = (from o in bd.concelhos
orderby o.descricao
select new { id = o.idDistrito + "/" + o.idConcelho,
descricao = o.descricao });
var results = query.ToList();
Unfortunately, this happened to me a lot. I'm not sure about the details of how Linq 2 Object works, but if you'll call ToList on the original object, like this:
from o in bd.concelhos.ToList()
...
It should do the trick.