Here is [a Link](http://stackoverflow.com/questions/32448987/how-to-retrieve-a-very-big-cassandra-table-and-delete-some-unuse-data-from-it#comment52844466_32464409) of my question before.
After I get the cassandra data row by row in my program, I'm confused by the convert between cassandra row to java class. In java class the table of cassandra is convert to a ResultSet class,when I iterator it and get the row data,it returns a NPE. In fact,I can see the Object (or the data) while debuging the program. Here is My Iterator Code:
ResultSet rs=CassandraTools.getInstance().execute(cql);
Iterator<Row> iterator = rs.iterator();
while (iterator.hasNext()) {
Row row = iterator.next();
row.getString() ---->return NPE
The CassandraTools class is:
public class CassandraTools {
private static CassandraTools instance;
private CassandraTools() {
}
public static synchronized CassandraTools getInstance() {
if (instance == null) {
instance = new CassandraTools();
instance.init();
}
return instance;
}
Cluster cluster;
Session session;
public void init() {
if (cluster == null) {
cluster = new Cluster.Builder().addContactPoint("10.16.34.96")
.build();
if (session == null) {
session = cluster.connect("uc_passport");
}
}
}
public ResultSet execute(String cql) {
ResultSet rs = session.execute(cql);
// rs.forEach(n -> {
// System.out.println(n);
// });
return rs;
}
}
SO how could I convert the data in the row to A java Class?I have read the convert class in the API of spring data cassandra,but it is complicated to use for me. Who can help?
IMHO, If you want to map the rows of Cassandra to a java class, you should try to use an Object-Datastore mapper which does these things for you.
If you try to do this by yourself, you need to handle the java-cassandra datatype mappings, validations etc all by yourself which is very hectic job.
There are few (Kundera, Hibernate OGM, etc) opensource object-datastore mappers available and you can use them. I suggest you to try Kundera and check this for getting started with Cassandra.
Related
We are migrating our code base from spark-java to PySpark. We were handling custom aggregations for merging data using preCombine() and combineAndGetUpdateValue() and had implemented this in our Spark-Java code. Example below:
package com.paytm.sparkjobs.utils.hudi;
public class MergeMdrPayloadAndPersist extends BaseAvroPayload implements HoodieRecordPayload<MergeMdrPayloadAndPersist> {
public static final Logger logger = LoggerFactory.getLogger(MergeMdrPayloadAndPersist.class);
private GenericRecord record = null;
public MergeMdrPayloadAndPersist(GenericRecord record, Comparable orderingVal) {
super(record, orderingVal);
this.record = record;
}
#Override
public MergeMdrPayloadAndPersist preCombine(MergeMdrPayloadAndPersist mergeMdrPayloadAndPersist) {
//custom logic for aggregations
return new MergeMdrPayloadAndPersist(mergeMdrPayloadAndPersist.record, mergeMdrPayloadAndPersist.orderingVal);
}
#Override
public Option<IndexedRecord> combineAndGetUpdateValue(IndexedRecord indexedRecord, Schema schema) throws IOException {
//custom logic for aggregations
MergeMdrPayloadAndPersist mergedDoc = new MergeMdrPayloadAndPersist(inputPayload.record, inputPayload.orderingVal);
return mergedDoc.getInsertValue(schema);
}
#Override
public Option<IndexedRecord> getInsertValue(Schema schema) throws IOException {
if (this.recordBytes.length == 0) {
return Option.empty();
} else {
IndexedRecord indexedRecord = HoodieAvroUtils.bytesToAvro(this.recordBytes, schema);
return this.isDeleteRecord((GenericRecord)indexedRecord) ? Option.empty() : Option.of(indexedRecord);
}
}
private boolean isDeleteRecord(GenericRecord genericRecord) {
Object deleteMarker = genericRecord.get("_hoodie_is_deleted");
return deleteMarker instanceof Boolean && (Boolean)deleteMarker;
}
}
Can I know how do we write a custom Payload class/function in python to handle our aggregation and merging logic? Some code examples would help.
There is no way to achieve this with pyspark, where Hudi doesn't have its own python API, it uses spark python API to interact with its java/scala classes, which is based on py4j, and you cannot create a java class using py4j because the java class needs to be created before compiling the java code.
The best way is creating a small java jar containing your classes, and adding it to your pyspark shell/submit.
Is there any way in which using Spring Data a query can be executed on all keyspaces in Cassandra?
There are two parts to this answer:
When using Spring Data Cassandra 1.x, you are need to setup individual CassandraTemplate instances for each keyspace you want to use.
With Spring Data Cassandra 2.x, we introduced the SessionFactory interface to control which Session to use. We ship with routing SessionFactory support so you can provide multiple sessions and a discriminator (usually something ThreadLocal-based) to select the appropriate Session.
Some example code for 2.0 would look like:
class MyRoutingSessionFactory extends AbstractRoutingSessionFactory {
ThreadLocal<String> lookupKey = ThreadLocal.withInitial(() -> "default-session");
void setLookupKey(String lookupKey) {
this.lookupKey.set(lookupKey);
}
#Override
protected Object determineCurrentLookupKey() {
return lookupKey.get();
}
}
class MyConfig extends AbstractCassandraConfiguration {
#Bean
#Override
public SessionFactory sessionFactory() {
MyRoutingSessionFactory factory = new MyRoutingSessionFactory();
factory.setDefaultTargetSessionFactory(getRequiredSession());
MapSessionFactoryLookup lookup = new MapSessionFactoryLookup();
Session myOtherSession = …;
lookup.addSessionFactory("default-session", getRequiredSession());
lookup.addSessionFactory("my-other-session", myOtherSession);
factory.setSessionFactoryLookup(lookup);
return factory;
}
// …
}
i am using hazelcast 3.6.1 and implementing distinct aggregate functionality using custom mapreduce to get solr facet kind of results.
public class DistinctMapper implements Mapper<String, Employee, String, Long>{
private transient SimpleEntry<String, Employee> entry = new SimpleEntry<String, Employee>();
private static final Long ONE = Long.valueOf(1L);
private Supplier<String, Employee, String> supplier;
public DistinctMapper(Supplier<String, Employee, String> supplier) {
this.supplier = supplier;
}
#Override
public void map(String key, Employee value, Context<String, Long> context) {
System.out.println("Object "+ entry + " and key "+key);
entry.setKey(key);
entry.setValue(value);
String fieldValue = (String) supplier.apply(entry);
//getValue(value, fieldName);
if (null != fieldValue){
context.emit(fieldValue, ONE);
}
}
}
and mapper is failing with NullPointerException. and sysout statement says entry object is null.
SimpleEntry : https://github.com/hazelcast/hazelcast/blob/v3.7-EA/hazelcast/src/main/java/com/hazelcast/mapreduce/aggregation/impl/SimpleEntry.java
Can you point me the issue in the above code ? Thanks.
entry field is transient. This means that it is not serialized, so when DistinctMapperobject is deserialized on hazecalst node, it's value is null.
Removing the transient will solve the NullPointerException.
On the side note:
Why do you need this entry field? It doesn't seem to have any use.
This has baffled me for a while now and I cannot seem to get the grasp of it. I'm using Cell Value Factory to populate a simple one column table and it does not populate in the table.
It does and I click the rows that are populated but I do not see any values in them- in this case String values. [I just edited this to make it clearer]
I have a different project under which it works under the same kind of data model. What am I doing wrong?
Here's the code. The commented code at the end seems to work though. I've checked to see if the usual mistakes- creating a new column instance or a new tableview instance, are there. Nothing. Please help!
//Simple Data Model
Stock.java
public class Stock {
private SimpleStringProperty stockTicker;
public Stock(String stockTicker) {
this.stockTicker = new SimpleStringProperty(stockTicker);
}
public String getstockTicker() {
return stockTicker.get();
}
public void setstockTicker(String stockticker) {
stockTicker.set(stockticker);
}
}
//Controller class
MainGuiController.java
private ObservableList<Stock> data;
#FXML
private TableView<Stock> stockTableView;// = new TableView<>(data);
#FXML
private TableColumn<Stock, String> tickerCol;
private void setTickersToCol() {
try {
Statement stmt = conn.createStatement();//conn is defined and works
ResultSet rsltset = stmt.executeQuery("SELECT ticker FROM tickerlist order by ticker");
data = FXCollections.observableArrayList();
Stock stockInstance;
while (rsltset.next()) {
stockInstance = new Stock(rsltset.getString(1).toUpperCase());
data.add(stockInstance);
}
} catch (SQLException ex) {
Logger.getLogger(WriteToFile.class.getName()).log(Level.SEVERE, null, ex);
System.out.println("Connection Failed! Check output console");
}
tickerCol.setCellValueFactory(new PropertyValueFactory<Stock,String>("stockTicker"));
stockTableView.setItems(data);
}
/*THIS, ON THE OTHER HAND, WORKS*/
/*Callback<CellDataFeatures<Stock, String>, ObservableValue<String>> cellDataFeat =
new Callback<CellDataFeatures<Stock, String>, ObservableValue<String>>() {
#Override
public ObservableValue<String> call(CellDataFeatures<Stock, String> p) {
return new SimpleStringProperty(p.getValue().getstockTicker());
}
};*/
Suggested solution (use a Lambda, not a PropertyValueFactory)
Instead of:
aColumn.setCellValueFactory(new PropertyValueFactory<Appointment,LocalDate>("date"));
Write:
aColumn.setCellValueFactory(cellData -> cellData.getValue().dateProperty());
For more information, see this answer:
Java: setCellValuefactory; Lambda vs. PropertyValueFactory; advantages/disadvantages
Solution using PropertyValueFactory
The lambda solution outlined above is preferred, but if you wish to use PropertyValueFactory, this alternate solution provides information on that.
How to Fix It
The case of your getter and setter methods are wrong.
getstockTicker should be getStockTicker
setstockTicker should be setStockTicker
Some Background Information
Your PropertyValueFactory remains the same with:
new PropertyValueFactory<Stock,String>("stockTicker")
The naming convention will seem more obvious when you also add a property accessor to your Stock class:
public class Stock {
private SimpleStringProperty stockTicker;
public Stock(String stockTicker) {
this.stockTicker = new SimpleStringProperty(stockTicker);
}
public String getStockTicker() {
return stockTicker.get();
}
public void setStockTicker(String stockticker) {
stockTicker.set(stockticker);
}
public StringProperty stockTickerProperty() {
return stockTicker;
}
}
The PropertyValueFactory uses reflection to find the relevant accessors (these should be public). First, it will try to use the stockTickerProperty accessor and, if that is not present fall back to getters and setters. Providing a property accessor is recommended as then you will automatically enable your table to observe the property in the underlying model, dynamically updating its data as the underlying model changes.
put the Getter and Setter method in you data class for all the elements.
I am working with JSF 2.1 and Primefaces 3.3. I am using the primefaces tree component for crating the tree from Database. I wanted to sort the tree nodes in alphabetical order at all levels. Please help me on this.
we had problems with the sorting via Comparator and found out, that there is a handy PrimeFaces TreeUtils.sortNode( TreeNode, Comparator ) class already provided which works like a charm :)
You would have to sort Primefaces DefaultTreeNode objects at the ManagedBean using Collections.sort and a Comparator class.
public TreeNodeComparator() implements Comparator<TreeNode> {
public int compare(TreeNode n1, TreeNode n2) {
// This assumes the tree node data is a string
return n1.getData().compareTo(n2.getData());
}
}
In your managed bean you will need to assemble your child lists without adding their parents yet. That can come later. For right now build your child lists out for each level and set the parent to null;
TreeNode node1 = new DefaultTreeNode("node1", null);
TreeNode node2 = new DefaultTreeNode("node2", null);
TreeNode child1node1 = new DefaultTreeNode("zgnagn", null);
TreeNode child2node1 = new DefaultTreeNode("vvnieeianag", null);
TreeNode child1node2 = new DefaultTreeNode("cajkgnagair", null);
TreeNode child2node2 = new DefaultTreeNode("ajaavnagwokd", null);
rootNodeChildren.add(node1);
rootNodeChildren.add(node2);
node1Children.add(child1node1);
node1Children.add(child2node1);
node2Children.add(child1node2);
node2Children.add(child2node2);
The reason why we are setting everything to null is because when the parent is set on the DefaultTreeNode it is added to the parents child list. The order in which you set a nodes parents determines the order they will appear in the Tree component.
Knowing that we can use our comparator to sort each list individually.
Collections.sort(rootNodeChildren, new TreeNodeComparator());
Collections.sort(node1Children, new TreeNodeComparator());
Collections.sort(node2Children, new TreeNodeComparator());
Now all of the lists are sorted so we can loop through and the appropriate parents one at a time. You can probably write an algorithm to determine this or you can keep a separate data stucture that builds the tree hierarchy without adding to the list.
Another way, and probably easier overall, is to just override the DefaultTreeNode class and give it a sort method:
public SortableDefaultTreeNode extends DefaultTreeNode {
public void sort() {
TreeNodeComparator comparator = new TreeNodeComparator();
Collections.sort(this.children, comparator);
for (TreeNode child : children) {
child.sort();
}
}
}
Now you can just build your TreeNodes out and then call root.sort() and it will recursively sort all of its children at each level alphabetically.
You could also use a generic comparable TreeNode aproach such as:
Base was taken from primefaces DefaultTreeNode, unmodified changes are left out in below code.
If child's should not be restricted on T one can use TreeNodeComparable<T extends Comparable<?>> and cast to Comparable in compareTo() method.
public class TreeNodeComparable<T extends Comparable<T>> implements TreeNode, Serializable,
Comparable<TreeNodeComparable<T>>
{
private static final long serialVersionUID = ...;
private T data;
private List<TreeNodeComparable<T>> children;
public TreeNodeComparable(final String type, final T data, final TreeNodeComparable<T> parent)
{
this.type = type;
this.data = data;
this.children = (List) new TreeNodeChildren(this);
if (parent != null)
parent.getChildren().add(this);
}
/**
* Comparison only depends on the underlying data
*
* #see ObjectUtils#compare(Comparable, Comparable)
*/
#Override
public int compareTo(final TreeNodeComparable<T> node)
{
if (node == null)
throw new NullPointerException("node");
return ObjectUtils.compare((T) this.getData(), (T) node.getData());
}
/**
* Recursively sorts the complete tree.
*/
public void sort()
{
Collections.sort(this.children);
for (final TreeNodeComparable<T> child : this.children)
{
child.sort();
// must reset parent due to PF problems
// http://forum.primefaces.org/posting.php?mode=reply&f=3&t=39752
child.setParent(this);
}
}
#SuppressWarnings("unchecked")
#Override
public boolean equals(final Object obj)
{
if (this == obj)
return true;
if (obj == null || this.getClass() != obj.getClass())
return false;
final TreeNodeComparable<T> other = (TreeNodeComparable<T>) obj;
return ObjectUtils.equals(this.data, other.data);
}
#Override
public int hashCode()
{
return new HashCodeBuilder().append(this.data).toHashCode();
}
public void setData(final Object data)
{
if (data != null && !(data instanceof Comparable))
throw new IllegalArgumentException();
this.data = (T) data;
}
#SuppressWarnings(
{
"unchecked", "rawtypes"
})
public List<TreeNode> getChildren()
{
return (List) this.children;
}
}