How to create and retrieve a graph database using java api in cassandra database - cassandra

I am trying to create a graph having nodes and edges with some weights in Cassandra database using Titan Graph api . so how to retrieve this graph so that I could visualize it.
rexster or gremlin is the solution for it.. ?? If it so.. Please tell me the process.

I have a 4 cluster cassandra setup. I run titan on top of it. I made this program which creates two nodes and then create an edge between them and finally queries and prints it.
public static void main(String args[])
{
BaseConfiguration baseConfiguration = new BaseConfiguration();
baseConfiguration.setProperty("storage.backend", "cassandra");
baseConfiguration.setProperty("storage.hostname", "192.168.1.10");
TitanGraph titanGraph = TitanFactory.open(baseConfiguration);
Vertex rash = titanGraph.addVertex(null);
rash.setProperty("userId", 1);
rash.setProperty("username", "rash");
rash.setProperty("firstName", "Rahul");
rash.setProperty("lastName", "Chaudhary");
rash.setProperty("birthday", 101);
Vertex honey = titanGraph.addVertex(null);
honey.setProperty("userId", 2);
honey.setProperty("username", "honey");
honey.setProperty("firstName", "Honey");
honey.setProperty("lastName", "Anant");
honey.setProperty("birthday", 201);
Edge frnd = titanGraph.addEdge(null, rash, honey, "FRIEND");
frnd.setProperty("since", 2011);
titanGraph.commit();
Iterable<Vertex> results = rash.query().labels("FRIEND").has("since", 2011).vertices();
for(Vertex result : results)
{
System.out.println("Id: " + result.getProperty("userId"));
System.out.println("Username: " + result.getProperty("username"));
System.out.println("Name: " + result.getProperty("firstName") + " " + result.getProperty("lastName"));
}
}
My pom.xml, I just have this dependency:
<dependency>
<groupId>com.thinkaurelius.titan</groupId>
<artifactId>titan-cassandra</artifactId>
<version>0.5.4</version>
</dependency>
I am running Cassandra 2.1.2.
I don’t know much on gremlin, but I believe Gremlin is the shell that you can use to query your database from the command line. Rexter is an API that you can use on top of any other API that uses Blueprints (like Titan), to make your query/code accessible to others via REST API. Since you want to use embedded Java with titan, you don’t need gremlin. With the dependency that I have stated, blueprint API comes with it and using that API (like I have done in my code), you can do everything with your graph.
You might find this cheatsheet useful. http://www.fromdev.com/2013/09/Gremlin-Example-Query-Snippets-Graph-DB.html

First note that TitanGraph uses the Blueprints API, thus the Titan API is the Blueprints API. As you are using Blueprints you can use Gremlin, Rexster or any part of the TinkerPop stack to process your graph.
How you visualize your graph once in Titan is dependent on the graph visualization tools you choose. If I assume you are using Gephi or similar tool that can consume GraphML, then the easiest way to get the data from Titan would be to open a Gremlin REPL, get a reference to the graph and just do:
g = TitanFactory.open(...)
g.saveGraphML('/tmp/my-graph.xml')
From there you could import my-graph.xml into Gephi and visualize it.

Related

Datastax Node.js Cassandra driver When to use a Mapper vs. Query

I'm working with the Datastax Node.js driver and I can't figure out when to use a mapper vs. query. Both seem to be able to perform the same CRUD operations.
With a query:
const q = SELECT * FROM mykeyspace.mytable WHERE id='12345';
client.execute(q).then(result => console.log('This is the data', result);
With mapper:
const tableRow = await tableMapper.find({ id: '12345' });
When should I use the mapper over a query and vice versa?
Mapper is a feature from cassandra-driver released in 2018. Using mapper, cassandra-driver can make a map from your cassandra table to an object in nodejs and you can handle in your nodejs application like a set of document.
Using mapper you can make selects or inserts in your database like said in this article:
https://www.datastax.com/blog/2018/12/introducing-datastax-nodejs-mapper-apache-cassandra
With query method, if you need to use or reuse any property from your json you will need to make a Json.Parse().
The short answer is: whatever you find more comfortable.
The Mapper lets you deal with database data as documents (JavaScript objects), builds the CQL query for you, executes the query and maps the results.
On the other hand, the core driver only supports executing CQL queries that you have to write yourself.

Read cassandra matrics using JMX in Java

How can i produce live metrics of Cassandra in Java using JMX/Metrics? I want to run cassandra JMX command to collect cassandra matrices.Examples will be much appreciated.
All Cassandra's metrics exposed via JMX are documented in official documentation. And because it uses the Metrics library, you may not need to use JMX to capture metrics - see the note at the end of the referenced page for more information (and conf/metrics-reporter-config-sample.yaml example file from Cassandra's distribution).
P.S. Maybe I misunderstood the question - can you provide more details? Are you looking for commands to collect that metrics from Cassandra? Or code snippets in Java?
From Java you can access the particular metrics with something like this:
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://[127.0.0.1]:7199/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
Set<ObjectInstance> objs = mbsc.queryMBeans(ObjectName
.getInstance("org.apache.cassandra.metrics:type=ClientRequest,scope=Read-ALL,name=TotalLatency"), null);
for (ObjectInstance obj : objs) {
Object proxy = JMX.newMBeanProxy(mbsc, obj.getObjectName(),
CassandraMetricsRegistry.JmxCounterMBean.class);
if (proxy instanceof CassandraMetricsRegistry.JmxCounterMBean) {
System.out.println("TotalLatency = " + ((CassandraMetricsRegistry.JmxCounterMBean) proxy).getCount());
}
}
jmxc.close();
More detailed example you can find at JmxCollector from cassandra-metrics-collector project...

Passing sets of properties and nodes as a POST statement wit KOA-NEO4J or BOLT

I am building a REST API which connects to a NEO4J instance. I am using the koa-neo4j library as the basis (https://github.com/assister-ai/koa-neo4j-starter-kit). I am a beginner at all these technologies but thanks to some help from this forum I have the basic functionality working. For example the below code allows me to create a new node with the label "metric" and set the name and dateAdded propertis.
URL:
/metric?metricName=Test&dateAdded=2/21/2017
index.js
app.defineAPI({
method: 'POST',
route: '/api/v1/imm/metric',
cypherQueryFile: './src/api/v1/imm/metric/createMetric.cyp'
});
createMetric.cyp"
CREATE (n:metric {
name: $metricName,
dateAdded: $dateAdded
})
return ID(n) as id
However, I am struggling to know how I can approach more complicated examples. How can I handle situations when I don't know how many properties will be added when creating a new node beforehand or when I want to create multiple nodes in a single post statement. Ideally I would like to be able to pass something like JSON as part of the POST which would contain all of the nodes, labels and properties that I want to create. Is something like this possible? I tried using the below Cypher query and passing a JSON string in the POST body but it didn't work.
UNWIND $props AS properties
CREATE (n:metric)
SET n = properties
RETURN n
Would I be better off switching tothe Neo4j Rest API instead of the BOLT protocol and the KOA-NEO4J framework. From my research I thought it was better to use BOLT but I want to have a Rest API as the middle layer between my front and back end so I am willing to change over if this will be easier in the longer term.
Thanks for the help!
Your Cypher syntax is bad in a couple of ways.
UNWIND only accepts a collection as its argument, not a string.
SET n = properties is only legal if properties is a map, not a string.
This query should work for creating a single node (assuming that $props is a map containing all the properties you want to store with the newly created node):
CREATE (n:metric $props)
RETURN n
If you want to create multiple nodes, then this query (essentially the same as yours) should work (but only if $prop_collection is a collection of maps):
UNWIND $prop_collection AS props
CREATE (n:metric)
SET n = props
RETURN n
I too have faced difficulties when trying to pass complex types as arguments to neo4j, this has to do with type conversions between js and cypher over bolt and there is not much one could do except for filing an issue in the official neo4j JavaScript driver repo. koa-neo4j uses the official driver under the hood.
One way to go about such scenarios in koa-neo4j is using JavaScript to manipulate the arguments before sending to Cypher:
https://github.com/assister-ai/koa-neo4j#preprocess-lifecycle
Also possible to further manipulate the results of a Cypher query using postProcess lifecycle hook:
https://github.com/assister-ai/koa-neo4j#postprocess-lifecycle

Add plot to sample jhipster app

A very basic question from a newbie to the wonderful jhipster ecosystem.
How do I add an additional page to my app that displays two plots based on queries on the database?
I have created the demo jhipster app that tracks authors and books in a postgres database as described in the tutorial.
I would now like to add a summary page to my sample jhipster app that displays two plots based on data retrieved from an sql query.
1) barplot of total books published per author
select author_id, count(*)
from book
group by author_id
2) line plot showing number of publications for each author per year
select author_id,extract(year from publication_date) as year,count(*)
from book
group by author_id,year
I would normally use R/Shiny and plotly to create this type of dashboard, but would really appreciate any guidance on how to achieve the same using the jhipster framework
Thanks
Iain
I see 2 tasks here. In general you first prepare 2 API endpoints to deliver the data to your frontend (keep in mind, JHI provides both server and client), and then using plotly (js) to do your plots
preparing API
you should translate your SQL query to JPA, like this:
public interface BookRepository extends JpaRepository<Book,Long> {
#Query("select b.authorId, count(b.id) from Book b group by b.authorId ")
List<Book> findAllGroupByAuthor();
#Query("select b.authorId, YEAR(b.publicationDate) as year,count(*) from Book b group by b.authorId, b.year")
List<Book> findAllGroupByAuthorAndYear();
}
then you add this to some RestControllers. Here is an example
#RestController
#RequestMapping("/api/books/query/")
public class CustomBookQueryResource {
private BookRepository bookRepository;
public CustomBookQueryResource(BookRepository bookRepository) {
this.bookRepository = bookRepository;
}
#GetMapping("/group_by_author")
public ResponseEntity<List<Book>> groupByAuthor() {
return ResponseEntity.ok(bookRepository.findAllGroupByAuthor());
}
#GetMapping("/group_by_author_and_year")
public ResponseEntity<List<Book>> groupByAuthorAndYear() {
return ResponseEntity.ok(bookRepository.findAllGroupByAuthorAndYear());
}
}
so until this point, you should already have some api endpoints, serving your data. Now you should add your custom query book service in angular
angular
.module('<yourApp>')
.factory('CustomBookQuery', CustomBookQuery);
CustomBookQuery.$inject = ['$http'];
function CustomBookQuery ($http) {
var resourceUrl = 'api/books/query/';
return {
findAllGroupByAuthor: function () {
return $http.get(resourceUrl + 'group_by_author');
},
findAllGroupByAuthorAndYear: function () {
return $http.get(resourceUrl + 'group_by_author_and_year');
}
};
}
Now you just can inject your service and pass its promises resolves to your plotly, which is already has and JS part and a Angular implementation
(I coded above code from mind, so it's not tested)
For simple cases where your query only involves one entity it might be as simple as adding a method or methods to the associated Repository interface and have Spring Data handle the query generation.
Take a look at https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.query-methods.query-creation
and if you'd prefer to supply the query
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods.at-query
You could then add an HTML page with JS to render the plot of the results of the query.
For situations that require a join over entities there’s a few possible approaches…
You could create a view in PostGresQL that would effectively make a table which is this query and then create an Entity for it (and then render them in your fav JS lib). This is my preferred option. You would need to update the liquidbase file to create the view. You would make an Entity and you would create a Spring Data repository interface (which will result in the autogeneration of a REST service etc) and then render using HTML and JS.
Alternatively, JPA entities can be queries using JPQL. That will give you a collection of objects that you can render with Angular.js or whatever your fav JS library.
http://www.oracle.com/technetwork/articles/vasiliev-jpql-087123.html
You can also use JPA to map native SQL to a PoJo (see especially the JPA 2.1 notes) and then create a REST service and render using your JS.
JPA : How to convert a native query result set to POJO class collection
http://www.thoughts-on-java.org/jpa-native-queries/

Not loading data from titan graph with cassandra backend using gremlin.

I have added data in titan(cassandra backend) using blueprint api with java. I used following configuration in java for inserting data.
TitanGraph getTitanGraph()
{
conf2 = new BaseConfiguration();
conf2.setProperty("storage.backend", "cassandra");
conf2.setProperty("storage.directory","/some/directory");
conf2.setProperty("storage.read-only", "false");
conf2.setProperty("attributes.allow-all", true);
return TitanFactory.open(conf2);
}
Now I am trying to query that database using gremlin. I used following cmd to load it
g = TitanFactory.open("bin/cassandra.local");
following is my cassandra.local file
conf = new BaseConfiguration();
conf.setProperty("storage.backend","cassandra");
conf.setProperty("storage.hostname","127.0.0.1");
conf.setProperty("storage.read-only", "false");
conf.setProperty("attributes.allow-all", true)
but when I am running "g.V", I am getting empty graph. Please help
thanks
Make sure that you commit the changes to your TitanGraph after making graph mutations in your Java program. If you're using Titan 0.5.x, the call is graph.commit(). If you're using Titan 0.9.x, the call is graph.tx().commit().
Note that storage.directory isn't valid for a Cassandra backend, however the default value for storage.hostname is 127.0.0.1 so those should be the same between your Java program and cassandra.local. It might be easier to use a properties file to store your connection properties.

Resources