Having a graph, actually it is a tree: vertexes are nodes, edges are labeled as "subnode" and directed from child to parent.
I need to make gremlin query to get recursive structure as this:
node_info = [properties: node.map(),
subnodes: [...list of node_info items...]]
Groovy function describes more precisely what I need to get:
def get_node_hierarchy(node_id) {
def get_hierarchy(node) {
def hierarchy_list = []
for (subnode in node.in('subnode')) {
sub_hierarchy = get_hierarchy(subnode)
hierarchy_list.add(sub_hierarchy)
}
[properties: node.map(), subnodes: hierarchy_list]
}
node = g.V('node_id', node_id).next()
get_hierarchy(node)
}
result = get_node_hierarchy(1)
Is it possible to implement this using a single Gremlin query?
Related
I am new to Gatling, I am trying to do the performance testing for couple of rest calls. In my scenario I need to extract a value from json response of the 1st call and add those values to the list after looping for few times. Again after looping for few times and adding the values into the list, I want to reuse each value in my next rest call by iterating over the values in the list. Can anyone please suggest on how to implement this. I tried something as below,
var datasetIdList = List.empty[String]
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator
def createData() = {
repeat(20){
feed("").exec(http("create dataset").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{"name":"name"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId"))))
.exec(session => { var usrid = session("userId").as[String].trim
datasetIdList:+= usrid session})
}}
def upload()= feed(datasetidsFeeder).exec(http("file upload").post("/compute-metaservice/datasets/${datasetId}/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
val scn = scenario("create data and upload").exec(createData()).exec(upload())
setUp(scn.inject(atOnceUsers(1))).protocols(httpConf)
}
I am seeing an exception that ListFeeder is empty when trying to run above script. Can someone please help
Updated Code:
class ParallelcallsSimulation extends Simulation{
var idNumbers = (1 to 50).iterator
val customFeeder = Iterator.continually(Map(
"name" -> ("test_gatling_"+ idNumbers.next())
))
val httpConf = http.baseUrl("http://localhost:8080")
.header("Authorization","Bearer 6a4aee03-9172-4e31-a784-39dea65e9063")
def createDatasetsAndUpload() = {
repeat(3) {
//create dataset
feed(customFeeder).exec(http("create data").post("/create/data").header("content-type", "application/json")
.body(StringBody("""{ "name": "${name}","description": "create data and upload file"}"""))
.asJson.check(jsonPath("$.id").saveAs("userId")))
.exec(session => {
val name = session("name").asOption[String]
println(name.getOrElse("COULD NOT FIND NAME"))
val userId = session("userId").as[String].trim
println("%%%%% User ID ====>"+userId)
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
})
}
}
// File Upload
def fileUpload() = foreach("${datasetIdList}","datasetId"){
exec(http("file upload").post("/uploadFile")
.formUpload("File","./src/test/resources/data/File.csv")
.header("content-type","multipart/form-data")
.check(status is 200))
}
def getDataSetId() = foreach("${datasetIdList}","datasetId"){
exec(http("get datasetId")
.get("/get/data/${datasetId}")
.header("content-type","application/json")
.asJson.check(jsonPath("$.dlp.dlp_job_status").optional
.saveAs("dlpJobStatus")).check(status is 200)
).exec(session => {
val datastId = session("datasetId").asOption[String]
println("request for datasetId >>>>>>>>"+datastId.getOrElse("datasetId not found"))
val jobStatus = session("dlpJobStatus").asOption[String]
println("JOB STATUS:::>>>>>>>>>>"+jobStatus.getOrElse("Dlp Job Status not Found"))
println("Time: >>>>>>"+System.currentTimeMillis())
session
}).pause(10)
}
val scn1 = scenario("create multiple datasets and upload").exec(createDatasetsAndUpload()).exec(fileUpload())
val scn2 = scenario("get datasetId").pause(100).exec(getDataSetId())
setUp(scn1.inject(atOnceUsers(1)),scn2.inject(atOnceUsers(1))).protocols(httpConf)
}
I see below error when I try to execute above script
[ERROR] i.g.c.s.LoopBlock$ - Condition evaluation crashed with message 'No attribute named 'datasetIdList' is defined', exiting loop
var datasetIdList = List.empty[String] defines a mutable variable pointing to a immutable list.
val datasetidsFeeder = datasetIdList.map(datasetId => Map("datasetId" -> datasetId)).iterator uses the immutable list. Further changes to datasetIdList is irrelevant to datasetidsFeeder.
Mutating a global variable with your virtual user is usually not a good idea.
You can save the value into the user's session instead.
In the exec block, you can write:
val userId = session("userId").as[String].trim
val datasetIdList = session("datasetIdList").asOption[List[_]].getOrElse(Nil)
session.set("datasetIdList", userId :: datasetIdList)
Then you can use foreach to iterate them all without using a feeder at all.
foreach("${datasetIdList}", "datasetId") {
exec(http("file upload")
...
}
You should put more work in your question.
Your code is not syntax-highlighted, and is formatted poorly.
You said "I am seeing an exception that ListFeeder is empty" but the words "ListFeeder" are not seen anywhere.
You should post the error message so that it's easier to see what went wrong.
In the documentation linked, there is a Warning. Quoted below:
Session instances are immutable!
Why is that so? Because Sessions are messages that are dealt with in a multi-threaded concurrent way, so immutability is the best way to deal with state without relying on synchronization and blocking.
A very common pitfall is to forget that set and setAll actually return new instances.
This is why the code in the updated question doesn't update the list.
session => {
...
session.set("datasetIdList", userId :: datasetIdList)
println("%%%% List =====>>>" + datasetIdList.toString())
session
}
The updated session is simply discarded. And the original session is returned in the anonymous function.
When creating rivers for composed objects, the resulting _mapping is set with the complete nested object definition rather than String field. This causes the data import to fail because the object references are not "dereferenced".
E.g.
collection1: {name: "test", items: [collection2/123, collection2/124] }
collection1: {somefield: "test"}
The resulting _mapping after creating the river for those collections within a single index is:
collection1: {name: String, items: { properties: { somefield: String } } }.
Importing data fails with the following error:
org.elasticsearch.index.mapper.MapperParsingException: object mapping [items] trying to serialize a value with no field associated with it, current value [collection1/123]
How can I either tell the arango db river to dereference the nested objects or set the mapping properly to work with references?
Rivers are now deprecated. I created a mixin for elasticsearch which updates the index when I save/update/delete objects (through my custom ODM).
Simply make yourself a wrapper around your data access layer with high level functions that also updates the ES index.
For example:
class Base(ArangoBase, es.Base):
def save(self):
ret = ArangoBase.save(self)
es.Base.save_es(self)
return ret
def update(self):
ret = ArangoBase.update(self)
es.Base.save_es(self)
return ret
def delete(self):
ret = ArangoBase.delete(self)
es.Base.delete_es(self)
return ret
from elasticsearch import Elasticsearch
class Base(object):
_es = None
_es_index = 'chopchop'
_es_type = None
def save_es(self):
self._es.index(index=self._es_index, doc_type=self._es_type, body=self._doc(), id=self.id)
def delete_es(self):
self._es.delete(index=self._es_index, doc_type=self._es_type, id=self.id)
I am trying to write a generic program in Groovy that will get the SQL from config file along with other parameters and put them into file.
here is the program:
def config = new ConfigSlurper().parse(new File("config.properties").toURL())
Sql sql = Sql.newInstance(config.db.url, config.db.login, config.db.password, config.db.driver);
def fileToWrite = new File(config.copy.location)
def writer = fileToWrite.newWriter()
writer.write(config.file.headers)
sql.eachRow(config.sql){ res->
writer.write(config.file.rows)
}
in the config the sql is something like this:
sql="select * from mydb"
and
file.rows="${res.column1}|${res.column2}|${res.column3}\n"
when I run it I get
[:]|[:]|[:]
[:]|[:]|[:]
[:]|[:]|[:]
in the file. If I substitute
writer.write(config.file.rows)
to
writer.write("${res.column1}|${res.column2}|${res.column3}\n")
it outputs the actual results. What do I need to do different to get the results?
You accomplish this by using lazy evaluation of the Gstring combined with altering the delegate.
First make the Gstring lazy by making the values be the results of calling Closures:
file.rows="${->res.column1}|${->res.column2}|${-> res.column3}"
Then prior to evaluating alter the delegate of the closures:
config.file.rows.values.each {
if (Closure.class.isAssignableFrom(it.getClass())) {
it.resolveStrategy = Closure.DELEGATE_FIRST
it.delegate = this
}
}
The delegate must have the variable res in scope. Here is a full working example:
class Test {
Map res
void run() {
String configText = '''file.rows="${->res.column1}|${->res.column2}|${-> res.column3}"
sql="select * from mydb"'''
def slurper = new ConfigSlurper()
def config = slurper.parse(configText)
config.file.rows.values.each {
if (Closure.class.isAssignableFrom(it.getClass())) {
it.resolveStrategy = Closure.DELEGATE_FIRST
it.delegate = this
}
}
def results = [
[column1: 1, column2: 2, column3: 3],
[column1: 4, column2: 5, column3: 6],
]
results.each {
res = it
println config.file.rows.toString()
}
}
}
new Test().run()
The good news is that the ConfigSlurper is more than capable of doing the GString variable substitution for you as intended. The bad news is that it does this substitution when it calls the parse() method, way up above, long before you have a res variable to substitute into the parser. The other bad news is that if the variables being substituted are not defined in the config file itself, then you have to supply them to the slurper in advance, via the binding property.
So, to get the effect you want you have to parse the properties through each pass of eachRow. Does that mean you have to create a new ConfigSlurper re-read the file once for every row? No. You will have to create a new ConfigObject for each pass, but you can reuse the ConfigSlurper and the file text, as follows:
def slurper = new ConfigSlurper();
def configText = new File("scripts/config.properties").text
def config = slurper.parse(configText)
Sql sql = Sql.newInstance(config.db.url, config.db.login, config.db.password, config.db.driver);
def fileToWrite = new File(config.copy.location)
def writer = fileToWrite.newWriter()
writer.write(config.file.headers)
sql.eachRow(config.sql){ result ->
slurper.binding = [res:result]
def reconfig = slurper.parse(configText)
print(reconfig.file.rows)
}
Please notice that I changed the name of the Closure parameter from res to result. I did this to emphasize that the slurper was drawing the name res from the binding map key, not from the closure parameter name.
If you want to reduce wasted "reparsing" time and effort, you could separate the file.rows property into its own separate file. i would still read in that file text once and reuse the text in the "per row" parsing.
I want to return the list of the computations from a method that uses a list of Futures:
def foo: List[Long] = {
val res = List(1, 2, 3) map {
x => Future { someCalculation(x) }
}
Future.sequence(res)
// what to do next?
}
def someCalculation(a: Int): Long = //....
How can I do this?
There is a key point to understand when it comes to futures: if you wanna go from Future[T] to T you need to await the result of the operation, but this is something you would like to avoid not to affect the performance of your program. The correct approach is to keep working with asynchronous abstractions as much as you can, and move blocking up to your calling stack.
The Future class has a number of methods you can use to enchain other async operations, such as map, onComplete, onSuccess, etc etc.
If you really need to wait the result, then there is Await.result
val listOfFutures:List[Future[Long]] = val res = List(1, 2, 3) map {
x => Future { someCalculation(x) }
}
// now we have a Future[List[Long]]
val futureList:Future[List[Long]] = Future.sequence(listOfFutures)
// Keep being async here, compute the results asynchronously. Remember the map function on future allows you to pass a f: A=>B on Future[A] and obtain a Future[B]. Here we call the sum method on the list
val yourOperationAsync:Future[Long] = futureList.map{_.sum}
// Do this only when you need to get the result
val result:Long = Await.result(yourOperationAsync, 1 second)
Well the whole point of using Future is to make it asynchronous. i.e
def foo: Future[List[Long]] = {
val res = List(1, 2, 3) map {
x => Future { someCalculation(x) }
}
Future.sequence(res)
}
This would be the ideal solution. But In case if you wish to wait, then you could wait for the result and then return:
val ans = Future.sequence(res)
Await.ready(ans, Duration.inf)
I have a scenario where users are assigned to team.
Different ClientServices are allocated to different teams and
we need to assign user Of these teams to clientservice in RoundRobin fashion
I was trying to solve it as follows to get a map where team name and a list of ClientServiceInstance will be mapped so I can do further processing on it
def teamMap = [:]
clientServicesList.each {clientServiceInstance->
if(teamMap[clientServiceInstance.ownerTeam] == null){
teamMap.putAt(clientServiceInstance.ownerTeam, new ArrayList().push(clientServiceInstance))
}else{
def tmpList = teamMap[clientServiceInstance.ownerTeam]
tmpList.push(clientServiceInstance)
teamMap[clientServiceInstance.ownerTeam] = tmpList
}
}
but instead of pushing clientServiceInstance it pushes true.
Any idea?
I believe another version would be:
def teamMap = clientServicesList.inject( [:].withDefault { [] } ) { map, instance ->
map[ instance.ownerTeam ] << instance
map
}
new ArrayList().push(clientServiceInstance) returns true, which means you're putting that into your teamMap instead of what I assume should be a list? Instead you might want
teamMap.putAt(clientServiceInstance.ownerTeam, [clientServiceInstance])
By the way, your code is not very Groovy ;)
You could rewrite it as something like
def teamMap = [:]
clientServicesList.each { clientServiceInstance ->
if (teamMap[clientServiceInstance.ownerTeam]) {
teamMap[clientServiceInstance.ownerTeam] << clientServiceInstance
} else {
teamMap[clientServiceInstance.ownerTeam] = [clientServiceInstance]
}
}
Although I'm sure there are even better ways to write that.