I have gone through various articles but didn't find any simple way to sort the API response in ascending or descending order using groovy. Can someone please help on this?
import groovy.json.JsonSlurper
def inputJson = '''{
"status" : "success",
"locale" : "",
"data" : {
"periods" : [
{
"payCycleId" : "custompayperiod",
"sequence" : 1,
"cycleStartDate" : "2018-10-01",
"cycleEndDate" : "2018-10-08"
},
{
"payCycleId" : "custompayperiod",
"sequence" : 2,
"cycleStartDate" : "2018-10-09",
"cycleEndDate" : "2018-10-16"
}
]
}
}
Want to sort above response based on sequence.
There are a lot of ways.
For example:
def json = new JsonSlurper().parseText(inputJson)
//Descending
json.data.periods = json.data.periods.toSorted { a, b -> b.sequence <=> a.sequence }
//Ascending
//json.data.periods = json.data.periods.toSorted { a, b -> a.sequence <=> b.sequence }
String output = JsonOutput.prettyPrint(JsonOutput.toJson(json))
Related
I've a JSON with next structure and data:
[ {
"id" : 716612,
"type" : "ad",
"stats" : [ {
"day" : "2020-06-01",
"impressions" : 1956,
"clicks" : 1,
"reach" : 1782
},
{
"day" : "2020-06-13",
"spent" : "73.32",
"reach" : 1059
} ]
}, {
"id" : 414290,
"type" : "campaign",
"stats" : [ {
"day" : "2020-05-21",
"effective_cost_per_click" : "31.200",
"effective_cost_per_mille" : "108.337"
},
{
"day" : "2020-05-17",
"impressions" : 1,
"reach" : 1,
"ctr" : "0.000",
"uniq_views_count" : 1
} ]
} ]
I need to map id and type from top level with data inside stats to get result like this:
[ {
"id" : 716612,
"type" : "ad",
"day" : "2020-06-01",
"impressions" : 1956,
"clicks" : 1,
"reach" : 1782
},
{
"id" : 716612,
"type" : "ad",
"day" : "2020-06-13",
"spent" : "73.32",
"reach" : 1059
},
...
I tried with:
def json = new JsonSlurper().parseText(text)
def result = json.collectMany{ a ->
a["stats"].collectMany{ b ->
b.collect{
[id: a.id,
type: a.type
]
}
}
}
But it returns only id and type fields without stats. I thought that I'm looping through stat and just adding needed fields from above. I guess I don't get the difference between collectMany and collect?
You were close 😁
You want to collect the stat plus the id and type, so you need:
def result = json.collectMany { a ->
a.stats.collect { b ->
[ id: a.id, type: a.type ] + b
}
}
I am having a data field in my index in which,
I want only doc 2 as result i.e logically where b comes before
a in the array field data.
doc 1:
data = ['a','b','t','k','p']
doc 2:
data = ['p','b','i','o','a']
Currently, I am trying terms must on [a,b] then checking the order in another code snippet.
Please suggest any better way around.
My understanding is that the only way to do that would be to make use of Span Queries, however it won't be applicable on an array of values.
You would need to concatenate the values into a single text field with whitespace as delimiter, reingest the documents and make use of Span Near query on that field:
Please find the below mapping, sample document, the query and response:
Mapping:
PUT my_test_index
{
"mappings": {
"properties": {
"data":{
"type": "text"
}
}
}
}
Sample Documents:
POST my_test_index/_doc/1
{
"data": "a b"
}
POST my_test_index/_doc/2
{
"data": "b a"
}
Span Query:
POST my_test_index/_search
{
"query": {
"span_near" : {
"clauses" : [
{ "span_term" : { "data" : "a" } },
{ "span_term" : { "data" : "b" } }
],
"slop" : 0, <--- This means only `a b` would return but `a c b` won't.
"in_order" : true <--- This means a should come first and the b
}
}
}
Note that slop controls the maximum number of intervening unmatched positions permitted.
Response:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.36464313,
"hits" : [
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.36464313,
"_source" : {
"data" : "a b"
}
}
]
}
}
Let me know if this helps!
I need to do a json transformation in apache nifi. The json keys in the payload would be dynamically generated.
For example in the input given below, the 'customer' has attributes 'fname' and 'lname'. I need to change this 'fname' -> 'firstname' and 'lname' -> 'lastname' as provided in the 'mappingvalues'.
Since I am newbie to nifi. I dont know where to start. I have tried some json transformers like jolt. But couldn't achieve the expected result.
The jolt transform that i have used is given below :
[
{
"operation": "shift",
"spec": {
"customer": {
"*": {
"#": "&"
}
}
}
}
]
which produced an output
{
"fname" : "akhil",
"lname" : "kumar"
}
The input and expected output of what I need to achieve is given below :
{
"customer": {
"fname": "akhil",
"lname": "kumar",
.
.
.
},
"mappingvalues": {
"fname": "firstname",
"lname": "lastname",
.
.
.
}
}
##OUTPUT
{
"customer": {
"firstname": "akhil",
"lastname": "kumar",
.
.
.
}
}
*Is there any way to achieve the same in nifi with or without using jolt transform? Is it possible to do the same with groovy script? *
Please help me on the same.
the code in groovy with recursive mapping:
import groovy.json.JsonSlurper
def ff = session.get()
if(!ff)return
def json = ff.read().withReader("UTF-8"){r-> new JsonSlurper().parse(r) }
def mappings = json.remove('mappingvalues')
def mapper(o, mappings){
if(o instanceof Map){
//json object. let's iterate it and do mapping
o = o.collectEntries{k,v-> [ (mappings[k] ?: k), mapper(v,mappings) ] }
}else if(o instanceof List){
//map elements in array
o = o.collect{v-> mapper(v,mappings) }
}
return o
}
json = mapper(json,mappings)
ff.write("UTF-8"){w-> new JsonBuilder(json).writeTo(w) }
REL_SUCCESS << ff
Hi I am new to groovy and API Automation. I have the following Json and i want to add assertion to check cyclestartdate and cycleEnddate based on sequence number.
{
"status" : "success",
"locale" : "",
"data" : {
"periods" : [
{
"payCycleId" : "custompayperiod",
"sequence" : 1,
"cycleStartDate" : "2018-10-01",
"cycleEndDate" : "2018-10-08"
},
{
"payCycleId" : "custompayperiod",
"sequence" : 2,
"cycleStartDate" : "2018-10-09",
"cycleEndDate" : "2018-10-16"
}
]
}
}
How do i check if sequence 1 cycleStartDate is 2018-10-01
Groovy provides JsonSlurper class that makes parsing JSON documents easier. Consider following example that reads JSON document as a String (it supports different initialization methods as well):
import groovy.json.JsonSlurper
def inputJson = '''{
"status" : "success",
"locale" : "",
"data" : {
"periods" : [
{
"payCycleId" : "custompayperiod",
"sequence" : 1,
"cycleStartDate" : "2018-10-01",
"cycleEndDate" : "2018-10-08"
},
{
"payCycleId" : "custompayperiod",
"sequence" : 2,
"cycleStartDate" : "2018-10-09",
"cycleEndDate" : "2018-10-16"
}
]
}
}'''
def json = new JsonSlurper().parseText(inputJson)
assert json.data.periods.find { it.sequence == 1 }.cycleStartDate == '2018-10-01'
Having JSON document loaded you can extract data by accessing nested fields. For instance, json.data.periods gives you an access to the array stored in your JSON document. Then method find { it.sequence == 1 } returns a node from this array where sequence field is equal to 1. And lastly you can extract cycleStartDate and compare it with the expected date.
You can find more useful examples in Groovy's official documentation.
I Have the following Collection :
/* 0 */
{
"clientID" : ObjectId("51b9c10d91d1a3a52b0000b8"),
"_id" : ObjectId("532b4f1cb3d2eacb1300002b"),
"answers" : [],
"questions" : []
}
/* 1 */
{
"clientID" : ObjectId("51b9c10d91d1a3a52b0000b8"),
"_id" : ObjectId("532b6b9eb3d2eacb1300002c"),
"answers" : [
"1",
"8"
],
"questions" : [
"1",
"2",
"3"
]
}
/* 2 */
{
"clientID" : ObjectId("51b9c10d91d1a3a52b0000b8"),
"_id" : ObjectId("532b6baeb3d2eacb1300002d"),
"answers" : [
"1",
"8"
],
"questions" : [
"1",
"2",
"3"
]
}
/* 3 */
{
"clientID" : ObjectId("5335f9d864e2b1290c00012e"),
"_id" : ObjectId("533b828146ca43634000002d"),
"answers" : [
"ORANGE"
],
"questions" : [
"Color"
]
}
/* 4 */
{
"clientID" : ObjectId("5335f9d864e2b1290c00012e"),
"_id" : ObjectId("5351be327b539a4d1a00002b"),
"answers" : [
"ORANGE"
],
"questions" : [
"Color"
]
}
/* 5 */
{
"clientID" : ObjectId("5335f9d864e2b1290c00012e"),
"_id" : ObjectId("5351be5ec89d717d1a00002b"),
"answers" : [
"ORANGE"
],
"questions" : [
"Color"
]
}
I am running the following code in order to find how many times the (questions,answers) combination appears in the collection:
o.map= function(){
emit({"questions" : this.questions, "answers" :this.answers },this.clientID)
};
o.reduce = function(answers, collection){
return collection.length;
};
logSearchDB.mapReduce(o,function (err, results) {
results.sort(function(a, b){return b.value-a.value});
for (var i = 0; i < results.length; i++) {
console.log(JSON.stringify(results[i]))
};
})
The output is:
{"_id":{"questions":[],"answers":[]},"value":"51b9c10d91d1a3a52b0000b8"}
{"_id":{"questions":["Color"],"answers":["ORANGE"]},"value":3}
{"_id":{"questions":["1","2","3"],"answers":["1","8"]},"value":2}
I expected that the first row will have "value" : 1
I guess the 'reduce' function got a 'collection' object : "51b9c10d91d1a3a52b0000b8", instead of getting an array : ["51b9c10d91d1a3a52b0000b8"].
Why the map reduce doesn't collect everything into an array?
The reason why you have just a plain value in that first row is because there was only one occurrence of your key value. This is generally how mapReduce works, at least in the way it was specified in the original papers.
So the reduce function is not actually called when there only is a single key. To work around this you use the finalize function in your map reduce:
var finalize = function(key,value) {
if ( typeof(value) != "number" )
value = 1;
return value;
};
db.collection.mapReduce(
mapper,
reducer,
{
"finalize": finalize,
"out": { "inline": 1 }
}
);
That runs over all of the output and sees that when the value is seen to be not a nunber, being the clientID you are emitting, then the value is set at 1 because that is how hany are in the grouping.
Really your query is better suited to the aggregation framework than mapReduce. The aggregation framework is a native code implementation as opposed to using a JavaScript interpreter. It runs much faster than mapReduce:
db.collection.aggregate([
{ "$group": {
"_id": {
"questions": "$questions",
"answers": "$answers"
},
"count": { "$sum": 1 }
}}
])
So it is the better option to use. It was a later introduction to MongoDB so people still tend to think in terms of mapReduce or otherwise there is legacy code from earlier versions of MongoDB. But this has been around for quite a while now.
Also see the operator reference for the aggregation framework.