I would like to remove the array SEO from the json when the keys "Description" and "Title" in the has no value.
json:
[
{
"SEO": [
{
"Description": "",
"Title": ""
}
],
"accesoires": [
"1167296"
],
"shortCode": "S-576",
"spareParts": [
"800236"
]
}]
I tried the below code but i'm not able to remove the array.
def Message processData(Message message) {
def body = message.getBody(String);
def json = new JsonSlurper().parseText(body)
json.each{
it.SEO.each{
if(!(it.findResults{k, v -> v?.size() > 0 && v[0]?.length() > 0 ? v[0] : null })){
json.remove("SEO")
} } }
def out= JsonOutput.toJson(json)
message.setBody(out)
return message}
To remove the array "SEO" from the JSON when the keys "Description" and "Title" have no value, you can use the following Groovy code:
def jsonString = '[{"SEO": [{"Description": "", "Title": ""}], "accesoires": ["1167296"], "shortCode": "S-576", "spareParts": ["800236"]}]'
def json = new JsonSlurper().parseText(jsonString)
for (item in json) {
if (!item.SEO[0].Description && !item.SEO[0].Title) {
item.remove('SEO')
}
}
println(JsonOutput.toJson(json))
This will first parse the JSON string into a list of maps using JsonSlurper. Then it iterates through each map in the list and checks if the "Description" and "Title" keys in the "SEO" array are empty. If they are, it removes the "SEO" array from the map using the remove() method. Finally, it prints the modified JSON using the JsonOutput.toJson() method.
Related
I'm trying to generate JSON body dynamically using values in csv file. For this i'm making use of JSR223 PreProcessor with groovy script.
I'm expecting the below format to be generate when i run the groovy script
{
"transactionId": "100",
"lineItems": [{
"lineItemNo": "1",
"cardInfo": {
"cardNumber": "3456"
}
},
{
"lineItemNo": "2",
"cardInfo": {
"cardNumber": "45698"
}
}
]
}
but when i execute script i'm getting below format
POST data:
{
"transactionId": "100",
"lineItems": [
{
"lineItemNo": "1",
"Cardinfo": [
9255000012794606,
9255000012794645
]
},
{
"lineItemNo": "1",
"Cardinfo": [
9255000012794606,
9255000012794645
]
}
]
}
Script to generate json body
File csvFile = new File("D:\\Project Related Docs\\Jmeter\\apache-jmeter-5.0\\bin\\Map_Performance\\Map_New_Auto_jmx\\2Cards.csv")
def cards = csvFile.readLines()
List<String> cardnumbmer = new ArrayList<>()
def counter = 1
cards.eachWithIndex{line,idx->cardnumbmer.add(line)}
log.info("size of csv = "+cardnumbmer.size())
log.info("File conents = "+cardnumbmer[0])
//build the json body
def ids = new groovy.json.JsonSlurper().parseText(cardnumbmer.toString())
log.info("cardnumbmer to string = "+cardnumbmer.toString())
def builder = new groovy.json.JsonBuilder()
builder([
transactionId:"100",
lineItems:ids.collect{[lineItemNo:"1",Cardinfo: ids.collect{carnumber: it}]}
])
//sampler.getArguments().removeAllArguments()
sampler.addNonEncodedArgument('',builder.toPrettyString(),'')
sampler.setPostBodyRaw(true);
--CSV FILE have cardnumbers listed in row-wise look like below
9255000012794606
9255000012794645
Request to help me to know how to fix this issue.
ids.collect{carnumber: it} is basically ids. Be explicit about returning the map: ids.collect{ [carnumber: it] }
Below code resolved the problem
//build the json body
def ids = new groovy.json.JsonSlurper().parseText(cardnumbmer.toString())
log.info("cardnumbmer to string = "+cardnumbmer.toString())
def builder = new groovy.json.JsonBuilder()
def count = 1
builder([
transactionId:"100",
//lineItems:ids.collect{[lineItemNo:"1",Cardinfo: count.collect{[carnumber: it]}]}
lineItems:ids.collect{[lineItemNo:count++,Cardinfo: [Cardnumber:it]]}
])
//sampler.getArguments().removeAllArguments()
sampler.addNonEncodedArgument('',builder.toPrettyString(),'')
sampler.setPostBodyRaw(true);
I need to do a json transformation in apache nifi. The json keys in the payload would be dynamically generated.
For example in the input given below, the 'customer' has attributes 'fname' and 'lname'. I need to change this 'fname' -> 'firstname' and 'lname' -> 'lastname' as provided in the 'mappingvalues'.
Since I am newbie to nifi. I dont know where to start. I have tried some json transformers like jolt. But couldn't achieve the expected result.
The jolt transform that i have used is given below :
[
{
"operation": "shift",
"spec": {
"customer": {
"*": {
"#": "&"
}
}
}
}
]
which produced an output
{
"fname" : "akhil",
"lname" : "kumar"
}
The input and expected output of what I need to achieve is given below :
{
"customer": {
"fname": "akhil",
"lname": "kumar",
.
.
.
},
"mappingvalues": {
"fname": "firstname",
"lname": "lastname",
.
.
.
}
}
##OUTPUT
{
"customer": {
"firstname": "akhil",
"lastname": "kumar",
.
.
.
}
}
*Is there any way to achieve the same in nifi with or without using jolt transform? Is it possible to do the same with groovy script? *
Please help me on the same.
the code in groovy with recursive mapping:
import groovy.json.JsonSlurper
def ff = session.get()
if(!ff)return
def json = ff.read().withReader("UTF-8"){r-> new JsonSlurper().parse(r) }
def mappings = json.remove('mappingvalues')
def mapper(o, mappings){
if(o instanceof Map){
//json object. let's iterate it and do mapping
o = o.collectEntries{k,v-> [ (mappings[k] ?: k), mapper(v,mappings) ] }
}else if(o instanceof List){
//map elements in array
o = o.collect{v-> mapper(v,mappings) }
}
return o
}
json = mapper(json,mappings)
ff.write("UTF-8"){w-> new JsonBuilder(json).writeTo(w) }
REL_SUCCESS << ff
1.I am having a list which contains multiple Maps that looks like below Map.
Each map contains many keys I want to get value of "name"
{
"question":{
"com.forms.tree":{
"requiredByDefault":true,
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"language":"en"
},
"formField":{
"name":"CUSTOM_347",
"tag":"input",
"url":"Demo"
}
}
},
"Field":"true"
},{
"question":{
"com.forms.tree":{
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"language":"en"
},
"formField":{
"name":"CUSTOM_348",
"url":"Demo"
}
}
},
"Field":"true"}
I want to get the value of "name" which falls in every Map but don't want to iterate like question?."com.forms.tree"?.formField?.name.
Is there any other approach in groovy?
So given the json:
def jsonTxt = '''{
"question":{
"com.forms.tree":{
"requiredByDefault":true,
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"name": "test",
"language":"en"
},
"formField":{
"name":"CUSTOM_347",
"tag":"input",
"url":"Demo"
}
}
},
"Field":"true"
}'''
We can parse it with:
import groovy.json.*
def json = new JsonSlurper().parseText(jsonTxt)
You want to find the "formField" entry in that object, so lets write a recursive finder that will walk through out map of maps looking for the first entry with the given key:
static findFirstByKey(Map map, key) {
map.get(key) ?: map.findResult { k, v -> if(v in Map) findFirstByKey(v, key) }
}
And you can then check it works:
assert findFirstByKey(json, 'formField')?.name == "CUSTOM_347"
Basically, I want to somehow create a JSON object from a Groovy object. The Groovy object has key value pairs, and one of the values is a Groovy Array:
import groovy.json.*
// Imagine "handler" gets called somehow and an event gets passed to it.
def handler(event) {
def capabilitiesList = event.device.capabilities.findAll { attr -> attr.name != null }
def json = new JsonBuilder({
id event.deviceId
displayName event.displayName
value event.value
})
}
log.debug capabilitiesList
log.debug json.toPrettyString()
At this point, json.toPrettyString() gives me this:
{
"id": "asdfl469934623sdglsi3aqaq",
"displayName": "Some Lightbulb",
"value": "on"
}
And capabilitiesList gives me this:
["Test 1", "Test 2", "Test 3"]
How can I add the capabilitiesList array to the Groovy object so it gets converted to JSON?
I can't seem to get anything to work; the only thing that does work is this:
// ...
def json = new JsonBuilder({
id event.deviceId
displayName event.displayName
value event.value
capabilitiesList "Test 1", "Test 2", "Test 3"
})
// ...
Which gives me this (correct) JSON output:
{
"id": "asdfl469934623sdglsi3aqaq",
"displayName": "Some Lightbulb",
"value": "on",
"capabilitiesList": ["Test 1", "Test 2", "Test 3"]
}
But that obviously isn't useful because it's hard coded. So I tried referencing the Array directly like this:
// ...
def capabilitiesList = event.device.capabilities.findAll { attr -> attr.name != null }
def json = new JsonBuilder({
id event.deviceId
displayName event.displayName
value event.value
capabilitiesList capabilitiesList
})
// ...
But that breaks the JsonBuilder somehow, and it doesn't output anything.
I'm probably doing something really silly here but I can't quite figure out how to get this done. First time with Groovy as well. Thanks for the help!
Using the builder DSL should work. For example:
List list = ['Test1', 'Test2', 'Test3']
def builder = new groovy.json.JsonBuilder()
builder {
id "asdfl469934623sdglsi3aqaq"
displayName "Some Lightbulb"
value "on"
capabilitiesList list
}
println builder.toPrettyString()
prints
{
"id": "asdfl469934623sdglsi3aqaq",
"displayName": "Some Lightbulb",
"value": "on",
"capabilitiesList": [
"Test1",
"Test2",
"Test3"
]
}
I have required to remove object from array that satisfies the condition, I am able to update the object of array on the basis of condition, which is as follow:
PUT twitter/twit/1
{"list":
[
{
"tweet_id": "1",
"a": "b"
},
{
"tweet_id": "123",
"a": "f"
}
]
}
POST /twitter/twit/1/_update
{"script":"foreach (item :ctx._source.list) {
if item['tweet_id'] == tweet_id) {
item['new_field'] = 'ghi';
}
}",
"params": {tweet_id": 123"}
}
this is working
for remove i am doing this
POST /twitter/twit/1/_update
{ "script": "foreach (item : ctx._source.list) {
if item['tweet_id'] == tweet_id) {
ctx._source.list.remove(item);
}
}",
"params": { tweet_id": "123" }
}
but this is not working and giving this error,
ElasticsearchIllegalArgumentException[failed to execute script];
nested: ConcurrentModificationException; Error:
ElasticsearchIllegalArgumentException[failed to execute script];
nested: ConcurrentModificationException
I am able to remove whole array or whole field using
"script": "ctx._source.remove('list')"
I am also able to remove object from array by specifying all the keys of an object using
"script":"ctx._source.list.remove(tag)",
"params" : {
"tag" : {"tweet_id": "123","a": "f"}
my node module elastic search version is 2.4.2 elastic search server is 1.3.2
You get that because you are trying to modify a list while iterating through it, meaning you want to change a list of object and, at the same time, listing those objects.
You instead need to do this:
POST /twitter/twit/1/_update
{
"script": "item_to_remove = nil; foreach (item : ctx._source.list) { if (item['tweet_id'] == tweet_id) { item_to_remove=item; } } if (item_to_remove != nil) ctx._source.list.remove(item_to_remove);",
"params": {"tweet_id": "123"}
}
If you have more than one item that matches the criteria, use a list instead:
POST /twitter/twit/1/_update
{
"script": "items_to_remove = []; foreach (item : ctx._source.list) { if (item['tweet_id'] == tweet_id) { items_to_remove.add(item); } } foreach (item : items_to_remove) {ctx._source.list.remove(item);}",
"params": {"tweet_id": "123"}
}
For people that need this working in elasticsearch 2.0 and up, the nil and foreach don't get recognized by groovy.
So here's a updated version, including a option to replace a item with the same id by a new object.
and also passing it the upsert will make sure the item gets added even if the document doesn't exist yet
{
"script": "item_to_remove = null; ctx._source.delivery.each { elem -> if (elem.id == item_to_add.id) { item_to_remove=elem; } }; if (item_to_remove != null) ctx._source.delivery.remove(item_to_remove); if (item_to_add.size() > 1) ctx._source.delivery += item_to_add;",
"params": {"item_to_add": {"id": "5", "title": "New item"}},
"upsert": [{"id": "5", "title": "New item"}]
}