Create a key:value pair dynamically - groovy

I have the following groovy-script:
#!/usr/bin/env groovy
def files = [
'file-1.bat' :
[
'content-1-1',
'content-1-2',
'content-1-3'
],
'file-1' :
[
'content-unix-1-1',
'content-unix-1-2',
'content-unix-1-3',
],
'file-2.bat' :
[
'content-2-1',
'content-2-2',
'content-2-3'
],
'file-2' :
[
'content-unix-2-1',
'content-unix-2-2',
'content-unix-2-3',
],
]
files.each {
file_key, file_value -> println file_key;
file_value.each {
file_content -> println "\t"+file_content;
files[ file_key ] = [file_content : true];
}
}
println ' Upgraded content';
files.each {
file_key, file_value -> println file_key;
file_value.each {
file_content -> println "\t"+file_content;
}
}
The following lines causes my problems:
files[ file_key ] = [file_content : true];
What I want to do is to create for every entry in the map also the true part of the key:value pair...I know that I didn't define the list in that way...
I've tried to enhance the map by using files[file_key].put(key, value); but this does not work...Maybe I'm thinking in a complete wrong direction...
The background of that construct is that in the files (file-1.bat, file-1 etc.) I will check the existence of the contents given as a Map
'file-1.bat' : [
'content-1-1',
'content-1-2',
'content-1-3'
],
I could do the following:
'file-1.bat' : [
'content-1-1' : false,
'content-1-2' : false,
'content-1-3' : false,
],
but that's exactly what I want to avoid.

You are right in thinking that put() will solve your issue, but you cannot put a map element into a list. You need to first create a map that can be put into, then assign the resulting map as the output.
For example:
files.each {
file_key, file_value -> println file_key;
def file_map = [:];
file_value.each {
file_content -> println "\t"+file_content;
file_map.put(file_content, true);
}
files[ file_key ] = file_map;
}

Related

Remove empty Keys from JSON arrays using Groovy

I would like to remove the array SEO from the json when the keys "Description" and "Title" in the has no value.
json:
[
{
"SEO": [
{
"Description": "",
"Title": ""
}
],
"accesoires": [
"1167296"
],
"shortCode": "S-576",
"spareParts": [
"800236"
]
}]
I tried the below code but i'm not able to remove the array.
def Message processData(Message message) {
def body = message.getBody(String);
def json = new JsonSlurper().parseText(body)
json.each{
it.SEO.each{
if(!(it.findResults{k, v -> v?.size() > 0 && v[0]?.length() > 0 ? v[0] : null })){
json.remove("SEO")
} } }
def out= JsonOutput.toJson(json)
message.setBody(out)
return message}
To remove the array "SEO" from the JSON when the keys "Description" and "Title" have no value, you can use the following Groovy code:
def jsonString = '[{"SEO": [{"Description": "", "Title": ""}], "accesoires": ["1167296"], "shortCode": "S-576", "spareParts": ["800236"]}]'
def json = new JsonSlurper().parseText(jsonString)
for (item in json) {
if (!item.SEO[0].Description && !item.SEO[0].Title) {
item.remove('SEO')
}
}
println(JsonOutput.toJson(json))
This will first parse the JSON string into a list of maps using JsonSlurper. Then it iterates through each map in the list and checks if the "Description" and "Title" keys in the "SEO" array are empty. If they are, it removes the "SEO" array from the map using the remove() method. Finally, it prints the modified JSON using the JsonOutput.toJson() method.

Is it possible to perform nested iterations in HCL resulting in a flat list without calling flatten?

Is it possible with HCL to have nested iterations returning a flat list(map) without resorting to flatten?
I have this:
locals {
mappings = flatten([
for record_type in var.record_types : [
for host in var.hosts : {
type = record_type,
host = host
}
]
])
}
I would like to remove the need for flatten like this:
locals {
mappings = [
for record_type in var.record_types :
for host in var.hosts : {
type = record_type,
host = host
}
]
}
But it seems like each for .. in must return data.
One alternative I could think of to only have a single for-loop is using setproduct():
variable "record_types" {
default = ["type1", "type2"]
}
variable "hosts" {
default = ["host1", "host2"]
}
locals {
mappings = [
for i in setproduct(var.record_types, var.hosts) : {
type = i[0],
host = i[1],
}
]
}
output "mappings" {
value = local.mappings
}
after terraform apply resulting in:
Outputs:
mappings = [
{
"host" = "host1"
"type" = "type1"
},
{
"host" = "host2"
"type" = "type1"
},
{
"host" = "host1"
"type" = "type2"
},
{
"host" = "host2"
"type" = "type2"
},
]
Of course, the two variables need to be independent sets here.
If you want to support duplicates or have dependent inputs, flatten() with two loops is the way.

Groovy object retrieval using String notation

Writing a utility method for Jenkins pipeline using groovy and noticed its not easy to get value from a deeply nested map using externalised object retrieval notation as string.
Question: If you see the example below, you will notice that its not possible to retrieve object in the form datas."bankctrl.deployment.clientConfigMap.enabled". You definitely can do datas."bankctrl"."deployment"."clientConfigMap"."enabled". I have a workaround, but question is if there is any known best approach to do this?
datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"true", name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
String name = "bankctrl"
datas."${name}".deployment.clientConfigMap.enabled = true
println datas
//following does't work as dots within string are not evaluated.
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
datas."${elementNameToUpdate}" = true
Following code works when you need to access values using externalised string:
String s = "bankctrl.deployment.clientConfigMap.enabled"
def q = s.split( /\./ ).inject( datas ) { obj1, prop -> obj1?."$prop" }
println q
Even this works when you need to access values using externalised string:
String s = "bankctrl.deployment.clientConfigMap.enabled"
Eval.x(datas,"x.${s}")
def gpathSet = { obj,path,value -> Eval.xy(obj, value, "x.${path}=y") }
datas = [
"bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
gpathSet(datas, elementNameToUpdate, true)
One way of workaround.
String s = "bankctrl.deployment.clientConfigMap.enabled"
def setDeepProps(s,datas, value) {
s.split(/\./).inject(datas) {
obj, prop ->
if (obj."$prop" instanceof Map)
obj?."$prop"
else
obj."$prop" = value
}
}
setDeepProps(s,datas,true)
println datas
Groovy doesn't have this functionality out-of-box, so you have to either go wild and use Eval :) which for this certain case is a bit fragile (if the path-string is misspelled it can produce RuntimeExceptions) and not the best performer (due to run-time compilation).
You can solve your problem with a simple recursion like so:
def datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"true", name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
def traverser
traverser = { map, path ->
int ix = path.indexOf '.'
if( -1 == ix ) return map[ path ]
def val = map[ path[ 0..<ix ] ]
if( val in Map )
traverser val, path.substring( ix + 1 )
else
null
}
traverser datas, "bankctrl.deployment.clientConfigMap.enabled"
Below workaround worked for me. The reason I preferred this over any other solution is cause Jenkins is very picky about what script you invoke and hence wanted to have a work around that would not force me to accept script execution in Jenkins while getting around this requirement.
On a side note, would have loved if Jenkins would let me explore full power of Groovy and unfortunately it doesn't because of security reasons.
datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
boolean customConfigFolderExists = true
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
String[] elementNameToUpdateArray = "${elementNameToUpdate}".split("\\.")
def tempDatas = datas
//get property like datas.bankctrl.deployment.clientConfigMap (note that we are leaving enabled out which would be invoked in the following steps)
for (int i=0; i<elementNameToUpdateArray.length-1; i++) {
println "elementNameToUpdateArray[i] -> ${elementNameToUpdateArray[i]}"
tempDatas = tempDatas?."${elementNameToUpdateArray[i]}"
}
//we are invoking property returned from for loop above using the last element from elementNameToUpdateArray
tempDatas?."${elementNameToUpdateArray[elementNameToUpdateArray.length-1]}" = true
println "datas = ${datas}"

Do json key transformation with apache nifi

I need to do a json transformation in apache nifi. The json keys in the payload would be dynamically generated.
For example in the input given below, the 'customer' has attributes 'fname' and 'lname'. I need to change this 'fname' -> 'firstname' and 'lname' -> 'lastname' as provided in the 'mappingvalues'.
Since I am newbie to nifi. I dont know where to start. I have tried some json transformers like jolt. But couldn't achieve the expected result.
The jolt transform that i have used is given below :
[
{
"operation": "shift",
"spec": {
"customer": {
"*": {
"#": "&"
}
}
}
}
]
which produced an output
{
"fname" : "akhil",
"lname" : "kumar"
}
The input and expected output of what I need to achieve is given below :
{
"customer": {
"fname": "akhil",
"lname": "kumar",
.
.
.
},
"mappingvalues": {
"fname": "firstname",
"lname": "lastname",
.
.
.
}
}
##OUTPUT
{
"customer": {
"firstname": "akhil",
"lastname": "kumar",
.
.
.
}
}
*Is there any way to achieve the same in nifi with or without using jolt transform? Is it possible to do the same with groovy script? *
Please help me on the same.
the code in groovy with recursive mapping:
import groovy.json.JsonSlurper
def ff = session.get()
if(!ff)return
def json = ff.read().withReader("UTF-8"){r-> new JsonSlurper().parse(r) }
def mappings = json.remove('mappingvalues')
def mapper(o, mappings){
if(o instanceof Map){
//json object. let's iterate it and do mapping
o = o.collectEntries{k,v-> [ (mappings[k] ?: k), mapper(v,mappings) ] }
}else if(o instanceof List){
//map elements in array
o = o.collect{v-> mapper(v,mappings) }
}
return o
}
json = mapper(json,mappings)
ff.write("UTF-8"){w-> new JsonBuilder(json).writeTo(w) }
REL_SUCCESS << ff

how to get child key without going with parent key or with iteration

1.I am having a list which contains multiple Maps that looks like below Map.
Each map contains many keys I want to get value of "name"
{
"question":{
"com.forms.tree":{
"requiredByDefault":true,
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"language":"en"
},
"formField":{
"name":"CUSTOM_347",
"tag":"input",
"url":"Demo"
}
}
},
"Field":"true"
},{
"question":{
"com.forms.tree":{
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"language":"en"
},
"formField":{
"name":"CUSTOM_348",
"url":"Demo"
}
}
},
"Field":"true"}
I want to get the value of "name" which falls in every Map but don't want to iterate like question?."com.forms.tree"?.formField?.name.
Is there any other approach in groovy?
So given the json:
def jsonTxt = '''{
"question":{
"com.forms.tree":{
"requiredByDefault":true,
"questionDetails":{
"com.forms.Details":{
"preferredFormComponent":"TEXT"
}
},
"locale":{
"name": "test",
"language":"en"
},
"formField":{
"name":"CUSTOM_347",
"tag":"input",
"url":"Demo"
}
}
},
"Field":"true"
}'''
We can parse it with:
import groovy.json.*
def json = new JsonSlurper().parseText(jsonTxt)
You want to find the "formField" entry in that object, so lets write a recursive finder that will walk through out map of maps looking for the first entry with the given key:
static findFirstByKey(Map map, key) {
map.get(key) ?: map.findResult { k, v -> if(v in Map) findFirstByKey(v, key) }
}
And you can then check it works:
assert findFirstByKey(json, 'formField')?.name == "CUSTOM_347"

Resources