Writing a utility method for Jenkins pipeline using groovy and noticed its not easy to get value from a deeply nested map using externalised object retrieval notation as string.
Question: If you see the example below, you will notice that its not possible to retrieve object in the form datas."bankctrl.deployment.clientConfigMap.enabled". You definitely can do datas."bankctrl"."deployment"."clientConfigMap"."enabled". I have a workaround, but question is if there is any known best approach to do this?
datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"true", name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
String name = "bankctrl"
datas."${name}".deployment.clientConfigMap.enabled = true
println datas
//following does't work as dots within string are not evaluated.
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
datas."${elementNameToUpdate}" = true
Following code works when you need to access values using externalised string:
String s = "bankctrl.deployment.clientConfigMap.enabled"
def q = s.split( /\./ ).inject( datas ) { obj1, prop -> obj1?."$prop" }
println q
Even this works when you need to access values using externalised string:
String s = "bankctrl.deployment.clientConfigMap.enabled"
Eval.x(datas,"x.${s}")
def gpathSet = { obj,path,value -> Eval.xy(obj, value, "x.${path}=y") }
datas = [
"bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
gpathSet(datas, elementNameToUpdate, true)
One way of workaround.
String s = "bankctrl.deployment.clientConfigMap.enabled"
def setDeepProps(s,datas, value) {
s.split(/\./).inject(datas) {
obj, prop ->
if (obj."$prop" instanceof Map)
obj?."$prop"
else
obj."$prop" = value
}
}
setDeepProps(s,datas,true)
println datas
Groovy doesn't have this functionality out-of-box, so you have to either go wild and use Eval :) which for this certain case is a bit fragile (if the path-string is misspelled it can produce RuntimeExceptions) and not the best performer (due to run-time compilation).
You can solve your problem with a simple recursion like so:
def datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"true", name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
def traverser
traverser = { map, path ->
int ix = path.indexOf '.'
if( -1 == ix ) return map[ path ]
def val = map[ path[ 0..<ix ] ]
if( val in Map )
traverser val, path.substring( ix + 1 )
else
null
}
traverser datas, "bankctrl.deployment.clientConfigMap.enabled"
Below workaround worked for me. The reason I preferred this over any other solution is cause Jenkins is very picky about what script you invoke and hence wanted to have a work around that would not force me to accept script execution in Jenkins while getting around this requirement.
On a side note, would have loved if Jenkins would let me explore full power of Groovy and unfortunately it doesn't because of security reasons.
datas = [
"ao":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-ao-custom"
], image:[
name:"test/mit/ao",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
], "bankctrl":[
deployment:[
clientConfigMap:[
enabled:"false",
name:"volume-bankctrl-custom"
], image:[
name:"test/mit/bankctrl",
tag:"10.0.0-3.0.0",
pullPolicy:"IfNotPresent"
]
]
]
]
boolean customConfigFolderExists = true
String elementNameToUpdate = "bankctrl.deployment.clientConfigMap.enabled"
String[] elementNameToUpdateArray = "${elementNameToUpdate}".split("\\.")
def tempDatas = datas
//get property like datas.bankctrl.deployment.clientConfigMap (note that we are leaving enabled out which would be invoked in the following steps)
for (int i=0; i<elementNameToUpdateArray.length-1; i++) {
println "elementNameToUpdateArray[i] -> ${elementNameToUpdateArray[i]}"
tempDatas = tempDatas?."${elementNameToUpdateArray[i]}"
}
//we are invoking property returned from for loop above using the last element from elementNameToUpdateArray
tempDatas?."${elementNameToUpdateArray[elementNameToUpdateArray.length-1]}" = true
println "datas = ${datas}"
Related
trying to create the below json structure with groovy jsonBuilder in jmeter using JSR223 sampler
{
"Catalogues":[
{
"Catalogue":{
"itemName":"XYZ",
"Level":"Class1",
"Color":"Orange",
"Id":"232f2d6820822840"
},
"sales":[
[
"7:19 PM 3-31-2022",
"gadfma53742w3585657er43"
],
[
"5:02 PM 3-30-2022",
"iytyvh53742w3585657er41"
]
]
}
]
}
I have tried the below groovy script
def json = new groovy.json.JsonBuilder()
json {
Catalogues(
[{
Catalogue {
itemName('XYZ')
Level('Class1')
Color('Orange')
Id('232f2d6820822840')
},
{
sales(
('7:19 PM 3-31-2022'), ('gadfma53742w3585657er43')
)
}
}]
)
}
log.info '\n\n\n' + json.toPrettyString() + '\n\n\n'
Output:
{
"Catalogues":[
{
"Catalogue":[
{
"itemName":"XYZ",
"Level":"Class1",
"Color":"Orange",
"Id":"232f2d6820822840"
},
{
"sales":[
"7:19 PM 3-31-2022",
"gadfma53742w3585657er43"
]
}
]
}
]
}
Problems:
If I remove the '{' before sales and after (at corresponding location), it adds sales values into catalogue
unable to include second set of sales values
I'm suggesting another way to use builder because it's easier to understand.
To declare array in groovy use [1, 2, 3]
To declare map [a:1, b:2, c:3]
So, if you replace in original json { to [ - you will get valid groovy object that corresponds to parsed json
def data = [
"Catalogues":[
[
"Catalogue":[
"itemName":"XYZ",
"Level":"Class1",
"Color":"Orange",
"Id":"232f2d6820822840"
},
"sales":[
[
"7:19 PM 3-31-2022",
"gadfma53742w3585657er43"
],
[
"5:02 PM 3-30-2022",
"iytyvh53742w3585657er41"
]
]
]
]
]
//then you could convert it to json:
def json = new groovy.json.JsonBuilder(data).toPrettyString()
log.info(json)
JsonBuilder translates Map to JSON Object and List to JSON Array
So I would recommend for better readability and clarity amending your code to look like:
def body = [:]
def Catalogues = []
def Catalogue = [:]
def entry = [:]
Catalogue.put('itemName', 'XYZ')
Catalogue.put('Level', 'Class1')
Catalogue.put('Color', 'Orange')
Catalogue.put('Id', '232f2d6820822840')
def sales = []
sales.add(['7:19 PM 3-31-2022', 'gadfma53742w3585657er43'])
sales.add(['5:02 PM 3-30-2022', 'iytyvh53742w3585657er41'])
entry.put('Catalogue', Catalogue)
entry.put('sales', sales)
Catalogues.add(entry)
body.put('Catalogues', Catalogues)
def json = new groovy.json.JsonBuilder(body)
More information:
JsonBuilder
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
dataset_bindings = {
"infra":[
"group:infra-team#xxxx.com",
],
"finance":[
"group:finance-data#xxx.com",
],
"marketing": [
"group:marketing#xxx.com"
]
}
How can I get all the emails as string. I need to loop thru the dict and get the values and convert those values to string.
You can do this with values and flatten:
locals {
dataset_bindings = {
"infra":[
"group:infra-team#xxxx.com",
],
"finance":[
"group:finance-data#xxx.com",
],
"marketing": [
"group:marketing#xxx.com"
]
}
list_of_emails = flatten(values(local.dataset_bindings))
}
results in:
list_of_emails = [
"group:finance-data#xxx.com",
"group:infra-team#xxxx.com",
"group:marketing#xxx.com",
]
I have the issue that I'm not able to execute the following code. The syntax seems to be okay, but when I try to execute it, I get the response, that:
Expression.Error: We cannot convert a value of type Record to type "Text".
Details:
Value=[Record]
Type=[Type]
let
body="{
""page"": ""1"",
""pageSize"": ""100"",
""requestParams"": {
""deviceUids"": [
""xxx-yyy-xxx-yyyy-xxxx"",
""yyy-xxx-yyy-xxxx-yyyy"",
""aaa-bbb-aaa-bbbb-aaaa"",
""ccc-ddd-ccc-dddd-cccc""
],
""entityColumns"": [
{
""entityId"": ""144"",
""joinColumnName"": ""device_uid"",
""columnName"": ""device_random_date""
}
],
""columnNames"": [
""ts"",
""device_uid"",
""1"",
""32"",
""55"",
""203"",
""204""
],
""startUnixTsMs"": ""1583413637000"",
""endUnixTsMs"": ""1583413640000"",
""columnFilters"": [
{
""filterType"": ""eq"",
""columnName"": ""55"",
""value"": ""1234""
}
],
""sortOrder"": [
{
""column"": ""ts"",
""order"": ""DESC""
},
{
""column"": ""55"",
""order"": ""ASC""
}
],
""entityFilters"": [
{
""entityId"": ""144"",
""entityEntryIds"": [
""12345-221-232-1231-123456""
]
}
]
}
}",
Parsed_JSON = Json.Document(body),
BuildQueryString = Uri.BuildQueryString(Parsed_JSON),
Quelle = Json.Document(Web.Contents("http://localhost:8101/device-data-reader-api/read-paginated/xxx-xxx-yyyy-yyyy", [Headers=[#"Content-Type"="application/json"], Content = Text.ToBinary(BuildQueryString)]))
in
Quelle
I tried to remove the quotes of the numbers, but this leads to the same issue, as system complains it cannot convert numbers into text.
I need the body which needs to be handed over with the request in order to do a POST request. What I'm doing wrong?
Since you seem to want to send this as application/json, I think you would change this bit in your code:
Content = Text.ToBinary(BuildQueryString)
to:
Content = Text.ToBinary(body)
and then you'd also get rid of the lines below (since you don't need them):
Parsed_JSON = Json.Document(body),
BuildQueryString = Uri.BuildQueryString(Parsed_JSON),
I don't think you would need Uri.BuildQueryString unless you wanted to send as application/x-www-form-urlencoded (i.e. URL encoded key-value pairs).
Unrelated: If it helps, you can build the structure in M and then use JSON.FromValue to turn the structure into bytes which can be put directly into the POST body. Untested example is below.
let
body = [
page = "1",
pageSize = "100",
requestParams = [
deviceUids = {
"xxx-yyy-xxx-yyyy-xxxx",
"yyy-xxx-yyy-xxxx-yyyy",
"aaa-bbb-aaa-bbbb-aaaa",
"ccc-ddd-ccc-dddd-cccc"
},
entityColumns = {
[
entityId = "144",
joinColumnName = "device_uid",
columnName = "device_random_date"
]
},
columnNames = {
"ts",
"device_uid",
"1",
"32",
"55",
"203",
"204"
},
startUnixTsMs = "1583413637000",
endUnixTsMs = "1583413640000",
columnFilters = {
[
filterType = "eq",
columnName = "55",
value = "1234"
]
},
sortOrder = {
[
column = "ts",
order = "DESC"
],
[
column = "55",
order = "ASC"
]
},
entityFilters = {
[
entityId = "144",
entityEntryIds = {
"12345-221-232-1231-123456"
}
]
}
]
],
Quelle = Json.Document(
Web.Contents(
"http://localhost:8101/device-data-reader-api/read-paginated/xxx-xxx-yyyy-yyyy",
[
Headers = [#"Content-Type" = "application/json"],
Content = Json.FromValue(body)
]
)
)
in
Quelle
It might look a little weird (since M uses [] instead of {}, {} instead of [] and = instead of :), but just mentioning in case it helps.
I have a List of Map with nested Map as well as below :-
def list = [
[
"description": "The issue is open and ready for the assignee to start work on it.",
"id": "1",
"name": "Open",
"statusCategory": [
"colorName": "blue-gray",
"id": 2,
"key": "new",
"name": "To Do",
]
],
[
"description": "This issue is being actively worked on at the moment by the assignee.",
"id": "3",
"name": "In Progress",
"statusCategory": [
"colorName": "yellow",
"id": 4,
"key": "indeterminate",
"name": "In Progress",
]
]
]
I have a task to get List of subMap with nested subMap. I'm doing some thing like as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"])}
}
println getSubMap(list)
But its give me output as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"colorName":"blue-gray",
"id":2,
"key":"new",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"colorName":"yellow",
"id":"4",
"key":"indeterminate",
"name":"In Progress"
]
]
]
As you can see I'm unable to get subMap of statusCategory key Map. Actually I want to get further subMap for nested Maps something like as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"id":"2",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"id":"4",
"name":"In Progress"
]
]
]
To achieve this I'm trying as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory":["id","name"]])}
}
def modifiedList = getSubMap(list)
But it throws me Excpetion. And If I'm doing as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"]).statusCategory.subMap(["id","name"])}
}
println getSubMap(list)
It gives only nested subMap as :-
[["id":"2", "name":"To Do"], ["id":"4", "name":"In Progress"]]
could anyone suggest me how to recurselvely find List of subMap with nested subMap if exist?
If your Map nesting is arbitrary, then you might want to consider something like this:
def nestedSubMap
nestedSubMap = { Map map, List keys ->
map.subMap(keys) + map.findAll { k, v -> v instanceof Map }.collectEntries { k, v -> [(k):nestedSubMap(v, keys)] }
}
Given your input and this closure, the following script:
def result = list.collect { nestedSubMap(it, ["id", "name"]) }
println '['
result.each { print it; println ',' }
println ']'
Produces this output:
[
[id:1, name:Open, statusCategory:[id:2, name:To Do]],
[id:3, name:In Progress, statusCategory:[id:4, name:In Progress]],
]
Given the original list, consider this:
def resultList = list.collect {
def fields = ["id", "name"]
def m = it.subMap(fields)
m["statusCategory"] = it["statusCategory"].subMap(fields)
return m
}
which supports these assertions:
assert 1 == resultList[0]["id"] as int
assert "Open" == resultList[0]["name"]
assert 2 == resultList[0]["statusCategory"]["id"] as int
assert "To Do" == resultList[0]["statusCategory"]["name"]
assert 3 == resultList[1]["id"] as int
assert "In Progress" == resultList[1]["name"]
assert 4 == resultList[1]["statusCategory"]["id"] as int
assert "In Progress" == resultList[1]["statusCategory"]["name"]
I have the following groovy-script:
#!/usr/bin/env groovy
def files = [
'file-1.bat' :
[
'content-1-1',
'content-1-2',
'content-1-3'
],
'file-1' :
[
'content-unix-1-1',
'content-unix-1-2',
'content-unix-1-3',
],
'file-2.bat' :
[
'content-2-1',
'content-2-2',
'content-2-3'
],
'file-2' :
[
'content-unix-2-1',
'content-unix-2-2',
'content-unix-2-3',
],
]
files.each {
file_key, file_value -> println file_key;
file_value.each {
file_content -> println "\t"+file_content;
files[ file_key ] = [file_content : true];
}
}
println ' Upgraded content';
files.each {
file_key, file_value -> println file_key;
file_value.each {
file_content -> println "\t"+file_content;
}
}
The following lines causes my problems:
files[ file_key ] = [file_content : true];
What I want to do is to create for every entry in the map also the true part of the key:value pair...I know that I didn't define the list in that way...
I've tried to enhance the map by using files[file_key].put(key, value); but this does not work...Maybe I'm thinking in a complete wrong direction...
The background of that construct is that in the files (file-1.bat, file-1 etc.) I will check the existence of the contents given as a Map
'file-1.bat' : [
'content-1-1',
'content-1-2',
'content-1-3'
],
I could do the following:
'file-1.bat' : [
'content-1-1' : false,
'content-1-2' : false,
'content-1-3' : false,
],
but that's exactly what I want to avoid.
You are right in thinking that put() will solve your issue, but you cannot put a map element into a list. You need to first create a map that can be put into, then assign the resulting map as the output.
For example:
files.each {
file_key, file_value -> println file_key;
def file_map = [:];
file_value.each {
file_content -> println "\t"+file_content;
file_map.put(file_content, true);
}
files[ file_key ] = file_map;
}