Groovy: How do iterate through a map to create a new map with values baed on a specific condition - groovy

I am in no way an expert with groovy so please don't hold that against me.
I have JSON that looks like this:
{
"metrics": [
{
"name": "metric_a",
"help": "This tracks your A stuff.",
"type": "GAUGE",
"labels": [
"pool"
],
"unit": "",
"aggregates": [],
"meta": [
{
"category": "CAT A",
"deployment": "environment-a"
}
],
"additional_notes": "Some stuff (potentially)"
},
...
]
...
}
I'm using it as a source for automated documentation of all the metrics. So, I'm iterating through it in various ways to get the information I need. So far so good, I'm most of the way there. The problem is this all needs to be organized per the deployment environment. Meaning, multiple metrics will share the same value for deployment.
My thought was I could create a map with deployment as the key and the metric name for any metric that has a matching deployment as the value. Once I have that map, it should be easy for me to organize things the way they should be. I can't figure out how to do that. The result is all the metric names are added which is expected since I'm not doing anything to filter them out. I was thinking that groupBy would make sense here but I can't figure out how to use it effectively and frankly I'm not sure it will solve my problem by itself. Here is my code so far:
parentChild = [:]
children = []
metrics.each { metric ->
def metricName = metric.name
def depName = metric.meta.findResult{ it.deployment }
children.add(metricName)
parentChild.put(depName, children)
}
What is the best way to create a new map where the values for each key are based off a specific condition?
EDIT: The desired result would be each key in the resulting map would be a unique deployment value from all the metrics (as a string). Each value would be name of each metric that contains that deployment (as an array).
[environment-a:
[metric_a,metric_b,metric_c,...],
environment-b:
[metric_d,metric_e,metric_f,...]
...]

I would use a combo of withDefault() to pre-fill each map-entry value with a fresh TreeSet-instance (sorted no-duplicates set) and standard inject().
I reduced your sample data to the bare minimum and added some new nodes:
import groovy.json.*
String input = '''\
{
  "metrics": [
{
"name": "metric_a",
"meta": [
{
"deployment": "environment-a"
}
]
},
{
"name": "metric_b",
"meta": [
{
"deployment": "environment-a"
}
]
},
{
"name": "metric_c",
"meta": [
{
"deployment": "environment-a"
},
{
"deployment": "environment-b"
}
]
},
{
"name": "metric_d",
"meta": [
{
"deployment": "environment-b"
}
]
}
  ]
}'''
def json = new JsonSlurper().parseText input
def groupedByDeployment = json.metrics.inject( [:].withDefault{ new TreeSet() } ){ res, metric ->
  metric.meta.each{ res[ it.deployment ] << metric.name }
res
}
assert groupedByDeployment.toString() == '[environment-a:[metric_a, metric_b, metric_c], environment-b:[metric_c, metric_d]]'
If your metrics.meta array is supposed to have a single value, you can simplify the code by replacing the line:
metric.meta.each{ res[ it.deployment ] << metric.name }
with
res[ metric.meta.first().deployment ] << metric.name

Related

How to extract selected key and value from nested dictionary object in a list?

I have a list example_list contains two dict objects, it looks like this:
[
{
"Meta": {
"ID": "1234567",
"XXX": "XXX"
},
"bbb": {
"ccc": {
"ddd": {
"eee": {
"fff": {
"xxxxxx": "xxxxx"
},
"www": [
{
"categories": {
"ppp": [
{
"content": {
"name": "apple",
"price": "0.111"
},
"xxx: "xxx"
}
]
},
"date": "A2020-01-01"
}
]
}
}
}
}
},
{
"Meta": {
"ID": "78945612",
"XXX": "XXX"
},
"bbb": {
"ccc": {
"ddd": {
"eee": {
"fff": {
"xxxxxx": "xxxxx"
},
"www": [
{
"categories": {
"ppp": [
{
"content": {
"name": "banana",
"price": "12.599"
},
"xxx: "xxx"
}
]
},
"date": "A2020-01-01"
}
]
}
}
}
}
}
]
now I want to filter the items and only keep "ID": "xxx" and the correspoding value for "price": "0.111", expected result can be something similar to :
[{"ID": "1234567", "price": "0.111"}, {"ID": "78945612", "price": "12.599"}]
or something like {"1234567":"0.111", "78945612":"12.599" }
Here's what I've tried:
map_list=[]
map_dict={}
for item in example_list:
#get 'ID' for each item in 'meta'
map_dict['ID'] = item['meta']['ID']
# get 'price'
data_list = item['bbb']['ccc']['ddd']['www']
for data in data_list:
for dataitem in data['categories']['ppp']
map_dict['price'] = item["content"]["price"]
map_list.append(map_dict)
print(map_list)
The result for this doesn't look right, feels like the item isn't iterating properly, it gives me result:
[{"ID": "78945612", "price": "12.599"}, {"ID": "78945612", "price": "12.599"}]
It gave me duplicated result for the second ID but where is the first ID?
Can someone take a look for me please, thanks.
Update:
From some comments from another question, I understand the reason for the output keeps been overwritten is because the key name in the dict is always the same, but I'm not sure how to fix this because the key and value needs to be extracted from different level of for loops, any help would be appreciated, thanks.
as #Scott Hunter has mentioned, you need to create a new map_dict everytime you are trying to do this. Here is a quick fix to your solution (I am sadly not able to test it right now, but it seems right to me).
map_list=[]
for item in example_list:
# get 'price'
data_list = item['bbb']['ccc']['ddd']['www']
for data in data_list:
for dataitem in data['categories']['ppp']:
map_dict={}
map_dict['ID'] = item['meta']['ID']
map_dict['price'] = item["content"]["price"]
map_list.append(map_dict)
print(map_list)
But what are you doing here is that you are basically just "forcing" your way through ... I recommend you to take a break and check out somekind of tutorial, which will help you to understand how it really works in the back-end. This is how I would have written it:
list_dicts = []
for example in example_list:
for www in item['bbb']['ccc']['ddd']['www']:
for www_item in www:
list_dicts.append({
'ID': item['meta']['ID'],
'price': www_item["content"]["price"]
})
Good luck with this problem and hope it helps :)
You need to create a new dictionary for map_dict for each ID.

Inject matchesJsonPath from Groovy into Spring Cloud Contract

When writing a Spring Cloud Contract in Groovy,
I want to specify an explicit JSON path expression.
The expression:
"$.['variants'][*][?(#.['name'] == 'product_0004' && #.['selected'] == true)]"
shall appear in the generated json, like so:
{
"request" : {
"bodyPatterns": [ {
"matchesJsonPath": "$.['variants'][*][?(#.['name'] == 'product_0004' && #.['selected'] == true)]"
} ]
}
}
in order to match e.g.:
{ "variants": [
{ "name": "product_0003", "selected": false },
{ "name": "product_0004", "selected": true },
{ "name": "product_0005", "selected": false } ]
}
and to not match e.g.:
{ "variants": [
{ "name": "product_0003", "selected": false },
{ "name": "product_0004", "selected": false },
{ "name": "product_0005", "selected": true } ]
}
Is this possible using consumers, bodyMatchers, or some other facility of the Groovy DSL?
There are some possibilities with matching on json path, but you wouldn't necessarily use it for matching on explicit values, but rather to make a flexible stub for the consumer by using regex. There are some possibilities though.
So the body section is your static request body with hardcoded values, while the bodyMatchers section provides you the ability to make the stub matching from the consumer side more flexible.
Contract.make {
request {
method 'POST'
url '/some-url'
body ([
id: id
items: [
foo: foo
bar: bar
],
[
foo: foo
bar: foo
]
])
bodyMatchers {
jsonPath('$.id', byEquality()) //1
jsonPath('$.items[*].foo', byRegex('(?:^|\\W)foo(?:$|\\W)')) //2
jsonPath('$.items[*].bar', byRegex(nonBlank())) //3
}
headers {
contentType(applicationJson())
}
}
response {
status 200
}
}
I referenced some lines
1: "byEquality()" in the bodyMatchers section means: the input from the consumer must be equal to the value provided in the body for this contract/stub to match, in other words must be "id".
2: I'm not sure how nicely the //1 solution will work when the property is in a list, and you want the stub to be flexible with the amount of items provided. Therefor I also included this byRegex which basically means, for any item in the list, the property foo must have exactly value "foo". However, I dont really know why you would want to do this.
3: This is where bodyMatchers are actually most useful. This line means: match to this contract if every property bar in the list of items is a non blank string. This allows you to have a dynamic stub with a flexible size of lists/arrays.
All the conditions in bodyMatchers need to be met for the stub to match.

How to Generate Counts of Elements Returned from Map Function?

I have a map function
function (doc) {
for(var n =0; n<doc.Observations.length; n++){
emit(doc.Scenario, doc.Observations[n].Label);
}
}
the above returns the following:
{"key":"Splunk","value":"Organized"},
{"key":"Splunk","value":"Organized"},
{"key":"Splunk","value":"Organized"},
{"key":"Splunk","value":"Generate"},
{"key":"Splunk","value":"Ingest"}
I"m looking to design a reduce function that will then return the counts of the above values, something akin to:
Organized: 3
Generate: 1
Ingest: 1
My map function has to filter on my Scenario field, hence why I have it as an emitted key in the map function.
I've tried using a number of the built in reduce functions, but I end up getting count of rows, or nothing at all as the functions available don't apply.
I just need to access the counts of each of the elements that appear in the values field. Also, the values present here are representative, there could 100s of different types of values found in the values field for what that's worth.
I really appreciate the help!
Here's sample input:
{
"_id": "dummyId",
"test": "test",
"Team": "Alpha",
"CreatedOnUtc": "2019-06-20T21:39:09.5940830Z",
"CreatedOnLocal": "2019-06-20T17:39:09.5940830-04:00",
"Participants": [
{
"Name": "A",
"Role": "Person"
}
],
"Observations": [
{
"Label": "Report",
},
{
"Label": "Ingest",
},
{
"Label": "Generate",
},
{
"Label": "Ingest",
}
]
}
You can set the map by "value" as your key and associate an increment to that key to make sure a count is maintained. And then you can print your map which should look as you are requesting for.

How can I merge the outputs from a For_Each loop in an Azure Logic App to a single flat array?

I have a For_Each loop in an Azure Logic App that calls another, nested, Logic App. The result from each iteration of the nested Logic Apps is a JSON object that contains an array of strings, like this:
{
"Results": ["string a", "string b"]
}
So the output from my For_Each loop in the parent Logic App looks like this:
[
{"Results": ["string a", "string b"]},
{"Results": ["string c", "string d"]}
]
I want to put all these strings into a single flat list that I can pass to another action.
How can I do this? Is it possible using the workflow definition language and built-in functions, or do I need to use an external function (in a service, or an Azure Function)?
There's a simpler solution, working with Array Variables.
At the top level, outside the For Each loop, declare a variable with an InitializeVariable action:
"Initialize_Items_variable": {
"inputs": {
"variables": [
{
"name": "Items",
"type": "Array",
"value": []
}
]
},
"runAfter": {},
"type": "InitializeVariable"
}
Inside the For Each, use a AppendToArrayVariable action. You can append the Response object of the Nested Logic App you just called.
"Append_to_Items_variable": {
"inputs": {
"name": "Items",
"value": "#body('Nested_Logic_App_Response')"
},
"runAfter": {
},
"type": "AppendToArrayVariable"
}
Hope it helps.
Picking up on #DerekLi's useful comment above, it seems this is not possible at the time of writing with Logic Apps schema version 2016-06-01.
One of the great strengths of Logic Apps is the ability to leverage the power of Azure Functions to solve problems like this that can't (yet) be solved in the schema language.
Re-writing the array is trivial in c# within a function:
using System.Net;
public class Result
{
public List<string> Results {get; set;}
}
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
var inputs = await req.Content.ReadAsAsync<List<Result>>();
var outputs = new List<string>();
foreach(var item in inputs)
{
log.Info(item.Results.ToString());
outputs.AddRange(item.Results.Where(x => !string.IsNullOrEmpty(x)));
}
return req.CreateResponse(HttpStatusCode.OK, outputs);
}
And this function can then be passed the result of the For_Each loop:
"MyFunction": {
"inputs": {
"body": "#body('Parse_JSON')",
"function": {
"id": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{function-app-name}/functions/{function-name}"
},
"method": "POST"
},
"runAfter": {
"For_each": [
"Succeeded"
]
},
"type": "Function"
}
There is also a way to do it using the workflow definition language. (https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-workflow-definition-language).
Using the fonctions string and replace you can work on your json as a string rather than on objects.
Here is a Flat_List action that follows a Parse_JSON action with your data:
Your data:
[
{"Results": ["string a", "string b"]},
{"Results": ["string c", "string d"]}
]
Flat_List component:
"Flat_List": {
"inputs": "#replace(replace(replace(string(body('Parse_JSON')),']},{\"Results\":[',','),'}]','}'),'[{','{')",
"runAfter": {
"Parse_JSON": [
"Succeeded"
]
},
"type": "Compose"
},
What happens here? First we use string that takes your json data and gives:
[{"Results":["string a", "string b"]},{"Results":["string c", "string d"]}]
We replace all the ]},{"Results":[ by ,.
We replace all the }] by }.
We replace all the [{ by {.
We get the string {"Results":["string a","string b","string c","string d"]}
Then you are free to parse it back to json with:
"Parse_JSON_2": {
"inputs": {
"content": "#outputs('Flat_List')",
"schema": {
"properties": {
"Results": {
"items": {
"type": "string"
},
"type": "array"
}
},
"type": "object"
}
},
"runAfter": {
"Flat_List": [
"Succeeded"
]
},
"type": "ParseJson"
}
You can see it as a proof of concept as the Azure Function may be easier to re-read later but there may be many reason not to want to instantiate a new Azure Function while you can do the job in Logic App.
Feel free to ask for more details if needed :)
This technique works pretty well, and only uses run-of-the-mill Logic App actions:
1. start with declaring an empty array variable (action Variable: Initialise variable)
2. iterate through your items (action Control: For each), e.g. the resultset from a previous action
in each iteration, first compose the JSON fragment you need (action Data Operations: Compose)
then append the output of your Compose action to the array (action: Variable: Append to array variable)
3. then, outside the loop, join the elements of the array (action Data Operations: Join)
4. do what you need with the output of the Join action, e.g. send as response payload (action Request: Response)
This is what it looks like in the end:
You can use #body(nestedLogicApp) outside of the for-each loop to access all the nested Logic Apps' response in an array.

Performing a query on the lowest level of a tree-structured Dojo store

Let's say we have a nested data structure like so:
[
{
"name": "fruits",
"items": [
{ "name": "apple" ...}
{ "name": "lemon" ...}
{ "name": "peach" ...}
]
}
{
"name": "veggies",
"items": [
{ "name": "carrot" ...}
{ "name": "cabbage" ...}
]
}
{
"name": "meat",
"items": [
{ "name": "steak" ...}
{ "name": "pork" ...}
]
}
]
The above data is placed in a dojo/store/Memory. I want to perform a query for items that contain the letter "c", but only on the lower level (don't want to query the categories).
With a generic dojo/store/Memory, it's query function only applies a filter on the top level, so the code
store.query(function(item) {
return item.name.indexOf("c") != -1;
});
will only perform the query on the category names (fruits, veggies, etc) instead of the actual items.
Is there a straight-forward way to perform this query on the child nodes, and if there's a match, return all children as well as the parent? For instance, the "c" query would return the "fruits" node with it's "peach" child only, "veggies" would remain intact, and "meat" would be left out of the query results entirely.
You can of course define your own checking method in the store's query method. I don't check if this code runs perfectly, but I guess you could pretty much get what it's meant to do.
store.query(function(item) {
var found = {
name: "",
items: []
};
var children = item.items;
d_array.forEach(children, function(child) {
if (child.name.indexOf("c") != -1) {
found.name = item.name;
found.items.push(child);
}
});
return found;
});
Hope this helps.

Resources