I have data file site24x7IPs.json, it looks like below:
{
"LocationDetails": [
{
"IPv6_Address_External": "2803:eb80:4000:d::0/64",
"City": "Buenos Aires",
"Place": "Argentina",
"external_ip": "170.78.75.88"
},
{
"IPv6_Address_External": "",
"City": "Buenos Aires",
"Place": "Argentina",
"external_ip": "170.78.75.87"
},
{
"IPv6_Address_External": "",
"City": "Melbourne",
"Place": "Australia",
"external_ip": "103.91.166.0/24"
},
{
"IPv6_Address_External": "2400:fa80:5:9:d68e:c0c1:fced:a31a",
"City": "Perth",
"Place": "Australia",
"external_ip": "103.77.234.74"
}
]
}
Below is part of my terraform main.tf
locals {
site24x7IPs = jsondecode(file("${path.module}/site24x7IPs.json"))
}
locals {
ipList = [for i in local.site24x7IPs.LocationDetails: i.external_ip if i.Place == "Australia"]
cidrList = [ for j in local.ipList: length(regexall("/", j)) <= 0 : "${j}/32" : j ]
}
My expectation:
cidrList = [ "103.91.166.0/24", "103.77.234.74/32" ]
Error:
Error: Invalid 'for' expression
on main.tf line 7, in locals: 7: cidrList = [ for j in
local.ipList: length(regexall("/", j)) <= 0 : "${j}/32" : j ]
Extra characters after the end of the 'for' expression.
Update:
The correct syntax is:
cidrList = [ for j in local.ipList:
length(regexall("/", j)) <= 0 ? "${j}/32" : j ]
but you are using double :, instead of ? and ':'.
Previous answer
Not sure what exactly are you trying to achieve, it seems to be that you are after something as follows:
cidrList = [ for j in local.ipList:
{"${j}/32" : j } if length(regexall("/", j)) <= 0 ]
which would give:
[
{
"103.77.234.74/32" = "103.77.234.74"
},
]
or
cidrList = [ for j in local.ipList:
"${j}/32" if length(regexall("/", j)) <= 0 ]
which gives:
[
"103.77.234.74/32",
]
Related
I would like to make comparison between two values in a OPA rule.
My OPA rule is :
package example
vm_name = input.planned_values.root_module.resources[0].values.name
vm_name_length = count(vm_name)
The output is :
~$ ./opa eval -i tfplan.json -d test1.rego "data.example"
{
"result": [
{
"expressions": [
{
"value": {
"vm_name": "XXXXXXXX",
"vm_name_length": 8
},
"text": "data.example",
"location": {
"row": 1,
"col": 1
}
}
]
}
]
}
I've tried to make a comparison between the length needed and the length of the VM name :
package example
vm_name = input.planned_values.root_module.resources[0].values.name
vm_name_length = count(vm_name)
if vm_name_length == 13 {
msg := "OK"
}
else {
msg := "Not ok"
}
But the output is :
{
"errors": [
{
"message": "unexpected assign token: non-terminated set",
"code": "rego_parse_error",
"location": {
"file": "test1.rego",
"row": 24,
"col": 7
},
"details": {
"line": " msg := \"OK\"",
"idx": 6
}
}
]
}
Does anyone know how to do that ?
Rego rules describe conditional assignment, so all conditions in the rule body (i.e. inside of the { ... }) will need to be true in order for the rule to evaluate the assignment.
package example
vm_name = input.planned_values.root_module.resources[0].values.name
vm_name_length = count(vm_name)
default msg := "Not ok"
msg := "OK" {
vm_name_length == 13
}
How can we generate an yaml out from a json
{
"users":[
{
"name":"rock",
"age":33,
"city":"paris"
},
{
"name":"austin",
"age":45,
"city":"greece"
}
]
}
Expected output with a additional field with random password
users:
- key: 'user[0].name'
value: rock
- key: 'user[0].age'
value: '33'
- key: 'user[0].city'
value: paris
- key: 'user[0].password'
value: '5]L+J7rA*<7+:PO6'
- key: 'user[1].name'
value: austin
- key: 'user[1].age'
value: '45'
- key: 'user[1].city'
value: greece
- key: 'user[1].password'
value: P=x&385YGMI0?!Is
The files are used only on a local machine.
A straight conversion from JSON to YAML would involve just passing the result of jsondecode to yamlencode.
However, the meat of your question seems to not be about conversion between serialization formats but rather about transforming from a typical nested data structure into a flattened structure using a compound key syntax.
Transforming from a multi-level structure into a single-level structure is a good job for flatten:
locals {
users = flatten([
for idx, u in local.users : [
for k, v in u : {
key = "user[${idx}].${v}"
value = v
}
]
])
}
The result would be a Terraform value corresponding to the data structure you illustrated in YAML:
[
{
key = "user[0].name"
value = "rock"
},
{
key = "user[0].age"
value = 33
},
# ...etc...
]
You can then pass the result to yamlencode to produce the YAML serialization of that data structure.
You can use file function of terraform to read the content of your json.
You can then use jsondecode to decode/represent the string as json.
The last step is to use , yamlencode function to encode returned json representation in yaml format.
Please see below terraform config.
locals {
test = yamlencode(jsondecode(file("${path.module}/test.json")))
}
output "test" {
value = local.test
}
The result from the above tf config is as below
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
test = "users":
- "age": 33
"city": "paris"
"name": "rock"
- "age": 45
"city": "greece"
"name": "austin"
locals {
users = jsondecode(file("${path.module}/user.json"))
}
resource "random_string" "password" {
count = length(local.users.users)
length = 16
special = true
}
output "pass" {
value = random_string.password
}
locals {
user_flat = yamlencode(flatten([for i,j in local.users.users:[
for k,v in j : {
key = "user.[${i}].${k}"
"value" = v
}
]
]
)
)
out = yamlencode({"users"= local.user_flat})
}
output "Desired_output" {
value = local.out
}
Try this code.
Outputs:
Desired_output = <<EOT
"users": |
- "key": "user.[0].age"
"value": 23
- "key": "user.[0].city"
"value": "barcelona"
- "key": "user.[0].name"
"value": "john"
- "key": "user.[1].age"
"value": 29
- "key": "user.[1].city"
"value": "london"
- "key": "user.[1].name"
"value": "bob"
EOT
pass = [
{
"id" = "XEXj*svXm#d&_%Fn"
"keepers" = tomap(null) /* of string */
"length" = 16
"lower" = true
"min_lower" = 0
"min_numeric" = 0
"min_special" = 0
"min_upper" = 0
"number" = true
"override_special" = tostring(null)
"result" = "XEXj*svXm#d&_%Fn"
"special" = true
"upper" = true
},
{
"id" = "Ews%V%Xk[YBd]D_M"
"keepers" = tomap(null) /* of string */
"length" = 16
"lower" = true
"min_lower" = 0
"min_numeric" = 0
"min_special" = 0
"min_upper" = 0
"number" = true
"override_special" = tostring(null)
"result" = "Ews%V%Xk[YBd]D_M"
"special" = true
"upper" = true
},
]
I have below data as input, looking to create a data structure as below.
Input:
Key,type,alias
Aggregator_aggregator_se,Sorter,So_so
Aggregator_aggregator_se,Sorter,So_so
Aggregator_aggregator_se,Sorter,So_so
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Sorter,So_so
Expression_expr_se,Sorter,So_so
Expression_expr_se,Aggregator,Ag_ag
Expression_expr_se,Aggregator,Ag_ag
Filter_filter_se,Expression,Ex_ex
Filter_filter_se,Expression,Ex_ex
Filter_filter_se,Expression,Ex_ex
Filter_filter_se,Expression,Ex_ex
Filter_filter_se,Expression,Ex_ex
Output:
{ 'Aggregator_aggregator_se' : [ {type: 'Sorter', count: 3, value: 'So_so'],
'Expression_expr_se' : [ {type: 'Aggregator', count: 7, value: 'Ag_ag'}, {type: 'Sorter', count: 2, value: 'So_so'}],
'Filter_filter_se' : [ {type: 'Expression', count: 5,value: 'Ex_ex']
}
How should I achive this data structure?
I am very new to python so need some help.
Try:
import csv
out = {}
with open("your_data_file.txt") as f_in:
reader = csv.reader(f_in)
# skip header
next(reader)
for line in reader:
# skip empty lines
if not line:
continue
key, type_, value = line
out.setdefault(key, {}).setdefault(type_, {}).setdefault(value, 0)
out[key][type_][value] += 1
out = {
k: [
{"type": kk, "count": vvv, "value": kkk}
for kk, vv in v.items()
for kkk, vvv in vv.items()
]
for k, v in out.items()
}
print(out)
Prints:
{
"Aggregator_aggregator_se": [
{"type": "Sorter", "count": 3, "value": "So_so"}
],
"Expression_expr_se": [
{"type": "Aggregator", "count": 7, "value": "Ag_ag"},
{"type": "Sorter", "count": 2, "value": "So_so"},
],
"Filter_filter_se": [{"type": "Expression", "count": 5, "value": "Ex_ex"}],
}
I want to get a specific map element from a lists of maps in Terraform's output.
For example, how would I access account in properties.json for each map item
and print a list as output conditionally for type also.
I tried for loops and splat expressions as below but it does not return exact value.
It seems that the props below is also a list.
output "resources_by_name" {
description = "Resource name of all machine type resources from a vRA deployment"
value = [
for props in deployment.deploy[*].resources.*.properties_json:
jsondecode(props).account
if jsondecode(props).type == "vsphere"
]
}
I am not sure how to use nested for loops or access map items within the lists.
properties.json
[
[
{
"id" = "b5336bf7-07fb-4026-aa3d-479bd974ca45"
"name" = "test1"
"properties_json" = "{"account":"test0","constraints":"anothertest4"}"
"type" = "vsphere"
},
{
"id" = "67a3380b-8008-4f9c-9c13-2a1a935d5820"
"name" = "test2"
"properties_json" = "{"account":"test1","constraints":"anothertest3"}"
"type" = "gcp"
},
],
[
{
"id" = "eeddd127-cba2-4b34-a2d7-e56dda5d2974"
"name" = "test3"
"properties_json" = "{"account":"test2","constraints":"anothertest2"}"
"type" = "aws"
},
{
"id" = "81de1857-c0c9-4c9e-8fbd-d8a1da64fa3c"
"name" = "test4"
"properties_json" = "{"account":"test3","constraints":"anothertest1"}"
"type" = "az"
},
],
]
Here is working example. I had to fill out the blanks missing from your question, thus you may need to modify it to suit your needs:
locals {
properties = [
[
{
"id" = "b5336bf7-07fb-4026-aa3d-479bd974ca45"
"name" = "test1"
"properties_json" = "{\"account\":\"test0\",\"constraints\":\"anothertest4\"}"
"type" = "vsphere"
},
{
"id" = "67a3380b-8008-4f9c-9c13-2a1a935d5820"
"name" = "test2"
"properties_json" = "{\"account\":\"test1\",\"constraints\":\"anothertest3\"}"
"type" = "gcp"
},
],
[
{
"id" = "eeddd127-cba2-4b34-a2d7-e56dda5d2974"
"name" = "test3"
"properties_json" = "{\"account\":\"test2\",\"constraints\":\"anothertest2\"}"
"type" = "aws"
},
{
"id" = "81de1857-c0c9-4c9e-8fbd-d8a1da64fa3c"
"name" = "test4"
"properties_json" = "{\"account\":\"test3\",\"constraints\":\"anothertest1\"}"
"type" = "az"
},
],
]
}
output "resources_by_name" {
value = [for props in flatten(local.properties):
jsondecode(props.properties_json).account
if props.type == "vsphere"
]
}
Outcome:
resources_by_name = [
"test0",
]
I have a List of Map with nested Map as well as below :-
def list = [
[
"description": "The issue is open and ready for the assignee to start work on it.",
"id": "1",
"name": "Open",
"statusCategory": [
"colorName": "blue-gray",
"id": 2,
"key": "new",
"name": "To Do",
]
],
[
"description": "This issue is being actively worked on at the moment by the assignee.",
"id": "3",
"name": "In Progress",
"statusCategory": [
"colorName": "yellow",
"id": 4,
"key": "indeterminate",
"name": "In Progress",
]
]
]
I have a task to get List of subMap with nested subMap. I'm doing some thing like as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"])}
}
println getSubMap(list)
But its give me output as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"colorName":"blue-gray",
"id":2,
"key":"new",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"colorName":"yellow",
"id":"4",
"key":"indeterminate",
"name":"In Progress"
]
]
]
As you can see I'm unable to get subMap of statusCategory key Map. Actually I want to get further subMap for nested Maps something like as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"id":"2",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"id":"4",
"name":"In Progress"
]
]
]
To achieve this I'm trying as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory":["id","name"]])}
}
def modifiedList = getSubMap(list)
But it throws me Excpetion. And If I'm doing as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"]).statusCategory.subMap(["id","name"])}
}
println getSubMap(list)
It gives only nested subMap as :-
[["id":"2", "name":"To Do"], ["id":"4", "name":"In Progress"]]
could anyone suggest me how to recurselvely find List of subMap with nested subMap if exist?
If your Map nesting is arbitrary, then you might want to consider something like this:
def nestedSubMap
nestedSubMap = { Map map, List keys ->
map.subMap(keys) + map.findAll { k, v -> v instanceof Map }.collectEntries { k, v -> [(k):nestedSubMap(v, keys)] }
}
Given your input and this closure, the following script:
def result = list.collect { nestedSubMap(it, ["id", "name"]) }
println '['
result.each { print it; println ',' }
println ']'
Produces this output:
[
[id:1, name:Open, statusCategory:[id:2, name:To Do]],
[id:3, name:In Progress, statusCategory:[id:4, name:In Progress]],
]
Given the original list, consider this:
def resultList = list.collect {
def fields = ["id", "name"]
def m = it.subMap(fields)
m["statusCategory"] = it["statusCategory"].subMap(fields)
return m
}
which supports these assertions:
assert 1 == resultList[0]["id"] as int
assert "Open" == resultList[0]["name"]
assert 2 == resultList[0]["statusCategory"]["id"] as int
assert "To Do" == resultList[0]["statusCategory"]["name"]
assert 3 == resultList[1]["id"] as int
assert "In Progress" == resultList[1]["name"]
assert 4 == resultList[1]["statusCategory"]["id"] as int
assert "In Progress" == resultList[1]["statusCategory"]["name"]