Transform an object (map) into a distinct list with terraform - terraform

With Terraform, is it possible to convert an object something like
locals {
data = {
"project1" = {
user_assigned = ["user1", "user2", "user3"]
}
"project2" = {
user_assigned = ["user2", "user3", "user4"]
}
}
to an output like
user1 = ["project1"]
user2 = ["project1","project2"]
user3 = ["project1","project2"]
user4 = ["project2"]
Note that data is an object with maps of keys(projects) and values(user_assigned)

Here's another way to do it, which I'm sharing just in case it's interesting -- the accepted answer is a fine approach too.
locals {
data = {
"project1" = {
user_assigned = ["user1", "user2", "user3"]
}
"project2" = {
user_assigned = ["user2", "user3", "user4"]
}
}
project_user = flatten([
for proj_name, proj in local.data : [
for username in proj.user_assigned : {
project_name = proj_name,
username = username
}
]
])
}
output "example" {
value = {
for pu in local.project_user :
pu.username => pu.project_name...
}
}
Outputs:
example = {
"user1" = [
"project1",
]
"user2" = [
"project1",
"project2",
]
"user3" = [
"project1",
"project2",
]
"user4" = [
"project2",
]
}
I typically use this sort of approach because a data structure like that intermediate local.project_user value -- which is a list with an element for each project/user pair -- often ends up being useful when declaring resources that represent those pairings.
There wasn't any context in the question about what these projects and users represent or which provider they might related to, so I'm going to use github_team and github_team_membership as an example to illustrate what I mean:
resource "github_team" "example" {
for_each = local.data
name = each.key
}
resource "github_team_membership" "example" {
for_each = {
for pu in local.project_user : "${pu.username}:${pu.project_name}" => pu
}
team_id = github_team.example[each.value.project_name].id
username = each.value.username
}
Lots of providers have resources that represent a relationship between two objects like this, and so having an intermediate data structure that contains an element for each pair is a useful building block for those cases, and then you can derive from that mappings in either direction as I did in the output "example" in my original snippet.

You can't dynamically create fully independent variables. Instead you can create a map in few ways. One way would be with the help of transpose and zipmap:
output "test1" {
value = transpose(zipmap(keys(local.data), values(local.data)[*].user_assigned))
}
Resulting in:
test1 = tomap({
"user1" = tolist([
"project1",
])
"user2" = tolist([
"project1",
"project2",
])
"user3" = tolist([
"project1",
"project2",
])
"user4" = tolist([
"project2",
])
})

Related

I want to get a unique storage size through Terraform output

I created a terraform code using "for_each". And I made the code as below. I thought the entire code was not important here, so I omitted the entire code. And then, The terraform resource code I want to get is as follows.
resource "kubernetes_persistent_volume_claim" "pvc" {
for_each = var.az_fileshare
metadata {
name = each.value.pvc_name # PVC 이름
}
spec {
access_modes = ["ReadWriteMany"]
storage_class_name = each.value.storage_class
resources {
requests = {
storage = each.value.pv_size # PVC 볼륨
}
}
volume_name = kubernetes_persistent_volume.pv[each.key].id
}
}
I want to output the size of the unique "storage" in the above code as terraform output.
output "kubernetes_pvc_size" {
value = [
for pvc_size in kubernetes_persistent_volume_claim.pvc : pvc_size.spec
]
}
However, unwanted values were also printed. What should I do? I ask for your help.
kubernetes_pvc_size = [
tolist([
{
"access_modes" = toset([
"ReadWriteMany",
])
"resources" = tolist([
{
"limits" = tomap({})
"requests" = tomap({
"storage" = "10Gi"
})
},
])
"selector" = tolist([])
"storage_class_name" = "azurefile-csi"
"volume_name" = "TEST-1-pv"
},
]),
tolist([
{
"access_modes" = toset([
"ReadWriteMany",
])
"resources" = tolist([
{
"limits" = tomap({})
"requests" = tomap({
"storage" = "10Gi"
})
},
])
"selector" = tolist([])
"storage_class_name" = "azurefile-csi"
"volume_name" = "TEST-2-pv"
},
]),
tolist([
{
"access_modes" = toset([
"ReadWriteMany",
])
"resources" = tolist([
{
"limits" = tomap({})
"requests" = tomap({
"storage" = "10Gi"
})
},
])
"selector" = tolist([])
"storage_class_name" = "azurefile-csi"
"volume_name" = "TEST-3-pv"
},
]),
]
I only want to print "storage" = "10 Gi".
Thank you for your reply. First of all, your answer didn't work well in my environment. However, I got a hint from your answer and generated the code as below.
output "kubernetes_pvc_size" {
value = {
for k,v in kubernetes_persistent_volume_claim.pvc : k => v.spec[*].resources[*].requests.storage
}
}
This works as follows.
kubernetes_pvc_size = {
"fileshare_1" = tolist([
tolist([
"100Gi",
]),
])
"fileshare_2" = tolist([
tolist([
"100Gi",
]),
])
"fileshare_3" = tolist([
tolist([
"100Gi",
]),
])
}
To be a little over-motivated, I want to remove the "tolist([])" item from this result value and print only the value. Is this possible?
The reason why I'm not satisfied with just printing the contents of a variable is that I want to leave the value of the variable null in the future and receive input from the user. It would be nice if the input value is output as terraform output.

Invert the association of yaml nodes in terraform

Assuming the following yaml input file
admins:
members:
- john#gmail.com
- peter#gmail.com
viewers:
members:
- maria#gmail.com
- john#gmail.com
that is passed to var.memberships via yamldecode, is it possible to produce a list of objects that has the member as key and the groups it belongs to as values, i.e. something like
{
"maria#gmail.com" = {
groups = ["viewers"]
},
"john#gmail.com" = {
groups = ["admins", "viewers"]
},
"peter#gmail.com" = {
groups = ["admins"]
}
}
The general answer for projecting one data structure into another in the Terraform language is for expressions.
In this particular case the Grouping Results mechanism would be a part of the solution, because you want to map potentially many elements from the input to a single element in the output by grouping by a particular key (the email addresses).
I think I'd start by first flattening out this data structure so that it's just a single set of member+group pairs where each element is therefore describing the relationship between one member and one group alone.
locals {
memberships = setunion([
for group_name, group in var.memberships : toset([
for member_email in group.members : {
member_email = member_email
group_name = group_name
}
])
]...)
}
With this declaration and your example input I would expect the intermediate result to be:
memberships = toset([
{ member_email = "john#gmail.com", group_name = "admins" },
{ member_email = "peter#gmail.com", group_name = "admins" },
{ member_email = "maria#gmail.com", group_name = "viewers" },
{ member_email = "john#gmail.com", group_name = "viewers" },
])
We can then project this a second time to transform it into a map from email addresses to collections of groups:
locals {
member_groups = tomap({
for membership in local.memberships :
membership.member_email => membership.group_name...
})
}
member_groups = tomap({
"john#gmail.com" = ["admins", "viewers"]
"maria#gmail.com" = ["viewers"]
"peter#gmail.com" = ["admins"]
})
This now has the same information as the data structure you wanted but not quite the right shape, so one more transformation can achieve exactly the map of objects you described in your question:
locals {
users = tomap({
for email, groups in local.member_groups : email => {
groups = toset(groups)
}
})
}
users = tomap({
"john#gmail.com" = {
groups = toset(["admins", "viewers"])
}
"maria#gmail.com" = {
groups = toset(["viewers"])
}
"peter#gmail.com" = {
groups = toset(["admins"])
}
})

did I do something wrong? or I found a bug?

this is to prove that there might be a bug in setintersection() or keys() in terraform v1.2.1
that it treats local and var differently.
I hope someone can help to confirm, or point out if I did something wrong?
# this file is to prove that there might be a bug in setintersection() or keys() in terraform v1.2.1
# that it treats local and var differently.
# I hope someone can help to confirm, or point out if I did something wrong?
# local.regions is to serve as the setintersection control set of strings
# as the keys(s_v) has non-region-related keys
locals {
regions = ["region-1", "region-2"]
}
# local.local_obj has the exact same data structure as of var.var_obj
locals {
local_obj = {
collection_local = {
subnet_local = {
other_keys = "somthing"
region-1 = "cidr-for-region-1-for-local-obj"
# region-2 = "cidr-for-region-2-for-local-obj"
}
}
}
}
# var.var_obj is a map(map(object())) data structure, outter map is the collection of subnets,
# inner map is the subnets within a collection, object is the subnet config
variable "var_obj" {
type = map(
map (
object ({
other_keys = "whatever"
region-1 = optional(string)
region-2 = optional(string)
})
)
)
default = {
collection_var = {
subnet_var = {
other_keys = "something_else"
region-1 = "cidr-for-region-1-for-var-obj"
# region-2 = "cidr-for-region-2-for-var-obj"
}
}
}
}
# local.transform-local and local.transform-var are two transforming locals to suite subnet resource needs
locals {
transform-local = flatten([
for c_k,c_v in local.local_obj : [
for s_k, s_v in c_v : [
for rg in setintersection(local.regions, keys(s_v)) : {
name = "${rg}-${s_k}"
region = rg
ip_cidr_range = s_v[rg]
}
]
]
])
transform-var = flatten([
for c_k,c_v in var.var_obj : [
for s_k, s_v in c_v : [
for rg in setintersection(local.regions, keys(s_v)) : {
name = "${rg}-${s_k}"
region = rg
ip_cidr_range = s_v[rg]
}
]
]
])
}
# output.contrast is to show the different behavior of setintersection() and keys() on local and var
# I think it is a bug?
# as commenting out the region-2 in local and var has different behavior,
# local behaves corretly, not producing region-2 record at all,
# but var transformer produced a region-2 record without ip_cidr_range, which will break the resource code execution
output "contrast" {
value = {
transform-local = local.transform-local
transform-var = local.transform-var
}
}
if uncommenting the region-2 line, setintersection() and keys() behave exactly the same on local and var:
output:
contrast : {
transform-local : [
{
name : "region-1-subnet_local"
ip_cidr_range : "cidr-for-region-1-for-local-obj"
region : "region-1"
}
{
name : "region-2-subnet_local"
ip_cidr_range : "cidr-for-region-2-for-local-obj"
region : "region-2"
}
]
transform-var : [
{
name : "region-1-subnet_var"
ip_cidr_range : "cidr-for-region-1-for-var-obj"
region : "region-1"
}
{
name : "region-2-subnet_var"
ip_cidr_range : "cidr-for-region-2-for-var-obj"
region : "region-2"
}
]
}
but if I comment out the region-2 line (as shown in code above), transform-var output is not desirable as it produces something that will cause malfunctioning
output:
contrast : {
transform-local : [
{
name : "region-1-subnet_local"
ip_cidr_range : "cidr-for-region-1-for-local-obj"
region : "region-1"
}
]
transform-var : [
{
name : "region-1-subnet_var"
ip_cidr_range : "cidr-for-region-1-for-var-obj"
region : "region-1"
}
{
name : "region-2-subnet_var"
region : "region-2"
}
]
}

Terraform access map

I am trying to access all groups and create groups in the below terraform code. But I am facing error This object does not have an attribute named "groups". Is there any logic I am missing here in the resource "og" "example"
for_each=toset(flatten(local.instances[*].groups)). Thanks
locals {
instances = {
test1 = {
baseUrl = "url1"
subDomain = "sd1"
groups = [
"app1",
"app2",
],
}
test2 = {
baseUrl = "url2"
subDomain = "sd2"
groups = [
"t1",
"t2",
],
}
}
}
resource "og" "example" {
for_each = toset(flatten(local.instances[*].groups))
name = each.value
description = "${each.value}-access"
}
Your local variable is a map, not a list. So it should be:
for_each = toset(flatten(values(local.instances)[*].groups))

terraform flatten loop for 3 times

Has any one tried to get flatten for 3 loops working? I keep getting an error when i try the 3rd one:
I am trying to loop through a list within a data resource - data.instances.sg.ids
Variable example:
alarms = [
{
instances = data.instances.ids ## a list of instance ids
config = [
metric_name = "disk_free"
threshold = "GreaterThan"
]
}
]
locals {
configs = flatten([
for config_key, config_list in var.alarms : [
for instance in config_list.instances : {
for config in config_list.configs : {
instance_id = instance
metric_name = config.name
threshold = config.threshold
}
}
]
])
}
how can i properly loop through and flatten this list with data instances list.
Thanks
Not sure what exactly you want to construct, but I think it should be:
locals {
configs = flatten([
for config_key, config_list in var.alarms : [
for instance in config_list.instances : [
for config in config_list.configs :
{
instance_id = instance
metric_name = config.name
threshold = config.threshold
}
]
]
])
}

Resources