How to create a set by loop over a nested map - terraform

I have a map that I want to read in locals and generate a new map from. One field in the new map will be a set containing the values from the nested data structure. I can't figure out the syntax to do this.
//I want to generate a set of all zones from the nested zone fields
variable "my_var" {
type = object({
name = string
google_bigtable_clusters = any
})
default = {
app_name = "sdfsdfds"
instances = {
instance01 = [
{
zone = "asia-east1-a"
num_nodes = 1
},
{
zone = "asia-east1-b"
num_nodes = 1
},
{
zone = "asia-east1-c"
num_nodes = 1
},
{
zone = "asia-east2-a"
num_nodes = 1
},
],
instance02 = [
{
zone = "europe-west2-a"
num_nodes = 1
},
{
zone = "europe-west2-b"
num_nodes = 1
},
{
zone = "europe-west2-c"
num_nodes = 1
},
{
zone = "europe-west3-a"
num_nodes = 1
},
]
}
}
}
This throws The key expression produced an invalid result: string required.
// locals
new_map = {
some_field = "arbitrary string"
set_of_zones = {
for item in var.my_var.instances : item => {
for subitem in item : subitem.zone => {
zone = subitem.zone
}
}
}
}
I also tried to get the key name but that didn't work: for item in var.my_var.instances : item.key => {
Edit
I was able to do this but I don't understand why I don't have access to the key name here. I want to use the instance01, instance02, etc key name here: for item in var.my_var.instances : item[0].zone => {.

First, your type for your variable is all messed up. You have:
type = object({
name = string
google_bigtable_clusters = any
})
This means that Terraform will accept a value for the variable only if it has these two fields: name (a string) and google_bigtable_clusters (can be anything).
Your default value has neither of these fields. Instead, it only contains app_name and instances, so that's likely the cause of the first issue.
Regarding why you can't access the key name in your for loop, you need to specify both the key and value:
set_of_zones = {
for key, val in var.my_var.instances :
key => {
for subval in val:
subval.zone => {
zone = subval.zone
}
}
}
This is a really odd thing to want to do though, because you're going to end up with a map that looks like:
set_of_zones = {
instance01 = {
"asia-east1-a" => {
zone = "asia-east1-a"
}
}
}
Which doesn't seem super helpful since there is only one attribute in each map, and that attribute's value is the same as the key for that map.

Related

How can I split out an 'any' variable in terraform?

I'm trying to get multiple values out of an 'any' type variable. I'm new to terraform and open to recommendations. Specifically for this example, I'd like to know how I can output the 'bucket_name' value in my outputs.
variable "replica_config" {
type = any
default = {
role = "role_name"
rules = [
{
id = "full-s3-replication"
status = true
priority = 10
delete_marker_replication = false
destination = {
bucket = "bucket_name"
storage_class = "STANDARD"
replica_kms_key_id = "key_id"
account_id = "account_id"
replication_time = {
status = "Enabled"
minutes = 15
}
}
}
]
}
}
Current Output:
output "output4" {
value = flatten(var.replica_config["rules"])
}
Since you you have a list for rules, you can use a splat expression as such:
output "output4" {
value = var.replica_config.rules[*].destination.bucket
}
Keep in mind, the output of this expression will also be a list. If you want a single item instead of a list, you can use an index.
For example:
output "output4" {
value = var.replica_config.rules[0].destination.bucket
}

Produce single item maps procedurally from nested map in terraform

I have a nested map variable of account name and ID by OU, like:
variable "aws_accounts" {
type = map(map(any))
default = {
first_ou = {
first_account = "111111111"
second_account = "222222222"
}
second_ou = {
third_account = "333333333"
fourth_account = "444444444"
}
}
}
This is great for passing a map of account_name to account_id as a sub-variable to do things by ou and the modules in question are constructed to accept a map input.
I would like to also render a local so that I can also reference single accounts but get a map value for them without having to maintain a separate list of variables, like
local.first_account = {
first_account = "111111111"
}
local.second_account = {
second_account = "222222222"
}
local.third_account = {
third_account = "33333333"
}
etc.
I have tried various techniques but without success:
I cannot work out how to refer to each map in the array iteratively- most documentation seems to be based on lists and when I try to do a for_each I get
The "each" object can be used only in "module" or "resource" blocks, and only when the "for_each" argument is set.
Based on your example, it seems like you want to take your two-level map and turn it into a single-level map where the keys are the account names and the "OU names" are just discarded.
Here's one way to achieve that:
locals {
account_ids = merge(values(var.aws_accounts)...)
}
This first uses values to take the values from the top-level map, producing a list of maps.
It then uses merge to take all of the elements from each of the maps and combine them into a single new map. I used the ... symbol to tell Terraform that it should treat each element of the list as a separate argument to merge, rather than just passing the whole list as a single argument.
After merging these together you could potentially split them apart again, creating a separate map each, using a for expression.
locals {
account_maps = tomap({
for k, id in local.account_ids :
k => { (k) = id }
})
}
Maybe not exactly what you're looking for, but could be helpful:
locals {
accounts = merge(var.aws_accounts["first_ou"], var.aws_accounts["second_ou"])
}
If you need to do this in a more dynamic way:
locals {
accounts = zipmap(
flatten([for item in var.aws_accounts : keys(item)]),
flatten([for item in var.aws_accounts : values(item)])
)
}
Now you can access each account with local.accounts["first_account"] etc.
OK So with help from Martin Atkins and Bryan Heden I have found an answer to this. It isn't exactly pretty but it does work:
variable "aws_accounts" {
type = map(map(any))
default = {
first_ou = {
first_account = "111111111"
second_account = "222222222"
}
second_ou = {
third_account = "333333333"
fourth_account = "444444444"
}
}
}
locals {
# gives single map from nested map
account_ids = merge(values(var.aws_accounts)...)
# gives separate structured map for each key
single_accounts_maps = {
for account, id in local.account_ids :
account => {
account = account
id = id
}
}
# gives map where values = keys plus values
single_accounts_maps_joined = zipmap(
flatten([for item in var.aws_accounts : keys(item)]),
[for item in local.single_accounts_maps :
join(" = ", values(item))]
)
# gives nested map by key = {key = "value"}
single_accounts_maps_keys_values = {
for item in local.single_accounts_maps_joined :
(split(" = ", item)[0]) => {
(split(" = ", item)[0]) = (split(" = ", item)[1])
}
}
}
Output that I wanted:
terraform console
> local.single_accounts_maps_keys_values
{
"first_account" = {
"first_account" = "111111111"
}
"fourth_account" = {
"fourth_account" = "444444444"
}
"second_account" = {
"second_account" = "222222222"
}
"third_account" = {
"third_account" = "333333333"
}
}
After the discussion with Martin Atkins and his subsequent edits below, I am recommending his answer instead as simpler, more legible and more graceful, although the tomap() nesting appears to be unneeded, i.e. do
locals {
account_ids = merge(values(var.aws_accounts)...)
account_maps = {
for k, id in local.account_ids :
k => { (k) = id }
}
}

Terraform - override a single value in a map

I would like to know if it is possible to merge two map of maps without replacing the main map object.
My map object is defined as follows:
variable "apps" {
type = map(object({
is_enabled = bool
cost_center = string
}))
default = {}
}
locals {
default_apps = {
"api-1" = {
is_enabled = false
cost_center = "1234"
},
"api-2" = {
is_enabled = false
cost_center = "1235"
},
}
apps = merge(
local.default_apps,
var.apps
)
}
If define my tfars as follows, to override the value of api-1['s_enabled']
apps = {
"api-1" = {
is_enabled = true
}
}
I get the following error:
Error: Invalid value for input variable
The environment variable TF_VAR_apps does not contain a valid value for
variable "apps": element "api-1": attribute "cost_center" is required.
It works if I define my tfvars like so:
apps = {
"api-1" = {
is_enabled = true
cost_center = "1234"
}
}
My goal is to override a single value of one of the pre defined local variables under default_apps (e.x is_enabled) in tfvars.
Edit: requirements
The error is not about your merge but about your tfars. The following variable is invalid in your case:
apps = {
"api-1" = {
is_enabled = true
}
}
as you explicitly defined it as:
type = map(object({
is_enabled = bool
cost_center = string
}))
Your apps is missing cost_center which is required. If you use object type, everything that you specify in type definition must be provided:
Values that match the object type must contain all of the specified keys, and the value for each key must match its specified type.

Merging module output map

I'm trying out the new for_each function on a module, which itself outputs some values that I need to pass into another resource.
module "vnets" {
source = "../caf-virtual-network"
for_each = var.vnet_list
ARM_ENVIRONMENT = var.ARM_ENVIRONMENT
ARM_LOCATION = var.ARM_LOCATION
ARM_SUBSCRIPTION_ID = var.ARM_SUBSCRIPTION_ID
diagnostics_map = local.diagnostics_map
location = var.ARM_LOCATION
netwatcher = local.netwatcher
networking_object = each.value
tags = var.global_settings.tags
virtual_network_rg = "${module.names.standard["resource-group"]}-${each.value.vnet.resource_group_name}"
depends_on = [
module.resource_groups_networking
]
}
I can grab the output of the module for one or more of those objects by specifying something like this
output "subnets" { value = module.vnets["vnet_shared_services_object"].vnet_subnets } , which in turn looks like this:
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
Here I'm specifying the output of ONE object, but I want to dynamically specify the output of both objects in one hit.
So I want this;
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
...output to look like this:
subnets = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
So i know doing the following will work, but the point i'm trying to make is that i don't know how many vnet modules i'm going to produce, and thus i need to make this dynamic:
output merge{
value = merge({
for key, value in module.vnets["vnet_shared_services_object"].vnet_subnets:
key => value
},
{
for key, value in module.vnets["vnet_transit_object"].vnet_subnets:
key => value
})
}
Using the guide on Terraform to flatten (https://www.terraform.io/docs/configuration/functions/flatten.html) the output object works, but it's not how i wish for it to be:
output stuff {
value = flatten([
for key, value in module.vnets: [
for subnet, id in value.vnet_subnets: {
"${subnet}" = id
}
]
])
}
...which equats to:
stuff = [
{
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
},
{
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
},
...and so on
]
an FYI, this does not help me :(
output {
value = merge(
for key, value in module.vnets:
key => value.vnets_subnets
)
}
Any help on this would be greatly appreciated!
I'm not sure if I correctly understand the input maps, but I tried to replicate the issue creating some mock variables.
For that I created the following variables:
variable "vnets" {
default = {
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
}
}
variable "vnet_subnets" {
default = {
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
}
}
Then the output was defiend as:
output stuff {
value = {for k,v in flatten([
for key, value in merge(var.vnets, var.vnet_subnets):
[for subkey1, subval1 in value: {"${subkey1}" = subval1}]
]): keys(v)[0] => values(v)[0]}
}
which resulted in:
stuff = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
A colleague was able to answer this question with the following code:
locals {
subnet_list = {
for key, value in module.vnets:
key => value.vnet_subnets
}
subnet_map = merge(values(local.subnet_list)...)
}
it is the ... operator which is the key takeaway from this. you can look it up here; https://www.terraform.io/docs/configuration/expressions.html#expanding-function-arguments
... will expand a list of items to function parameters, hence you can call merge to merge a list of map

terraform how to describe variable type with changing keys in object

I've got an ever changing list of objects as variable and wanted to know how to properly describe its type
variable "lifecycle_rules" {
type = set(object({
# set(object({
# action = map(string)
# condition = map(string)
# }))
}))
default = [
{
first = [
{
condition = {
age = "1"
}
action = {
type = "Delete"
}
},
{
condition = {
age = "2"
}
action = {
type = "Delete"
}
}
]},
{
second = [
{
condition = {
age = "3"
}
action = {
type = "Delete"
}
},
{
condition = {
age = "4"
}
action = {
type = "Delete"
}
}
]
}
]
}
Here should be line with smth like this string = set(object({...
the first and second are always changing, so key value should be
string but can't really set it - any other thoguhts, how to write
type for the default below ?
You are almost there. I think the correct one is:
type = set(
map(
set(
object({condition = map(string),
action = map(string)})
)
)
)
In the map you don't specify attributes, as they can be different. In the most inner one you have object as condition and action are constant.

Resources