Looking for a more concise way of doing a nested loop - terraform

Looking for a cleaner/more-readable way to achieve a nested loop in Terraform. I will illustrate with an example.
Let's say we have variable for roles that looks like this:
variable "roles" {
type = "list"
default = [
{
name = "LOADER"
schemas = {
RAW = ["USAGE", "ALL"]
SRC = ["ALL"]
}
},
{
name = "USER"
schemas = {
RAW = ["DELETE", "OBJECT"]
SRC = ["USE"]
}
}
]
}
From this, I want to end up with a List of dictionaries that looks something like:
output = [
{
"privilege" = "USAGE"
"role" = "LOADER"
"schema" = "RAW"
},
{
"privilege" = "ALL"
"role" = "LOADER"
"schema" = "RAW"
},
{
"privilege" = "ALL"
"role" = "LOADER"
"schema" = "SRC"
},
{
"privilege" = "DELETE"
"role" = "USER"
"schema" = "RAW"
},
{
"privilege" = "OBJECT"
"role" = "USER"
"schema" = "RAW"
},
{
"privilege" = "USE"
"role" = "USER"
"schema" = "SRC"
},
]
What I have tried so far (seems to work but I am looking for a more concise/readable way to do it):
locals {
# FlatMapping to a list of dictionaries. Each dict in the form of {schema=<schema>, role=<role>, privilege=<privilege>}
key_val = [for role in var.roles : [for schema, privilege in role["schemas"]: {
role = role["name"]
schema = schema
privilege = privilege
}]]
other_key_val = [for dict in flatten(local.key_val): [for priv in dict["privilege"]: {
role = dict["role"]
schema = dict["schema"]
privilege = priv
}]]
}
output "output" {
value = flatten(local.other_key_val)
}
My main objective is to have readable code that can be understood better by others. Given that I am using loops in Terraform for the first time, I can't judge if my implementation is considered readable.

Maybe this would be a little bit simpler way to achieve the same result:
locals {
roles = [
{
name = "LOADER"
schemas = {
RAW = ["USAGE", "ALL"]
SRC = ["ALL"]
}
},
{
name = "USER"
schemas = {
RAW = ["DELETE", "OBJECT"]
SRC = ["USE"]
}
}
]
out = flatten([
for item in local.roles: [
for schema, privileges in item.schemas: [
for privilege in privileges: {
role = item.name
privilege = privilege
schema = schema
}
]
]
])
}

Related

How can I get a map using flatten in Terraform?

I have this variable:
applications = {
"app-name" = {
more_stuff = "x"
environments = ["dev", "stg"]
}
"workload" = {
random_thing = "y"
environments = ["dev"]
}
}
I want to create this map from it:
application_envs = {
"app-name-dev" = { more_stuff = "x" }
"app-name-stg" = { more_stuff = "x" }
"workload-dev" = { random_thing = "y" }
}
I've tried using flatten but I didn't have any success:
application_envs = flatten([
for application_key, application in var.applications : [
for env in application.environments : [
{"${application_key}-${env}" = workspace}
]
]
])
The problem is that it creates a list of objects:
result = [
{ "app-name-dev" = { ... } },
{ "app-name-stg" = { ... } },
...
]
How can I achieve the desired result?
You are headed in the correct direction, but for this kind of data transformation the algorithm requires two steps. Your first step is completely correct, but now you need the second step:
result = { for app_env in application_envs : keys(app_env)[0] => values(app_env)[0] }
This transforms the list(object) by iteratively mapping the key of each object element to the value of each object element. Testing the output produces:
result = {
app-name-dev = {
more_stuff = "x"
}
app-name-stg = {
more_stuff = "x"
}
}
as desired. Since the namespaces of the variables have been omitted from the question, I have likewise omitted them from the answer, but you may need to re-add them yourself when converting between the question code and the actual code.

How to use ellipsis (…) in nested for loop

I see this error "If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key."
locals {
key_id = {
for x in var.security_rules :
"${x.type}" => x}
}
Is it possible to use ellipsis in a nested for this loop and how can i do it?
The error message means that var.security_rules has multiple items with the same type. For example:
variable "security_rules" {
default = [
{
type = "a"
},
{
type = "b"
},
{
type = "a"
}
]
}
We can see that there are at least 2 items with the same type, which wont be accepted as key in map. What we can do here is to group the items with the same type. This is exactly what ellipsis (...) will accomplish. So:
locals {
key_id = {
for x in var.security_rules : "${x.type}" => x... }
}
The value of key_id in this example will be:
key_id = {
"a" = [
{
"type" = "a"
},
{
"type" = "a"
},
]
"b" = [
{
"type" = "b"
},
]
}

Add to list if not empty

I have defined variable users:
users = [
{
userName = "john.doe"
roles = ["ORG_ADMIN"]
profile_attributes = null
appId = null
},
{
userName = "doe.john"
roles = ["ORG_ADMIN"]
profile_attributes = <<EOT
{
"testParameter":"value"
}
EOT
appId = "test123"
}
]
and now I want to create okta_app_user resource:
resource "okta_app_user" "app_users" {
count = length(var.users)
app_id = var.users[count.index].appId
user_id = okta_user.users[count.index].id
username = "${var.users[count.index].userName}#example.com"
profile = var.users[count.index].profile_attributes
}
but app_id can't be empty in this resource but may be empty in my configuration. Is it possible to skip that user in okta_app_user when var.users[count.index].appId is empty ?
so something similar to what can be achieved:
foreach($users in $user) {
if (!$user.app_id) {
continue;
}
}
Yes, you can filter out users without the appId. For example, by creating helper local variable users_with_appId:
locals {
users_with_appId = [
for user in var.users: user if user.appId != null
]
}
And then:
resource "okta_app_user" "app_users" {
count = length(local.users_with_appId)
app_id = local.users_with_appId[count.index].appId
user_id = okta_user.users[count.index].id
username = "${local.users_with_appId[count.index].userName}#example.com"
profile = local.users_with_appId[count.index].profile_attributes
}
In the above its not clear what okta_user.users[count.index].id is? Thus further adjustments may be needed.

Merging module output map

I'm trying out the new for_each function on a module, which itself outputs some values that I need to pass into another resource.
module "vnets" {
source = "../caf-virtual-network"
for_each = var.vnet_list
ARM_ENVIRONMENT = var.ARM_ENVIRONMENT
ARM_LOCATION = var.ARM_LOCATION
ARM_SUBSCRIPTION_ID = var.ARM_SUBSCRIPTION_ID
diagnostics_map = local.diagnostics_map
location = var.ARM_LOCATION
netwatcher = local.netwatcher
networking_object = each.value
tags = var.global_settings.tags
virtual_network_rg = "${module.names.standard["resource-group"]}-${each.value.vnet.resource_group_name}"
depends_on = [
module.resource_groups_networking
]
}
I can grab the output of the module for one or more of those objects by specifying something like this
output "subnets" { value = module.vnets["vnet_shared_services_object"].vnet_subnets } , which in turn looks like this:
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
Here I'm specifying the output of ONE object, but I want to dynamically specify the output of both objects in one hit.
So I want this;
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
...output to look like this:
subnets = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
So i know doing the following will work, but the point i'm trying to make is that i don't know how many vnet modules i'm going to produce, and thus i need to make this dynamic:
output merge{
value = merge({
for key, value in module.vnets["vnet_shared_services_object"].vnet_subnets:
key => value
},
{
for key, value in module.vnets["vnet_transit_object"].vnet_subnets:
key => value
})
}
Using the guide on Terraform to flatten (https://www.terraform.io/docs/configuration/functions/flatten.html) the output object works, but it's not how i wish for it to be:
output stuff {
value = flatten([
for key, value in module.vnets: [
for subnet, id in value.vnet_subnets: {
"${subnet}" = id
}
]
])
}
...which equats to:
stuff = [
{
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
},
{
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
},
...and so on
]
an FYI, this does not help me :(
output {
value = merge(
for key, value in module.vnets:
key => value.vnets_subnets
)
}
Any help on this would be greatly appreciated!
I'm not sure if I correctly understand the input maps, but I tried to replicate the issue creating some mock variables.
For that I created the following variables:
variable "vnets" {
default = {
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
}
}
variable "vnet_subnets" {
default = {
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
}
}
Then the output was defiend as:
output stuff {
value = {for k,v in flatten([
for key, value in merge(var.vnets, var.vnet_subnets):
[for subkey1, subval1 in value: {"${subkey1}" = subval1}]
]): keys(v)[0] => values(v)[0]}
}
which resulted in:
stuff = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
A colleague was able to answer this question with the following code:
locals {
subnet_list = {
for key, value in module.vnets:
key => value.vnet_subnets
}
subnet_map = merge(values(local.subnet_list)...)
}
it is the ... operator which is the key takeaway from this. you can look it up here; https://www.terraform.io/docs/configuration/expressions.html#expanding-function-arguments
... will expand a list of items to function parameters, hence you can call merge to merge a list of map

Terraform extract account id from aws_organizations_organization.main.accounts

Given an account name, is it possible to extract the account id from
resource "aws_organizations_organization" "main" {
}
So something like:
output "account_id" {
value = "aws_organizations_organization.main.accounts[name == 'account1']"
}
account_id = 012345678901
accounts = [
{
"arn" = "arn:aws:organizations::012345678901:account/o-abc123/012345678901"
"email" = "account1#email.com"
"id" = "012345678901"
"name" = "account1"
},
{
"arn" = "arn:aws:organizations::012345678902:account/o-abc123/012345678902"
"email" = "account2#email.com"
"id" = "012345678902"
"name" = "account2"
},
{
"arn" = "arn:aws:organizations::012345678903:account/o-abc123/012345678903"
"email" = "account3#email.com"
"id" = "320413348752"
"name" = "account3"
}
]
In case anyone else stumbles upon this, I was able to solve with the following:
data "aws_organizations_organization" "main" {}
locals {
account-name = "account1"
account-index = index(data.aws_organizations_organization.main.accounts[*].name, local.account-name)
account-id = data.aws_organizations_organization.main.accounts[local.account-index].id
}
output "account_id" {
value = local.account-id
}
Theoretically, if you get version 2.21.0 or newer, you should be able to use the new aws_organizations_organization data source and filter it based on the account name. For example, though not tested:
data "aws_organizations_organization" "org" {
filter = {
name = "name"
values = ["account1"]
}
}
And then where you need the account id use data.aws_organizations_organization.org.id
You can use null_data_source for creating an email list. and then extract accounts using matchkeys.
data "aws_organizations_organization" "main" {}
data "null_data_source" "main" {
count = length(data.aws_organizations_organization.main.accounts)
inputs = {
emails = data.aws_organizations_organization.main.accounts[count.index]["email"]
}
}
output "accounts" {
value = matchkeys(data.aws_organizations_organization.main.accounts, data.null_data_source.main.*.outputs.emails, list("account1"))
}

Resources