Add to list if not empty - terraform

I have defined variable users:
users = [
{
userName = "john.doe"
roles = ["ORG_ADMIN"]
profile_attributes = null
appId = null
},
{
userName = "doe.john"
roles = ["ORG_ADMIN"]
profile_attributes = <<EOT
{
"testParameter":"value"
}
EOT
appId = "test123"
}
]
and now I want to create okta_app_user resource:
resource "okta_app_user" "app_users" {
count = length(var.users)
app_id = var.users[count.index].appId
user_id = okta_user.users[count.index].id
username = "${var.users[count.index].userName}#example.com"
profile = var.users[count.index].profile_attributes
}
but app_id can't be empty in this resource but may be empty in my configuration. Is it possible to skip that user in okta_app_user when var.users[count.index].appId is empty ?
so something similar to what can be achieved:
foreach($users in $user) {
if (!$user.app_id) {
continue;
}
}

Yes, you can filter out users without the appId. For example, by creating helper local variable users_with_appId:
locals {
users_with_appId = [
for user in var.users: user if user.appId != null
]
}
And then:
resource "okta_app_user" "app_users" {
count = length(local.users_with_appId)
app_id = local.users_with_appId[count.index].appId
user_id = okta_user.users[count.index].id
username = "${local.users_with_appId[count.index].userName}#example.com"
profile = local.users_with_appId[count.index].profile_attributes
}
In the above its not clear what okta_user.users[count.index].id is? Thus further adjustments may be needed.

Related

Create aws_ses_receipt_rule in predefined order with for_each loop

I am trying to define SES rule sets with an order defined by a collection of rules in a variable.
I have tried the solution in https://github.com/hashicorp/terraform-provider-aws/issues/24067 to use the after property of the resource, and It does create the first rule, and fails when creating the second, and all subsequent rules, because the first rule does not exist yet (the one with after=null). I guess it needs some time to finalize. depends_on does not work with dynamic dependencies, as far as i know, so this will not make it either.
If I re-run the apply, then the second rule is created, but all the other rules fail.
my recipients_by_position map is indexed by 0-padded position (i.e. "01", "02", etc):
This is my code
locals {
recipients = {
"mail1-recipient1" = {
short_name = "mail1"
domain = "mail1.domain.com"
recipient = "recipient1"
position = 1
target_bucket = "bucket1"
}
"mail1-recipient2" = {
short_name = "mail1"
domain = "mail1.domain.com"
recipient = "recipient2"
position = 2
target_bucket = "bucket1"
}
"mail2-recipient1" = {
short_name = "mail2"
domain = "mail2.domain.com"
recipient = "recipient1"
position = 3
target_bucket = "bucket2"
}
}
spec_by_domain = {
"mail1.domain.com" = {
irrelevant ={}
}
"mail2.domain.com" = {
irrelevant ={}
}
}
recipients_by_position = {for r in local.recipients: "${format("%02s",r.position)}" => r}
}
resource "aws_ses_domain_identity" "domains" {
for_each = local.spec_by_domain
domain = each.key
}
resource "aws_ses_receipt_rule_set" "main" {
rule_set_name = "new-rules"
}
# store it in S3
resource "aws_ses_receipt_rule" "store" {
for_each = local.recipients_by_position
after = each.value.position == 1 ? null : "${format("%02s",each.value.position - 1)}"
# name = "${each.value.short_name}-store_on_s3-${each.value.recipient}"
name = each.key
rule_set_name = aws_ses_receipt_rule_set.main.rule_set_name
recipients = ["${each.value.recipient}#${each.value.domain}"]
enabled = true
scan_enabled = true
s3_action {
bucket_name = aws_s3_bucket.mailboxes[each.value.domain].bucket
object_key_prefix = each.value.recipient
position = 1
}
}
apply fails with a bunch of
Error: Error creating SES rule: RuleDoesNotExist: Rule does not exist: xx
with xx from 01 to whatever number of rules were defined

Looking for a more concise way of doing a nested loop

Looking for a cleaner/more-readable way to achieve a nested loop in Terraform. I will illustrate with an example.
Let's say we have variable for roles that looks like this:
variable "roles" {
type = "list"
default = [
{
name = "LOADER"
schemas = {
RAW = ["USAGE", "ALL"]
SRC = ["ALL"]
}
},
{
name = "USER"
schemas = {
RAW = ["DELETE", "OBJECT"]
SRC = ["USE"]
}
}
]
}
From this, I want to end up with a List of dictionaries that looks something like:
output = [
{
"privilege" = "USAGE"
"role" = "LOADER"
"schema" = "RAW"
},
{
"privilege" = "ALL"
"role" = "LOADER"
"schema" = "RAW"
},
{
"privilege" = "ALL"
"role" = "LOADER"
"schema" = "SRC"
},
{
"privilege" = "DELETE"
"role" = "USER"
"schema" = "RAW"
},
{
"privilege" = "OBJECT"
"role" = "USER"
"schema" = "RAW"
},
{
"privilege" = "USE"
"role" = "USER"
"schema" = "SRC"
},
]
What I have tried so far (seems to work but I am looking for a more concise/readable way to do it):
locals {
# FlatMapping to a list of dictionaries. Each dict in the form of {schema=<schema>, role=<role>, privilege=<privilege>}
key_val = [for role in var.roles : [for schema, privilege in role["schemas"]: {
role = role["name"]
schema = schema
privilege = privilege
}]]
other_key_val = [for dict in flatten(local.key_val): [for priv in dict["privilege"]: {
role = dict["role"]
schema = dict["schema"]
privilege = priv
}]]
}
output "output" {
value = flatten(local.other_key_val)
}
My main objective is to have readable code that can be understood better by others. Given that I am using loops in Terraform for the first time, I can't judge if my implementation is considered readable.
Maybe this would be a little bit simpler way to achieve the same result:
locals {
roles = [
{
name = "LOADER"
schemas = {
RAW = ["USAGE", "ALL"]
SRC = ["ALL"]
}
},
{
name = "USER"
schemas = {
RAW = ["DELETE", "OBJECT"]
SRC = ["USE"]
}
}
]
out = flatten([
for item in local.roles: [
for schema, privileges in item.schemas: [
for privilege in privileges: {
role = item.name
privilege = privilege
schema = schema
}
]
]
])
}

Merging module output map

I'm trying out the new for_each function on a module, which itself outputs some values that I need to pass into another resource.
module "vnets" {
source = "../caf-virtual-network"
for_each = var.vnet_list
ARM_ENVIRONMENT = var.ARM_ENVIRONMENT
ARM_LOCATION = var.ARM_LOCATION
ARM_SUBSCRIPTION_ID = var.ARM_SUBSCRIPTION_ID
diagnostics_map = local.diagnostics_map
location = var.ARM_LOCATION
netwatcher = local.netwatcher
networking_object = each.value
tags = var.global_settings.tags
virtual_network_rg = "${module.names.standard["resource-group"]}-${each.value.vnet.resource_group_name}"
depends_on = [
module.resource_groups_networking
]
}
I can grab the output of the module for one or more of those objects by specifying something like this
output "subnets" { value = module.vnets["vnet_shared_services_object"].vnet_subnets } , which in turn looks like this:
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
Here I'm specifying the output of ONE object, but I want to dynamically specify the output of both objects in one hit.
So I want this;
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
...output to look like this:
subnets = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
So i know doing the following will work, but the point i'm trying to make is that i don't know how many vnet modules i'm going to produce, and thus i need to make this dynamic:
output merge{
value = merge({
for key, value in module.vnets["vnet_shared_services_object"].vnet_subnets:
key => value
},
{
for key, value in module.vnets["vnet_transit_object"].vnet_subnets:
key => value
})
}
Using the guide on Terraform to flatten (https://www.terraform.io/docs/configuration/functions/flatten.html) the output object works, but it's not how i wish for it to be:
output stuff {
value = flatten([
for key, value in module.vnets: [
for subnet, id in value.vnet_subnets: {
"${subnet}" = id
}
]
])
}
...which equats to:
stuff = [
{
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
},
{
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
},
...and so on
]
an FYI, this does not help me :(
output {
value = merge(
for key, value in module.vnets:
key => value.vnets_subnets
)
}
Any help on this would be greatly appreciated!
I'm not sure if I correctly understand the input maps, but I tried to replicate the issue creating some mock variables.
For that I created the following variables:
variable "vnets" {
default = {
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
}
}
variable "vnet_subnets" {
default = {
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
}
}
Then the output was defiend as:
output stuff {
value = {for k,v in flatten([
for key, value in merge(var.vnets, var.vnet_subnets):
[for subkey1, subval1 in value: {"${subkey1}" = subval1}]
]): keys(v)[0] => values(v)[0]}
}
which resulted in:
stuff = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
A colleague was able to answer this question with the following code:
locals {
subnet_list = {
for key, value in module.vnets:
key => value.vnet_subnets
}
subnet_map = merge(values(local.subnet_list)...)
}
it is the ... operator which is the key takeaway from this. you can look it up here; https://www.terraform.io/docs/configuration/expressions.html#expanding-function-arguments
... will expand a list of items to function parameters, hence you can call merge to merge a list of map

Terraform extract account id from aws_organizations_organization.main.accounts

Given an account name, is it possible to extract the account id from
resource "aws_organizations_organization" "main" {
}
So something like:
output "account_id" {
value = "aws_organizations_organization.main.accounts[name == 'account1']"
}
account_id = 012345678901
accounts = [
{
"arn" = "arn:aws:organizations::012345678901:account/o-abc123/012345678901"
"email" = "account1#email.com"
"id" = "012345678901"
"name" = "account1"
},
{
"arn" = "arn:aws:organizations::012345678902:account/o-abc123/012345678902"
"email" = "account2#email.com"
"id" = "012345678902"
"name" = "account2"
},
{
"arn" = "arn:aws:organizations::012345678903:account/o-abc123/012345678903"
"email" = "account3#email.com"
"id" = "320413348752"
"name" = "account3"
}
]
In case anyone else stumbles upon this, I was able to solve with the following:
data "aws_organizations_organization" "main" {}
locals {
account-name = "account1"
account-index = index(data.aws_organizations_organization.main.accounts[*].name, local.account-name)
account-id = data.aws_organizations_organization.main.accounts[local.account-index].id
}
output "account_id" {
value = local.account-id
}
Theoretically, if you get version 2.21.0 or newer, you should be able to use the new aws_organizations_organization data source and filter it based on the account name. For example, though not tested:
data "aws_organizations_organization" "org" {
filter = {
name = "name"
values = ["account1"]
}
}
And then where you need the account id use data.aws_organizations_organization.org.id
You can use null_data_source for creating an email list. and then extract accounts using matchkeys.
data "aws_organizations_organization" "main" {}
data "null_data_source" "main" {
count = length(data.aws_organizations_organization.main.accounts)
inputs = {
emails = data.aws_organizations_organization.main.accounts[count.index]["email"]
}
}
output "accounts" {
value = matchkeys(data.aws_organizations_organization.main.accounts, data.null_data_source.main.*.outputs.emails, list("account1"))
}

Optional list element processing in Terraform

I am trying to add redrive policies to existing queues.
I have managed to define a list like this:
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLetterQueue = ""
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLetterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLetterQueue = "PrimaryQueue3_DL"
}
]
}
I have defined a list of DL queues like this:
variable "sqsq_primary_dl" {
type = "list"
default = [
"PrimaryQueue2_DL",
"PrimaryQueue3_DL"
]
}
In my module I define resources like this:
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLetterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary_dl)}"
name = "${element(var.sqsq_primary_dl, count.index)}-${var.environment}"
}
This works. However, I don't like the duplicated information which is the names of the DL queues.
So the question is, how could I get rid of the second list? How could I iterate in the second resource over the first list instead and only create a DL queue if deadLetterQueue != "" ?
Thanks for your help!
I think you may have encountered a limitation of terraform interpolation. Unless you deconstruct your list of maps to separate maps, the best is probably below.
If you keep your definitions for queues with no dl at the bottom and use a static value for minus maths on the dl resource count, the plan stays the same as before.
As a side note, it's dead letter not dead leater.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLeaterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLeaterQueue = "PrimaryQueue3_DL"
},
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLeaterQueue = ""
}
]
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary) - 1}"
name = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")-var.environment}"
}
My colleague has come up with a solution that seems slightly more flexible than the one provided by #henry-dobson.
We have also refactored it so now it doesn't require the deadLeaterQueue value - we conform to a naming standard now, so the resulting names of the DL queues are different from the ones in the question.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = 0
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
}
]
}
data "empty_data_source" "deadletterq" {
count = "${length(var.sqsq_primary)}"
inputs = {
dl = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount", "") > 0 ? "${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}" : ""}"
}
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount") > 0 ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(compact(data.empty_data_source.deadletterq.*.outputs.dl))}"
name = "${element(compact(data.empty_data_source.deadletterq.*.outputs.dl), count.index)}-${var.environment}"
}

Resources