How to merge two level nested maps in terraform? - terraform

I know there is an open feature request for deepmerge but I just wanted to see if there is any work around for my use case. lets consider the following local variables:
locals {
default = {
class = "class1"
options = {
option1 = "1"
option2 = "2"
}
}
configs = {
configA = {
name = "A"
max = 10
min = 5
enabled = true
options = {
option3 = "3"
}
}
configB = {
name = "B"
max = 20
min = 10
enabled = false
}
}
}
so I can merge the configs with default like this:
for key, config in local.configs : key => merge(local.default, config)
and the result will be:
configs = {
configA = {
name = "A"
class = "class1"
max = 10
min = 5
enabled = true
options = {
option3 = "3"
}
}
configB = {
name = "B"
class = "class1"
max = 20
min = 10
enabled = false
options = {
option1 = "1"
option2 = "2"
}
}
}
The problem is the nested map (options property) gets completely replaced by configA since merge cannot handle nested merge. Is there any work around for it in terraform 1.1.3 ?

If you know the structure of the map, you can merge the included elements separately as you wish.
In this case this should work:
merged = {
for key, config in local.configs : key =>
merge(
local.default,
config,
{ options = merge(local.default.options, lookup(config, "options", {})) }
)
}
So first merge the top-level elements, and then handle the options separately.

There are various ways to do this including using this deepmerge provider:
https://registry.terraform.io/modules/cloudposse/config/yaml/0.5.0/submodules/deepmerge
Here is a way that assumes only that /usr/bin/jq exists. I am not saying it is pretty, but it does work and ensures that you get the same semantics as a jq * operator.
locals {
left = {...}
right = {...}
merged = {
for k, v in data.external.merge.result : k => jsondecode(v)
}
}
data "external" "merge" {
program = [
"/usr/bin/jq",
"((.left|fromjson) * (.right|fromjson))|with_entries(.value|=tojson)"
]
query = {
left = jsonencode(local.left)
right = jsonencode(local.right)
}
}

Related

Create a second loop in Terraform using a condition

I'm a true beginner with Terraform, and here is my problem:
I need to create multiple objects using the same resource of this type:
resource "jamf_smartComputerGroup" "test_smart_1" {
name = "Test Smart 1"
criteria {
priority = 0
name = "UDID"
search_type = "is"
search_value = "FAKE-UDID-THAT-ALSO-DOES-NOT-EXIST"
}
criteria {
priority = 1
name = "UDID"
search_type = "is not"
search_value = "FAKE-UDID-THAT-DOES-NOT-EXIST-LIKE-REALLY"
}
}
IMPORTANT: this resource can have zero or more criterias!
I have created the variables.tf and terraform.vartf files as follow:
variables.tf
variable "jamf_smartComputerGroup_list" {
type = list(object({
SMCG_NAME = string
SMCG_CRITERIA = list(object({
SMCG_CRITERIA_PRIORITY = number
SMCG_CRITERIA_NAME = string
SMCG_CRITERIA_TYPE = string
SMCG_CRITERIA_VALUE = string
}))
}))
}
terraform.vartf
jamf_smartComputerGroup_list = [
{
SMCG_NAME = "smcg_1"
SMCG_CRITERIA = [] # THIS OBJECT HAS ZERO CRITERIA
},
{
SMCG_NAME = "smcg_2"
SMCG_CRITERIA = [ # THIS OBJECT HAS ONE CRITERIA
{
SMCG_CRITERIA_PRIORITY = 0
SMCG_CRITERIA_NAME = "crit"
SMCG_CRITERIA_TYPE = "is not"
SMCG_CRITERIA_VALUE = "false"
}
]
},
{
SMCG_NAME = "smcg_3"
SMCG_CRITERIA = [ # THIS OBJECT HAS TWO CRITERIAS
{
SMCG_CRITERIA_PRIORITY = 0
SMCG_CRITERIA_NAME = "crit 1"
SMCG_CRITERIA_TYPE = "contains"
SMCG_CRITERIA_VALUE = "foo"
},
{
SMCG_CRITERIA_PRIORITY = 1
SMCG_CRITERIA_NAME = "crit 2"
SMCG_CRITERIA_TYPE = "exact match"
SMCG_CRITERIA_VALUE = "bar"
}
]
}
]
In the main.tf file I was able to loop through the objects, without criterias, using this:
resource "jamf_smartComputerGroup" "default" {
for_each = { for idx, val in var.jamf_smartComputerGroup_list : idx => val }
name = each.value.SMCG_NAME
}
But and I can't find the appropriate way to determine if one or more criterias are present; and if there is one more criterias, how to loop through them.
A far as I understand, I can't use two for_each verbs at the same time, and I can't use count with for_each.
Any examples will be appreciated :-) !
Regards,
Emmanuel Canault
You have to use dynamic blocks:
resource "jamf_smartComputerGroup" "test_smart_1" {
for_each = { for idx, val in var.jamf_smartComputerGroup_list : idx => val }
name = each.value.SMCG_NAME
dynamic "criteria" {
` for_each = each.value.SMCG_CRITERIA
content {
priority = criteria.SMCG_CRITERIA_PRIORITY
name = criteria.SMCG_CRITERIA_NAME
search_type = criteria.SMCG_CRITERIA_TYPE
search_value = criteria.SMCG_CRITERIA_VALUE
}
}
}
Thanks #Marcin!
It works with small adaptation : criteria.value.SMCG_... instead of criteria.SMCG_...
Regards,
Emmanuel

How can I get a map using flatten in Terraform?

I have this variable:
applications = {
"app-name" = {
more_stuff = "x"
environments = ["dev", "stg"]
}
"workload" = {
random_thing = "y"
environments = ["dev"]
}
}
I want to create this map from it:
application_envs = {
"app-name-dev" = { more_stuff = "x" }
"app-name-stg" = { more_stuff = "x" }
"workload-dev" = { random_thing = "y" }
}
I've tried using flatten but I didn't have any success:
application_envs = flatten([
for application_key, application in var.applications : [
for env in application.environments : [
{"${application_key}-${env}" = workspace}
]
]
])
The problem is that it creates a list of objects:
result = [
{ "app-name-dev" = { ... } },
{ "app-name-stg" = { ... } },
...
]
How can I achieve the desired result?
You are headed in the correct direction, but for this kind of data transformation the algorithm requires two steps. Your first step is completely correct, but now you need the second step:
result = { for app_env in application_envs : keys(app_env)[0] => values(app_env)[0] }
This transforms the list(object) by iteratively mapping the key of each object element to the value of each object element. Testing the output produces:
result = {
app-name-dev = {
more_stuff = "x"
}
app-name-stg = {
more_stuff = "x"
}
}
as desired. Since the namespaces of the variables have been omitted from the question, I have likewise omitted them from the answer, but you may need to re-add them yourself when converting between the question code and the actual code.

How can I split out an 'any' variable in terraform?

I'm trying to get multiple values out of an 'any' type variable. I'm new to terraform and open to recommendations. Specifically for this example, I'd like to know how I can output the 'bucket_name' value in my outputs.
variable "replica_config" {
type = any
default = {
role = "role_name"
rules = [
{
id = "full-s3-replication"
status = true
priority = 10
delete_marker_replication = false
destination = {
bucket = "bucket_name"
storage_class = "STANDARD"
replica_kms_key_id = "key_id"
account_id = "account_id"
replication_time = {
status = "Enabled"
minutes = 15
}
}
}
]
}
}
Current Output:
output "output4" {
value = flatten(var.replica_config["rules"])
}
Since you you have a list for rules, you can use a splat expression as such:
output "output4" {
value = var.replica_config.rules[*].destination.bucket
}
Keep in mind, the output of this expression will also be a list. If you want a single item instead of a list, you can use an index.
For example:
output "output4" {
value = var.replica_config.rules[0].destination.bucket
}

terraform how to describe variable type with changing keys in object

I've got an ever changing list of objects as variable and wanted to know how to properly describe its type
variable "lifecycle_rules" {
type = set(object({
# set(object({
# action = map(string)
# condition = map(string)
# }))
}))
default = [
{
first = [
{
condition = {
age = "1"
}
action = {
type = "Delete"
}
},
{
condition = {
age = "2"
}
action = {
type = "Delete"
}
}
]},
{
second = [
{
condition = {
age = "3"
}
action = {
type = "Delete"
}
},
{
condition = {
age = "4"
}
action = {
type = "Delete"
}
}
]
}
]
}
Here should be line with smth like this string = set(object({...
the first and second are always changing, so key value should be
string but can't really set it - any other thoguhts, how to write
type for the default below ?
You are almost there. I think the correct one is:
type = set(
map(
set(
object({condition = map(string),
action = map(string)})
)
)
)
In the map you don't specify attributes, as they can be different. In the most inner one you have object as condition and action are constant.

Optional list element processing in Terraform

I am trying to add redrive policies to existing queues.
I have managed to define a list like this:
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLetterQueue = ""
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLetterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLetterQueue = "PrimaryQueue3_DL"
}
]
}
I have defined a list of DL queues like this:
variable "sqsq_primary_dl" {
type = "list"
default = [
"PrimaryQueue2_DL",
"PrimaryQueue3_DL"
]
}
In my module I define resources like this:
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLetterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary_dl)}"
name = "${element(var.sqsq_primary_dl, count.index)}-${var.environment}"
}
This works. However, I don't like the duplicated information which is the names of the DL queues.
So the question is, how could I get rid of the second list? How could I iterate in the second resource over the first list instead and only create a DL queue if deadLetterQueue != "" ?
Thanks for your help!
I think you may have encountered a limitation of terraform interpolation. Unless you deconstruct your list of maps to separate maps, the best is probably below.
If you keep your definitions for queues with no dl at the bottom and use a static value for minus maths on the dl resource count, the plan stays the same as before.
As a side note, it's dead letter not dead leater.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLeaterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLeaterQueue = "PrimaryQueue3_DL"
},
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLeaterQueue = ""
}
]
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary) - 1}"
name = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")-var.environment}"
}
My colleague has come up with a solution that seems slightly more flexible than the one provided by #henry-dobson.
We have also refactored it so now it doesn't require the deadLeaterQueue value - we conform to a naming standard now, so the resulting names of the DL queues are different from the ones in the question.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = 0
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
}
]
}
data "empty_data_source" "deadletterq" {
count = "${length(var.sqsq_primary)}"
inputs = {
dl = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount", "") > 0 ? "${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}" : ""}"
}
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount") > 0 ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(compact(data.empty_data_source.deadletterq.*.outputs.dl))}"
name = "${element(compact(data.empty_data_source.deadletterq.*.outputs.dl), count.index)}-${var.environment}"
}

Resources