Optional list element processing in Terraform - terraform

Question

I am trying to add redrive policies to existing queues.
I have managed to define a list like this:
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLetterQueue = ""
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLetterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLetterQueue = "PrimaryQueue3_DL"
}
]
}
I have defined a list of DL queues like this:
variable "sqsq_primary_dl" {
type = "list"
default = [
"PrimaryQueue2_DL",
"PrimaryQueue3_DL"
]
}
In my module I define resources like this:
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLetterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary_dl)}"
name = "${element(var.sqsq_primary_dl, count.index)}-${var.environment}"
}
This works. However, I don't like the duplicated information which is the names of the DL queues.
So the question is, how could I get rid of the second list? How could I iterate in the second resource over the first list instead and only create a DL queue if deadLetterQueue != "" ?
Thanks for your help!

Answer 1

I think you may have encountered a limitation of terraform interpolation. Unless you deconstruct your list of maps to separate maps, the best is probably below.
If you keep your definitions for queues with no dl at the bottom and use a static value for minus maths on the dl resource count, the plan stays the same as before.
As a side note, it's dead letter not dead leater.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue2"
maxReceiveCount = 5
deadLeaterQueue = "PrimaryQueue2_DL"
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
deadLeaterQueue = "PrimaryQueue3_DL"
},
{
name = "PrimaryQueue1"
maxReceiveCount = -1
deadLeaterQueue = ""
}
]
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue") != "" ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(var.sqsq_primary) - 1}"
name = "${lookup(var.sqsq_primary[count.index], "deadLeaterQueue")-var.environment}"
}

Answer 2

My colleague has come up with a solution that seems slightly more flexible than the one provided by #henry-dobson.
We have also refactored it so now it doesn't require the deadLeaterQueue value - we conform to a naming standard now, so the resulting names of the DL queues are different from the ones in the question.
variable "sqsq_primary" {
type = "list"
default = [
{
name = "PrimaryQueue1"
maxReceiveCount = 0
},
{
name = "PrimaryQueue2"
maxReceiveCount = 5
},
{
name = "PrimaryQueue3"
maxReceiveCount = 20
}
]
}
data "empty_data_source" "deadletterq" {
count = "${length(var.sqsq_primary)}"
inputs = {
dl = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount", "") > 0 ? "${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}" : ""}"
}
}
resource "aws_sqs_queue" "q" {
count = "${length(var.sqsq_primary)}"
name = "${lookup(var.sqsq_primary[count.index], "name")}-${var.environment}"
## Conditionally Sets A Redrive Policy ##
redrive_policy = "${lookup(var.sqsq_primary[count.index], "maxReceiveCount") > 0 ? "{\"deadLetterTargetArn\":\"arn:aws:sqs:${var.region}:${var.acc_number}:${replace(lookup(var.sqsq_primary[count.index], "name"),"Queue","DeadLetterQueue")}-${var.environment}\",\"maxReceiveCount\":${lookup(var.sqsq_primary[count.index], "maxReceiveCount")}}" : ""}"
depends_on = ["aws_sqs_queue.qdl"]
}
resource "aws_sqs_queue" "qdl" {
count = "${length(compact(data.empty_data_source.deadletterq.*.outputs.dl))}"
name = "${element(compact(data.empty_data_source.deadletterq.*.outputs.dl), count.index)}-${var.environment}"
}