Terraform List of Maps - terraform

I currently have a single resource defined for
aws_ebs_volume & aws_volume_attachment
I use a count based on a variable to determine how many devices I want to created followed by a count on the attachment:
data_volumes = ["50"]
data_device = ["xvde"]
resource "aws_ebs_volume" "datavolumes" {
count = "${length(var.data_volumes)}"
size = "${var.data_volumes[count.index]}"
tags = "${var.instance_tags}"
encrypted = "true"
availability_zone = "us-east-2b"
kms_key_id = "${var.kms_key}"
}
resource "aws_volume_attachment" "attachvolumes" {
count = "${length(var.data_volumes)}"
device_name = "${var.data_device[count.index]}"
volume_id = "${aws_ebs_volume.datavolumes.*.id[count.index]}"
instance_id = "${aws_instance.general.id}"
}
I'm struggling with finding a way to assign unique tags to each of these volumes that get created, as you can see I'm using a static list of "instance_tags" for each of the volumes but I'd like to have unique tags applied to each of the volumes. I'm trying to avoid having to specify a resource/volume but might be easiest at this point.
Hoping someone can help me understand if its possible and an example of what it looks like.

I think I found an approach that works for what I'm trying to accomplish:
resource "aws_ebs_volume" "datavolumes" {
count = "${length(var.data_volumes)}"
size = "${var.data_volumes[count.index]}"
tags = "${merge(
var.instance_tags,
map(
"DriveLetter", "${var.data_letters[count.index]}",
"DriveLabel", "${var.data_labels[count.index]}"
)
)}"
data_volumes = ["50","50","50"]
data_device = ["xvde","xvdf","xvdg"]
data_letters = ["E:", "F:", "G:"]
data_labels = ["Data", "Logs", "TempDB"]
This gives me unique EBS volumes with Unique tag properties for each volumes and maintains my single resource

Related

How to extract generated attribute from a previously defined resource?

This is not related to aws but to a technique of extracting data from a resource collection; therefore the content is most likely not correct relative to aws provider. I just used some words from that provider to prove the idea.
Given that the aws_instance.web resources are created as a collection by use of a for_each loop like described below:
resource "aws_instance" "web" {
for_each = {for k,v in var.input_var: k => v if v.enabled}
name = each.key
ami = each.value.ami
instance_type = each.value.instance_type
}
resource "aws_db_instance" "db" {
for_each = var.another_map
aws_instance_id = aws_instance.web[index(aws_instance.web[*].name, each.value.name)].id
}
At creation of the first collection of resources, to each element is assigned a unique read-only id by terraform/provider. Given that var.input_var.key is always unique, results that also aws_instance.web.name will always be unique for each element created.
In the second resources block, I also use a for_each loop to cycle through all elements of var.another_map. I want to attribute to aws_instance_id, the generated id from the first resources collection. So I need to first find the element from aws_instance.web where the name of it is equal to each.value.name while creating aws_db_instance.db and than extract the id form it.
I tried several ways to achieve this. The closest one is the one exposed above: aws_instance.web[index(aws_instance.web[*].name, each.value.name)].id.
So there are two questions that arise from this:
What is the type of aws_instance.web (a list of objects, a map of objects, an object which contains a map)?
How would a correct syntax would look like for matching the element and extracting the id from it?
Researching on Matt Schuchard's answer and expanding on it, the output of first resource creation is object(map(object)) where the keys of the generated map are the same keys of the input variable var.input_var.
eg: given the input variable
input_var = {
"website1" = {
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
other_var = "some value"
},
"webst2" = {
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
other_var = "some other value"
}
}
resource "aws_instance" "web" block would produce a aws_instance.web variable with contents like:
aws_instance.web = {
"website1" = {
id = 43262 # (read-only) generated by `aws_instance` resource block
name = "website1"
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
# other entries generated by `aws_instance` resource block
},
"webst2" = {
id = 43263 # (read-only) generated by `aws_instance` resource block
name = "webst2"
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
# other entries generated by `aws_instance` resource block
}
}
So when trying to access a specific element from aws_instance.web in the second resource block (aws_db_instance.db), one can access it by its key, rather than trying to match its name attribute.
Therefore, the line aws_instance_id = aws_instance.web[index(aws_instance.web[*].name, each.value.name)].id should be replaced with aws_instance_id = aws_instance.web[each.value.name].id, if and only if the set of names is a subset of the keys of aws_instance.web (I will represent this like each.value.name[*] ⊆ aws_instance.web.keys[*])

Terraform: Conditional creation of a resource based on a variable in .tfvars

I have resources defined in .tf files that are generic to several applications. I populate many of the fields via a .tfvars file. I need to omit some of the resources entirely based on variables in the .tfvars.
For example if I have a resource like:
resource "cloudflare_record" "record" {
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
But then I declare something like cloudflare = false in my .tfvars file I'd like to be able to do something like this:
if var.cloudflare {
resource "cloudflare_record" "record" {
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
}
I've looked at dynamic blocks but that looks like you can only use those to edit fields and blocks within a resource. I need to be able to ignore an entire resource.
Add a count parameter with a ternary conditional using the variable declared in .tfvars like this:
resource "cloudflare_record" "record" {
count = var.cloudflare ? 1 : 0
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
In this example var.cloudflare is a boolean declared in the .tfvars file. If it is true a count of 1 record will be created. If it is false a count of 0 record will be created.
After the count apply the resource becomes a group, so later in the reference use 0-index of the group:
cloudflare_record.record[0].some_field
Expanding on #Joel Guerra's answer, after you use count to determine whether to deploy the resource or not, you can use the one() function to refer to the resource without an index (i.e. without having to use [0]).
For example, after defining the resource like below
resource "cloudflare_record" "record" {
count = var.cloudflare ? 1 : 0
}
Define a local variable like below
locals {
cloudflare_record_somefield = one(cloudflare_record.record[*].some_field)
}
Now instead of cloudflare_record.record[0].some_field, you can use
local.cloudflare_record_somefield
If the count is 0 (e.g. var.cloudflare is false and the resource wasn't created) then local.cloudflare_record_somefield would return null (instead of returning an error when indexing using [0]).
Reference: https://developer.hashicorp.com/terraform/language/functions/one
An issue i'm seeing this with is if the resource your trying to create is already using a for_each then you can't use both count and for_each in the resource. I'm still trying to find an answer on this will update if I find something better.

Terraform looking local vars that are calculated using mapping when using count to create multiple resources

I’m using mapped variables in order to create local vars based on longer variable names, I'm using them where we would have abbreviations or where the resource wants a sanitized or shortened version of a value used elsewhere.
Eg
variable "env-short" {
description = "create a shortened version of the name of use in resource naming"
type = "map"
default = {
"Proof Of Concept" = "poc"
"User Acceptance Testing" = "uat"
"Production" = "prd"
}
}
variable "org-short" {
description = "create a shortened version of the name of use in resource naming"
type = map(string)
default = {
"My Big Company" = "MBC"
"My Little Company" = "MLC"
}
}
variable "loc-short" {
description = "create a shortened version of the name of use in resource naming"
type = map(string)
default = {
"UK South" = "UKS"
"UK West" = "UKW"
"North Europe" = "NEU"
"West Europe" = "WEU"
}
}
And use corresponding variables for their full length mapping equiverlants.
Now I could use as is within a resource block by something like
Name = “${lower(“${var.loc-short[$var.location]}”)-${lower(“${var.org-short[$var.organisation]}”)-${lower(“${var.env-short[$var.environment]}”)-myresource”
But like all good coders I like to keep things neat and readable by declaring local variables that I can then refer to.
locals {
org-short = "${lower("${var.org-short["${var.organisation}"]}")}"
loc-short = "${lower("${var.loc-short["${var.location}"]}")}"
env-short = "${lower("${var.env-short["${var.environment}"]}")}"
# I also create additional for commonly used configurations of them
name-prefix = "${lower("${var.org-short["${var.organisation}"]}")}-${lower("${var.loc-short["${var.location}"]}")}"
name-prefix-storage = "${lower("${var.org-short["${var.organisation}"]}")}${lower("${var.loc-short["${var.location}"]}")}"
}
This works really great and keeps things neat tidy and readable.
resource "provisioner_example" "test" {
location = var.location
name = “${local.loc-short}-${local.env-short}-my resource”
I would like however to be able to use this format when I start creating multiple resources using the count functionality.
resource "provisioner_example" "test" {
count = length(var.location)
location = var.location[count.index]
name = “${local.loc-short[count.index]}-${local.env-short}-my resource”
Terraform then complains that the index is invalid in the locals lookup, varlocation is tuple with 2 elements,| var.loc-short is map of string with 4 elements. The given key does not identify an element in this collection value: string required.
Now I know I can work around this by getting rid of the locals variables andincluding the variable calculation directly
name =”${lower("${var.loc-short["${var.locations[count.index]}"]}")}-${local.env-short}-my resource"
But to me it then makes the code seem more messy and less structured.
Any ideas on how I can pass the count index value to the map lookup?

When/where does the data aws_availability_zones is initialised or injected?

I am reading the terraform code here https://github.com/linuxacademy/content-terraform/blob/master/course/terraform-aws/networking/main.tf and here are the code I don't quite understand.
data "aws_availability_zones" "available" {}
....
resource "aws_subnet" "tf_public_subnet" {
count = 2
vpc_id = "${aws_vpc.tf_vpc.id}"
cidr_block = "${var.public_cidrs[count.index]}"
map_public_ip_on_launch = true
availability_zone = "${data.aws_availability_zones.available.names[count.index]}"
tags {
Name = "tf_public_${count.index + 1}"
}
}
I don't understand when the data.aws_availability_zones is populated.
data.aws_availability_zones is data source, different from resource (the next code you pasted)
Please go through this url terraform data source to understand how data sources work
If you don't understand how data.aws_availability_zones works, go through this url Data Source: aws_availability_zones
So from its Attributes Reference, we know it will help to get current account's availability zones.
zone_ids - A list of the Availability Zone IDs available to the account.
Its attribute name is alias of zone_ids, their outputs are same.

creation order of subnet with terraform

I need to create 6 subnets with below cidr value but it's order has been changed while creating it with terraform.
private_subnets = {
"10.1.80.0/27" = "x"
"10.1.80.32/27" = "x"
"10.1.80.64/28" = "y"
"10.1.80.80/28" = "y"
"10.1.80.96/27" = "z"
"10.1.80.128/27" = "z"
}
Terraform is creating with 10.1.80.0/27 , 10.1.80.128/27,10.1.80.32/27,10.1.80.64/28,10.1.80.80/28,10.1.80.96/27 order
Module of terraform:
resource "aws_subnet" "private" {
vpc_id = "${var.vpc_id}"
cidr_block = "${element(keys(var.private_subnets), count.index)}"
availability_zone = "${element(var.availability_zones, count.index)}"
count = "${length(var.private_subnets)}"
tags {
Name = "${lookup(var.private_subnets, element(keys(var.private_subnets), count.index))}
}
}
Updated Answer:
Thanks to the discussion in the comments, I revise my answer:
You are assuming an order within a dictionary. This is not intended behaviour. As from your example, one can see that terraform orders the keys alphabetically internally, i.e., you can "think" of your variable as
private_subnets = {
"10.1.80.0/27" = "x"
"10.1.80.128/27" = "z"
"10.1.80.32/27" = "x"
"10.1.80.64/28" = "y"
"10.1.80.80/28" = "y"
"10.1.80.96/27" = "z"
}
You are running into problems, because you are having mismatches with your other variable var.availability_zones where you assume the index to be sorted the same as for var.private_subnets.
Relying on the above ordering (alphabetically), is not a good solution, since it may change with any version of terraform (order of keys is not guaranteed).
Hence, I propose to use a list of maps:
private_subnets = [
{
"cidr" = "10.1.80.0/27"
"name" = "x"
"availability_zone" = 1
},
{
"cidr" = "10.1.80.32/27"
"name" = "x"
"availability_zone" = 2
},
…
]
I encoded the availability zone as index of your var.availability_zones list. However, you could also consider using the availability zone directly.
The adaption of your code is straightforward: Get (element(…)) the list element to get the map and then lookup(…) the desired key.
Old Answer (not applicable here):
Before Terraform creates any resources, it creates a graphstructure to represent all the objects it wants to track (create, update, delete) and the dependencies upon one another.
In your example, 6 different aws_subnet objects are created in the graph which do not depend on each other (there is no variable in one subnet dependent on another subnet).
When Terraform now tries to create the attributes, it does so concurrently in (potentially) multiple threads and creates resources potentially simultaniously, if they do not depend on each other.
This is why you might see very different orders of execution within multiple runs of terraform.
Note that this is a feature, since if you have many resources to be created that have no dependency on each other, they all are created simultaneously saving a lot of time with long-running creation operations.
A solution to your problem is to explicitly model the dependencies you are thinking of. Why should one subnet be created before the other? And if so, how can you make them dependent (e.g. via depends_on parameter)?
Answering this questions should bring you into the right direction to model your code according to your required layout.

Resources