Terraform iterate over list - terraform

I would like to replace the 3 indepedent variables (dev_id, prod_id, stage_id), for a single list containing all the three variables, and iterate over them, applying them to the policy.
Is this something terraform can do?
data "aws_iam_policy_document" "iam_policy_document_dynamodb" {
statement {
effect = "Allow"
resources = ["arn:aws:dynamodb:${var.region}:${var.account_id}:table:${var.dynamodb_table_name}"]
actions = [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem",
]
principals {
type = "AWS"
identifiers = [
"arn:aws:iam::${var.dev_id}:root",
"arn:aws:iam::${var.prod_id}:root",
"arn:aws:iam::${var.stage_id}:root"
]
}
}
}
I looked into cycles and interpolation, but It seems that 99% of the time the interpolation is done with "count" which only works for the creation of multiple resources (I hope I am not saying a big lie).
For example, I used
principals {
count = "${length(var.list)}"
identifiers = ["arn:aws:iam::${var.list[count.index]}"]
}
but that was unsuccessful.
Is there some way of achieving the final goal of replacing those 3 variables by a single list (or map) and iterate over them?

Given you have the list of account ids, have you tried this?
var "accounts" {
default = ["123", "456", "789"]
type = "list"
}
locals {
accounts_arn = "${formatlist("arn:aws:iam::%s", var.accounts)}"
}
Then in your policy document:
principals {
type = "AWS"
identifiers = ["${local.accounts_arn}"]
}
I haven't actually tried it, but can't think of a reason it wouldn't work.

Related

Merging complex types

I'm struggling with merging maps that have duplicate keys, since the built-in merge function will only keep the latest argument that has matching keys or attributes.
My maps are of the following shape:
{mykey = ["item1","item2","item3"]}
{mykey = ["item4","item5","item6"]}
The merge function simply returns {mykey = ["item4","item5","item6"]} (or whatever is the last argument). I'd like to compose a map like
{mykey = ["item1","item2","item3","item4","item5","item6"]}
I don't believe there's a function I can use to achieve this. However, I suspect that a for loop is the right approach, yet my knowledge is failing me.
Hi #MattSchuchard, thanks for your help. The parent map structure (if I am following you correctly), is a local variable. I am using keys and values from this local in order to retrieve object_ids from the azuread_users and azuread_groups data sources. My locals looks like this:
locals {
roles = {
role1 = {
users = ["user1#domain.com","user2#domain.com"]
groups = ["myGroup1","myGroup2"]
}
}
I am referencing this local in the data sources as follows:
data azuread_users.lookup {
for_each = local.roles
user_principal_names = each.values.users
}
data azuread_groups.lookup {
for_each = local.roles
display_names = each.values.groups
}
These data sources export object_ids as an attribute (which I will then use when adding members to groups in a resource block). The exported attributes are in the shape detailed in my original post.
I would like to compose a combined list of all object_ids exported from both data.azuread_users.lookup and data.azuread_groups.lookup which I can then provide as an argument when creating a group:
resource "azuread_group" "my_group" {
members = $myCombinedListOfObjectIDs
}
Depending on whether you want a list/map you can use this:
locals {
input = { mykey = ["item1", "item2", "item3"] }
input2 = { mykey = ["item4", "item5", "item6"] }
merged = concat(local.input.mykey, local.input2.mykey)
merged_mykey = {
mykeys = concat(local.input.mykey, local.input2.mykey)
}
}
The output is
> local.merged_mykey
{
"mykeys" = [
"item1",
"item2",
"item3",
"item4",
"item5",
"item6",
]
}
> local.merged
[
"item1",
"item2",
"item3",
"item4",
"item5",
"item6",
]
If you're worried about duplicates you can wrap it in a distinct call. https://www.terraform.io/language/functions/distinct

Terraform data structures

maybe a really stupid question here and I'm guessing this is actually nothing like as complicated as I'm finding it. However, after spending hours and hours reading and trying, while I've learnt a lot about Terraform, I'm really no closer to a solution which I can understand or reuse.
I'm trying to do something that would be pretty simple in bash or python but seems to require some trickery in Terraform. I have two variables that I want to "combine" so that I have a valid block of data usable in for_each looping for resource creation. It's quite likely that I have failed to do this properly from the beginning in terms of variable declaration. So please forgive my beginner invompetence.
Just to note also, this is mostly for my own learning so not necessarily a practical example bur rather a use case that illustrates what I want to understand better. In this context, my use case is to provision workspaces in Terraform cloud and to insert the same set of private variables in to each workspace created. aka create workspaces and push in azure service principle authentication variables.
So in variables.tf I have
variable "env_names" {
type = set(string)
default = ["dev", "uat", "prod"]
}
variable "auth_vars" {
default = [
{ key = "subscription_id"
value = "XXXXXXXXXXXXXXXXXXXXXXXX"
},
{
key = "client_id"
value = "XXXXXXXXXXXXXXXXXXXX"
},
{
key = "client_secret"
value = "XXXXXXXXXXXXXXXXXXX"
},
{
key = "tenant_id"
value = "XXXXXXXXXXXXXXXXXXX"
}
]
}
Then I want to use these variables to create workspaces with the set of variables applied to each workspace. For the workspace creation, I have no problems creating them using.
resource "tfe_workspace" "cloud_workspace" {
for_each = var.env_names
name = "MyWorkspace-${each.key}"
organization = "MyOrg"
execution_mode = "remote"
auto_apply = "false"
allow_destroy_plan = "true"
global_remote_state = "false"
}
It's the creation of the variables for the workspaces which is a major headache for me. I have tried all sorts of manipulations using locals, flattening the data, doing a setproduct etc. I think it's here that I really don't know how to approach this problem. I have tried to do things like:
locals {
auth_map = flatten([
for w in var.env_names : [
for v in var.auth_vars : {
workspace = w
key = v.key
value = v.value
}
]
])
}
This does seem to create a structure that makes sense as I have a list of all the tuple values required. aka
{
key = "subscription_id"
value = "XXXXXX"
workspace = "dev"
},
{
key = "client_id"
value = "XXXXXXXXXXXX"
workspace = "dev"
},
ETC.....
I just have no idea how to get this list of tuples in to a useful form so that I can create all the variables for the workspaces. I'd want to use the data to create multiple variable blocks of the type below. I'm not really sure how to get there though.
resource "tfe_variable" "azure-credentials" {
key = ""
value = ""
category = "terraform"
workspace_id = ""
}
I'm aware that I could have simply done a few blocks of code and solved this use case long ago, but it's more for my understanding of how Terraform works and how to deal with these more complex situations. Any advice appreciated as I'm really at the start of learning Terraform properly and am well out of my depth here.
Thank you so much for taking the time
If I understand correctly, it should be:
resource "tfe_variable" "azure-credentials" {
for_each = {for idx, value in local.auth_map: idx => value}
key = each.value.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test[each.value.workspace].id
}
In the above, you convert your list of maps local.auth_map into a map, as for_each will not work with your list.

terraform - how to use variables inside attributes

I am not sure if this is the right approach to do this but I want to use a variable as an attribute.
For example, I have a variable that changes based on user input: os_name = ubuntu.
I want to use this variable name like the following,
resource "aws_instance" "workerNode" {
..................
ami = data.aws_ami.${var.os_name}.image_id
..................
}
Following is an example of the data block,
data "aws_ami" "suse" {
count = "${var.os_name == "suse" ? 1 : 0}"
owners = ["amazon"]
most_recent = true
filter {
name = "name"
values = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
Which result the following,
"architecture" = "x86_64"
"hypervisor" = "xen"
"id" = "ami-0d3905203a039e3b0"
"image_id" = "ami-0d3905203a039e3b0"
But terraform is not allowing me to do this. Is there any way I can do this or I have to change the workflow?
In situations where it's not appropriate to gather all of your instances under a single resource using for_each (which would implicitly make that resource appear as a map of objects), you can get a similar result explicitly by writing a local value expression to construct an equivalent map:
locals {
amis = {
suse = data.aws_ami.suse
ubuntu = data.aws_ami.ubuntu
}
}
Then you can refer to local.amis["ubuntu"] or local.amis["suse"] (possibly replacing the element key with a variable, if you need to.
With that said, it does seem like there is a possible different approach for your case which would get there with only one data block:
locals {
os_ami_queries = {
suse = {
owners = ["amazon"]
filters = {
name = ["suse-sles-${var.os_version}-sp*-v????????-hvm-ssd-x86_64"]
}
}
ubuntu = {
owners = ["amazon"]
filters = {
name = ["ubuntu-${var.os_version}-something-something"]
}
}
}
ami_query = local.os_ami_queries[var.os_name]
}
data "aws_ami" "selected" {
owners = local.ami_query.owners
dynamic "filter" {
for_each = local.ami_query.filters
content {
name = filter.key
values = filter.value
}
}
}
This different permutation does the OS selection before the data "aws_ami" lookup, so it can use the settings associated with whichever OS was selected by the caller. The AMI id would then be in data.aws_ami.selected.id.
With that said, this approach has the disadvantage of being quite indirect and using a dynamic block, so I'd weigh that against the readability of the alternatives to pick the one which seems subjectively easiest to follow for someone who isn't familiar with this configuration. There isn't a single answer to that because to some extent it's a matter of taste, and so if you are working in a team setting this could be something to discuss with colleagues to see which approach best suits tradeoffs like how often you expect to be adding and removing supported operating systems vs. changing the details of how you use the result.
You can make it work by specifying your AMI's with a for_each and thus getting a map which you can access by key.
My data.aws_ami.myamis looks like this:
data "aws_ami" "myamis" {
for_each = toset(["suse", "ubuntu"])
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["${each.value}*"]
}
}
For test purposes I define a variable foo like this:
variable "foo" {
type = string
default = "suse"
}
Now I can access the AMI like this:
$ tf console
> data.aws_ami.myamis[var.foo].image_id
"ami-0ea50c090ba6e85c5"
You can adapt this to suit your needs for os_name and os_version.
I have solved the issue just by using conditional expression.
I am not sure if it is a standard way of doing things but it works for me.
I have tried to emulate if/elif/else with nested conditional expression.
output "name" {
value = "${ var.os_name == "ubuntu" ? data.aws_ami.ubuntu[0].image_id : (var.os_name == "redhat" ? data.aws_ami.redhat[0].image_id : (var.os_name == "centos" ? data.aws_ami.suse[0].image_id : data.aws_ami.suse[0].image_id ))}"
}

Iterate Through Map of Maps in Terraform 0.12

I need to build a list of templatefile's like this:
templatefile("${path.module}/assets/files_eth0.nmconnection.yaml", {
interface-name = "eth0",
addresses = element(values(var.virtual_machines), count.index),
gateway = element(var.gateway, count.index % length(var.gateway)),
dns = join(";", var.dns_servers),
dns-search = var.domain,
}),
templatefile("${path.module}/assets/files_etc_hostname.yaml", {
hostname = element(keys(var.virtual_machines), count.index),
}),
by iterating over a map of maps like the following:
variable templatefiles {
default = {
"files_eth0.nmconnection.yaml" = {
"interface-name" = "eth0",
"addresses" = "element(values(var.virtual_machines), count.index)",
"gateway" = "element(var.gateway, count.index % length(var.gateway))",
"dns" = "join(";", var.dns_servers)",
"dns-search" = "var.domain",
},
"files_etc_hostname.yaml" = {
"hostname" = "host1"
}
}
}
I've done something similar with a list of files:
file("${path.module}/assets/files_90-disable-console-logs.yaml"),
file("${path.module}/assets/files_90-disable-auto-updates.yaml"),
...but would like to expand this to templatefiles (above).
Here's the code I've done for the list of files:
main.tf
variable files {
default = [
"files_90-disable-auto-updates.yaml",
"files_90-disable-console-logs.yaml",
]
}
output "snippets" {
value = flatten(module.ingition_snippets.files)
}
modules/main.tf
variable files {}
resource "null_resource" "files" {
for_each = toset(var.files)
triggers = {
snippet = file("${path.module}/assets/${each.value}")
}
}
output "files" {
value = [for s in null_resource.files: s.triggers.*.snippet]
}
Appreciate any help!
Both of these use-cases can be met without using any resource blocks at all, because the necessary features are built in to the Terraform language.
Here is a shorter way to write the example with static files:
variable "files" {
type = set(string)
}
output "files" {
value = tomap({
for fn in var.files : fn => file("${path.module}/assets/${fn}")
})
}
The above would produce a map from filenames to file contents, so the calling module can more easily access the individual file contents.
We can adapt that for templatefile like this:
variable "template_files" {
# We can't write down a type constraint for this case
# because each file might have a different set of
# template variables, but our later code will expect
# this to be a mapping type, like the default value
# you shared in your comment, and will fail if not.
type = any
}
output "files" {
value = tomap({
for fn, vars in var.template_files : fn => templatefile("${path.module}/assets/${fn}", vars)
})
}
Again, the result will be a map from filename to the result of rendering the template with the given variables.
If your goal is to build a module for rendering templates from a source directory to publish somewhere, you might find the module hashicorp/dir/template useful. It combines fileset, file, and templatefile in a way that is hopefully convenient for static website publishing and similar use-cases. (At the time I write this the module is transitioning from being in my personal GitHub account to being in the HashiCorp organization, so if you look at it soon after you may see some turbulence as the docs get updated, etc.)

Terraform - Variable inside a variable

I would like to use a variable inside a variable.
This is my resource:
resource "aws_route" "vpc_peering_accepter" {
provider = "aws.accepter"
count = length(data.terraform_remote_state.vpc.outputs.${var.region}-vpc-private_routing_tables)
route_table_id = tolist(data.terraform_remote_state.vpc.outputs.${var.region}-vpc-private_routing_tables)[count.index]
destination_cidr_block = var.vpc_cidr
vpc_peering_connection_id = aws_vpc_peering_connection.peer.*.id[0]
}
Of course this one is not working.
What's the best practice to do it?
Thanks,
Elad
You can combine Local Values with the lookup function to accomplish this.
In the following example the null datasource is mimicking data.terraform_remote_state.vpc.outputs:
variable "region" {
default = "us-east1"
}
locals {
vpc_private_routing_tables = "${var.region}-vpc-private_routing_tables"
}
data "null_data_source" "values" {
inputs = {
us-east1-vpc-private_routing_tables = "11111111"
us-east2-vpc-private_routing_tables = "22222222"
}
}
output "vpc_peering" {
value = lookup(data.null_data_source.values.inputs, local.vpc_private_routing_tables)
}
Because data.terraform_remote_state.vpc.outputs is a mapping, you can use either attribute syntax or index syntax to access the values inside:
Attribute syntax: data.terraform_remote_state.vpc.outputs.us-west-1-vpc-private_routing_tables
Index syntax: data.terraform_remote_state.vpc.outputs["us-west-1-vpc-private_routing_tables"]
An advantage of index syntax is that you can use any expression within those brackets as long as its result is a string. In particular, you can use the template interpolation syntax:
data.terraform_remote_state.vpc.outputs["${var.region}-vpc-private_routing_tables"]
With that said, in this sort of situation where you are producing the same information for a number of different objects -- regions, in this case -- it's more conventional to gather all of these values into a single mapping when you declare the output, so that these related values are explicitly grouped together in a single collection. For example:
output "vpc_private_routing_table_ids" {
value = {
us-east-1 = aws_route_table.us-east-1.id
us-west-2 = aws_route_table.us-west-2.id
}
}
Then from the perspective of the consumer -- that is, the module that is using data "terraform_remote_state" to access these outputs -- this appears as a simple map keyed by region:
data.terraform_remote_state.vpc.outputs.vpc_private_routing_table_ids[var.region]
If you are producing many different objects on a per-region basis then you might choose to gather all of their ids together into a single output, which might be more convenient to use elsewhere:
output "regions" {
value = {
us-east-1 = {
vpc_id = aws_vpc.us-east-1.id
subnet_ids = aws_subnet.us-east-1[*].id
private_route_table_id = aws_route_table.us-east-1.id
}
us-west-1 = {
vpc_id = aws_vpc.us-west-1.id
subnet_ids = aws_subnet.us-west-1[*].id
private_route_table_id = aws_route_table.us-west-1.id
}
}
}
...which would then appear as follows in the consumer module:
data.terraform_remote_state.vpc.outputs.regions[var.region].private_route_table_id
Ultimately you can structure your output values however you like, but I'd recommend choosing a shape that optimizes for clarity in the configuration that is referring to the data. That usually means making the referring expressions as simple as possible, and ideally avoiding complex expressions like string template syntax whenever possible.

Resources