Following hashicorp doc to leverage for_each to provision multiple instance using local var map.
I am unable to get the instance_ids into a single lists for output:
output "instance_ids" {
description = "IDs of EC2 instances"
value = { for p in sort(keys(var.project)) : p => module.ec2_instances[p].instance_ids }
}
This is the output:
instance_ids = {
"client-webapp" = [
"i-0e11fcc341e6ce292",
"i-0b7ddd178c0590116",
"i-0c570628d3997874b",
"i-0a1642d7cc173f329",
]
"internal-webapp" = [
"i-0e65c8569f2d2c6f5",
"i-0c62e911e9446c53b",
]
}
Looking to get both objects lists of instance_ids into single list. Any good recommendation? Attempt to use merge, flatten, concat fail with various errors.
The var context for ids in above output loops thru the KEYs 'client-webapp' & 'internal-webapp'
variable "project" {
description = "Map of project names to configuration."
type = map
default = {
client-webapp = {
public_subnets_per_vpc = 2,
private_subnets_per_vpc = 2,
instances_per_subnet = 2,
instance_type = "t2.micro",
environment = "dev"
},
internal-webapp = {
public_subnets_per_vpc = 1,
private_subnets_per_vpc = 1,
instances_per_subnet = 2,
instance_type = "t2.nano",
environment = "test"
}
}
}
Any suggestions?
You can concatenate both lists together.
locals {
instance_ids = {
"client-webapp" = [
"i-0e11fcc341e6ce292",
"i-0b7ddd178c0590116",
"i-0c570628d3997874b",
"i-0a1642d7cc173f329",
]
"internal-webapp" = [
"i-0e65c8569f2d2c6f5",
"i-0c62e911e9446c53b",
]
}
new-list = concat(local.instance_ids["client-webapp"], local.instance_ids["internal-webapp"])
}
output "new-list" {
description = "my new list"
value = local.new-list
}
Here is the output
Changes to Outputs:
+ new-list = [
+ "i-0e11fcc341e6ce292",
+ "i-0b7ddd178c0590116",
+ "i-0c570628d3997874b",
+ "i-0a1642d7cc173f329",
+ "i-0e65c8569f2d2c6f5",
+ "i-0c62e911e9446c53b",
]
Instead of creating a map with the p =>, can you just return the array? And flatten.
Something like...
output "instance_ids" {
description = "IDs of EC2 instances"
value = flatten({ for p in sort(keys(var.project)) : module.ec2_instances[p].instance_ids })
}
Related
I will be providing values with terragrunt. The variable, push_subscriptions, is a list of maps and I want to modify the values of the map. For example, append a prefix to the push subscription name in place like so (within the main.tf):
push_subscriptions[index]['name'] = "$pbsb-push-${var.product_environment_code}-push_subscriptions[index]['name']"
main.tf
module "pubsub" {
push_subscriptions = var.push_subscriptions
}
terragrunt.hcl
include "product_vars" {
path = find_in_parent_folders("_terragrunt.hcl")
}
inputs = {
push_subscriptions = [
{
name = "push-sub-1"
ack_deadline_seconds = 20
push_endpoint = "https://example.com"
},
{
name = "push-sub-2"
ack_deadline_seconds = 20
push_endpoint = "https://example.com"
}
]
}
Shouldn't be a problem. Just create a local where you'll be using it, that iterates over the list and returns another list of objects with the updated values.
In this example, local.subs is used in lieu of your variable, but you would just replace local.subs with var.push_subscriptions in your case.
locals {
subs = [
{ name = "foo" },
{ name = "bar" },
]
updated = [for sub in local.subs : { name = "some-prefix-${sub.name}" }]
}
output "updated" {
value = local.updated
}
Which gives:
Changes to Outputs:
+ updated = [
+ {
+ name = "some-prefix-foo"
},
+ {
+ name = "some-prefix-bar"
},
]
So that is a new value you can use with prefixes.
Or you could do this entirely in line, with something like:
module "pubsub" {
push_subscriptions = [for sub in var.push_subscriptions : merge(sub, {
name = "pbsb-push-${var.product_environment_code}-some-prefix-${sub.name}"
})]
}
Using merge here allows you to maintain all the other values.
I'm creating multiple Google Cloud Projects using a module which creates a custom service account for each one and provides that as an output eg
module "app_projects" {
count = var.number_application_projects
etc
}
I then have a list of IAM roles I want to assign to each Project Service account like this:
sa_roles = ["roles/run.developer","roles/appengine.deployer"]
resource "google_project_iam_member" "proj_sa_iam_roles" {
count = var.number_application_projects
for_each = toset(var.sa_roles)
project = module.app_projects[count.index].project_id
role = each.value
member = "serviceAccount:${module.app_projects[count.index].service_account_email}"
}
This runs into the error about "count" and "for_each" being mutually-exclusive, and I can't for the life of me figure out what to use instead, any help would be appreciated!
You can probably use setproduct, but the standard method is using flatten.
They even have a section tailored for similar use cases. Flattening nested structures for for_each.
Here is an example, that doesn't use your exact resource, but should be instructive and you can actually run it to test it out.
modules/app-project/variables.tf
variable "name" {}
modules/app-project/outputs.tf
output "name" {
value = var.name
}
modules/member/variables.tf
variable "project" {}
variable "role" {}
modules/member/outputs.tf
output "project_role" {
value = "${var.project}-${var.role}"
}
main.tf
locals {
roles = ["rad", "tubular"]
}
module "app_project" {
source = "./modules/app-project"
count = 2
name = "app-project-${count.index}"
}
module "project_role" {
source = "./modules/member"
for_each = { for pr in flatten([for p in module.app_project[*] : [
for r in local.roles : {
app_project_name = p.name
role = r
}]
]) : "${pr.app_project_name}-${pr.role}" => pr }
project = each.value.app_project_name
role = each.value.role
}
output "project_roles" {
value = values(module.project_role)[*].project_role
}
terraform plan output
Changes to Outputs:
+ project_roles = [
+ "app-project-0-rad",
+ "app-project-0-tubular",
+ "app-project-1-rad",
+ "app-project-1-tubular",
]
In your case specifically, I think something like this would work:
resource "google_project_iam_member" "proj_sa_iam_roles" {
for_each = { for i, pr in flatten([for p in module.app_project[*] : [
for r in var.sa_roles : {
app_project = p
role = r
}]
]) : "${i}-${pr.role}" => pr }
project = each.value.app_project.project_id
role = each.value.role
member = "serviceAccount:${each.value.app_project.service_account_email}"
}
Has any one tried to get flatten for 3 loops working? I keep getting an error when i try the 3rd one:
I am trying to loop through a list within a data resource - data.instances.sg.ids
Variable example:
alarms = [
{
instances = data.instances.ids ## a list of instance ids
config = [
metric_name = "disk_free"
threshold = "GreaterThan"
]
}
]
locals {
configs = flatten([
for config_key, config_list in var.alarms : [
for instance in config_list.instances : {
for config in config_list.configs : {
instance_id = instance
metric_name = config.name
threshold = config.threshold
}
}
]
])
}
how can i properly loop through and flatten this list with data instances list.
Thanks
Not sure what exactly you want to construct, but I think it should be:
locals {
configs = flatten([
for config_key, config_list in var.alarms : [
for instance in config_list.instances : [
for config in config_list.configs :
{
instance_id = instance
metric_name = config.name
threshold = config.threshold
}
]
]
])
}
I'd like to create every type of DNS record for a domain using a generic module, and so be able to call it with something like:
module "example_com_dns" {
source = "[PATH_TO_MODULES]/modules/dns"
domain = "example.com"
a_records = {
"#" = [SOME IP]
"www" = [SOME IP]
"home" = [SOME IP]
}
txt_records = {
"#" = "txt-foobar1"
"#" = "txt-foobar2"
"mail._domainkey.self" = "foobar"
}
mx_entries = {
"10" = "mail.someprovider.com"
"20" = "mail2.someprovider.com"
}
cname_records {
"cname-foo" = "cname-bar
}
}
I have something that works fine for A, CNAME , and MX records, but TXT has an edge case which I need to work around. My module has resource blocks for each type of record, which run through loops. I'll just paste the TXT one, but they're all the same:
resource "digitalocean_record" "this_txt_record" {
for_each = var.txt_records
domain = var.domain
type = "TXT"
name = each.key
value = each.value
}
This all works fine, except for the fact that since there are 2 records with "#" for their key, it results in only the last one being created (in my example above, this being "txt-foobar2"):
...
# module.example_com.digitalocean_record.this_txt_record["#"] will be created
+ resource "digitalocean_record" "this_txt_record" {
+ domain = "example.com"
+ fqdn = (known after apply)
+ id = (known after apply)
+ name = "#"
+ ttl = (known after apply)
+ type = "TXT"
+ value = "txt-foobar2"
}
I'd like for it to create both "txt-foobar1" and "txt-foobar2", even given non-unique keys in the map.
Perhaps this is the wrong way and I just need to figure out a clever loop for for parsing this structure instead?:
txt_records = [
{ "#" = "foo" },
{ "#" = "bar"},
{ "mail._domainkey.self" = "foobar"}
]
If so, I'm currently failing there too :)
Resources cannot be created by for_each'ing a list since there must be a unique key that will become part of the terraform resource name. List indexes cannot be a reliable key since your TF plan will be all messed up if you reorder items in the list.
Maps on the other hand do have unique keys by definition.
You can generate a map from a list though! I found this little trick here. Note you additionally need to manually compute the unique map key (${txt_record[0]}=${txt_record[1]} in the example below).
Your resources with updates in place:
module "example_com_dns" {
...
txt_records = [
["#", "txt-foobar1"],
["#", "txt-foobar2"],
["mail._domainkey.self", "foobar"],
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record[0]}=${txt_record[1]}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value[0]
value = each.value[1]
}
or slightly more verbose if you prefer:
module "example_com_dns" {
...
txt_records = [
{name: "#", value: "txt-foobar1"},
{name: "#", value: "txt-foobar2"},
{name: "mail._domainkey.self", value: "foobar"},
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record.name}=${txt_record.value}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value.name
value = each.value.value
}
Alternative way to already given one is to use the following:
variable "txt_records" {
default = {
"#" = ["foo", "bar"],
"mail._domainkey.self" = ["foobar"]
}
}
Then you can flatten the txt_records using:
locals {
txt_records_flat = merge([
for key, values in var.txt_records:
{for value in values:
"${key}-${value}" => {"record_name" = key, "record_value" = value}
}
]...)
}
which results in local.txt_records_flat of:
{
"#-bar" = {
"record_name" = "#"
"record_value" = "bar"
}
"#-foo" = {
"record_name" = "#"
"record_value" = "foo"
}
"mail._domainkey.self-foobar" = {
"record_name" = "mail._domainkey.self"
"record_value" = "foobar"
}
}
Then you use it:
resource "digitalocean_record" "this_txt_record" {
for_each = local.txt_records_flat
domain = var.domain
type = "TXT"
name = each.value.record_name
value = each.value.record_value
}
If resources use a count parameter to specify multi resources in terraform there is a simple syntax for providing a list/array of dedicated fields for the resource instances.
for example
aws_subnet.foo.*.id
Since quite a number of versions it is possible to declare variables with a complex structure, for example lists of maps.
variable "data" {
type = "list"
default = [
{
id = "1"
...
},
{
id = "10"
...
}
]
}
I'm looking for a possibility to do the same for varaibles I can do for multi resources: a projection of an array to an array of field values of the array elements.
Unfortunately
var.data.*.id
does not work as for resources. Is there any possibility to do this?
UPDATE
Massive fancy features have been added into terraform since Terraform 0.12 was released, e.g., list comprehension, with which the solution is super easy.
locals {
ids = [for d in var.data: d.id]
#ids = [for d in var.data: d["id"]] #same
}
# Then you could get the elements this way,
# local.ids[0]
Solution before terraform 0.12
template_file can help you out.
data "template_file" "data_id" {
count = "${length(var.data)}"
template = "${lookup(var.data[count.index], "id")}"
}
Then you get a list "${data.template_file.data_id.*.rendered}", whose elements are value of "id".
You can get its element by index like this
"${data.template_file.data_id.*.rendered[0]}"
or through function element()
"${element(data.template_file.data_id.*.rendered, 0)}"
NOTE: This answer and its associated question are very old at this point, and this answer is now totally stale. I'm leaving it here for historical reference, but nothing here is true of modern Terraform.
At the time of writing, Terraform doesn't have a generalized projection feature in its interpolation language. The "splat syntax" is implemented as a special case for resources.
While deep structure is possible, it is not yet convenient to use, so it's recommended to still keep things relatively flat. In future it is likely that new language features will be added to make this sort of thing more usable.
If have found a working solution using template rendering to by-pass the list of map's issue:
resource "aws_instance" "k8s_master" {
count = "${var.master_count}"
ami = "${var.ami}"
instance_type = "${var.instance_type}"
vpc_security_group_ids = ["${aws_security_group.k8s_sg.id}"]
associate_public_ip_address = false
subnet_id = "${element(var.subnet_ids,count.index % length(var.subnet_ids))}"
user_data = "${file("${path.root}/files/user_data.sh")}"
iam_instance_profile = "${aws_iam_instance_profile.master_profile.name}"
tags = "${merge(
local.k8s_tags,
map(
"Name", "k8s-master-${count.index}",
"Environment", "${var.environment}"
)
)}"
}
data "template_file" "k8s_master_names" {
count = "${var.master_count}"
template = "${lookup(aws_instance.k8s_master.*.tags[count.index], "Name")}"
}
output "k8s_master_name" {
value = [
"${data.template_file.k8s_master_names.*.rendered}",
]
}
This will result in the following output:
k8s_master_name = [
k8s-master-0,
k8s-master-1,
k8s-master-2
]
A potentially simpler answer is to use the zipmap function.
Starting with an environment variable map compatible with ECS template definitions:
locals {
shared_env = [
{
name = "DB_CHECK_NAME"
value = "postgres"
},
{
name = "DB_CONNECT_TIMEOUT"
value = "5"
},
{
name = "DB_DOCKER_HOST_PORT"
value = "35432"
},
{
name = "DB_DOCKER_HOST"
value = "localhost"
},
{
name = "DB_HOST"
value = "my-db-host"
},
{
name = "DB_NAME"
value = "my-db-name"
},
{
name = "DB_PASSWORD"
value = "XXXXXXXX"
},
{
name = "DB_PORT"
value = "5432"
},
{
name = "DB_QUERY_TIMEOUT"
value = "30"
},
{
name = "DB_UPGRADE_TIMEOUT"
value = "300"
},
{
name = "DB_USER"
value = "root"
},
{
name = "REDIS_DOCKER_HOST_PORT"
value = "6380"
},
{
name = "REDIS_HOST"
value = "my-redis"
},
{
name = "REDIS_PORT"
value = "6379"
},
{
name = "SCHEMA_SCRIPTS_PATH"
value = "db-scripts"
},
{
name = "USE_LOCAL"
value = "false"
}
]
}
In the same folder launch terraform console for testing built-in functions. You may need to terraform init if you haven't already.
terraform console
Inside the console type:
zipmap([for m in local.shared_env: m.name], [for m in local.shared_env: m.value])
Observe the output of each list-item-map being a name-value-pair of a single map:
{
"DB_CHECK_NAME" = "postgres"
"DB_CONNECT_TIMEOUT" = "5"
"DB_DOCKER_HOST" = "localhost"
"DB_DOCKER_HOST_PORT" = "35432"
"DB_HOST" = "my-db-host"
"DB_NAME" = "my-db-name"
"DB_PASSWORD" = "XXXXXXXX"
"DB_PORT" = "5432"
"DB_QUERY_TIMEOUT" = "30"
"DB_UPGRADE_TIMEOUT" = "300"
"DB_USER" = "root"
"REDIS_DOCKER_HOST_PORT" = "6380"
"REDIS_HOST" = "my-redis"
"REDIS_PORT" = "6379"
"SCHEMA_SCRIPTS_PATH" = "db-scripts"
"USE_LOCAL" = "false"
}