In terraform, how to loop over a list of objects and update values? - terraform

I will be providing values with terragrunt. The variable, push_subscriptions, is a list of maps and I want to modify the values of the map. For example, append a prefix to the push subscription name in place like so (within the main.tf):
push_subscriptions[index]['name'] = "$pbsb-push-${var.product_environment_code}-push_subscriptions[index]['name']"
main.tf
module "pubsub" {
push_subscriptions = var.push_subscriptions
}
terragrunt.hcl
include "product_vars" {
path = find_in_parent_folders("_terragrunt.hcl")
}
inputs = {
push_subscriptions = [
{
name = "push-sub-1"
ack_deadline_seconds = 20
push_endpoint = "https://example.com"
},
{
name = "push-sub-2"
ack_deadline_seconds = 20
push_endpoint = "https://example.com"
}
]
}

Shouldn't be a problem. Just create a local where you'll be using it, that iterates over the list and returns another list of objects with the updated values.
In this example, local.subs is used in lieu of your variable, but you would just replace local.subs with var.push_subscriptions in your case.
locals {
subs = [
{ name = "foo" },
{ name = "bar" },
]
updated = [for sub in local.subs : { name = "some-prefix-${sub.name}" }]
}
output "updated" {
value = local.updated
}
Which gives:
Changes to Outputs:
+ updated = [
+ {
+ name = "some-prefix-foo"
},
+ {
+ name = "some-prefix-bar"
},
]
So that is a new value you can use with prefixes.
Or you could do this entirely in line, with something like:
module "pubsub" {
push_subscriptions = [for sub in var.push_subscriptions : merge(sub, {
name = "pbsb-push-${var.product_environment_code}-some-prefix-${sub.name}"
})]
}
Using merge here allows you to maintain all the other values.

Related

convert list to strings by iterating resource with variable

Trying to convert list to strings by iterating resource with variable using if condition
How to use if condition for resource to iterate on variable(list) in for_each
locals {
new_out = flatten([
for opi, clsan in var.module_cl_list : {
opic_R_P = reverse(split("#", "${opi}"))[1]
#op_R = ["WRITE"]
op_R = ["READ","DESCRIBE"]
}
])
}
for "Write" it is working, because resource takes one value per variable at a time
resource "something" "some" {
for_each = {for opi1, clsa1 in local.new_out: opi1 => clsa1}
name = local.new_out_opic_R_P
op_R = join(", ", each.value.op_R)
permission = "ALLOW"
}
for resource if we want to iterate the op_R variable, ["READ","DESCRIBE"], How to convert list to string iteratively?
Input
if input variable is write only, variable should be assigned as write
if input variable is Read only, variable should be assigned itereatively with "READ" and "Describe"
note: WO = WRITE, RO = READ and DESCRIBE
Example :-
west = {
name = "random1"
operation = "WRITE"
permission = "ALLOW"
},
west2 = {
name = "random2"
operation = "READ"
permission = "ALLOW"
},
west2 = {
name = "random2"
operation = "DESCRIBE"
permission = "ALLOW"
}
Current code works with this variable
op_R = ["WRITE"]
If given below variable, It fails to process i need this to do in iterative manner
op_R = ["READ","DESCRIBE"]
Input
module_cl_list = {
"west#WO" = {appid = "456"},
"west2#RO" = {appid = "123"}
}
Error
Current code taking value as "Read, Describe" , resource takes only one value at a time for each variable
You need to use double for loop:
new_out = flatten([
for opi, clsan in var.module_cl_list : [
for op in (split("#", "${opi}")[1] == "RO" ? ["READ","DESCRIBE"] : ["WRITE"]): {
opic_R_P = reverse(split("#", "${opi}"))[1]
op_R = op
}
]
])
which will produce:
[
{
"op_R" = "READ"
"opic_R_P" = "west2"
},
{
"op_R" = "DESCRIBE"
"opic_R_P" = "west2"
},
{
"op_R" = "WRITE"
"opic_R_P" = "west"
},
]

Is there possibility to dynamicly pass user-defined variable (key = value) to terraform module?

There is resource:
resource "resource_name" "foo" {
name = "test"
config {
version = 14
resources {
disk_type_id = "network-ssd"
}
postgresql_config = {
enable_parallel_hash = true
}
}
}
I need a module which accepts optional user variables in "postgresql_config". There can be many such variables.
I tried next:
variables.tf
variable "postgresql_config" {
description = "User defined for postgresql_config"
type = list(object({
# key1 = value1
# ...
# key50 = value50
}))
}
variable "config" {
description = "for dynamic block 'config' "
type = list(object({
version = number
}))
default = [{
version = 14
}]
}
variable "resources" {
description = "for dynamic block 'resources' "
type = list(object({
disk_type_id = string
}))
default = [{
disk_type_id = "network-hdd"
}]
}
module/postgresql/main.tf
resource "resource_name" "foo" {
name = "test"
dynamic "config" {
for_each = var.config
content {
version = config.value["version"]
dynamic "resources" {
for_each = var.resources
content {
disk_type_id = resources.value["disk_type_id"]
}
}
# problem is here
postgresql_config = {
for_each = var.postgresql_config
each.key = each.value
}
}
}
example/main.tf
module "postgresql" {
source = "../module/postgresql"
postgresql_config = [{
auto_explain_log_buffers = true
log_error_verbosity = "LOG_ERROR_VERBOSITY_UNSPECIFIED"
max_connections = 395
vacuum_cleanup_index_scale_factor = 0.2
}]
That is, I understand that I need to use "dynamic", but it can only be applied to the block "config" and the nested block "resource_name".
How can I pass values for "postgresql_config" from main.tf to module? Of course, my example with for_each = var.postgresql_config doesn't work, but I hope this way to give an idea of what I need.
Or does terraform have no such option to use custom variables dynamically at all, and all of them must be specified explicitly?
Any help would be appreciated, thank you
from what I understand , you are trying to create a map dynamically for your resource postgres_config.
I would recommend using a for expression to solve that problem.
However, I think your problem lies in how you have defined variables for your module . You might run into a problem if your postgress_config list has multiple configs in it because that config can only take a map by the looks of it.
have a look at the following documentation:
this one is for how to define your variables
https://www.terraform.io/language/expressions/dynamic-blocks#multi-level-nested-block-structures
for expressions
https://www.terraform.io/language/expressions/for
my solution for your config problem ,would be something like this assuming that the postgres_config list has one element all the time:
# problem is here
postgresql_config = var.postgresql_config[0]

terraform combine 2 objects with 2 attributes for aws instance_ids

Following hashicorp doc to leverage for_each to provision multiple instance using local var map.
I am unable to get the instance_ids into a single lists for output:
output "instance_ids" {
description = "IDs of EC2 instances"
value = { for p in sort(keys(var.project)) : p => module.ec2_instances[p].instance_ids }
}
This is the output:
instance_ids = {
"client-webapp" = [
"i-0e11fcc341e6ce292",
"i-0b7ddd178c0590116",
"i-0c570628d3997874b",
"i-0a1642d7cc173f329",
]
"internal-webapp" = [
"i-0e65c8569f2d2c6f5",
"i-0c62e911e9446c53b",
]
}
Looking to get both objects lists of instance_ids into single list. Any good recommendation? Attempt to use merge, flatten, concat fail with various errors.
The var context for ids in above output loops thru the KEYs 'client-webapp' & 'internal-webapp'
variable "project" {
description = "Map of project names to configuration."
type = map
default = {
client-webapp = {
public_subnets_per_vpc = 2,
private_subnets_per_vpc = 2,
instances_per_subnet = 2,
instance_type = "t2.micro",
environment = "dev"
},
internal-webapp = {
public_subnets_per_vpc = 1,
private_subnets_per_vpc = 1,
instances_per_subnet = 2,
instance_type = "t2.nano",
environment = "test"
}
}
}
Any suggestions?
You can concatenate both lists together.
locals {
instance_ids = {
"client-webapp" = [
"i-0e11fcc341e6ce292",
"i-0b7ddd178c0590116",
"i-0c570628d3997874b",
"i-0a1642d7cc173f329",
]
"internal-webapp" = [
"i-0e65c8569f2d2c6f5",
"i-0c62e911e9446c53b",
]
}
new-list = concat(local.instance_ids["client-webapp"], local.instance_ids["internal-webapp"])
}
output "new-list" {
description = "my new list"
value = local.new-list
}
Here is the output
Changes to Outputs:
+ new-list = [
+ "i-0e11fcc341e6ce292",
+ "i-0b7ddd178c0590116",
+ "i-0c570628d3997874b",
+ "i-0a1642d7cc173f329",
+ "i-0e65c8569f2d2c6f5",
+ "i-0c62e911e9446c53b",
]
Instead of creating a map with the p =>, can you just return the array? And flatten.
Something like...
output "instance_ids" {
description = "IDs of EC2 instances"
value = flatten({ for p in sort(keys(var.project)) : module.ec2_instances[p].instance_ids })
}

Iterate over list of list of maps in terraform

Consider I have a variable that is a list of list of maps.
Example:
processes = [
[
{start_cmd: "a-server-start", attribute2:"type_a"},
{start_cmd: "a-worker-start", attribute2:"type_b"}
{start_cmd: "a--different-worker-start", attribute2:"type_c"}
],
[
{start_cmd: "b-server-start", attribute2:"type_a"},
{start_cmd: "b-worker-start", attribute2:"type_b"}
]
]
In each iteration, I need to take out the array of maps, then iterate over that array and take out the values of the map. How do I achieve this in terraform?
I have considered having two counts and doing some arithmetic to trick terraform into performing a lookalike nested iteration Check reference here. But in our case the number of maps in the inner array can vary.
Also we are currently using the 0.11 terraform version but dont mind using the alpha 0.12 version of terraform if it is possible to achieve this in that version.
Edit:
Added how I would use this variable:
resource “create_application” “applications” {
// Create a resource for every array in the variable processes. 2 in this case
name = ""
migration_command = ""
proc {
// For every map create this attribute for the resource.
name = ““
init_command = “a-server-start”
type = “server”
}
}
Not sure if this clears up the requirement. Please do ask if it is still not clear.
Using terraform 0.12.x
locals {
processes = [
[
{ start_cmd: "a-server-start", type: "type_a", name: "inglorious bastards" },
{ start_cmd: "a-worker-start", type: "type_b", name: "kill bill" },
{ start_cmd: "a--different-worker-start", type: "type_c", name: "pulp fiction" },
],
[
{ start_cmd: "b-server-start", type: "type_a", name: "inglorious bastards" },
{ start_cmd: "b-worker-start", type: "type_b", name: "kill bill" },
]
]
}
# just an example
data "archive_file" "applications" {
count = length(local.processes)
type = "zip"
output_path = "applications.zip"
dynamic "source" {
for_each = local.processes[count.index]
content {
content = source.value.type
filename = source.value.name
}
}
}
$ terraform apply
data.archive_file.applications[0]: Refreshing state...
data.archive_file.applications[1]: Refreshing state...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
If a create_application resource existed, it can be modeled like so
resource "create_application" "applications" {
count = length(local.processes)
name = ""
migration_command = ""
dynamic "proc" {
for_each = local.processes[count.index]
content {
name = proc.value.name
init_command = proc.value.start_cmd
type = proc.value.type
}
}
}
Here is my solution that work like charm. Just note the tricks google_service_account.purpose[each.value["name"]].name where I can retrieve the named array element by using its name.
variable "my_envs" {
type = map(object({
name = string
bucket = string
}))
default = {
"dev" = {
name = "dev"
bucket = "my-bucket-fezfezfez"
}
"prod" = {
name = "prod"
bucket = "my-bucket-ezaeazeaz"
}
}
}
resource "google_service_account" "purpose" {
for_each = var.my_envs
display_name = "blablabla (terraform)"
project = each.value["name"]
account_id = "purpose-${each.value["name"]}"
}
resource "google_service_account_iam_binding" "purpose_workload_identity_binding" {
for_each = var.my_envs
service_account_id = google_service_account.purpose[each.value["name"]].name
role = "roles/iam.whatever"
members = [
"serviceAccount:${each.value["name"]}.svc.id.goog[purpose/purpose]",
]
}
resource "google_storage_bucket_iam_member" "purpose_artifacts" {
for_each = var.my_envs
bucket = each.value["bucket"]
role = "roles/storage.whatever"
member = "serviceAccount:${google_service_account.purpose[each.value["name"]].email}"
}

map list of maps to a list of selected field values in terraform

If resources use a count parameter to specify multi resources in terraform there is a simple syntax for providing a list/array of dedicated fields for the resource instances.
for example
aws_subnet.foo.*.id
Since quite a number of versions it is possible to declare variables with a complex structure, for example lists of maps.
variable "data" {
type = "list"
default = [
{
id = "1"
...
},
{
id = "10"
...
}
]
}
I'm looking for a possibility to do the same for varaibles I can do for multi resources: a projection of an array to an array of field values of the array elements.
Unfortunately
var.data.*.id
does not work as for resources. Is there any possibility to do this?
UPDATE
Massive fancy features have been added into terraform since Terraform 0.12 was released, e.g., list comprehension, with which the solution is super easy.
locals {
ids = [for d in var.data: d.id]
#ids = [for d in var.data: d["id"]] #same
}
# Then you could get the elements this way,
# local.ids[0]
Solution before terraform 0.12
template_file can help you out.
data "template_file" "data_id" {
count = "${length(var.data)}"
template = "${lookup(var.data[count.index], "id")}"
}
Then you get a list "${data.template_file.data_id.*.rendered}", whose elements are value of "id".
You can get its element by index like this
"${data.template_file.data_id.*.rendered[0]}"
or through function element()
"${element(data.template_file.data_id.*.rendered, 0)}"
NOTE: This answer and its associated question are very old at this point, and this answer is now totally stale. I'm leaving it here for historical reference, but nothing here is true of modern Terraform.
At the time of writing, Terraform doesn't have a generalized projection feature in its interpolation language. The "splat syntax" is implemented as a special case for resources.
While deep structure is possible, it is not yet convenient to use, so it's recommended to still keep things relatively flat. In future it is likely that new language features will be added to make this sort of thing more usable.
If have found a working solution using template rendering to by-pass the list of map's issue:
resource "aws_instance" "k8s_master" {
count = "${var.master_count}"
ami = "${var.ami}"
instance_type = "${var.instance_type}"
vpc_security_group_ids = ["${aws_security_group.k8s_sg.id}"]
associate_public_ip_address = false
subnet_id = "${element(var.subnet_ids,count.index % length(var.subnet_ids))}"
user_data = "${file("${path.root}/files/user_data.sh")}"
iam_instance_profile = "${aws_iam_instance_profile.master_profile.name}"
tags = "${merge(
local.k8s_tags,
map(
"Name", "k8s-master-${count.index}",
"Environment", "${var.environment}"
)
)}"
}
data "template_file" "k8s_master_names" {
count = "${var.master_count}"
template = "${lookup(aws_instance.k8s_master.*.tags[count.index], "Name")}"
}
output "k8s_master_name" {
value = [
"${data.template_file.k8s_master_names.*.rendered}",
]
}
This will result in the following output:
k8s_master_name = [
k8s-master-0,
k8s-master-1,
k8s-master-2
]
A potentially simpler answer is to use the zipmap function.
Starting with an environment variable map compatible with ECS template definitions:
locals {
shared_env = [
{
name = "DB_CHECK_NAME"
value = "postgres"
},
{
name = "DB_CONNECT_TIMEOUT"
value = "5"
},
{
name = "DB_DOCKER_HOST_PORT"
value = "35432"
},
{
name = "DB_DOCKER_HOST"
value = "localhost"
},
{
name = "DB_HOST"
value = "my-db-host"
},
{
name = "DB_NAME"
value = "my-db-name"
},
{
name = "DB_PASSWORD"
value = "XXXXXXXX"
},
{
name = "DB_PORT"
value = "5432"
},
{
name = "DB_QUERY_TIMEOUT"
value = "30"
},
{
name = "DB_UPGRADE_TIMEOUT"
value = "300"
},
{
name = "DB_USER"
value = "root"
},
{
name = "REDIS_DOCKER_HOST_PORT"
value = "6380"
},
{
name = "REDIS_HOST"
value = "my-redis"
},
{
name = "REDIS_PORT"
value = "6379"
},
{
name = "SCHEMA_SCRIPTS_PATH"
value = "db-scripts"
},
{
name = "USE_LOCAL"
value = "false"
}
]
}
In the same folder launch terraform console for testing built-in functions. You may need to terraform init if you haven't already.
terraform console
Inside the console type:
zipmap([for m in local.shared_env: m.name], [for m in local.shared_env: m.value])
Observe the output of each list-item-map being a name-value-pair of a single map:
{
"DB_CHECK_NAME" = "postgres"
"DB_CONNECT_TIMEOUT" = "5"
"DB_DOCKER_HOST" = "localhost"
"DB_DOCKER_HOST_PORT" = "35432"
"DB_HOST" = "my-db-host"
"DB_NAME" = "my-db-name"
"DB_PASSWORD" = "XXXXXXXX"
"DB_PORT" = "5432"
"DB_QUERY_TIMEOUT" = "30"
"DB_UPGRADE_TIMEOUT" = "300"
"DB_USER" = "root"
"REDIS_DOCKER_HOST_PORT" = "6380"
"REDIS_HOST" = "my-redis"
"REDIS_PORT" = "6379"
"SCHEMA_SCRIPTS_PATH" = "db-scripts"
"USE_LOCAL" = "false"
}

Resources