I'd appreciate some help in better understanding when to use each.key vs each.value.
I have a transit gateway and wanted to ensure that any new transit gateway attachment is propagated into all available route tables. My code looked like this:
locals {
propagation_rt_ids = {
tgw-rt-001 = "tgw-rtb-xxx”
tgw-rt-002 = "tgw-rtb-yyy"
tgw-rt-003 = "tgw-rtb-zzz”
}
}
resource "aws_ec2_transit_gateway_route_table_propagation" "propagate_attachment” {
for_each = local.propagation_rt_ids
transit_gateway_attachment_id = data.aws_ec2_transit_gateway_vpc_attachment.tenant_id.id
transit_gateway_route_table_id = local.propagation_rt_ids[each.value]
}
When I referenced each.value and ran terraform plan, I got errors saying: The given key does not identify an element in this collection value
I eventually found a similar example where [each.key] was used. I tried [each.key] and it worked.
What I am trying to understand is this:
I was thinking that the “transit_gateway_route_table_id” argument under aws_ec2_transit_gateway_route_table_propagation is expecting a route table ID as its value. So I assumed the correct thing to specify there was local.propagation_rt_ids[each.value] which should retrieve each of the route-table IDs in the key/value pair. Why is [each.value] not valid in this scenario?
Maps and objects in terraform are represented by key/value pairs. From the documentation [1]:
{
name = "John"
age = 52
}
Map attributes can be accessed by using either the dot notation, e.g., local.propagation_rt_ids.tgw-rt-001 or using a square-bracket index notation local.propagation_rt_ids["tgw-rt-001"]. So, for maps, in order to get a value of an attribute, you have to reference a certain key. If we take the example you posted and use terraform console:
> local.propagation_rt_ids
{
"tgw-rt-001" = "tgw-rtb-xxx" # key = value
"tgw-rt-002" = "tgw-rtb-yyy" # key = value
"tgw-rt-003" = "tgw-rtb-zzz" # key = value
}
> local.propagation_rt_ids.tgw-rt-001
"tgw-rtb-xxx"
> local.propagation_rt_ids["tgw-rt-001"]
"tgw-rtb-xxx"
Now, if I were to try and reference a value instead of a key:
> local.propagation_rt_ids["tgw-rtb-xxx"]
╷
│ Error: Invalid index
│
│ on <console-input> line 1:
│ (source code not available)
│
│ The given key does not identify an element in this collection value.
╵
The first example works, because the attribute value is being fetched by using one of the keys, i.e. "tgw-rt-001". In the second example, the error is the same as the one you got, because I tried to get an attribute value based on a key that does not exist, as it is in fact a value. In other words, a key references a value but a value references nothing, so something like:
key1 -> value1 -> no reference
key2 -> value2 -> no reference
key3 -> value3 -> no reference
[1] https://developer.hashicorp.com/terraform/language/expressions/types#maps-objects
Related
I try to pass some parameters to the google pub sub terraform module where they use a code block like this
for_each = var.create_topic ? { for i in var.push_subscriptions : i.name => i if try(i.dead_letter_topic, "") != "" } : {}
When I pass variables for the push_subscription like:
push_subscriptions = [
{
name = "push"
push_endpoint = "https://example.com/push"
dead_letter_topic = null
},
]
I will get an error with:
on .terraform/modules/pubsub/main.tf line 62, in resource "google_pubsub_topic_iam_member" "push_topic_binding":
│ 62: topic = each.value.dead_letter_topic
│
│ The argument "topic" is required, but no definition was found.
When I completely remove the dead_letter_topic variable it works fine.
Im wondering why this is the case? I thought (and read) when something is null then terraform threads it like it does not exist? So in my opinion both options should result in the same outcome.
Your code is trying to create a google_pubsub_topic_iam_member by passing the value of dead_letter_topic as the topic value for that resource. The topic value of that resource is a required value that you have to set. If dead_letter_topic is null, then you are trying to create a google_pubsub_topic_iam_member that has a null or empty string for a topic.
Terraform treats optional attributes as "unset" when you pass a null value. You can't do that at all for required attributes.
I believe the logic in your for_each is flawed. I think part of the problem may be that you are turning a list into a map, instead of a set.
since the last update of the Logicmonitor provider in Terraform we're struggling with a sorting isse.
In LogicMonitor the properties of a device are a name-value pair, and they are presented alfabetically by name. Also in API requests the result is alphabetical. So far nothing fancy.
But... We build our Cloud devices using a module. Calling the module we provide some LogicMonitor properties specially for this device, and a lot more are provided in the module itself.
In the module this looks like this:
`
custom_properties = concat([
{
name = "host_fqdn"
value = "${var.name}.${var.dns_domain}"
},
{
name = "ocid"
value = oci_core_instance.server.id
},
{
name = "private_ip"
value = oci_core_instance.server.private_ip
},
{
name = "snmp.version"
value = "v2c"
}
],
var.logicmonitor_properties)
`
The first 4 properties are from the module and combined with anyting what is in var.logicmonitor_properties. On the creation of the device in LogicMonitor all properties are set in the order the are and no problem.
The issue arises when there is any update on a terraform file in this environment. Due to the fact the properties are presented in alphabetical order, Terraform is showing a lot of changes if finds (but which are in fact just a mixed due to sorting).
The big question is: How can I sort the complete list of properties bases on the "name".
Tried to work with maps, sort and several other functions and examples, but got nothing working on key-value pairs. Merging single key's works fine in a map, but how to deal with name/value pairs/
I think you were on the right track with maps and sorting. Terraform maps do not preserve any explicit ordering themselves, and so whenever Terraform needs to iterate over the elements of a map in some explicit sequence it always do so by sorting the keys lexically (by Unicode codepoints) first.
Therefore one answer is to project this into a map and then project it back into a list of objects again. The projection back into list of objects will implicitly sort the map elements by their keys, which I think will get the effect you wanted.
variable "logicmonitor_properties" {
type = list(object({
name = string
value = string
}))
}
locals {
base_properties = tomap({
host_fqdn = "${var.name}.${var.dns_domain}"
ocid = oci_core_instance.server.id
private_ip = oci_core_instance.server.private_ip
"snmp.version" = "v2c"
})
extra_properties = tomap({
for prop in var.logicmonitor_properties : prop.name => prop.value
})
final_properties = merge(local.base_properties, local.extra_properties)
# This final step will implicitly sort the final_properties
# map elements by their keys.
final_properties_list = tolist([
for k, v in local.final_properties : {
name = k
value = v
}
])
}
With all of the above, local.final_properties_list should be similar to the custom_properties structure you showed in your question except that the elements of the list will be sorted by their names.
This solution assumes that the property names will be unique across both base_properties and extra_properties. If there are any colliding keys between both of those maps then the merge function will prefer the value from extra_properties, overriding the element of the same key from base_properties.
First, use the sort() function to sort the keys in alphabetical order:
sorted_keys = sort(keys(var.my_map))
Next, use the map() function to create a new map with the sorted keys and corresponding values:
sorted_map = map(sorted_keys, key => var.my_map[key])
Finally, you can use the jsonencode() function to print the sorted map in JSON format:
jsonencode(sorted_map)```
I've searched quite a bit and don't think I've found the answer I really need. I'm trying to loop through a nested block and am successful in doing this if all of the attributes are on the same root object. This is great if I want to loop over the entire set of attributes. However this situation is a bit different. I need to loop over an entire set of attributes and also a sub-set.
In this Terragrunt example, you can see the desired inputs since we want to loop over the escalation policy entirely as well as loop the rule and its targets so that we can create many escalation policies with many rules/targets in them.
/// PagerDuty Escalation Policies
create_escalation_policy = true
escalation_policies = [
{
name = "TEST Engineering Escalation 1"
description = "My TEST engineering escalation policy 1"
teams = ["111N1CV"]
num_loops = 2
rule = [
{
escalation_delay_in_minutes = 15
target = {
type = "user_reference"
id = "ABCB8F3"
}
},
{
escalation_delay_in_minutes = 15
target = {
type = "user_reference"
id = "NBCB1A1"
}
}
}
]
However, after quite a bit of trial and error, I'm able to loop over the entire escalation policy but not if we have values inside of rule = { which returns a generic error that Terraform can't find those attributes in the object which I have confirmed is the root object instead of the nested one. This was validated by simply moving those attributes out to the root of the object input block.
│ Error: Unsupported attribute
│
│ on main.tf line 121, in resource "pagerduty_escalation_policy" "this":
│ 121: id = rule.value.id
│ ├────────────────
│ │ rule.value is object with 5 attributes
│
│ This object does not have an attribute named "id".
For reference, here is the variable for var.escalation_policies
variable "escalation_policies" {
description = "A list of escalation policies and rules for a given PagerDuty service."
type = any
}
and the resource
resource "pagerduty_escalation_policy" "this" {
for_each = {
for key in var.escalation_policies : key.name => {
name = key.name
description = key.description
num_loops = key.num_loops
teams = key.teams
}
if var.create_escalation_policy == true
}
name = each.value.name
description = each.value.description
num_loops = each.value.num_loops
teams = each.value.teams
dynamic "rule" {
for_each = {
for k, v in var.escalation_policies : k => v }
content {
escalation_delay_in_minutes = rule.value.escalation_delay_in_minutes
target {
type = rule.value.type
id = rule.value.id
}
}
}
}
With your current example the dynamic "rule" block has a for expression that isn't really doing anything useful:
{ for k, v in var.escalation_policies : k => v }
This expression is strange in two ways:
Taking the expression alone, it's unusual to project k, v directly to k => v because that doesn't really change anything about the key or the value. Since your source var.escalation_policies is a list rather than a map this is changing the data type of the result and making Terraform convert the integer indices to strings instead, but otherwise the elements are the same as var.escalation_policies.
Considering the context, this is also unusual because it's repeating the nested block based on the same collection as the containing resource: there will be one pagerduty_escalation_policy.this instance per var.escalation_policy element and then each one will have one nested rule block for each of your escalation policies.
To get a useful result the for_each in your dynamic block should use a different collection as the basis for its repetition. I think in your case you're intending to use the nested lists inside the rule attributes of each of your policies, but your outermost for_each expression doesn't include the rules so you'll first need to update that:
resource "pagerduty_escalation_policy" "this" {
for_each = {
for policy in var.escalation_policies : policy.name => {
name = policy.name
description = policy.description
num_loops = policy.num_loops
teams = policy.teams
rules = policy.rule
}
if var.create_escalation_policy == true
}
# ...
}
This means that each.value will now include an additional attribute rules which has the same value as the corresponding attribute in each element of var.escalation_policies.
You can then refer to that rules attribute in your dynamic block:
dynamic "rule" {
for_each = each.value.rules
content {
escalation_delay_in_minutes = rule.value.escalation_delay_in_minutes
target {
type = rule.value.target.type
id = rule.value.target.id
}
}
}
This tells Terraform to generate a dynamic rule block for each element of each.value.rules, which is the rules attribute for the current policy.
Inside the content block rule.value is the current rule object, so you can refer to attributes like escalation_delay_in_minutes and target from that object.
id is a key within the target object and not within rule:
id = rule.value.target.id
Note also that a for expression which iterates through key-value pairs within a map and outputs the exact same structure is extraneous and can be removed for efficiency and readability:
dynamic "rule" {
for_each = var.escalation_policies
...
}
I am trying to create a GCP project with this:
module "project-factory" {
source = "terraform-google-modules/project-factory/google"
version = "11.2.3"
name = var.project_name
random_project_id = "true"
org_id = var.organization_id
folder_id = var.folder_id
billing_account = var.billing_account
activate_apis = [
"iam.googleapis.com",
"run.googleapis.com"
]
}
After that, I am trying to create a service account, like so:
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "4.0.3"
project_id = module.project-factory.project_id
generate_keys = "true"
names = ["backend-runner"]
project_roles = [
"${module.project-factory.project_id}=>roles/cloudsql.client",
"${module.project-factory.project_id}=>roles/pubsub.publisher"
]
}
To be honest, I am fairly new to Terraform. I have read a few answers on the topic (this and this) but I am unable to understand how that would apply here.
I am getting the error:
│ Error: Invalid for_each argument
│
│ on .terraform/modules/pubsub-exporter-service-account/main.tf line 47, in resource "google_project_iam_member" "project-roles":
│ 47: for_each = local.project_roles_map_data
│ ├────────────────
│ │ local.project_roles_map_data will be known only after apply
│
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the
│ -target argument to first apply only the resources that the for_each depends on.
Looking forward to learn more about Terraform through this challenge.
With only parts of the configuration visible here I'm guessing a little bit, but let's see. You mentioned that you'd like to learn more about Terraform as part of this exercise, so I'm going to go into a lot of detail about the chain here to explain why I'm recommending what I'm going to recommend, though you can skip to the end if you find this extra detail uninteresting.
We'll start with that first module's definition of its project_id output value:
output "project_id" {
value = module.project-factory.project_id
}
module.project-factory here is referring to a nested module call, so we need to look one level deeper in the nested module terraform-google-modules/project-factory/google//modules/core_project_factory:
output "project_id" {
value = module.project_services.project_id
depends_on = [
module.project_services,
google_project.main,
google_compute_shared_vpc_service_project.shared_vpc_attachment,
google_compute_shared_vpc_host_project.shared_vpc_host,
]
}
Another nested module call! 😬 That one declares its project_id like this:
output "project_id" {
description = "The GCP project you want to enable APIs on"
value = element(concat([for v in google_project_service.project_services : v.project], [var.project_id]), 0)
}
Phew! 😅 Finally an actual resource. This expression in this case seems to be taking the project attribute of a google_project_service resource instance, or potentially taking it from var.project_id if that resource was disabled in this instance of the module. Let's have a look at the google_project_service.project_services definition:
resource "google_project_service" "project_services" {
for_each = local.services
project = var.project_id
service = each.value
disable_on_destroy = var.disable_services_on_destroy
disable_dependent_services = var.disable_dependent_services
}
project here is set to var.project_id, so it seems like either way this innermost project_id output just reflects back the value of the project_id input variable, so we need to jump back up one level and look at the module call to this module to see what that was set to:
module "project_services" {
source = "../project_services"
project_id = google_project.main.project_id
activate_apis = local.activate_apis
activate_api_identities = var.activate_api_identities
disable_services_on_destroy = var.disable_services_on_destroy
disable_dependent_services = var.disable_dependent_services
}
project_id is set to the project_id attribute of google_project.main:
resource "google_project" "main" {
name = var.name
project_id = local.temp_project_id
org_id = local.project_org_id
folder_id = local.project_folder_id
billing_account = var.billing_account
auto_create_network = var.auto_create_network
labels = var.labels
}
project_id here is set to local.temp_project_id, which is declared further up in the same file:
temp_project_id = var.random_project_id ? format(
"%s-%s",
local.base_project_id,
random_id.random_project_id_suffix.hex,
) : local.base_project_id
This expression includes a reference to random_id.random_project_id_suffix.hex, and .hex is a result attribute from random_id, and so its value won't be known until apply time due to how that random_id resource type is implemented. (It generates a random value during the apply step and saves it in the state so it'll stay consistent on future runs.)
This means that (after all of this indirection) module.project-factory.project_id in your module is not a value defined statically in the configuration, and might instead be decided dynamically during the apply step. That means it's not an appropriate value to use as part of the instance key of a resource, and thus not appropriate to use as a key in a for_each map.
Unfortunately the use of for_each here is hidden inside this other module terraform-google-modules/service-accounts/google, and so we'll need to have a look at that one too and see how it's making use of the project_roles input variable. First, let's look at the specific resource block the error message was talking about:
resource "google_project_iam_member" "project-roles" {
for_each = local.project_roles_map_data
project = element(
split(
"=>",
each.value.role
),
0,
)
role = element(
split(
"=>",
each.value.role
),
1,
)
member = "serviceAccount:${google_service_account.service_accounts[each.value.name].email}"
}
There's a couple somewhat-complex things going on here, but the most relevant thing for what we're looking at here is that this resource configuration is creating multiple instances based on the content of local.project_roles_map_data. Let's look at local.project_roles_map_data now:
project_roles_map_data = zipmap(
[for pair in local.name_role_pairs : "${pair[0]}-${pair[1]}"],
[for pair in local.name_role_pairs : {
name = pair[0]
role = pair[1]
}]
)
A little more complexity here that isn't super important to what we're looking for; the main thing to consider here is that this is constructing a map whose keys are built from element zero and element one of local.name_role_pairs, which is declared directly above, along with local.names that it refers to:
names = toset(var.names)
name_role_pairs = setproduct(local.names, toset(var.project_roles))
So what we've learned here is that the values in var.names and the values in var.project_roles both contribute to the keys of the for_each on that resource, which means that neither of those variable values should contain anything decided dynamically during the apply step.
However, we've also learned (above) that the project and role arguments of google_project_iam_member.project-roles are derived from the prefixes of elements in the two lists you provided as names and project_roles in your own module call.
Let's return back to where we started then, with all of this extra information in mind:
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "4.0.3"
project_id = module.project-factory.project_id
generate_keys = "true"
names = ["backend-runner"]
project_roles = [
"${module.project-factory.project_id}=>roles/cloudsql.client",
"${module.project-factory.project_id}=>roles/pubsub.publisher"
]
}
We've learned that names and project_roles must both contain only static values decided in the configuration, and so it isn't appropriate to use module.project-factory.project_id because that won't be known until the random project ID has been generated during the apply step.
However, we also know that this module is expecting the prefix of each item in project_roles (the part before the =>) to be a valid project ID, so there isn't any other value that would be reasonable to use there.
Therefore we're at a bit of an empasse: this second module has a rather awkward design decision that it's trying to derive a both a local instance key and a reference to a real remote object from the same value, and those two situations have conflicting requirements. But this isn't a module you created, so you can't easily modify it to address that design quirk.
Given that, I see two possible approaches to move forward, neither ideal but both workable with some caveats:
You could take the approach the error message offered as a workaround, asking Terraform to plan and apply the resources in the first module alone first, and then plan and apply the rest on a subsequent run once the project ID is already decided and recorded in the state:
terraform apply -target=module.factory
terraform apply
Although it's annoying to have to do this initial create in two steps, it does at least only matter for the initial creation of this infrastructure. If you update it later then you won't need to repeat this two-step process unless you've changed the configuration in a way that requires generating a new project ID.
While working through the above we saw that this approach of generating and returning a random project ID was optional based on that first module's var.random_project_id, which you set to "true" in your configuration. Without that, the project_id output would be just a copy of your given name argument, which seems to be statically defined by reference to a root module variable.
Unless you particularly need that random suffix on your project ID, you could leave random_project_id unset and thus just get the project ID set to the same static value as your var.project_name, which should then be an acceptable value to use as a for_each key.
Ideally this second module would be designed to separate the values it's using for instance keys from the values it's using to refer to real remote objects, and thus it would be possible to use the random-suffixed name for the remote object but a statically-defined name for the local object. If this were a module under your control then I would've suggested a design change like that, but I assume the current unusual design of that third-party module (packing multiple values into a single string with a delimiter) is a compromise resulting from wanting to retain backward compatibility with an earlier iteration of the module.
I am trying to pass multiple values to pricipals's identifiers in the data resource "aws_iam_policy_document". getting the following error
Inappropriate value for attribute "identifiers": element 0: string required.
s3_values variable is defined type = any and set the values as
....
s3_values:
bucket: bucketname1
s3_arns:
- arn:aws:iam::1234567890:root
- arn:aws:iam::2345678901:role/s3-read-role
data "aws_iam_policy_document" "s3_policy" {
count = length(var.s3_arns)
statement {
sid = "1"
effect = "Allow"
principals {
type = "AWS"
identifiers = ["${var.s3_values[count.index]["s3_arns"]}"]
}
actions = ["s3:PutObject"]
resources = ["arn:aws:s3:::${var.s3_values[count.index]["bucket"]}/*"]
}
}
I get the following error
Inappropriate value for attribute "identifiers": element 0: string required.
its working , when only one value is passed , but not working when we pass multiple values to the variable s3_arns.
It looks like you're trying to create multiple policy documents for a single S3 bucket. Rather than using count to create many documents, it would be best if you created a single policy document that gives access to each ARN you pass.
Currently it works for one ARN because the identifiers field gets passed a single string and creates a list with one string element. When you pass a list of ARNs, the identifiers field is instead creating a list with a list element that contains the ARN strings.
I would fix this by making the s3_arns field always be a list of strings, and removing the count field on the data resource. Once you do that you can change the line identifiers to be identifiers = var.s3_values.s3_arns and the resources line to be resources = ["arn:aws:s3:::${var.s3_values.bucket}/*"]