I'm trying to iterate over a simple list of maps. Here's a segment of what my module code looks like:
resource "helm_release" "nginx-external" {
count = var.install_ingress_nginx_chart ? 1 : 0
name = "nginx-external"
repository = "https://kubernetes.github.io/ingress-nginx"
chart = "ingress-nginx"
version = var.nginx_external_version
namespace = "default"
lint = true
values = [
"${file("chart_values/nginx-external.yaml")}"
]
dynamic "set" {
for_each = { for o in var.nginx_external_overrides : o.name => o }
content {
name = each.value.name
value = each.value.value
}
}
}
variable "nginx_external_overrides" {
description = "A map of maps to override customizations from the default chart/values file."
type = any
}
And here's a snippet of how I'm trying to call it from terragrunt:
nginx_external_overrides = [
{ name = "controller.metrics.enabled", value = "false" }
]
When trying to use this in a dynamic block, I'm getting:
Error: each.value cannot be used in this context
A reference to "each.value" has been used in a context in which it
unavailable, such as when the configuration no longer contains the value in
its "for_each" expression. Remove this reference to each.value in your
configuration to work around this error.
Ideally, I would be able to pass any number of maps in nginx_external_overrides to override the settings in the yaml being passed, but am struggling to do so. Thanks for the help.
If you are using for_each in dynamic blocks, you can't use each. Instead, in your case, it should be set:
dynamic "set" {
for_each = { for o in var.nginx_external_overrides : o.name => o }
content {
name = set.value.name
value = set.value.value
}
}
Related
I would like to create an AWS account with SSO Account Assignments in the same first terraform run without hit the for_each limitation with dynamic values that cannot be predicted during plan.
I've tried to separate the aws_organizations_account resource from aws_ssoadmin_account_assignment in completely separate TF module and also I tried to use depends_on between those resources and modules.
What is the simplest and correct way to fix this issue?
Terraform v1.2.4
AWS SSO Account Assignments Module
Closed Pull Request that did not fix this issue
main.tf file (aws module)
resource "aws_organizations_account" "account" {
name = var.aws_account_name
email = "${var.aws_account_name}#gmail.com"
tags = {
Name = var.aws_account_name
}
parent_id = var.aws_org_folder_id
}
data "aws_identitystore_group" "this" {
for_each = local.group_list
identity_store_id = local.identity_store_id
filter {
attribute_path = "DisplayName"
attribute_value = each.key
}
}
data "aws_identitystore_user" "this" {
for_each = local.user_list
identity_store_id = local.identity_store_id
filter {
attribute_path = "UserName"
attribute_value = each.key
}
}
data "aws_ssoadmin_instances" "this" {}
locals {
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.id, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
identity_store_id = tolist(data.aws_ssoadmin_instances.this.identity_store_ids)[0]
sso_instance_arn = tolist(data.aws_ssoadmin_instances.this.arns)[0]
group_list = toset([for mapping in var.account_assignments : mapping.principal_name if mapping.principal_type == "GROUP"])
user_list = toset([for mapping in var.account_assignments : mapping.principal_name if mapping.principal_type == "USER"])
}
resource "aws_ssoadmin_account_assignment" "this" {
for_each = local.assignment_map
instance_arn = local.sso_instance_arn
permission_set_arn = each.value.permission_set_arn
principal_id = each.value.principal_type == "GROUP" ? data.aws_identitystore_group.this[each.value.principal_name].id : data.aws_identitystore_user.this[each.value.principal_name].id
principal_type = each.value.principal_type
target_id = aws_organizations_account.account.id
target_type = "AWS_ACCOUNT"
}
main.tf (root)
module "sso_account_assignments" {
source = "./modules/aws"
account_assignments = [
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Administrators"
},
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-955c264e8f20fea3",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Developers"
},
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Developers"
},
]
}
The important thing about a map for for_each is that all of the keys must be made only of values that Terraform can "see" during the planning step.
You defined local.assignment_map this way in your example:
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.id, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
I'm not personally familiar with the aws_organizations_account resource type, but I'm guessing that aws_organizations_account.account.id is an attribute whose value gets decided by the remote system during the apply step (once the object is created) and so this isn't a suitable value to use as part of a for_each map key.
If so, I think the best path forward here is to use a different attribute of the resource that is defined statically in your configuration. If var.aws_account_name has a static value defined in your configuration (that is, it isn't derived from an apply-time attribute of another resource) then it might work to use the name attribute instead of the id attribute:
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.name, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
Another option would be to remove the organization reference from the key altogether. From what you've shared it seems like there is only one account and so all of these keys would end up starting with exactly the same account name anyway, and so that string isn't contributing to the uniqueness of those keys. If that's true then you could drop that part of the key and just keep the other parts as the unique key:
assignment_map = {
for a in var.account_assignments :
format(
"%v-%v-%v",
substr(a.principal_type, 0, 1),
a.principal_name,
a.permission_set_name,
) => a
}
If I want to define a lambda function with a VPC config. I can do it like this:
resource "aws_lambda_function" "lambda" {
function_name = "..."
...
vpc_config {
subnet_ids = ["..."]
security_group_ids = ["..."]
}
}
I would like to create the lambda in a terraform module and define the vpc_config in the module definition. I can define the module like this:
resource "aws_lambda_function" "lambda" {
function_name = "..."
...
dynamic "vpc_config" {
for_each = var.vpc_configs
content {
subnet_ids = vpc_config.value["subnet_ids"]
security_group_ids = vpc_config.value["security_group_ids"]
}
}
}
variable "vpc_configs" {
type = list(object({
subnet_ids = list(string)
security_group_ids = list(string)
}))
default = []
}
And then use it:
module "my_lambda" {
source = "./lambda"
...
vpc_configs = [
{
subnet_ids = ["..."]
security_group_ids = ["..."]
}
]
}
However, since there is only one vpc_config block allowed there is no point in defining the variable as a list. I would prefer the following syntax:
module "my_lambda" {
source = "./lambda"
...
vpc_config = {
subnet_ids = ["..."]
security_group_ids = ["..."]
}
# or:
#vpc_config {
# subnet_ids = ["..."]
# security_group_ids = ["..."]
#}
}
However, I can't figure out if it is possible to define a variable like this and then use it in a dynamic block. I defined it as a list in the first place because I don't always need a VPC config and this way I can simply leave the list empty and no VPC config will be created. Is there a way to create an optional vpc_config block through a simple map or object definition?
dynamic blocks work by generating one block for each element in a collection, if any, whereas you want to define a variable that is an optional non-collection value. Therefore the key to this problem is to translate from a single value that might be null (representing absence) into a list of zero or one elements.
Due to how commonly this arises, Terraform has a concise way to represent that conversion using the splat operator, [*]. If you apply it to a value that isn't a list, then it will implicitly convert it into a list of zero or one elements, depending on whether the value is null.
The example in the documentation I just linked to shows a practical example of this pattern. The following is essentially the same approach, but adapted to use the resource type that you are using in your question:
variable "vpc_config" {
type = object({
subnet_ids = list(string)
security_group_ids = list(string)
})
default = null
}
resource "aws_lambda_function" "lambda" {
function_name = "..."
...
dynamic "vpc_config" {
for_each = var.vpc_config[*]
content {
subnet_ids = vpc_config.value.subnet_ids
security_group_ids = vpc_config.value.security_group_ids
}
}
}
The default value of var.vpc_config is null, so if the caller doesn't set it then that is the value it will take.
var.vpc_config[*] will either return an empty list or a list containing one vpc_config object, and so this dynamic block will generate either zero or one vpc_config blocks depending on the "null-ness" of var.vpc_config.
so you are wanting a conditional dynamic block
you could possibly get away with it by doing a check similar to the one on the object below
dynamic "vpc_config"{
for_each = length(var.vpc_config) > 0 ? {config=var.vpc_config}: {}
content{
...
}
}
if no vpc_config is passed in the module then the input variable should default to something like an empty object {}, that way the dynamic conditional check will still work if no config is passed
Turns out it doesn't seem to be possible what I want to do (building an optional type safe configuration through an object definition without having to nest it in a list).
Instead I now use the lambda module provided by Terraform:
module "email_lambda" {
source = "terraform-aws-modules/lambda/aws"
version = "3.3.1"
function_name = "${var.stack_name}-email"
handler = "pkg.email.App::handleRequest"
runtime = "java11"
architectures = ["x86_64"]
memory_size = 512
timeout = 30
layers = [aws_lambda_layer_version.lambda_layer.arn]
create_package = false
local_existing_package = "../email/target/email.jar"
environment_variables = {
# https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-function-performance-for-java/
JAVA_TOOL_OPTIONS = "-XX:+TieredCompilation -XX:TieredStopAtLevel=1"
}
vpc_subnet_ids = module.vpc.private_subnets
vpc_security_group_ids = [aws_security_group.lambda_security_group.id]
attach_policies = true
policies = [
"arn:aws:iam::aws:policy/service-role/AWSLambdaSQSQueueExecutionRole",
]
number_of_policies = 1
attach_policy_json = true
policy_json = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "SESBulkTemplatedPolicy"
Effect = "Allow"
Resource = [...]
Action = [
"ses:SendEmail",
"ses:SendRawEmail",
"ses:SendTemplatedEmail",
"ses:SendBulkTemplatedEmail",
]
}
]
})
}
As one can see in this configuration I had to set the VPC parameters individually and in case of the policy I had to specify a boolean parameter to tell Terraform that the configuration was set (I even had to specify the length of the provided list). Looking at the source code of the module reveals that there may not be a better way how to achieve this in the most up to date version of Terraform.
There is resource:
resource "resource_name" "foo" {
name = "test"
config {
version = 14
resources {
disk_type_id = "network-ssd"
}
postgresql_config = {
enable_parallel_hash = true
}
}
}
I need a module which accepts optional user variables in "postgresql_config". There can be many such variables.
I tried next:
variables.tf
variable "postgresql_config" {
description = "User defined for postgresql_config"
type = list(object({
# key1 = value1
# ...
# key50 = value50
}))
}
variable "config" {
description = "for dynamic block 'config' "
type = list(object({
version = number
}))
default = [{
version = 14
}]
}
variable "resources" {
description = "for dynamic block 'resources' "
type = list(object({
disk_type_id = string
}))
default = [{
disk_type_id = "network-hdd"
}]
}
module/postgresql/main.tf
resource "resource_name" "foo" {
name = "test"
dynamic "config" {
for_each = var.config
content {
version = config.value["version"]
dynamic "resources" {
for_each = var.resources
content {
disk_type_id = resources.value["disk_type_id"]
}
}
# problem is here
postgresql_config = {
for_each = var.postgresql_config
each.key = each.value
}
}
}
example/main.tf
module "postgresql" {
source = "../module/postgresql"
postgresql_config = [{
auto_explain_log_buffers = true
log_error_verbosity = "LOG_ERROR_VERBOSITY_UNSPECIFIED"
max_connections = 395
vacuum_cleanup_index_scale_factor = 0.2
}]
That is, I understand that I need to use "dynamic", but it can only be applied to the block "config" and the nested block "resource_name".
How can I pass values for "postgresql_config" from main.tf to module? Of course, my example with for_each = var.postgresql_config doesn't work, but I hope this way to give an idea of what I need.
Or does terraform have no such option to use custom variables dynamically at all, and all of them must be specified explicitly?
Any help would be appreciated, thank you
from what I understand , you are trying to create a map dynamically for your resource postgres_config.
I would recommend using a for expression to solve that problem.
However, I think your problem lies in how you have defined variables for your module . You might run into a problem if your postgress_config list has multiple configs in it because that config can only take a map by the looks of it.
have a look at the following documentation:
this one is for how to define your variables
https://www.terraform.io/language/expressions/dynamic-blocks#multi-level-nested-block-structures
for expressions
https://www.terraform.io/language/expressions/for
my solution for your config problem ,would be something like this assuming that the postgres_config list has one element all the time:
# problem is here
postgresql_config = var.postgresql_config[0]
Let's say I have a map of environments to supply to for_each
environments = {
"0" = "dev"
"1" = "test"
"2" = "stage"
}
Then for whatever reason I want to create an Azure Resource Group for each environment.
resource "azurerm_resource_group" "resource_group" {
for_each = var.environments
name = "${var.resource-group-name}-${each.value}-rg"
location = var.location
}
How do I get the outputs? I've tried the new splat to no avail.
output "name" {
value = "${azurerm_resource_group.resource_group[*].name}"
}
output "id" {
value = "${azurerm_resource_group.resource_group[*].id}"
}
output "location" {
value = "${azurerm_resource_group.resource_group[*].location}"
}
Error: Unsupported attribute
in output "id":
6: value = "${azurerm_resource_group.resource_group[*].id}"
This object does not have an attribute named "id".
How do I output an attribute of multiple instances of a resource created with for_each?
The [*] is a shorthand for extracting attributes from a list of objects. for_each makes a resource appear as a map of objects instead, so the [*] operator is not appropriate.
However, for expressions are a more general mechanism that can turn either a list or a map into another list or map by evaluating an arbitrary expression for each element of the source collection.
Therefore we can simplify a map of azurerm_resource_group objects into a map of names of those objects like this:
output "name" {
value = { for k, group in azurerm_resource_group.resource_group: k => group.name }
}
Your input map uses numeric indexes as keys, which is unusual but allowed. Because of that, the resulting value for the output would be something like this:
{
"0" = "something-dev-rg"
"1" = "something-test-rg"
"2" = "something-stage-rg"
}
It's more common for a map in for_each to include a meaningful name as the key, so that the resulting instances are identified by that meaningful name rather than by incrementing integers. If you changed your configuration to use the environment name as the key instead, the map of names would look like this instead:
{
"dev" = "something-dev-rg"
"test" = "something-test-rg"
"stage" = "something-stage-rg"
}
EDIT: for_each doesn't work with output
output "name"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].name }
}
output "id"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].id }
}
output "location"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].location }
}
Example output,
id = {
"dev" = "xxx"
"stage" = "yyy"
"test" = "zzz"
}
I am using Terraform v12.19 with the aws provider v2.34.0.
Imagine, I have a resource generated with a count value:
resource "aws_iam_role" "role" {
count = length(var.somevariable)
name = var.somevariable[count.index]
}
Later on, I want to reference one specific resource instance in that way, e. g.:
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role["TheRoleNameIWant"].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
I don't know the index, I can just rely on the name, provided by the variable. Thats because the values of the variable are provided by an external source and the order could change...
Any ideas how to do this?
You should be able to accomplish this using the index terraform function.
Here's a minimal example using null_resources to test it out
locals {
role_names = [
"role-a",
"role-b",
"role-c",
"role-d",
]
target_role_name = "role-c"
}
resource "null_resource" "hi" {
count = length(local.role_names)
}
output "target_resource" {
value = null_resource.hi[index(local.role_names, local.target_role_name)].id
}
output "all_resources" {
value = [for r in null_resource.hi : r.id]
}
This outputs, for example
all_resources = [
"4350570701002192774",
"9173388682753384584",
"1634695740603384613",
"2098863759573339880",
]
target_resource = 1634695740603384613
So your example, I suppose, would look like
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role[index(var.somevariable, "TheRoleNameIWant")].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
Update
Your comment below mentions that you actually have a more complicated data structure than just a list of names. I just wanted to mention that you can derive names from your JSON structure.
Assuming you have something like the following
variable "role_values" {
value = [
{
name = "foo",
other = "details",
fields = 3
},
{
name = "bar",
other = "yet more details",
fields = 3
}
]
}
you could derive just the names by using a local and the newer for loops TF 0.12 offers
locals {
role_names = [for role in var.role_values: role.name]
}
That way you don't have to store the names twice.