Get resources based on a value created using count - terraform

I am using Terraform v12.19 with the aws provider v2.34.0.
Imagine, I have a resource generated with a count value:
resource "aws_iam_role" "role" {
count = length(var.somevariable)
name = var.somevariable[count.index]
}
Later on, I want to reference one specific resource instance in that way, e. g.:
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role["TheRoleNameIWant"].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
I don't know the index, I can just rely on the name, provided by the variable. Thats because the values of the variable are provided by an external source and the order could change...
Any ideas how to do this?

You should be able to accomplish this using the index terraform function.
Here's a minimal example using null_resources to test it out
locals {
role_names = [
"role-a",
"role-b",
"role-c",
"role-d",
]
target_role_name = "role-c"
}
resource "null_resource" "hi" {
count = length(local.role_names)
}
output "target_resource" {
value = null_resource.hi[index(local.role_names, local.target_role_name)].id
}
output "all_resources" {
value = [for r in null_resource.hi : r.id]
}
This outputs, for example
all_resources = [
"4350570701002192774",
"9173388682753384584",
"1634695740603384613",
"2098863759573339880",
]
target_resource = 1634695740603384613
So your example, I suppose, would look like
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role[index(var.somevariable, "TheRoleNameIWant")].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
Update
Your comment below mentions that you actually have a more complicated data structure than just a list of names. I just wanted to mention that you can derive names from your JSON structure.
Assuming you have something like the following
variable "role_values" {
value = [
{
name = "foo",
other = "details",
fields = 3
},
{
name = "bar",
other = "yet more details",
fields = 3
}
]
}
you could derive just the names by using a local and the newer for loops TF 0.12 offers
locals {
role_names = [for role in var.role_values: role.name]
}
That way you don't have to store the names twice.

Related

terraform - use count and for_each together

I'm creating multiple Google Cloud Projects using a module which creates a custom service account for each one and provides that as an output eg
module "app_projects" {
count = var.number_application_projects
etc
}
I then have a list of IAM roles I want to assign to each Project Service account like this:
sa_roles = ["roles/run.developer","roles/appengine.deployer"]
resource "google_project_iam_member" "proj_sa_iam_roles" {
count = var.number_application_projects
for_each = toset(var.sa_roles)
project = module.app_projects[count.index].project_id
role = each.value
member = "serviceAccount:${module.app_projects[count.index].service_account_email}"
}
This runs into the error about "count" and "for_each" being mutually-exclusive, and I can't for the life of me figure out what to use instead, any help would be appreciated!
You can probably use setproduct, but the standard method is using flatten.
They even have a section tailored for similar use cases. Flattening nested structures for for_each.
Here is an example, that doesn't use your exact resource, but should be instructive and you can actually run it to test it out.
modules/app-project/variables.tf
variable "name" {}
modules/app-project/outputs.tf
output "name" {
value = var.name
}
modules/member/variables.tf
variable "project" {}
variable "role" {}
modules/member/outputs.tf
output "project_role" {
value = "${var.project}-${var.role}"
}
main.tf
locals {
roles = ["rad", "tubular"]
}
module "app_project" {
source = "./modules/app-project"
count = 2
name = "app-project-${count.index}"
}
module "project_role" {
source = "./modules/member"
for_each = { for pr in flatten([for p in module.app_project[*] : [
for r in local.roles : {
app_project_name = p.name
role = r
}]
]) : "${pr.app_project_name}-${pr.role}" => pr }
project = each.value.app_project_name
role = each.value.role
}
output "project_roles" {
value = values(module.project_role)[*].project_role
}
terraform plan output
Changes to Outputs:
+ project_roles = [
+ "app-project-0-rad",
+ "app-project-0-tubular",
+ "app-project-1-rad",
+ "app-project-1-tubular",
]
In your case specifically, I think something like this would work:
resource "google_project_iam_member" "proj_sa_iam_roles" {
for_each = { for i, pr in flatten([for p in module.app_project[*] : [
for r in var.sa_roles : {
app_project = p
role = r
}]
]) : "${i}-${pr.role}" => pr }
project = each.value.app_project.project_id
role = each.value.role
member = "serviceAccount:${each.value.app_project.service_account_email}"
}

Terraform Invalid for_each argument local will be known only after apply

I would like to create an AWS account with SSO Account Assignments in the same first terraform run without hit the for_each limitation with dynamic values that cannot be predicted during plan.
I've tried to separate the aws_organizations_account resource from aws_ssoadmin_account_assignment in completely separate TF module and also I tried to use depends_on between those resources and modules.
What is the simplest and correct way to fix this issue?
Terraform v1.2.4
AWS SSO Account Assignments Module
Closed Pull Request that did not fix this issue
main.tf file (aws module)
resource "aws_organizations_account" "account" {
name = var.aws_account_name
email = "${var.aws_account_name}#gmail.com"
tags = {
Name = var.aws_account_name
}
parent_id = var.aws_org_folder_id
}
data "aws_identitystore_group" "this" {
for_each = local.group_list
identity_store_id = local.identity_store_id
filter {
attribute_path = "DisplayName"
attribute_value = each.key
}
}
data "aws_identitystore_user" "this" {
for_each = local.user_list
identity_store_id = local.identity_store_id
filter {
attribute_path = "UserName"
attribute_value = each.key
}
}
data "aws_ssoadmin_instances" "this" {}
locals {
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.id, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
identity_store_id = tolist(data.aws_ssoadmin_instances.this.identity_store_ids)[0]
sso_instance_arn = tolist(data.aws_ssoadmin_instances.this.arns)[0]
group_list = toset([for mapping in var.account_assignments : mapping.principal_name if mapping.principal_type == "GROUP"])
user_list = toset([for mapping in var.account_assignments : mapping.principal_name if mapping.principal_type == "USER"])
}
resource "aws_ssoadmin_account_assignment" "this" {
for_each = local.assignment_map
instance_arn = local.sso_instance_arn
permission_set_arn = each.value.permission_set_arn
principal_id = each.value.principal_type == "GROUP" ? data.aws_identitystore_group.this[each.value.principal_name].id : data.aws_identitystore_user.this[each.value.principal_name].id
principal_type = each.value.principal_type
target_id = aws_organizations_account.account.id
target_type = "AWS_ACCOUNT"
}
main.tf (root)
module "sso_account_assignments" {
source = "./modules/aws"
account_assignments = [
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Administrators"
},
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-955c264e8f20fea3",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Developers"
},
{
permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66",
permission_set_name = "ReadOnlyAccess",
principal_type = "GROUP",
principal_name = "Developers"
},
]
}
The important thing about a map for for_each is that all of the keys must be made only of values that Terraform can "see" during the planning step.
You defined local.assignment_map this way in your example:
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.id, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
I'm not personally familiar with the aws_organizations_account resource type, but I'm guessing that aws_organizations_account.account.id is an attribute whose value gets decided by the remote system during the apply step (once the object is created) and so this isn't a suitable value to use as part of a for_each map key.
If so, I think the best path forward here is to use a different attribute of the resource that is defined statically in your configuration. If var.aws_account_name has a static value defined in your configuration (that is, it isn't derived from an apply-time attribute of another resource) then it might work to use the name attribute instead of the id attribute:
assignment_map = {
for a in var.account_assignments :
format("%v-%v-%v-%v", aws_organizations_account.account.name, substr(a.principal_type, 0, 1), a.principal_name, a.permission_set_name) => a
}
Another option would be to remove the organization reference from the key altogether. From what you've shared it seems like there is only one account and so all of these keys would end up starting with exactly the same account name anyway, and so that string isn't contributing to the uniqueness of those keys. If that's true then you could drop that part of the key and just keep the other parts as the unique key:
assignment_map = {
for a in var.account_assignments :
format(
"%v-%v-%v",
substr(a.principal_type, 0, 1),
a.principal_name,
a.permission_set_name,
) => a
}

Writing DRY code with terraform for loop?

I am very new to terraform and had a task dropped upon me to create 2 AWS KMS keys.
So I am doing this:
resource "aws_kms_key" "ebs_encryption_key" {
description = "EBS encryption key"
... omitted for brevity ...
tags = merge(map(
"Name", format("%s-ebs-encryption-key", var.name_prefix),
"component", "kms",
"dataclassification","low",
), var.extra_tags)
}
resource "aws_kms_alias" "ebs_encryption_key" {
name = format("alias/%s-ebs-encryption-key", var.name_prefix)
target_key_id = aws_kms_key.ebs_encryption_key.key_id
}
# Repeated code!
resource "aws_kms_key" "rds_encryption_key" {
description = "RDS encryption key"
... omitted for brevity ...
tags = merge(map(
"Name", format("%s-rds-encryption-key", var.name_prefix),
"component", "kms",
"dataclassification","low",
), var.extra_tags)
}
resource "aws_kms_alias" "rds_encryption_key" {
name = format("alias/%s-rds-encryption-key", var.name_prefix)
target_key_id = "${aws_kms_key.rds_encryption_key.key_id}"
}
As you can see the only difference between the two blocks of code is "ebs" and "rds"?
How could I use a for loop to avoid repeating the code blocks?
This seems like it could be a candidate for a small module that encapsulates the details of declaring a key and an associated alias, since a key and an alias are typically declared together in your system.
The module itself would look something like this:
variable "name" {
type = string
}
variable "description" {
type = string
}
variable "tags" {
type = map(string)
}
resource "aws_kms_key" "main" {
description = var.description
# ...
tags = var.tags
}
resource "aws_kms_alias" "main" {
name = "alias/${var.name}"
target_key_id = aws_kms_key.main.key_id
}
output "key_id" {
value = aws_kms_key.main.key_id
}
output "alias_name" {
value = aws_kms_alias.main.name
}
(As written here this module feels a little silly because there's not really much here that isn't derived only from the variables, but I'm assuming that the interesting stuff you want to avoid repeating is in "omitted for brevity" in your example, which would go in place of # ... in my example.)
Your calling module can then include a module block that uses for_each to create two instances of the module, systematically setting the arguments to populate its input variables:
module "kms_key" {
for_each = {
kms = "KMS"
ebs = "EBS"
}
name = "${var.name_prefix}-${each.key}-encryption-key"
description = "${each.value} Encryption Key"
tags = merge(
var.extra_tags,
{
Name = "${var.name_prefix}-${each.key}-encryption-key"
component = "kms"
dataclassification = "low"
},
)
}
Since the for_each map here has the keys kms and ebs, the result of this will be to declare resource instances which should have the following addresses in the plan:
module.kms_key["kms"].aws_kms_key.main
module.kms_key["kms"].aws_kms_alias.main
module.kms_key["ebs"].aws_kms_key.main
module.kms_key["ebs"].aws_kms_alias.main
Since they are identified by the map keys, you can add new keys to that map in future to create new key/alias pairs without disturbing the existing ones.
If you need to use the key IDs or alias names elsewhere in your calling module then you can access them via the outputs exposed in module.kms_key elsewhere in that calling module:
module.kms_key["kms"].key_id
module.kms_key["kms"].alias_name
module.kms_key["ebs"].key_id
module.kms_key["ebs"].alias_name

terraform - Iterate over two linked resources

I’m trying to write some code which would take an input structure like this:
projects = {
"project1" = {
namespaces = ["mynamespace1"]
},
"project2" = {
namespaces = ["mynamespace2", "mynamespace3"]
}
}
and provision multiple resources with for_each which would result in this:
resource "rancher2_project" "project1" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_project" "project2" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_namespace" "mynamespace1" {
provider = rancher2.admin
project_id = rancher2_project.project1.id
depends_on = [rancher2_project.project1]
}
resource "rancher2_namespace" "mynamespace2" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
resource "rancher2_namespace" "mynamespace3" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
namespaces are dependent on Projects and the generate id needs to be passed into namespace.
Is there any good way of doing this dynamically ? We might have a lot of Projects/namespaces.
Thanks for any help and advise.
The typical answer for systematically generating multiple instances of a resource based on a data structure is resource for_each. The main requirement for resource for_each is to have a map which contains one element per resource instance you want to create.
In your case it seems like you need one rancher2_project per project and then one rancher2_namespace for each pair of project and namespaces. Your current data structure is therefore already sufficient for the rancher2_project resource:
resource "rancher2_project" "example" {
for_each = var.projects
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
The above will declare two resource instances with the following addresses:
rancher2_project.example["project1"]
rancher2_project.example["project2"]
You don't currently have a map that has one element per namespace, so it will take some more work to derive a suitable value from your input data structure. A common pattern for this situation is flattening nested structures for for_each using the flatten function:
locals {
project_namespaces = flatten([
for pk, proj in var.projects : [
for nsk in proj.namespaces : {
project_key = pk
namespace_key = ns
project_id = rancher2_project.example[pk].id
}
]
])
}
resource "rancher2_namespace" "example" {
for_each = {
for obj in local.project_namespaces :
"${obj.project_key}.${obj.namespace_key}" => obj
}
provider = rancher2.admin
project_id = each.value.project_id
}
This produces a list of objects representing all of the project and namespace pairs, and then the for_each argument transforms it into a map using compound keys that include both the project and namespace keys to ensure that they will all be unique. The resulting instances will therefore have the following addresses:
rancher2_namespace.example["project1.mynamespace1"]
rancher2_namespace.example["project2.mynamespace2"]
rancher2_namespace.example["project2.mynamespace3"]
This seems to work too:
resource "rancher2_namespace" "example" {
count = length(local.project_namespaces)
provider = rancher2.admin
name = local.project_namespaces[count.index].namespace_name
project_id = local.project_namespaces[count.index].project_id
}

Terraform azure-remove subcription details from output

I declared security group in following way:
resource "azurerm_network_security_group" "wan" {
count = "${var.enable_wan_subnet ? 1 : 0}"
provider = "azurerm.base"
name = "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}"
location = "${azurerm_resource_group.this.location}"
resource_group_name = "${azurerm_resource_group.this.name}"
tags = "${
merge(map("Name", format("%s-%s-%s",var.environment_name,"WAN-Subnets", "Security-Group")),
var.tags_global,
var.tags_module)
}"
}
and created output for that security group:
output "security_groups_id_wan" {
value = "${azurerm_network_security_group.wan.*.id}"
depends_on = [
"azurerm_subnet.wan",
]
}
In output i'm getting
Actual output
security_groups_id_wan = [
/subscriptions/111-222-333-4445/resourceGroups/default_resource_group/providers/Microsoft.Network/networkSecurityGroups/DF-DTAP-WAN-Subnet-Security-Group
]
How, from output, to remove all except resource name (DF-DTAP-WAN-Subnet-Security-Group)
Desired output:
security_groups_id_wan = [
DF-DTAP-WAN-Subnet-Security-Group
]
You can just use the Terraform functions and change the output value like this:
output "security_groups_id_wan" {
value = "${slice(split("/",azurerm_network_security_group.wan.*.id), length(split("/",azurerm_network_security_group.wan.*.id))-1, length(split("/",azurerm_network_security_group.wan.*.id)))}"
depends_on = [
"azurerm_subnet.wan",
]
}
With the functions, you can output every resource as you need. For more details, see Terraform Supported built-in functions.
Update
The test with an existing NSG through the Terraform data and the template here:
data "azurerm_network_security_group" "test" {
name = "azureUbuntu18-nsg"
resource_group_name = "charles"
}
output "substring" {
value = "${slice(split("/",data.azurerm_network_security_group.test.id), length(split("/",data.azurerm_network_security_group.test.id))-1, length(split("/",data.azurerm_network_security_group.test.id)))}"
}
The screenshot of the result here:
You built that name yourself with "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}" so why not just output that?
To save repeating yourself you could put that in a local and refer to it in both the resource and the output:
locals {
security_group_name = "${format("%s-%s", var.environment_name, "WAN-Subnet-Security-Group")}"
}
resource "azurerm_network_security_group" "wan" {
count = "${var.enable_wan_subnet ? 1 : 0}"
provider = "azurerm.base"
name = "${local.security_group_name}"
# ...
}
output "security_groups_id_wan" {
value = "${local.security_group_name}"
}
Note that you also didn't need the depends_on because a) it's an output, it happens at the end of things anyway and b) you already have an implicit dependency on that resource because you used an interpolation that included the resource.
You can read more about Terraform dependencies via the Hashicorp Learn platform.
Addition to #Charles Xu's answer:Had to convert list to string first
output "subnets_id_wan" {
value = "${slice(split("/",join(",",azurerm_subnet.wan.*.id)), length(split("/",join(",",azurerm_subnet.wan.*.id)))-1, length(split("/",join(",",azurerm_subnet.wan.*.id))))}"
depends_on = [
"azurerm_subnet.wan",
]
}

Resources