I’m trying to write some code which would take an input structure like this:
projects = {
"project1" = {
namespaces = ["mynamespace1"]
},
"project2" = {
namespaces = ["mynamespace2", "mynamespace3"]
}
}
and provision multiple resources with for_each which would result in this:
resource "rancher2_project" "project1" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_project" "project2" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_namespace" "mynamespace1" {
provider = rancher2.admin
project_id = rancher2_project.project1.id
depends_on = [rancher2_project.project1]
}
resource "rancher2_namespace" "mynamespace2" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
resource "rancher2_namespace" "mynamespace3" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
namespaces are dependent on Projects and the generate id needs to be passed into namespace.
Is there any good way of doing this dynamically ? We might have a lot of Projects/namespaces.
Thanks for any help and advise.
The typical answer for systematically generating multiple instances of a resource based on a data structure is resource for_each. The main requirement for resource for_each is to have a map which contains one element per resource instance you want to create.
In your case it seems like you need one rancher2_project per project and then one rancher2_namespace for each pair of project and namespaces. Your current data structure is therefore already sufficient for the rancher2_project resource:
resource "rancher2_project" "example" {
for_each = var.projects
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
The above will declare two resource instances with the following addresses:
rancher2_project.example["project1"]
rancher2_project.example["project2"]
You don't currently have a map that has one element per namespace, so it will take some more work to derive a suitable value from your input data structure. A common pattern for this situation is flattening nested structures for for_each using the flatten function:
locals {
project_namespaces = flatten([
for pk, proj in var.projects : [
for nsk in proj.namespaces : {
project_key = pk
namespace_key = ns
project_id = rancher2_project.example[pk].id
}
]
])
}
resource "rancher2_namespace" "example" {
for_each = {
for obj in local.project_namespaces :
"${obj.project_key}.${obj.namespace_key}" => obj
}
provider = rancher2.admin
project_id = each.value.project_id
}
This produces a list of objects representing all of the project and namespace pairs, and then the for_each argument transforms it into a map using compound keys that include both the project and namespace keys to ensure that they will all be unique. The resulting instances will therefore have the following addresses:
rancher2_namespace.example["project1.mynamespace1"]
rancher2_namespace.example["project2.mynamespace2"]
rancher2_namespace.example["project2.mynamespace3"]
This seems to work too:
resource "rancher2_namespace" "example" {
count = length(local.project_namespaces)
provider = rancher2.admin
name = local.project_namespaces[count.index].namespace_name
project_id = local.project_namespaces[count.index].project_id
}
Related
I'm trying to create a module in Terraform for creating Azure resources and facing some issues. This module creates a resource group, subnet, vnet and Role bindings. I see that the below code creates the resources twice because of the loop. Does the for_each loop work in such a way that the entire resource or module block will be executed each time it loops? I'm new to Terraform and come from a Java background.
Also, ideally would like to use the flatten inside the module without locals possibly, any way to do that? Code is below.
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
}
]
])
}
module "testproject1" {
source = "C:\\Users\\ebo1h8h\\Documents\\Project\\Automation\\Terraform\\Code\\Azure\\modules\\sandbox-module"
short_name = "testproj"
# Resource Group Variables
az_rg_location = "eastus"
az_tags = {
Environment = "Sandbox"
CostCenter = "Department"
ResourceOwner = "Vikram"
Project = "testproj"
Role = "Resource Group"
}
address_space = ["10.0.0.0/16"]
subnet_prefixes = ["10.0.1.0/24"]
subnet_names = ["a-npe-snet01-sbox"]
vnet_location = var.az_rg_location
for_each = {
for group in local.groupsbyrole : "${group.role_name}.${group.group_name}}" => group
}
principal_id = each.value.group_name
role_definition_name = each.value.role_name
}
And here is the role_assignments variable
variable "testproject1_role_assignments" {
type = map(list(string))
default = {
"Contributor" = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
"Owner" = ["gcp-org-cloud-delivery"]
}
}
The above code creates 12 resources when it should be only 6. The only was I was able to get around this is have the resource "azurerm_role_assignment" "role_assignment" as a separate module. Ideally, I want to pass the role assignments variable in each of the module to be created so that it creates a set of resources.
Any pointers on how to achieve that?
Thanks,
The docs state
If a resource or module block includes a for_each argument whose value is a map or a set of strings, Terraform will create one instance for each member of that map or set.
So in your scenario you are creating 3 instances of the module, whereas it sounds like you want to pass in the local.groupsbyrole object as a variable in the module and only attach the for_each to the resources you want multiple instances of.
Sidenote: You could simplify your local by adding group like below:
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
group = "${roleName}.${groupName}"
}
]
])
}
Tip: I find adding an output to see the shape of the object whilst developing can also be useful
output "test_output" {
value = local.groupsbyrole
}
Then when you run plan you will see your object
test_output = [
+ {
+ group = "Contributor.prod-azure-contrib-sbox"
+ group_name = "prod-azure-contrib-sbox"
+ role_name = "Contributor"
},
+ {
+ group = "Contributor.gcp-org-network-engineering"
+ group_name = "gcp-org-network-engineering"
+ role_name = "Contributor"
},
+ {
+ group = "Owner.gcp-org-cloud-delivery"
+ group_name = "gcp-org-cloud-delivery"
+ role_name = "Owner"
},
]
Currently I have a powershell script that reads a yaml config file with all the objects I need created and creates a .tfvars file which contains all the variables, maps, lists of maps etc.
It would be something like the following:
global_tags = {
Provisioner = "Terraform"
}
resource_groups = {
myrg1 = {
location = "uksouth",
tags = {
ResourceType = "resourcegroup"
}
}
}
storage_accounts = {
mystorage1 = {
resource_group_name = "myrg1",
location = "uksouth",
account_tier = "Standard",
account_replication_type = "GRS",
tags = {
ResourceType = "storageaccount"
}
containers_list = [
{ name = "test_private_x", access_type = "private" },
{ name = "test_blob_x", access_type = "blob" },
{ name = "test_container_x", access_type = "container" }
]
}
The idea is to then pump each list of maps into each module to create the resources, e.g. main.tf would be just:
module "resourcegroup" {
source = "./modules/azure-resourcegroup"
resource_groups = var.resource_groups
global_tags = var.global_tags
}
module "storageaccount" {
source = "./modules/azure-storageaccount"
depends_on = [module.resourcegroup]
storage_accounts = var.storage_accounts
global_tags = var.global_tags
}
Also, an example of a simple module would be:
resource "azurerm_resource_group" "rg" {
for_each = var.resource_groups
name = each.key
location = each.value.location
tags = lookup(each.value,"tags",null) == null ? var.global_tags : merge(var.global_tags,each.value.tags)
}
The issue is that writing a complex module, say around storage account, isn't too bad if you are just feeding in all the params, but feeding in a list of maps and writing a module to read that list and create multiple flattened lists to perform say 15 different calls (to create containers, shares, network rules etc.) is very complex.
Obviously the reason I want to use for_each loops in the modules is so that my main.tf doesn't have to call the module multiple times with hard coded values for say 50 storage accounts.
Just wondering if I am missing an obvious way to create complicated multiples of each resource type ?
I appreciate I could do separate modules for containers, shares etc and break the complex maps down into simpler ones to pass to the additional modules, but I was trying to just have 1 storage account module that could handle anything and be fed by a complex list of maps so main.tf did not need editing, I could just control the config completely via a .tfvars file
Need your kind help. I am stuck with the following resource creation. Using Terraform v1.0.6
I need to create appropriate subnets dynamically in two VPCs
variables.tf
vpc_resource_networks = {
pnw-01 = [
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.72.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "wb-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
],
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.80.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "web-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
]
]
}
main.tf
locals {
wlb_net = element(keys(var.vpc_resource_networks), 0)
}
resource "aws_subnet" "wlb" {
count = length(module.aws_vpc_app_resource)
vpc_id = element(module.aws_vpc_app_resource.*.vpc_id, count.index)
dynamic "subnet_group" {
for_each = var.vpc_resource_networks[local.wlb_net][count.index]
content {
dynamic "subnet" {
for_each = subnet_group.value
content {
cidr_block = subnet.subnet_ip
availability_zone = element(var.azs, subnet.index)
tags = {
Name = subnet.subnet_name
}
}
}
}
}
I intend to create subnets dynamically which is var.vpc_resource_networks.pnw01[0] should be on one vpc and other index on another VPC.
The above block returns
dynamic “subnet_group” {
Blocks of type “subnet_group” are not expected here.
Please assist
Looking at the resource definition of aws_subnet, I can see that, as the error message suggests, there's no property for a "subnet_group".
There are several different resource types that are subnet groups though for different services; such as DMS, DocumentDB, DAX, Elasticache, MemoryDB, Neptune, RDS, and Redshift. Search the term "subnet_group" on the left panel within the provider page.
Perhaps an AWS expert can comment here but I believe you're trying to do two things in one motion here.
First you should create the subnets and define their ranges, then you should create subnet groups that need access to different subnets for a particular service.
Here's some more information on subnets and subnet groups.
I am learning terraform and trying to understand the for_each loop iteration in terraform.
I am iterating through a loop for creating RGs in Azure cloud and what I want to understand is the difference between accessing the value of an instance using . or [""].
So for example, below is my tfvar file:
resource_groups = {
resource_group_1 = {
name = "terraform-apply-1"
location = "eastus2"
tags = {
created_by = "vivek89#test.com"
}
},
resource_group_2 = {
name = "terraform-apply-2"
location = "eastus2"
tags = {
created_by = "vivek89#test.com"
}
},
resource_group_3 = {
name = "terraform-apply-3"
location = "eastus2"
tags = {
created_by = "vivek89#test.com"
contact_dl = "vivek89#test.com"
}
}
}
and below is my terraform main.tf file:
resource "azurerm_resource_group" "terraformRG" {
for_each = var.resource_groups
name = each.value.name
location = each.value.location
tags = each.value.tags
}
I am confused with the expression in for_each in RG creation block. Both the below codes works and create RGs:
name = each.value.name
name = each.value["name"]
I want to understand the difference between the two and which one is correct.
They are equivalent as explained in the docs:
Map/object attributes with names that are valid identifiers can also be accessed using the dot-separated attribute notation, like local.object.attrname. In cases where a map might contain arbitrary user-specified keys, we recommend using only the square-bracket index notation (local.map["keyname"]).
The main difference is that dot notation requires key attributes to be valid identifiers. In contrast, the square-bracket notation works with any identifiers.
I am using Terraform v12.19 with the aws provider v2.34.0.
Imagine, I have a resource generated with a count value:
resource "aws_iam_role" "role" {
count = length(var.somevariable)
name = var.somevariable[count.index]
}
Later on, I want to reference one specific resource instance in that way, e. g.:
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role["TheRoleNameIWant"].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
I don't know the index, I can just rely on the name, provided by the variable. Thats because the values of the variable are provided by an external source and the order could change...
Any ideas how to do this?
You should be able to accomplish this using the index terraform function.
Here's a minimal example using null_resources to test it out
locals {
role_names = [
"role-a",
"role-b",
"role-c",
"role-d",
]
target_role_name = "role-c"
}
resource "null_resource" "hi" {
count = length(local.role_names)
}
output "target_resource" {
value = null_resource.hi[index(local.role_names, local.target_role_name)].id
}
output "all_resources" {
value = [for r in null_resource.hi : r.id]
}
This outputs, for example
all_resources = [
"4350570701002192774",
"9173388682753384584",
"1634695740603384613",
"2098863759573339880",
]
target_resource = 1634695740603384613
So your example, I suppose, would look like
resource "aws_iam_role_policy_attachment" "polatt" {
role = aws_iam_role.role[index(var.somevariable, "TheRoleNameIWant")].id
policy_arn = "arn:aws:iam::aws:policy/..."
}
Update
Your comment below mentions that you actually have a more complicated data structure than just a list of names. I just wanted to mention that you can derive names from your JSON structure.
Assuming you have something like the following
variable "role_values" {
value = [
{
name = "foo",
other = "details",
fields = 3
},
{
name = "bar",
other = "yet more details",
fields = 3
}
]
}
you could derive just the names by using a local and the newer for loops TF 0.12 offers
locals {
role_names = [for role in var.role_values: role.name]
}
That way you don't have to store the names twice.