Nested Dynamic block on counted resource not working as expected - terraform

Need your kind help. I am stuck with the following resource creation. Using Terraform v1.0.6
I need to create appropriate subnets dynamically in two VPCs
variables.tf
vpc_resource_networks = {
pnw-01 = [
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.72.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "wb-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
],
[
{
subnet_name = "wb-01"
subnet_ip = "10.58.80.0/25"
description = "WEB01"
index = 0
},
{
subnet_name = "web-02"
subnet_ip = "10.58.72.128/25"
description = "WEB02"
index = 1
}
]
]
}
main.tf
locals {
wlb_net = element(keys(var.vpc_resource_networks), 0)
}
resource "aws_subnet" "wlb" {
count = length(module.aws_vpc_app_resource)
vpc_id = element(module.aws_vpc_app_resource.*.vpc_id, count.index)
dynamic "subnet_group" {
for_each = var.vpc_resource_networks[local.wlb_net][count.index]
content {
dynamic "subnet" {
for_each = subnet_group.value
content {
cidr_block = subnet.subnet_ip
availability_zone = element(var.azs, subnet.index)
tags = {
Name = subnet.subnet_name
}
}
}
}
}
I intend to create subnets dynamically which is var.vpc_resource_networks.pnw01[0] should be on one vpc and other index on another VPC.
The above block returns
dynamic “subnet_group” {
Blocks of type “subnet_group” are not expected here.
Please assist

Looking at the resource definition of aws_subnet, I can see that, as the error message suggests, there's no property for a "subnet_group".
There are several different resource types that are subnet groups though for different services; such as DMS, DocumentDB, DAX, Elasticache, MemoryDB, Neptune, RDS, and Redshift. Search the term "subnet_group" on the left panel within the provider page.
Perhaps an AWS expert can comment here but I believe you're trying to do two things in one motion here.
First you should create the subnets and define their ranges, then you should create subnet groups that need access to different subnets for a particular service.
Here's some more information on subnets and subnet groups.

Related

Creating subnets and assinging rout tables to them

I am new to terraform and I'm trying to create a VPC with multiple subnets and adding route tables to all those subnets in a for loop manner.
VPC: 10.207.0.0/16
There's number_of_subnets which will create subnets like this: 10.207.x.0/24
This code works fine:
variable "region" {
default = "us-east-1"
}
variable "availability_zone" {
default = "us-east-1a"
}
variable "cidr_block" {
default = "207"
}
variable "number_of_subnets" {
default = 5
}
provider "aws" {
region = var.region
}
resource "aws_vpc" "test_vpc" {
cidr_block = "10.${var.cidr_block}.0.0/16"
instance_tenancy = "default"
enable_dns_support = true
enable_dns_hostnames = true
tags = {
Name = "test_vpc_${var.cidr_block}"
}
}
resource "aws_subnet" "test_subnets" {
count = var.number_of_subnets
vpc_id = aws_vpc.test_vpc.id
cidr_block = "10.${var.cidr_block}.${count.index+1}.0/24" # start from x.x.1.0/24
availability_zone = var.availability_zone
map_public_ip_on_launch = false
tags = {
Name = "test_subnet_${var.cidr_block}_${count.index+1}"
}
}
Now if I try to add this code to the bottom of the same file (everything is in one file called main.tf) to get the subnets and add route table to each:
# get all subnet IDs
data "aws_subnets" "q_subnets" {
filter {
name = "vpc-id"
values = [aws_vpc.test_vpc.id]
}
}
# add route table to all subnets
resource "aws_route_table_association" "rt_assoc_subnet" {
depends_on = [aws_subnet.test_subnets]
for_each = toset(data.aws_subnets.q_subnets.ids)
subnet_id = each.value
route_table_id = aws_route_table.test_rt.id
}
and run terraform apply it will give this error:
invalid for_each argument...
The "for_each" value depends on resource attribute that cannot be deteremined until apply,...
which doesn't make scense. First create vpc, then subnet, then get all subnets...
I also tried depends_on and didn't help.
How would I write this to make it work?
Any help is appreciated.
UPDATE1:
I tried to use aws_subnet.test_subnets.*.id instead of data and it still gives depencendy error:
variable "region" {
default = "us-east-1"
}
variable "availability_zone" {
default = "us-east-1a"
}
variable "cidr_block" {
default = "207"
}
variable "number_of_subnets" {
default = 5
}
provider "aws" {
region = var.region
}
resource "aws_vpc" "test_vpc" {
cidr_block = "10.${var.cidr_block}.0.0/16"
instance_tenancy = "default"
enable_dns_support = true
enable_dns_hostnames = true
tags = {
Name = "test_vpc_${var.cidr_block}"
}
}
resource "aws_route_table" "test_rt" {
vpc_id = aws_vpc.test_vpc.id
route = []
tags = {
Name = "test_rt_${var.cidr_block}"
}
}
resource "aws_subnet" "test_subnets" {
count = var.number_of_subnets
vpc_id = aws_vpc.test_vpc.id
cidr_block = "10.${var.cidr_block}.${count.index+1}.0/24" # start from x.x.1.0/24
availability_zone = var.availability_zone
map_public_ip_on_launch = false
tags = {
Name = "test_subnet_${var.cidr_block}_${count.index+1}"
}
}
output "subnets" {
value = aws_subnet.test_subnets.*.id
}
# add route table to all subnets
resource "aws_route_table_association" "rt_assoc_subnet" {
depends_on = [aws_subnet.test_subnets]
for_each = toset(aws_subnet.test_subnets.*.id)
subnet_id = each.value
route_table_id = aws_route_table.test_rt.id
}
is there another way to pass the subnets to aws_route_table_association without getting dependency error?
Since you are using count, it is very hard to make count work with for_each. It would be better to continue using count for route table association as well. If you decide to go down that route, the only change you need is:
resource "aws_route_table_association" "rt_assoc_subnet" {
count = var.number_of_subnets
subnet_id = aws_subnet.test_subnets.*.id[count.index]
route_table_id = aws_route_table.test_rt.id
}
This will work as intended. However, if you must use for_each I would suggest defining a variable that could be used with it in all the resources you are now using count. If you really want to use for_each with the current code, then you can use the -target option [1]:
terraform apply -target=aws_vpc.test_vpc -target=aws_route_table.test_rt -target=aws_subnet.test_subnets
When running this command, this will be shown in the command output:
│ Warning: Resource targeting is in effect
│
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current
│ configuration.
│
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform
│ specifically suggests to use it as part of an error message.
After the targeted resources are created, you could re-run terraform apply and it should create the route table associations.
[1] https://www.terraform.io/cli/commands/plan#target-address

Terraform: Reference multiple resources without naming each of them

I am creating multiple subnets via a variable:
variable "private_subnets" {
description = "Private Subnets"
default = ["10.0.0.0/20", "10.0.32.0/20"]
}
resource "aws_subnet" "private" {
vpc_id = aws_vpc.main.id
cidr_block = element(var.private_subnets, count.index)
availability_zone = element(var.availability_zones, count.index)
count = length(var.private_subnets)
}
I count how many subnets I have listed in my Var and then create a subnet for each. The problem is so far I can only figure out how to reference them by each individual index:
subnets = [ aws_subnet.private[0].id, aws_subnet.private[1].id ]
What is the correct way to do this? I tried a similar element() and count section to the ECS network config where I'm referencing this, but it isn't working.
I am not sure if this is what you are after, but you can try:
subnets = aws_subnet.private[*].id
This syntax is called splat expressions

How can I get active address space of tagged Azure VNets inside Terraform?

I would like to get with Terraform the active address space of VNets for Azure in Terraform that have a certain tag. For this I thought I could use Resource data source for virtual networks:
data "azurerm_resources" "vnets"{
type = "Microsoft.Network/virtualNetworks"
required_tags = {
tag_name = "tag"
}
}
Then I realized that required attribute "address_space" belongs actually to the Virtual Networks Data Source (https://www.terraform.io/docs/providers/azurerm/r/virtual_network.html). Still I need to get the information about existing virtual networks from the Resources Data Source. So I tried nesting the data sources, but the following code does not work:
data "azurerm_virtual_network" "vnets"{
for_each = [for r in data.azurerm_resources.vnets.resources: {r.name = split("/", split("/resourceGroups", r.id))}]
name = each.key
resource_group_name = each.value
}
If that were to work, the idea would then be to determine the lowest possible VNet address space I can give to a new VNet based on the currently allocated VNet addresses (active_vnet_addresses) and a predefined address space (eligible_vnet_addresses) constrained by the number of resource groups (980=3*255+215) per subscription in Azure:
locals {
active_vnet_addresses = azurerm_virtual_network.vnets.address_space
eligible_vnet_addresses = concat([for s in range(1,255,1): "10.${s}.0.0/16"], [for s in range(1,255,1): "11.${s}.0.0/16"], [for s in range(1,255,1): "12.${s}.0.0/16"], [for s in range(1,215,1): "13.${s}.0.0/16"])
available_vnet_addresses = setsubtract(local.eligible_vnet_addresses, local.active_vnet_addresses)
available_vnet_numbers_sorted = sort([for a in local.available_vnet_addresses: split(".", a)[1]])
lowest_available_address_num = (length(local.available_vnet_numbers_sorted) == 0 ? "no more resource groups available" : local.available_vnet_numbers_sorted[0])
}
I am quite new to Terraform and this is my best effort, so I greatly appreciate suggestion on code improvements and would highly appreciate if someone could point me to a solution on how to get already active address spaces from Azure in Terraform.
To make the nesting data sources that you tried works, you need to change your code like this:
data "azurerm_virtual_network" "vnet" {
for_each = zipmap(flatten([for id in data.azurerm_resources.vnets.resources[*].id: element(split("/", id), 4)]), data.azurerm_resources.vnets.resources[*].name)
resource_group_name = each.key
name = each.value
}
These data sources only get a map of the existing VNet details. And each element of the map display like this:
"group_name" = {
"address_space" = [
"172.18.44.0/24",
]
"guid" = "7212a68b-94a0-4be9-b972-b1c61e6ec007"
"id" = "xxxxxxxxxxxxx"
"location" = "southcentralus"
"name" = "romungi-ml-studio-vnet"
"resource_group_name" = "group_name"
"subnets" = [
"default",
]
"vnet_peerings" = {}
}
So if you just want to get the address space of all the VNet, you need to continue to convert the data sources. To convert the details into a map and each member looks like this:
"VNet_name" = address_space
Here the code:
locals {
vnet_address_spaces = zipmap(values(data.azurerm_virtual_network.vnet)[*].name, values(data.azurerm_virtual_network.vnet)[*].address_space)
}
The whole map shows here:
vnets = {
"vnet1" = [
"172.18.1.0/24",
]
"vnet2" = [
"172.17.2.0/24",
]
"vnet3" = [
"172.18.44.0/24",
]
"vnet4" = [
"10.0.0.0/16",
]
}

how to conditionally create aws_eip and aws_eip_association?

I need to be able to conditionally create an EIP and associate it to an instance:
resource "aws_eip" "gateway" {
vpc = true
tags = {
Name = "${var.project_id}-gateway"
Project = "${var.project_id}"
user = "${var.user}"
}
}
resource "aws_eip_association" "eip_assoc_gateway" {
instance_id = aws_instance.gateway.id
allocation_id = aws_eip.gateway.id
}
resource "aws_instance" "gateway" {
...
}
Unfortunately, aws_eip and aws_eip_association don't appear to support the count attribute, so I'm not clear if this is even possible?
Any ideas?
As mentioned in comment, count is supported by all Terraform primitive resources. Example for aws_eip below:
resource "aws_eip" "eip" {
instance = "${element(aws_instance.manager.*.id,count.index)}"
count = "${var.eip_count}"
vpc = true
}

Terraform: How to use multiple locals and Variables inside "for_each"

I have a terraform template that creates multiple EC2 instances.
I then create a few Elastic Network interfaces in the AWS console and added them as locals in the terraform template.
Now, I want to map the appropriate ENI to the instance hence I added locals and variables as below.
locals {
instance_ami = {
A = "ami-11111"
B = "ami-22222"
C = "ami-33333"
D = "ami-4444"
}
}
variable "instance_eni" {
description = "Pre created Network Interfaces"
default = [
{
name = "A"
id = "eni-0a15890a6f567f487"
},
{
name = "B"
id = "eni-089a68a526af5775b"
},
{
name = "C"
id = "eni-09ec8ad891c8e9d91"
},
{
name = "D"
id = "eni-0fd5ca23d3af654a9"
}
]
}
resource "aws_instance" "instance" {
for_each = local.instance_ami
ami = each.value
instance_type = var.instance_type
key_name = var.keypair
root_block_device {
delete_on_termination = true
volume_size = 80
volume_type = "gp2"
}
dynamic "network_interface" {
for_each = [for eni in var.instance_eni : {
eni_id = eni.id
}]
content {
device_index = 0
network_interface_id = network_interface.value.eni_id
delete_on_termination = false
}
}
}
I am getting below error:
Error: Error launching source instance: InvalidParameterValue: Each network interface requires a
unique device index.
status code: 400, request id: 4a482753-bddc-4fc3-90f4-2f1c5e2472c7
I think terraform is tyring to attach all 4 ENI's to single instance only.
What should be done to attach ENI's to an individual instance?
The configuration you shared in your question is asking Terraform to manage four instances, each of which has four network interfaces associated with it. That's problematic in two different ways:
All for of the network interfaces on each instance are configured with the same device_index, which is invalid and is what the error message here is reporting.
Even if you were to fix that, it would then try to attach the same four network interfaces to four different EC2 instances, which is invalid: each network interface can be attached to only one instance at a time.
To address that and get the behavior you wanted, you only need one network_interface block, whose content is different for each of the instances:
locals {
instance_ami = {
A = "ami-11111"
B = "ami-22222"
C = "ami-33333"
D = "ami-4444"
}
}
variable "instance_eni" {
description = "Pre created Network Interfaces"
default = [
{
name = "A"
id = "eni-0a15890a6f567f487"
},
{
name = "B"
id = "eni-089a68a526af5775b"
},
{
name = "C"
id = "eni-09ec8ad891c8e9d91"
},
{
name = "D"
id = "eni-0fd5ca23d3af654a9"
}
]
}
locals {
# This expression is transforming the instance_eni
# value into a more convenient shape: a map from
# instance key to network interface id. You could
# also choose to just change directly the
# definition of variable "instance_eni" to already
# be such a map, but I did it this way to preserve
# your module interface as given.
instance_network_interfaces = {
for ni in var.instance_eni : ni.name => ni.id
}
}
resource "aws_instance" "instance" {
for_each = local.instance_ami
ami = each.value
instance_type = var.instance_type
key_name = var.keypair
root_block_device {
delete_on_termination = true
volume_size = 80
volume_type = "gp2"
}
network_interface {
device_index = 0
network_interface_id = local.instance_network_interfaces[each.key]
delete_on_termination = false
}
}
Now each instance has only one network interface, with each one attaching to the corresponding ENI ID given in your input variable. Referring to each.key and each.value is how we can create differences between each of the instances declared when using resource for_each; we don't need any other repetition constructs inside unless we want to create nested repetitions, like having a dynamic number of network interfaces for each instance.

Resources