Looping in terraform with multi-level Maps - azure

I have a terraform question. I want to create a vnet with multiple subnets by reading in all the variable information in from a yaml file into locals. For instance:
vnets:
vnet1:
name: "core-euw"
location: "westeurope"
address_space: ["10.0.0.0/22"]
subnets:
- name: "adds"
address_prefixes: "10.0.0.64/27"
- name: "mgmt"
address_prefixes: "10.0.0.96/27"
vnet2:
name: "core-ussc"
location: "westeurope"
address_space: ["10.1.0.0/22"]
subnets:
- name: "adds"
address_prefixes: "10.1.0.64/27"
- name: "mgmt"
address_prefixes: "10.1.0.96/27"
In my .tf file, I want to create only one resource block for azurerm_virtual_network and loop on the number of vnets (vnet1, vnet2) in the yaml file. The code I have works great for creating the two vnets, but I can't seem to figure out how to parse the subnets: section under each vnet for creating the multiple subnets within it. My .tf file is below:
provider "azurerm" {
features {}
}
locals {
settings = yamldecode(file("./settings.yaml"))
}
resource "azurerm_resource_group" "rg-network" {
name = "rg-global-core-network"
location = "South Central US"
}
resource "azurerm_virtual_network" "vnets" {
for_each = local.settings.vnets
name = "vnet-${each.value["name"]}"
resource_group_name = azurerm_resource_group.rg-network.name
location = each.value["location"]
address_space = each.value["address_space"]
dynamic "subnet" {
for_each = local.settings.vnets.subnets
content {
name = each.value["name"]
address_prefix = each.value["address_prefixes"]
}
}
}
output "name" {
value = [for subnet in local.settings.vnets]
}

In your dynamic subnet block you should not use each. Instead it should be subnet:
dynamic "subnet" {
for_each = each.subnets
content {
name = subnet.value["name"]
address_prefix = subnet.value["address_prefixes"]
}
}

Related

Iterating over multi-level yaml values in terraform

I'm trying to shorten my terraform code for deploying azure vnets by iterating over values I provide in a yaml file. I want to write one .tf file with the code for the vnets, the subnets, the NSGs, etc. but I'm struggling to get the locals block right to correctly iterate through my yaml file (see below)
vnets:
- name: adds
location: eastus
address_space: ["10.1.0.0/24"]
subnets:
- name: adds
address_prefix: "10.1.0.0/27"
- name: dns
location: eastus
address_space: ["10.1.53.0/24"]
subnets:
- name: dns-inbound
address_prefix: "10.1.53.0/28"
- name: dns-outbound
address_prefix: "10.1.53.16/28"
Any help on how I should right my locals block would be appreciated!
This code will transform your yaml file into local map:
locals {
vnets = yamldecode(file("./test.yaml"))
vnets_map = {
for vnet in local.vnets.vnets :
vnet.name => {
address_space = vnet.address_space
location = vnet.location
subnets = {
for subnet in vnet.subnets :
subnet.name => subnet.address_prefix
}
}
}
}
output "example-output-dns-inbound-subnet" {
value = local.vnets_map.dns.subnets.dns-inbound
}
I took the liberty of changing lists to maps - it is better to navigate in terraform. Entire vnets_map object looks like this:
{
"adds" = {
"address_space" = [
"10.1.0.0/24",
]
"location" = "eastus"
"subnets" = {
"adds" = "10.1.0.0/27"
}
}
"dns" = {
"address_space" = [
"10.1.53.0/24",
]
"location" = "eastus"
"subnets" = {
"dns-inbound" = "10.1.53.0/28"
"dns-outbound" = "10.1.53.16/28"
}
}
}

Subnet is showing as destroyed in terraform destroy but it is not getting removed from azure portal

I have written a module called network_resources in which I am creating vnets and subnets attached to it. Below is the code for the module and how the module is being called:
main.tf
resource "azurerm_virtual_network" "vnets" {
for_each = var.vnets
name = each.key
resource_group_name = var.resource_group_name
location = var.location
address_space = [each.value.address_space]
dns_servers = each.value.dns_servers
}
resource "azurerm_subnet" "subnets" {
for_each = local.subnets
name = each.value.subnet_name
resource_group_name = var.resource_group_name
virtual_network_name = azurerm_virtual_network.vnets[each.value.vnet_name].name
address_prefixes = [each.value.subnet_address]
service_endpoints = each.value.service_endpoints
}
local.tf
locals {
subnets_flatlist = flatten([for key, val in var.vnets : [
for subnet in val.subnets : {
vnet_name = key
subnet_name = subnet.subnet_name
subnet_address = subnet.subnet_address
service_endpoints = subnet.service_endpoints
}
]
])
subnets = { for subnet in local.subnets_flatlist : subnet.subnet_address => subnet }
}
variables.tf
variable "resource_group_name" {
description = "Name of the resource group to be imported."
type = string
}
variable "location" {
description = "The location of the vnet to create. Defaults to the location of the resource group."
type = string
default = null
}
variable "vnets" {
type = map(object({
address_space = string
dns_servers = list(string)
subnets = list(object({
subnet_name = string
subnet_address = string
service_endpoints = list(string)
}))
}))
}
Code to call the module:
module "network_aks_prod1" {
source = "./network_resources_dns"
vnets = var.vnets_aks_prod1
resource_group_name = azurerm_resource_group.rg2.name
location = azurerm_resource_group.rg2.location
}
Variables.tf
vnets_aks_prod1 = {
"bupaanz-mel-prod-caas-vnet01" = {
address_space = "10.80.0.0/18"
#dns_servers = ["10.0.0.4" , "10.0.0.5"]
dns_servers = ["10.0.0.6","10.0.0.5", "10.0.0.6", "10.0.0.4","10.64.150.11"]
subnets = [
{
subnet_name = "subnet-mel-prod-aks-mgmt-10.80.9.64"
subnet_address = "10.80.9.64/26"
service_endpoints = []
},
{
subnet_name = "subnet-mel-prod-aks-internal1-10.80.0.0"
subnet_address = "10.80.0.0/22"
service_endpoints = []
},
{
subnet_name = "GatewaySubnet"
subnet_address = "10.80.9.0/26"
service_endpoints = []
},
{
subnet_name = "subnet-mel-prod-aks-internal2-10.80.4.0"
subnet_address = "10.80.4.0/22"
service_endpoints = []
},
{
subnet_name = "subnet-mel-prod-aks-pa1-ext-10.80.8.0"
subnet_address = "10.80.8.0/25"
service_endpoints = []
},
{
subnet_name = "subnet-mel-prod-aks-pa2-int-10.80.8.128"
subnet_address = "10.80.8.128/25"
service_endpoints = []
},
]
},
}
All the vnets and subnets get created successfully with the above code.
Now if I have to delete one subnet, i am removing the code for one of the subnet with all its attributes.
Now when I run the terraform plan and apply again, it informs that one subnet will be deleted. It is showing in terraform apply also the subnet has got deleted but it is not getting deleted in the portal. Please can you let me know how the subnet will get deleted in the portal with the terraform code only using the code which I am using.
Update as of 18th May 2022
Following further triage, it appears that azurerm_virtual_network resource is being modified because of a change of tags/edge_zone/bgp_community/flow_timeout_in_minutes attributes.
While deleting a subnet, terraform plan shows it is going to modify above attributes but under the hoods, it is updating the azurerm_virtual_network resource from the state file which includes the subnet azurerm_subnet just deleted.
Workaround to get it working with azurerm_virtual_network is to add lifecycle block as shown below..
resource "azurerm_virtual_network" "vnets" {
for_each = var.vnets
name = each.key
resource_group_name = var.resource_group_name
location = var.location
address_space = [each.value.address_space]
dns_servers = each.value.dns_servers
lifecycle {
ignore_changes = [
# one or more of the below attributes
tags, edge_zone, bgp_community, flow_timeout_in_minutes
]
}
}
TBH, I'm not sure of the side-effects of the above attributes by including them into lifecycle block but it doesn't update the vnet resources when these gets changed.
However this is a bug on Terraform provider, you must create an issue there to get this fixed.
You are unable to delete the subnet because you haven't set subnet = [] in the azurerm_virtual_network resource. Below is what the documentation here says::
Since subnet can be configured both inline and via the separate
azurerm_subnet resource, we have to explicitly set it to empty slice
([]) to remove it.
Also, the doc says below::
Terraform currently provides both a standalone Subnet resource, and
allows for Subnets to be defined in-line within the Virtual Network
resource. At this time you cannot use a Virtual Network with in-line
Subnets in conjunction with any Subnet resources. Doing so will cause
a conflict of Subnet configurations and will overwrite subnets.
Nevertheless, applying an empty slice gave even worse results as it wiped off all subnets. I think it's better you use dynamic-blocks to create subnets as part of vnet resource rather than using subnet specific resource.
subnet {
name = "subnet-mel-prod-aks-pa1-ext-10.80.8.0"
address_prefix = "10.80.8.0/25"
}
/*
subnet {
name = "subnet-mel-prod-aks-pa2-int-10.80.8.128"
address_prefix = "10.80.8.128/25"
}
*/
...
From the above snippet, when I commented on the subnet, it did delete the only subnet I commented out. I'm not sure whether there is a bug to fix azurerm_subnet as its behavior is very weird.

Declare Terraform Variable

Cloud Provider : Azure
Can I declare resource group , Virtual network , or subnets in Azure as terraform Variable - Infrastructure as code
without variable :-
resource "azurerm_resource_group" "example" {
name = "example"
location = var.location
}
Yes you can
resource "azurerm_resource_group" "example" {
name = "${var.prefix}-resources"
location = "westeurope"
}

Azure Databricks workspace using terraform

Trying to create Databricks workspace using terraform but unsupported arguments:
resource "azurerm_databricks_workspace" "workspace" {
name = "testdata"
resource_group_name = "cloud-terraform"
location = "east us"
sku = "premium"
virtual_network_id = azurerm_virtual_network.vnet.id
public_subnet_name = "databrickpublicsubnet"
public_subnet_cidr = "10.0.0.0/22"
private_subnet_name = "databrickprivatesubnet"
private_subnet_cidr = "10.0.0.0/22"
tags = {
Environment = "terraformtest"
}
}
Error: An argument named "virtual_network_id" is not expected here. An argument named "public_subnet_name" is not expected here. An argument named "public_subnet_cidr" is not expected here.
I haven't tried to set up databricks via Terraform, but I believe (per the docs) you need add those properties in a block:
resource "azurerm_databricks_workspace" "workspace" {
name = "testdata"
resource_group_name = "cloud-terraform"
location = "east us"
sku = "premium"
custom_parameters {
virtual_network_id = azurerm_virtual_network.vnet.id
public_subnet_name = "databrickpublicsubnet"
private_subnet_name = "databrickprivatesubnet"
}
tags = {
Environment = "terraformtest"
}
}
The two cidr entries aren't part of the TF documentation.
true. you can add terraform commands to create the subnets (assuming vnet already exists, you can use data azurerm_virtual_network then create the two new subnets, then reference the names of the two new public/private subnets.
Then you run into what seems to be a chicken/egg issue though.
You get Error: you must define a value for 'public_subnet_network_security_group_association_id' if 'public_subnet_name' is set.
Problem is, the network security group is typically auto-generated on creation of the databrick workspace (like databricksnsgrandomstring), which works when creating it in the portal, but via terraform, I have to define it to create the workspace, but it doesn't yet exist until I create the workspace. The fix is to not let it generate it's own nsg name, but name it yourself with an nsg resource block.
below is code I use (dbname means databricks name!). here I'm
adding to an existing resource group 'qa' and existing vnet as well, only showing the public subnet and nsg association, you can easily add the private ones). just copy/modify in your own tf file(s). and you'll definitely need to change the address_prefix to your own CIDR values that works within your vnet and not stomp on existing subnets within.
resource "azurerm_subnet" "public" {
name = "${var.dbname}-public-subnet"
resource_group_name = data.azurerm_resource_group.qa.name
virtual_network_name = data.azurerm_virtual_network.vnet.name
address_prefixes = ["1.2.3.4/24"]
delegation {
name = "databricks_public"
service_delegation {
name = "Microsoft.Databricks/workspaces"
}
}
}
resource "azurerm_network_security_group" "nsg" {
name = "${var.dbname}-qa-databricks-nsg"
resource_group_name = data.azurerm_resource_group.qa.name
location= data.azurerm_resource_group.qa.location
}
resource "azurerm_subnet_network_security_group_association" "nsga_public" {
network_security_group_id = azurerm_network_security_group.nsg.id
subnet_id = azurerm_subnet.public.id
}
Then in your databricks_workspace block, replace your custom parameters with
custom_parameters {
public_subnet_name = azurerm_subnet.public.name
public_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.nsga_public.id
private_subnet_name = azurerm_subnet.private.name
private_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.nsga_private.id
virtual_network_id = data.azurerm_virtual_network.vnet.id
}

Use the existing Subnet details instead of creating again while creating Network Interface in Azure using terraform

I'm trying to create network Interface in Azure through terraform using below script :
resource "azurerm_subnet" "internal" {
name = "Subnet1"
resource_group_name = "${var.VNetResourceGroup}"
virtual_network_name = "${var.VNetName}"
address_prefix = "10.0.2.0/24"
}
resource "azurerm_network_interface" "main" {
name = "${var.prefix}-nic"
location = "${var.location}"
resource_group_name = "${var.resourceGroup}"
ip_configuration {
name = "ipconfig1"
subnet_id = "${azurerm_subnet.internal.id}"
private_ip_address_allocation = "dynamic"
}
}
This script creates the Subnet Subnet1 and assigning that Subnet.id in ipconfiguration.
But if I have to create another network interface again through another .tf file with the same Subnet1 , how to do I get the ${azurerm_subnet.internal.id} value again.
That is if the Subnet is already existing and I do not want to create it, how to I set those values and use them?
EDIT
I have figured out, in powershell this is the script used to determine Subnet id :
$vnetId= "[resourceId("VNetRG",'Microsoft.Network/virtualNetworks', "VNetName")]"
$subnetRef = "[concat($vnetId, '/subnets/', "Subnet1")]"
where VNetRG - resource group of VNet ,
VNetName - Name of VNet ,
Subnet1 - Name of Subnet.
Can anyone tell me what is the equivalent script in terraform?
Use a subnet data source:
data "azurerm_subnet" "subnet1" {
name = "Subnet1"
virtual_network_name = "${var.VNetName}"
resource_group_name = "${var.VNetResourceGroup}"
}
Then reference it in your NIC code with
subnet_id = "${data.azurerm_subnet.subnet1.id}"

Resources