Terraform referencing output from another module with for_each - terraform

I am having trouble referencing an output from a module in another module. The resources in the first module was deployed using for_each. The resources in the second module is trying to reference the resources from first module
There are 2 modules created
Security Group
VM
The intention is to assign the Security Group to the VM
The following is the module for the Security Group
variable "configserver" {
type = map(object({
name = string
location = string
subnet = string
availability_zone = string
vm_size = string
hdd_size = string
}))
}
module "configserver_nsg" {
for_each = var.configserver
source = "../../../terraform/modules/azure-network-security-group"
resource_group_name = var.resource_group_name
tags = var.tags
location = each.value.location
nsg_name = "${each.value.name}-nsg"
security_rules = [
{
name = "Office",
priority = "100"
direction = "Inbound"
access = "Allow"
protocol = "TCP"
source_port_range = "*"
destination_port_ranges = [
"22"]
source_address_prefix = "192.168.1.100"
destination_address_prefixes = [
module.configserver_vm[each.key].private_ip
]
},
{
name = "Deny-All-Others"
priority = 4096
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
]
}
// Value
configserver = {
config1 = {
name = "config1"
location = "eastus"
subnet = "services"
availability_zone = 1
vm_size = "Standard_F2s_v2"
hdd_size = 30
}
}
The security group module source has an output file which outputs the id of the nsg
output "nsg_id" {
description = "The ID of the newly created Network Security Group"
value = azurerm_network_security_group.nsg.id
}
Generally, if there isn't a for_each, I could access the nsg_id like this
module.configserver_nsg.id
So far this is good, now the issue is that I am not able to access the nsg_id from another module
module "configserver_vm" {
for_each = var.configserver
source = "../../../terraform/modules/azure-linux-vm"
resource_group = module.resource_group.name
ssh_public_key = var.ssh_public_key
tags = var.tags
vm_name = each.value.name
location = each.value.location
subnet_id = each.value.subnet
availability-zones = each.value.availability_zone
vm_size = each.value.vm_size
hdd-size = each.value.hdd_size
nsg_id = module.configserver_nsg[each.key].nsg_id
}
Based on my research these, a number of posts (here, here, here say I should be able to loop through the map using each.key however
nsg_id = module.configserver_nsg[each.key].nsg_id
this produces the error
Error: Cycle: module.configserver_nsg (close), module.configserver_vm.var.nsg_id (expand), module.configserver_vm.azurerm_network_interface_security_group_association.this, module.configserver_vm (close), module.configserver_nsg.var.security_rules (expand), module.configserver_nsg.azurerm_network_security_group.nsg, module.configserver_nsg.output.nsg_id (expand)
Is there any other way to reference the value?

As I see the first problem is that you use the wrong way to quote the things from the module configserver_nsg for the NSG id, it should be like this:
nsg_id = module.configserver_nsg[each.value.name].nsg_id
And the second problem has been said by #Matt. It's a cyclical dependency between the two modules. The thing that makes the cyclical dependency is the NSG rule, it seems the NSG rule needs the VM private IP address. According to my knowledge, you cannot solve the cyclical dependency if you do not make a change. So I recommend you make a change that separates the NSG rule from the module configserver_nsg and use the resource azurerm_network_security_rule instead after the two modules.
And finally, it seems like this:
variable "configserver" {
type = map(object({
name = string
location = string
subnet = string
availability_zone = string
vm_size = string
hdd_size = string
}))
}
module "configserver_nsg" {
for_each = var.configserver
source = "../../../terraform/modules/azure-network-security-group"
resource_group_name = var.resource_group_name
tags = var.tags
location = each.value.location
nsg_name = "${each.value.name}-nsg"
security_rules = [
{
},
{
name = "Deny-All-Others"
priority = 4096
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
]
}
// Value
configserver = {
config1 = {
name = "config1"
location = "eastus"
subnet = "services"
availability_zone = 1
vm_size = "Standard_F2s_v2"
hdd_size = 30
}
}
module "configserver_vm" {
for_each = var.configserver
source = "../../../terraform/modules/azure-linux-vm"
resource_group = module.resource_group.name
ssh_public_key = var.ssh_public_key
tags = var.tags
vm_name = each.value.name
location = each.value.location
subnet_id = each.value.subnet
availability-zones = each.value.availability_zone
vm_size = each.value.vm_size
hdd-size = each.value.hdd_size
nsg_id = module.configserver_nsg[each.value.name].nsg_id
}
resource "azurerm_network_security_rule" "configserver_nsg" {
for_each = var.configserver
name = "Office",
priority = "100"
direction = "Inbound"
access = "Allow"
protocol = "TCP"
source_port_range = "*"
destination_port_ranges = ["22"]
source_address_prefix = "192.168.1.100"
destination_address_prefixes = [
module.configserver_vm[each.key].private_ip
]
resource_group_name = var.resource_group_name
network_security_group_name = "${each.value.name}-nsg"
}

Related

Terraform azurerm_linux_web_app dynamic ip_restriction

i'm trying to create several applications in one go and i can successfully do it with count = length(var.webapp_name) but the problem that i'm facing is that some of the apps are required to communicate with each other and i need to whitelist outbound_ip_addresses on each app
the code that i use which is causing issues is the following:
resource "azurerm_linux_web_app" "API" {
depends_on = [azurerm_subnet.subnet]
count = length(var.webapp_name)
name = lower("${var.customer4letter}-${var.env3letter}-${var.locationid3letter}-${var.servicetype}-${element(var.webapp_name, count.index)}")
location = var.location //West US 2
resource_group_name = azurerm_resource_group.rg.name
service_plan_id = azurerm_service_plan.api-farm.id
https_only = "true"
app_settings = {
"WEBSITE_USE_DIAGNOSTIC_SERVER" = "True"
}
identity {
type = "SystemAssigned"
}
site_config {
ftps_state = "FtpsOnly"
websockets_enabled = "false"
use_32_bit_worker = "false"
always_on = "true"
application_stack {
dotnet_version = "6.0"
}
dynamic "ip_restriction" {
for_each = local.ip_address_list3
content{
action = "Allow"
name = ip_restriction.value["name"]
service_tag = ip_restriction.value["service_tag"]
priority = ip_restriction.value["prior"]
} }
dynamic "ip_restriction" {
for_each = azurerm_linux_web_app.API[0].outbound_ip_addresses
content {
ip_address = cidrhost(ip_restriction.value,0)
} } } }
Local and variables set are
variable "webapp_name" {
default = [ "app1", "app2", "app3" ]
}
locals {
ip_address_list3 = [
{
service_tag = "AppService"
prior : "102",
name = "VirtualNetwork"
}
]
}
the error that i have is the following:
I Tried to reproduce the same in my environment to azure web app with outbound IP using Terraform:
Terraform Code:
provider "azurerm" {
features {}
}
locals {
resource_group_name = "test-rg"
location = "East US"
app_name_prefix = "venkatdemoapp"
}
resource "azurerm_resource_group" "test-rg" {
name = local.resource_group_name
location = local.location
}
# Create multiple web apps
resource "azurerm_app_service_plan" "test_plan" {
count = 2
name = "${local.app_name_prefix}-plan-${count.index}"
location = local.location
resource_group_name = azurerm_resource_group.test-rg.name
sku {
tier = "Basic"
size = "B1"
}
}
resource "azurerm_app_service" "thejawebapp" {
count = 2
name = "${local.app_name_prefix}-${count.index}"
location = local.location
resource_group_name = azurerm_resource_group.test-rg.name
app_service_plan_id = azurerm_app_service_plan.test_plan[count.index].id
}
# Restrict access to each web app
resource "azurerm_network_security_group" "test_nsg" {
count = 2
name = "${local.app_name_prefix}-nsg-${count.index}"
location = local.location
resource_group_name = azurerm_resource_group.test-rg.name
security_rule {
name = "Allow_HTTP_Traffic"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "80"
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "Deny_All_Traffic"
priority = 200
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
# Associate each web app with its NSG.
resource "azurerm_network_interface" "test_nic" {
count = 2
name = "${local.app_name_prefix}-nic-${count.index}"
location = local.location
resource_group_name = azurerm_resource_group.test-rg.name
ip_configuration {
name = "${local.app_name_prefix}-ipconfig-${count.index}"
subnet_id = azurerm_subnet.test_subnet.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_subnet" "test_subnet" {
name = "${local.app_name_prefix}-subnet"
resource_group_name = azurerm_resource_group.test-rg.name
virtual_network_name = azurerm_virtual_network.test_vnet.name
address_prefixes = ["10.0.1.0/24"]
}
resource "azurerm_virtual_network" "test_vnet" {
name = "${local.app_name_prefix}-vnet"
location = local.location
resource_group_name = azurerm_resource_group.test-rg.name address_space = ["10.0.0.0/16"]
}
Terraform Apply:
Once ran the above code resources are created automatically with restriction.
NSG Rule:

Reference to specific object created with for each terraform

I create two subnets with for each. But then when I want to associate nsg with ONLY ONE SPECIFIC subnet I don't know how to reference to it(((
thanks
resource "azurerm_subnet" "subnets" {
for_each = {
def-subnet = var.subnet_address_prefixes[0]
GatewaySubnet = var.subnet_address_prefixes[1]
}
address_prefixes = each.value
name = each.key
virtual_network_name = azurerm_virtual_network.testvm-VNET.name
resource_group_name = azurerm_resource_group.testvm-RG.name
}
resource "azurerm_network_security_group" "def-nsg" {
location = var.region
name = "def-nsg"
resource_group_name = azurerm_resource_group.testvm-RG.name
security_rule {
access = "Allow"
direction = "Inbound"
name = "rdp"
priority = 300
protocol = "Tcp"
destination_port_range = "3389"
source_port_range = "3389"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
resource "azurerm_subnet_network_security_group_association" "def-subnet-nsg-association" {
network_security_group_id = azurerm_network_security_group.def-nsg.id
subnet_id = here I want to use only def-subnet id
}
Since you've used for_each, you refer to individual instances of azurerm_subnet using key, such as def-subnet:
subnet_id = azurerm_subnet.subnets["def-subnet"].id

Use output value from module that has a for_each set

I had my code setup to export the dynamic private ip address when the VM is created. I did this via an outputs value. Since then, I have updated to tf 0.13 and I'm using a for_each in the module but when I reference this value now I get the below error. I'm not sure how I can export the dynamic private address attribute of the NIC now the for_each has been set to be used in the source_address_prefixes. I understand what the error is saying but not sure on correct way of exporting the value to the object map?
Error: Unsupported attribute
on main.tf line 66, in resource "azurerm_network_security_rule" "fico-app-sr-80":
66: source_address_prefixes = module.fico_web_vm.linux_vm_ips[0]
|----------------
| module.fico_web_vm is object with 1 attribute "web-1"
This object does not have an attribute named "linux_vm_ips".
The outputs.tf file:
output "linux_vm_names" {
value = [azurerm_linux_virtual_machine.virtual_machine.*.name]
}
output "linux_vm_ips" {
value = [azurerm_network_interface.network_interface.*.private_ip_address]
}
output "linux_vm_nsg" {
value = azurerm_network_security_group.network_security_group.name
}
Security rule where the error is referencing, located in the main.tf:
resource "azurerm_network_security_rule" "fico-app-sr-80" {
name = "nsr-${var.environment}-${var.directorate}-${var.business_unit}-${var.vm_identifier}${var.instance_number}-http80"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "80"
source_address_prefixes = module.fico_web_vm.linux_vm_ips[0]
destination_address_prefix = "VirtualNetwork"
resource_group_name = azurerm_resource_group.rg_dwp_fico_app.name
network_security_group_name = "module.fico_app_vm.linux_vm_nsg"
}
The NIC which is in the root module:
resource "azurerm_network_interface" "network_interface" {
name = "nic-${var.environment}-${var.directorate}-${var.business_unit}-${var.vm_identifier}-${var.vm_name}"
resource_group_name = var.resource_group
location = var.location
enable_ip_forwarding = "false"
enable_accelerated_networking = "false"
ip_configuration {
name = "ipconfig1"
subnet_id = data.azurerm_subnet.dwp_subnet.id
private_ip_address_allocation = "Dynamic"
primary = "true"
}
The module that builds the web_vm in main.tf:
module "fico_web_vm" {
for_each = var.web_servers
source = "../modules/compute/linux_vm"
source_image_id = var.web_image_id
location = var.location
vm_name = each.key
vm_identifier = "${var.vm_identifier}${var.instance_number}"
vm = each.value
disks = each.value["disks"]
resource_group = azurerm_resource_group.rg_dwp_fico_web.name
directorate = var.directorate
business_unit = var.business_unit
environment = var.environment
network_rg_identifier = var.network_rg_identifier
subnet_name = "sub-dwp-${var.environment}-${var.directorate}-${var.business_unit}-fe01"
diag_storage_account_name = var.diag_storage_account_name
ansible_storage_account_name = var.ansible_storage_account_name
ansible_storage_account_key = var.ansible_storage_account_key
log_analytics_workspace_name = var.log_analytics_workspace_name
backup_policy_name = var.backup_policy_name
}
variables for the app in the main.tf:
# Web VM Variables
variable "web_servers" {
description = "Variable for defining each instance"
type = map(object({
size = string
admin_username = string
public_key = string
disks = list(number)
zone_vm = string
zone_disk = list(string)
}))
}
The tfvars file that the variables.tf translates to and which the for_each loops over:
web_servers ={
web-1 = {
size = "Standard_B2s"
admin_username = xxxx
public_key = xxxx
disks = [32, 32]
zone_vm = "1"
zone_disk = ["1"]
}
}
As I see, there two mistakes in your code.
First:
network_security_group_name = "module.fico_app_vm.linux_vm_nsg"
need to change into:
network_security_group_name = module.fico_app_vm.linux_vm_nsg
The double quotes are usually used to set a string when you use them only, not other resources.
Second:
source_address_prefixes = module.fico_web_vm.linux_vm_ips[0]
need to change into:
source_address_prefixes = module.fico_web_vm.linux_vm_ips
and the output also should be change into:
output "linux_vm_ips" {
value = azurerm_network_interface.network_interface.*.private_ip_address
}
I see you just create one NIC without count and for_each, but when you set the value with azurerm_network_interface.network_interface.*.private_ip_address then it also returns a list. And the source_address_prefixes option expects a list, so you just need to give the output.
Maybe it's better to change the output like this:
output "linux_vm_ip" {
value = azurerm_network_interface.network_interface.private_ip_address
}
And then you can set the source_address_prefixes like this:
source_address_prefixes = [ module.fico_web_vm.linux_vm_ip ]

For azure terraform I am unclear about the output command?

Do not understand when the Output command is used in azure terraform? Where is the output going? Why do we need this output for? We have multiple tf files in aws which I am converting to azure. What type of information is outputted?
Without this output information what would happen?
Please provide examples.
Thank you
A lot of questions, I'm not sure I can cover all this but I can deo give an example.
A good practice would be to build terraform modules, then use the output from these modules to pass along arguments to other modules consuming the same type of input.
The following is a "base module" that creates a resource group, a Vnet, couple of subnets (external and internal for example), NSG and binds the rules to the subnets
resource "azurerm_resource_group" "main_rg" {
name = var.resource_group_name
location = var.location
tags = {
group = var.resource_group_name
Customer = var.tag_reference
stack = "resource group"
}
}
resource "azurerm_virtual_network" "main_vnet" {
name = "${azurerm_resource_group.main_rg.name}-primary-vnet"
address_space = ["${var.vnet_cidr}"]
location = azurerm_resource_group.main_rg.location
resource_group_name = azurerm_resource_group.main_rg.name
tags = {
group = var.resource_group_name
Customer = var.tag_reference
stack = "virtual network"
}
}
resource "azurerm_subnet" "backend_subnet" {
name = "${azurerm_resource_group.main_rg.name}-backend-subnet"
resource_group_name = azurerm_resource_group.main_rg.name
virtual_network_name = azurerm_virtual_network.main_vnet.name
address_prefix = var.backend_subnet_cidr
service_endpoints = ["Microsoft.Sql", "Microsoft.Storage"]
}
resource "azurerm_subnet" "frontend_subnet" {
name = "${azurerm_resource_group.main_rg.name}-frontend-subnet"
resource_group_name = azurerm_resource_group.main_rg.name
virtual_network_name = azurerm_virtual_network.main_vnet.name
address_prefix = var.frontend_subnet_cidr
service_endpoints = ["Microsoft.Sql"]
}
resource "azurerm_network_security_group" "default_nsg" {
name = "${azurerm_resource_group.main_rg.name}-nsg"
location = azurerm_resource_group.main_rg.location
resource_group_name = azurerm_resource_group.main_rg.name
security_rule {
name = "appgwV1Exception"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "65503-65534"
source_address_prefix = "Internet"
destination_address_prefix = "*"
description = "This is needed to allow helth check of backend server to pass according to official documentation"
}
security_rule {
name = "appgwV2Exception"
priority = 200
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "65200-65535"
source_address_prefix = "Internet"
destination_address_prefix = "*"
description = "This is needed to allow helth check of backend server to pass according to official documentation"
}
security_rule {
name = "Office"
priority = 500
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "443"
source_address_prefix = "1.2.3.4/32"
destination_address_prefix = "*"
description = "Allow 443 access from the office"
}
tags = {
group = var.resource_group_name
Customer = var.tag_reference
stack = "NSG"
}
}
resource "azurerm_subnet_network_security_group_association" "nsg_to_backend" {
network_security_group_id = azurerm_network_security_group.default_nsg.id
subnet_id = azurerm_subnet.backend_subnet.id
}
resource "azurerm_subnet_network_security_group_association" "nsg_to_frontend" {
network_security_group_id = azurerm_network_security_group.default_nsg.id
subnet_id = azurerm_subnet.frontend_subnet.id
}
output "resource_group_name" {
value = azurerm_resource_group.main_rg.name
}
output "vnet_name" {
value = azurerm_virtual_network.main_vnet.name
}
output "vnet_id" {
value = azurerm_virtual_network.main_vnet.id
}
output "backend_subnet_id" {
value = azurerm_subnet.backend_subnet.id
}
output "frontend_subnet_id" {
value = azurerm_subnet.frontend_subnet.id
}
output "nsg_id" {
value = azurerm_network_security_group.default_nsg.id
}
Again the terraform best practice is to use an output file but I'll spare you that for now, see the outputs at the bottom of the file, now I have another module for creating a VM, so my main.tf if you will or the complete file might look a bit something like this:
provider "azurerm" {
version = "~> 1.21"
}
terraform {
backend "azurerm" {}
}
module "base_infra" {
source = "../../base_infra"
location = var.location
resource_group_name = var.resource_group_name
vnet_cidr = var.vnet_cidr
backend_subnet_cidr = var.backend_subnet_cidr
frontend_subnet_cidr = var.frontend_subnet_cidr
tag_reference = var.tag_reference
}
module "webapp_vm" {
source = "../../webapp"
resource_group_name = module.base_infra.resource_group_name
location = var.location
vnet_cidr = module.base_infra.main_vnet_id
subnet_id = module.base_infra.backend_subnet_id
tag_reference = var.tag_reference
datadisk_size_gb = "200"
instance_count = "1"
instance_name_prefix = "${module.base_infra.resource_group_name}-webapp"
vm_size = var.vm_size
vm_username = var.vm_username
vm_password = module.webapp_vm_password.password_result
}
I did cut some corners in my answer, hope I gave you enough

Can a VNET address space be used for NSG source address prefix?

I would like to allow traffic through a NSG from all local subnets (not including peered subnets). As I only have one address space, it seems like the most direct way to do this would be to use the address_space of the VNET as the source_address_prefix of the security rule.
resource "azurerm_resource_group" "west01-rg" {
name = "west01-rg"
location = "West US"
}
resource "azurerm_virtual_network" "virtual-network" {
name = "west01-vnet"
location = "${azurerm_resource_group.west01-rg.location}"
resource_group_name = "${azurerm_resource_group.west01-rg.name}"
address_space = ["10.10.20.0/21"]
}
resource "azurerm_subnet" "servers-subnet" {
name = "ServersNet"
resource_group_name = "${azurerm_resource_group.west01-rg.name}"
virtual_network_name = "${azurerm_virtual_network.virtual-network.name}"
address_prefix = "10.10.20.0/24"
}
resource "azurerm_network_security_group" "dc-nsg" {
name = "dc-nsg"
location = "${azurerm_resource_group.west01-rg.location}"
resource_group_name = "${azurerm_resource_group.west01-rg.name}"
security_rule {
name = "AllowCidrSubnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "${azurerm_virtual_network.virtual-network.address_space}"
destination_address_prefix = "*"
}
tags {
environment = "Testing"
}
}
Per the documentation, this value can be in CIDR notation. However, my example above results in the error
Error: azurerm_network_security_group.dc: security_rule.0.source_address_prefix must be a single value, not a list
If I switch to source_address_prefixes, which should accept a list, I get this error
Error: azurerm_network_security_group.dcx: security_rule.0.source_address_prefixes: should be a list
So it seems the value is both a list and not a list. Should this work? Or should I be going about it a different way?
Terraform v0.11.11
provider.azurerm v1.21.0
In Terraform pre 0.12, every variable is a string type by default and if you want to use a list or map type you must use that type consistently as you pass the variable around. This should change in Terraform 0.12 as HCL2 has better support for types including more complex type handling.
To solve your issue you need to either index the list to return a single element which would then be a string or you need to be consistent with your list type.
So either of these should work:
resource "azurerm_network_security_group" "dc-nsg" {
name = "dc-nsg"
location = "${azurerm_resource_group.west01-rg.location}"
resource_group_name = "${azurerm_resource_group.west01-rg.name}"
security_rule {
name = "AllowCidrSubnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "${azurerm_virtual_network.virtual-network.address_space[0]}"
destination_address_prefix = "*"
}
tags {
environment = "Testing"
}
}
or using a list directly:
resource "azurerm_network_security_group" "dc-nsg" {
name = "dc-nsg"
location = "${azurerm_resource_group.west01-rg.location}"
resource_group_name = "${azurerm_resource_group.west01-rg.name}"
security_rule {
name = "AllowCidrSubnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefixes = ["${azurerm_virtual_network.virtual-network.address_space}"]
destination_address_prefix = "*"
}
tags {
environment = "Testing"
}
}

Resources