Terraform plan output shows recreating existing configuration / resources - azure

I'm currently getting a strange issue. I have a Application Gateway deployed via terraform. If I try to add port 443 in both front end and backend end, the terraform plan shows it will delete the frontend and backend for port 80 and then recreate port 80 again along with the addition of 443.
~ resource "azurerm_application_gateway" "xyz" {
id = "xyz"
name = "xyz"
tags = {
"BusinessUnit" = "ehs"
"LineOfBusiness" = "corp"
}
# (8 unchanged attributes hidden)
- backend_http_settings {
- cookie_based_affinity = "Disabled" -> null
- id = "xyz" -> null
- name = "xyz" -> null
- path = "/path1/" -> null
- pick_host_name_from_backend_address = false -> null
- port = 80 -> null
- protocol = "Http" -> null
- request_timeout = 60 -> null
- trusted_root_certificate_names = [] -> null
}
+ backend_http_settings {
+ cookie_based_affinity = "Disabled"
+ host_name = "xyz"
+ id = (known after apply)
+ name = "xyz"
+ path = "/path1/"
+ pick_host_name_from_backend_address = false
+ port = 443
+ probe_id = (known after apply)
+ protocol = "Https"
+ request_timeout = 60
+ trusted_root_certificate_names = [
+ "irmscer",
]
}
+ backend_http_settings {
+ cookie_based_affinity = "Disabled"
+ id = "xyz"
+ name = "xyz"
+ path = "/path1/"
+ pick_host_name_from_backend_address = false
+ port = 80
+ protocol = "Http"
+ request_timeout = 60
+ trusted_root_certificate_names = []
}
+ frontend_port {
+ id = (known after apply)
+ name = "xyz"
+ port = 443
}
How to get around this issue? I'm not pointing the terraform to use an specific version
This is the terraform backend
terraform {
backend "azurerm" {
storage_account_name = "xyz"
resource_group_name = "xyz"
container_name = "appgw"
tenant_id = "xyz"
subscription_id = "xyz"
key = "xyz"
}
}
provider "azurerm" {
features {}
}

It is not re-creating entire application gateway. It is re-creating the settings with port 80 and 443 which is normal. This is normal behavior and not an issue. May be, the Terraform addresses these kind of issues in future versions.

backend_http_settings protocol will not listen on port 443 and port doesn't support for backend pools.
As a result, you cannot change the port for the backend, and the only supported one is 80.
And for front end configuration, if you want to add any existing listener port to the previous port 80, you must add one more frontend port block so that it will consider two ports and listen to the specific port that we provide.
Add frontend_port as shown:
frontend_port{
name = local.frontend_port_name_new
port = 443
}
Firstly, Deployed application_gateway with listener Port 80:
I've written below script by following terraform registry template and made a few changes as per your requirement and was able to update the port changes successfully.
provider "azurerm"{
features{}
}
resource "azurerm_resource_group" "example" {
name = "xxxresources"
location = "West Europe"
}
resource "azurerm_virtual_network" "example" {
name = "xxxnetwork"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
address_space = xxxx
}
resource "azurerm_subnet" "frontend" {
name = "frontend"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = xxxx
}
resource "azurerm_subnet" "backend" {
name = "backend"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = xxx
}
resource "azurerm_public_ip" "example" {
name = "xxxx"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
allocation_method = "Dynamic"
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
backend_address_pool_name = "${azurerm_virtual_network.example.name}-beapname"
frontend_port_name = "${azurerm_virtual_network.example.name}-fendport"
frontend_port_name_new = "${azurerm_virtual_network.example.name}-feportnew"
frontend_ip_configuration_name = "${azurerm_virtual_network.example.name}-fconfig"
http_setting_name = "${azurerm_virtual_network.example.name}-htstname"
listener_name = "${azurerm_virtual_network.example.name}-httplisten"
request_routing_rule_name = "${azurerm_virtual_network.example.name}-rt"
redirect_configuration_name = "${azurerm_virtual_network.example.name}-rcfg"
}
resource "azurerm_application_gateway" "network" {
name = "xxxxappgateway"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
sku {
name = "Standard_Small"
tier = "Standard"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_port{
name = local.frontend_port_name_new
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.example.id
}
backend_address_pool {
name = local.backend_address_pool_name
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/path1/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
}
terraform init:
After updating the port terraform plan showed output as below:
terraform apply:
Changes deployed successfully and you can track the change analysisby going to the Activity Log under deployed application_gateway resource:

Related

Issue creating backend pool for Azure load balancer with private link service

I am planning to access an application hosted on two servers using azure load balancer which will be accessed using private end point and private link server from on-prem network for private access. while i try to execute the code, getting the below error. If i don't use back end pool, i am able to create the load balancer with private link service and private end point, what could be an issue?
Error: creating Private Link Service: (Name "privatelink" / Resource Group "XXXXXXXX"): network.PrivateLinkServicesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="PrivateLinkServiceIsNotSupportedForIPBasedLoadBalancer" Message="Private link service is not supported for load balancer /subscriptions/XXXXXXXX/providers/Microsoft.Network/privateLinkServices/privatelink with backend addresses set by (virtualNetwork, ipAddress) or (subnet, ipAddress)." Details=[]
resource "azurerm_subnet" "lbsubnet" {
name = "lbsubnet"
resource_group_name = local.resource_group
virtual_network_name = azurerm_virtual_network.devvm_net.name
address_prefixes = ["10.20.1.0/24"]
enforce_private_link_service_network_policies = true
depends_on = [
azurerm_virtual_network.devvm_net
]
}
resource "azurerm_lb" "app_balancer" {
name = "app-balancer"
location = local.location
resource_group_name = local.resource_group
sku="Standard"
sku_tier = "Regional"
frontend_ip_configuration {
name = "frontend-ip"
subnet_id = azurerm_subnet.lbsubnet.id
# private_ip_address_allocation = "Dynamic"
}
}
// the backend pool
resource "azurerm_lb_backend_address_pool" "PoolA" {
loadbalancer_id = azurerm_lb.app_balancer.id
name = "PoolA"
depends_on=[
azurerm_lb.app_balancer
]
}
resource "azurerm_lb_backend_address_pool_address" "vm1" {
name = "vm1"
backend_address_pool_id = azurerm_lb_backend_address_pool.PoolA.id
virtual_network_id = azurerm_virtual_network.devvm_net.id
ip_address = azurerm_network_interface.devvm1_interface1.private_ip_address
#ip_address= "10.20.0.10"
}
resource "azurerm_lb_backend_address_pool_address" "appvm2_address" {
name = "appvm2"
backend_address_pool_id = azurerm_lb_backend_address_pool.PoolA.id
virtual_network_id = azurerm_virtual_network.devvm_net.id
#ip_address = azurerm_network_interface.devvm2_interface2.private_ip_address
ip_address = "10.20.0.5"
depends_on=[
azurerm_lb_backend_address_pool.PoolA
]
}
// Health Probe
resource "azurerm_lb_probe" "ProbeA" {
resource_group_name = local.resource_group
loadbalancer_id = azurerm_lb.app_balancer.id
name = "probeA"
port = 80
protocol = "Tcp"
depends_on=[
azurerm_lb.app_balancer
]
}
// Load Balancing Rule
resource "azurerm_lb_rule" "RuleA" {
resource_group_name = local.resource_group
loadbalancer_id = azurerm_lb.app_balancer.id
name = "RuleA"
protocol = "Tcp"
frontend_port = 80
backend_port = 80
frontend_ip_configuration_name = "frontend-ip"
backend_address_pool_ids = [ azurerm_lb_backend_address_pool.PoolA.id ]
depends_on=[
azurerm_lb.app_balancer
]
}
// the NAT Rules
resource "azurerm_lb_nat_rule" "NATRuleA" {
resource_group_name = local.resource_group
loadbalancer_id = azurerm_lb.app_balancer.id
name = "RDPAccess"
protocol = "Tcp"
frontend_port = 3389
backend_port = 3389
frontend_ip_configuration_name = "frontend-ip"
depends_on=[
azurerm_lb.app_balancer
]
}
resource "azurerm_virtual_network" "pvt-endpoint-vnet" {
name = "pvtendpoint-network"
location = local.location
resource_group_name = local.resource_group
address_space = ["10.50.0.0/16"]
}
resource "azurerm_subnet" "endpoint-subnet" {
name = "endpoint-subnet"
resource_group_name = local.resource_group
virtual_network_name = azurerm_virtual_network.pvt-endpoint-vnet.name
address_prefixes = ["10.50.0.0/24"]
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_private_link_service" "privatelink-service" {
name = "privatelink"
location = local.location
resource_group_name = local.resource_group
load_balancer_frontend_ip_configuration_ids = [azurerm_lb.app_balancer.frontend_ip_configuration.0.id]
nat_ip_configuration {
name = "pls-ip"
primary = true
subnet_id = azurerm_subnet.lbsubnet.id
}
}
resource "azurerm_private_endpoint" "private_endpoint" {
name = "private-endpoint"
location = local.location
resource_group_name = local.resource_group
subnet_id = azurerm_subnet.endpoint-subnet.id
private_service_connection {
name = "privateserviceconnection"
private_connection_resource_id = azurerm_private_link_service.privatelink-service.id
is_manual_connection = false
}
}

502 Bad Gateway from Azure Application Gateway Connecting to Azure Container Instance

I am working on learning Terraform and Azure Web Services. After following a series of tutorials, I've been working on getting an Azure Container Instance setup that talks to a CosmosDB instance within a virtual network, and I want an Application Gateway setup that will allow HTTP connections to the Azure Container Instance.
Currently, when I call the IP address assigned to the Application Gateway, I receive a 502 Bad Gateway. I've verified that the image I'm running in the Azure Container Instance works locally. I have a feeling that the issues I'm facing are in relation to the back-end address pool I've configured, and possibly an issue with the rules I've setup in my network security group (nsg-myapp).
I was wondering if someone could look at my Terraform and identify what I've not configured correctly? The closest question I found similar to my scenario on StackOverflow as this unresolved question from last year.
network.tf
resource "azurerm_virtual_network" "myappdb" {
name = "myappdb-vnet"
address_space = ["10.7.0.0/16"]
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.2.0/24"]
service_endpoints = ["Microsoft.AzureCosmosDB"]
delegation {
name = "acidelegationservice"
service_delegation {
name = "Microsoft.ContainerInstance/containerGroups"
actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
}
}
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "frontend" {
name = "myapp-frontend"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.0.0/24"]
}
resource "azurerm_network_security_group" "nsg-myapp" {
name = "nsg-aci"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
security_rule {
name = "from-gateway-subnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [22, 80, 443, 445, 8000]
source_address_prefixes = azurerm_subnet.internal.address_prefixes
destination_address_prefix = azurerm_subnet.internal.address_prefixes[0]
}
security_rule {
name = "DenyAllInBound-Override"
priority = 900
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "to-internet"
priority = 100
direction = "Outbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [80, 443, 445]
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "DenyAllOutBound-Override"
priority = 900
direction = "Outbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
resource "azurerm_subnet_network_security_group_association" "sn-nsg-aci" {
subnet_id = azurerm_subnet.internal.id
network_security_group_id = azurerm_network_security_group.nsg-myapp.id
}
resource "azurerm_network_profile" "containergroup_profile" {
name = "acg-profile"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
container_network_interface {
name = "acg-nic"
ip_configuration {
name = "aciipconfig"
subnet_id = azurerm_subnet.internal.id
}
}
}
resource "azurerm_public_ip" "myappip" {
name = "myappip"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
allocation_method = "Static"
sku = "Standard"
}
locals {
backend_address_pool_name = "${azurerm_virtual_network.myappdb.name}-beap"
frontend_port_name = "${azurerm_virtual_network.myappdb.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.myappdb.name}-feip"
http_setting_name = "${azurerm_virtual_network.myappdb.name}-be-htst"
listener_name = "${azurerm_virtual_network.myappdb.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.myappdb.name}-rqrt"
redirect_configuration_name = "${azurerm_virtual_network.myappdb.name}-rdrcfg"
}
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/path1/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
}
container.tf
resource "azurerm_container_group" "tf_cg_sampleapi" {
depends_on = [azurerm_cosmosdb_account.db]
name = "cg_myapp"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
network_profile_id = azurerm_network_profile.containergroup_profile.id
ip_address_type = "Private"
# dns_name_label = "sampleapitf"
os_type = "Linux"
identity {
type = "SystemAssigned"
}
container {
name = "myapp"
image = "sample/myapp"
cpu = 1
memory = 1
ports {
port = 80
protocol = "TCP"
}
ports {
port = 443
protocol = "TCP"
}
secure_environment_variables = {
"MYAPP_CONNECTION_STRING" = azurerm_cosmosdb_account.db.connection_strings[0]
}
}
}
I met the similar issue and in my case(containers on top of Azure App Service) I needed to put the depends_on block inside the application gateway resource creation with regards to app services being created in the first place. So in your case should be:
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/path1/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
depends_on = [ azurerm_container_group.tf_cg_sampleapi, ]
}
I figured out the root cause of my 502 Gateway error was due to health checks not being setup / not working. Consequently, I setup custom probes that would go to an API endpoint to return a 200 OK response. Of course, I will configure this endpoint to actually check to see if I can connect to my services, but this was just a test to verify this was the issue.
I also removed the DenyAllInBound-Override and DenyAllOutBound-Override rules within my nsg-aci security group, as this was causing issues with my ACI to connect to my Cosmos DB.
This was my resulting network.tf and container.tf files:
network.tf
resource "azurerm_virtual_network" "myappdb" {
name = "myappdb-vnet"
address_space = ["10.7.0.0/16"]
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.2.0/24"]
service_endpoints = ["Microsoft.AzureCosmosDB"]
delegation {
name = "acidelegationservice"
service_delegation {
name = "Microsoft.ContainerInstance/containerGroups"
actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
}
}
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "frontend" {
name = "myapp-frontend"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.0.0/24"]
}
resource "azurerm_network_security_group" "nsg-myapp" {
name = "nsg-aci"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
security_rule {
name = "from-gateway-subnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [22, 80, 443, 445, 8000]
source_address_prefixes = azurerm_subnet.internal.address_prefixes
destination_address_prefixes = azurerm_subnet.internal.address_prefixes
}
security_rule {
name = "to-internet"
priority = 100
direction = "Outbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [80, 443, 445]
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
resource "azurerm_subnet_network_security_group_association" "sn-nsg-aci" {
subnet_id = azurerm_subnet.internal.id
network_security_group_id = azurerm_network_security_group.nsg-myapp.id
}
resource "azurerm_network_profile" "containergroup_profile" {
name = "acg-profile"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
container_network_interface {
name = "acg-nic"
ip_configuration {
name = "aciipconfig"
subnet_id = azurerm_subnet.internal.id
}
}
}
resource "azurerm_public_ip" "myappip" {
name = "myappip"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
allocation_method = "Static"
sku = "Standard"
}
locals {
backend_address_pool_name = "${azurerm_virtual_network.myappdb.name}-beap"
frontend_port_name = "${azurerm_virtual_network.myappdb.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.myappdb.name}-feip"
http_setting_name = "${azurerm_virtual_network.myappdb.name}-be-htst"
listener_name = "${azurerm_virtual_network.myappdb.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.myappdb.name}-rqrt"
redirect_configuration_name = "${azurerm_virtual_network.myappdb.name}-rdrcfg"
}
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
probe {
interval = 60
timeout = 60
name = "status"
protocol = "Http"
path = "/api/status/"
unhealthy_threshold = 3
host = "127.0.0.1"
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/"
port = 80
protocol = "Http"
request_timeout = 60
probe_name = "status"
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
depends_on = [azurerm_container_group.tf_cg_sampleapi, ]
}
container.tf
resource "azurerm_container_group" "tf_cg_sampleapi" {
depends_on = [azurerm_cosmosdb_account.db]
name = "cg_myapp"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
network_profile_id = azurerm_network_profile.containergroup_profile.id
ip_address_type = "Private"
# dns_name_label = "sampleapitf"
os_type = "Linux"
container {
name = "myapp"
image = "sample/myapp"
cpu = 1
memory = 1
ports {
port = 80
protocol = "TCP"
}
ports {
port = 443
protocol = "TCP"
}
secure_environment_variables = {
"MYAPP_CONNECTION_STRING" = azurerm_cosmosdb_account.db.connection_strings[0]
}
}
}

Terraform Upgrade to 0.14.0/azurerm to 2.65.0 Causing an Issue regarding SSL Cert in Application Gateway Configuration

After upgrading terraform to 0.14.0 and azurerm to 2.65.0 I got three errors regarding ssl certificate configuration in the application gateway section.
Error: expected "ssl_certificate.0.key_vault_secret_id" to not be an empty string, got
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
102: ssl_certificate {
Error: Computed attributes cannot be set
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
120: ssl_certificate {
Computed attributes cannot be set, but a value was set for
"ssl_certificate.0.id".
Error: Computed attributes cannot be set
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
120: ssl_certificate {
Computed attributes cannot be set, but a value was set for
"ssl_certificate.0.public_cert_data".
But key_vault_secret_id configuratiom is not existing in my code:
ssl_certificate {
name = local.certificate_name
data = filebase64("./ssl-cert/appgwcert.pfx")
password = "SecretPwd"
}
Snippet of the application gateway:
terraform {
required_version = "= 0.14.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "2.65.0"
}
}
}
provider "azurerm" {
features {}
}
# #################################################################
resource "azurerm_public_ip" "pub-ip" {
name = "appgw-pubIP"
resource_group_name = local.resour_group_name
location = local.resour_group_location
allocation_method = "Dynamic"
}
resource "azurerm_subnet" "subnet-01" {
name = "seubnet-app-gateway"
resource_group_name = local.resour_group_name
virtual_network_name = "vnet-app-gateway"
address_prefixes = ["10.21.0.0/24"]
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
resour_group_name = "app-gateway-test-01"
resour_group_location = "westus2"
backend_address_pool_name = "backend-pool-test-01"
frontend_port_name = "port_443"
frontend_ip_configuration_name = "appGwPublicFrontendIp"
http_setting_name = "http-settings-test-01"
listener_name = "https-listener-01"
request_routing_rule_name = "routrul-test-01"
certificate_name = "appgw-cert-test-01"
}
resource "azurerm_application_gateway" "network" {
name = "app-gateway-test-01"
resource_group_name = local.resour_group_name
location = local.resour_group_location
sku {
name = "Standard_V2"
tier = "Standard"
capacity = 2
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.subnet-01.id
}
frontend_port {
name = local.frontend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pub-ip.id
}
backend_address_pool {
name = local.backend_address_pool_name
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 20
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Https"
ssl_certificate_name = local.certificate_name
}
ssl_certificate {
name = local.certificate_name
# reference the dummy certificate
data = filebase64("./ssl-cert/appgwcert.pfx")
# this is only a dummy and not the actual certificate to be used thus no harm in storing the password
password = "SecretPwd"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
lifecycle {
ignore_changes = [ssl_certificate, http_listener]
}
}
I have no idea how to fix it?
There are few problems in your code, I have fixed those and tested on my environment.
The Public IP should have Sku = Standard and allocation_method = static.
The Sku name in application Gateway is standard_v2 but the
tier is Standard , it should be the same i.e. standard_v2 .
I have used terraform version 1.0.5 instead of using terraform
version 0.14.0.
So, after the changes the code is as below:
terraform {
required_version = "1.0.5"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "2.65.0"
}
}
}
provider "azurerm" {
features {}
}
# #################################################################
resource "azurerm_public_ip" "pub-ip" {
name = "appgw-pubIP"
resource_group_name = local.resour_group_name
location = local.resour_group_location
allocation_method = "Static"
sku = "standard"
}
resource "azurerm_subnet" "subnet-01" {
name = "seubnet-app-gateway"
resource_group_name = local.resour_group_name
virtual_network_name = "ansuman-vnet"
address_prefixes = ["172.31.10.0/24"]
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
resour_group_name = "myresourcegroup"
resour_group_location = "westus2"
backend_address_pool_name = "backend-pool-test-01"
frontend_port_name = "port_443"
frontend_ip_configuration_name = "appGwPublicFrontendIp"
http_setting_name = "http-settings-test-01"
listener_name = "https-listener-01"
request_routing_rule_name = "routrul-test-01"
certificate_name = "appgw-cert-test-01"
}
resource "azurerm_application_gateway" "network" {
name = "app-gateway-test-01"
resource_group_name = local.resour_group_name
location = local.resour_group_location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.subnet-01.id
}
frontend_port {
name = local.frontend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pub-ip.id
}
backend_address_pool {
name = local.backend_address_pool_name
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 20
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Https"
ssl_certificate_name = local.certificate_name
}
ssl_certificate {
name = local.certificate_name
# reference the dummy certificate
data = filebase64("C:/powershellpfx.pfx")
# this is only a dummy and not the actual certificate to be used thus no harm in storing the password
password = "password#1234"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
lifecycle {
ignore_changes = [ssl_certificate, http_listener]
}
}
Outputs:

Application gateway request_routing_rules does not exist

I am trying to deploy a azure application gateway. I set the configuration as follow:
resource "azurerm_application_gateway" "demo-app-gateway" {
location = var.location
resource_group_name = azurerm_resource_group.rg-hri-testing-env.name
name = "demo-app-gateway"
autoscale_configuration {
max_capacity = 10
min_capacity = 2
}
frontend_port {
name = "port_443"
port = 443
}
sku {
name = "Standard_v2"
tier = "Standard_v2"
}
frontend_ip_configuration {
name = "appGwPublicFrontendIp"
public_ip_address_id = azurerm_public_ip.demo-app-gateway-public-ip.id
private_ip_address_allocation = "Dynamic"
}
backend_http_settings {
cookie_based_affinity = "Disabled"
name = "demo-http-settings"
port = 443
protocol = "Https"
host_name = "apim.test.com"
pick_host_name_from_backend_address = false
path = "/external/"
request_timeout = 20
probe_name = "demo-apim-probe"
trusted_root_certificate_names = ["demo-trusted-root-ca-certificate"]
}
probe {
interval = 30
name = "demo-apim-probe"
path = "/status-0123456789abcdef"
protocol = "Https"
timeout = 30
unhealthy_threshold = 3
pick_host_name_from_backend_http_settings = true
match {
body = ""
status_code = [
"200-399"
]
}
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.GatewaSubnet.id
}
backend_address_pool {
name = "demo-backend-pool"
}
http_listener {
frontend_ip_configuration_name = "appGwPublicFrontendIp"
frontend_port_name = "port_443"
name = "demo-app-gateway-listener"
protocol = "Https"
require_sni = false
ssl_certificate_name = "demo-app-gateway-certificate"
}
ssl_certificate {
data = filebase64(var.ssl_certificate_path)
name = "demo-app-gateway-certificate"
password = var.ssl_certificate_password
}
trusted_root_certificate {
data = filebase64(var.ssl_certificate_path)
name = "demo-trusted-root-ca-certificate"
}
request_routing_rule {
http_listener_name = "demo-app-gateway-listener"
name = "demo-rule"
rule_type = "Basic"
backend_address_pool_name = "demo-backend-pool"
backend_http_settings_name = "demo-http-setting"
}
}
But when I run terraform apply I get this error.
Error: creating/updating Application Gateway: (Name "demo-app-gateway" / Resource Group "rg-hri-testing-apim"): network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="InvalidResourceReference" Message="Resource /subscriptions/my-sub/resourceGroups/rg-hri-testing-apim/providers/Microsoft.Network/applicationGateways/demo-app-gateway/backendHttpSettingsCollection/demo-http-setting referenced by resource /subscriptions/mysub/resourceGroups/rg-hri-testing-apim/providers/Microsoft.Network/applicationGateways/demo-app-gateway/requestRoutingRules/demo-rule was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[]
on app-gateway-main.tf line 1, in resource "azurerm_application_gateway" "demo-app-gateway":
1: resource "azurerm_application_gateway" "demo-app-gateway" {
The resource causing the error is the request_routing_rule not being found, but what it confuses me is that is looking for it before to create it?
Can anyone please help me to understand what am I doing wrong here?
Please if you need more infos, just let me know.
Thank you very much
Please check the Backend HTTP settings name which is being referenced by request routing rule block. You have to change it to demo-http-settings in request_routing_rule to resolve the error.
Issue:
You are using below as backend http setting :
backend_http_settings {
cookie_based_affinity = "Disabled"
name = "demo-http-settings"
port = 443
protocol = "Https"
host_name = "apim.test.com"
pick_host_name_from_backend_address = false
path = "/external/"
request_timeout = 20
probe_name = "demo-apim-probe"
trusted_root_certificate_names = ["demo-trusted-root-ca-certificate"]
}
But while referencing it in request request routing rule you are using :
request_routing_rule {
http_listener_name = "demo-app-gateway-listener"
name = "demo-rule"
rule_type = "Basic"
backend_address_pool_name = "demo-backend-pool"
backend_http_settings_name = "demo-http-setting"
As you have given the name of backend_http_setting_name = demo-http-settings and giving it as demo-http-setting in request_routing_rule. It will error out as it can't find the backend http setting.

Allow Application Gateway Backend pool pointing at a specific VM

How do we add virtual machine as target in backend pool of an Application Gateway using terraform code ?
What property I should use in the following code to point at particular virtual Machine ?
resource "azurerm_application_gateway" "network" {
name = "example-appgateway"
resource_group_name = "${azurerm_resource_group.test.name}"
location = "${azurerm_resource_group.test.location}"
sku {
name = "Standard_Small"
tier = "Standard"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = "${azurerm_subnet.frontend.id}"
}
frontend_port {
name = "${local.frontend_port_name}"
port = 80
}
frontend_ip_configuration {
name = "${local.frontend_ip_configuration_name}"
public_ip_address_id = "${azurerm_public_ip.test.id}"
}
backend_address_pool {
name = "${local.backend_address_pool_name}"
}
backend_http_settings {
name = "${local.http_setting_name}"
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 1
}
http_listener {
name = "${local.listener_name}"
frontend_ip_configuration_name = "${local.frontend_ip_configuration_name}"
frontend_port_name = "${local.frontend_port_name}"
protocol = "Http"
}
request_routing_rule {
name = "${local.request_routing_rule_name}"
rule_type = "Basic"
http_listener_name = "${local.listener_name}"
backend_address_pool_name = "${local.backend_address_pool_name}"
backend_http_settings_name = "${local.http_setting_name}"
}
}
The backend_address_pool block supports:
name - (Required) User defined name for a backend address pool.
ip_addresses - (Optional) List of public IPAdresses, or internal IP
addresses in a backend address pool.
fqdns - (Optional) List of FQDNs in a backend address pool
Terraform is a bit silly in this regard, and you have to use a separate proxy resource (which doesnt even exist in Azure) to configure this:
resource "azurerm_resource_group" "test" {
name = "example-resources"
location = "West Europe"
}
resource "azurerm_virtual_network" "test" {
name = "example-network"
address_space = ["10.0.0.0/16"]
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
}
resource "azurerm_subnet" "frontend" {
name = "frontend"
resource_group_name = "${azurerm_resource_group.test.name}"
virtual_network_name = "${azurerm_virtual_network.test.name}"
address_prefix = "10.254.0.0/24"
}
resource "azurerm_subnet" "backend" {
name = "backend"
resource_group_name = "${azurerm_resource_group.test.name}"
virtual_network_name = "${azurerm_virtual_network.test.name}"
address_prefix = "10.254.2.0/24"
}
resource "azurerm_public_ip" "test" {
name = "example-pip"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
allocation_method = "Dynamic"
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
backend_address_pool_name = "${azurerm_virtual_network.test.name}-beap"
frontend_port_name = "${azurerm_virtual_network.test.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.test.name}-feip"
http_setting_name = "${azurerm_virtual_network.test.name}-be-htst"
listener_name = "${azurerm_virtual_network.test.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.test.name}-rqrt"
}
resource "azurerm_application_gateway" "network" {
name = "example-appgateway"
resource_group_name = "${azurerm_resource_group.test.name}"
location = "${azurerm_resource_group.test.location}"
removed for brievity, check full example at the link below
}
# binding happens here
resource "azurerm_network_interface_application_gateway_backend_address_pool_association" "test" {
network_interface_id = "${azurerm_network_interface.test.id}"
ip_configuration_name = "testconfiguration1"
backend_address_pool_id = "${azurerm_application_gateway.test.backend_address_pool.0.id}"
}
Reading:
https://www.terraform.io/docs/providers/azurerm/r/network_interface_application_gateway_backend_address_pool_association.html
If someone will be looking at adding App Service to backend. This is following what you need to do:
backend_address_pool {
name = local.backend_address_pool_name
fqdns = ["${azurerm_function_app.function_app.name}.azurewebsites.net"]
}

Resources