ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup error when destroying resources with terraform - azure

Question

I am creating Azure infra using terraform. I am able to create AppGateway in gateway subnet. The AppGateway required NSG rule to all access on ports 65200 - 65535, I have added the NSG. I am able to communicate with app behind AppGateway. But my jenkins pipeline fails when I try to destroy the complete setup, it says -
Error: Deleting Security Rule: (Name "AllowGatewayManagerInbound" / Network Security Group
Name "gateway" / Resource Group "primary"): network.SecurityRulesClient#Delete: Failure
sending request: StatusCode=400 -- Original Error:
Code="ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup" Message="Network
security group /subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/networkSecurityGroups
/gateway blocks incoming internet traffic on ports 65200 - 65535 to subnet
/subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/virtualNetworks/primary/subnets/gateway,
associated with Application Gateway subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/applicationGateways/primary-centralus.
This is not permitted for Application Gateways that have V2 Sku." Details=[]
Terraform code to create subnet, NSG and create AppGateway.
resource "azurerm_network_security_group" "gateway" {
name = "gateway"
location = var.location
resource_group_name = azurerm_resource_group.app.name
tags = var.tags
}
resource "azurerm_network_security_rule" "gateway_allow_gateway_manager_https_inbound" {
name = "AllowGatewayManagerInbound"
description = "Allow Azure application GatewayManager on management ports"
resource_group_name = azurerm_network_security_group.gateway.resource_group_name
network_security_group_name = azurerm_network_security_group.gateway.name
priority = 2510
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
source_address_prefix = "GatewayManager"
destination_port_range = "65200-65535"
destination_address_prefix = "*"
}
module "app_gateway" {
source = "../../modules/app_gateway"
name = "${azurerm_resource_group.app.name}-${var.location}"
location = azurerm_resource_group.app.location
resource_group_name = azurerm_resource_group.app.name
vnet_subnet_id = azurerm_subnet.gateway.id
app_public_dns_zone = local.app_public_dns_zone
a_record_domain_name = local.a_record_subdomain
key_vault = local.key_vault
ssl_certificates = local.ssl_certificates
env = local.suffix
tags = var.tags
depends_on = [
azurerm_network_security_group.gateway
]
}
I have added depends_on relationship between AppGateway and NSG as AppGateway depends on NSG.
I need help to destry these resources using terraform.

Answer 1

• The ‘Destroy’ task through the terraform code that you are using is failing because inbound connectivity from the Jenkins pipeline is not possible through the NSG to the Azure resources, i.e., Application gateway in this case since the NSG is blocking the Jenkins pipeline access to the Azure resources on ports 65200 – 65535.
Thus, since you have deployed the ‘Application gateway’ in the ‘Gateway’ subnet and you have already allowed inbound network connectivity through the NSG to the application deployed behind the application gateway.
• Therefore, ensure that this allow rule’s priority is set higher than the deny rules for the same category. Also, allow TCP ports 65200 - 65535 for the application gateway v2 SKU with the destination subnet as ‘Any’ and source as ‘GatewayManager’ service tag for the communication between the Jenkin pipeline and the Azure Resource manager to happen.
Do check and ensure that the below rules in the NSG are set correctly: -
a) Outbound Internet connectivity can't be blocked. Default outbound rules in the NSG allow Internet connectivity.
b) Don't remove the default outbound rules.
c) Don't create other outbound rules that deny any outbound connectivity.
d) Traffic from the ‘AzureLoadBalancer’ tag with the destination subnet as Any must be allowed.
Finally, do check the priority for all the above stated rules and configurations for if the priority of the inbound rules is set higher than the deny rules, then they won’t be effective. Please find the below snapshot for your reference: -