I am trying to pass PFX certificate from local machine to code azure terraform but when i am applying this method in terraform its showing one error certificate or password invalid. i have tested certificate and password both but its working fine. i have attache code and error details below
Thanks-Onkar
resource "azurerm_application_gateway" "network" {
name = "my-application-gateway-12345"
resource_group_name = azurerm_resource_group.rg.name
location = "West US"
sku {
name = "Standard_Small"
tier = "Standard"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = "${azurerm_virtual_network.vnet.id}/subnets/${azurerm_subnet.sub1.name}"
}
ssl_certificate {
name = "certificate"
data = "${base64encode(filemd5("test.pfx"))}"
password = "*****"
}
frontend_port {
name = "feport"
port = 80
}
frontend_ip_configuration {
name = "feip"
public_ip_address_id = "${azurerm_public_ip.pip.id}"
}
backend_address_pool {
name = "beap"
}
backend_http_settings {
name = "be-htst"
cookie_based_affinity = "Disabled"
port = 443
protocol = "Https"
request_timeout = 1
}
http_listener {
name = "httplstn"
frontend_ip_configuration_name = "feip"
frontend_port_name = "feport"
protocol = "https"
ssl_certificate_name = "certificate"
}
request_routing_rule {
name = "rqrt"
rule_type = "Basic"
http_listener_name = "httplstn"
backend_address_pool_name = "beap"
backend_http_settings_name = "be-htst"
}
}
Update
after changes as per your instruction following error is occurred
For the error message, you could use ssl_certificate block like this
ssl_certificate {
name = "certificate"
data = "${base64encode(file("test.pfx"))}"
password = "*****"
}
Here is an example in that question.
Update
For the error message, you could use the filebase64 function to obtain the Base64 encoded contents.
change it to
ssl_certificate {
name = "certificate"
data = "${filebase64("test.pfx")}"
password = "xxxx"
}
Below is the module for application gateway:
locals {
backend_address_pool_name = format("appgwbeap-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
frontend_port_name = format("appgwfeport-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
frontend_ip_configuration_name = format("appgwfeip-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
http_setting_name = format("appgwhtst-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
listener_name = format("appgwhttplstnr-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
request_routing_rule_name = format("appgwrqrt-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
redirect_configuration_name = format("appgwrdrcfg-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
}
resource "azurerm_application_gateway" "appgw" {
name = format("appgw-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
resource_group_name = var.rg_name
location = var.location
sku {
name = var.sku_name
tier = var.sku_tier
capacity = var.sku_capacity
}
gateway_ip_configuration {
name = format("appgwipcfg-%[1]s-%[2]s%[3]sweb-gw",var.project_code,var.env,var.zone)
subnet_id = var.subnet_id
}
frontend_port {
name = "appgwfeport-app1-uatizweb-gw"
port = "443"
}
frontend_port {
name = "appgwfeport-app2-uatizweb-gw"
port = "443"
}
ssl_certificate {
name = "UAT-APP1-APPGW-SSL-CERT-SGCORE-12Jan21-12Jan23"
data = filebase64("./certificates/web.app1.sso.gwwu.xxx.com.de-12Jan2021.pfx")
password = "${var.app1_pfx_password}"
}
authentication_certificate {
name = "UAT-APP1-APPGW-SSL-CERT-SGCORE-12Jan21-12Jan23"
data = file("./certificates/web_app1_sso_gwwu_xxx_com_de-12Jan21.cer")
}
ssl_certificate {
name = "UAT-APP2-APPGW-SSL-CERT-01Mar21"
data = filebase64("./certificates/selfsigned-app2-uat-01Mar21.pfx")
password = "${var.app1_pfx_password}"
}
authentication_certificate {
name = "UAT-APP2-APPGW-SSL-CERT-01Mar21"
data = file("./certificates/selfsigned-app2-uat-01Mar21.cer")
}
frontend_ip_configuration {
name = "${local.frontend_ip_configuration_name}"
subnet_id = var.subnet_id
private_ip_address = var.frontend_private_ip
private_ip_address_allocation = "Static"
}
backend_address_pool {
name = "beap-path-app1-app"
#fqdns = var.fqdn_list
ip_addresses = ["10.xxx.xxx.36"]
}
backend_address_pool {
name = "beap-path-app2-app"
#fqdns = var.fqdn_list
ip_addresses = ["10.xxx.xxx.37"]
}
backend_http_settings {
name = "behs-path-app1-app"
cookie_based_affinity = var.backend_cookie_based_affinity
affinity_cookie_name = "ApplicationGatewayAffinity"
path = var.backend_path
port = "443"
#probe_name = "probe-app1"
protocol = "Https"
request_timeout = var.backend_request_timeout
authentication_certificate {
name = "UAT-APP1-APPGW-SSL-CERT-SGCORE-12Jan21-12Jan23"
}
}
backend_http_settings {
name = "behs-path-app2-app"
cookie_based_affinity = var.backend_cookie_based_affinity
affinity_cookie_name = "ApplicationGatewayAffinity"
path = var.backend_path
port = "443"
#probe_name = "probe-app2"
protocol = "Https"
request_timeout = var.backend_request_timeout
authentication_certificate {
name = "UAT-APP2-APPGW-SSL-CERT-01Mar21"
}
}
http_listener {
name = "appgwhttplsnr-app1-uatizweb-gw"
frontend_ip_configuration_name = "${local.frontend_ip_configuration_name}"
frontend_port_name = "appgwfeport-app1-uatizweb-gw"
protocol = "Https"
ssl_certificate_name = "UAT-APP1-APPGW-SSL-CERT-SGCORE-12Jan21-12Jan23"
require_sni = true
host_name = "web.app1.sso.gwwu.xxx.com.de"
}
http_listener {
name = "appgwhttplsnr-app2-uatizweb-gw"
frontend_ip_configuration_name = "${local.frontend_ip_configuration_name}"
frontend_port_name = "appgwfeport-app2-uatizweb-gw"
ssl_certificate_name = "UAT-APP2-APPGW-SSL-CERT-01Mar21"
require_sni = true
protocol = "Https"
host_name = "web.app2.sso.gwwu.xxx.com.de"
}
request_routing_rule {
name = "appgwrqrt-app2-uatizweb-gw"
rule_type = var.backend_rule_type
http_listener_name = "appgwhttplsnr-app2-uatizweb-gw"
backend_address_pool_name = "beap-path-app2-app"
backend_http_settings_name = "behs-path-app2-app"
}
request_routing_rule {
name = "appgwrqrt-app1-uatizweb-gw"
rule_type = var.backend_rule_type
http_listener_name = "appgwhttplsnr-app1-uatizweb-gw"
backend_address_pool_name = "beap-path-app1-app"
backend_http_settings_name = "behs-path-app1-app"
}
}
Below is the main.tf that calls the module:
module "app_gateway" {
source = "../../../modules/appgateway"
rg_name = var.rg_name
agency = local.agency
project_code = local.project_code
env = var.env
zone = var.zone
tier = "appgw"
location = local.location
vnet_name = var.vnet_name
subnet_id = module.agw_subnet.subnet_id
sku_name = var.appgw_sku_name
sku_capacity = var.appgw_sku_capacity
frontend_private_ip = var.appgw_frontend_ip
frontend_port = var.frontend_port
frontend_protocol = var.frontend_protocol
app1_pfx_password = "${var.app1_pfx_password}"
backend_protocol = var.backend_protocol
backend_port = var.backend_port
backend_path = "/"
providers = {
azurerm = azurerm.corpapps
}
}
Related
I have created a application gateway, WAF policy, public IP via terraform.
From Azure GUI I have created a Key vault in which I have uploaded the pfx certificate also I have created managed identity and granted full access to azure key vault.
I am trying to create a additional https listener and calling the certificate stored in the keyvault via data block but somehow landing in this error .
Note: Kayvault , managed identity , appgw, waf policy are all in same region.
Error :
│ Error: updating Application Gateway: (Name "abc-xyz-Nonprod-test-us6-Extappgw0001" / Resource Group "xyz-network-vnet-devtest"): network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidResourceReference" Message="Resource /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xyz-network-vnet-devtest/providers/Microsoft.Network/applicationGateways/abc-xyz-Nonprod-test-us6-Extappgw0001/sslCertificates/firepfx referenced by resource /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xyz-network-vnet-devtest/providers/Microsoft.Network/applicationGateways/abc-xyz-Nonprod-test-us6-Extappgw0001/httpListeners/External_app_gtw_nonprod_backend_listener_https was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[]
│
│ with azurerm_application_gateway.abc-xyz-Nonprod-test-us6-Extappgw0001,
│ on abc-xyz-Nonprod-test-us6-Extappgw0001.tf line 102, in resource "azurerm_application_gateway" "abc-xyz-Nonprod-test-us6-Extappgw0001":
│ 102: resource "azurerm_application_gateway" "abc-xyz-Nonprod-test-us6-Extappgw0001"
code
terraform {
backend "azurerm" {
storage_account_name = "abccloudlbstorage"
resource_group_name = "xyz-NETENG-AppResources-Prod"
container_name = "testlb"
tenant_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
subscription_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
key = "abc-xyz-Nonprod-test-us6-Extappgw0001.tfstate"
}
}
provider "azurerm" {
features {}
}
data "azurerm_client_config" "current" {}
data "azurerm_subnet" "abc-xyz-devtest-us6-vnet00002-sub00001-AppGW" {
name = "abc-xyz-devtest-us6-vnet00002-sub00001-AppGW"
resource_group_name = "xyz-network-vnet-devtest"
virtual_network_name = "abc-xyz-devtest-us6-vnet00002"
}
data "azurerm_user_assigned_identity" "test-appgw-identity-us6"{
name = "test-appgw-identity-us6"
resource_group_name = "xyz-network-vnet-devtest"
}
data "azurerm_key_vault" "xyz-network-kv" {
name = "xyz-network-kv"
resource_group_name = "xyz-network-vnet-devtest"
}
data "azurerm_key_vault_certificate" "firepfx" {
name = "firepfx"
key_vault_id = data.azurerm_key_vault.xyz-network-kv.id
}
resource "azurerm_public_ip" "abc-test-us6-nonprod-FE0001" {
name = "abc-test-us6-nonprod-FE0001"
resource_group_name = "xyz-network-vnet-devtest"
location = "eastus2"
allocation_method = "Static"
sku = "Standard"
zones = ["1", "2", "3"]
tags = {
BusinessUnit = "enterprise-management"
LineOfBusiness = "xyz"
}
}
resource "azurerm_web_application_firewall_policy" "abc-test-us6-nonprod-WFW0001" {
name = "abc-test-us6-nonprod-WFW0001"
resource_group_name = "xyz-network-vnet-devtest"
location = "eastus2"
tags = {
BusinessUnit = "enterprise-management"
LineOfBusiness = "xyz"
}
custom_rules {
name = "Rule1"
priority = 1
rule_type = "MatchRule"
match_conditions {
match_variables {
variable_name = "RemoteAddr"
}
operator = "IPMatch"
negation_condition = false
match_values = ["8.8.8.8"]
}
action = "Block"
}
policy_settings {
enabled = true
mode = "Prevention"
request_body_check = true
file_upload_limit_in_mb = 100
max_request_body_size_in_kb = 128
}
managed_rules {
exclusion {
match_variable = "RequestHeaderNames"
selector = "x-company-secret-header"
selector_match_operator = "Equals"
}
managed_rule_set {
type = "OWASP"
version = "3.2"
}
}
}
resource "azurerm_application_gateway" "abc-xyz-Nonprod-test-us6-Extappgw0001" {
name = "abc-xyz-Nonprod-test-us6-Extappgw0001"
resource_group_name = "xyz-network-vnet-devtest"
location = "eastus2"
zones = ["1", "2", "3"]
firewall_policy_id = azurerm_web_application_firewall_policy.abc-test-us6-nonprod-WFW0001.id
tags = {
BusinessUnit = "enterprise-management"
LineOfBusiness = "xyz"
}
sku {
name = "WAF_v2"
tier = "WAF_v2"
}
autoscale_configuration {
min_capacity = 2
max_capacity = 10
}
gateway_ip_configuration {
name = "abc-test-us6-nonprod-GIP0001"
subnet_id = data.azurerm_subnet.abc-xyz-devtest-us6-vnet00002-sub00001-AppGW.id
}
frontend_port {
name = "abc-us6-gpt-nonprod-PRT-FE0001"
port = 80
}
frontend_ip_configuration {
name = "abc-test-us6-nonprod-CFG-FE0001"
public_ip_address_id = azurerm_public_ip.abc-test-us6-nonprod-FE0001.id
}
frontend_ip_configuration {
name = "abc-test-us6-nonprod-CFG-FE0002"
subnet_id = data.azurerm_subnet.abc-xyz-devtest-us6-vnet00002-sub00001-AppGW.id
private_ip_address = "10.46.72.200"
private_ip_address_allocation = "Static"
}
backend_address_pool {
name = "External_app_gtw_nonprod_backend"
}
backend_http_settings {
name = "External_app_gtw_nonprod_http_setting"
cookie_based_affinity = "Disabled"
path = "/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = "External_app_gtw_nonprod_backend_listener"
frontend_ip_configuration_name = "abc-test-us6-nonprod-CFG-FE0001"
frontend_port_name = "abc-us6-gpt-nonprod-PRT-FE0001"
protocol = "Http"
}
request_routing_rule {
name = "External_app_gtw_nonprod_RR"
rule_type = "Basic"
http_listener_name = "External_app_gtw_nonprod_backend_listener"
backend_address_pool_name = "External_app_gtw_nonprod_backend"
backend_http_settings_name = "External_app_gtw_nonprod_http_setting"
priority = 1
}
frontend_port {
name = "abc-us6-gpt-nonprod-PRT-FE00011"
port = 443
}
backend_http_settings {
name = "External_app_gtw_nonprod_https_setting"
cookie_based_affinity = "Disabled"
path = "/"
port = 443
protocol = "Https"
request_timeout = 60
host_name = "irms.abc.com"
}
http_listener {
name = "External_app_gtw_nonprod_backend_listener_https"
frontend_ip_configuration_name = "abc-test-us6-nonprod-CFG-FE0001"
frontend_port_name = "abc-us6-gpt-nonprod-PRT-FE00011"
protocol = "Https"
ssl_certificate_name = data.azurerm_key_vault_certificate.firepfx.name
}
identity {
type = "UserAssigned"
identity_ids = [data.azurerm_user_assigned_identity.test-appgw-identity-us6.id]
}
request_routing_rule {
name = "External_app_gtw_nonprod_https"
rule_type = "Basic"
http_listener_name = "External_app_gtw_nonprod_backend_listener_https"
backend_address_pool_name = "External_app_gtw_nonprod_backend"
backend_http_settings_name = "External_app_gtw_nonprod_https_setting"
priority = 3
}
}
For Application Gateway, you have to create an ssl_certificate block that references the Key Vault secret ID under the key_vault_secret_id property. Then your listener will reference the name of this ssl_certificate resource instead of the locals variable you declared.
ssl_certificate {
name = "cert2023"
key_vault_secret_id "https://mykv.vault.azure.net/secrets/cert2023"
}
I am working on learning Terraform and Azure Web Services. After following a series of tutorials, I've been working on getting an Azure Container Instance setup that talks to a CosmosDB instance within a virtual network, and I want an Application Gateway setup that will allow HTTP connections to the Azure Container Instance.
Currently, when I call the IP address assigned to the Application Gateway, I receive a 502 Bad Gateway. I've verified that the image I'm running in the Azure Container Instance works locally. I have a feeling that the issues I'm facing are in relation to the back-end address pool I've configured, and possibly an issue with the rules I've setup in my network security group (nsg-myapp).
I was wondering if someone could look at my Terraform and identify what I've not configured correctly? The closest question I found similar to my scenario on StackOverflow as this unresolved question from last year.
network.tf
resource "azurerm_virtual_network" "myappdb" {
name = "myappdb-vnet"
address_space = ["10.7.0.0/16"]
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.2.0/24"]
service_endpoints = ["Microsoft.AzureCosmosDB"]
delegation {
name = "acidelegationservice"
service_delegation {
name = "Microsoft.ContainerInstance/containerGroups"
actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
}
}
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "frontend" {
name = "myapp-frontend"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.0.0/24"]
}
resource "azurerm_network_security_group" "nsg-myapp" {
name = "nsg-aci"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
security_rule {
name = "from-gateway-subnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [22, 80, 443, 445, 8000]
source_address_prefixes = azurerm_subnet.internal.address_prefixes
destination_address_prefix = azurerm_subnet.internal.address_prefixes[0]
}
security_rule {
name = "DenyAllInBound-Override"
priority = 900
direction = "Inbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "to-internet"
priority = 100
direction = "Outbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [80, 443, 445]
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "DenyAllOutBound-Override"
priority = 900
direction = "Outbound"
access = "Deny"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
resource "azurerm_subnet_network_security_group_association" "sn-nsg-aci" {
subnet_id = azurerm_subnet.internal.id
network_security_group_id = azurerm_network_security_group.nsg-myapp.id
}
resource "azurerm_network_profile" "containergroup_profile" {
name = "acg-profile"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
container_network_interface {
name = "acg-nic"
ip_configuration {
name = "aciipconfig"
subnet_id = azurerm_subnet.internal.id
}
}
}
resource "azurerm_public_ip" "myappip" {
name = "myappip"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
allocation_method = "Static"
sku = "Standard"
}
locals {
backend_address_pool_name = "${azurerm_virtual_network.myappdb.name}-beap"
frontend_port_name = "${azurerm_virtual_network.myappdb.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.myappdb.name}-feip"
http_setting_name = "${azurerm_virtual_network.myappdb.name}-be-htst"
listener_name = "${azurerm_virtual_network.myappdb.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.myappdb.name}-rqrt"
redirect_configuration_name = "${azurerm_virtual_network.myappdb.name}-rdrcfg"
}
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/path1/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
}
container.tf
resource "azurerm_container_group" "tf_cg_sampleapi" {
depends_on = [azurerm_cosmosdb_account.db]
name = "cg_myapp"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
network_profile_id = azurerm_network_profile.containergroup_profile.id
ip_address_type = "Private"
# dns_name_label = "sampleapitf"
os_type = "Linux"
identity {
type = "SystemAssigned"
}
container {
name = "myapp"
image = "sample/myapp"
cpu = 1
memory = 1
ports {
port = 80
protocol = "TCP"
}
ports {
port = 443
protocol = "TCP"
}
secure_environment_variables = {
"MYAPP_CONNECTION_STRING" = azurerm_cosmosdb_account.db.connection_strings[0]
}
}
}
I met the similar issue and in my case(containers on top of Azure App Service) I needed to put the depends_on block inside the application gateway resource creation with regards to app services being created in the first place. So in your case should be:
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/path1/"
port = 80
protocol = "Http"
request_timeout = 60
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
depends_on = [ azurerm_container_group.tf_cg_sampleapi, ]
}
I figured out the root cause of my 502 Gateway error was due to health checks not being setup / not working. Consequently, I setup custom probes that would go to an API endpoint to return a 200 OK response. Of course, I will configure this endpoint to actually check to see if I can connect to my services, but this was just a test to verify this was the issue.
I also removed the DenyAllInBound-Override and DenyAllOutBound-Override rules within my nsg-aci security group, as this was causing issues with my ACI to connect to my Cosmos DB.
This was my resulting network.tf and container.tf files:
network.tf
resource "azurerm_virtual_network" "myappdb" {
name = "myappdb-vnet"
address_space = ["10.7.0.0/16"]
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.2.0/24"]
service_endpoints = ["Microsoft.AzureCosmosDB"]
delegation {
name = "acidelegationservice"
service_delegation {
name = "Microsoft.ContainerInstance/containerGroups"
actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
}
}
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_subnet" "frontend" {
name = "myapp-frontend"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.myappdb.name
address_prefixes = ["10.7.0.0/24"]
}
resource "azurerm_network_security_group" "nsg-myapp" {
name = "nsg-aci"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
security_rule {
name = "from-gateway-subnet"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [22, 80, 443, 445, 8000]
source_address_prefixes = azurerm_subnet.internal.address_prefixes
destination_address_prefixes = azurerm_subnet.internal.address_prefixes
}
security_rule {
name = "to-internet"
priority = 100
direction = "Outbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [80, 443, 445]
source_address_prefix = "*"
destination_address_prefix = "*"
}
}
resource "azurerm_subnet_network_security_group_association" "sn-nsg-aci" {
subnet_id = azurerm_subnet.internal.id
network_security_group_id = azurerm_network_security_group.nsg-myapp.id
}
resource "azurerm_network_profile" "containergroup_profile" {
name = "acg-profile"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
container_network_interface {
name = "acg-nic"
ip_configuration {
name = "aciipconfig"
subnet_id = azurerm_subnet.internal.id
}
}
}
resource "azurerm_public_ip" "myappip" {
name = "myappip"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
allocation_method = "Static"
sku = "Standard"
}
locals {
backend_address_pool_name = "${azurerm_virtual_network.myappdb.name}-beap"
frontend_port_name = "${azurerm_virtual_network.myappdb.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.myappdb.name}-feip"
http_setting_name = "${azurerm_virtual_network.myappdb.name}-be-htst"
listener_name = "${azurerm_virtual_network.myappdb.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.myappdb.name}-rqrt"
redirect_configuration_name = "${azurerm_virtual_network.myappdb.name}-rdrcfg"
}
resource "azurerm_application_gateway" "network" {
name = "myapp-appgateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.frontend.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.myappip.id
}
backend_address_pool {
name = local.backend_address_pool_name
ip_addresses = [azurerm_container_group.tf_cg_sampleapi.ip_address]
}
probe {
interval = 60
timeout = 60
name = "status"
protocol = "Http"
path = "/api/status/"
unhealthy_threshold = 3
host = "127.0.0.1"
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/"
port = 80
protocol = "Http"
request_timeout = 60
probe_name = "status"
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
depends_on = [azurerm_container_group.tf_cg_sampleapi, ]
}
container.tf
resource "azurerm_container_group" "tf_cg_sampleapi" {
depends_on = [azurerm_cosmosdb_account.db]
name = "cg_myapp"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
network_profile_id = azurerm_network_profile.containergroup_profile.id
ip_address_type = "Private"
# dns_name_label = "sampleapitf"
os_type = "Linux"
container {
name = "myapp"
image = "sample/myapp"
cpu = 1
memory = 1
ports {
port = 80
protocol = "TCP"
}
ports {
port = 443
protocol = "TCP"
}
secure_environment_variables = {
"MYAPP_CONNECTION_STRING" = azurerm_cosmosdb_account.db.connection_strings[0]
}
}
}
I'm struggling to find the use or benefit of private DNS in web apps with application gateways.
Using terraform I am trying to create a web app with a private connection behind an application gateway.
I am using terraform to create this, and if I set the back end pool to the private IP address changing the host name to the .azurewebsite.net name then it works great.
However whenever I try to create a private DNS and point the back end pool to the web app using the back end pool DNS name, it gives me an error 502.
Here is the code I am using, I have read through a few guides and everyone I read people still seem to be pointing to the IP address rather than the DNS name. A push in the right direction would be appreciated!
resource "azurerm_virtual_network" "uks-network" {
name = "mrp-uks-tf-vnet"
location = azurerm_resource_group.uks-rg.location
resource_group_name = azurerm_resource_group.uks-rg.name
address_space = ["10.0.0.0/16"]
# dns_servers = ["10.0.0.4", "10.0.0.5"]
tags = {
environment = "staging"
Location = "UK South"
terraform = "True"
}
}
resource "azurerm_subnet" "mrp-uks-tf-sn-ag" {
name = "applicationgatewaysubnet"
resource_group_name = azurerm_resource_group.uks-rg.name
virtual_network_name = azurerm_virtual_network.uks-network.name
address_prefixes = ["10.0.1.0/24"]
enforce_private_link_endpoint_network_policies = "true"
}
resource "azurerm_subnet" "mrp-uks-tf-sn-ws" {
name = "websitesubnet"
resource_group_name = azurerm_resource_group.uks-rg.name
virtual_network_name = azurerm_virtual_network.uks-network.name
address_prefixes = ["10.0.2.0/24"]
enforce_private_link_endpoint_network_policies = "true"
}
resource "azurerm_private_dns_zone" "mrp-tf-uks-dns" {
name = "privatelink.azurewebsites.net"
resource_group_name = azurerm_resource_group.uks-rg.name
}
resource "azurerm_subnet" "mrp-uks-tf-sn-sql" {
name = "sqlsubnet"
resource_group_name = azurerm_resource_group.uks-rg.name
virtual_network_name = azurerm_virtual_network.uks-network.name
address_prefixes = ["10.0.3.0/24"]
enforce_private_link_endpoint_network_policies = "true"
}
resource "azurerm_private_dns_a_record" "uks-webapp-privatendpoint" {
name = "webappuks"
zone_name = azurerm_private_dns_zone.mrp-tf-uks-dns.name
resource_group_name = azurerm_resource_group.uks-rg.name
ttl = 300
records = [azurerm_private_endpoint.uks-webapp-privatendpoint.private_service_connection[0].private_ip_address]
}
resource "azurerm_private_dns_zone_virtual_network_link" "ukswebapp" {
name = "${azurerm_app_service.uks-webapp.name}-dnslink"
resource_group_name = azurerm_resource_group.uks-rg.name
private_dns_zone_name = azurerm_private_dns_zone.mrp-tf-uks-dns.name
virtual_network_id = azurerm_virtual_network.uks-network.id
registration_enabled = false
}
#Create Private Endpoints for UKS app service
resource "azurerm_private_endpoint" "uks-webapp-privatendpoint" {
name = "uks-webapp-privatendpoint"
location = azurerm_resource_group.uks-rg.location
resource_group_name = azurerm_resource_group.uks-rg.name
subnet_id = azurerm_subnet.mrp-uks-tf-sn-ws.id
private_service_connection {
name = "uks-webapp-privatendpoint-com"
private_connection_resource_id = azurerm_app_service.uks-webapp.id
is_manual_connection = false
subresource_names = ["sites"]
}
}
#Create Private Endpoints for UKS app service
resource "azurerm_private_endpoint" "uks-webapp-privatendpoint" {
name = "uks-webapp-privatendpoint"
location = azurerm_resource_group.uks-rg.location
resource_group_name = azurerm_resource_group.uks-rg.name
subnet_id = azurerm_subnet.mrp-uks-tf-sn-ws.id
private_service_connection {
name = "uks-webapp-privatendpoint-com"
private_connection_resource_id = azurerm_app_service.uks-webapp.id
is_manual_connection = false
subresource_names = ["sites"]
}
}
resource "azurerm_public_ip" "mrp-tf-uks-ag-pip" {
name = "mrp-tf-uks-ag-pip"
resource_group_name = azurerm_resource_group.uks-rg.name
location = azurerm_resource_group.uks-rg.location
allocation_method = "Static"
sku = "Standard"
}
locals {
backend_address_pool_name = "${azurerm_virtual_network.uks-network.name}-beap"
frontend_port_name = "${azurerm_virtual_network.uks-network.name}-feport"
frontend_ip_configuration_name = "${azurerm_virtual_network.uks-network.name}-feip"
http_setting_name = "${azurerm_virtual_network.uks-network.name}-be-htst"
listener_name = "${azurerm_virtual_network.uks-network.name}-httplstn"
request_routing_rule_name = "${azurerm_virtual_network.uks-network.name}-rqrt"
redirect_configuration_name = "${azurerm_virtual_network.uks-network.name}-rdrcfg"
}
resource "azurerm_application_gateway" "mrp-tf-uks-ag" {
name = "mrp-tf-uks-ag"
resource_group_name = azurerm_resource_group.uks-rg.name
location = azurerm_resource_group.uks-rg.location
sku {
name = "WAF_V2"
tier = "WAF_V2"
capacity = 1
}
waf_configuration {
enabled = "true"
firewall_mode = "Detection"
rule_set_type = "OWASP"
rule_set_version = "3.0"
}
gateway_ip_configuration {
name = "mrp-tf-uks-ag-ipc"
subnet_id = azurerm_subnet.mrp-uks-tf-sn-ag.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.mrp-tf-uks-ag-pip.id
}
backend_address_pool {
name = local.backend_address_pool_name
fqdns = ["${azurerm_app_service.uks-webapp.name}.azurewebsites.net"]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 1
pick_host_name_from_backend_address = true
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
}
After upgrading terraform to 0.14.0 and azurerm to 2.65.0 I got three errors regarding ssl certificate configuration in the application gateway section.
Error: expected "ssl_certificate.0.key_vault_secret_id" to not be an empty string, got
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
102: ssl_certificate {
Error: Computed attributes cannot be set
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
120: ssl_certificate {
Computed attributes cannot be set, but a value was set for
"ssl_certificate.0.id".
Error: Computed attributes cannot be set
on ~/modules/someservice/gateways.tf line 120, in resource "azurerm_application_gateway" "network":
120: ssl_certificate {
Computed attributes cannot be set, but a value was set for
"ssl_certificate.0.public_cert_data".
But key_vault_secret_id configuratiom is not existing in my code:
ssl_certificate {
name = local.certificate_name
data = filebase64("./ssl-cert/appgwcert.pfx")
password = "SecretPwd"
}
Snippet of the application gateway:
terraform {
required_version = "= 0.14.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "2.65.0"
}
}
}
provider "azurerm" {
features {}
}
# #################################################################
resource "azurerm_public_ip" "pub-ip" {
name = "appgw-pubIP"
resource_group_name = local.resour_group_name
location = local.resour_group_location
allocation_method = "Dynamic"
}
resource "azurerm_subnet" "subnet-01" {
name = "seubnet-app-gateway"
resource_group_name = local.resour_group_name
virtual_network_name = "vnet-app-gateway"
address_prefixes = ["10.21.0.0/24"]
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
resour_group_name = "app-gateway-test-01"
resour_group_location = "westus2"
backend_address_pool_name = "backend-pool-test-01"
frontend_port_name = "port_443"
frontend_ip_configuration_name = "appGwPublicFrontendIp"
http_setting_name = "http-settings-test-01"
listener_name = "https-listener-01"
request_routing_rule_name = "routrul-test-01"
certificate_name = "appgw-cert-test-01"
}
resource "azurerm_application_gateway" "network" {
name = "app-gateway-test-01"
resource_group_name = local.resour_group_name
location = local.resour_group_location
sku {
name = "Standard_V2"
tier = "Standard"
capacity = 2
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.subnet-01.id
}
frontend_port {
name = local.frontend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pub-ip.id
}
backend_address_pool {
name = local.backend_address_pool_name
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 20
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Https"
ssl_certificate_name = local.certificate_name
}
ssl_certificate {
name = local.certificate_name
# reference the dummy certificate
data = filebase64("./ssl-cert/appgwcert.pfx")
# this is only a dummy and not the actual certificate to be used thus no harm in storing the password
password = "SecretPwd"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
lifecycle {
ignore_changes = [ssl_certificate, http_listener]
}
}
I have no idea how to fix it?
There are few problems in your code, I have fixed those and tested on my environment.
The Public IP should have Sku = Standard and allocation_method = static.
The Sku name in application Gateway is standard_v2 but the
tier is Standard , it should be the same i.e. standard_v2 .
I have used terraform version 1.0.5 instead of using terraform
version 0.14.0.
So, after the changes the code is as below:
terraform {
required_version = "1.0.5"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "2.65.0"
}
}
}
provider "azurerm" {
features {}
}
# #################################################################
resource "azurerm_public_ip" "pub-ip" {
name = "appgw-pubIP"
resource_group_name = local.resour_group_name
location = local.resour_group_location
allocation_method = "Static"
sku = "standard"
}
resource "azurerm_subnet" "subnet-01" {
name = "seubnet-app-gateway"
resource_group_name = local.resour_group_name
virtual_network_name = "ansuman-vnet"
address_prefixes = ["172.31.10.0/24"]
}
# since these variables are re-used - a locals block makes this more maintainable
locals {
resour_group_name = "myresourcegroup"
resour_group_location = "westus2"
backend_address_pool_name = "backend-pool-test-01"
frontend_port_name = "port_443"
frontend_ip_configuration_name = "appGwPublicFrontendIp"
http_setting_name = "http-settings-test-01"
listener_name = "https-listener-01"
request_routing_rule_name = "routrul-test-01"
certificate_name = "appgw-cert-test-01"
}
resource "azurerm_application_gateway" "network" {
name = "app-gateway-test-01"
resource_group_name = local.resour_group_name
location = local.resour_group_location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 2
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.subnet-01.id
}
frontend_port {
name = local.frontend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pub-ip.id
}
backend_address_pool {
name = local.backend_address_pool_name
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 80
protocol = "Http"
request_timeout = 20
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Https"
ssl_certificate_name = local.certificate_name
}
ssl_certificate {
name = local.certificate_name
# reference the dummy certificate
data = filebase64("C:/powershellpfx.pfx")
# this is only a dummy and not the actual certificate to be used thus no harm in storing the password
password = "password#1234"
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
lifecycle {
ignore_changes = [ssl_certificate, http_listener]
}
}
Outputs:
I am trying to deploy a azure application gateway. I set the configuration as follow:
resource "azurerm_application_gateway" "demo-app-gateway" {
location = var.location
resource_group_name = azurerm_resource_group.rg-hri-testing-env.name
name = "demo-app-gateway"
autoscale_configuration {
max_capacity = 10
min_capacity = 2
}
frontend_port {
name = "port_443"
port = 443
}
sku {
name = "Standard_v2"
tier = "Standard_v2"
}
frontend_ip_configuration {
name = "appGwPublicFrontendIp"
public_ip_address_id = azurerm_public_ip.demo-app-gateway-public-ip.id
private_ip_address_allocation = "Dynamic"
}
backend_http_settings {
cookie_based_affinity = "Disabled"
name = "demo-http-settings"
port = 443
protocol = "Https"
host_name = "apim.test.com"
pick_host_name_from_backend_address = false
path = "/external/"
request_timeout = 20
probe_name = "demo-apim-probe"
trusted_root_certificate_names = ["demo-trusted-root-ca-certificate"]
}
probe {
interval = 30
name = "demo-apim-probe"
path = "/status-0123456789abcdef"
protocol = "Https"
timeout = 30
unhealthy_threshold = 3
pick_host_name_from_backend_http_settings = true
match {
body = ""
status_code = [
"200-399"
]
}
}
gateway_ip_configuration {
name = "appGatewayIpConfig"
subnet_id = azurerm_subnet.GatewaSubnet.id
}
backend_address_pool {
name = "demo-backend-pool"
}
http_listener {
frontend_ip_configuration_name = "appGwPublicFrontendIp"
frontend_port_name = "port_443"
name = "demo-app-gateway-listener"
protocol = "Https"
require_sni = false
ssl_certificate_name = "demo-app-gateway-certificate"
}
ssl_certificate {
data = filebase64(var.ssl_certificate_path)
name = "demo-app-gateway-certificate"
password = var.ssl_certificate_password
}
trusted_root_certificate {
data = filebase64(var.ssl_certificate_path)
name = "demo-trusted-root-ca-certificate"
}
request_routing_rule {
http_listener_name = "demo-app-gateway-listener"
name = "demo-rule"
rule_type = "Basic"
backend_address_pool_name = "demo-backend-pool"
backend_http_settings_name = "demo-http-setting"
}
}
But when I run terraform apply I get this error.
Error: creating/updating Application Gateway: (Name "demo-app-gateway" / Resource Group "rg-hri-testing-apim"): network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="InvalidResourceReference" Message="Resource /subscriptions/my-sub/resourceGroups/rg-hri-testing-apim/providers/Microsoft.Network/applicationGateways/demo-app-gateway/backendHttpSettingsCollection/demo-http-setting referenced by resource /subscriptions/mysub/resourceGroups/rg-hri-testing-apim/providers/Microsoft.Network/applicationGateways/demo-app-gateway/requestRoutingRules/demo-rule was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[]
on app-gateway-main.tf line 1, in resource "azurerm_application_gateway" "demo-app-gateway":
1: resource "azurerm_application_gateway" "demo-app-gateway" {
The resource causing the error is the request_routing_rule not being found, but what it confuses me is that is looking for it before to create it?
Can anyone please help me to understand what am I doing wrong here?
Please if you need more infos, just let me know.
Thank you very much
Please check the Backend HTTP settings name which is being referenced by request routing rule block. You have to change it to demo-http-settings in request_routing_rule to resolve the error.
Issue:
You are using below as backend http setting :
backend_http_settings {
cookie_based_affinity = "Disabled"
name = "demo-http-settings"
port = 443
protocol = "Https"
host_name = "apim.test.com"
pick_host_name_from_backend_address = false
path = "/external/"
request_timeout = 20
probe_name = "demo-apim-probe"
trusted_root_certificate_names = ["demo-trusted-root-ca-certificate"]
}
But while referencing it in request request routing rule you are using :
request_routing_rule {
http_listener_name = "demo-app-gateway-listener"
name = "demo-rule"
rule_type = "Basic"
backend_address_pool_name = "demo-backend-pool"
backend_http_settings_name = "demo-http-setting"
As you have given the name of backend_http_setting_name = demo-http-settings and giving it as demo-http-setting in request_routing_rule. It will error out as it can't find the backend http setting.