Related
Hopefully someone can help there.
I am trying to integrate Azure Application Gateway with Key Vault using ARM template and getting an issue:
SecretIdSpecifiedIsInvalid: SecretId '==' specified in '/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourceGroups/rg-ProjectX-dev-infra/providers/Microsoft.Network/applicationGateways/appgw-ProjectX-dev/sslCertificates/appGwSslCert' is invalid. []
User assigned managed identity of Application Gateway has proper permissions ('Get' and 'List' under secrets and certificates) in Azure Key Vault.
Certificate is self-signed and generated in Azure Key Vault. It works as expected if I add certificate using Azure portal but it fails to add using ARM template.
The following guides were used during deployment:
TLS termination with Key Vault certificates and Pass sensitive values
Parameters.json file:
"app-gateway-httpsvaultCert": {
"reference": {
"keyVault": {
"id": "/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourceGroups/rg-projeX-dev-infra/providers/Microsoft.KeyVault/vaults/kv-ProjectX-dev"
},
"secretName": "zzz-zzz-zzz-zzz"
}
},
Defined parameter as secure string in the template file:
"app-gateway-httpsvaultCert": {
"type": "securestring",
"metadata": {
"description": "Secure access string from Azure Application Gateway to Key Vault."
}
},
Template.json file:
{
"type": "Microsoft.Network/applicationGateways",
"apiVersion": "2020-11-01",
"name": "[variables('app-gateway-name')]",
"location": "[parameters('location')]",
"tags": "[parameters('resource-Tags')]",
"dependsOn": [
"[resourceId('Microsoft.Insights/components', variables('app-insights-name'))]",
"[resourceId('Microsoft.Network/publicIPAddresses', variables('public-ip-name'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('vnet-name'))]",
"[resourceId('Microsoft.KeyVault/vaults', variables('kv-name'))]"
],
"identity": {
"type": "UserAssigned",
"userAssignedIdentities": {
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('appgw-managed-id'))]": {
}
}
},
"properties": {
"sku": {
"name": "Standard_v2",
"tier": "Standard_v2",
"capacity": "[parameters('app-gateway-capacity')]"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vnet-name'), 'appgw-subnet')]"
}
}
}
],
"sslCertificates": [
{
"name": "appGwSslCert",
"properties": {
"keyVaultSecretId": "[parameters('app-gateway-httpsvaultCert')]"
}
}
],
"trustedRootCertificates": [],
"frontendIPConfigurations": [
{
"name": "appGwPublicFrontendIp",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('public-ip-name'))]"
}
}
}
],
"frontendPorts": [
{
"name": "port_443",
"properties": {
"port": 443
}
}
],
"backendAddressPools": [
{
"name": "gatewayBackEnd",
"properties": {
"backendAddresses": [
{
"fqdn": "[concat(variables('apim-name'), '.azure-api.net')]"
},
{
"fqdn": "[concat(variables('fr-name'), '.cognitiveservices.azure.com')]"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "global-gateway-https-setting",
"properties": {
"port": 443,
"protocol": "Https",
"cookieBasedAffinity": "Disabled",
"pickHostNameFromBackendAddress": true,
"requestTimeout": 20,
"probe": {
"id": "[resourceId('Microsoft.Network/applicationGateways/probes', variables('app-gateway-name'), 'global-gateway-probe')]"
}
}
}
],
"httpListeners": [
{
"name": "global-listener-https",
"properties": {
"frontendIPConfiguration": {
"id": "[resourceId('Microsoft.Network/applicationGateways/frontEndIPConfigurations', variables('app-gateway-name'), 'appGwPublicFrontendIp')]"
},
"frontendPort": {
"id": "[resourceId('Microsoft.Network/applicationGateways/frontEndPorts', variables('app-gateway-name'), 'port_443')]"
},
"protocol": "Https",
"sslCertificate": {
"id": "[resourceId('Microsoft.Network/applicationGateways/sslCertificates', variables('app-gateway-name'), 'appGwSslCert')]"
},
"hostNames": [],
"requireServerNameIndication": false
}
}
],
"urlPathMaps": [],
"requestRoutingRules": [
{
"name": "global-routing-rule",
"properties": {
"ruleType": "Basic",
"httpListener": {
"id": "[resourceId('Microsoft.Network/applicationGateways/httpListeners', variables('app-gateway-name'), 'global-listener-https')]"
},
"backendAddressPool": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendAddressPools', variables('app-gateway-name'), 'gatewayBackEnd')]"
},
"backendHttpSettings": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendHttpSettingsCollection', variables('app-gateway-name'), 'global-gateway-https-setting')]"
}
}
}
],
"probes": [
{
"name": "global-gateway-probe",
"properties": {
"protocol": "Https",
"port": 443,
"path": "/status-0123456789abcdef",
"interval": 30,
"timeout": 30,
"unhealthyThreshold": 3,
"pickHostNameFromBackendHttpSettings": true,
"minServers": 0
}
}
],
"rewriteRuleSets": [],
"redirectConfigurations": [],
"privateLinkConfigurations": [],
"sslPolicy": {
"policyType": "Predefined",
"policyName": "AppGwSslPolicy20170401S"
},
"enableHttp2": true
}
},
An issue has been fixed changing template.json and parameters.json files accordingly:
Template.json:
"app-gateway-httpsvaultCert": {
"type": "String",
"defaultValue": "https://[KeyVaultName].vault.azure.net/secrets/[CertName]",
"metadata": {
"description": "The base-64 encoded SSL certificate PFX data. Must be supplied via a parameters file references to a Key Vault / Secret Name."
}
}
Parameters.json:
"app-gateway-httpsvaultCert": {
"value":
"https://[KeyVaultName].vault.azure.net/secrets/[CertName]"
}
I am trying to configure Azure Application Gateway with Basic Rule. For my Frontend IP, I have created set DNS name to whatever.canadacentral.cloudapp.azure.com and uploaded a self-signed certificate. When I hit https:// everything works correctly however when I go to https://whatever.canadacentral.cloudapp.azure.com it returns 307 redirecting me to my backend pool https://whatever.azurewebsites.net/
Is this something to do with canadacentral.cloudapp.azure.com and I need to provide custom DNS?
Here's my template for Application Gateway:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"applicationGateways_ExampleDev_name": {
"defaultValue": "ExampleDev",
"type": "String"
},
"virtualNetworks_Ex_DEV_externalid": {
"defaultValue": "/subscriptions/xxx/resourceGroups/Example-Ex-DEV/providers/Microsoft.Network/virtualNetworks/Ex-DEV",
"type": "String"
},
"publicIPAddresses_ExampleDevIP_externalid": {
"defaultValue": "/subscriptions/xxx/resourceGroups/Example-Ex-DEV/providers/Microsoft.Network/publicIPAddresses/ExampleDevIP",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Network/applicationGateways",
"apiVersion": "2019-09-01",
"name": "[parameters('applicationGateways_ExampleDev_name')]",
"location": "canadacentral",
"properties": {
"sku": {
"name": "WAF_v2",
"tier": "WAF_v2"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[concat(parameters('virtualNetworks_Ex_DEV_externalid'), '/subnets/default')]"
}
}
}
],
"sslCertificates": [
{
"name": "ApplicationGateway",
"properties": {}
}
],
"trustedRootCertificates": [],
"frontendIPConfigurations": [
{
"name": "appGwPublicFrontendIp",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[parameters('publicIPAddresses_ExampleDevIP_externalid')]"
}
}
}
],
"frontendPorts": [
{
"name": "port_80",
"properties": {
"port": 80
}
},
{
"name": "port_443",
"properties": {
"port": 443
}
}
],
"backendAddressPools": [
{
"name": "ExampleApiDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleapi-dev.azurewebsites.net"
}
]
}
},
{
"name": "ExampleAuthDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleauth-dev.azurewebsites.net"
}
]
}
},
{
"name": "ExampleAppDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleapp-dev.azurewebsites.net"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "default",
"properties": {
"port": 80,
"protocol": "Http",
"cookieBasedAffinity": "Disabled",
"pickHostNameFromBackendAddress": true,
"affinityCookieName": "ApplicationGatewayAffinity",
"requestTimeout": 20,
"probe": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/probes/defaultxxx')]"
}
}
}
],
"httpListeners": [
{
"name": "public-https",
"properties": {
"frontendIPConfiguration": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/frontendIPConfigurations/appGwPublicFrontendIp')]"
},
"frontendPort": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/frontendPorts/port_443')]"
},
"protocol": "Https",
"sslCertificate": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/sslCertificates/ApplicationGateway')]"
},
"hostNames": [],
"requireServerNameIndication": false
}
}
],
"urlPathMaps": [],
"requestRoutingRules": [
{
"name": "basic",
"properties": {
"ruleType": "Basic",
"httpListener": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/httpListeners/public-https')]"
},
"backendAddressPool": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/backendAddressPools/ExampleApiDev')]"
},
"backendHttpSettings": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/backendHttpSettingsCollection/default')]"
}
}
}
],
"probes": [
{
"name": "default07a3e3ac-3c07-40f6-ad80-837f4cdd1009",
"properties": {
"protocol": "Http",
"path": "/swagger/index.html",
"interval": 30,
"timeout": 30,
"unhealthyThreshold": 3,
"pickHostNameFromBackendHttpSettings": true,
"minServers": 0,
"match": {
"statusCodes": [
"200-399"
]
}
}
}
],
"rewriteRuleSets": [],
"redirectConfigurations": [],
"webApplicationFirewallConfiguration": {
"enabled": true,
"firewallMode": "Prevention",
"ruleSetType": "OWASP",
"ruleSetVersion": "3.0",
"disabledRuleGroups": [],
"exclusions": [],
"requestBodyCheck": true,
"maxRequestBodySizeInKb": 128,
"fileUploadLimitInMb": 50
},
"enableHttp2": false,
"autoscaleConfiguration": {
"minCapacity": 0,
"maxCapacity": 2
}
}
}
]
}
In this case, for application gateway V2, you have two solutions from this document.
Rewrite the location header
Set the host name in the location header to the application gateway's
domain name. To do this, create a rewrite rule with a condition that
evaluates if the location header in the response contains
azurewebsites.net. It must also perform an action to rewrite the
location header to have the application gateway's host name.
Use a custom domain name
In this way, you must own a custom domain and add custom domain in app servvice, see Map an existing custom DNS name to Azure App Service. You could follow this process:
I'm creating an ARM template that has rules for FTP traffic. I'd like to use the copy command within the probes property of the load balancer to create probes for a range of ports, but I also have other probes that do not need to be repeated (fall outside of the copy). When I try to add both, the ARM template says it's invalid because the probes property is already present. Is it possible to use the "copy" syntax in addition to other value or must the "copy" be the only ARM syntax that builds the entire list of array values?
{
"type": "Microsoft.Network/loadBalancers",
"sku": {
"name": "Standard",
"tier": "Regional"
},
"name": "[variables('lb-csl-private.name')]",
"apiVersion": "2018-10-01",
"location": "[resourceGroup().location]",
"scale": null,
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"privateIPAddress": "[variables('ip-lb-csl-private')]",
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vnet-csl.name'), variables('vnet-csl.subnets.loadBalancerName'))]"
}
}
}
],
"copy": [
{
"name": "probes",
"count": 10,
"input": {
"name": "[concat('probe-CSL-PASV-', copyIndex('probes'))]",
"properties": {
"protocol": "Tcp",
"port": "[copyIndex('probes')]",
"inervalInSeconds": 5,
"numberOfProbes": 2
}
}
}
],
"probes": [
{
"name": "probe-CSL-FTP",
"properties": {
"protocol": "Tcp",
"port": 21,
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
],
"inboundNatRules": [],
"outboundRules": [],
"inboundNatPools": []
}
}
it is possible to do that using variables:
"variables": {
"copy": [
{
"name": "probes",
"count": 10,
"input": {
"name": "[concat('probe-CSL-PASV-', copyIndex('probes'))]",
"properties": {
"protocol": "Tcp",
"port": "[copyIndex('probes')]",
"inervalInSeconds": 5,
"numberOfProbes": 2
}
}
}
],
"otherProbes": [
{
probe1
},
{
probe2
},
etc
]
}
and then, in your load balancer:
"probes": "[concat(variables('probes'), variables('otherProbes'))]"
I have created template1 which will deploy HDI cluster and template2 which will deploy Azure VM seperately.
Now I want to get the Head-node Private IP from cluster and pass it to Azure VM template for processing using ARM template.
How can I do so?
Considering this is the object you are getting from HDcluster:
{
"id": "xxx",
"name": "xxx",
"type": "Microsoft.HDInsight/clusters",
"location": "East US",
"etag": "xxx",
"tags": null,
"properties": {
"clusterVersion": "3.5.1000.0",
"osType": "Linux",
"clusterDefinition": {
"blueprint": "https://blueprints.azurehdinsight.net/spark-3.5.1000.0.9865375.json",
"kind": "SPARK",
"componentVersion": {
"Spark": "1.6"
}
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "ExtraLarge"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Medium"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
}
]
},
"provisioningState": "Succeeded",
"clusterState": "Running",
"createdDate": "2017-04-11T09:07:44.68",
"quotaInfo": {
"coresUsed": 20
},
"connectivityEndpoints": [
{
"name": "SSH",
"protocol": "TCP",
"location": "xxx.azurehdinsight.net",
"port": 22
},
{
"name": "HTTPS",
"protocol": "TCP",
"location": "xxx.azurehdinsight.net",
"port": 443
}
],
"tier": "standard"
}
}
I'm guessing this is the best output you can get, so you can use something like:
"outputs": {
"test": {
"type": "Object",
"value": "[reference(parameters('clusterName'),'2015-03-01-preview').connectivityEndpoints[0].location]"
}
}
This will get you an output of xxx.azurehdinsight.net
And you can either create a new deployment with this data or (just like I said) add RHEL VM to the same template and make it dependOn on the HDCluster deployment and reference the same thing as an input to VMextension.
I am trying to modify an ARM template that I have which deploys some VMs and defines some autoscale rules (you can see the full template at https://gist.github.com/jinky32/d80e0ab2137236ff262484193f93c946, it is based on the template at https://github.com/gbowerman/azure-myriad/tree/master/vmss-ubuntu-scale).
I am trying to add in some load balancer rules so that traffic is spread across the new VMs as they are generated in reponse to the autoscale rules that are defined.
When I run this template via Azure CLI I get no errors in terminal but the deployment fails. Digging into the error events I see two:
statusCode:BadRequest serviceRequestId:ef42ec66-600e-4fb9-b4e2-dc2c06dda79c statusMessage:{"error":{"code":"InvalidRequestFormat","message":"Cannot parse the request.","details":[{"code":"InvalidJsonReferenceFormat","message":"Reference Id cc2bepool is not formatted correctly. The Id is expected to reference resources of type loadBalancers/backendAddressPools. Path properties.loadBalancingRules[0].properties.backendAddressPool."}]}} responseBody:{"error":{"code":"InvalidRequestFormat","message":"Cannot parse the request.","details":[{"code":"InvalidJsonReferenceFormat","message":"Reference Id cc2bepool is not formatted correctly. The Id is expected to reference resources of type loadBalancers/backendAddressPools. Path properties.loadBalancingRules[0].properties.backendAddressPool."}]}}
and
statusCode:BadRequest statusMessage:{"error":{"code":"InvalidRequestFormat","message":"Cannot parse the request.","details":[{"code":"InvalidJsonReferenceFormat","message":"Reference Id cc2bepool is not formatted correctly. The Id is expected to reference resources of type loadBalancers/backendAddressPools. Path properties.loadBalancingRules[0].properties.backendAddressPool."}]}}
I've put some of the relevant variables below and have also included my loadbalancer object but I believe that the issue is related to how I am referencing backendAddressPool:
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"backendAddressPool": {
"id": "[variables('bePoolName')]"
},
but I'm confused because I refer to it the same way elsewhere. Any advice on how to do this correctly much appreciated.
"variables": {
....
"loadBalancerName": "[concat(parameters('vmssName'), 'lb')]",
"lbProbeID": "[concat(variables('lbID'),'/probes/tcpProbe')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbID": "[resourceId('Microsoft.Network/loadBalancers',variables('loadBalancerName'))]",
"natPoolName": "[concat(parameters('vmssName'), 'natpool')]",
"bePoolName": "[concat(parameters('vmssName'), 'bepool')]",
....
....
}
.....
.....
{
"type": "Microsoft.Network/loadBalancers",
"name": "[variables('loadBalancerName')]",
"location": "[variables('location')]",
"apiVersion": "[variables('networkApiVersion')]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]"
],
"properties": {
"frontendIPConfigurations": [
....
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
...
},
{
"name": "natpooltileserver",
....
},
{
"name": "natpool2",
....
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"backendAddressPool": {
"id": "[variables('bePoolName')]"
},
"protocol": "tcp",
"frontendPort": 80,
"backendPort": 80,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 5,
"probe": {
"id": "[variables('lbProbeID')]"
}
}
}
],
"probes": [
{
"name": "tcpProbe",
"properties": {
"protocol": "tcp",
"port": 80,
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
]
}
},
please go to the portal and open a support request to see what's wrong with your template edits.