Listeners and SSL certs are getting deleted while redeploying Application gateway with ARM template - arm-template

I am using below ARM template to create Application Gateway. When I deploy it for the first time everything works. When I redeploy it (as a part of continuous deployment in the same env) it fails and I see the Listeners get deleted which were present previously and the SSL certificate also gets deleted.
Is there any option I can update the sub-resources present in ARM template based on a criteria something like not to update the listener if SSL cert is already present. I am updating some of the properties like creating rules and probe after creation of application gateway and not using ARM template.
ARM Template for reference-
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"applicationgateway_platform": {
"defaultValue": "",
"type": "String",
"maxLength": 6
},
"applicationgateway_location_shortname": {
"defaultValue": "",
"type": "String",
"maxLength": 3
},
"applicationgateway_project": {
"defaultValue": "",
"type": "String",
"maxLength": 6
},
"applicationgateway_environment": {
"defaultValue": "",
"type": "String",
"maxLength": 7
},
"applicationgateway_uniqueid": {
"defaultValue": "1",
"type": "String",
"maxLength": "1"
},
"vnetName": {
"type": "string",
"metadata": {
"description": "Name of the Virtual Network"
}
},
"subnetName": {
"type": "string",
"metadata": {
"description": "Name of subnet"
}
},
"vnetResourceGroup": {
"type": "string",
"defaultValue": "[resourceGroup().name]",
"metadata": {
"description": "Name of Resource group where Vnet and subnet resides"
}
},
"applicationGatewayTier": {
"type": "string",
"allowedValues": [
"Standard_v2",
"WAF_v2"
],
"defaultValue": "WAF_v2",
"metadata": {
"description": "application gateway tier"
}
},
"frontendPort": {
"type": "int",
"defaultValue": 80,
"metadata": {
"description": "application gateway front end port"
}
},
"secureFrontendPort": {
"type": "int",
"defaultValue": 443,
"metadata": {
"description": "application gateway secure front end port"
}
},
"backendPort": {
"type": "int",
"defaultValue": 80,
"metadata": {
"description": "application gateway back end port"
}
},
"applicationGatewayAutoScaleMinimumCapacity": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "Minimum appgateway instance to be running always"
}
},
"applicationGatewayAutoScaleMaximumCapacity": {
"type": "int",
"defaultValue": 10,
"metadata": {
"description": "Maximum appgateway instance that it can scale up."
}
}
},
"variables": {
"basename": "[concat(parameters('applicationgateway_platform'), '-', parameters('applicationgateway_project'), '-', parameters('applicationgateway_location_shortname'), '-', parameters('applicationgateway_environment'))]",
"applicationGatewayName": "[concat(variables('basename'), '-ag-', parameters('applicationgateway_uniqueid'))]",
"publicIPAddressName": "[concat(variables('basename'),'-agip-',parameters('applicationgateway_uniqueid'))]",
"subnetRef": "[concat(resourceId(parameters('vnetResourceGroup'), 'Microsoft.Network/virtualNetworks', parameters('vnetName')), '/subnets/', parameters('subnetName'))]",
"publicIPRef": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"webApplicationFirewallConfigurationProperties": {
"enabled": true,
"firewallMode": "Detection",
"ruleSetType": "OWASP",
"ruleSetVersion": "3.0"
},
"apiVersion": "2019-09-01"
},
"resources": [
{
"apiVersion": "[variables('apiVersion')]",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('publicIPAddressName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard"
},
"zones": [],
"properties": {
"publicIPAllocationMethod": "Static",
"dnsSettings": {
"domainNameLabel": "[variables('applicationGatewayName')]"
}
}
},
{
"apiVersion": "[variables('apiVersion')]",
"name": "[variables('applicationGatewayName')]",
"type": "Microsoft.Network/applicationGateways",
"location": "[resourceGroup().location]",
"dependsOn": [
"[variables('publicIPRef')]"
],
"properties": {
"sku": {
"name": "[parameters('applicationGatewayTier')]",
"tier": "[parameters('applicationGatewayTier')]"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
],
"frontendIPConfigurations": [
{
"name": "appGatewayFrontendIP",
"properties": {
"PublicIPAddress": {
"id": "[variables('publicIPRef')]"
}
}
}
],
"frontendPorts": [
{
"name": "appGatewayFrontendPort",
"properties": {
"Port": "[parameters('frontendPort')]"
}
},
{
"name": "appGatewaySecurePort",
"properties": {
"Port": "[parameters('secureFrontendPort')]"
}
}
],
"backendAddressPools": [
{
"name": "appGatewayBackendPool",
"properties": {
"BackendAddresses": []
}
}
],
"backendHttpSettingsCollection": [
{
"name": "appGatewayBackendHttpSettings",
"properties": {
"Port": "[parameters('backendPort')]",
"Protocol": "Http",
"CookieBasedAffinity": "disabled",
"requestTimeout": 20
}
}
],
"httpListeners": [
{
"name": "appGatewayHttpListener",
"properties": {
"FrontendIpConfiguration": {
"Id": "[concat(resourceId('Microsoft.Network/applicationGateways', variables('applicationGatewayName')), '/frontendIPConfigurations/appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(resourceId('Microsoft.Network/applicationGateways', variables('applicationGatewayName')), '/frontendPorts/appGatewayFrontendPort')]"
},
"Protocol": "Http",
"SslCertificate": null
}
}
],
"requestRoutingRules": [
{
"Name": "basicRule",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', variables('applicationGatewayName')), '/httpListeners/appGatewayHttpListener')]"
},
"backendAddressPool": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', variables('applicationGatewayName')), '/backendAddressPools/appGatewayBackendPool')]"
},
"backendHttpSettings": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', variables('applicationGatewayName')), '/backendHttpSettingsCollection/appGatewayBackendHttpSettings')]"
}
}
}
],
"enableHttp2": false,
"sslCertificates": [],
"probes": [],
"autoscaleConfiguration": {
"minCapacity": "[parameters('applicationGatewayAutoScaleMinimumCapacity')]",
"maxCapacity": "[parameters('applicationGatewayAutoScaleMaximumCapacity')]"
},
"webApplicationFirewallConfiguration": "[if(equals(toUpper(parameters('applicationGatewayTier')), 'WAF_V2'), variables('webApplicationFirewallConfigurationProperties'), json('null'))]"
}
}
]
}

By default, Resource Manager deployment uses incremental mode. In incremental mode, Resource Manager leaves unchanged resources that exist in the resource group but aren't specified in the template.
However, when redeploying an existing resource in incremental mode,
the outcome is different. Specify all properties for the resource,
not just the ones you're updating. A common misunderstanding is to
think properties that aren't specified are left unchanged. If you
don't specify certain properties, Resource Manager interprets the
update as overwriting those values.
So, if you want some properties to leave unchanged when redeploying the template, you can specify certain properties (listeners for HTTPS, rules for HTTPS, SSL certificate) in your template. Here is a quickstart template for an end to end SSL with an application gateway that you can refer to.
To update a resource in an Azure Resource Manager template, you could follow the link for more details.
First, you must reference the resource once in the template to create
it and then reference the resource by the same name to update it
later. However, if two resources have the same name in a template,
Resource Manager throws an exception. To avoid this error, specify the
updated resource in a second template that's either linked or included
as a subtemplate using the Microsoft.Resources/deployments resource
type.
Second, you must either specify the name of the existing property to
change or a new name for a property to add in the nested template. You
must also specify the original properties and their original values.
If you fail to provide the original properties and values, Resource
Manager assumes you want to create a new resource and deletes the
original resource.

Related

Only one SFTP Server for one Azure Resource group possible?

Is it only possible to create one on-demand SFPT Server with one Resource group in Azure?
This is a link regards to SFPT on Azure. https://learn.microsoft.com/en-us/samples/azure-samples/sftp-creation-template/sftp-on-azure/
I tried to create a second SFPT in the same Resource group, but previous SFPT got replaced with the new one.
I tried Goolging on this one, but I was not able to find the answer, so I am posting this question here.
Yes we can deploy multiple SFTP server to our Azure resource group.
But the template you are using already they have declare default variables ,Instead of that we need to declare parameters as shown in below template, So that you can use the same template multiple times.
TEMPLATE:-
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"metadata": {
"_generator": {
"name": "bicep",
"version": "0.4.63.48766",
"templateHash": "17013458610905703770"
}
},
"parameters": {
"storageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"metadata": {
"description": "Storage account type"
},
"allowedValues": [
"Standard_LRS",
"Standard_ZRS",
"Standard_GRS"
]
},
"storageAccountPrefix": {
"type": "string",
"defaultValue": "sftpstg",
"metadata": {
"description": "Prefix for new storage account"
}
},
"fileShareName": {
"type": "string",
"defaultValue": "sftpfileshare",
"metadata": {
"description": "Name of file share to be created"
}
},
"sftpUser": {
"type": "string",
"defaultValue": "sftp",
"metadata": {
"description": "Username to use for SFTP access"
}
},
"sftpPassword": {
"type": "securestring",
"metadata": {
"description": "Password to use for SFTP access"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Primary location for resources"
}
},
"containerGroupDNSLabel": {
"type": "string",
"defaultValue": "[uniqueString(resourceGroup().id, deployment().name)]",
"metadata": {
"description": "DNS label for container group"
}
},
"sftpContainerGroupName": {
"type": "string",
"metadata": {
"description": "cngroup for container group"
}
},
"sftpContainerName": {
"type": "string",
"metadata": {
"description": "container name"
}
}
},
"functions": [],
"variables": {
"sftpContainerImage": "atmoz/sftp:debian",
"sftpEnvVariable": "[format('{0}:{1}:1001', parameters('sftpUser'), parameters('sftpPassword'))]",
"storageAccountName": "[take(toLower(format('{0}{1}', parameters('storageAccountPrefix'), uniqueString(resourceGroup().id))), 24)]"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"name": "[variables('storageAccountName')]",
"location": "[parameters('location')]",
"kind": "StorageV2",
"sku": {
"name": "[parameters('storageAccountType')]"
}
},
{
"type": "Microsoft.Storage/storageAccounts/fileServices/shares",
"apiVersion": "2019-06-01",
"name": "[toLower(format('{0}/default/{1}', variables('storageAccountName'), parameters('fileShareName')))]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]"
]
},
{
"type": "Microsoft.ContainerInstance/containerGroups",
"apiVersion": "2019-12-01",
"name": "[parameters('sftpContainerGroupName')]",
"location": "[parameters('location')]",
"properties": {
"containers": [
{
"name": "[parameters('sftpContainerName')]",
"properties": {
"image": "[variables('sftpContainerImage')]",
"environmentVariables": [
{
"name": "SFTP_USERS",
"secureValue": "[variables('sftpEnvVariable')]"
}
],
"resources": {
"requests": {
"cpu": 1,
"memoryInGB": 1
}
},
"ports": [
{
"port": 22,
"protocol": "TCP"
}
],
"volumeMounts": [
{
"mountPath": "[format('/home/{0}/upload', parameters('sftpUser'))]",
"name": "sftpvolume",
"readOnly": false
}
]
}
}
],
"osType": "Linux",
"ipAddress": {
"type": "Public",
"ports": [
{
"port": 22,
"protocol": "TCP"
}
],
"dnsNameLabel": "[parameters('containerGroupDNSLabel')]"
},
"restartPolicy": "OnFailure",
"volumes": [
{
"name": "sftpvolume",
"azureFile": {
"readOnly": false,
"shareName": "[parameters('fileShareName')]",
"storageAccountName": "[variables('storageAccountName')]",
"storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value]"
}
}
]
},
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]"
]
}
],
"outputs": {
"containerDNSLabel": {
"type": "string",
"value": "[format('{0}.{1}.azurecontainer.io', reference(resourceId('Microsoft.ContainerInstance/containerGroups', parameters('sftpContainerGroupName'))).ipAddress.dnsNameLabel, reference(resourceId('Microsoft.ContainerInstance/containerGroups', parameters('sftpContainerGroupName')), '2019-12-01', 'full').location)]"
}
}
}
Deployment details:-

Trying to set up diagnostic settings for an API arm template

I am new to ARM templates and I have the following issue:
I have an ARM template which creates an API Management service and I want this API Management service to use Log Analytics workspace in order to store it's logs there.
I have already created a log analytics workspace resource.
So, according to Microsoft documentation: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-manager-diagnostic-settings
the only thing I need to do is to edit the API management ARM template and include a new resource:
"type": "Microsoft.Insights/diagnosticSettings"
but when I do that, I get the following error:
"Values must be one of the following values......" getting a long long list.
Am I doing something wrong here?
thank you for your time!
If we want to create a diagnostic setting for an Azure API Management resource, we can add a resource of type Microsoft.ApiManagement/service/providers/diagnosticSettings to the template.
For example
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"publisherEmail": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The email address of the owner of the service"
},
"defaultValue": "test#gmail.com"
},
"publisherName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the owner of the service"
},
"defaultValue": "test",
},
"sku": {
"type": "string",
"defaultValue": "Developer",
"allowedValues": [
"Developer",
"Standard",
"Premium"
],
"metadata": {
"description": "The pricing tier of this API Management service"
}
},
"skuCount": {
"type": "string",
"defaultValue": "1",
"allowedValues": [
"1",
"2"
],
"metadata": {
"description": "The instance size of this API Management service."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"apiManagementServiceName": "[concat('myapiservice', uniqueString(resourceGroup().id))]"
},
"resources": [{
"type": "Microsoft.ApiManagement/service",
"apiVersion": "2019-12-01",
"name": "[variables('apiManagementServiceName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('sku')]",
"capacity": "[parameters('skuCount')]"
},
"properties": {
"publisherEmail": "[parameters('publisherEmail')]",
"publisherName": "[parameters('publisherName')]"
}
}, {
"type": "Microsoft.ApiManagement/service/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(variables('apiManagementServiceName'), '/Microsoft.Insights/', 'mytest')]",
"dependsOn": ["[resourceId('Microsoft.ApiManagement/service', variables('apiManagementServiceName'))]"],
"properties": {
"logs": [{
"category": "GatewayLogs",
"enabled": true,
}
],
"metrics": [{
"enabled": true,
"category": "AllMetrics"
}
],
"workspaceId": "<the resource id of workspace>",
}
}
]
}

HDInsight azure adls gen2 'InternalServerError' ARM Template deployment

Creating Azure HDinsight Spark cluster with ADLS Gen 2,Userassigned managed idnetity with StorageBlobdataOwner role.
Successfully assigned msi role to storage but getting error with HDInsight deployment(Internal server error)
Theres some issue near HDInsight cluster(Storage profile)resource code in the template i think. I could use some help here.Attached image below.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"clusterType": {
"type": "string",
"allowedValues": [ "hadoop", "hbase", "storm", "spark" ],
"metadata": {
"description": "The type of the HDInsight cluster to create."
}
},
"clusterName": {
"type": "string",
"metadata": {
"description": "The name of the HDInsight cluster to create."
}
},
"clusterLoginUserName": {
"type": "string",
"metadata": {
"description": "These credentials can be used to submit jobs to the cluster and to log into cluster dashboards."
}
},
"clusterLoginPassword": {
"type": "securestring",
"minLength": 10,
"metadata": {
"description": "The clusterloginpassword must be at least 10 characters in length and must contain at least one digit, one upper case letter, one lower case letter, and one non-alphanumeric character except (single-quote, double-quote, backslash, right-bracket, full-stop). Also, the password must not contain 3 consecutive characters from the cluster username or SSH username."
}
},
"sshUserName": {
"type": "string",
"metadata": {
"description": "These credentials can be used to remotely access the cluster and should not be same as clusterLoginUserName."
}
},
"sshPassword": {
"type": "securestring",
"minLength": 6,
"maxLength": 72,
"metadata": {
"description": "SSH password must be 6-72 characters long and must contain at least one digit, one upper case letter, and one lower case letter. It must not contain any 3 consecutive characters from the cluster login name"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"HeadNodeVirtualMachineSize": {
"type": "string",
"defaultValue": "Standard_D12_v2",
"allowedValues": [
"Standard_A4_v2",
"Standard_A8_v2",
"Standard_D3_v2",
"Standard_D4_v2",
"Standard_D5_v2",
"Standard_D12_v2",
"Standard_D13_v2"
],
"metadata": {
"description": "This is the headnode Azure Virtual Machine size, and will affect the cost. If you don't know, just leave the default value."
}
},
"WorkerNodeVirtualMachineSize": {
"type": "string",
"defaultValue": "Standard_D13_v2",
"allowedValues": [
"Standard_A4_v2",
"Standard_A8_v2",
"Standard_D1_v2",
"Standard_D2_v2",
"Standard_D3_v2",
"Standard_D4_v2",
"Standard_D5_v2",
"Standard_D12_v2",
"Standard_D13_v2"
],
"metadata": {
"description": "This is the workerdnode Azure Virtual Machine size, and will affect the cost. If you don't know, just leave the default value."
}
},
"clusterHeadNodeCount": {
"type": "int",
"defaultValue": 2,
"metadata": {
"description": "Number of worker nodes"
}
},
"clusterWorkerNodeCount": {
"type": "int",
"defaultValue": 4,
"metadata": {
"description": "Number of worker nodes"
}
},
"StorageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the Storage Account"
}
},
"StorageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Standard_ZRS",
"Standard_RA-GRS"
],
"metadata": {
"description": "Type of the Storage Account"
}
},
"filesystemname": {
"type": "string",
"metadata": {
"description": "Name of the container"
}
},
"UserAssignedIdentityName": {
"type": "string",
"metadata": {
"description": "Name of the User Assigned Identity"
}
}
},
"variables": {
"managedIdentityId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/',resourceGroup().name, '/providers/Microsoft.ManagedIdentity/userAssignedIdentities/', parameters('UserAssignedIdentityName'))]",
"StorageApiVersion": "2019-06-01",
"msiApiVersion": "2018-11-30",
"HDInsightApiVersion": "2015-03-01-preview",
"StorageBlobDataOwner": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', 'b7e6dc6d-f1e8-4753-8033-0f276bb0955b')]",
"StorageBlobDataContributor": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]"
},
"resources": [
{
"name": "[parameters('UserAssignedIdentityName')]",
"type": "Microsoft.ManagedIdentity/userAssignedIdentities",
"apiVersion": "[variables('msiApiVersion')]",
"location": "[resourceGroup().location]"
},
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "[variables('StorageApiVersion')]",
"name": "[parameters('StorageAccountName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('StorageAccountType')]"
},
"kind": "StorageV2",
"properties": {
"encryption": {
"keySource": "Microsoft.Storage",
"services": {
"blob": {
"enabled": true
},
"file": {
"enabled": true
}
}
},
"isHnsEnabled": true,
"supportsHttpsTrafficOnly": true
}
},
{
"type": "Microsoft.Storage/storageAccounts/providers/roleAssignments",
"apiVersion": "2018-01-01-preview",
"name": "[concat(parameters('StorageAccountName'),'/Microsoft.Authorization/',guid(subscription().subscriptionId))]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts',parameters('StorageAccountName'))]",
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities',parameters('UserAssignedIdentityName'))]"
],
"properties": {
"roleDefinitionId": "[variables('StorageBlobDataOwner')]",
"principalId": "[reference(variables('managedIdentityId'),variables('msiApiVersion')).principalId]"
}
},
{
"apiVersion": "[variables('HDInsightApiVersion')]",
"name": "[parameters('clusterName')]",
"type": "Microsoft.HDInsight/clusters",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts',parameters('StorageAccountName'))]",
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities',parameters('UserAssignedIdentityName'))]"
],
"properties": {
"clusterVersion": "4.0",
"osType": "Linux",
"tier": "standard",
"clusterDefinition": {
"kind": "[parameters('clusterType')]",
"componentVersion": {
"Spark": "2.3"
},
"configurations": {
"gateway": {
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "[parameters('clusterLoginUserName')]",
"restAuthCredential.password": "[parameters('clusterLoginPassword')]"
}
}
},
"identity": {
"type": "UserAssigned",
"userAssignedIdentities": {
"[variables('managedIdentityId')]": {}
}
},
"storageProfile": {
"storageaccounts": [
{
"name": "[concat(parameters('StorageAccountName'),'.blob.core.windows.net')]",
"isDefault": true,
"fileSystem": "[parameters('filesystemname')]",
"resourceId": "[reference(resourceId('Microsoft.Storage/storageAccounts',parameters('StorageAccountName')),variables('StorageApiVersion'))]",
"msiResourceId": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities',parameters('UserAssignedIdentityName')),variables('msiApiVersion'))]"
}
]
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"minInstanceCount": 1,
"targetInstanceCount": "[parameters('clusterHeadNodeCount')]",
"hardwareProfile": {
"vmSize": "[parameters('HeadNodeVirtualMachineSize')]"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "[parameters('sshUserName')]",
"password": "[parameters('sshPassword')]"
}
},
"virtualNetworkProfile": null,
"scriptActions": []
},
{
"name": "workernode",
"targetInstanceCount": "[parameters('clusterWorkerNodeCount')]",
"autoscale": {
"capacity": {
"minInstanceCount": 3,
"maxInstanceCount": 10
}
},
"hardwareProfile": {
"vmSize": "[parameters('WorkerNodeVirtualMachineSize')]"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "[parameters('sshUserName')]",
"password": "[parameters('sshPassword')]"
}
},
"virtualNetworkProfile": null,
"scriptActions": []
}
]
}
}
}
],
"outputs": {
"storage": {
"type": "object",
"value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('StorageAccountName')))]"
},
"cluster": {
"type": "object",
"value": "[reference(resourceId('Microsoft.HDInsight/clusters', parameters('clusterName')))]"
}
}
}
InternalServerError and Operation detail shows "Anerror has occured" and no other info
Update: Ensure that your storage account has the user-assigned identity with Storage Blob Data Contributor role permissions, otherwise cluster creation will fail.
If you are using Azure Data Lake Storage Gen2 and receive the error AmbariClusterCreationFailedErrorCode: "Internal server error occurred while processing the request. Please retry the request or contact support.".
To resolve this issue, open the Azure portal, go to your Storage account, and under Access Control (IAM), ensure that the Storage Blob Data Contributor or the Storage Blob Data Owner role has Assigned access to the User assigned managed identity for the subscription. See Set up permissions for the managed identity on the Data Lake Storage Gen2 account for detailed instructions.
Make sure you have followed the necessary steps to configure a Data Lake Storage gen2 account.
Reference: Use Azure Data Lake Storage Gen2 with Azure HDInsight clusters

Azure Kubernetes Service ARM template is not idempotent

I have created an ARM template to deploy an Azure Kubernetes Service instance, which I am trying to plug into a CI/CD pipeline in VSTS. On the first deployment, everything works as expected and the K8s cluster is created successfully. However, upon redeployment, the template fails the validation stage with the following error:
{
"message": "The template deployment 'Microsoft.Template' is not valid according to the validation procedure."
"details": [
{
"code":"PropertyChangeNotAllowed",
"message":"Provisioning of resource(s) for container service <cluster name> in resource group <resource group name> failed. Message:"
{
"code": "PropertyChangeNotAllowed",
"message": "Changing property 'linuxProfile.ssh.publicKeys.keyData' is not allowed.",
"target": "linuxProfile.ssh.publicKeys.keyData"
}
}
]
}
The template is therefore clearly not idempotent which completely dishonours the intended nature of ARM template deployments.
Has anyone managed to find a workaround for this?
The solution to this is to specify the SSH RSA Public Key as a template parameter and use it when configuring the Linux profile. I have posted my ARM template below:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"clusterName": {
"type": "string",
"metadata": {
"description": "The name of the Kubernetes cluster."
}
},
"location": {
"type": "string",
"metadata": {
"description": "The data center in which to deploy the Kubernetes cluster."
}
},
"dnsPrefix": {
"type": "string",
"metadata": {
"description": "DNS prefix to use with hosted Kubernetes API server FQDN."
}
},
"osDiskSizeGB": {
"defaultValue": 32,
"minValue": 0,
"maxValue": 1023,
"type": "int",
"metadata": {
"description": "Disk size (in GB) to provision for each of the agent pool nodes. This value ranges from 0 to 1023. Specifying 0 will apply the default disk size for that agentVMSize."
}
},
"agentCount": {
"defaultValue": 1,
"minValue": 1,
"maxValue": 50,
"type": "int",
"metadata": {
"description": "The number of agent nodes for the cluster."
}
},
"agentVMSize": {
"defaultValue": "Standard_D1_v2",
"type": "string",
"metadata": {
"description": "The size of the Virtual Machine."
}
},
"servicePrincipalClientId": {
"type": "securestring",
"metadata": {
"description": "The Service Principal Client ID."
}
},
"servicePrincipalClientSecret": {
"type": "securestring",
"metadata": {
"description": "The Service Principal Client Secret."
}
},
"osType": {
"defaultValue": "Linux",
"allowedValues": [
"Linux"
],
"type": "string",
"metadata": {
"description": "The type of operating system."
}
},
"kubernetesVersion": {
"defaultValue": "1.10.6",
"type": "string",
"metadata": {
"description": "The version of Kubernetes."
}
},
"enableOmsAgent": {
"defaultValue": true,
"type": "bool",
"metadata": {
"description": "boolean flag to turn on and off of omsagent addon"
}
},
"enableHttpApplicationRouting": {
"defaultValue": true,
"type": "bool",
"metadata": {
"description": "boolean flag to turn on and off of http application routing"
}
},
"networkPlugin": {
"defaultValue": "kubenet",
"allowedValues": [
"azure",
"kubenet"
],
"type": "string",
"metadata": {
"description": "Network plugin used for building Kubernetes network."
}
},
"enableRBAC": {
"defaultValue": true,
"type": "bool",
"metadata": {
"description": "Flag to turn on/off RBAC"
}
},
"logAnalyticsWorkspaceName": {
"type": "string",
"metadata": {
"description": "Name of the log analytics workspace which will be used for container analytics"
}
},
"logAnalyticsWorkspaceLocation": {
"type": "string",
"metadata": {
"description": "The data center in which the log analytics workspace is deployed"
}
},
"logAnalyticsResourceGroup": {
"type": "string",
"metadata": {
"description": "The resource group in which the log analytics workspace is deployed"
}
},
"vmAdminUsername": {
"type": "string",
"metadata": {
"description": "User name for the Linux Virtual Machines."
}
},
"sshRsaPublicKey": {
"type": "securestring",
"metadata": {
"description": "Configure all linux machines with the SSH RSA public key string. Your key should include three parts, for example: 'ssh-rsa AAAAB...snip...UcyupgH azureuser#linuxvm'"
}
}
},
"variables": {
"logAnalyticsWorkspaceId": "[resourceId(parameters('logAnalyticsResourceGroup'), 'Microsoft.OperationalInsights/workspaces', parameters('logAnalyticsWorkspaceName'))]",
"containerInsightsName": "[concat(parameters('clusterName'),'-containerinsights')]"
},
"resources": [
{
"type": "Microsoft.ContainerService/managedClusters",
"name": "[parameters('clusterName')]",
"apiVersion": "2018-03-31",
"location": "[parameters('location')]",
"properties": {
"kubernetesVersion": "[parameters('kubernetesVersion')]",
"enableRBAC": "[parameters('enableRBAC')]",
"dnsPrefix": "[parameters('dnsPrefix')]",
"addonProfiles": {
"httpApplicationRouting": {
"enabled": "[parameters('enableHttpApplicationRouting')]"
},
"omsagent": {
"enabled": "[parameters('enableOmsAgent')]",
"config": {
"logAnalyticsWorkspaceResourceID": "[variables('logAnalyticsWorkspaceId')]"
}
}
},
"agentPoolProfiles": [
{
"name": "agentpool",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "[parameters('osType')]",
"storageProfile": "ManagedDisks"
}
],
"linuxProfile": {
"adminUsername": "[parameters('vmAdminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[parameters('sshRsaPublicKey')]"
}
]
}
},
"servicePrincipalProfile": {
"clientId": "[parameters('servicePrincipalClientId')]",
"secret": "[parameters('servicePrincipalClientSecret')]"
},
"networkProfile": {
"networkPlugin": "[parameters('networkPlugin')]"
}
},
"dependsOn": [
"[concat('Microsoft.Resources/deployments/', 'SolutionDeployment')]"
]
},
{
"type": "Microsoft.Resources/deployments",
"name": "SolutionDeployment",
"apiVersion": "2017-05-10",
"resourceGroup": "[parameters('logAnalyticsResourceGroup')]",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2015-11-01-preview",
"type": "Microsoft.OperationsManagement/solutions",
"location": "[parameters('logAnalyticsWorkspaceLocation')]",
"name": "[variables('containerInsightsName')]",
"properties": {
"workspaceResourceId": "[variables('logAnalyticsWorkspaceId')]"
},
"plan": {
"name": "[variables('containerInsightsName')]",
"product": "OMSGallery/ContainerInsights",
"promotionCode": "",
"publisher": "Microsoft"
}
}
]
}
}
}
],
"outputs": {
"controlPlaneFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/managedClusters/', parameters('clusterName'))).fqdn]"
},
"sshMaster0": {
"type": "string",
"value": "[concat('ssh ', parameters('vmAdminUsername'), '#', reference(concat('Microsoft.ContainerService/managedClusters/', parameters('clusterName'))).fqdn, ' -A -p 22')]"
}
}
}

How do I deploy a Application Gateway with VMs behind it

I've been trying to deploy an Azure Application Gateway to front application I have on existing VMs and use hostnames for the pool selection. I started with this template from git https://github.com/Azure/azure-quickstart-templates/tree/master/201-application-gateway-multihosting based on the article https://github.com/Azure/azure-content/blob/master/articles/application-gateway/application-gateway-multi-site-overview.md
Here is the modifed tempate I used
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vnetAddressPrefix": {
"type": "string",
"defaultValue": "10.0.0.0/16",
"metadata": {
"description": "Address prefix for the Virtual Network"
}
},
"subnetPrefix": {
"type": "string",
"defaultValue": "10.0.0.0/28",
"metadata": {
"description": "Gateway Subnet prefix"
}
},
"skuName": {
"type": "string",
"allowedValues": [
"Standard_Small",
"Standard_Medium",
"Standard_Large"
],
"defaultValue": "Standard_Small",
"metadata": {
"description": "Sku Name"
}
},
"capacity": {
"type": "int",
"defaultValue": 4,
"metadata": {
"description": "Number of instances"
}
},
"backendIpAddress1": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 1"
}
},
"backendIpAddress2": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 2"
}
},
"backendIpAddress3": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 3"
}
},
"backendIpAddress4": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 4"
}
},
"backendIpAddress5": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 5"
}
},
"backendIpAddress6": {
"type": "string",
"metadata": {
"description": "IP Address for Backend Server 6"
}
},
"hostName1": {
"type": "string",
"metadata": {
"description": "HostName for listener 1"
}
},
"hostName2": {
"type": "string",
"metadata": {
"description": "HostName for listener 2"
}
},
"certData1": {
"type": "securestring",
"metadata": {
"description": "Base-64 encoded form of the .pfx file"
}
},
"certPassword1": {
"type": "securestring",
"metadata": {
"description": "Password for .pfx certificate"
}
}
},
"variables": {
"applicationGatewayName": "PortalGateway",
"publicIPAddressName": "PortalGatewayFrontendIP",
"virtualNetworkName": "PalitonNetworks-East-VirtualNetwork",
"subnetName": "GWSubnet1",
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
"subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]",
"publicIPRef": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"applicationGatewayID": "[resourceId('Microsoft.Network/applicationGateways',variables('applicationGatewayName'))]",
"apiVersion": "2015-06-15"
},
"resources": [
{
"apiVersion": "[variables('apiVersion')]",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('publicIPAddressName')]",
"location": "[resourceGroup().location]",
"properties": {
"publicIPAllocationMethod": "Dynamic"
}
},
{
"apiVersion": "[variables('apiVersion')]",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[parameters('subnetPrefix')]"
}
}
]
}
},
{
"apiVersion": "[variables('apiVersion')]",
"name": "[variables('applicationGatewayName')]",
"type": "Microsoft.Network/applicationGateways",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]"
],
"properties": {
"sku": {
"name": "[parameters('skuName')]",
"tier": "Standard",
"capacity": "[parameters('capacity')]"
},
"sslCertificates": [
{
"name": "appGatewaySslCert1",
"properties": {
"data": "[parameters('certData1')]",
"password": "[parameters('certPassword1')]"
}
}
],
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
],
"frontendIPConfigurations": [
{
"name": "appGatewayFrontendIP",
"properties": {
"PublicIPAddress": {
"id": "[variables('publicIPRef')]"
}
}
}
],
"frontendPorts": [
{
"name": "appGatewayFrontendPort1",
"properties": {
"Port": 443
}
},
{
"name": "appGatewayFrontendPort2",
"properties": {
"Port": 80
}
}
],
"backendAddressPools": [
{
"name": "appGatewayBackendPool1",
"properties": {
"BackendAddresses": [
{
"IpAddress": "[parameters('backendIpAddress1')]"
},
{
"IpAddress": "[parameters('backendIpAddress2')]"
},
{
"IpAddress": "[parameters('backendIpAddress3')]"
}
]
}
},
{
"name": "appGatewayBackendPool2",
"properties": {
"BackendAddresses": [
{
"IpAddress": "[parameters('backendIpAddress4')]"
},
{
"IpAddress": "[parameters('backendIpAddress5')]"
},
{
"IpAddress": "[parameters('backendIpAddress6')]"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "appGatewayBackendHttpSettings",
"properties": {
"Port": 80,
"Protocol": "Http",
"CookieBasedAffinity": "Disabled"
}
},
{
"name": "appGatewayBackendHttpsSettings",
"properties": {
"Port": 443,
"Protocol": "Https",
"CookieBasedAffinity": "Disabled"
}
}
],
"httpListeners": [
{
"name": "appGatewayHttpsListener-Group1",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGatewayFrontendPort1')]"
},
"Protocol": "Https",
"SslCertificate": {
"Id": "[concat(variables('applicationGatewayID'), '/sslCertificates/appGatewaySslCert1')]"
},
"HostName": "[parameters('hostName1')]",
"RequireServerNameIndication": "false"
}
},
{
"name": "appGatewayHttpsListener-Group2",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGatewayFrontendPort1')]"
},
"Protocol": "Https",
"SslCertificate": {
"Id": "[concat(variables('applicationGatewayID'), '/sslCertificates/appGatewaySslCert1')]"
},
"HostName": "[parameters('hostName2')]",
"RequireServerNameIndication": "false"
}
},
{
"name": "appGatewayHttpListener-Group1",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGatewayFrontendPort2')]"
},
"Protocol": "Http",
"SslCertificate": null,
"HostName": "[parameters('hostName1')]",
"RequireServerNameIndication": "false"
}
},
{
"name": "appGatewayHttpListener-Group2",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGatewayFrontendPort2')]"
},
"Protocol": "Http",
"SslCertificate": null,
"HostName": "[parameters('hostName2')]",
"RequireServerNameIndication": "false"
}
}
],
"requestRoutingRules": [
{
"Name": "Group1-SSL",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpsListener-Group1')]"
},
"backendAddressPool": {
"id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool1')]"
},
"backendHttpSettings": {
"id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGatewayBackendHttpSettings')]"
}
}
},
{
"Name": "Group2-SSL",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpsListener-Group2')]"
},
"backendAddressPool": {
"id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool2')]"
},
"backendHttpSettings": {
"id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGatewayBackendHttpSettings')]"
}
}
},
{
"Name": "Group2-www",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpListener-Group1')]"
},
"backendAddressPool": {
"id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool1')]"
},
"backendHttpSettings": {
"id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGatewayBackendHttpSettings')]"
}
}
},
{
"Name": "Group1-www",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpListener-Group2')]"
},
"backendAddressPool": {
"id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool2')]"
},
"backendHttpSettings": {
"id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGatewayBackendHttpSettings')]"
}
}
}
]
}
}
]
}
As you can see I specify GWSubnet1 as the App Gateway subnet. My backend IPs are in the VMnet1 subnet under the same Virtual Network. When I deploy it fails saying that it can't delete VMnet1. VMNet1 is only indirectly referenced as the backend IP so why would it be trying remove it. GWSubnet1 is an unused empty subenet as per the deployment rules from Azure.
If I use the GUI I can create the gateway and select GWSubnet1. However using the GUI the advanced feature of putting the hostname in the listner isn't an option and therefore won't let you create multiple listners using the same front end port. I tried using the GUI and and then adding listners via Poweshell (version 3.0.0) using the following
$hostname = "example1.foo.com"
$listnername = "group2-az"
$appgwname = "PortalGateway"
$rmname = "myrmg"
$feipname = "appGatewayFrontendIP"
$fepname = "appGatewayFrontendPort"
$behttpname = "appGatewayBackendHttpSettings"
$appgw = Get-AzureRmApplicationGateway -Name $appgwname -ResourceGroupName $rmname
$bepool = Get-AzureRmApplicationGatewayBackendAddressPool -ApplicationGateway $appgw -Name "appGatewayBackendPool"
$behttp = Get-AzureRmApplicationGatewayBackendHttpSettings -ApplicationGateway $appgw -Name $behttpname
$fipc = Get-AzureRmApplicationGatewayFrontendIPConfig -Name $feipname -ApplicationGateway $appgw
$fep = Get-AzureRmApplicationGatewayFrontendPort -Name $fepname -ApplicationGateway $appgw
$result = Add-AzureRmApplicationGatewayHttpListener -ApplicationGateway $appgw -Name "appGatewayHttpListenerGroup1" -Protocol Http -FrontendIPConfiguration $fipc -FrontendPort $fep -HostName $hostname -RequireServerNameIndication false
However what appears to happen is that it doesn't add a listener it just modifies the existing default listener that is created when you create the appgateway via the GUI. It does this no matter what name I choose as the listener.
I know the deployment template works as I can create a new empty resource group and deploy it in there and it deploys. I just can't seem to get it to deploy where there are existing VMs. What is the correct way to do this?
ARM Templates are declarative and in your template there is only a single Subnet. If you deploy that template ARM will try to make it exactly as you have defined = it tries to remove any subnets in that subnet which arent defined withing itself.
That is the reason for your error. ARM tries to delete your VMnet1 and it cant do that as long as it has NICs associated with it.
Check the Documentation here:
Deploy resources with Resource Manager templates and Azure PowerShell
The interesting Part for you is:
Incremental and complete deployments
When deploying your resources, you specify that the deployment is either an incremental update or a complete update. By default, Resource Manager handles deployments as incremental updates to the resource group.
With incremental deployment, Resource Manager:
leaves unchanged resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
With complete deployment, Resource Manager:
deletes resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
To solve your Problem you need to either make the subnet config exactly represent your existing setup or your manually create the new subnet and dont define the vnet in your template.
If you create the subnet manually you can reference your existing vnet and subnet in the template like this:
"parameters": {
"existingVirtualNetworkName": {
"type": "string"
},
"existingVirtualNetworkResourceGroup": {
"type": "string"
},
"existingSubnet1Name": {
"type": "string"
},
"existingSubnet2Name": {
"type": "string"
},
}
"variables": {
"existingVnetID": "[resourceId(parameters('existingVirtualNetworkResourceGroup'), 'Microsoft.Network/virtualNetworks', parameters('existingVirtualNetworkName'))]",
"existingSubnet1Ref": "[concat(variables('existingVnetID'),'/subnets/', parameters('existingSubnet1Name'))]",
"existingSubnet2Ref": "[concat(variables('existingVnetID'),'/subnets/', parameters('existingSubnet2Name'))]",
}
After passing the existing RessourceGroup, Vnet and Subnetnames through parameters you can just use the variables "existingSubnet1Name" to point to the correct IDs.
The magic lies withing the [resourceId()] functions optional parameters: [subscriptionId], [resourceGroupName].
resourceId ([subscriptionId], [resourceGroupName], resourceType, resourceName1, [resourceName2]...)
Documentation: Template functions

Resources