Azure ARM deployment - copyIndex error with multiple NICs - azure

I am trying to deploy the below Palo Alto deployment to an Azure environment. We are using it as an internal firewall for the various features it provides that NSGs don't, so we don't want a public IP. We're using a virtual gateway and sticking the firewall behind it in an active-active pair (the load balancer, NSGs, etc. are being done in a separate template).
I've been trying to get this template to work for a while now, but I'm stuck on deploying the two VMs with multiple NICs. I keep getting this error:
ERROR: Azure Error: InvalidTemplate
Message: Deployment template validation failed: 'The template variable 'nicName' is not valid: The template function 'copyIndex' is not expected at this location. The function can only be used in a resource with copy specified. Please see https://aka.ms/arm-copy for usage details.. Please see https://aka.ms/arm-template-expressions for usage details.'.
I've tried a bunch of fixes - changing the variable syntax, changing the syntax of the resource, but none of them are working. I've checked the Azure documentation on using the copyIndex feature, but I can't see where I'm going wrong. I was hoping someone with a bit more experience could point out where my syntax is wrong and provide suggestions on how to correct it?
Many thanks, template is below:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"LoadBalancerSku": {
"type": "string",
"allowedValues": [
"Standard",
"Basic"
],
"metadata": {
"description": "Std LB load balances across all the ports where as Basic LB load balances on a port-to-port basis"
},
"defaultValue": "Standard"
},
"storageName": {
"type": "string",
"metadata": {
"description": "Name of the storage account created to store the VM's disks. Storage account name must be globally unique."
},
"defaultValue": "Enter a globally unique name"
},
"mgmtPublicIPDns": {
"type": "string",
"metadata": {
"description": "DNS Name prefix of public IP resource for Management interface of VM-Series firewall. Name must be globally unique."
},
"defaultValue": "Enter a globally unique name"
},
"networkSecurityGroupName": {
"type": "string",
"defaultValue": "nsg",
"metadata": {
"description": "Network Security Group Name"
}
},
"networkSecurityGroupInboundIP": {
"type": "string",
"metadata": {
"description": "Your source public IP address. Added to the inbound NSG on eth0 (MGMT), to restrict access to the deployment."
},
"defaultValue": "1.1.1.1/32"
},
"avSetName": {
"type": "string",
"metadata": {
"description": "Name of the availability set for outbound firewall"
},
"defaultValue": "outbound-avset"
},
"storageType": {
"type": "string",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Premium_LRS",
"Standard_RAGRS"
],
"metadata": {
"description": "Type of the storage account created"
},
"defaultValue": "Standard_LRS"
},
"virtualNetworkName": {
"type": "string",
"defaultValue": "firewall-test",
"metadata": {
"description": "Virtual Network Name"
}
},
"virtualNetworkAddressPrefix": {
"type": "string",
"defaultValue": "10.0.0.0/16",
"metadata": {
"description": "CIDR for Virtual Network"
}
},
"mgmtSubnetName": {
"type": "string",
"defaultValue": "Mgmt",
"metadata": {
"description": "Subnet for Management Network"
}
},
"mgmtSubnetPrefix": {
"type": "string",
"defaultValue": "10.0.0.0/24",
"metadata": {
"description": "CIDR for Management Network"
}
},
"untrustSubnetName": {
"type": "string",
"defaultValue": "Untrust",
"metadata": {
"description": "Subnet for Untrusted Network"
}
},
"untrustSubnetPrefix": {
"type": "string",
"defaultValue": "10.0.1.0/24",
"metadata": {
"description": "CIDR for Untrusted Network"
}
},
"trustSubnetName": {
"type": "string",
"defaultValue": "Trust",
"metadata": {
"description": "Subnet for Trusted Network"
}
},
"trustSubnetPrefix": {
"type": "string",
"defaultValue": "10.0.2.0/24",
"metadata": {
"description": "CIDR for Trusted Network"
}
},
"mgmtPublicIPName": {
"type": "string",
"metadata": {
"description": "Name prefix of public IP resource for Management interface of VM-Series firewall."
},
"defaultValue": "mgmt-pip"
},
"loadBalancerName": {
"type": "string",
"metadata": {
"description": "Name for the outbound load balancer resource."
},
"defaultValue": "outbound-lb"
},
"loadBalancerIP": {
"type": "string",
"metadata": {
"description": "IP Address for the outbound load balancer resource in the Trust network."
},
"defaultValue": "10.0.2.4"
},
"imageSku": {
"type": "string",
"defaultValue": "bundle1",
"allowedValues": [
"byol",
"bundle1",
"bundle2"
],
"metadata": {
"description": "byol = Bring Your Own License; bundle1 = Bundle 1 PAYG (Hourly); bundle2 = Bundle 2 PAYG (Hourly)"
}
},
"virtualMachineName": {
"type": "string",
"metadata": {
"description": "Name prefix of VM-Series VM in the Azure portal"
},
"defaultValue": "outbound-vm-series"
},
"vmSize": {
"type": "string",
"allowedValues": [
"Standard_D3",
"Standard_D4",
"Standard_D3_v2",
"Standard_D4_v2",
"Standard_D5_v2",
"Standard_D14_v2",
"Standard_A4"
],
"metadata": {
"description": "Azure VM size for VM-Series"
},
"defaultValue": "Standard_D3_v2"
},
"authenticationType": {
"type": "string",
"metadata": {
"description": "Type of administrator user authentication "
},
"allowedValues": [
"sshPublicKey",
"password"
],
"defaultValue": "password"
},
"adminUsername": {
"type": "string",
"defaultValue": "pandemo",
"metadata": {
"description": "Username of the administrator account of VM instances"
}
},
"adminPassword": {
"type": "securestring",
"defaultValue": "Dem0pa$$w0rd",
"metadata": {
"description": "Password for the administrator account of all VM instances. This must be specified if Authentication Type is 'password'."
}
},
"sshKey": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "SSH RSA public key file as a string. Must be specified if Authentication Type is 'sshPublicKey'."
}
},
"vmCount": {
"type": "int",
"defaultValue": 2,
"metadata": {
"description": "Number of VM-Series firewall"
}
}
},
"variables": {
"baseUrl": "http://git.lr.net/Azure/management/firewall/tree/master/azure-pan-hub",
"deployStorageURL": "[concat(variables('baseUrl'),'/deployStorage.json')]",
"deployVnetURL": "[concat(variables('baseUrl'),'/deployVnet.json')]",
"deployFirewallURL": "[concat(variables('baseUrl'),'/deployFirewall.json')]",
"location": "[resourceGroup().location]",
"rgname": "[resourceGroup().name]",
"nicName": "[concat(parameters('virtualMachineName'), copyindex())]",
"imagePublisher": "paloaltonetworks",
"imageOffer": "vmseries1",
"version": "latest",
"vnetname": "[parameters('virtualNetworkName')]",
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('vnetname'))]",
"mgmtSubnetRef": "[concat(variables('vnetID'),'/subnets/',parameters('mgmtSubnetName'))]",
"untrustSubnetRef": "[concat(variables('vnetID'),'/subnets/',parameters('untrustSubnetName'))]",
"trustSubnetRef": "[concat(variables('vnetID'),'/subnets/',parameters('trustSubnetName'))]",
"subnets": [
{
"name": "[parameters('mgmtSubnetName')]",
"properties": {
"addressPrefix": "[parameters('mgmtSubnetPrefix')]"
}
},
{
"name": "[parameters('untrustSubnetName')]",
"properties": {
"addressPrefix": "[parameters('untrustSubnetPrefix')]"
}
},
{
"name": "[parameters('trustSubnetName')]",
"properties": {
"addressPrefix": "[parameters('trustSubnetPrefix')]"
}
}
]
},
"resources": [
{
"name": "deployStorage",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2015-01-01",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('storageName')]",
"apiVersion": "2015-06-15",
"location": "[variables('location')]",
"properties": {
"accountType": "[parameters('storageType')]"
}
}
]
}
}
},
{
"name": "deployVnet",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2017-05-10",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2015-06-15",
"location": "[variables('location')]",
"name": "[variables('vnetname')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('virtualNetworkAddressPrefix')]"
]
},
"subnets": "[variables('subnets')]"
}
}
]
}
}
},
{
"name": "deployAvailabilitySet",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2015-01-01",
"dependsOn": [
"Microsoft.Resources/deployments/deployStorage",
"Microsoft.Resources/deployments/deployVNet"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Compute/availabilitySets",
"name": "[parameters('avSetName')]",
"location": "[variables('location')]"
}
]
}
}
},
{
"name": "deployMgmtNetworkInterface",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-08-01",
"dependsOn": [
"Microsoft.Resources/deployments/deployStorage",
"Microsoft.Resources/deployments/deployVNet"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[concat(parameters('virtualMachineName'), copyindex(), '-nic0')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[variables('location')]",
"apiVersion": "2015-06-15",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', parameters('mgmtPublicIPName'), copyindex())]"
],
"copy": {
"name": "nicLoop",
"count": "[parameters('vmCount')]"
},
"properties": {
"ipConfigurations": [
{
"name": "ipconfig-mgmt",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[concat(variables('vnetId'),'/subnets/', parameters ('mgmtSubnetName'))]"
}
}
}
]
}
}
]
}
}
},
{
"name": "deployUntrustNetworkInterface",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-08-01",
"dependsOn": [
"Microsoft.Resources/deployments/deployStorage",
"Microsoft.Resources/deployments/deployVNet"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[concat(parameters('virtualMachineName'), copyindex(), '-nic1-std')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[variables('location')]",
"apiVersion": "2015-06-15",
"copy": {
"name": "nicLoop",
"count": "[parameters('vmCount')]"
},
"properties": {
"enableIPForwarding": true,
"ipConfigurations": [
{
"name": "ipconfig-untrust",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[concat(variables('vnetId'),'/subnets/', parameters ('untrustSubnetName'))]"
}
}
}
]
}
}
]
}
}
},
{
"name": "deploytrustNetworkInterface",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-08-01",
"dependsOn": [
"Microsoft.Resources/deployments/deployStorage",
"Microsoft.Resources/deployments/deployVNet"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[concat(parameters('virtualMachineName'), copyindex(), '-nic2')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[variables('location')]",
"apiVersion": "2015-06-15",
"copy": {
"name": "nicLoop",
"count": "[parameters('vmCount')]"
},
"properties": {
"enableIPForwarding": true,
"ipConfigurations": [
{
"name": "ipconfig-trust",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[concat(variables('vnetId'),'/subnets/', parameters ('trustSubnetName'))]"
}
}
}
]
}
}
]
}
}
},
{
"name": "[concat(parameters('virtualMachineName'), '-std-', copyindex())]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-08-01",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', parameters('virtualMachineName'), copyindex(), '-nic0')]",
"[concat('Microsoft.Network/networkInterfaces/', parameters('virtualMachineName'), copyindex(), '-nic1-std')]",
"[concat('Microsoft.Network/networkInterfaces/', parameters('virtualMachineName'), copyindex(), '-nic2')]"
],
"copy": {
"name": "vmLoop",
"count": "[parameters('vmCount')]"
},
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Compute/virtualMachines",
"name": "[concat(parameters('virtualMachineName'), '-std')]",
"location": "[variables('location')]",
"apiVersion": "2015-05-01-preview",
"plan": {
"name": "[parameters('imageSku')]",
"product": "[variables('imageOffer')]",
"publisher": "[variables('imagePublisher')]"
},
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"osProfile": {
"computerName": "[parameters('virtualMachineName')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[variables('imagePublisher')]",
"offer": "[variables('imageOffer')]",
"sku": "[parameters('imageSku')]",
"version": "latest"
},
"osDisk": {
"name": "osdisk",
"vhd": {
"uri": "[concat('http://', parameters('storageName'), '.blob.core.windows.net/vhds/', parameters('virtualMachineName'), '-', variables('imageOffer'), '-', parameters('imageSku'), '.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),'-nic0'))]",
"properties": {
"primary": true
}
},
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),'-nic1-std'))]",
"properties": {
"primary": false
}
},
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('nicName'),'-nic2'))]",
"properties": {
"primary": false
}
}
]
}
}
}
]
}
}
}
]
}

so in general with loops, you can only use copyIndex() function inside loops (and you are trying to use it outside of loop). with variables you can use this (same method applies to property loops):
"variables": {
"copy": [
{
"name": "real_var_name_goes_here",
"count": "how_many_items_with_var",
"input": {
"key": "value" << have to use copyIndex('real_var_name_goes_here')
}
}
]
}
and you'd use normal way for regular loops
Reading:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-multiple

Related

Add VM to Recovery Services Vault in different Resource Group

I'm fairly new to Azure ARM Templates so any help it's appreciated. I'm trying to deploy various vm's and at the same time configure the backup but I'm getting this error:
"InvalidTemplate","message":"Deployment template validation failed: 'The template resource '[concat(parameters('VMNames')[copyIndex()], '-' , 'BackupIntent')]' at line '195' and column '9' is not valid: The language expression property '0' can't be evaluated.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"VnetResourceGroup": {
"type": "String"
},
"VnetName": {
"type": "String"
},
"SubnetName": {
"type": "String"
},
"OSVersion": {
"defaultValue": "2016-Datacenter",
"allowedValues": [
"2016-Datacenter",
"2019-Datacenter",
"2022-datacenter",
"2022-datacenter-g2",
"2019-datacenter-gensecond",
"2016-datacenter-gensecond"
],
"type": "String",
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version."
}
},
"virtualMachineSize": {
"type": "String"
},
"AdminUsername": {
"defaultValue": "evtcladm",
"type": "String",
"metadata": {
"description": "Admin username for VM"
}
},
"AdminPassword": {
"type": "SecureString",
"metadata": {
"description": "Admin password for VM"
}
},
"backupVaultName": {
"defaultValue": "vault-corenet-01",
"allowedValues": [
"vault-corenet-01",
"vault-cps-01",
"Vault-Multi-VM",
"Vault-RecoveryMultiVM",
"vault-evt-cps-chile"
],
"type": "String",
"metadata": {
"description": "Recovery Vault to be used for backup"
}
},
"BackupPolicy": {
"defaultValue": "FASE1",
"type": "String"
},
"RSVResourceGroup": {
"type": "String"
},
"backupFabricName": {
"defaultValue": "Azure",
"type": "String"
},
"VMNames": {
"type": "String"
},
"numberOfInstances": {
"type": "Int",
"metadata": {
"description": "Number of VMs to deploy"
}
}
},
"variables": {
"v2Vm": "vm;iaasvmcontainerv2;",
"maxZones": 3,
"nicSuffix": "NIC",
"networkInterfaceName": "[concat(parameters('VMNames'), variables('nicSuffix'))]"
},
"resources": [
{
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2016-03-30",
"name": "[concat(variables('networkInterfaceName'), copyindex(1))]",
"location": "[resourceGroup().location]",
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[resourceId(parameters('VnetResourceGroup'),'Microsoft.Network/virtualNetworks/subnets',parameters('VnetName'),parameters('subnetName'))]"
}
}
}
]
},
"copy": {
"name": "nicLoop",
"count": "[parameters('numberOfInstances')]"
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2016-04-30-preview",
"name": "[concat(parameters('VMNames'), copyIndex(1))]",
"location": "[resourceGroup().location]",
"dependsOn": [
"nicLoop"
],
"zones": [
"[string(add(mod(copyIndex(0), variables('maxZones')), 1))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('virtualMachineSize')]"
},
"osProfile": {
"computerName": "[concat(parameters('VMNames'), copyIndex(1))]",
"adminUsername": "[parameters('AdminUsername')]",
"adminPassword": "[parameters('AdminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('OSVersion')]",
"version": "latest"
},
"osDisk": {
"createOption": "FromImage",
"managedDisk": {
"storageAccountType": "Premium_LRS"
}
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('networkInterfaceName'), copyindex(1)))]"
}
]
}
},
"copy": {
"name": "virtualMachineLoop",
"count": "[parameters('numberOfInstances')]"
}
},
{
"apiVersion": "2017-05-10",
"name": "[concat(parameters('VMNames')[copyIndex()], '-' , 'BackupIntent')]",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('RSVResourceGroup')]",
"copy": {
"name": "AzureBackupLoop",
"count": "[length(parameters('VMNames'))]"
},
"dependsOn": [
"virtualMachineLoop"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[concat(parameters('backupVaultName'), '/', 'Azure', '/', variables('v2Vm'), resourceGroup().name, ';', parameters('VMNames')[copyIndex()])]",
"apiVersion": "2017-07-01",
"type": "Microsoft.RecoveryServices/vaults/backupFabrics/backupProtectionIntent",
"properties": {
"friendlyName": "[concat(parameters('VMNames')[copyIndex()], 'BackupIntent')]",
"protectionIntentItemType": "AzureResourceItem",
"policyId": "[resourceId(parameters('RSVResourceGroup'), 'Microsoft.RecoveryServices/vaults/backupPolicies', parameters('backupVaultName'), parameters('BackupPolicy'))]",
"sourceResourceId": "[resourceId(resourceGroup().name, 'Microsoft.Compute/virtualMachines', parameters('VMNames')[copyIndex()])]"
}
}
]
}
}
}
]
}
Tested in my environment was getting same error as you are getting.
Solution
please remove sqaure bracket from [copyIndex()] and seprate with , for wherever you are using like below.
"name": "[concat(parameters('VMNames'),copyIndex(1),'_','BackupIntent')]",
instead of
"name": "[concat(parameters('VMNames')[copyIndex()], '-' , 'BackupIntent')]"

Subnet DemoSubnet is in use and cannot be updated

I have created a DemoSubnet and an ElasticIP, then I am trying to deploy a ARM template which has a Virtual machine which is trying to attach with that existing elastic IP. The template works fine initially, and after some time its failing due to below error
"At least one resource deployment operation failed. Please list
deployment operations for details"
Here is the detailed error message from the arm template deployment:
Details=[{"code":"BadRequest","message":"{\r\n \"error\": {\r\n
\"code\": \"InUseSubnetCannotBeUpdated\",\r\n \"message\": \"Subnet
DemoSubnet is in use and cannot be updated.\",\r\n \"details\":
[]\r\n }\r\n}"}]
ARM Temaplate
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"addressPrefixes": {
"defaultValue": [
"10.0.0.0/16"
],
"type": "Array",
"metadata": {
"description": "Address prefix of the virtual network"
}
},
"location": {
"defaultValue": "[resourceGroup().location]",
"type": "String",
"metadata": {
"description": "Location for all resources."
}
},
"publicIpNewOrExisting": {
"defaultValue": "existing",
"type": "String",
"metadata": {
"description": "Determines whether or not a new public ip should be provisioned."
}
},
"publicIpResourceGroupName": {
"defaultValue": "[resourceGroup().name]",
"type": "String",
"metadata": {
"description": "Name of the resource group for the public ip address"
}
},
"storageAccountResourceGroupName": {
"defaultValue": "[resourceGroup().name]",
"type": "String",
"metadata": {
"description": "Name of the resource group for the existing storage account"
}
},
"storageNewOrExisting": {
"defaultValue": "existing",
"type": "String",
"metadata": {
"description": "Determines whether or not a new storage account should be provisioned."
}
},
"subnetPrefix": {
"defaultValue": "10.0.0.0/24",
"type": "String",
"metadata": {
"description": "Subnet prefix of the virtual network"
}
},
"virtualNetworkNewOrExisting": {
"defaultValue": "existing",
"type": "String",
"metadata": {
"description": "Determines whether or not a new virtual network should be provisioned."
}
},
"virtualNetworkResourceGroupName": {
"defaultValue": "[resourceGroup().name]",
"type": "String",
"metadata": {
"description": "Name of the resource group for the existing virtual network"
}
}
},
"resources": [
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-10-01",
"name": "SampleVM",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId(parameters('storageAccountResourceGroupName'),'Microsoft.Storage/storageAccounts/', 'SampleStorageAccnt')]",
"[resourceId('Microsoft.Network/networkInterfaces/', 'SampleNIF')]"
],
"properties": {
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[reference(resourceId(parameters('storageAccountResourceGroupName'),'Microsoft.Storage/storageAccounts/', 'SampleStorageAccnt')).primaryEndpoints.blob]"
}
},
"hardwareProfile": {
"vmSize": "Standard_A2"
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces','SampleNIF')]"
}
]
},
"osProfile": {
"adminPassword": "rajnikaur#253",
"adminUsername": "adminuser",
"computerName": "SampleVM"
},
"storageProfile": {
"dataDisks": [
{
"createOption": "Empty",
"diskSizeGB": 1023,
"lun": 0
}
],
"imageReference": {
"offer": "WindowsServer",
"publisher": "MicrosoftWindowsServer",
"sku": "2016-Datacenter",
"version": "latest"
},
"osDisk": {
"createOption": "FromImage"
}
}
}
},
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-04-01",
"name": "SampleStorageAccnt",
"location": "[parameters('location')]",
"kind": "Storage",
"condition": "[equals(parameters('storageNewOrExisting'), 'existing')]"
},
{
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2019-09-01",
"name": "SampleNIF",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId(parameters('virtualNetworkResourceGroupName'),'Microsoft.Network/virtualNetworks/', 'sampleVnet')]",
"[resourceId(parameters('publicIpResourceGroupName'),'Microsoft.Network/publicIPAddresses', 'DemoEIP')]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId(parameters('publicIpResourceGroupName'),'Microsoft.Network/publicIPAddresses', 'DemoEIP')]"
},
"subnet": {
"id": "[resourceId(parameters('virtualNetworkResourceGroupName'),'Microsoft.Network/virtualNetworks/subnets', 'sampleVnet', 'DemoSubnet')]"
}
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"apiVersion": "2017-09-01",
"name": "DemoEIP",
"location": "[parameters('location')]",
"properties": {
"publicIPAllocationMethod": "Static"
},
"condition": "[equals(parameters('publicIpNewOrExisting'), 'existing')]"
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2017-09-01",
"name": "sampleVnet",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": "[parameters('addressPrefixes')]"
},
"subnets": [
{
"name": "DemoSubnet",
"properties": {
"addressPrefix": "[parameters('subnetPrefix')]"
}
}
]
},
"condition": "[equals(parameters('virtualNetworkNewOrExisting'), 'existing')]"
}
]
}

How to create custom script extension on virtual machine scale set of linux using azure resource manager template?

Here is my template1.json which is used to create cluster with virtual machine scale set. I want to add extensions to it using the same arm template1. I tried including script of extensions in the properties section of managed clusters ( template2.json) but it is throwing error of parent resource vnetName not found.
I tried adding extensionProfile in the properties section of managed clusters but that also didn't work.
can anyone help me out how can I set custom script extension using same arm template?
template1.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vnetName": {
"type": "string",
"metadata": {
"description": "VNet name"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Azure resource location"
}
},
"vnetAddressPrefix": {
"type": "string",
"metadata": {
"description": "Virtual network address range"
}
},
"subnetName": {
"type": "string",
"metadata": {
"description": "Azure subnet Name"
}
},
"subnetPrefix": {
"type": "string",
"metadata": {
"description": "Azure subnet prefix"
}
},
"kubernetesVersion": {
"type": "string",
"metadata": {
"description": "kubernetesVersion"
}
},
"enableRBAC": {
"defaultValue": false,
"type": "bool",
"metadata": {
"description": "boolean flag to turn on and off of RBAC"
}
},
"dnsPrefix": {
"type": "string",
"metadata": {
"description": "Optional DNS prefix to use with hosted Kubernetes API server FQDN."
}
},
"enableHttpApplicationRouting": {
"defaultValue": false,
"type": "bool",
"metadata": {
"description": "boolean flag to turn on and off of http application routing"
}
},
"osDiskSizeGB": {
"defaultValue": 0,
"minValue": 0,
"maxValue": 1023,
"type": "int",
"metadata": {
"description": "Disk size (in GB) to provision for each of the agent pool nodes. This value ranges from 0 to 1023. Specifying 0 will apply the default disk size for that agentVMSize."
}
},
"agentCount": {
"minValue": 1,
"maxValue": 50,
"type": "int",
"metadata": {
"description": "The number of agent nodes for the cluster. Production workloads have a recommended minimum of 3."
}
},
"agentVMSize": {
"type": "string",
"metadata": {
"description": "The size of the Virtual Machine."
}
},
"masterCount": {
"type": "int",
"allowedValues": [
1
],
"metadata": {
"description": "The number of Kubernetes masters for the cluster."
}
},
"adminUsername": {
"type": "string",
"defaultValue": "azureuser",
"metadata": {
"description": "User name for the Linux Virtual Machines."
}
},
"osType": {
"defaultValue": "Linux",
"allowedValues": [
"Linux"
],
"type": "string",
"metadata": {
"description": "The type of operating system."
}
},
"maxPods": {
"defaultValue": 30,
"type": "int",
"metadata": {
"description": "Maximum number of pods that can run on a node."
}
},
"ServicePrincipalClientId": {
"type": "string",
"metadata": {
"description": "ServicePrincipalClientId"
}
},
"ServicePrincipalClientSecret": {
"type": "string",
"metadata": {
"description": "ServicePrincipalClientId"
}
},
"ServicePrincipalObjectId": {
"type": "string",
"metadata": {
"description": "ServicePrincipalClientId"
}
},
"sshRSAPublicKey": {
"type": "securestring",
"metadata": {
"description": "Configure all linux machines with the SSH RSA public key string"
}
},
"networkPlugin": {
"allowedValues": [
"azure",
"kubenet"
],
"defaultValue": "kubenet",
"type": "string",
"metadata": {
"description": "Network plugin used for building Kubernetes network."
}
},
"dockerBridgeCidr": {
"type": "string",
"metadata": {
"description": "A CIDR notation IP for Docker bridge."
}
},
"dnsServiceIP": {
"type": "string",
"metadata": {
"description": "Containers DNS server IP address."
}
},
"serviceCidr": {
"type": "string",
"metadata": {
"description": "A CIDR notation IP range from which to assign service cluster IPs."
}
},
"resourceGroup": {
"type": "string",
"defaultValue": "[resourceGroup().name]",
"metadata": {
"description": "Azure resource group."
}
},
"autoscalingEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Flag to enable autoscaling."
}
},
"nodesMinimum": {
"type": "int",
"defaultValue": 2,
"metadata": {
"description": "Minimum number of nodes for autoscaling."
}
},
"nodesMaximum": {
"type": "int",
"defaultValue": 4,
"metadata": {
"description": "Maximum number of nodes for autoscaling."
}
}
},
"variables": {
"mastersEndpointDNSNamePrefix":"[concat(parameters('dnsPrefix'),'mgmt')]"
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-08-01",
"name": "nestedTemplate",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2019-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnetName')]",
"properties": {
"addressPrefix": "[parameters('subnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.ContainerService/managedClusters",
"name":"[concat('Kluster-',resourceGroup().name)]",
"apiVersion": "2019-08-01",
"location": "[parameters('location')]",
"tags": {},
"dependsOn": [
"[concat('Microsoft.Resources/deployments/', 'ClusterSubnetRoleAssignmentDeployment')]"
],
"properties": {
"kubernetesVersion": "[parameters('kubernetesVersion')]",
"enableRBAC": "[parameters('enableRBAC')]",
"dnsPrefix": "[parameters('dnsPrefix')]",
"addonProfiles": {
"httpApplicationRouting": {
"enabled": "[parameters('enableHttpApplicationRouting')]"
}
},
"masterProfile": {
"count": "[parameters('masterCount')]",
"vmSize": "[parameters('agentVMSize')]",
"dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]"
},
"agentPoolProfiles": [
{
"name": "agentpool",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "[parameters('osType')]",
"storageProfile": "ManagedDisks",
"vnetSubnetID": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]",
"maxPods": "[parameters('maxPods')]",
"enableAutoScaling": "[parameters('autoscalingEnabled')]",
"minCount": "[parameters('nodesMinimum')]",
"maxCount": "[parameters('nodesMaximum')]",
"type": "VirtualMachineScaleSets"
}
],
"linuxProfile": {
"adminUsername": "[parameters('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[parameters('sshRSAPublicKey')]"
}
]
}
},
"servicePrincipalProfile": {
"clientId": "[parameters('ServicePrincipalClientId')]",
"secret": "[parameters('ServicePrincipalClientSecret')]"
},
"networkProfile": {
"networkPlugin": "[parameters('networkPlugin')]",
"serviceCidr": "[parameters('serviceCidr')]",
"dnsServiceIP": "[parameters('dnsServiceIP')]",
"dockerBridgeCidr": "[parameters('dockerBridgeCidr')]"
}
}
},
{
"type": "Microsoft.Resources/deployments",
"name": "ClusterSubnetRoleAssignmentDeployment",
"apiVersion": "2019-08-01",
"subscriptionId": "[subscription().subscriptionId]",
"resourceGroup": "[parameters('resourceGroup')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"type": "Microsoft.Network/virtualNetworks/subnets/providers/roleAssignments",
"apiVersion": "2019-04-01-preview",
"name": "[concat(parameters('vnetName'), '/', parameters('subnetName'), '/Microsoft.Authorization/', guid(resourceGroup().id, deployment().name))]",
"properties": {
"roleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', '4d97b98b-1d4f-4787-a291-c67834d212e7')]",
"principalId": "[parameters('ServicePrincipalObjectId')]",
"scope": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]"
}
}
]
}
}
}
]
}
}
}
]
}
template2.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
Please copy from template1.json (due to characters limit I have removed)
},
"variables": {
"mastersEndpointDNSNamePrefix":"[concat(parameters('dnsPrefix'),'mgmt')]",
"vmssName":"aks-agentpool-43678249-vmss"
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-08-01",
"name": "nestedTemplate",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2019-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnetName')]",
"properties": {
"addressPrefix": "[parameters('subnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.ContainerService/managedClusters",
"name":"[concat('Kluster-',resourceGroup().name)]",
"apiVersion": "2019-08-01",
"location": "[parameters('location')]",
"tags": {},
"dependsOn": [
"[concat('Microsoft.Resources/deployments/', 'ClusterSubnetRoleAssignmentDeployment')]"
],
"properties": {
"kubernetesVersion": "[parameters('kubernetesVersion')]",
"enableRBAC": "[parameters('enableRBAC')]",
"dnsPrefix": "[parameters('dnsPrefix')]",
"addonProfiles": {
"httpApplicationRouting": {
"enabled": "[parameters('enableHttpApplicationRouting')]"
}
},
"masterProfile": {
"count": "[parameters('masterCount')]",
"vmSize": "[parameters('agentVMSize')]",
"dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]"
},
"agentPoolProfiles": [
{
"name": "agentpool",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "[parameters('osType')]",
"storageProfile": "ManagedDisks",
"vnetSubnetID": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]",
"maxPods": "[parameters('maxPods')]",
"enableAutoScaling": "[parameters('autoscalingEnabled')]",
"minCount": "[parameters('nodesMinimum')]",
"maxCount": "[parameters('nodesMaximum')]",
"type": "virtualMachineScaleSets"
}
],
"linuxProfile": {
"adminUsername": "[parameters('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[parameters('sshRSAPublicKey')]"
}
]
}
},
"servicePrincipalProfile": {
"clientId": "[parameters('ServicePrincipalClientId')]",
"secret": "[parameters('ServicePrincipalClientSecret')]"
},
"networkProfile": {
"networkPlugin": "[parameters('networkPlugin')]",
"serviceCidr": "[parameters('serviceCidr')]",
"dnsServiceIP": "[parameters('dnsServiceIP')]",
"dockerBridgeCidr": "[parameters('dockerBridgeCidr')]"
}
},
"resources": [
{
"apiVersion": "2019-03-01",
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat('aks-agentpool-43678249-vmss','/', 'Extensions')]",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]",
"[concat('Microsoft.ContainerService/managedClusters/', concat('Kluster-',resourceGroup().name))]"
],
"properties": {
"publisher": "Microsoft.Azure.Extensions",
"type": "CustomScript",
"typeHandlerVersion": "2.0",
"autoUpgradeMinorVersion":true,
"settings": {
"skipDos2Unix":false,
"fileUris": ["filelink"],
"commandToExecute": "./filename.sh"
}
}
}
]
},
{
"type": "Microsoft.Resources/deployments",
"name": "ClusterSubnetRoleAssignmentDeployment",
"apiVersion": "2019-08-01",
"subscriptionId": "[subscription().subscriptionId]",
"resourceGroup": "[parameters('resourceGroup')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"type": "Microsoft.Network/virtualNetworks/subnets/providers/roleAssignments",
"apiVersion": "2019-04-01-preview",
"name": "[concat(parameters('vnetName'), '/', parameters('subnetName'), '/Microsoft.Authorization/', guid(resourceGroup().id, deployment().name))]",
"properties": {
"roleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', '4d97b98b-1d4f-4787-a291-c67834d212e7')]",
"principalId": "[parameters('ServicePrincipalObjectId')]",
"scope": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]"
}
}
]
}
}
}
]
}
}
}
]
}
Error thrown : Parent resource vnetname not found as shown below
template1_params.json
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vnetName": {
"value": "aks-vnet"
},
"vnetAddressPrefix": {
"value": "10.0.0.0/8"
},
"subnetName": {
"value": "aks-subnet"
},
"subnetPrefix": {
"value": "10.240.0.0/16"
},
"kubernetesVersion": {
"value": "1.13.12"
},
"dnsPrefix": {
"value": "test"
},
"agentCount": {
"value": 2
},
"agentVMSize": {
"value": "Standard_E2_v3"
},
"masterCount": {
"value": 1
},
"ServicePrincipalClientId": {
"value": "clientid..."
},
"ServicePrincipalClientSecret": {
"value": "clientsecret..."
},
"ServicePrincipalObjectId": {
"value": "objectid.."
},
"sshRSAPublicKey": {
"value": "sshrsa-...."
},
"dnsServiceIP": {
"value": "10.0.0.10"
},
"serviceCidr": {
"value": "10.0.0.0/16"
},
"dockerBridgeCidr": {
"value": "172.17.0.1/16"
},
"nodesMinimum": {
"value": 2
},
"nodesMaximum": {
"value": 4
}
}
}
For your issue, there are two mistakes in your template as I see, maybe not all.
One is that the resources block in the AKS cluster, it's not for the extensions, only for the agent pools. You can learn its definition in Microsoft.ContainerService/managedClusters object.
Another is that you create the AKS cluster with virtual machine scale sets, so the extension should be the type Microsoft.Compute/virtualMachineScaleSets/extensions.
Maybe you can provide more details about the issue to get an exact solution. But before, you can try to change your template to fix the mistakes I found above.

Azure Databricks with custom vnet arm template won't connect to the custom vnet

With the following ARM template, I deploy an Azure Databricks with a custom managed Resource Group Name and add the workers to a custom VNET. In the portal this works fine. But When I try to do this inside an ARM template the managed resource groups keep deploying a workers vnet for the workers. I am thinking that I am on the right track but missing one setting. But can't figure it out. Is there anyone who can see what I am missing ?
Source ARM: https://github.com/Azure/azure-quickstart-templates/tree/master/101-databricks-workspace-with-vnet-injection
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
"databricksName": {
"type": "string",
"metadata": {
"description": "The name of the databricks workspace"
}
},
"pricingTier": {
"type": "string",
"allowedValues": [
"trial",
"standard",
"premium"
],
"metadata": {
"description": "The pricing tier of workspace."
}
},
"managedResourceGroupName": {
"type": "string",
"metadata": {
"description": "The name of the managed resource group that databricks will create"
}
},
"Location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "The Location of the deployment"
}
},
"vnetName": {
"type": "string",
"metadata": {
"description": "The Name of the virtual network where the Workers would be connected to"
}
},
"privateSubnetName": {
"defaultValue": "public-subnet",
"type": "string",
"metadata": {
"description": "The name of the private subnet to create."
}
},
"publicSubnetName": {
"defaultValue": "private-subnet",
"type": "string",
"metadata": {
"description": "The name of the public subnet to create."
}
}
},
"variables": {
"ManagedResourceGroupId": "[concat(subscription().id, '/resourceGroups/', parameters('managedResourceGroupName'))]",
"vnetId": "[resourceId('Microsoft.Network/virtualNetworks', parameters('vnetName'))]"
},
"resources": [
{
"name": "[parameters('databricksName')]",
"type": "Microsoft.Databricks/workspaces",
"apiVersion": "2018-04-01",
"tags": {
"description": "MIG6 databricks workspace",
"costCenter": "WPIPM12SG552"
},
"location": "[parameters('Location')]",
"properties": {
"managedResourceGroupId": "[variables('managedResourceGroupId')]",
"parameters": {
"customVirtualNetworkId": {
"value": "[variables('vnetId')]"
},
"customPublicSubnetName": {
"value": "[parameters('publicSubnetName')]"
},
"customPrivateSubnetName": {
"value": "[parameters('privateSubnetName')]"
}
}
},
"sku": {
"name": "[parameters('pricingTier')]"
}
}
]
}
You need to nest the vnet in the template, this works for me:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vnetName": {
"type": "string"
},
"vnetRG": {
"type": "string"
},
"publicSubnetName": {
"type": "string"
},
"publicSubnetCIDR": {
"type": "string"
},
"privateSubnetName": {
"type": "string"
},
"privateSubnetCIDR": {
"type": "string"
},
"workspaceName": {
"type": "string"
},
"tier": {
"type": "string"
},
"location": {
"type": "string"
},
"nsgName": {
"defaultValue": "databricks-nsg",
"type": "string"
},
"environment": {
"type": "string"
}
},
"resources": [
{
"apiVersion": "2017-05-10",
"name": "nestedTemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('vnetRG')]",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2018-04-01",
"type": "Microsoft.Network/virtualNetworks/subnets",
"name": "[concat(parameters('vnetName'), '/', parameters('publicSubnetName'))]",
"location": "[parameters('location')]",
"properties": {
"addressPrefix": "[parameters('publicSubnetCIDR')]",
"networkSecurityGroup": {
"id": "[variables('nsgId')]"
}
}
},
{
"apiVersion": "2018-04-01",
"type": "Microsoft.Network/virtualNetworks/subnets",
"name": "[concat(parameters('vnetName'), '/', parameters('privateSubnetName'))]",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'), '/subnets/', parameters('publicSubnetName'))]"
],
"properties": {
"addressPrefix": "[parameters('privateSubnetCIDR')]",
"networkSecurityGroup": {
"id": "[variables('nsgId')]"
}
}
}
]
},
"parameters": {}
}
},
{
"apiVersion": "2018-04-01",
"type": "Microsoft.Databricks/workspaces",
"location": "[parameters('location')]",
"name": "[parameters('workspaceName')]",
"dependsOn": [
"['Microsoft.Resources/deployments/nestedTemplate']"
],
"sku": {
"name": "[parameters('tier')]"
},
"comments": "Please do not use an existing resource group for ManagedResourceGroupId.",
"properties": {
"ManagedResourceGroupId": "[variables('managedResourceGroupId')]",
"parameters": {
"customVirtualNetworkId": {
"value": "[variables('vnetId')]"
},
"customPublicSubnetName": {
"value": "[parameters('publicSubnetName')]"
},
"customPrivateSubnetName": {
"value": "[parameters('privateSubnetName')]"
}
}
}
}
],
"variables": {
"managedResourceGroupId": "[concat(subscription().id, '/resourceGroups/', variables('managedResourceGroupName'))]",
"managedResourceGroupName": "[concat(resourceGroup().name,'-DATABRICKS-MANAGED')]",
"vnetId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('vnetRG'), '/providers/Microsoft.Network/virtualNetworks/', parameters('vnetName'))]",
"nsgId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('vnetRG'), '/providers/Microsoft.Network/networkSecurityGroups/', parameters('nsgName'))]"
},
"outputs": {}
}

Unable to add vNet peering to nested ARM template

I am trying to create an ARM template which creates 2 resource groups, and then deploys vNets and creates a peering between them. The vNet peering resource in the template is not working. I keep getting the same error and I believe it's because i'm trying to create a peering with a VNET which is in a different VNET/Resource group than the one i'm deploying into.
I am using nested templates to deploy into separate resource groups.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/Microsoft.Resources.json",
"contentVersion": "1.0.0.0",
"parameters": {
"vnetName": {
"type": "string",
"metadata": {
"description": "VNet name"
}
},
"vnetAddressPrefix": {
"type": "string",
"metadata": {
"description": "Address prefix"
}
},
"DMZResourceGroup": {
"type": "string",
"metadata": {
"description": "resource group for DMZ environment"
}
},
"HUBResourceGroup": {
"type": "string",
"metadata": {
"description": "resource group for HUB environment"
}
},
"DMZvnet": {
"type": "string",
"metadata": {
"description": "VNet for DMZ environment"
}
},
"DMZvnetAddressPrefix": {
"type": "string",
"metadata": {
"description": "DMZ VNet Address prefix"
}
},
"DMZFESubnet1Prefix": {
"type": "string",
"metadata": {
"description": "Subnet1 for DMZ environment. This is the Front-End subnet"
}
},
"DMZBESubnet2Prefix": {
"type": "string",
"metadata": {
"description": "Subnet1 for DMZ environment. This is the Back-End subnet"
}
},
"DMZNSG": {
"type": "string",
"metadata": {
"description": "NSG for DMZ environment"
}
},
"DMZFEsubnetName": {
"type": "string",
"metadata": {
"description": "Front-End subnet for DMZ environment"
}
},
"DMZBEsubnetName": {
"type": "string",
"metadata": {
"description": "Back-End subnet for DMZ environment"
}
},
"subnet1Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 1 prefix"
}
},
"subnet1Name": {
"type": "string",
"metadata": {
"description": "Subnet 1 Name"
}
},
"subnet2Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 2 Prefix"
}
},
"subnet2Name": {
"type": "string",
"metadata": {
"description": "Subnet 2 Name"
}
},
"subnet3Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 3 Prefix"
}
},
"subnet3Name": {
"type": "string",
"metadata": {
"description": "Subnet 3 Name"
}
},
"networkSecurityGroupName": {
"type": "string"
}
},
"variables": {
"location": "West Europe",
"HUBvNettoDMZvNet": "[concat(parameters('VNetName'), '-', parameters('DMZvnet'))]",
"DMZvNettoHUBvNet": "[concat(parameters('DMZvnet'), '-', parameters('VNetName'))]"
},
"resources": [
{
"name": "[parameters('DMZResourceGroup')]",
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[variables('Location')]",
"tags": {},
"properties": {}
},
{
"name": "[parameters('HUBResourceGroup')]",
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[variables('Location')]",
"tags": {},
"properties": {}
},
{
"apiVersion": "2017-05-10",
"name": "hubtemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('Hubresourcegroup')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('HUBResourceGroup'))]"
],
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentversion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2017-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[variables('Location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet1Name')]",
"properties": {
"addressPrefix": "[parameters('subnet1Prefix')]"
}
},
{
"name": "[parameters('subnet2Name')]",
"properties": {
"addressPrefix": "[parameters('subnet2Prefix')]"
}
},
{
"name": "[parameters('subnet3Name')]",
"properties": {
"addressPrefix": "[parameters('subnet3Prefix')]"
}
}
]
}
},
{
"name": "[concat(parameters('dmzvnet'), '/peered-to-', parameters('vnetname'))]",
"type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings",
"apiVersion": "2018-08-01",
"location": "[variables('location')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('VNetName'))]"
],
"properties": {
"allowVirtualNetworkAccess": true,
"allowForwardedTraffic": true,
"allowGatewayTransit": true,
"useRemoteGateways": true,
"remoteVirtualNetwork": {
"id": "/subscriptions/d01e22a2-bdb3-4c7a-92a2-613cc0dc1f30/resourceGroups/dmz-rg/providers/Microsoft.Network/virtualNetworks/dmz-vnet"
}
}
},
{
"apiVersion": "2015-06-15",
"type": "Microsoft.Network/networkSecurityGroups",
"name": "[parameters('networkSecurityGroupName')]",
"location": "[variables('location')]",
"properties": {
"securityRules": [
{
"name": "allow-rdp",
"properties": {
"description": "allow rdp",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "3389",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "10.0.1.0",
"access": "Allow",
"priority": 110,
"direction": "Inbound"
}
},
{
"name": "allow-smb",
"properties": {
"description": "allow smb",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "445",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "10.0.2.0",
"access": "Allow",
"priority": 112,
"direction": "Inbound"
}
}
]
}
}
]
}
}
},
{
"apiVersion": "2017-05-10",
"name": "dmztemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('DMZresourcegroup')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('DMZResourceGroup'))]"
],
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentversion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2017-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('DMZvNet')]",
"location": "[variables('Location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('DMZvNetaddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('DMZFEsubnetName')]",
"properties": {
"addressPrefix": "[parameters('DMZFESubnet1Prefix')]"
}
},
{
"name": "[parameters('DMZBEsubnetName')]",
"properties": {
"addressPrefix": "[parameters('DMZBESubnet2Prefix')]"
}
}
]
}
}
]
}
}
}
]
}
This is the error I am receiving
New-AzureRmDeployment : 16:13:36 - Resource
Microsoft.Network/virtualNetworks/virtualNetworkPeerings
'DMZ-VNET/peered-to-HUB-VNET' failed with message '{ "error": {
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.Network/virtualNetworks/DMZ-VNET' under resource group
'HUB-RG' was not found." } }' At line:1 char:1
+ New-AzureRmDeployment -Name $jobname -location $location -TemplatePar ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzureRmDeployment], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implement
ation.NewAzureDeploymentCmdlet
first of all, you can amend "outputs": {} everywhere, they dont really add anything useful, only clatter the template :). ok, there's just too much stuff going on here, to get into details of everything, unfortunately, here's a working template, you can diff yours and figure out what is wrong. if you need some questions answered, just get back to me here.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/Microsoft.Resources.json",
"contentVersion": "1.0.0.0",
"parameters": {
"VNetName": {
"type": "string",
"metadata": {
"description": "vnetname"
}
},
"vnetAddressPrefix": {
"type": "string",
"metadata": {
"description": "Address prefix"
}
},
"DMZResourceGroup": {
"type": "string",
"metadata": {
"description": "resource group for DMZ environment"
}
},
"HUBResourceGroup": {
"type": "string",
"metadata": {
"description": "resource group for HUB environment"
}
},
"DMZvnet": {
"type": "string",
"metadata": {
"description": "VNet for DMZ environment"
}
},
"DMZvnetAddressPrefix": {
"type": "string",
"metadata": {
"description": "DMZ VNet Address prefix"
}
},
"DMZFESubnet1Prefix": {
"type": "string",
"metadata": {
"description": "Subnet1 for DMZ environment. This is the Front-End subnet"
}
},
"DMZBESubnet2Prefix": {
"type": "string",
"metadata": {
"description": "Subnet1 for DMZ environment. This is the Back-End subnet"
}
},
"DMZNSG": {
"type": "string",
"metadata": {
"description": "NSG for DMZ environment"
}
},
"DMZFEsubnetName": {
"type": "string",
"metadata": {
"description": "Front-End subnet for DMZ environment"
}
},
"DMZBEsubnetName": {
"type": "string",
"metadata": {
"description": "Back-End subnet for DMZ environment"
}
},
"subnet1Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 1 prefix"
}
},
"subnet1Name": {
"type": "string",
"metadata": {
"description": "Subnet 1 Name"
}
},
"subnet2Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 2 Prefix"
}
},
"subnet2Name": {
"type": "string",
"metadata": {
"description": "Subnet 2 Name"
}
},
"subnet3Prefix": {
"type": "string",
"metadata": {
"description": "Subnet 3 Prefix"
}
},
"subnet3Name": {
"type": "string",
"metadata": {
"description": "Subnet 3 Name"
}
},
"networkSecurityGroupName": {
"type": "string"
}
},
"variables": {
"location": "West Europe",
"HUBvNettoDMZvNet": "[concat(parameters('VNetName'), '-', parameters('DMZvnet'))]",
"DMZvNettoHUBvNet": "[concat(parameters('DMZvnet'), '-', parameters('VNetName'))]"
},
"resources": [
{
"name": "[parameters('DMZResourceGroup')]",
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[variables('Location')]",
"tags": {},
"properties": {}
},
{
"name": "[parameters('HUBResourceGroup')]",
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[variables('Location')]",
"tags": {},
"properties": {}
},
{
"apiVersion": "2017-05-10",
"name": "hubtemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('Hubresourcegroup')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('HUBResourceGroup'))]",
"dmztemplate"
],
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentversion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2017-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[variables('Location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet1Name')]",
"properties": {
"addressPrefix": "[parameters('subnet1Prefix')]"
}
},
{
"name": "[parameters('subnet2Name')]",
"properties": {
"addressPrefix": "[parameters('subnet2Prefix')]"
}
},
{
"name": "[parameters('subnet3Name')]",
"properties": {
"addressPrefix": "[parameters('subnet3Prefix')]"
}
}
]
}
},
{
"name": "[concat(parameters('vnetName'), '/peered-to-', parameters('vnetname'))]",
"type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings",
"apiVersion": "2018-08-01",
"location": "[variables('location')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('VNetName'))]"
],
"properties": {
"allowVirtualNetworkAccess": true,
"allowForwardedTraffic": true,
"allowGatewayTransit": true,
"useRemoteGateways": false,
"remoteVirtualNetwork": {
"id": "[concat(subscription().id, '/resourceGroups/', parameters('DMZresourcegroup'), '/providers/Microsoft.Network/virtualNetworks/', parameters('DMZvNet'))]"
}
}
},
{
"apiVersion": "2015-06-15",
"type": "Microsoft.Network/networkSecurityGroups",
"name": "[parameters('networkSecurityGroupName')]",
"location": "[variables('location')]",
"properties": {
"securityRules": [
{
"name": "allow-rdp",
"properties": {
"description": "allow rdp",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "3389",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "10.0.1.0",
"access": "Allow",
"priority": 110,
"direction": "Inbound"
}
},
{
"name": "allow-smb",
"properties": {
"description": "allow smb",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "445",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "10.0.2.0",
"access": "Allow",
"priority": 112,
"direction": "Inbound"
}
}
]
}
}
]
}
}
},
{
"apiVersion": "2017-05-10",
"name": "dmztemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('DMZresourcegroup')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('DMZResourceGroup'))]"
],
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentversion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2017-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('DMZvNet')]",
"location": "[variables('Location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('DMZvNetaddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('DMZFEsubnetName')]",
"properties": {
"addressPrefix": "[parameters('DMZFESubnet1Prefix')]"
}
},
{
"name": "[parameters('DMZBEsubnetName')]",
"properties": {
"addressPrefix": "[parameters('DMZBESubnet2Prefix')]"
}
}
]
}
}
]
}
}
}
]
}
resourceId() in subscription level deployments: https://learn.microsoft.com/en-us/azure/azure-resource-manager/deploy-to-subscription#using-template-functions
ps. I'm sure you need peerings on both vnets.

Resources