applicationGatewayBackendAddressPools configurations does not apply in virtual machine scale set - azure

I have a VMSS which I deployed using ARM templates. This is the networkProfile block under VMSS resource section.
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": true,
"ipConfigurations": [
{
"name": "[concat(variables('VMSSName'), '-ipconfig')]",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
},
"applicationGatewayBackendAddressPools": "[variables('AppGatewayBackendAddressPool')]"
}
}
]
}
}
]
},
In Variable section, if I use resourceId() function and provide values from parameters then it does not apply the configuration in VMSS. for example:
"AppGatewayBackendAddressPool": "[resourceId(parameters('VirtualNetworkResourceGroup'),'Microsoft.Network/applicationGateways/backendAddressPools', parameters('ApplicationGatewayName'), parameters('BackendAddressPool'))]",
I've also tried adding parameters('SubscriptionName') but the result is same.
"AppGatewayBackendAddressPool": "[resourceId(parameters('SubscriptionName') ,parameters('VirtualNetworkResourceGroup'),'Microsoft.Network/applicationGateways/backendAddressPools', parameters('ApplicationGatewayName'), parameters('BackendAddressPool'))]",
When I declare variable like below then it applies backendAddressPool configuration in Networking -> Load Balancing.
"AppGatewayBackendAddressPool": [
{ "id": "/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Network/applicationGateways/<applicationGatewayName>/backendAddressPools/<backendAddressPool>" }
],
Similar I'm doing with subnetRef like below and that is working fine.
"subnetRef": "[resourceId(parameters('VirtualNetworkResourceGroup'), 'Microsoft.Network/virtualNetworks/subnets', parameters('VirtualNetworkName'), parameters('SubnetName'))]",
I want to parametrize the deployment by defining separate parameters.json file so I can attach applicationGatewayBackendAddressPools with different virtual machine scale sets.

This is how I achieved it by following Ked Mardemootoo answer.
IP configuration section under networkProfile of VMSS resource.
"ipConfigurations": [
{
"name": "[concat(variables('VMSSName'), '-ipconfig')]",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
},
"applicationGatewayBackendAddressPools": [
{ "id": "[concat(parameters('AapplicationGatewayExternalid'), '/backendAddressPools/', parameters('BackendAddressPool'))]" }
]
}
}
]
Template file parameters:
"BackendAddressPool": {
"type": "string",
"metadata": {
"description": "Backend pool to host blue/green vmss."
}
},
"AapplicationGatewayExternalid": {
"type": "string",
"metadata": {
"description": "Application Gateway Id."
}
}
Now, ARM template is calling and referencing applicationGatewayBackendAddressPools attribute dynamically under VMSS' resource section.
I have these two parameters in parameters.json file where I can define values according to environment.
"BackendAddressPool": {
"value": "<backendPoolName>"
},
"AapplicationGatewayExternalid": {
"value": "/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Network/applicationGateways/<ApplicationGatewayName>"
}
Overriding template variables in release pipeline vars:
overriding template vars
Defining in pipeline vars
pipeline var

You seem to be missing the concat in the variables. Looking at the raw json on my end, this is how it's configured. See if you can do something similar, and convert the subnet name and backend address pool to variables.
"ipConfigurations": [
{
"name": "ip-vmss-name",
"properties": {
"primary": true,
"subnet": {
"id": "[concat(parameters('virtualNetworks_vnet_externalid'), '/subnets/snet-vm')]"
},
"privateIPAddressVersion": "IPv4",
"applicationGatewayBackendAddressPools": [
{
"id": "[concat(parameters('applicationGateways_agw_1_externalid'), '/backendAddressPools/be-addr-pool-vmss-1')]"
}
]
}
}
]

Nothing seems wrong with your variables/parameters call but applicationGatewayBackendAddressPools is not a valid attribute for neither VMSS nor Application Gateway.
You can do it check AKS and Application Gateway documentations. I achieve the same goal by setting backendAddressPools, which is in Application Gateway section, in different parameters.json files.

Related

BICEP - Parameter file variable assignment

I was following the repo for separate parameter file to each env as defined in the https://github.com/Azure/bicep/discussions/4586
I tried the separate parameters file for dev, stage, prod but the value assignment in main module variable remains flagged by intelligence even though it exists same param exist in the respective parameter file.
Other approach I tried is loadjson variable, but it does not show auto completion for items under subnet block as it stopes right after value.
Maybe I am overthinking and not applying the correct approach, Perhaps I should ignore intellisense and try deploying by applying parameter and hope it will auto pick correct value during the deployment param check.
Here is my parameter file and the same value applies to each env param json.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"department": {
"value": "finance"
},
"saAccountCount": {
"value": 1
},
"vmCount": {
"value": 1
},
"locationIndex": { //idenx 1 = app server, 2=AD, 3=Tool server, 4= dchp server
"value": 1
},
"appRoleIndex": { //idenx 1 = westus2, 2= westus, 3= eastus, 4=centralus, 5=uswest3
"value": 1
},
"appRole": {
"value": {
"Applicatoin Server": "ap",
"Active Directory": "dc",
"Tool server": "tool",
"DHCP server": "dhcp"
}
},
"environment": {
"value": "dev"
},
"addressPrefixes": {
"value": [
"172.16.0.0/20"
]
},
"dnsServers": {
"value": [
"1.1.1.1",
"4.4.4.4"
]
},
"locationList": {
"value": {
"westus2": "azw2",
"westus": "azw",
"Eastus": "aze",
"CentralUS": "azc",
"westus3": "azw3"
}
},
"subnets": {
"value": [
{
"name": "frontend",
"subnetPrefix": "172.16.2.0/24",
"delegation": "Microsoft.Web/serverfarms",
"privateEndpointNetworkPolicies": "disabled",
"serviceEndpoints": [
{
"service": "Microsoft.KeyVault",
"locations": [
"*"
]
},
{
"service": "Microsoft.Web",
"locations": [
"*"
]
}
]
},
{
"name": "backend",
"subnetPrefix": "172.16.3.0/24",
"delegation": "Microsoft.Web/serverfarms",
"privateEndpointNetworkPolicies": "enabled",
"serviceEndpoints": [
{
"service": "Microsoft.KeyVault",
"locations": [
"*"
]
},
{
"service": "Microsoft.Web",
"locations": [
"*"
]
},
{
"service": "Microsoft.AzureCosmosDB",
"locations": [
"*"
]
}
]
}
]
}
}
}
You appear to be attempting to deploy an Azure Resource Management (ARM) template using a parameter file.
The parameter file is used to pass values to the ARM template during deployment. The parameter file must use the same types as the ARM template and can only include values for the ARM template's parameters.
You will receive an error if the parameter file contains extra parameters that do not match the ARM template's parameters.
In the same deployment process, you can use both inline parameters and a local parameter file. If you specify a parameter's value in both the local parameter file and inline, the inline value takes priority.
Refer to create a parameter file of an ARM template
About the different parameters file for dev, stage, and prod, it's likely that the parameter file is not correctly linked to the ARM template.
You can deploy the ARM template with the parameter file to determine if it will automatically select the proper value during the deployment parameter check.
Regarding the loadjson variable, it is possible that the loadjson variable is not properly formatted.
You can double-check the loadjson variable's format to ensure it's proper.
After a workaround on this, I created a sample parameter.json file for a webapp to deploy in a production environment and that worked for me.
Note: Alternatively, You can use az deployment group create with a parameters file and deploy into Azure to avoid these conflicts.

Key Vault ipRules property as parameter issue

I am trying to add Firewall rules for Azure Key Vault using ARM templates. It works as expected if ipRules property in conjunction with multiple IPs are defined in template (not as parameter).
However, if I try to define it as parameter getting "Bad JSON content found in the request."
Property defined in Template ("apiVersion": "2019-09-01"):
"kv-ipRules": {
"type": "array",
"metadata": {
"description": "The address space (in CIDR notation) to use for the Azure Key Vault to be deployed as Firewall rules."
}
}
"networkAcls": {
"defaultAction": "Deny",
"bypass": "AzureServices",
"virtualNetworkRules": [
{
"id": "[concat(parameters('kv-virtualNetworks'), '/subnets/','kv-subnet')]",
"ignoreMissingVnetServiceEndpoint": false
}
],
"ipRules": "[parameters('kv-ipRules')]"
}
Property defined in Parameters:
"kv-ipRules": {
"value": [
"xx.xx.xx.xxx",
"yy.yy.yy.yyy"
]
}
Given the documentation (https://learn.microsoft.com/en-us/azure/templates/Microsoft.KeyVault/vaults?tabs=json#IPRule), I would use:
"kv-ipRules": {
"value": [
{
"value": "xx.xx.xx.xxx"
},
{
"value": "yy.yy.yy.yyy"
}
]
}

Deployment Agents in Azure VM Scale Set

I am currently deploying a VM Scale Set (VMSS) using an ARM template which has a resource inside VMSS to install Azure extension for Azure DevOps (ADO) Deployment Agent. All is deployed successfully and a node is registered in ADO with all details as are in the ARM template. However the problem is that it installs the agent only on first node and (as far as I see) ignores the rest of the nodes. I've tested this with multiple nodes during creation of the scale set and with auto-scale as well. Both scenarios result in only first agent registered.
This is the code layout I'm using (I've removed the VMSS bits to reduce the template length here, there are of course OS, storage and network settings inside):
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('VMSSName')]",
"apiVersion": "2018-10-01",
"location": "[resourceGroup().location]",
"sku": {
"name": "[parameters('VMSSSize')]",
"capacity": "[parameters('VMSSCount')]",
"tier": "Standard"
},
"dependsOn": [],
"properties": {
"overprovision": "[variables('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {},
"storageProfile": {},
"networkProfile": {},
"extensionProfile": {
"extensions": [
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-NetworkWatcher",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.Azure.NetworkWatcher",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'NetworkWatcherAgentWindows', 'NetworkWatcherAgentLinux')]",
"typeHandlerVersion": "1.4",
"autoUpgradeMinorVersion": true
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-TeamServicesAgent",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.VisualStudio.Services",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'TeamServicesAgent', 'TeamServicesAgentLinux')]",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"VSTSAccountName": "[parameters('VSTSAccountName')]",
"TeamProject": "[parameters('VSTSTeamProjectName')]",
"DeploymentGroup": "[parameters('VSTSDeploymentGroupName')]",
"AgentName": "[concat(parameters('VMSSName'),'-DG')]",
"Tags": "[parameters('VSTSDeploymentAgentTags')]"
},
"protectedSettings": {
"PATToken": "[parameters('VSTSPATToken')]"
}
}
}
]
}
}
}
}
Now the desired state, of course, is that all nodes will have agent installed so that I can use the Deployment Group inside Release pipeline.
your problem is in the fact that all agents have the same AgentName, so it effectively overwrites the agent and only the latest one "survives". I dont think there is anything you can do, unless you just amend the AgentName and it auto assigns based on computer name.
You can convert this to a script\dsc extension, that way you can calculate everything on the fly.

arm template virtualNetworkName creation appendix issue

I am trying to get a arm template running and have hit an issue with the virtualnetwork creation.
azuredeploy.json
"virtualNetworkName": {
"type": "string",
"metadata": {
"description": "Name of virtual network to be created"
},
"defaultValue": "autohav2VNET"
},
vnet-net.json
"resources": [
{
"name": "[parameters('virtualNetworkName')]",
"type": "Microsoft.Network/virtualNetworks",
"location": "[parameters('location')]",
"apiVersion": "2015-06-15",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('virtualNetworkAddressRange')]"
]
},
"subnets": "[parameters('subnets')]"
}
}
]
The issue I am getting is that the vnet gets created with an appendix such as this: autohav2VNETl5g
So when this gets used to create a loadblancer, the names doe not match the defined parameter and the creation fails.
..../virtualNetworks/AUTOHAV2VNET referenced by resource .... /Microsoft.Network/loadBalancers/sqlLoadBalancer was not found.
Any suggestions?
with the data given it impossible to be sure why this is happening. you are probably passing in a value to the parameter virtualNetworkName. because if you wouldn't, than the vnet name would be: autohav2VNET.
ARM templates do not append anything anywhere just because they are arm templates. they only do what you designed them to do.
to help with debugging: how you are invoking the template and full template + full parameters file.

ARM - get VMSS's private IPs

I have a VMSS deployed with am ARM template.
I want add it's first VM's IP address to the output.
In the Azure Resource Explorer - I see resource as:
...
"ipConfigurations": [
{
"name": "jm-website-script-4-master-ip",
"id": "/subscriptions/0a4f2b9c-***-40b17ef8c3ab/resourceGroups/jm-website-script-4/providers/Microsoft.Compute/virtualMachineScaleSets/jm-website-script-4-master-vmss/virtualMachines/1/networkInterfaces/jm-website-script-4-master-nic/ipConfigurations/jm-website-script-4-master-ip",
"etag": "W/\"09be80d2-76f5-49fc-ad47-0ef836a3799a\"",
"properties": {
"provisioningState": "Succeeded",
"privateIPAddress": "10.0.0.5",
...
So I tried to add to the variables:
"masterVM": "[concat('Microsoft.Compute/virtualMachineScaleSets/jm-website-script-4-master-vmss/virtualMachines/1/networkInterfaces/jm-website-script-4-master-nic/ipConfigurations/jm-website-script-4-master-ip')]",
And then call in outputs:
"outputs": {
"MasterFirstIPConfig": {
"type": "string",
"value": "[reference(variables('masterVM').properties.privateIPAddress)]"
}
}
Which returns me an error:
The language expression property 'Microsoft.WindowsAzure.ResourceStack.Frontdoor.Expression.Expressions.JTokenExpression' can't be evaluated..
I guess something is completely wrong with my masterVM variable definition here, but can't get it.
UPD The solution
Thanks for the answer by 4c74356b41.
The solution looks like next:
variable definition:
"masterVM": "Microsoft.Compute/virtualMachineScaleSets/jm-website-script-4-master-vmss/virtualMachines/1/networkInterfaces/jm-website-script-4-master-nic"
outputs:
"outputs": {
"MasterFirstIPConfig": {
"type": "string",
"value": "[reference(variables('masterVM'),'2016-09-01').ipConfigurations[0].properties.privateIPAddress]"
}
}
Well, you are targeting the wrong resource. You should be targeting the networkInterface (and you don't need to concat in this case), you also have to reference it properly:
"masterVM": "Microsoft.Compute/virtualMachineScaleSets/jm-website-script-4-master-vmss/virtualMachines/1/networkInterfaces/jm-website-script-4-master-nic]"
And reference like so
"value": "[reference(variables('masterVM'),'2016-09-01').ipConfigurations[0].properties.privateIPAddress]"

Resources