Linked ARM templates result in Invalid Template - azure

Let's start with what I'm trying to accomplish.
What I'd like to do is create an ARM template where I'm retrieving secrets from the Azure Key Vault, without specifying too much details on the specific Key Vault. Sounds easy enough and probably something which is implemented in every production system.
Doing a quick search I discovered the syntax for such a thing is the following.
"parameters": {
"adminPassword": {
"reference": {
"keyVault": {
"id": "[resourceId(subscription().subscriptionId, parameters('vaultResourceGroup'), 'Microsoft.KeyVault/vaults', parameters('vaultName'))]"
},
"secretName": "[parameters('secretName')]"
}
},
From what I gather you need to add this in an external template as the used methods can't be used in the 'main' method.
So I started creating a 'main' ARM template and a new template called appservice.json which contains all of the stuff necessary for my App Service, including the appSettings block which need the secrets from Key Vault.
In my main template I've done the following, as described in the documentation.
{
"apiVersion": "2017-05-10",
"name": "linkedTemplate",
"type": "Microsoft.Resources/deployments",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[uri(deployment().properties.templateLink.uri, 'appservice.json')]",
"contentVersion": "1.0.0.0"
},
However, when deploying I'm confronted with the following error.
"error": {
"code": "InvalidTemplate",
"message": "Unable to process template language expressions for resource '/subscriptions/ba49bae7-2b37-4504-914b-441763a2bcd3/resourceGroups/cfpexchange-jan-test/providers/Microsoft.Resources/deployments/linkedTemplate' at line '1' and column '1526'. 'The language expression property 'templateLink' doesn't exist, available properties are 'name, properties'.'"
}
I also tried the following because I noticed IntelliSense in Visual Studio told me properties doesn't exits and I should use templateLink directly.
{
"apiVersion": "2017-05-10",
"name": "linkedTemplate",
"type": "Microsoft.Resources/deployments",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[uri(deployment().templateLink.uri, 'appservice.json')]",
"contentVersion": "1.0.0.0"
},
This of course isn't right either.
"error": {
"code": "InvalidTemplate",
"message": "Unable to process template language expressions for resource '/subscriptions/ba49bae7-2b37-4504-914b-441763a2bcd3/resourceGroups/cfpexchange-jan-test/providers/Microsoft.Resources/deployments/linkedTemplate' at line '1' and column '1526'. 'The language expression property 'templateLink' doesn't exist, available properties are 'name, properties'.'"
}
And when using it as a variable, like in the documentation
"variables": {
"sharedTemplateUrl": "[uri(deployment().properties.templateLink.uri, 'shared-resources.json')]"
},
...
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[variables('sharedTemplateUrl')]",
"contentVersion": "1.0.0.0"
},
I'm also getting an error.
2018-07-04T19:14:34.4204720Z ##[error]Deployment template validation failed: 'The template variable 'sharedTemplateUrl' is not valid: The language expression property 'templateLink' doesn't exist, available properties are 'template, parameters, mode, debugSetting, provisioningState'.. Please see https://aka.ms/arm-template-expressions for usage details.'.
At this time I'm a bit lost. From what I understand from the documentation it appears I'm doing everything correct. Apparently I'm not. Any ideas on how to continue with this?
For completeness, the two actual files which I'm using at this time:
The main ARM template: https://github.com/Jandev/CfpExchange/blob/b998bb0c49cf369b2f7584e20556aefb5224ace0/Deployment/CFPExchange.json
The linked template: https://github.com/Jandev/CfpExchange/blob/522f48aa19d19730e5474ce11ead57a48d330389/Deployment/appservice.json
There have been multiple iterations on it in order to try and fix it, but as mentioned noting appeared to be working so far.

First of all, this is how you are supposed to use KV in a nested template. Example with Admin password:
"adminPassword": {
"reference": {
"keyVault": {
"id": "kv_resource_id"
},
"secretName": "[concat('secret', copyindex(1))]"
}
},
This section supposed to be in the nested template parameters WHEN you invoke it (just look at the example link).
Your error seems to be in the variable. So the templateLink property is only available when you deploy your main template from the url, if you use local file to deploy main template it wont work.
Compare this to remote template execution:
New-AzureRmResourceGroupDeployment -ResourceGroupName xxx -TemplateUri 'https://paste.ee/d/XI1Rc/0'
As this is a remote url, it should show you the same output, but this time with a templateLink property.
Name Type Value
=============== ========================= ==========
test Object {
"name": "theDeploymentName",
"properties": {
"templateLink": {
"uri": "theRemoteTemplateUri"
},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [],
"outputs": {
"test": {
"type": "Object",
"value": "[deployment()]"
}
}
},
"parameters": {},
"mode": "Incremental",
"provisioningState": "Accepted"
}
}

Related

Resource [parameters('mgName')] Location must be an expression or 'global'

I am experimenting with Azure Management Groups Arm template.
As you can see in this link, I have this Arm template:
{
"$schema": "https://schema.management.azure.com/schemas/2019-08-01/managementGroupDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"mgName": {
"type": "string",
"defaultValue": "[concat('mg-', uniqueString(newGuid()))]"
}
},
"resources": [
{
"type": "Microsoft.Management/managementGroups",
"apiVersion": "2021-04-01",
"name": "[parameters('mgName')]",
"scope": "/",
"location": "eastus",
"properties": {}
}
],
"outputs": {
"output": {
"type": "string",
"value": "[parameters('mgName')]"
}
}
}
Saved as mg.json and it works fine.
Later I start experimenting with validating and testing Arm template using Test-AzTemplate (https://github.com/Azure/arm-ttk). When I run following command to test Arm Template:
Test-AzTemplate -TemplatePath .\mg.json
I get this test error:
[-] Resources Should Have Location (3 ms)
Resource [parameters('mgName')] Location must be an expression or 'global'
Now when I remove "location": "eastus", line form Arm template, the test does not fail and pass the test.
My Question:
Is this location in Management Group Arm required or not required? And why it is failing when it is part of Microsoft documentation! Any idea?
Location is not required in Management Group. As you can check this Azure Create Management Group REST API documentation, location is not needed here.
That's why in the template either you can remove the location or you can provide 'global' as the value, as the test command output specifies.

Use [deployment().properties.templateLink.uri] on local deployments through ARM templates

When I try to use an ARM linked Template like...
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-05-01",
"name": "[concat('EventHubLinkedTemplate-', parameters('eventHubNames')[copyindex('eventHubNameIterator')])]",
"copy": {
"name": "eventHubNameIterator",
"count": "[length(parameters('eventHubNames'))]"
},
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[uri(deployment().properties.templateLink.uri, '/eventHub/template.json')]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"consumerGroups": "[parameters('consumerGroups')]",
"eventHubName": "[concat(variables('eventHubNamespace'), '/', parameters('eventHubNames'))]"
}
}
}
... from parent ARM template, using deploy option of Visual Studio 2019 ARM project, the deployment fails within the next message:
07:48:12 - Resource Microsoft.Resources/deployments 'EventHubLinkedTemplate-test' failed with message '{
"error": {
"code": "InvalidTemplate",
"message": "Unable to process template language expressions for resource '/subscriptions/********-****-****-****-************/resourceGroups/*****/providers/Microsoft.Resources/deployments/EventHubLinkedTemplate-test' at line '127' and column '9'. 'The language expression property 'templateLink' doesn't exist, available properties are 'template, templateHash, parameters, mode, provisioningState'.'",
"additionalInfo": [
{
"type": "TemplateViolation",
"info": {
"lineNumber": 127,
"positionNumber": 9,
"snippet": ""
}
}
]
}
}'
Does anyone knows any way to use [deployment().properties.templateLink.uri] on local deployments through ARM templates?
As far as I can found on the documentation, seems it is not supported yet...
https://github.com/Azure/azure-quickstart-templates/blob/master/1-CONTRIBUTION-GUIDE/best-practices.md
https://github.com/MicrosoftDocs/azure-docs/issues/8748
No, it's impossible.
As the official article said:
The templateLink property is only returned when linking to a remote template with a URL. If you're using a local template, that property isn't available.
No, this is not possible, because for that to work you have to upload the template (else, how you would get a link??).
A simple workaround would be to have a powershell script that uploads everything and starts the deployment. Using Visual Studio for ARM Template development is not the best experience anyway.

Is it possible to do an iterative string replace in an Azure ARM template?

I suspect it's not possible to do what I'm looking for but it's worth a shot!
I have a pipeline for provisioning Azure log query alert rules. The individual alert rules are defined as ARM parameter files, and I use a shared ARM template file to deploy them.
Here's a stripped down version of my template file with most of the parameters omitted.
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logQuery": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Query to execute against the AI resource"
}
}
},
"variables": {
"appInsightsResourceId": "[concat(resourceGroup().id,'/providers/','microsoft.insights/components/', parameters('appInsightsResourceName'))]",
"actionGroupId": "[concat(resourceGroup().id,'/providers/','microsoft.insights/actionGroups/', parameters('actionGroupName'))]",
"linkToAiResource" : "[concat('hidden-link:', variables('appInsightsResourceId'))]"
},
"resources":[{
"name":"[parameters('alertName')]",
"type":"Microsoft.Insights/scheduledQueryRules",
"location": "northeurope",
"apiVersion": "2018-04-16",
"tags": {
"[variables('linkToAiResource')]": "Resource"
},
"properties":{
"description": "[parameters('alertDescription')]",
"enabled": "[parameters('isEnabled')]",
"source": {
"query": "[parameters('logQuery')]",
"dataSourceId": "[variables('appInsightsResourceId')]",
"queryType":"[parameters('logQueryType')]"
},
"schedule":{
"frequencyInMinutes": "[parameters('alertSchedule').Frequency]",
"timeWindowInMinutes": "[parameters('alertSchedule').Time]"
},
"action":{
"odata.type": "Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.Microsoft.AppInsights.Nexus.DataContracts.Resources.ScheduledQueryRules.AlertingAction",
"severity": "[parameters('alertSeverity')]",
"aznsAction":{
"actionGroup":"[array(variables('actionGroupId'))]"
},
"trigger":{
"thresholdOperator":"[parameters('alertTrigger').Operator]",
"threshold":"[parameters('alertTrigger').Threshold]"
}
}
}
}
]
}
You can see how I'm providing the App Insights query as a parameter, so my parameters file could look something like:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logQuery": {
"value": "requests | where resultCode >= 500"
}
}
}
However, these queries can be very long and hard to understand when viewing as an unbreakable JSON string. So I want to parametize this parameter (if you know what I mean) so that the key variables are defined and supplied separately. I was thinking about changing the parameters to something like this, introducing a new parameter holding an array of placeholder replacements for the parametized query...
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logQueryVariables": [
{ "{minCode}": "500" }
],
"logQuery": {
"value": "requests | where resultCode >= {minCode}"
}
}
}
...then finding a way to iterate over the variables array and replace the placeholders in the logQuery parameter, I thought maybe I could use an ARM function or something. But I'm afraid to admit I'm stuck with this part. Is it possible to use the copy syntax to do something like this?
depends on what the end result should look like you can do this different ways, but I'd suggest against doing this in the template and suggest you do this outside of the template and feed in the result. If you really want to achieve exactly what you describe you can do that** with nested deployments. I dont think there is any other way of iterating through an array to build a string in ARM Templates.
This is the closest I could come. We do the input looping in powershell for secretObjects.
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(variables('KeyVaultName'), '/',
variables('secretsObject').secrets[copyIndex()].secretName)]",
"apiVersion": "2018-02-14",
"dependsOn": [
"[concat('Microsoft.KeyVault/vaults/', variables('KeyVaultName'))]"
],
"copy": {
"name": "secretsCopy",
"count": "[length(variables('secretsObject').secrets)]"
},
"properties": {
"value": "[variables('secretsObject').secrets[copyIndex()].secretValue]"
}
}

ARM nested templates: _artifactLocation parameter is not populated correctly when deploying template with nested templates

I am trying to figure out how nested templates work and I have the below templates. I am trying to deploy from VS using the VS deploy mechanism:
Right click on the project > Deploy > New
"Artifact storage account" field is prepopulated with "Automatically create a storage account" and I leave it like that
Click on Deploy button
If you take a look in HelloWorldParent.json template in variables you will see two variables "nestedTemplateUri" and "nestedTemplateUriWithBlobContainerName".
It is my understanding that "nestedTemplateUri" should contain the "blob container name" but that doesn't seem to be the case.
If I deploy with resources > properties > templateLink > "uri": "[variables('nestedTemplateUri')]"
The deployment fails with:
Error: Code=InvalidContentLink; Message=Unable to download deployment
content from
'https://********.blob.core.windows.net/NestedTemplates/HelloWorld.json?sv=2017-07-29&sr=c&sig=ZCJAoOdp08qDWxbzKbXSZzX1VBCf7%2FNSt4aIznFCTPQ%3D&se=2019-03-12T03:39:09Z&sp=r'
The Storage Account is created, the templates, parameters and PS1 script are uploaded
A new deployment is not created in resource group / deployments
If I deploy with resources > properties > templateLink > "uri": "[variables('nestedTemplateUriWithBlobContainerName')]"
The deployments succeeds.
Any idea? Any help is highly appreciated!
HelloWorldParent.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"_artifactsLocation": {
"type": "string",
"metadata": {
"description": "The base URI where artifacts required by this template are located including a trailing '/'"
}
},
"_artifactsLocationSasToken": {
"type": "securestring",
"metadata": {
"description": "The sasToken required to access _artifactsLocation. When the template is deployed using the accompanying scripts, a sasToken will be automatically generated. Use the defaultValue if the staging location is not secured."
},
"defaultValue": ""
}
},
"variables": {
"blobContainerName": "[concat(resourceGroup().name, '-stageartifacts/')]",
"nestedTemplateUriWithBlobContainerName": "[uri(parameters('_artifactsLocation'), concat(variables('blobContainerName'), 'NestedTemplates/HelloWorld.json', parameters('_artifactsLocationSasToken')))]",
"nestedTemplateUri": "[uri(parameters('_artifactsLocation'), concat('NestedTemplates/HelloWorld.json', parameters('_artifactsLocationSasToken')))]"
},
"resources": [
{
"apiVersion": "2017-05-10",
"name": "linkedTemplate",
"type": "Microsoft.Resources/deployments",
"properties": {
"mode": "incremental",
"templateLink": {
"uri": "[variables('nestedTemplateUri')]",
"contentVersion": "1.0.0.0"
}
}
}
],
"outputs": {
"messageFromLinkedTemplate": {
"type": "string",
"value": "[reference('linkedTemplate').outputs.greetingMessage.value]"
},
"_artifactsLocation": {
"type": "string",
"value": "[parameters('_artifactsLocation')]"
}
}
}
HelloWorldParent.parameters.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
}
}
NestedTemplates/HelloWorld.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [],
"outputs": {
"greetingMessage": {
"value": "Hello World (1)",
"type": "string"
}
}
}
Unfortunately VS is a bit "dated" in it's support for your scenario... the problem is that you're using the URI function and the _artifactsLocation does not have a trailing slash. So you have a few options to fix:
1) in the PS1 file in VS there is a line that looks like this:
$OptionalParameters[$ArtifactsLocationName] = $StorageAccount.Context.BlobEndPoint + $StorageContainerName
If you change it to this (add a trailing /):
$OptionalParameters[$ArtifactsLocationName] = $StorageAccount.Context.BlobEndPoint + $StorageContainerName + "/"
It should work - alternatively you can just replace the entire script with this one: https://github.com/Azure/azure-quickstart-templates/blob/master/Deploy-AzureResourceGroup.ps1
Note that if you have other templates that work without the trailing slash, this will be a breaking change.
2) use concat() to create the uri instead of the uri() function. You still have to know whether there is a trailing slash but this change can be done in the template and not the PS1 file.
"nestedTemplateUri": "[concat(parameters('_artifactsLocation'), '/NestedTemplates/HelloWorld.json', parameters('_artifactsLocationSasToken'))]"
Either should work.

ARM deployment fails when properties are passed an an object

When deploying an ARM template using resource iteration, I'd like to pass the resource properties as an object.
Doing this would allow for a different set of parameters to exist within each element the copy array. The reason for this is because some properties may need to be conditionally included or excluded depending on the values of others. For example, in the case of an API Management product, the documentation states the following with regard to the subscriptionsLimit property -
Can be present only if subscriptionRequired property is present and has a value of false.
However, when deploying the example template below the deployment hangs in Azure. Looking in to the related events, I can see that the action the deploy the resource keeps failing with an Internal Server Error (500), but there are no additional details.
If I refer to each parameter in the properties object using variables('productsJArray')[copyIndex()].whatever then the deployment succeeds. However, this is undesirable as it means that every properties object would have to contain identical parameters, which is not always permissible and may cause the deployment to fail.
Example template
Note that I've output variables('productsJArray')[copyIndex()] and it is a valid object. I've even copied the output in to the template and deployed it successfully.
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"apiManagementServiceName": {
"type": "string",
"metadata": {
"description": "The name of the API Management instance."
}
},
"productsJson": {
"type": "string",
"metadata": {
"description": "A JSON representation of the Products to add."
}
}
},
"variables": {
"productsJArray": "[json(parameters('productsJson'))]"
},
"resources": [
{
"condition": "[greater(length(variables('productsJArray')), 0)]",
"type": "Microsoft.ApiManagement/service/products",
"name": "[concat(parameters('apiManagementServiceName'), '/', variables('productsJArray')[copyIndex()].name)]",
"apiVersion": "2018-06-01-preview",
"properties": "[variables('productsJArray')[copyIndex()]]",
"copy": {
"name": "productscopy",
"count": "[if(greater(length(variables('productsJArray')), 0), length(variables('productsJArray')), 1)]"
}
}
]
}
Example parameters
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"apiManagementServiceName": {
"value": "my-api-management"
},
"productsJson": {
"value": "[{\"name\":\"my-product\",\"displayName\":\"My Product\",\"description\":\"My product is awesome.\",\"state\":\"published\",\"subscriptionRequired\":true,\"approvalRequired\":false}]"
}
}
}
Output of variable 'productsJArray[0]'
"outputs": {
"properties": {
"type": "Object",
"value": {
"approvalRequired": false,
"description": "My product is awesome.",
"displayName": "My Product",
"name": "my-product",
"state": "published",
"subscriptionRequired": true
}
}
}
The issue here was that I was passing including name parameter along with other parameters when setting resource properties. This is obviously wrong, but it would have been helpful if MS had handled the error in a more human friendly way (guess they can't think of everything).
I've updated my incoming productsJson parameter -
[{\"name\":\"cs-automation\",\"properties\":{\"displayName\":\"CS Automation Subscription\",\"state\":\"published\",\"description\":\"Allows access to the ConveyorBot v1 API.\",\"subscriptionRequired\":true,\"approvalRequired\":false}}]
And I'm now passing only the required 'properties' -
"resources": [
{
"type": "Microsoft.ApiManagement/service/products",
"name": "[concat(parameters('apiManagementServiceName'), '/', variables('productsJArray')[copyIndex()].name)]",
"apiVersion": "2018-06-01-preview",
"properties": "[variables('productsJArray')[copyIndex()].properties]"
}
]

Resources