Azure ARM templates - using the output of other deployments - azure

What I am interested in is reading the output parameters of another deployment in a different resource group.
My ARM templates are something like:
platform.json - sets up the DNS, virtual networks and security
storage.json - sets up databases and other stores
app.json - sets up the web app/api
Each is deployed in different resource groups as they have different life cycles. However, when I deploy the app.json I want to pull in the outputs of the latest platform and storage deployments and use them to configure the app.
Linked templates are not a solution as the linked templates end up deployed in the same resource group as the app which defeats the purpose of segregating your resources in resource groups.
Is there any way I can read the output parameters of a deployment from a different resource group? if not, is Azure planning to support it?
I know there is a way to get the resources by id, using the resourceId function, and look at their properties but I am trying to avoid doing that to not get into a resource reference spagetti.

I know this is an old question but for others who come along, as of 03/12/18 you can definitely do this.
You need to ensure your output is formatted as per the Microsoft documentation for output variables which broadly has the format
"outputs": {
"resourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPAddresses_name'))]"
}
}
You can then use these outputs in your templates by referencing the deployment using a resource reference which has the format
reference(resourceName or resourceIdentifier, [apiVersion], ['Full'])
Note that you will need to provide the api version, as the deployment may use a different api version to the one your parent template uses.
Your reference would then look something like the following
{
"comments": "This would have an output named myOutput you want to use",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-deployment",
"resourceGroup": "...",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": { }
},
{
"comments": "This makes use of myOutput from my-deployment",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-dependent-deployment",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": {
"myValueFromAnotherDeployment": { "value": "[reference('my-deployment', '2017-05-10').outputs.myOutput.value]" }
}
}
}
Note the slightly awkward "repackaging" of the value, where we use myOutput.value as the input to the dependent deployment, and put that in an object with the key "value": "....". This is because ARM parameters must have a 'value' property in order to be valid.
You'll get invalid template errors if you try to use the output directly (because output variables have a 'type' and this is not an allowed key in a parameter). That's why you need to get the value property and then put it back into the value in downstream templates.

How are you doing your deployments? In PowerShell you can do something like:
(Get-AzureResourceGroupDeployment NameOfResourceGroup).Outputs.NameOfOuputProperty.Value
And that will give you the output of the most recent deployment. You can also throw the entire deployment object into a var and have at it that way.
$d = Get-AzureResourceGroupDeployment NameOfResourceGroup
Which would be faster if you needed a many output properties.
That help?
Update for AzureRM cmdlet
The syntax is largely the same:
(Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup -Name NameOfDeployment).Outputs.NameOfOutputProperty.value
If you have multiple deployments you can use:
Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup
To see them all and see what the names are...

Are you using Azure DevOps Release Pipelines? You could just set the output to be created as variables so you can re-use them in the same or a different stage.
We use these extensions on our projects
ARM Outputs
https://github.com/keesschollaart81/vsts-arm-outputs
VSTS replacetokens
https://github.com/qetza/vsts-replacetokens-task

Related

Parallel deployment of Azure AppService using ARM template

I am trying to solve a use-case of deploying 20 to 30 Azure AppServices using ARM template based on Admin decision.
This is happening through c# webapi using Microsoft Fluent library(Microsoft.Azure.Management.Fluent & Microsoft.Azure.Management.ResourceManager.Fluent),
var creds = new AzureCredentialsFactory().FromServicePrincipal(clientId,
clientSecret,
tenantId,
AzureEnvironment.AzureGlobalCloud);
var azure = Azure.Authenticate(creds).WithSubscription(subscriptionId);
var deployment = azure.Deployments.Define($"deployment-{userName}")
.WithExistingResourceGroup(resourceGroupName)
.WithTemplate(templateJson.ToString())
.WithParametersLink(templateParamsBlobURL, "1.0.0.0")
.WithMode(Microsoft.Azure.Management.ResourceManager.Fluent.Models.DeploymentMode.Incremental)
.Create();
Problem statement
When the admin decides to run 20 AppServices, then the above lines of code will execute and provision these AppServices, the decision is based on admin.
For me the Deployment is happening in sequential manner i.e., upon completing one AppService deployment then it triggers the next AppService deployment, which takes huge time to complete the entire operation. I am trying to achieve the deployment of 20 AppServices(decided by admin) in parallel so that the provisioning completes at the earliest
Kindly assist me on how to achieve the parallel deployment of AppServices using ARM template
Thanks for confirming #Guptha, posted the same as an answer to help other community members for the similar query , To iteration multiple resources at a time in ARM TEMPLATE .
Use your Azure Resource Manager template to create multiple instances
of a resource (ARM template). You can dynamically set the quantity of
resources to deploy by adding a copy loop to the resources section of
your template.
For example to create multiple storage accounts ;
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageCount": {
"type": "int",
"defaultValue": 3
}
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-04-01",
"name": "[concat(copyIndex(),'storage', uniqueString(resourceGroup().id))]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "Storage",
"properties": {},
"copy": {
"name": "storagecopy",
"count": "[parameters('storageCount')]"
}
}
]
}
For complete setup please refer this MICROSOFT DOCUMENTATION|Resource iteration in ARM templates

How to deploy multiple Azure resources from a local environment? (VS Code)

I have a web app that requires an Azure App Service, Azure Function, and Cosmos DB Database to run. I'm thinking about setting up a way for myself and the other developers to trigger a deployment of the full application to a development resource group specific to the developer that will contain a full version of the app. For example, a dev-John resource group that would contain a dev-John-AppService, dev-John-Function, and dev-John-CosmosDB. Is there an easy way to do this in VSCode using ARM templates?
The Azure Resource Manager Tools for Visual Studio Code provide language support, resource snippets, and resource autocompletion. These tools help create and validate Azure Resource Manager templates (ARM templates).
Create and open with Visual Studio Code a new file for example deploy.json. Just enter arm! into the code editor, which initiates Azure Resource Manager snippets. You can use this snippet to create your ARM template as you desire.
You can achieve your goal by using the Parameters and Functions element of your ARM template.
In the parameters section of the template, you specify the values that are provided when deployment is executed to customize resource deployment. Similarly you can also use functions, ARM template functions add flexibility to your ARM template by dynamically getting values during deployment.
The below ARM template can be used to deploy same resource with customization with the help of parameters and functions.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageName": {
"type": "string",
"minLength": 3,
"maxLength": 24
}
},
"functions": [],
"variables": {},
"resources": [{
"name": "[parameters('storageName')]",
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"tags": {
"displayName": "[parameters('storageName')]"
},
"location": "[resourceGroup().location]",
"kind": "StorageV2",
}],
"outputs": {}
}
Here when we added "[parameters('storageName')]" we used the parameters function. Lastly you can even create PowerShell scripts to call the ARM templates.
I would suggest to read this Deploy to multiple Azure environments by using JSON ARM template features document and Exploring ARM Templates: Azure Resource Manager Tutorial for more information.

ARM template deployment : re-use task parameters in parameters file

I use the ARM template deployment task in my release pipeline. The task has a parameter called Location.
In my parameters.json file, I also have a parameter called Location. How could I change this file so that it reads the value set on the task itself ?
Currently the value is read from a variable group's variable coming from azure keyvault. But I think it's overkill to have this location parameter coming from the vault.
Based on my test, I notice that the value in xxx.parameters.json couldn't read the value of the variable in pipeline.
For example:
I set the variable(test : abc) in Pipeline variables. Then I use it in the xxx.parameters.json file.
When the Resource group creates, the vaiable doesn't be read in the resource group.
The parameters.json sample:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"adminUsername": {
"value": "test"
},
"location": {
"value": "$(test)"
}
}
}
From the result, it seems that the location value couldn't be automatic used in the json file.
You may need to use the tokenizer task to manually override the specific value in the json file. Then the value could be used in the resource group.
Or you could directly use the Override template parameters in the ARM Template task.
Hope this could help.
Location*:
For Resource Group deployment scope: Location for deploying the resource group. If the resource group already exists in the subscription, then this value will be ignored.
For other deployment scopes: Location for storing the deployment metadata.
https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/AzureResourceManagerTemplateDeploymentV3
Given this information, you can set the location of the resource group via the task, and use this for all of your resources as well. If you want your resources to be in a different location than your resource group, you will not be able to leverage the task parameter.
This is how it looks like in ARM:
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/resource-location?tabs=azure-powershell#use-location-parameter

What is the best way to manage your environment specific app settings in Azure?

My current setup stores the environment specific application settings within a section the ARM parameter files. So, the structure of ARM parameters files looks something like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deploymentSpecificValues": {
"value": {
"subscriptionId": "0ce7bbfc-9fa4-46b5-8f38-303db907fd89",
"environmentName": "dev",
"azureBlobUri": "https://mytestblob.blob.core.windows.net/"
}
},
"regions": {
"value": [
{
"location": "West US",
"additionalRegionData": "abc"
},
{
"location": "West US 2",
"additionalRegionData": "pqr"
}
]
},
"webAppInfo": {
"value": {
"appName": "mytestapp",
"appSettings": {
"SqlServerConnectionString": "mySqlServerConnectionString",
"Service1Url": "https://www.service1url.com",
"Service2Url": "https://www.service2url.com"
}
}
},
"sku": {
"value": "Dynamic"
}
}
}
Notice the appSettings located under the webAppInfo node of the above code. Each environment has it's own parameters file like the above. I believe that the number of appsettings can keep growing with time and it might clutter the parameters files.
Questions:
Is this the best practice to manage appsetting?
If the answer to the above question is a no, then what's the best practice to handle the appsettings in such scenarios?
A couple thoughts:
What you're doing is fine, but you're right in that 1) it could get unwieldy in param files and 2) generally these param files are in source control, so make sure what you have in there, should be in source.
Check to see if there are values that you can get from the deployment at deployment time, e.g. subscriptionId can be returned from the subscription() function (assuming you want the id of the subscription you are deploying to).
There are two resources in Azure that you can use to store "config" - you can use them slightly differently.
KeyVault can store arbitrary strings (e.g. blobs of json) securely and they can be dynamically referenced (via parameters) at deployment time.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/key-vault-parameter
App-Config is a resource to store config values for just such an occasion - you can pull values out of the config store using the listKeyValue() template function
https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-resource-manager
https://github.com/Azure/azure-quickstart-templates/tree/master/101-app-configuration
That help?
Yes, I think this is a good practice to manage appsettings in ARM template like this.
But if you just want to add/update a setting, it is recommended to use azure portal or Azure CLI.
Reference:
az webapp config appsettings

Azure ARM Templates and REST API

I'm trying to learn Azure Resource Templates and am trying to understand the workflow behind when to use them and when to use the REST API.
My sense is that creating a Virtual Network and Subnets in Azure is a fairly uncommon occurance, once you get that set up as you want you don't modify it too frequently, you deploy things into that structure.
So with regard to an ARM Template let's say I have a template with resources for VNET and Subnet. To take the example from https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-template-walkthrough#virtual-network-and-subnet I might have:
{
"apiVersion": "2015-06-15",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"10.0.0.0/16"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "10.0.0.0/24"
}
}
]
}
}
which I deploy to a Resource Group. Let's say I then add a Load Balancer and redeploy the template. In this case the user then gets asked to supply the value for the vnetName parameter again and of course cannot supply the same value so we would end up with another VNET which is not what we want.
So is the workflow that you define your ARM Template (VNET, LBs, Subnets, NICs etc) in one go and then deploy? Then when you want to deploy VMs, Scale Sets etc you use the REST API to deploy then to the Resource Group / VNET Subnet? Or is there a way to incrementally build up an ARM Template and deploy it numerous times in a way that if a VNET already exists (for example) the user is not prompted to supply details for another one?
I've read around and seen incremental mode (default unless complete is specified) but not sure if this is relevant and if it is how to use it.
Many thanks for any help!
Update
OK so I can now use azure group deployment create -f azuredeploy.json -g ARM-Template-Tests -m Incremental and have modified the VNET resource in my template from
{
"apiVersion": "2016-09-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
to
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('addressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet1Name')]",
"properties": {
"addressPrefix": "[parameters('subnet1Prefix')]"
}
},
{
"name": "[parameters('gatewaySubnet')]",
"properties": {
"addressPrefix": "[parameters('gatewaySubnetPrefix')]"
}
}
]
}
},
but the subnets don't change. Should they using azure group deployment create -f azuredeploy.json -g ARM-Template-Tests -m Incremental
I am going to piggy back on this Azure documentation. Referencing appropriate section below:
Incremental and complete deployments
When deploying your resources,
you specify that the deployment is either an incremental update or a
complete update. By default, Resource Manager handles deployments as
incremental updates to the resource group.
With incremental deployment, Resource Manager
leaves unchanged resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
With complete deployment, Resource Manager:
deletes resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
To choose Incremental update or Complete update it depends on if you have resources that are in use. If devops requirement is to always have resources in sync with what is defined in the json template then Complete Update mode should be used. The biggest benefit of using templates and source code for deploying resources is to prevent configuration drift and it is beneficial to use Complete Update mode.
As for specifying the parameters if you specify in parameters file then you don't have to specify them again.
A new template can be deployed in incremental mode which would add new resources to the existing resource group. Define only the new resources in the template, existing resources would not be altered.
From powershell use the following cmdlet
New-AzureRmResourceGroupDeployment -ResourceGroupName "YourResourceGroupName" -TemplateFile "path\to\template.json" -Mode Incremental -Force
My rule of thumb is for things that I want to tear down, or for things I want to replicate across Subscriptions, I use ARM templates.
For example we want things in test environments, I just ARM it up, build on the scripts as developers request things ("Hey I need a cache", "Oh by the way I need to start using a Service Bus"), using incremental mode we can just push it out to Dev, then as we migrate up to different environments you just deploy to a different Subscription in Azure, and it should have everything ready to go.
Also, we've started provisioning our own Cloud Load Test agents in a VMSS, a simple ARM template that's called by a build to scale up to x number of machines, then when done, we just trash the Resource Group. It's repeatable and reliable, sure you can script it up, but as TFS has a task to deploy these things (also with schedules)...
One of the beautiful things I've come across is Key Vault, when you ARM it up and poke all the values from your service busses, storage accounts/whatevers, you can simply get the connection strings/keys/whatevers and put them straight into the Key Vault, so you never need to worry about it, and if you want to regenerate anything (say a dev wants to change the name of a cache or whatever, or accidentally posted the key to GitHub), just redeploy (often I'll just trash the whole Resource Group) and it updates the vault for you.

Resources