I am about to script the deployment of our Azure solution. For that reason I create an Azure IoTHub with a Resource Manager Template. This works very well. But the problem is, I need the
Event Hub-compatible endpoint string for further deployments.
See: https://picload.org/image/rrdopcia/untitled.png
I think, the solution would be, to output it in the template, but I cant get it to work.
The output-section of my template.json actually looks like this:
"outputs": {
"clusterProperties": {
"value": "[reference(parameters('clusterName'))]",
"type": "object"
},
"iotHubHostName": {
"type": "string",
"value": "[reference(variables('iotHubResourceId')).hostName]"
},
"iotHubConnectionString": {
"type": "string",
"value": "[concat('HostName=', reference(variables('iotHubResourceId')).hostName, ';SharedAccessKeyName=', variables('iotHubKeyName'), ';SharedAccessKey=', listkeys(variables('iotHubKeyResource'), variables('iotHubVersion')).primaryKey)]"
}
}
And here are the variables I used:
"variables": {
"iotHubVersion": "2016-02-03",
"iotHubResourceId": "[resourceId('Microsoft.Devices/Iothubs', parameters('iothubname'))]",
"iotHubKeyName": "iothubowner",
"iotHubKeyResource": "[resourceId('Microsoft.Devices/Iothubs/Iothubkeys', parameters('iothubname'), variables('iotHubKeyName'))]",
},
You can read the endpoint from the provisioned IoT Hub within the ARM template and build a connection string like this:
"EventHubConnectionString": "[concat('Endpoint=',reference(resourceId('Microsoft.Devices/IoTHubs',parameters('iothub_name'))).eventHubEndpoints.events.endpoint,';SharedAccessKeyName=iothubowner;SharedAccessKey=',listKeys(resourceId('Microsoft.Devices/IotHubs',parameters('iothub_name')),variables('devices_provider_apiversion')).value[0].primaryKey)]"
The important bit to get the EventHub-compatible endpoint was: resourceId('Microsoft.Devices/IoTHubs', parameters('iothub_name'))).eventHubEndpoints.events.endpoint
That was ripped out of my working ARM template. For clarity, here are some details about the variables/parameters in the above:
variables('devices_provider_apiversion') is "2016-02-03"
parameters('iothub_name') is the name of the IoT Hub that same ARM template is provisioning elsewhere in the template
The output of "listKeys" returns a array of key objects, where in my case the first item was "iothubowner". (... I like the approach to get this described in the question better. :)
One helpful trick that helped me learn what is available for me to read from the resources during execution of the ARM template is to output the entire resource and then find the property I am interested in. Here is how I output all details of the IoT Hub from running the ARM template:
"outputs": {
"iotHub": {
"value": "[reference(resourceId('Microsoft.Devices/IoTHubs',parameters('iothub_name')))]",
"type": "object"
}
}
You can also use this method to output the endpoint (among other things) to be used as input to other templates.
Related
I successfully manually configured a Stream Analytics Job that outputs data into Data Explorer. However, I am unable to set up my infrastructure pipeline using ARM templates. All the documentation only describes the other types of output (e.g. ServiceBus).
I also tried exporting the template in the resource group, but this does not work for my Stream Analytics Job.
How can I configure this output using ARM templates?
For many of such issues, you can use the following process:
Setup the desired configuration manually
Check https://resources.azure.com and find your resource
OR Export the resource using the export template feature
In this case, resources.azure.com is sufficient.
If you look at the Data Explorer output resource, you will see the required ARM representation that you can use in your ARM template:
"name": "xxxxxxx",
"type": "Microsoft.StreamAnalytics/streamingjobs/outputs",
"properties": {
"datasource": {
"type": "Microsoft.Kusto/clusters/databases",
"properties": {
"cluster": "https://yyyyy.westeurope.kusto.windows.net",
"database": "mydatabase",
"table": "mytable",
"authenticationMode": "Msi"
}
},
I'm working on creating an ARM template for a QnAMaker resource type in Azure. After creating a QnAMaker resource and exporting the template it creates a number of resources:
"Microsoft.CognitiveServices/accounts" of kind "QnAMaker"
"Microsoft.Search/searchServices"
"Microsoft.Web/serverfarms"
"Microsoft.Web/sites" including appsettings including "PrimaryEndpointKey": "[concat(parameters('appName'), '-PrimaryEndpointKey')]"
The ARM template has a hardcoded value in the "/sites" resource rather than picking it up from another resource.
In the outputs, there is a link to qna runtime:
"qnaRuntimeEndpoint": {
"type": "String",
"value": "[concat('https://',reference(resourceId('Microsoft.Web/sites', parameters('appName'))).hostNames[0])]"
}
What should the output value be for retrieving a subscription key for the QnAMaker resource?
This uses cognitive services related functions. Looking at some other examples I arrived at...
"outputs": {
"qnaKey":{
"type": "string",
"value": "[listKeys(concat(resourceGroup().id,'/providers/','Microsoft.CognitiveServices/accounts/', parameters('name')),'2016-02-01-preview').key1]"
}
}
Cognitive Services API docs for listKeys
My current setup stores the environment specific application settings within a section the ARM parameter files. So, the structure of ARM parameters files looks something like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deploymentSpecificValues": {
"value": {
"subscriptionId": "0ce7bbfc-9fa4-46b5-8f38-303db907fd89",
"environmentName": "dev",
"azureBlobUri": "https://mytestblob.blob.core.windows.net/"
}
},
"regions": {
"value": [
{
"location": "West US",
"additionalRegionData": "abc"
},
{
"location": "West US 2",
"additionalRegionData": "pqr"
}
]
},
"webAppInfo": {
"value": {
"appName": "mytestapp",
"appSettings": {
"SqlServerConnectionString": "mySqlServerConnectionString",
"Service1Url": "https://www.service1url.com",
"Service2Url": "https://www.service2url.com"
}
}
},
"sku": {
"value": "Dynamic"
}
}
}
Notice the appSettings located under the webAppInfo node of the above code. Each environment has it's own parameters file like the above. I believe that the number of appsettings can keep growing with time and it might clutter the parameters files.
Questions:
Is this the best practice to manage appsetting?
If the answer to the above question is a no, then what's the best practice to handle the appsettings in such scenarios?
A couple thoughts:
What you're doing is fine, but you're right in that 1) it could get unwieldy in param files and 2) generally these param files are in source control, so make sure what you have in there, should be in source.
Check to see if there are values that you can get from the deployment at deployment time, e.g. subscriptionId can be returned from the subscription() function (assuming you want the id of the subscription you are deploying to).
There are two resources in Azure that you can use to store "config" - you can use them slightly differently.
KeyVault can store arbitrary strings (e.g. blobs of json) securely and they can be dynamically referenced (via parameters) at deployment time.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/key-vault-parameter
App-Config is a resource to store config values for just such an occasion - you can pull values out of the config store using the listKeyValue() template function
https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-resource-manager
https://github.com/Azure/azure-quickstart-templates/tree/master/101-app-configuration
That help?
Yes, I think this is a good practice to manage appsettings in ARM template like this.
But if you just want to add/update a setting, it is recommended to use azure portal or Azure CLI.
Reference:
az webapp config appsettings
As part of a Stream Analytics deployment solution I want to retrieve the API key for a Azure Function App in an ARM template via e.g. the listkeys() function. Is there a way to retrieve this key via an ARM template respectively during an ARM deployment and if yes, how?
Thanks
The new Key Management API of Azure Functions has gone live. Its possible via the following ARM Script. Also check this Github issue
"variables": {
"functionAppId": "[concat(parameters('functionAppResourceGroup'),'/providers/Microsoft.Web/sites/', parameters('functionAppName'))]"
},
"resources": [
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVaultName'),'/', parameters('functionAppName'))]",
"apiVersion": "2015-06-01",
"properties": {
"contentType": "text/plain",
"value": "[listkeys(concat(variables('functionAppId'), '/host/default/'),'2016-08-01').functionKeys.default]"
},
"dependsOn": []
}
]
This question has already been answered here:
listKeys for Azure function app
Get Function & Host Keys of Azure Function In Powershell
What is important in this context is to set the 'Minimum TLS Version' to '1.0' before deploying the job. Otherwise you will get failures when testing the connection health.
What I am interested in is reading the output parameters of another deployment in a different resource group.
My ARM templates are something like:
platform.json - sets up the DNS, virtual networks and security
storage.json - sets up databases and other stores
app.json - sets up the web app/api
Each is deployed in different resource groups as they have different life cycles. However, when I deploy the app.json I want to pull in the outputs of the latest platform and storage deployments and use them to configure the app.
Linked templates are not a solution as the linked templates end up deployed in the same resource group as the app which defeats the purpose of segregating your resources in resource groups.
Is there any way I can read the output parameters of a deployment from a different resource group? if not, is Azure planning to support it?
I know there is a way to get the resources by id, using the resourceId function, and look at their properties but I am trying to avoid doing that to not get into a resource reference spagetti.
I know this is an old question but for others who come along, as of 03/12/18 you can definitely do this.
You need to ensure your output is formatted as per the Microsoft documentation for output variables which broadly has the format
"outputs": {
"resourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPAddresses_name'))]"
}
}
You can then use these outputs in your templates by referencing the deployment using a resource reference which has the format
reference(resourceName or resourceIdentifier, [apiVersion], ['Full'])
Note that you will need to provide the api version, as the deployment may use a different api version to the one your parent template uses.
Your reference would then look something like the following
{
"comments": "This would have an output named myOutput you want to use",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-deployment",
"resourceGroup": "...",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": { }
},
{
"comments": "This makes use of myOutput from my-deployment",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-dependent-deployment",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": {
"myValueFromAnotherDeployment": { "value": "[reference('my-deployment', '2017-05-10').outputs.myOutput.value]" }
}
}
}
Note the slightly awkward "repackaging" of the value, where we use myOutput.value as the input to the dependent deployment, and put that in an object with the key "value": "....". This is because ARM parameters must have a 'value' property in order to be valid.
You'll get invalid template errors if you try to use the output directly (because output variables have a 'type' and this is not an allowed key in a parameter). That's why you need to get the value property and then put it back into the value in downstream templates.
How are you doing your deployments? In PowerShell you can do something like:
(Get-AzureResourceGroupDeployment NameOfResourceGroup).Outputs.NameOfOuputProperty.Value
And that will give you the output of the most recent deployment. You can also throw the entire deployment object into a var and have at it that way.
$d = Get-AzureResourceGroupDeployment NameOfResourceGroup
Which would be faster if you needed a many output properties.
That help?
Update for AzureRM cmdlet
The syntax is largely the same:
(Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup -Name NameOfDeployment).Outputs.NameOfOutputProperty.value
If you have multiple deployments you can use:
Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup
To see them all and see what the names are...
Are you using Azure DevOps Release Pipelines? You could just set the output to be created as variables so you can re-use them in the same or a different stage.
We use these extensions on our projects
ARM Outputs
https://github.com/keesschollaart81/vsts-arm-outputs
VSTS replacetokens
https://github.com/qetza/vsts-replacetokens-task