How do I get a secret from an azure Vault using an ARM template?
My template:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"resources": [],
"outputs": {
"referenceOutput": {
"type": "object",
"value": "[listSecrets(resourceId('my-resource-group', 'Microsoft.KeyVault/vaults/secrets', 'myKeyVault','mySecret'), '2016-10-01')]"
}
}
}
And then run with:
az group deployment create -g some-rg --template-file ./arm.json
Error:
Deployment failed. Correlation ID: f76de3f2-a9ff-427c-9ae0-b7b24c3fde5d. {
"error": { "code": "BadRequest", "message": "<!DOCTYPE html P
....
<h2>404 - File or directory not found.</h2>\r\n <h3>The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.</h3>\r\n </fieldset></div>\r\n</div>\r\n</body>\r\n</ht
ml>\r\n"
There is no way of doing that with an ARM Template, but you can refecence secrets in an ARM Template to pass those as values.
"password": {
"reference": {
"keyVault": {
"id": "vaultId"
},
"secretName": "secretName"
}
}
But you have to know that this expression cannot be used directly in the template. You can use this in the parameters file and\or when invoking a nested template.
Also, you can use similar expressions for some of the properties of some of the resources (like VM password)
Related
I am experimenting with Azure Management Groups Arm template.
As you can see in this link, I have this Arm template:
{
"$schema": "https://schema.management.azure.com/schemas/2019-08-01/managementGroupDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"mgName": {
"type": "string",
"defaultValue": "[concat('mg-', uniqueString(newGuid()))]"
}
},
"resources": [
{
"type": "Microsoft.Management/managementGroups",
"apiVersion": "2021-04-01",
"name": "[parameters('mgName')]",
"scope": "/",
"location": "eastus",
"properties": {}
}
],
"outputs": {
"output": {
"type": "string",
"value": "[parameters('mgName')]"
}
}
}
Saved as mg.json and it works fine.
Later I start experimenting with validating and testing Arm template using Test-AzTemplate (https://github.com/Azure/arm-ttk). When I run following command to test Arm Template:
Test-AzTemplate -TemplatePath .\mg.json
I get this test error:
[-] Resources Should Have Location (3 ms)
Resource [parameters('mgName')] Location must be an expression or 'global'
Now when I remove "location": "eastus", line form Arm template, the test does not fail and pass the test.
My Question:
Is this location in Management Group Arm required or not required? And why it is failing when it is part of Microsoft documentation! Any idea?
Location is not required in Management Group. As you can check this Azure Create Management Group REST API documentation, location is not needed here.
That's why in the template either you can remove the location or you can provide 'global' as the value, as the test command output specifies.
I am trying to deploy a synapse instance via an ARM template and the deployment is successful via the Azure DevOps portal, but when I try to deploy the same template with an Azure Keyvault linked service I encounter the following error:
##[error]At least one resource deployment operation failed. Please list deployment
operations for details. Please see https://aka.ms/DeployOperations for usage details.
##[error]Details:
##[error]BadRequest:
After inspecting the activity logs from the Synapse instance I found out the following:
"resourceGroupName": "platform-test-group",
"resourceProviderName": {
"value": "Microsoft.Synapse",
"localizedValue": "Microsoft.Synapse"
},
"resourceType": {
"value": "Microsoft.Synapse/workspaces/linkedservices",
"localizedValue": "Microsoft.Synapse/workspaces/linkedservices"
},
"resourceId": "/subscriptions/xxxx-xxxx-xxxx-xxxx/resourcegroups/platform-test-group/providers/Microsoft.Synapse/workspaces/synapsedataapp/linkedservices/AzureKeyVault",
"status": {
"value": "Failed",
"localizedValue": "Failed"
},
"subStatus": {
"value": "NotFound",
"localizedValue": "Not Found (HTTP Status Code: 404)"
},
"submissionTimestamp": "2022-02-01T02:30:31.1471914Z",
"subscriptionId": "xxxx-xxxx-xxxx-xxxx",
"tenantId": "0f44c5d4-xxxx-xxxx-xxxxx",
"properties": {
"statusCode": "NotFound",
"serviceRequestId": null,
"statusMessage": "{\"error\":{\"code\":\"BadRequest\",\"message\":\"\"}}",
"eventCategory": "Administrative",
"entity": "/subscriptions/xxxx-xxxx-xxxx-xxxx/resourcegroups/platform-test-group/providers/Microsoft.Synapse/workspaces/synapsedataapp/linkedservices/AzureKeyVault",
"message": "Microsoft.Synapse/workspaces/linkedservices/write",
"hierarchy": "xxxx-xxxx-xxxx-xxxx/Enterprise/Group/Group-Test/xxxx-xxxx-xxxx-xxxx"
},
"relatedEvents": []
}
As you can see, the 404 error appears when the template tries to deploy to the tenant id which is not found, however, when I deploy the keyvault via the synapse UI I encounter no error.
Below is the code snippet that I use in my ARM template to deploy the keyvault to the synapse instance:
{
"name": "[concat(variables('workspaceName'), '/AzureKeyVault')]",
"type": "Microsoft.Synapse/workspaces/linkedservices",
"apiVersion": "2021-06-01-preview",
"properties": {
"annotations": [],
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "https://data-test-kv.vault.azure.net/"
}
},
"dependsOn": [
"[variables('workspaceName')]"
]
}
Am I missing some kind of permission or connection that I need to enable? Why am I able to deploy successfully through the UI but not through the ARM template? Any comment or suggestion is greatly valued, so please feel free to comment or improve this question.
I had to contact Microsoft support and their reply was the following:
ARM templates cannot be used to create a linked service. This is due to the fact that linked services are not ARM resources, for examples, synapse workspaces, storage account, virtual networks, etc. Instead, a linked service is classified as an artifact. To still complete the task at hand, you will need to use the Synapse REST API or PowerShell. Below is the link that provides guidance on how to use the API. https://learn.microsoft.com/en-us/powershell/module/az.synapse/set-azsynapselinkedservice?view=azps-7.1.0
This limitation in ARM is applied only to Synapse and they might fix this in the future.
Additional references:
https://feedback.azure.com/d365community/idea/05e41bf1-0925-ec11-b6e6-000d3a4f07b8
https://feedback.azure.com/d365community/idea/48f1bf78-2985-ec11-a81b-6045bd7956bb
In Synapse unlike ADF, linked-services are not part of arm-templates. They are called artifacts and it comprises: Note Books, Spark Definitions, Linked Services, Pipelines etc.
You can find the full article here: https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/how-to-use-ci-cd-integration-to-automate-the-deploy-of-a-synapse/ba-p/2248060
In short, first, deploy Synapse using arm templates. And then set up the linked services:
- task: Synapse workspace deployment#1
displayName: 'Setup:Synapse KeyVault Linked Service'
inputs:
TemplateFile: '$(Build.Repository.LocalPath)/TemplateForWorkspace.json'
ParametersFile: '$(Build.Repository.LocalPath)/TemplateParametersForWorkspace.json'
azureSubscription: '${{ parameters.environments.serviceConnectionId }}'
ResourceGroupName: '$(computeResourceGroupName)'
TargetWorkspaceName: '$(synapseWorkspaceName)'
DeleteArtifactsNotInTemplate: true
OverrideArmParameters: |
synapseLinkedServiceKV: $(synapseLinkedServiceKV)
workspaceName: $(synapseWorkspaceName)
Environment: 'prod'
TemplateForWorkspace.json:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string"
},
"synapseLinkedServiceKV": {
"type": "string"
}
},
"variables": {
"workspaceId": "[concat('Microsoft.Synapse/workspaces/', parameters('workspaceName'))]"
},
"resources": [
{
"name": "[concat(parameters('workspaceName'), '/' , parameters('synapseLinkedServiceKV'))]",
"type": "Microsoft.Synapse/workspaces/linkedServices",
"apiVersion": "2019-06-01-preview",
"properties": {
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "[concat('https://', parameters('synapseLinkedServiceKV'), '.vault.azure.net/')]"
},
"annotations": [],
"description": "Linked Service to Azure KeyVault. KeyVault is used to primarily fetch secrets"
},
"dependsOn": []
}
]
}
TemplateParametersForWorkspace.json:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"value": ""
},
"synapseLinkedServiceKV": {
"value": ""
}
}
}
It deletes the existing artifacts and deploys the one above. You would first need to install the task extension on your Azure Devops for Synapse workspace deployment#1
Note above template was auto-generated. In synapse studio, goto Git Configuration and point it to your repo. It will submit the changes to the branch workspace_publish. You can copy and build on top of the specific artifact code.
I am trying to create a Resource Group using the below ARM template.
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.1",
"parameters": {
"rgName": {
"type": "string"
},
"rgLocation": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[parameters('rgLocation')]",
"name": "[parameters('rgName')]",
"properties": {}
}
],
"outputs": {}
}
And the Parameter file is
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"rgName": {
"value": "sriram"
},
"rgLocation": {
"value": "southcentralus"
}
}
}
Is there any mistake in the above json files. Because am getting the following error.
Unable to load schema from 'https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json'. No schema request service available(768)
I ignored this error and tried running these templates in the Azure pipeline and got the following error.
"No HTTP resource was found that matches the request URI 'https://management.azure.com/subscriptions/****-****-*****-****/resourcegroups/<Rsource Group Name>/providers/Microsoft.Resources/resourceGroups/<new RG name>?api-version=2018-05-01'
Can anyone help me out? Thank you.
Your parameters file should use this https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentParameters.json# as a schema reference.
The schema your parameter file are using does not has affect on the creation of resource groups. With the command and your original script
az deployment create --template-file tem.json --parameters #para.json --location southcentralus
I could create the new resource group:
The error you are facing just caused by you are trying to create a new resource group within a exists resource group. This does not allowed.
Here suggest you using Command line task to achieve what you want, by using the command I show you above.
az deployment create --template-file $(Build.SourcesDirectory)/{Template}.json --parameters #$(Build.SourcesDirectory)/{parameter}.json --location southcentralus
I've searched online and browsed the available powershell cmdlets to try and find a solution for this problem but have been unsuccessful. Essentially, I have a few Data Factory pipelines that copy/archive incoming files and will use a web http post component that will invoke a Logic App that connects to a Blob container and will delete the incoming file. The issue I'm facing is that we have several automation runbooks that will rest Blob access keys every X days. When the Blob keys get reset the Logic App will fail whenever this happens because the connection is manually created in the designer itself and I can't specify a connection string that could pull from the Keyvault, as an example. Inside of the {Logic App > API Connections > Edit API Connection} we can manually update the connection string/key but obviously for an automated process we should be able to do this programmatically.
Is there a powershell cmdlet or other method I'm not seeing that would allow me to update/edit the API Connections that get created when using and Blob component inside a Logic App?
Any insights is appreciated!
Once you've rotated your key in the storage account, you can use an ARM template to update your connection API. In this ARM template, the connection api is created referencing the storage account internally so you don't have to provide the key:
azuredeploy.json file:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"azureBlobConnectionAPIName": {
"type": "string",
"metadata": {
"description": "The name of the connection api to access the azure blob storage."
}
},
"storageAccountName": {
"type": "string",
"metadata": {
"description": "The Storage Account Name."
}
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Web/connections",
"name": "[parameters('azureBlobConnectionAPIName')]",
"apiVersion": "2016-06-01",
"location": "[resourceGroup().location]",
"scale": null,
"properties": {
"displayName": "[parameters('azureBlobConnectionAPIName')]",
"parameterValues": {
"accountName": "[parameters('storageAccountName')]",
"accessKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')),'2015-05-01-preview').key1]"
},
"api": {
"id": "[concat('subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/azureblob')]"
}
},
"dependsOn": []
}
]
}
azuredeploy.parameters.json file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"azureBlobConnectionAPIName": {
"value": "myblobConnectionApiName"
},
"storageAccountName": {
"value": "myStorageAccountName"
}
}
}
You can them execute the arm template like that:
Connect-AzureRmAccount
Select-AzureRmSubscription -SubscriptionName <yourSubscriptionName>
New-AzureRmResourceGroupDeployment -Name "ExampleDeployment" -ResourceGroupName "MyResourceGroupName" `
-TemplateFile "D:\Azure\Templates\azuredeploy.json" `
-TemplateParameterFile "D:\Azure\Templates\azuredeploy.parameters.json"
to get started with ARM template and powerhsell, you cam have a look at this article:
Deploy resources with Resource Manager templates and Azure PowerShell
I need to use a tenant (directory tenant) name in my ARM templates (especially when creating Web Apps).
It is possible to get subscription name using subscription().displayName however, how can I get my associated directory tenant name?
The expressions like [subscription().tenantId.displayName] or [subscription().tenantId.Name] aren't working and also I'm unable to find any presence of this property on the web.
The way I won't hardcode it is that it can be easily changed by subscription owner or account admin that's why I'm looking for some existing variable\parameter\etc
The tenantId is now available with template functions. Use the subscription function: subscription().
Try the following output examples for reference:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [],
"outputs": {
"subscriptionOutput": {
"value": "[subscription()]",
"type" : "object"
}
}
}
For the tenant id:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [],
"outputs": {
"subscriptionOutput": {
"value": "[subscription().tenantId]",
"type" : "object"
}
}
}
Reference:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-functions-resource#subscription
There is no way to do this. you can only get these values back:
{
"id": "/subscriptions/{subscription-id}",
"subscriptionId": "{subscription-id}",
"tenantId": "{tenant-id}",
"displayName": "{name-of-subscription}"
}
reference: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-functions-resource#subscription