I am trying to use nested templates to deploy a resource group and multiple resources within it on subscription level.
Microsoft documentation has an example of deploying resource group and storage account that I'm trying to follow. I am trying to create another inner level of dependency between a Storage Account resource and a Container resource. That is, the container should only be deployed after the deployment of the storage account is finished. Here is simplified version of my template:
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"rgName": {
"type": "string"
},
"rgLocation": {
"type": "string"
},
"storagePrefix": {
"type": "string",
"maxLength": 11
},
"containerName": {
"type": "string"
}
},
"variables": {
"storageName": "[concat(parameters('storagePrefix'), uniqueString(subscription().id, parameters('rgName')))]"
},
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2021-04-01",
"name": "[parameters('rgName')]",
"location": "[parameters('rgLocation')]",
"properties": {}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"name": "storageDeployment",
"resourceGroup": "[parameters('rgName')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('rgName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('storageName')]",
"location": "[parameters('rgLocation')]",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2"
},
{
"type": "Microsoft.Storage/storageAccounts/blobServices/containers",
"apiVersion": "2021-06-01",
"name": "[format('{0}/default/{1}', variables('storageName'), parameters('containerName'))]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('storageName'))]"
]
}
],
"outputs": {}
}
}
}
],
"outputs": {}
}
When I try to deploy this template using PowerShell script New-AzSubscriptionDeployment, I get the following error:
| InvalidTemplate - Long running operation failed with status 'Failed'. Additional Info:'Deployment template validation failed: 'The resource 'Microsoft.Storage/storageAccounts/myStorageAccount' is not defined in the template. Please see https://aka.ms/arm-template for usage details.'.'
I kind of know it has to do with the dependsOn part of the container resource. But how can I resolve this problem?
EDIT: The selected answer solves the problem with dependencies, however the issue still persists in cases where a value needs to be called using concat or listKeys expressions. Here's an example where setting the value for AzureWebJobsStorage throws an error in a nested template:
{
"type": "Microsoft.Web/sites",
[ ... ]
"dependsOn": [
"[variables('hostingPlanName')]",
"[variables('functionAppStorageAccountName')]"
],
"properties": {
"serverFarmId": "[variables('hostingPlanName')]",
"siteConfig": {
"appSettings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('functionAppstorageAccountName'), ';EndpointSuffix=', environment().suffixes.storage, ';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('functionAppstorageAccountName')), '2021-04-01').keys[0].value)]"
}
[ ... ]
]
}
The value for AzureWebJobsStorage causes the deployment to fail with the following error:
`Status Message: The Resource 'Microsoft.Storage/storageAccounts/stfuncaedotestfeb16g' under resource group '<null>' was not found. For more details
| please go to https://aka.ms/ARMResourceNotFoundFix (Code:ResourceNotFound) CorrelationId: 55942377-6d0f-40ec-9733-33b9c3ea13de
I tried being more verbose by using resource group name (and then subscription ID), but that didn't solve the problem.
You should be able to use:
"dependsOn": [
"[variables('storageName')]"
]
Note that will only work if there is no other resource in the template with the same name - otherwise you have to manually construct the full resourceId, like:
[format('{0}/resourceGroups/{1}/providers/Microsoft.Storage/storageAccounts/{2}', subscription().id, parameters('rgName'), variables('storageName'))]
The latter form will always work, just a bit more verbose.
A bit more detail is that the resourceId function doesn't work as you would expect at subscription scope.
Related
My goal is to deploy a logic app with a system managed identity and a role assignment for that identity. Preferably, this is done in one ARM template.
I have a setup that fails the first run, but succeeds successive runs. Correct me if I'm wrong, but I think that the reason for this is that the deployment of the role assignment happens before the managed identity of the logic app is "ready", hence the following error I get the first time that I deploy the template. I don't get this error the second time I deploy the template, probably because the Identity already exists at that time.
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "PrincipalNotFound",
"message": "Principal *** does not exist in the directory ***."
}
]
}
My template (removed logic app definition for brevity). In this case the identity of the logic app requires access to a storage account which is located in another resource group.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logicAppName": {
"type": "string"
},
"storageAccountResourceGroup": {
"type": "String"
},
"storageAccountName": {
"type": "String"
}
},
"variables": {
"logicAppResourceId": "[resourceId('Microsoft.Logic/workflows', parameters('logicAppName'))]",
"Storage Blob Data Contributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]"
},
"resources": [
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[parameters('logicAppName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"state": "Enabled",
"definition": {
...
}
}
},
{
"type": "Microsoft.Resources/deployments",
"name": "[concat('RoleAssignment-', parameters('logicAppName'))]",
"apiVersion": "2020-06-01",
"resourceGroup": "[parameters('storageAccountResourceGroup')]",
"subscriptionId": "[subscription().subscriptionId]",
"dependsOn": [
"[parameters('logicAppName')]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2018-09-01-preview",
"type": "Microsoft.Storage/storageAccounts/providers/roleAssignments",
"name": "[concat(parameters('storageAccountName'), '/Microsoft.Authorization/', guid(subscription().subscriptionId, parameters('logicAppName')))]",
"properties": {
"roleDefinitionId": "[variables('Storage Blob Data Contributor')]",
"principalId": "[reference(variables('logicAppResourceId'), '2019-05-01', 'Full').identity.principalId]"
}
}
]
}
}
}
]
}
As you can see in the template, I added a dependsOn on the logic app itself. However that doesn't seem to be sufficient.
Does anyone have a solution for this?
Thank you!
I found the answer here: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-template#new-service-principal
Deployment works consistently after adding "principalType": "ServicePrincipal"
I have an ARM template which creates a Storage Account, App Service Plan, Application Insights and a Function App. In the dependsOn section of the Function App, I configured it to be depending on the other three resources. But when I deploy the template, it fails with the following error:
Status Message: The Resource 'Microsoft.Storage/storageAccounts/dummystorage' under resource group 'dummy-rg' was not found.
I see that the Storage Account, ASP and App Insights are being created first, followed by the Function App. So it seems the dependency is being honored. It's almost like the Storage Account provisioning is not completely finished when the Function App is being created. As you can see in the template below, the Function App resource uses the listKeys to get the Storage Account keys as part of the provisioning.
Does anybody have any idea how I can prevent this from happening? I've read everything in the docs regarding dependencies, and I believe this should just work.
One remark is that the storage account is being deployed based on a condition. I can't imagine it has something to do with my issue, but I just wanted to mention it.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"functionAppName": {
"type": "string"
},
"appServicePlanName": {
"type": "string"
},
"appInsightsName": {
"type": "string"
},
"deployStorage": {
"type": "bool",
"defaultValue": true
},
"storageAccountName": {
"type": "string"
},
"utcDateTime": {
"type": "string",
"defaultValue": "[utcNow()]"
}
},
"variables": {
"storageAccountDeploymentName": "[concat('storageDeploy-', parameters('utcDateTime'))]",
},
"resources": [
{
"name": "[parameters('functionAppName')]",
"apiVersion": "2018-11-01",
"type": "Microsoft.Web/sites",
"kind": "functionapp",
"location": "[parameters('location')]",
"tags": "[parameters('tags')]",
"dependsOn": [
"[parameters('appInsightsName')]",
"[parameters('appServicePlanName')]",
"[resourceId('Microsoft.Resources/deployments', variables('storageAccountDeploymentName'))]"
],
"properties": {
"name": "[parameters('functionAppName')]",
"siteConfig": {
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~3"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "powershell"
},
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "[reference(concat('Microsoft.Insights/components/', parameters('appInsightsName')), '2015-05-01').InstrumentationKey]"
},
{
"name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
"value": "[reference(concat('Microsoft.Insights/components/', parameters('appInsightsName')), '2015-05-01').ConnectionString]"
},
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',parameters('storageAccountName'),';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',parameters('storageAccountName'),';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[concat(toLower(parameters('functionAppName')), '813b')]"
}
],
"use32BitWorkerProcess": false,
"powerShellVersion": "[variables('powerShellVersion')]"
},
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
"clientAffinityEnabled": true
}
},
{
"name": "[parameters('appServicePlanName')]",
"apiVersion": "2018-11-01",
"type": "Microsoft.Web/serverfarms",
"location": "[parameters('location')]",
"tags": "[parameters('tags')]"
// snip
},
{
"name": "[parameters('appInsightsName')]",
"apiVersion": "2020-02-02-preview",
"type": "microsoft.insights/components",
"location": "[parameters('location')]",
"tags": "[parameters('tags')]",
// snip
},
{
"name": "[variables('storageAccountDeploymentName')]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-10-01",
"condition": "[parameters('deployStorage')]",
"properties": {
"expressionEvaluationOptions": {
"scope": "outer"
},
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[parameters('storageAccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"tags": "[parameters('tags')]",
"location": "[parameters('location')]",
// snip
}
]
}
}
}
]
}
I think you're running into this: https://bmoore-msft.blog/2020/07/26/resource-not-found-dependson-is-not-working/
TLDR;
listKeys is called in a separate job in the deployment engine and gets called early if that resource (i.e. your storage) is not in the same deployment.
you don't need to nest your storageAccount deployment, doesn't hurt but doesn't help. If your storageAccount deployment is unconditional, then it will help because ARM will wait before running listKeys.
if you need the condition, the only fix is to nest the function app deployment and set a dependency on the conditional storageAccount (the dependency need not be conditional ARM will handle that). You need to set the evaluation scope to "inner" on that nested deployment.
That help?
I noticed your dependsOn for Storage looks like this:
"dependsOn": [
"[parameters('appInsightsName')]",
"[parameters('appServicePlanName')]",
"[resourceId('Microsoft.Resources/deployments', variables('storageAccountDeploymentName'))]"
Whereas my despendsOn for Storage Account and Blob Container looks like this... could that be the reason it's not working? I have never used Microsoft.Resources/deployments (FYI my ARM Template deployment is working)
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts/blobServices/containers', variables('StorageAccountName'), 'default', variables('BlobContainerName2'))]",
"[resourceId('Microsoft.Storage/storageAccounts', variables('StorageAccountName'))]"
Alternatively:
You could drop the conditional and deploy the Storage Account each time. If it exists no change, if it doesn't exist then it's created.
The only caveat to this is if you change something on the storage account post-deployment or if the Storage account has a dynamically set name based on Timestamp or GUID etc.
I realize this is a work around, but it should solve your problem barring the caveat above.
I'm trying to set up an Event Grid subscription to my Storage Queue from a custom topic.
This is easy to do when navigating in the portal, but I'm failing to create the appropriate ARM template for this. After having searched and tried out a lot, I've come up with the following piece of template.
{
"name": "MyCustomTopicName/Microsoft.EventGrid/MySubscriptionName",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"location": "[resourceGroup().location]",
"apiVersion": "2019-06-01",
"properties": {
"destination": {
"endpointType": "StorageQueue",
"properties": {
"resourceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('theNameOfMyStorageAccount'))]",
"queueName": "[variables('theNameOfMyQueue')]"
}
},
"filter": {
"advancedFilters": []
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
}
}
This looks rather OK to me but fails because the Event Grid topic isn't in the resource group to which I'm deploying the template.
Deployment failed. Correlation ID: [guid]. {
"error": {
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.EventGrid/topics/MyCustomTopicName' under resource group 'TheResourceGroupTheStorageAccountIsIn' was not found."
}
}
I'm deploying the complete ARM template to TheResourceGroupTheStorageAccountIsIn.
The MyCustomTopicName topic is in a resource group where we place the custom topics, so all services can use it.
I've tried using the full identifier (resource id) of the custom topic, but this isn't valid. Ideas?
PS: I'm using a similar template for creating a subscription to an Azure Function, which does work properly. The main difference over there is the destination block, which makes sense.
If I'm reading this right you just need to use a nested deployment and target the resource group where the topic is in:
{
"apiVersion": "2017-05-10",
"name": "nestedTemplate",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "your_topic_resource_roup",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"name": "MyCustomTopicName/Microsoft.EventGrid/MySubscriptionName",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"location": "[resourceGroup().location]",
"apiVersion": "2019-06-01",
"properties": {
"destination": {
"endpointType": "StorageQueue",
"properties": {
"resourceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('theNameOfMyStorageAccount'))]",
"queueName": "[variables('theNameOfMyQueue')]"
}
},
"filter": {
"advancedFilters": []
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
}
}
]
}
}
},
Big picture: I want to use the ARM template to create multiple topics on a service bus.
Known fact: The app service that deploys the template is in a different resource group than the service bus.
Pain point: I'm using a nested template because I'm trying to create resources (topics) that are external to the targeted resource group. Within this nested template, I'm not sure how to get copy to work correctly.
From this MS doc , I believe my syntax is correct.
This is how my parameters are listed:
"sharedResourcesResourceGroupName": {
"type": "string",
"defaultValue": "sharedResourceGroupName",
"metadata": {
"description": "Resource Group in which platform shared resources live"
}
},
"serviceBusNamespaceName": {
"type": "string",
"defaultValue": "serviceBusName",
"metadata": {
"description": "Name of the Service Bus namespace"
}
},
"topics": {
"type": "array",
"metadata": {
"description": "List of topics"
},
"defaultValue": [
"topic1",
"topic2"
]
}
This is my resource object for creating the topics with the copyIndex() method:
{
"apiVersion": "2018-05-01",
"type": "Microsoft.Resources/deployments",
"name": "[concat(parameters('serviceBusNamespaceName'))]",
"resourceGroup": "[parameters('sharedResourcesResourceGroupName')]",
"properties": {
"mode": "Incremental",
"template":{
"$schema": "2018-05-01",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"type": "Microsoft.ServiceBus/namespaces/topics",
"name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('topics')[copyIndex()])]",
"apiVersion": "2017-04-01",
"location": "[resourceGroup().location]",
"properties": {},
"copy": {
"name": "topics",
"count": "[length(parameters('topics'))]"
},
"dependsOn": [
"[parameters('serviceBusNamespaceName')]"
]
}
]
}
}
}
I am testing the arm template deployment using the Azure Powershell with these commands:
Connect-AzAccount
Set-AZContext -SubscriptionName subscriptionWhereTheAppServiceLives
New-AzResourceGroupDeployment -ResourceGroupName resourceGroupWhereAppServiceLives -TemplateFile <path to template file>\azuredeploy.json -TemplateParameterFile <path to parameters file>\azuredeploy.parameters.json
The error I'm getting from the Azure powershell console is:
The template function 'copyIndex' is not expected at this location. The function can only be used in a resource with copy specified.
If I remove the "copy" object and replace "name" with something like "[concat(parameters('serviceBusNamespaceName'), '/topicName')]", then the template is able to create ONE topic in the right service bus. But I'm looking to create multiple topics.
Any insight would be greatly appreciated!
I think you can do this:
{
"apiVersion": "2018-05-01",
"type": "Microsoft.Resources/deployments",
"name": "[concat(parameters('serviceBusNamespaceName'), copyIndex())]",
"resourceGroup": "[parameters('sharedResourcesResourceGroupName')]",
"copy": {
"name": "topics",
"count": "[length(parameters('topics'))]"
},
"dependsOn": [
"[parameters('serviceBusNamespaceName')]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "2018-05-01",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.ServiceBus/namespaces/topics",
"name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('topics')[copyIndex()])]",
"apiVersion": "2017-04-01",
"location": "[resourceGroup().location]",
"properties": {}
}
]
}
}
}
I'm deploying arm template to create an SSL certificate to existing traffic managers and to bind the certificates to the app services.
Since the app services in one resource group and the traffic manager and certificate in a different resource group - I use nested template.
i got an error with my certificate SSL:
Deployment template validation failed: 'The template reference
'blabla-ssl1' is not valid: could not find template resource or
resource copy with this name
"comments": "Get the Traffic Manager SSL cert that will be binded to the app",
"copy": {
"name": "loop",
"count": "[length(variables('locations'))]"
},
"type": "Microsoft.Web/certificates",
"name": "[concat(variables('tmsslcert')['secretname'], copyIndex())]",
"apiVersion": "2016-03-01",
"location": "[variables('locations')[copyIndex()]]",
"dependsOn": [
"[variables('TMName')]"
],
"properties": {
"keyVaultId": "[variables('tmsslcert')['KeyVaultId']]",
"secretname": "[variables('tmsslcert')['secretname']]"
}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-05-01",
"resourceGroup": "[variables('webappResourceGroup')]",
"name": "[concat('AddTMSSLCert_',variables('locations')[copyIndex()],'_nestedTemplate')]",
"copy": {
"name": "endpointloop",
"count": "[length(variables('locations'))]"
},
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"comments": "app hostname binding of TM CNAME",
"type": "Microsoft.Web/sites/hostNameBindings",
"name": "[concat(variables('webappDNSNamePrefix'), '-', variables('locations')[copyIndex()], '/', variables('tmcname'))]",
"apiVersion": "2016-08-01",
"location": "[variables('locations')[copyIndex()]]",
"scale": null,
"properties": {
"siteName": "variables('webappDNSNamePrefix'), '-', variables('locations')[copyIndex()]",
"sslState": "SniEnabled",
"thumbprint": "[reference(resourceId(variables('webappResourceGroup'),'Microsoft.Web/certificates', concat(variables('tmsslcert')['secretname'], copyIndex())),'2016-03-01').Thumbprint]"
},
"dependsOn": [
"[concat(variables('tmsslcert')['secretname'], copyIndex())]",
//"[concat('Microsoft.Web/certificates/', variables('tmsslcert')['secretname'], copyIndex())]"
]
}
]
}
}
}
its impossible to tell where the error is exactly (given the data you provided), but this means either your references or dependsOn are trying to reach the resource that's either not created or in a different resource group. One thing that looks specifically wrong is this:
"dependsOn": [
"[concat(variables('tmsslcert')['secretname'], copyIndex())]",
//"[concat('Microsoft.Web/certificates/', variables('tmsslcert')['secretname'], copyIndex())]"
]
this would not work, because it will work in the context of the nested deployment, so in a different resource group