roleAssignment & diagnosticLog deployments fail if role/log already exist - azure

I'm using bicep to assign roles to my resources - the first run works perfectly, but any consecutive run fails because the role already exists. The same goes for diagnosticLogs - if they already exist, the pipeline fails.
Is there any way to check if the resource exist and skip the resource-deployment if so? Or at the very least to reduce the severity to a "warning", so the pipeline doesn't fail?
It took me a while to figure out the problem, because the log of Azure Pipelines does not even return an error description, but just fails...
##[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
##[error]Details:
##[error]DeploymentFailed: At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Task failed while creating or updating the template deployment.
Here is the log from the resource-group deployments - diagnosticLogs:
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed.
Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{
"code":"Conflict",
"message":"Data sink '/subscriptions/X-X-X-X-X/resourceGroups/<NAME>/providers/Microsoft.Storage/storageAccounts/<NAME>'
is already used in diagnostic setting '<NAME>' for category 'allLogs'.
Data sinks can't be reused in different settings on the same category for the same resource."
}]}
The error from the roleAssignment:
{"code":"DeploymentFailed",
"message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{
"code":"RoleAssignmentExists",
"message":"The role assignment already exists."
}]}
Here is the bicep code for the deployment:
// roleAssignment
resource role_developer_adls_blob_contributors 'Microsoft.Authorization/roleAssignments#2022-04-01' = {
name: guid(resourceGroup().id, aad_admin_developer_group_object_id)
scope: resourceGroup()
properties:{
description: 'Developer Group - BlobStorageContributor.'
principalId: aad_admin_developer_group_object_id
principalType: 'Group'
roleDefinitionId: resourceId('Microsoft.Authorization/roleDefinitions', storageBlobDataContributorRoleID)
}
}
// diagnosticLogs
resource keyvault_diagnostic_settings 'Microsoft.Insights/diagnosticSettings#2021-05-01-preview' = {
name: '${keyVaultName}-log-adls'
scope: key_vault
properties: {
storageAccountId: adls_storage_base.id
logs: [
{
categoryGroup: 'allLogs'
enabled : true
}
]
}
}

The roleAssignment name needs to be unique for a given principal, role and scope. The seed you have for the guid() function is not unique enough. It should be:
name: guid(resourceGroup().id, aad_admin_developer_group_object_id, storageBlobDataContributorRoleID)
Note that since you already have a roleAssigment for the principal with those perms at that scope, you'll have to remove the "old" roleAssignment before you can use the new naming scheme.

Related

Azure pipeline Release pipeline: ARM Template deployment: Resource Group scope fails with ,,Error code: DeploymentFailed"

The ARM template seems fine, no validation errors occur, and variables seem to be defined, not sure how should I debug this weird error message.
2023-01-25T16:46:49.4710207Z ==============================================================================
2023-01-25T16:46:49.4710434Z Task : ARM template deployment
2023-01-25T16:46:49.4710576Z Description : Deploy an Azure Resource Manager (ARM) template to all the deployment scopes
2023-01-25T16:46:49.4710814Z Version : 3.210.1
2023-01-25T16:46:49.4710932Z Author : Microsoft Corporation
2023-01-25T16:46:49.4711075Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment
2023-01-25T16:46:49.4711307Z ==============================================================================
2023-01-25T16:46:49.6711981Z ARM Service Connection deployment scope - Subscription
2023-01-25T16:46:49.6897349Z Checking if the following resource group exists: APP-RG-DEV-NE.
2023-01-25T16:46:49.9703739Z Resource group exists: true.
2023-01-25T16:46:49.9707493Z Creating deployment parameters.
2023-01-25T16:46:50.7941256Z Starting template validation.
2023-01-25T16:46:50.8120966Z Deployment name is azuredeploy-20230125-164650-9976
2023-01-25T16:46:55.8082119Z Template deployment validation was completed successfully.
2023-01-25T16:46:55.8082485Z Starting Deployment.
2023-01-25T16:46:55.8083063Z Deployment name is azuredeploy-20230125-164650-9976
2023-01-25T16:48:05.5609488Z There were errors in your deployment. Error code: DeploymentFailed.
2023-01-25T16:48:05.5641699Z ##[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
2023-01-25T16:48:05.5651004Z ##[error]Details:
2023-01-25T16:48:05.5651895Z ##[error]BadRequest:
2023-01-25T16:48:05.5652688Z ##[error]BadRequest:
2023-01-25T16:48:05.5653453Z ##[error]BadRequest:
2023-01-25T16:48:05.5654221Z ##[error]BadRequest:
2023-01-25T16:48:05.5654988Z ##[error]BadRequest:
2023-01-25T16:48:05.5656695Z ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
2023-01-25T16:48:05.5658246Z ##[error]Task failed while creating or updating the template deployment.
2023-01-25T16:48:05.5682468Z ##[section]Finishing: ARM Template deployment: Resource Group scope
```
Ok found the issue, which was pretty stupid not sure why the error message was not clearer about name violations.
The secret name for the key vault contained a dot (.) in it and was failing because of it.

Why does my BICEP template, fail to create authorization rules consistently?

I've created a bicep template for deploying the Azure Service Bus which includes creation of multiple topics, subscriptions, filters, and authorisation rules.
I'm attempting to deploy 24 authorisation rules in a serial for loop after the rest of the servicebus has been created. The first time deployment will always fail with one or two authorisation rules returning with the error MessagingGatewayTooManyRequests or AuthorizationRuleNotFound. A subsequent deployment will always work as expected.
I have tried only using a template that only deploys authorisation rules, and have run into the same problem. The first 18 rules were created almost instantly, then after that they start to show as duplicated in the azure portal and fail.
I have found that I can get closer to my goal by splitting up the policies into multiple dependent deployments, which does slow down the request speed due to the physical overhead from creating a new deployment. I'd rather create a pure solution that is low effort, easy to maintain, and doesn't abuse the limitations of ARM deployments in order to succeed.
Please see the cut down version of my module below;
#description('The namespace of the servicebus resource')
param namespace string = 'myservicebus'
#description('An array of shared access policy configurations for service bus topics')
param sharedAccessPolicies array = [
{
topicName: 'mytopic'
policyName: 'listen-policy'
policyRights: ['Listen']
secretName: 'sb-mytopic-listen'
}
{
topicName: 'mytopic'
policyName: 'send-policy'
policyRights: ['Send']
secretName: 'sb-mytopic-send'
}
]
#batchSize(1)
resource topic_auth_rule 'Microsoft.ServiceBus/namespaces/topics/authorizationRules#2021-11-01' = [for policy in sharedAccessPolicies: {
name: '${namespace}/${policy.topicName}/${policy.policyName}'
properties: {
rights: policy.policyRights
}
}]
I've found a similar post around this issue which is what lead to my current solution. Although I don't understand why this single API endpoint is so aggressively rate limited.
Any advice on this would be much appreciated.
The above code in my question now works as expected. I spent the past month talking to multiple levels of Microsoft support, I managed to get in touch with the ARM team who looked into and resolved the problem.
The alternative solution which I was suggested was to individually register each resource and create a huge dependency chain, see example below.
resource topic_auth_rule_listen 'Microsoft.ServiceBus/namespaces/topics/authorizationRules#2021-11-01' = {
name: '${namespace}/mytopic/listen-policy'
properties: {
rights: [ 'Listen' ]
}
}
resource topic_auth_rule_send 'Microsoft.ServiceBus/namespaces/topics/authorizationRules#2021-11-01' = {
name: '${namespace}/mytopic/send-policy'
properties: {
rights: [ 'Send' ]
}
dependsOn: [ topic_auth_rule_listen ]
}
...

How to create Microsoft.DBforPostgreSQL with Bicep?

I would like to create PostgreSQL for location based service needs.
I would install GIS extensions next.
I have created manually Azure Database for PostgreSQL flexible server to decide correct config.
"sku": {"name": "Standard_B1ms","tier": "Burstable"}
I wanted to create Single server, but it was not available in Europe for some reason. I thought that missing Burstable is good for initial POC, but general purpose is good as well.
Now trying to create PostgreSQL with Bicep.
However I have difficulty to set valid server. Firstly Burstable was not available. Next I cannot set valid sku name.
az deployment group create:
{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource
deployment operation failed. Please list deployment operations for details. Please see
https://aka.ms/DeployOperations for usage details.","details":
[{"code":"BadRequest","message":"{\r\n \"error\": {\r\n \"code\":
\"InvalidEditionSloCombination\",\r\n \"message\": \"The edition
GeneralPurpose does not support the service objective Standard_D2s_v3\"\r\n }\r\n}"}]}}
main.bicep:
resource symbolicname 'Microsoft.DBforPostgreSQL/servers#2017-12-01' = {
name: 'my-postgresql-dev'
location: 'West Europe'
tags: {
tagName1: 'tagValue1'
tagName2: 'tagValue2'
}
sku: {
name: 'Standard_D2s_v3'
tier: 'GeneralPurpose'
}
identity: {
type: 'SystemAssigned'
}
properties: {
administratorLogin: 'sqladmin'
administratorLoginPassword: 'asfar43efw!sdf'
storageProfile: {
backupRetentionDays: 7
geoRedundantBackup: 'Disabled'
storageMB: 32000
}
version: '11'
createMode: 'Default'
// For remaining properties, see ServerPropertiesForCreate objects
}
}
The error you received is related to the sku name:
The edition GeneralPurpose does not support the service objective Standard_D2s_v3
Looking at the documentation, the sku name has a specific naming convention:
The name of the sku, typically, tier + family + cores, e.g. B_Gen4_1, GP_Gen5_8.
In you case, for general purpose it will be GP_GEN5_{number of cores}.

ARM template deployment - The request failed due to conflict with a concurrent request error

When re-deploying ARM template with extensions getting error as
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "Conflict",
"message": "The request failed due to conflict with a concurrent request. To resolve it, please refer to https://aka.ms/activitylog to get more details on the conflicting requests."
}
]
}
Using Get-AzureRMLog -CorrelationId "xxxx-xxx-xxx-xxxx-Xxxxxx" -DetailedOutput
{"error":{"code":"Conflict","message":"The request failed
due to conflict with a concurrent request. To resolve it, please refer to
https://aka.ms/activitylog to get more details on the conflicting
requests."}
The deployment name remains same in both iterations of deployment. Have tried changing the name but received the same error.
When I remove Geneva extensions and Anti-Malware extension this error no longer occurs.
You need to fill in the dependsOn [] section for those resources in the resources section of your ARM template. In the Deployments section of your resource group, you should be able to find out which other resource they are conflicting with and use that information to set the deployment order.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/define-resource-dependency

How to update queue/topic of Azure Service Bus via ARM?

I have a ARM (Azure Resource Manager) script that creates Service bus with topic and subscriber inside. It worked perfectly for some time, but I decided to enable session on topic and disable partitioning. Script was changed and during deployment it gives me:
Template deployment returned the following errors:
07:56:00 - Resource Microsoft.ServiceBus/namespaces/topics 'ops-ServiceBus/default-topic' failed with message '{
"error": {
"message": "SubCode=40000. Partitioning cannot be changed for Topic. . TrackingId:<some_guid>_M11CH3_M11CH3_G1, SystemTracker:ops-servicebus.servicebus.windows.net:default-topic, Timestamp:2019-03-28T04:55:56 CorrelationId: <some_guid>",
"code": "BadRequest"
}
}'
07:56:21 - Template output evaluation skipped: at least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.
Is it possible to perform update operation on Queue/Topic using ARM?
We did configure queues\topics with arm templates, but according to the error - some parameters are immutable, so you'd have to recreate in this case.

Resources