Azure policy deployment using visual studio - azure

I am trying to use in built allowed locations Azure policy.
Below my ARM template definition
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"listOfAllowedLocations": {
"type": "Array"
}
},
"variables": {},
"resources": [{
"type": "Microsoft.Authorization/policyDefinitions",
"name": "Test",
"apiVersion": "2018-03-01",
"properties": {
"displayName": "Test allowed locations",
"policyType": "BuiltIn",
"description": "This policy enables you to restrict the locations your organization can specify when deploying resources. Use to enforce your geo-compliance requirements.",
"parameters": {
"listOfAllowedLocations": {
"type": "Array",
"metadata": {
"description": "The list of locations that can be specified when deploying resources.",
"strongType": "location",
"displayName": "Allowed locations"
}
}
},
"policyRule": {
"if": {
"not": {
"field": "location",
"in": "[parameters('listOfAllowedLocations')]"
}
},
"then": {
"effect": "Deny"
}
}
}
}],
"outputs": {}
}
I am getting below error when I try to deploy this using Visual Studio deploy option
{
"error": {
"code": "InvalidPolicyUri",
"message": "The policy request scope '/subscriptions/xxx/resourcegroups/Test' should be '/', '/subscriptions/id' or '/providers/Microsoft.Management/managementGroups/id'."
}
}
I really appreciate if someone can guide me the right way for deploying policies using Visual Studio. This template will go into DevOps release pipeline later once it is successful in VS deploy testing.

I figured it out. By default visual studio uses resource group deployment, that is the reason this is not working. We need to use New-AzureRmDeployment instead of New-AzureRmResourceGroupDeployment.

Related

Updating key vault secret via Arm template release from devops CI/CD fails

I have managed to release secrets to my Azure key vault via CI/CD from DevOps using my arm templates. The initial release went fine and added my new non existing secrets to my key vault resource. Though men trying to update the value of the secret in my ARM template and then pushing it to my GIT-repo to in turn release it as to update my secret in azure it fails giving me:
At least one resource deployment operation failed. Please list deployment operations for
details. Please see https://aka.ms/DeployOperations for usage details.
Details:
BadRequest:
Check out the troubleshooting guide to see if your issue is addressed:
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
Task failed while creating or updating the template deployment.
My template looks like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyVault": {
"value": "test-kv-devopstest01-d"
},
"TestCedential_1": {
"value": "TestCedentialSecretValue1"
},
"TestCedentialName_1": {
"value": "TestCedentialSecretName1_SecondVersion"
}
}
}
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyVault": {
"type": "string"
},
"TestCedential_1": {
"type": "secureString"
},
"TestCedentialName_1": {
"type": "string"
}
},
"variables": {
},
"resources": [
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVault'), '/', parameters('TestCedentialName_1'))]",
"apiVersion": "2015-06-01",
"properties": {
"contentType": "text/plain",
"value": "[parameters('TestCedential_1')]"
}
}
],
"outputs": {}
}
I've also tried granting permissions for the pipelines in access control in the key vault resource in azure.
Am i missing something maybe?
I tested the same code in my environment and it resulted in same error :
The issue is with the below :
"TestCedentialName_1": {
"value": "TestCedentialSecretName1_SecondVersion"
}
In Key vault secret '_' (underscore) is not allowed in name. The allowed values are alphanumeric characters and dashes.
Changing underscore to dash fixes the issue :
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyVault": {
"type": "string",
"defaultValue" :"test-kv-ansuman-d"
},
"TestCedential_1": {
"type": "secureString",
"defaultValue":"TestCedentialSecretValue1"
},
"TestCedentialName_1": {
"type": "string",
"defaultValue": "TestCedentialSecretName1-SecondVersion"
}
},
"variables": {
},
"resources": [
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVault'), '/', parameters('TestCedentialName_1'))]",
"apiVersion": "2015-06-01",
"properties": {
"contentType": "text/plain",
"value": "[parameters('TestCedential_1')]"
}
}
],
"outputs": {}
}
Output:

Azure ARM role assignment for System Assigned Managed Identity fails the first run

My goal is to deploy a logic app with a system managed identity and a role assignment for that identity. Preferably, this is done in one ARM template.
I have a setup that fails the first run, but succeeds successive runs. Correct me if I'm wrong, but I think that the reason for this is that the deployment of the role assignment happens before the managed identity of the logic app is "ready", hence the following error I get the first time that I deploy the template. I don't get this error the second time I deploy the template, probably because the Identity already exists at that time.
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "PrincipalNotFound",
"message": "Principal *** does not exist in the directory ***."
}
]
}
My template (removed logic app definition for brevity). In this case the identity of the logic app requires access to a storage account which is located in another resource group.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logicAppName": {
"type": "string"
},
"storageAccountResourceGroup": {
"type": "String"
},
"storageAccountName": {
"type": "String"
}
},
"variables": {
"logicAppResourceId": "[resourceId('Microsoft.Logic/workflows', parameters('logicAppName'))]",
"Storage Blob Data Contributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]"
},
"resources": [
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[parameters('logicAppName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"state": "Enabled",
"definition": {
...
}
}
},
{
"type": "Microsoft.Resources/deployments",
"name": "[concat('RoleAssignment-', parameters('logicAppName'))]",
"apiVersion": "2020-06-01",
"resourceGroup": "[parameters('storageAccountResourceGroup')]",
"subscriptionId": "[subscription().subscriptionId]",
"dependsOn": [
"[parameters('logicAppName')]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2018-09-01-preview",
"type": "Microsoft.Storage/storageAccounts/providers/roleAssignments",
"name": "[concat(parameters('storageAccountName'), '/Microsoft.Authorization/', guid(subscription().subscriptionId, parameters('logicAppName')))]",
"properties": {
"roleDefinitionId": "[variables('Storage Blob Data Contributor')]",
"principalId": "[reference(variables('logicAppResourceId'), '2019-05-01', 'Full').identity.principalId]"
}
}
]
}
}
}
]
}
As you can see in the template, I added a dependsOn on the logic app itself. However that doesn't seem to be sufficient.
Does anyone have a solution for this?
Thank you!
I found the answer here: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-template#new-service-principal
Deployment works consistently after adding "principalType": "ServicePrincipal"

Create API Connection for Azure Data Factory with service principal authentication using ARM Template

I've created a logic app that uses azure data factory connectors.
I can create the API Connection with service principal authentication from the portal:
But I can't find any documentation on how to create an API connection using ARM template.
But I need to create using ARM template with the same service principal authentication.
You can create an API connection for Azure Data factory using ARM template like that:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"connectionAPIName": {
"type": "string"
},
"clientId": {
"type": "string"
},
"clientSecret": {
"type": "securestring"
}
},
"resources": [
{
"type": "Microsoft.Web/connections",
"apiVersion": "2018-07-01-preview",
"name": "[parameters('connectionAPIName')]",
"location": "[resourceGroup().location]",
"properties": {
"displayName": "[parameters('connectionAPIName')]",
"parameterValues": {
"token:clientId": "[parameters('clientId')]",
"token:clientSecret": "[parameters('clientSecret')]",
"token:TenantId": "[subscription().tenantId]",
"token:grantType": "client_credentials"
},
"api": {
"id": "[concat('subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/azuredatafactory')]"
}
}
}
],
"outputs": {}
}
I did some test in visual studio 2019 because VS will show as much content of ARM template as possible(sometimes show the content more than in Azure portal). I installed "Azure Logic Apps Tools for Visual Studio 2019" and then create my logic app in VS2019. After adding an action "Create a pipeline run", click "code view" in VS2019. The template shows as below:
{
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Create_a_pipeline_run": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azuredatafactory_2']['connectionId']"
}
},
"method": "post",
"path": "/subscriptions/#{encodeURIComponent('**********')}/resourcegroups/#{encodeURIComponent('andywebbot')}/providers/Microsoft.DataFactory/factories/#{encodeURIComponent('andydatafactory2')}/pipelines/#{encodeURIComponent('pipeline1')}/CreateRun",
"queries": {
"x-ms-api-version": "2017-09-01-preview"
}
},
"runAfter": {}
}
},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"type": "Recurrence",
"recurrence": {
"frequency": "Month",
"interval": 3
}
}
},
"contentVersion": "1.0.0.0",
"outputs": {}
}
We can see the template doesn't show us the details of the connection(such as "tenantId", "Client ID" and "Client Secret"). So I'm afraid we can not use ARM template to create the service principal.

Azure ARM condition fails during validation

In my azure template I have a condition where I chose if I want my webapps deployed on a dedicated App Service Plan or if I want to use a shared App Service plan.
If I chose to not use a dedicated plan I want to ignore:
- the first section where I deploy the dedicated App Service Plan
- the second section where I deploy the Web Apps and use the dedicated Service Plan.
The third section is then used and deploy the web apps with a shared app plan.
Below is an extract of my ARM template.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"_artifactsLocation": {
"type": "string"
},
"_artifactsLocationSasToken": {
"type": "string"
},
"environmentConfiguration": {
"type": "object"
}
},
"variables": {},
"resources": [
{
"comments": "App Service Plan hosting all websites",
"apiVersion": "2017-05-10",
"name": "AppServicePlan",
"type": "Microsoft.Resources/deployments",
"condition": "[equals(parameters('environmentConfiguration').serverFarm.useDedicatedPlan, 'true')]",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[concat(parameters('_artifactsLocation'),'/Microsoft.Web/Asp.json',parameters('_artifactsLocationSasToken'))]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"environmentConfiguration": {
"value": "[parameters('environmentConfiguration')]"
}
}
}
},
{
"comments": "Web apps on dedicated plan",
"apiVersion": "2017-05-10",
"name": "[concat('WebAppsDedicatedPlan-',parameters('environmentConfiguration').webApp.webApps[copyIndex()].name)]",
"type": "Microsoft.Resources/deployments",
"condition": "[equals(parameters('environmentConfiguration').serverFarm.useDedicatedPlan, 'true')]",
"copy": {
"name": "webAppCopy",
"count": "[length(parameters('environmentConfiguration').webApp.webApps)]"
},
"dependsOn": [
"[resourceId('Microsoft.Resources/deployments', 'AppServicePlan')]"
],
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[concat(parameters('_artifactsLocation'),'/Microsoft.Web/WebApp.json',parameters('_artifactsLocationSasToken'))]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"environmentConfiguration": {
"value": "[parameters('environmentConfiguration')]"
},
"dependencies": {
"value": {
"webAppInfo": "[parameters('environmentConfiguration').webApp.webApps[copyIndex()]]",
"serverFarmId": "[reference('AppServicePlan').outputs.serverFarmId.value]"
}
}
}
}
},
{
"comments": "Web apps on shared plan",
"apiVersion": "2017-05-10",
"name": "[concat('WebAppsOnSharedPlan-',parameters('environmentConfiguration').webApp.webApps[copyIndex()].name)]",
"type": "Microsoft.Resources/deployments",
"condition": "[equals(parameters('environmentConfiguration').serverFarm.useDedicatedPlan, 'false')]",
"copy": {
"name": "webAppCopy",
"count": "[length(parameters('environmentConfiguration').webApp.webApps)]"
},
"dependsOn": [],
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[concat(parameters('_artifactsLocation'),'/Microsoft.Web/WebApp.json',parameters('_artifactsLocationSasToken'))]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"environmentConfiguration": {
"value": "[parameters('environmentConfiguration')]"
},
"dependencies": {
"value": {
"webAppInfo": "[parameters('environmentConfiguration').webApp.webApps[copyIndex()]]",
"serverFarmId": "[resourceId('sharedResources','Microsoft.Web/serverfarms','sharedasp')]"
}
}
}
}
}
],
"outputs": {}
}
What is working: If I remove the condition in the App Service Plan section and I ask to not use the dedicated plan, my web apps are deployed using the shared plan. (The app service plan is also deployed).
What is not working: If I let the condition in the App Service Plan section to not deploy it when I ask for to not use the dedicated plan the validation fails with the following message:
2017-09-25T11:55:49.7343682Z Creating deployment parameters.
2017-09-25T11:55:49.7373683Z The detected encoding for file
'd:\a\r1\a\output\iac\myapp.json' is 'utf-8'
2017-09-25T11:55:49.7373683Z The detected encoding for file
'd:\a\r1\a\output\iac\myapp.parameters.qa.json' is 'utf-8'
2017-09-25T11:55:49.7373683Z Starting Deployment.
2017-09-25T11:55:51.3725072Z There were errors in your deployment.
Error code: InvalidTemplate. 2017-09-25T11:55:51.3735078Z
##[error]Deployment template validation failed: 'The template resource 'Microsoft.Resources/deployments/WebAppsDedicatedPlan-appadmin'
reference to 'Microsoft.Resources/deployments/AppServicePlan' requires
an API version. Please see https://aka.ms/arm-template for usage
details.'. 2017-09-25T11:55:51.3735078Z ##[error]Task failed while
creating or updating the template deployment.
2017-09-25T11:55:51.4295112Z ##[section]Finishing: Azure Deployment:
Update resource group
How can I solve this issue?
'The template resource
'Microsoft.Resources/deployments/WebAppsDedicatedPlan-appadmin'
reference to 'Microsoft.Resources/deployments/AppServicePlan' requires
an API version.
The error gives that away. Check docs for better understanding. This is why it errors out:
API version of the specified resource. Include this parameter when the resource is not provisioned within same template. Typically, in the format, yyyy-mm-dd.
So you need to add api version to the reference function for resources created outside of the template

Retrieve Service Bus event hub connection string

I have an existing Service Bus with one queue and event hub deployed using Azure Resource Manager.
Now I am interested to retrieve the primary key and connection string using Azure PowerShell wiithout using the ServiceBus.dll. Is it possible??
As a workaround I have created an ARM template which does not deploy anything but just query the existing resource and retrieve the information I need. The below template retrieves the connection string and primary key of an event hub/queue for a specific service bus namespace
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"serviceBusNamespace": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the service bus namespace to create."
}
},
"resourceName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the resource to be retreived."
}
},
"resourceType": {
"type": "string",
"minLength": 1,
"allowedValues": [
"queues",
"eventhubs"
],
"metadata": {
"description": "The type of the resource"
}
},
"policy": {
"type": "string",
"minLength": 1,
"defaultValue": "ManagePolicy",
"allowedValues": [
"ManagePolicy",
"SendPolicy",
"ListenPolicy"
],
"metadata": {
"description": "The type of the resource"
}
}
},
"variables": {
},
"resources": [ ],
"outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryConnectionString]"
},
"primaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryKey]"
}
}
}
Is it abusing to use ARM template to query for a resource and not actually deploy anything?
EDIT
To capture the output of the ARM template within PowerShell use the below code
$ep = New-AzureRmResourceGroupDeployment -Name "getEventHub" -ResourceGroupName myResourceGroup -Mode Incremental -TemplateFile getEventHub.json -TemplateParameterFile getEventHub.param.json
$RuleConnString = $ep.Outputs.connectionString.value
$RulePrimaryKey = $ep.Outputs.primaryKey.value
Note that the property names connectionString and primaryKey are same as defined in my template file
EDIT 2
If I re-run the ARM template to deploy the event hub second time I get the below error.
I din't find any option other than to use the ARM template to query the details.
I don’t see what’s wrong with what you’re doing. In my view Resource Manager templates in their nature are incremental. So you could author a template to create your existing service bus with the same resources. If the properties are the same then it will leave the existing resources intact and return you the connection string and primary key of the relevant resource.
I have a need to automate the creation of a service bus and queue and separate send/listen shared access policies. You can retrieve the connection string on the service bus itself using PowerShell natively without using the .Net ServiceBus.dll assembly by using Get-AzureSBAuthorizationRule but due to a still current bug this doesn’t work at the queue level.
I tried using the ServiceBus.dll to create the shared access policies but sometimes it would randomly fail but subsequently work if you ran it a second time immediately afterwards. I also tried Resource Manager templates but previously you had to pass in the keys you’d generated yourself. Now I see Microsoft generate those for you but you’re still left trying to get the key in an automated fashion so I like your solution.
One question though, can you capture the Resource Manager template outputs and pass them back to a PowerShell script, do you know?
Cheers
Rob
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {
"servicebusNamespace": {
"type": "string",
"metadata": {
"description": "The service bus namespace"
}
},
"notificationssmsqueue": {
"type": "string",
"metadata": {
"description": "Notifications SMS queue"
}
} }, "variables": {
"location": "[resourceGroup().location]", }, "resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('servicebusNamespace')]",
"type": "Microsoft.ServiceBus/namespaces",
"location": "[variables('location')]",
"properties": {
"messagingSku": 2
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('notificationssmsqueue')]",
"type": "Queues",
"dependsOn": [
"[concat('Microsoft.ServiceBus/namespaces/', parameters('servicebusNamespace'))]"
],
"properties": {
"path": "[parameters('notificationssmsqueue')]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Listen" ],
"revision": -1
}
},
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.send')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.send')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Send" ],
"revision": -1
}
}
]
}
]
} ], "outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/AuthorizationRules'),parameters('serviceBusNamespace'),'RootManageSharedAccessKey'),'2015-08-01').primaryConnectionString]"
},
"smsSendPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.send')),'2015-08-01').PrimaryKey]"
},
"smsListenPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.listen')),'2015-08-01').PrimaryKey]"
} } }
But I call my templates like this:
New-AzureRMResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile "$scripts_folder$SB_create_script" -TemplateParameterObject `
#{ servicebusNamespace = $servicebusNamespace;
notificationssmsqueue = $NotificationSMSqueue }
This is the correct way to get the information you are seeking. The Resource Manager provides a common interface to interact with all the services. It is how the Portal access the services, and each of the language SDKs are just wrappers for similar requests to the one you have created.
I usually use the Python or java SDKs, but I have been told that NodeJS is a very easy way to wrap the Web APIs that ARM calls to construct similar calls like the one you made, if you are looking for a none ARM way to do this.

Resources