There is no support from Azure for Azure Key vault backup(existing options are error prone when it comes to automation). There is soft delete and I can reset passwords and put it back in keyvault, in case something goes wrong. So it seems okay still as an alternative to backup I would like to take a screenshot of the Secret Names (not values) and put that image in storage account. Is this safe? The reason why I do this is because it will be easy to recreate the secrets in case key-vault goes down(.5 % chance).
You can definitely use Soft Delete feature as an alternative. Apart from that, in case KeyVault goes down and you want to recreate the secrets, it's easier and safer to setup ARM template and ADO pipeline to achieve your goal with restricted access to the ADO (only people who are part of your organization in ADO can see the pipeline).
The ARM template for Key Vault looks like this:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyVaultName": {
"type": "string",
"metadata": {
"description": "Specifies the name of the key vault."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Specifies the Azure location where the key vault should be created."
}
},
"enabledForDeployment": {
"type": "bool",
"defaultValue": false,
"allowedValues": [
true,
false
],
"metadata": {
"description": "Specifies whether Azure Virtual Machines are permitted to retrieve certificates stored as secrets from the key vault."
}
},
"enabledForDiskEncryption": {
"type": "bool",
"defaultValue": false,
"allowedValues": [
true,
false
],
"metadata": {
"description": "Specifies whether Azure Disk Encryption is permitted to retrieve secrets from the vault and unwrap keys."
}
},
"enabledForTemplateDeployment": {
"type": "bool",
"defaultValue": false,
"allowedValues": [
true,
false
],
"metadata": {
"description": "Specifies whether Azure Resource Manager is permitted to retrieve secrets from the key vault."
}
},
"tenantId": {
"type": "string",
"defaultValue": "[subscription().tenantId]",
"metadata": {
"description": "Specifies the Azure Active Directory tenant ID that should be used for authenticating requests to the key vault. Get it by using Get-AzSubscription cmdlet."
}
},
"objectId": {
"type": "string",
"metadata": {
"description": "Specifies the object ID of a user, service principal or security group in the Azure Active Directory tenant for the vault. The object ID must be unique for the list of access policies. Get it by using Get-AzADUser or Get-AzADServicePrincipal cmdlets."
}
},
"keysPermissions": {
"type": "array",
"defaultValue": [
"list"
],
"metadata": {
"description": "Specifies the permissions to keys in the vault. Valid values are: all, encrypt, decrypt, wrapKey, unwrapKey, sign, verify, get, list, create, update, import, delete, backup, restore, recover, and purge."
}
},
"secretsPermissions": {
"type": "array",
"defaultValue": [
"list"
],
"metadata": {
"description": "Specifies the permissions to secrets in the vault. Valid values are: all, get, list, set, delete, backup, restore, recover, and purge."
}
},
"skuName": {
"type": "string",
"defaultValue": "Standard",
"allowedValues": [
"Standard",
"Premium"
],
"metadata": {
"description": "Specifies whether the key vault is a standard vault or a premium vault."
}
},
"secretName": {
"type": "string",
"metadata": {
"description": "Specifies the name of the secret that you want to create."
}
},
"secretValue": {
"type": "securestring",
"metadata": {
"description": "Specifies the value of the secret that you want to create."
}
}
},
"resources": [
{
"type": "Microsoft.KeyVault/vaults",
"name": "[parameters('keyVaultName')]",
"apiVersion": "2019-09-01",
"location": "[parameters('location')]",
"properties": {
"enabledForDeployment": "[parameters('enabledForDeployment')]",
"enabledForDiskEncryption": "[parameters('enabledForDiskEncryption')]",
"enabledForTemplateDeployment": "[parameters('enabledForTemplateDeployment')]",
"tenantId": "[parameters('tenantId')]",
"accessPolicies": [
{
"objectId": "[parameters('objectId')]",
"tenantId": "[parameters('tenantId')]",
"permissions": {
"keys": "[parameters('keysPermissions')]",
"secrets": "[parameters('secretsPermissions')]"
}
}
],
"sku": {
"name": "[parameters('skuName')]",
"family": "A"
},
"networkAcls": {
"defaultAction": "Allow",
"bypass": "AzureServices"
}
}
},
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVaultName'), '/', parameters('secretName'))]",
"apiVersion": "2019-09-01",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName'))]"
],
"properties": {
"value": "[parameters('secretValue')]"
}
}
]
}
The ADO Release pipeline will look like this:
Example build pipeline for .NET core solution repo:
Then, you can use Azure Resource Group Deployment task in your ADO release pipeline.
Azure Key Vault Backup to another Key Vault using LogicApp
Pre-Requisites
Enable LogicApp system assigned managed identity.
Two key vault in same subscription and geography.
Add LogicApp objectID in key vault access policy with following key and secret permissions,
backup,list(source key vault)
restore(destination key vault)
Limitation
Source and Destination key vault should be in same subscription and geography.
Steps
Design
Step1
Paste output generated from Step1 in Parse JSON to generate schema
Step2
Step3
Step4
above steps are same for secrets backup
References
Key Vault REST
LogicApp Backup/Restore using Storage Account
azureazure-keyvaultazure-logic-apps
Related
I have a template that creates a storage account and a fileshare. Like this:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"metadata": {
"_generator": {
"name": "bicep",
"version": "0.5.6.12127",
"templateHash": "3186185032530874094"
}
},
"parameters": {
"storageAccountName": {
"type": "string",
"defaultValue": "[format('storage{0}', uniqueString(resourceGroup().id))]",
"metadata": {
"description": "Specifies the name of the Azure Storage account."
}
},
"fileShareName": {
"type": "string",
"maxLength": 63,
"minLength": 3,
"metadata": {
"description": "Specifies the name of the File Share. File share names must be between 3 and 63 characters in length and use numbers, lower-case letters and dash (-) only."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Specifies the location in which the Azure Storage resources should be deployed."
}
}
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[parameters('storageAccountName')]",
"location": "[parameters('location')]",
"kind": "StorageV2",
"sku": {
"name": "Standard_LRS"
},
"properties": {
"accessTier": "Hot",
"minimumTlsVersion" : "TLS1_2",
"supportsHttpsTrafficOnly": "true"
}
},
{
"type": "Microsoft.Storage/storageAccounts/fileServices/shares",
"apiVersion": "2021-04-01",
"name": "[format('{0}/default/{1}', parameters('storageAccountName'), parameters('fileShareName'))]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
]
}
]
}
From the YAML pipeline, you are using AzureFileCopy#4 task.
#[error]Upload to container: 'fsndevinternetsuite' in storage account: 'strdevinternetsuite' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.'
From the error message, the cause of this issue could be that the Service Principal created by the Service connection does not have sufficient permissions to copy files to the Azure Storage account.
Refer to the following steps to find the service principal and add the Storage Blob Data Owner & Storage Blob Data Contributor to the service principal.
Step1: Navigate to Azure Portal -> AAD -> App registrations to find the related Service Principal.
Step2: Navigate to Storage account and grant the Storage Blob Data Owner & Storage Blob Data Contributor to the service principal.
For more detailed steps, you can refer to this doc: Assign an Azure role for access to blob data
Or you can change to use the AzureFileCopy#3 to replace the AzureFileCopy#4. You can check if it can work.
I need to get the latest version of the Key in the output section of the arm template(generated inside Azure's Key Vault) which is generated from an ARM template . How can I get that ? I need to use the output as input for my next Job in pipeline.
Newer versions of the Key Vault provider for ARM deployments support creating keys, which you can reference as shown in the example ARM template below.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vaultName": {
"type": "string"
},
"objectId": {
"type": "string",
"metadata": {
"description": "The unique principal ID within the tenant to which key wrap and unwrap permissions are given."
}
},
"keyName": {
"type": "string",
"defaultValue": "test-key"
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]"
},
"tenantId": {
"type": "string",
"defaultValue": "[subscription().tenantId]",
"metadata": {
"description": "Tenant ID of the ACtive Directory to authenticate access. Default is the current subscription's tenant ID."
}
}
},
"variables": {
"apiVersion": "2019-09-01"
},
"resources": [
{
"type": "Microsoft.KeyVault/vaults",
"apiVersion": "[variables('apiVersion')]",
"name": "[parameters('vaultName')]",
"location": "[parameters('location')]",
"properties": {
"sku": {
"family": "A",
"name": "standard"
},
"tenantId": "[parameters('tenantId')]",
"accessPolicies": [
{
"tenantId": "[parameters('tenantId')]",
"objectId": "[parameters('objectId')]",
"permissions": {
"keys": [
"wrapKey",
"unwrapKey"
]
}
}
]
}
},
{
"type": "Microsoft.KeyVault/vaults/keys",
"apiVersion": "[variables('apiVersion')]",
// The name must include the vault name and key name separated by a slash.
"name": "[concat(parameters('vaultName'), '/', parameters('keyName'))]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', parameters('vaultName'))]"
],
"properties": {
"kty": "RSA",
"keySize": 4096,
"keyOps": [
"wrapKey",
"unwrapKey"
]
}
}
],
"outputs": {
"keyName": {
"type": "string",
"value": "[parameters('keyName')]"
},
// Despite the delimited resource name above, we need to construct a 2-parameter resource ID to reference the created key.
"keyUri": {
"type": "string",
"value": "[reference(resourceId('Microsoft.KeyVault/vaults/keys', parameters('vaultName'), parameters('keyName'))).keyUri]"
},
"keyVersionUri": {
"type": "string",
"value": "[reference(resourceId('Microsoft.KeyVault/vaults/keys', parameters('vaultName'), parameters('keyName'))).keyUriWithVersion]"
}
}
}
Note the comments about how you create and reference the keys differently. Simply using the reference() template function against the delimited name results in an invalid template, so you must construct a resourceId() despite being in the sample template.
Adjust access policies as needed. This example gives the principal you pass key wrap and unwrap capabilities, which you can use for block encryption ciphers.
To use this template (e.g. saved as keyvault-template.json),
az group create -n rg-mytestkv -l westus2
az deployment group create -g rg-mytestkv --template-file keyvault-template.json --parameters vaultName=mytestkv objectId=$(az account show --query id)
I am trying to deploy a key vault with secretName and secretValue and I have created a variable group in azure devops with all the secrets and I am using the below parameters in parameter file, but when this gets deployed the secret value gets stored as $(secret) and not the password actually stored in the task group in Azure DevOps.
"secretsObject": {
"value": {
"secrets": [
{
"secretName": "App012",
"secretValue": "$(mysecret)"
},
and this is what I got in the key vault template:
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVaultName'), '/', parameters('secretsObject').secrets[copyIndex()].secretName)]",
"apiVersion": "2018-02-14",
"dependsOn": [
"[concat('Microsoft.KeyVault/vaults/', parameters('keyVaultName'))]"
],
"copy": {
"name": "secretsCopy",
"count": "[length(parameters('secretsObject').secrets)]"
},
"properties": {
"value": "[parameters('secretsObject').secrets[copyIndex()].secretValue]"
}
}
]
}
Any idea how to pass the "secretvalue" as a variable?
I believe your asking how to leverage your secrets that are stored as a variable group to be deployed securely with your ARM template via Azure DevOps. If that is the case look at using Override Template Parameters in your release task.
This would be in the format of -NameOfARMParameter $(NameofDevOpsVariable)
In your case it would be -mysecret $(NameOfDevOpsVariable)
The deployment .json should look like this for parameter declaration:
"secretValue": {
"type": "string",
"metadata": {
"description": "This is for receiving a value from DevOps releases of the secret to be stored in the key vault"
}
},
"secretName": {
"type": "string",
"metadata": {
"description": "Name of the Secret"
}
},
For the actual deployment
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(variables('keyVaultName'),'/',parameters('secretName'))]",
"apiVersion": "2018-02-14",
"properties": {
"contentType": "text/plain",
"value": "[parameters('secretValue')]"
},
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]"
]
},
And the parameters file doesn't need to have anything in it if these values will be fed from Dev Ops
You need to create a parameter file with the secret / link to Key Vault.
Here's a sample of it:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"adminLogin": {
"value": "exampleadmin"
},
"adminPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.KeyVault/vaults/<vault-name>"
},
"secretName": "ExamplePassword"
}
},
"sqlServerName": {
"value": "<your-server-name>"
}
}
}
More info:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/parameter-files
and https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/add-template-to-azure-pipelines
I'm using VSTS to deploy azure resources.
I use task "Azure Resource Group Deployment" to deploy ARM templates.
How can I, for a specific parameter, override the value with a ARM function (concat, listkeys, etc)?
Example: My ARM template has a parameter that is a storage account key and instead of providing the key directly, I want to provide it by passing [listkeys(...)]
You cannot do that, several functions (like listKeys()) are evaluated at runtime only. I don't know what you are trying to achieve, so there are probably ways of doing what you try to achieve.
If you want to hide the keys you can store them in the Key Vault and retrieve at deployment time:
"password": {
"reference": {
"keyVault": {
"id": "[resourceId('kvGroup', 'Microsoft.KeyVault/vaults', 'kvName')]"
},
"secretName": "secret"
}
},
If the storage account isn't created within the same ARM template, I'd use the parameter to supply the name of the storage account and then listkeys() within the ARM template to get at the storage account connection string.
If you're creating the storage account in a previous ARM template deployment in your pipeline you could use output parameters to make the connection string available in the pipeline. Here is an example where xxx represents your company naming prefix:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environment": {
"type": "string",
"defaultValue": "d",
"metadata": {
"description": "The deployment environment, given by develop (d), testing (t), production (p) or quality assurance (q)"
}
}
},
"variables": {
"busUnit": "vendor_name_here",
//storage account names must be lowercase and are limited to 24 alpha numeric characters
"storage_account_name": "[concat('xxx', parameters('environment'), variables('busUnit'), 'stor')]"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS", //this is a hard coded SKU
"tier": "Standard" //general purpose versus blob-only
},
"kind": "Storage",
"name": "[variables('storage_account_name')]",
"apiVersion": "2017-06-01",
"location": "[resourceGroup().location]", //add it to the same region/location as the resource group
"properties": {
"encryption": {
"keySource": "Microsoft.Storage",
"services": {
"blob": {
"enabled": true
}
}
},
"networkAcls": {
"bypass": "AzureServices",
"defaultAction": "Allow",
"ipRules": [],
"virtualNetworkRules": []
}
},
"dependsOn": []
}
],
"outputs": {
"storageAccountKey": {
//"description": "This works if the storage account is in the same resource group. It returns the access key for the account",
"type": "securestring",
"value": "[listKeys(variables('storage_account_name'),'2015-05-01-preview').key1]"
},
"storageAccountName": {
//"description": "This is the computed name of the storage account, based on naming conventions in the variables",
"type": "string",
"value": "[variables('storage_account_name')]"
}
}
}
I have an existing Service Bus with one queue and event hub deployed using Azure Resource Manager.
Now I am interested to retrieve the primary key and connection string using Azure PowerShell wiithout using the ServiceBus.dll. Is it possible??
As a workaround I have created an ARM template which does not deploy anything but just query the existing resource and retrieve the information I need. The below template retrieves the connection string and primary key of an event hub/queue for a specific service bus namespace
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"serviceBusNamespace": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the service bus namespace to create."
}
},
"resourceName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the resource to be retreived."
}
},
"resourceType": {
"type": "string",
"minLength": 1,
"allowedValues": [
"queues",
"eventhubs"
],
"metadata": {
"description": "The type of the resource"
}
},
"policy": {
"type": "string",
"minLength": 1,
"defaultValue": "ManagePolicy",
"allowedValues": [
"ManagePolicy",
"SendPolicy",
"ListenPolicy"
],
"metadata": {
"description": "The type of the resource"
}
}
},
"variables": {
},
"resources": [ ],
"outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryConnectionString]"
},
"primaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryKey]"
}
}
}
Is it abusing to use ARM template to query for a resource and not actually deploy anything?
EDIT
To capture the output of the ARM template within PowerShell use the below code
$ep = New-AzureRmResourceGroupDeployment -Name "getEventHub" -ResourceGroupName myResourceGroup -Mode Incremental -TemplateFile getEventHub.json -TemplateParameterFile getEventHub.param.json
$RuleConnString = $ep.Outputs.connectionString.value
$RulePrimaryKey = $ep.Outputs.primaryKey.value
Note that the property names connectionString and primaryKey are same as defined in my template file
EDIT 2
If I re-run the ARM template to deploy the event hub second time I get the below error.
I din't find any option other than to use the ARM template to query the details.
I don’t see what’s wrong with what you’re doing. In my view Resource Manager templates in their nature are incremental. So you could author a template to create your existing service bus with the same resources. If the properties are the same then it will leave the existing resources intact and return you the connection string and primary key of the relevant resource.
I have a need to automate the creation of a service bus and queue and separate send/listen shared access policies. You can retrieve the connection string on the service bus itself using PowerShell natively without using the .Net ServiceBus.dll assembly by using Get-AzureSBAuthorizationRule but due to a still current bug this doesn’t work at the queue level.
I tried using the ServiceBus.dll to create the shared access policies but sometimes it would randomly fail but subsequently work if you ran it a second time immediately afterwards. I also tried Resource Manager templates but previously you had to pass in the keys you’d generated yourself. Now I see Microsoft generate those for you but you’re still left trying to get the key in an automated fashion so I like your solution.
One question though, can you capture the Resource Manager template outputs and pass them back to a PowerShell script, do you know?
Cheers
Rob
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {
"servicebusNamespace": {
"type": "string",
"metadata": {
"description": "The service bus namespace"
}
},
"notificationssmsqueue": {
"type": "string",
"metadata": {
"description": "Notifications SMS queue"
}
} }, "variables": {
"location": "[resourceGroup().location]", }, "resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('servicebusNamespace')]",
"type": "Microsoft.ServiceBus/namespaces",
"location": "[variables('location')]",
"properties": {
"messagingSku": 2
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('notificationssmsqueue')]",
"type": "Queues",
"dependsOn": [
"[concat('Microsoft.ServiceBus/namespaces/', parameters('servicebusNamespace'))]"
],
"properties": {
"path": "[parameters('notificationssmsqueue')]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Listen" ],
"revision": -1
}
},
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.send')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.send')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Send" ],
"revision": -1
}
}
]
}
]
} ], "outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/AuthorizationRules'),parameters('serviceBusNamespace'),'RootManageSharedAccessKey'),'2015-08-01').primaryConnectionString]"
},
"smsSendPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.send')),'2015-08-01').PrimaryKey]"
},
"smsListenPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.listen')),'2015-08-01').PrimaryKey]"
} } }
But I call my templates like this:
New-AzureRMResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile "$scripts_folder$SB_create_script" -TemplateParameterObject `
#{ servicebusNamespace = $servicebusNamespace;
notificationssmsqueue = $NotificationSMSqueue }
This is the correct way to get the information you are seeking. The Resource Manager provides a common interface to interact with all the services. It is how the Portal access the services, and each of the language SDKs are just wrappers for similar requests to the one you have created.
I usually use the Python or java SDKs, but I have been told that NodeJS is a very easy way to wrap the Web APIs that ARM calls to construct similar calls like the one you made, if you are looking for a none ARM way to do this.