Unable to obtain azure key vault secret in ansible - azure

I try to get a secret from azure key vault in my ansible 4 playbook using azcollection 1.9.0.
- name: Get secret value
azure_rm_keyvaultsecret_info:
vault_uri: https://my-vault.vault.azure.net/
register: kvSecret
According to the docs the result should contain a list of secrets with a property called secret containing the secret value.
However, this property is not present on the result set. This is the result I get:
{
"changed": False,
"secrets": [
{
"sid": "https: //my-vault.vault.azure.net/secrets/ssh-user-username",
"version": "",
"tags": {},
"attributes": {
"enabled": True,
"not_before": None,
"expires": None,
"created": "2021-09-05T14:32:10+00:00",
"updated": "2021-09-05T14:32:10+00:00",
"recovery_level": "Recoverable+Purgeable"
}
}
],
"failed": False
}
If I try to get this exact secret using the name option I get an empty result set.
My vault contains this secret, it has a value and the service principal has access to my key vault through IAM with the roles Key Vault Reader and Key Vault Secrets User.

I tested it on my environment and my service principal is having Key vault reader and Key vault secrets user with the below yml code.
---
- hosts: localhost
connection: local
collections:
- azure.azcollection
vars:
vault_name: Testansumankeyvault01
secret_name: adminPassword
tasks:
- name: Get Key Vault by name
azure_rm_keyvault_info:
resource_group: test-rg
name: "{{ vault_name }}"
register: keyvault
- name: Set key vault URI fact
set_fact: keyvaulturi="{{ keyvault['keyvaults'][0]['vault_uri'] }}"
- name: Get secret value
azure_rm_keyvaultsecret_info:
vault_uri: "{{ keyvaulturi }}"
name: "{{ secret_name }}"
register: kvSecret
- name: set secret fact
set_fact: secretValue="{{ kvSecret['secrets'][0]['secret'] }}"
- name: Output key vault secret
debug:
msg="{{ secretValue }}"
Reference:
Azure built-in roles - Azure RBAC | Microsoft Docs
Use Azure Key Vault to store VM secrets with Ansible | Microsoft Docs

Turns out that this was an issue with the authentication. Ansible is connecting to my remote machine via ssh and therefore I needed to set the authentication for azure. I was doing this with environment variables in my ansible playbook but it turns out that they are not set when the playbook runs it's tasks. Passing them explicitly to the command does the trick.

Related

How to handle special characters in Azure DevOps yaml pipeline?

I am using Data thirst AzDo task to create bearer token in my yaml pipeline. Reference to the task: https://marketplace.visualstudio.com/items?itemName=DataThirstLtd.databricksDeployScriptsTasks.
The SPN secret I have has special characters in it and is being passed from Azure DevOps library. The sample secret is something like this SP=X$fg#ab*a].
When I pass this value to the Azure DevOps yaml pipeline, I am getting a 403 error. My guess is, the secret value is not being parsed the right way and hence the 403. How should one handle special characters in an Azure DevOps yaml pipeline ?
My AzDo task is something like this:
- task: DataThirstLtd.databricksDeployScriptsTasks.databricksDeployCreateBearer.databricksDeployCreateBearer#0
displayName: 'Get Databricks Bearer Token'
inputs:
applicationId: '$(client_id)'
spSecret: '$(client_secret)' #this is where the secret value is passed from AzDo library
resourceGroup: '$(rg_name)'
workspace: '$(adb_workspace)'
subscriptionId: '$(subscription_id)'
tenantId: '$(tenant_id)'
region: '$(location)'
Based on my research, the variable seems been passed correctly:
trigger:
- none
pool:
vmImage: windows-latest
variables:
- group: xxx
# The below setting can help check what happened.
- name: system.debug
value: true
steps:
- task: databricksDeployCreateBearer#0
inputs:
applicationId: 'xxx'
spSecret: '$(SP)'
resourceGroup: 'xxx'
workspace: 'xxx'
subscriptionId: 'xxx'
tenantId: 'xxx'
region: 'westeurope'
Result:
In fact, the returned 403 code also proves that the incoming secret is correct from another aspect. Code 403 often means that the server side knows who initiated the request but rejected the request (403 Forbidden). The problem may come from insufficient AAD app permissions or restricted ip.
I suggest you take a look at this case:
Error 403 User not authorized when trying to access Azure Databricks API through Active Directory

Azure ARM template how to get dynamic keyvault appsettings to function

I took the template from quickstart templates (https://github.com/Azure/azure-quickstart-templates/blob/master/101-functions-managed-identity/azuredeploy.json).
I would like to add "#Microsoft.KeyVault(SecretUri=secret_uri_with_version)" application setting to my keyvault within ARM template. How can I do this? Lets say my keyvault name is "MyKeyVault" and my secret name is "MySecret". I found from MS docs that this kind of reference should work:
"WEBSITE_CONTENTAZUREFILECONNECTIONSTRING": "[concat('#Microsoft.KeyVault(SecretUri=', reference(variables('keyVaultResourceId')).secretUriWithVersion, ')')]",
But I just get error message "The resource 'Microsoft.KeyVault/vaults/MyKeyVault' is not defined in the template."
I am creating resource Id with this line:
"keyVaultResourceId": "[resourceId(subscription().subscriptionId, parameters('vaultResourceGroupName'), 'Microsoft.KeyVault/vaults', parameters('keyVaultName'))]"
ah, ok I see, you are trying to get reference from the Key Vault, not from the secret. the key vault obviously doesn't have that property, because its not a secret, its a key vault. So you need to construct a reference to your secret:
reference(resourceId('rg','Microsoft.KeyVault/vaults/secrets','kvname','secretname').secretUriWithVersion)

Can Secrets From Objects Created in ARM Templates Get Auto Added to Key Vault

If I have an Azure ARM template that can create:
Azure Container Registry
Azure Key Vault
Is there a way for the username and password for the Azure Container Registry to be automatically be added to the Azure Key Vault using ARM templates?
Is there some way to refer to the Azure Container Registry username and password secrets in ARM templates for this purpose?
UPDATE
#EdBoykin's answer is correct, this is what I ended up with:
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('key_vault_name'), '/AzureContainerRegistryKey1')]",
"apiVersion": "2015-06-01",
"properties": {
"contentType": "text/plain",
"value": "[listCredentials(resourceId('Microsoft.ContainerRegistry/registries', parameters('container_registry_name')), '2017-10-01').passwords[0].value]"
},
"dependsOn": [
"[concat('Microsoft.KeyVault/vaults/', parameters('key_vault_name'))]",
"[concat('Microsoft.ContainerRegistry/registries/', parameters('container_registry_name'))]"
]
}
Muhammad,
To create the secrets in KeyVault you will need to create an ARM template that looks something like this. Make sure to update the 'dependson' section so this resource depends on your ACR being created first. The username is going to be the ACR resource name. So, whatever you set that to in your ARM script, you can store in your key vault as a key vault secret.
For the passwords, or keys, this is what you do. Here is a sample template for adding a KeyVault secret
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(variables('keyVaultName'), '/{YourACRKey1SecretName}')]",
"apiVersion": "2015-06-01",
"properties": {
"contentType": "text/plain",
"value": "[listCredentials(resourceId('Microsoft.ContainerRegistry/registries', parameters('YourACRName')), '2017-10-01').passwords[0].value]"
},
"dependsOn": []
}
{YourACRKey1SecretName} should be changed to the secret name for your ACR Key1 value.
To set the other key in your keyvault, create another key vault secret resource with a new name and use this for the value:
For Key 2
[listCredentials(resourceId('Microsoft.ContainerRegistry/registries', parameters('YourACRName')), '2017-10-01').passwords[1].value]

Azure Key Vault Understanding

The following github deployment allows me to deploy a simple Windows VM by retrieving the password that is stored in a Key Vault. Therefore the password is never put in plain text in the template parameter file.
Can someone explain what is meant with the statement:
'the password is never put in plain text in the template parameter
file?'
If you don't use Key Vault, even adminPassword type is securestring, When you enter the adminPassword during deployment, it is showing as plaintext, not ******. It is not safe. But if you use Key Vault, the password stores in Key Vault and encrypted save, other could not see your password.
In template, you should configure your template using Key Vault like below:
"adminPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/XXXXXXX/resourceGroups/resourceGroupName/providers/Microsoft.KeyVault/vaults/vaultName"
},
"secretName": "secretName"
}
},
Please refer to the similar question:How to hide password in shell script duing ARM template deployment.

Error when deploying ARM template that includes certificate stored in key vault

I am attempting to deploy an ARM template from Release Management that includes a 'Microsoft.Web/certificates' resource which references a certificate stored in a key vault. This works fine when the key vault exists in the same subscription as the resource group I am deploying to. When the key vault exists in a different subscription however, I receive the below error.
Resource Microsoft.Web/certificates 'cert name' failed with message
{
"Code": "BadRequest",
"Message": "The parameter Properties.KeyVaultId has an invalid value.",
"Target": null,
"Details": [
{
"Message": "The parameter Properties.KeyVaultId has an invalid value."
},
{
"Code": "BadRequest"
},
{
"ErrorEntity": {
"Code": "BadRequest",
"Message": "The parameter Properties.KeyVaultId has an invalid value.",
"ExtendedCode": "51008",
"MessageTemplate": "The parameter {0} has an invalid value.",
"Parameters": [
"Properties.KeyVaultId"
],
"InnerErrors": null
}
}
],
"Innererror": null
}'
The certificate resource is defined as below in my template.
{
"type":"Microsoft.Web/certificates",
"name": "SomeName",
"location": "East US 2",
"apiVersion": "2016-03-01",
"properties": {
"keyVaultId": "/subscriptions/<subscriptionId>/resourceGroups/<vault resource group>/providers/Microsoft.KeyVault/vaults/<vault name>",
"keyVaultSecretName": "SecretName"
}
}
I am using the Azure Resource Group Deployment Task in VSTS to deploy the resource group. The task is configured to use an endpoint with a service principal that has the below permissions set in Azure:
Key Vault Contributor Role on the resource group containing the key vault.
Get secret permissions on the key vault
The Microsoft.Azure.WebSites principal was granted Get permissions on the key vault secrets.
The key vault also has the 'Enable access to Azure Resource Manager for template deployment' option enabled. The certificate was uploaded to the key vault using powershell, not via the portal.
Am I missing something here?
Thanks
I think I found the cause of this issue. Apparently, when a resource group has been created, you cannot change the secret name. If you do so, the error above will be thrown.
If you want to change the secret name, you need to delete the resource group and redeploy everything.
Have you been changing the secret name in the ARM template, without removing the full resource group in the azure portal?
You can get this if you have referenced a certificate (secret) in keyvault on a previous deployment and the certificate has been removed or replaced in keyvault. The new deployment will fail with the above error(51008). An example could be if you have migrated a secret from another keyvault store.
This is not the same as versions of the same certificate. New versions will work fine.
Replace the secret with the original in keyvault or delete the secret and add a new one.

Resources