In my Azure Pipeline (YAML), I am deploying an ARM template to create a key vault (among other resources), then running a PowerShell script to generate a certificate and store it in the key vault. This gives me a Forbidden error. In Use secrets from Azure Key Vault in Azure Pipelines, Microsoft suggests that I need to create an access policy on the key vault for the pipeline service principal. When I do this, the script succeeds.
Image from cache404
I would now like to create this access policy programmatically as part of the ARM template, but I don't know how to retrieve the object id for the pipeline service principal programmatically within the pipeline. Could someone please help?
Whilst Hugh Lin's answer is valid, I found it simpler to adapt Nick Graham's answer for granting key vault access to the pipeline service principal as part of the PowerShell script itself:
$Context = Get-AzContext
Set-AzKeyVaultAccessPolicy -VaultName $vaultName -ServicePrincipalName $Context.Account.Id -PermissionsToCertificates Get,List,Create
I would now like to create this access policy programmatically as part
of the ARM template, but I don't know how to retrieve the object id
for the pipeline service principal programmatically within the
pipeline.
If you want to automatically obtain the service principal object ID in the ARM template, I am afraid this is impossible.
You can try to create a script(Get-AzADServicePrincipal) to get the service principal and pass it to the arm template.
Here is the ticket with similar issue you can refer to.
Related
For a azure keyvault connection in Power Automate I am using an app registration. Users of a PowerApp I made can't seem to get secrets from the azure key vault unless I give them access to the keyvault. I was hoping adding the users to the acces policies in the keyvault would be enough.
Is there a way to let users get secrets in a PowerApp (through Power Automate) without giving them full access to the keyvault?
I am trying to do something similar as this
You could grant them the "get" permission only on secrets:
az keyvault set-policy --name myvault --secret-permissions get --upn <user ID/email>
However, a better approach might be to run your application as a service principal (or have middleware service that does - really depends on why users need access to the secrets) and it contact Key Vault directly. That service principal should be given minimal rights - basically the same command as above, except using --spn instead of --upn.
I am trying to do an ARM deployment in Azure Devops whereby I add a key vault access policy to an existing key vault in Azure.
I want to use the following ARM template which adds an access policy to an existing Key Vault: https://github.com/Azure/azure-quickstart-templates/tree/master/101-keyvault-add-access-policy
I have a separate template that deploys an App service and creates a system assigned managed identity.
What is the best way to link the two templates? how do i reference the System Assigned identity of the app service in the 'Add Key Vault Policy' ARM template if I dont know what the object ID of the service Principle is yet?
here's a sample how you would retrieve the managed identity Id in your other template:
"[reference(concat(resourceId('Microsoft.Web/sites/', %wep_app_name%), '/providers/Microsoft.ManagedIdentity/Identities/default'), '2015-08-31-PREVIEW').principalId]"
https://github.com/Azure/azure-quickstart-templates/blob/master/101-functions-managed-identity/azuredeploy.json#L295
I am trying to link Azure Key Vault secrets to a variable group in Azure Pipelines (part in Azure DevOps). Microsoft documentation here.
However, the "Authorize" button does not seem to work. It spins endlessly. Screenshot.
My target Azure Key Vault already has the service principal included in its access policy with Get and List permissions. Screenshot.
Anyone seen this issue before?
This workaround also seems like a bug for Azure Key Vault deployments using ARM templates.
If the service principal in question is added to the Azure Key Vault (AKV) access policies through an ARM template by referencing the service principal's Object ID (as Microsoft documentation calls for), permission errors with Azure Pipelines follow.
However, if I manually add the service principal to the AKV's access policies by referencing the service principal's application (client) ID, the permissions errors go away entirely.
Again, feels like a bug. And now my automated deployment pipeline doesn't quite work because of this manual step.
Also, in the AKV ARM template, if I were to combine the mandatory field objectId with the optional field applicationId, the service principal shows up as a "compound identity". That does not fix the permissions issues in Azure Pipelines. I do not see a way of adding a service principal properly without doing it manually.
Firstly, please make sure the service connection is working correctly. Then refresh the page and try it again. Alternately you can also try in browser inprivate session.
Just as the message said "The specified Azure service connection needs to have "Get, List" secret management permissions on the selected key vault."
Basically, we need to click the "Authorize" button to enable Azure Pipelines to set these permissions for the specific service connection.
If that doesn't work, we can also manually set the permissions for the specific service connection.
Go to Project settings - > Service connections -> Select the
specific ARM service connection
Click Edit to popup the Update Authentication for xxx dialog
Click the "use the full version of the service connection dialog."
link, to get the Service principal client ID
Go to your key vault in Azure portal -> Access Policies -> Add a new
Access Policy -> Select a template (e.g Key&Secret Management) - >
Select Get, List for Secret permissions.
Click Select Principal -> Copy and paste the Service principal client ID
to search the user/application -> Select the searched
user/application
After that you can see the new APPLICATION access policy.
Try it again after successfully adding the application access policy.
UPDATE:
Generally in Azure DevOps we need to create a ARM service connection (the client which can access the azure sources) first before deploying an Azure Key Vault through an ARM template.
Actually when you select the Azure subscription then click Authorize in Azure resource group deployment task
, the ARM service connection is created automatically. You just need to check the AppID and get the ObjectID to use in the ARM template.
We can get the Service principal client ID (AppID) by following above steps. After that we can get ObjectId by the AppID with running the following command: (See Find service principal object ID using PowerShell for details.)
$(Get-AzureADServicePrincipal -Filter "AppId eq 'a89c3dee-f5bf-4ea1-a805-d4c729a4add3'").ObjectId
Then you can specific the ObjectId when deploying the Azure Key Vault through an ARM template.
When running an Azure Powershell task in an Azure DevOps Release Pipeline with system.debug=true, you will get an output similar to this:
# anonymized
...
2019-09-05T12:19:41.8983585Z ##[debug]INPUT_CONNECTEDSERVICENAMEARM: '7dd40b2a-1c37-4c0a-803e-9b0044a8b54e'
2019-09-05T12:19:41.9156487Z ##[debug]ENDPOINT_URL_7dd40b2a-1c37-4c0a-803e-9b0044a8b54e: 'https://management.azure.com/'
2019-09-05T12:19:41.9188051Z ##[debug]ENDPOINT_AUTH_7dd40b2a-1c37-4c0a-803e-9b0044a8b54e: '********'
2019-09-05T12:19:41.9221892Z ##[debug]ENDPOINT_DATA_7dd40b2a-1c37-4c0a-803e-9b0044a8b54e: '{"subscriptionId":"b855f753-d5b3-48f4-b7cd-5beb58fb5508","subscriptionName":"Entenhausen","environment":"AzureCloud","creationMode":"Automatic","azureSpnRoleAssignmentId":"5ddcc3fe-f93c-4771-8041-50b49f76b828","azureSpnPermissions":"[{\"roleAssignmentId\":\"5ddcc3fe-f93c-4771-8041-50b49f76b828\",\"resourceProvider\":\"Microsoft.RoleAssignment\",\"provisioned\":true}]","spnObjectId":"76055cb6-3b75-4191-9309-306b32dad443","appObjectId":"e4b90b9d-7a73-42a3-ae6e-4daec910def4","environmentUrl":"https://management.azure.com/","galleryUrl":"https://gallery.azure.com/","serviceManagementUrl":"https://management.core.windows.net/","resourceManagerUrl":"https://management.azure.com/","activeDirectoryAuthority":"https://login.microsoftonline.com/","environmentAuthorityUrl":"https://login.windows.net/","graphUrl":"https://graph.windows.net/","managementPortalUrl":"https://manage.windowsazure.com/","armManagementPortalUrl":"https://portal.azure.com/","activeDirectoryServiceEndpointResourceId":"https://management.core.windows.net/","sqlDatabaseDnsSuffix":".database.windows.net","AzureKeyVaultDnsSuffix":"vault.azure.net","AzureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","StorageEndpointSuffix":"core.windows.net","EnableAdfsAuthentication":"false"}'
2019-09-05T12:19:41.9284444Z ##[debug]AuthScheme ServicePrincipal
...
I need to add the SPN of the Azure DevOps connection to a resource. When changing subscriptions or pipelines, the SPN also changes and I do not want to hardcode the value.
As the value is printed in the system.debug=true output, I am wondering how to access my own SPN within a pipeline task. Is it possible to read out spnObjectId":"76055cb6-3b75-4191-9309-306b32dad443" somehow using Powershell?
Information about the Service Principal can be accessed using Get-AzureRmContext but the information is limited and some is obfuscated in the logs so you need to make a second call to Get-AzureRmServicePrincipal to access the ObjectId
$Context = Get-AzureRmContext
$AzureDevOpsServicePrincipal = Get-AzureRmADServicePrincipal -ApplicationId $Context.Account.Id
$ObjectId = $AzureDevOpsServicePrincipal.Id
The Id exposed in $Context.Account.Id is the Service Principals ApplicationId
SPN within a pipeline task is nothing but the Azure subscription you have passed on to the task. You can click on manage connections and copy the details of the SPN under connections and use them as you need. But, I am not sure why do you want to use the SPN directly as you can always use an Azure Powershell Task and just select the subscription. Once you store the Connection, you can always reuse it in different pipelines.
I am following directions here for learning the AzureKeyVault config settings
Key Vault Configuration Provider sample application (ASP.NET Core 2.x)
This sample illustrates the use of the Azure Key Vault Configuration
Provider for ASP.NET Core 2.x. For the ASP.NET Core 1.x sample, see
Key Vault Configuration Provider sample application (ASP.NET Core
1.x).
For more information on how the sample works, see the Azure Key Vault
configuration provider topic.
Using the sample
Create a key vault and set up Azure Active Directory (Azure AD) for the application following the guidance in Get started with Azure Key
Vault.
Add secrets to the key vault using the AzureRM Key Vault PowerShell Module available from the
PowerShell
Gallery,
the Azure Key Vault REST API, or the Azure
Portal. Secrets are created as either
Manual or Certificate secrets. Certificate secrets are certificates for use by apps and services but are not supported by the
configuration provider. You should use the Manual option to create
name-value pair secrets for use with the configuration provider.
Simple secrets are created as name-value pairs. Azure Key Vault secret names are limited to alphanumeric characters and dashes.
Hierarchical values (configuration sections) use -- (two dashes) as a separator in the sample. Colons, which are normally used
to delimit a section from a subkey in ASP.NET Core
configuration, aren't allowed
in secret names. Therefore, two dashes are used and swapped for a
colon when the secrets are loaded into the app's configuration.
Create two Manual secrets with the following name-value pairs. The first secret is a simple name and value, and the second
secret creates a secret value with a section and subkey in the secret
name:
SecretName: secret_value_1
Section--SecretName: secret_value_2
Register the sample app with Azure Active Directory.
Authorize the app to
access the key vault. When you use the
Set-AzureRmKeyVaultAccessPolicy PowerShell cmdlet to authorize the
app to access the key vault, provide List and Get access to
secrets with -PermissionsToSecrets list,get.
Update the app's appsettings.json file with the values of Vault, ClientId, and ClientSecret.
Run the sample app, which obtains its configuration values from IConfigurationRoot with the same name as the secret name. *
Non-hierarchical values: The value for SecretName is obtained with
config["SecretName"]. * Hierarchical values (sections): Use :
(colon) notation or the GetSection extension method. Use either of
these approaches to obtain the configuration value:
config["Section:SecretName"]
config.GetSection("Section")["SecretName"]
Okay so I have copied the name of my application into Azure Active Directory as an 'Enterprise Application'. And I have added 'Access policies' for 'get' and 'list' in Azure for my ADD object I just created. Yet I get this error in the program when attempting to start the application:
Exception: {"error":"unauthorized_client","error_description":"AADSTS70001:
Application with identifier '(guid)' was not found in the directory ...(continues)
Update 8-4-18
Okay I found out that Azure uses the 'ClientId' and 'ClientSecret' in the local appsettings.json to connect to what Azure registers in this tutorial: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#log-in-as-the-application
I get the clientId in appsettings.json from the applicationId on ADD I create with ADD>App Registrations>New
I click settings in ADD on the app I just created and create a key with an expiration to store as ClientSecret in appsettings.json.
I change my 'Vault' in appsettings to my named vault.
I run the powershell above to give access or else do it in ADD.
So now I am getting a simpler error:
'Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: 'Access denied''
I have tried running as Administrator in Visual Studio. I went under Subscriptions in Azure>Access Control>(IAM)>set my new apps to Reader.
So the reason your powershell is failing is because you are trying to assign a User Principal - a user - when actually you want a Service Principal.
I can’t see your C# to support more there than saying when you use the SDK to log in as the Service Principal you use the application id of the Application/Service Principal (its the same id).
The service principal acts like a user in the local directory but you log in as the application.
Edit:
I looked at the example you posted and ran it myself and had very similar problems. However I have got it working. Here's the steps:
Creating the Application
Create the Registered Application. I do this through the Azure Portal so
a Service Principal is created automatically. Make a note of the ApplicationId.
Generate a key credential on the created application and make a note of it.
In the Application click on the link to the Managed app in local directory. This is the Service Principal, make a note of the ObjectId
Creating the Key Vault
Create KeyVault - I used PowerShell to do this. New-AzureRmKeyVault
Apply the Service Principal to the Key Vault.
Set-AzureRmKeyVaultAccessPolicy -VaultName <vault> -ResourceGroupName <ResourceGroupName> -ObjectId <Object Id of the Created Service Principal> -PermissionsToSecrets Get,List
Running the Sample App
In your application settings follow this format:
{
"Vault": <the name of your vault>,
"ClientId": <ApplicationId of the Registered Application>,
"ClientSecret": <Credential generated from the Registered Application>
}
This worked for me and allowed me to run the sample and retrieve the secrets from the vault.
The ultimate problem for me became that running 'Set-AzureRmKeyVaultAccessPolicy' was not needed and for whatever reason it was easier to just ignore it and follow this subsection: https://azure.microsoft.com/documentation/articles/key-vault-get-started/#authorize
I kept trying to set up Object Id and Keys and really I had just overlooked a section mentioning a 'ServerPrincipalName'
They set one commandlet for keys
Set-AzureRmKeyVaultAccessPolicy -VaultName '<vaultName>' -ServicePrincipalName <ApplicationIdGuid> -PermissionsToKeys decrypt,sign
They set one commandlet for secrets
Set-AzureRmKeyVaultAccessPolicy -VaultName '<vaultName>' -ServicePrincipalName <ApplicationIdGuid> -PermissionsToSecrets Get, List
But I decided to follow the immediate proceeding section on doing it all in the Portal. The key take away for me was that the instructions were not wrong. Just vague when it says: "Register a sample app" then "Authorize the App". Really they should be saying
Register a sample app (https://learn.microsoft.com/en-us/azure/key-vault/key-vault-get-started#register)
Authorize the app with Key Vault (https://azure.microsoft.com/documentation/articles/key-vault-get-started/#authorize)
Ultimately all the information is there it was just confusing if you happen to already have a vault and an application and don't understand the prerequisite is that really you need to have a 1. A Vault, 2. An ADD Web Application, 3. Associate permissions for 2 in 1.