I'm attempting to build out my DevOps pipeline to deploy a DataFactory, Databricks Notebooks & Azure Data Warehouse,
I have my resource subscriptions setup for both Dev and Prod. deploying to Prod is more tricky than it seems.
my keyvault has GET/LIST Permissions for both Secret & Keys for the Target DataFactory.
https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment
I have used the above guide to setup my target data factory in prod - and it is stood up correctly with all the connection strings setup and keyvault permissions set.
but I am stuck on this portion :
Grant permissions to the Azure Pipelines agent The Azure Key Vault
task may fail with an Access Denied error if the proper permissions
aren't present. Download the logs for the release, and locate the .ps1
file with the command to give permissions to the Azure Pipelines
agent. You can run the command directly, or you can copy the principal
ID from the file and add the access policy manually in the Azure
portal. Get and List are the minimum permissions required.
when I deploy my release I get the following error on the KeyVault task :
The specified Azure service connection needs to have Get, List secret management permissions on the selected key vault. To set these permissions, download the ProvisionKeyVaultPermissions.ps1 script from build/release logs and execute it, or set them from the Azure portal
I've added this power shell script ProvisionKeyVaultPermissions.ps1 to my repo and added it to my task but it just runs forever ? unsure if I'm missing something here.
hope this is clear/ please ask for any additional info.
I wonder if it's the DevOps service connection that's missing the permissions.
You can check access policies for the vault from the console. You should see your service connection as an APPLICATION; it needs the GET and LIST privileges as the document your following says. My understanding is that these are privileges for the account that's deploying your code, rather than the account that will run your code.
Related
I have a DevOps release pipeline pushing out (via CLI on a VM) to a dev resource group happily. When I add a stage pushing to an identical QA resource group I get the below.
ERROR: (AuthorizationFailed) The client '' with object id '' does not have authorization to perform action 'Microsoft.Web/sites/config/list/action' over scope '/subscriptions//resourceGroups//providers/Microsoft.Web/sites//config/publishingcredentials' or the scope is invalid. If access was recently granted, please refresh your credentials.
Is there a place I should be looking at first?
The error is due to the service principa(in azure devops, typically is 'service connection') doesn't have correct role/permission on the target resource.
Please find your service connection in Project settings.
Click Manage Service Principal which will redirect you to the Application Registration of the Service Principal. All you need to do here is copy the name (the default format is <account>-<prject>-<GUID>).
Go back and click Manage service connection roles which will redirect you to the IAM blade of the Azure Subscription. Here you need to assign a role to the service principal of which you copied the name of in the previous step. you can add it as subscription owner/contributor role on the target subscription simply.
You can check similar answer here.
I have created an Azure Container Registry.
I have an Azure DevOps project.
I have created an Azure DevOps Pipeline using the wizard so
that it uses the standard template to build and push a Docker image.
When validating the Pipeline the following error is thrown:
Failed to set Azure permission 'RoleAssignmentId: ****' for the service principal '****' on subscription ID '****': error code: Forbidden, inner error code: AuthorizationFailed, inner error message The client '****' with object id '****' does not have authorization to perform action 'Microsoft.Authorization/roleAssignments/write' over scope '/subscriptions/****/resourceGroups/****/providers/Microsoft.ContainerRegistry/registries/****/providers/Microsoft.Authorization/roleAssignments/****' or the scope is invalid. If access was recently granted, please refresh your credentials. Ensure that the user has 'Owner' or 'User Access Administrator' permissions on the Subscription.
What configuration could I be missing? The documentation for this is all very sparse and written as though it should all just work.
Thanks
You need to add AcrPull permission to service principal you used here. Please got to you ACR and add it.
Here you have id of you service principal:
Failed to set Azure permission 'RoleAssignmentId: ' for the service principal ''
Also please check also networking on ACR if it blocks you or not.
To build and push your Docker image in Azure pipeline, you need follow the steps below:
Navigate to Project Settings > Service connections to create a Docker Registry service connection that can connect to your ACR.
In the pipeline, add the Docker task to build and push your Docker image to ACR.
Yours service principal on service connection is missing permissions link to documentation https://learn.microsoft.com/en-us/azure/container-registry/container-registry-roles?tabs=azure-cli. You can use custom role or build like contributor on resource group level. Example of custom role and allowed actions "Microsoft.ContainerRegistry/registries/push/write",
I'm using the Microsoft Extension to access KeyVaults from Azure DevOps to fetch a secret from the vault. I'm getting this error message which appears to say that I need to allow rights the Azure Devops agent access to the keyvault.
"Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request""
I have this running on a separate test subscription where the AzureDevOps account has contributor access under Role Assignments, same on the PROD subscription where the error message is coming up.
Any insights would be greatly appreciated.
THanks,
This seems to be an issue with Azure keyvaults defined with service endpoints. As Azure DevOps is not a trusted service it does not have access to the endpoint. The workaround is to whitelist the ip of the agent right from the pipeline, removing it after you've obtained the secrets(figuring out how to do this now). This is referenced in here and here
I am trying to link Azure Key Vault secrets to a variable group in Azure Pipelines (part in Azure DevOps). Microsoft documentation here.
However, the "Authorize" button does not seem to work. It spins endlessly. Screenshot.
My target Azure Key Vault already has the service principal included in its access policy with Get and List permissions. Screenshot.
Anyone seen this issue before?
This workaround also seems like a bug for Azure Key Vault deployments using ARM templates.
If the service principal in question is added to the Azure Key Vault (AKV) access policies through an ARM template by referencing the service principal's Object ID (as Microsoft documentation calls for), permission errors with Azure Pipelines follow.
However, if I manually add the service principal to the AKV's access policies by referencing the service principal's application (client) ID, the permissions errors go away entirely.
Again, feels like a bug. And now my automated deployment pipeline doesn't quite work because of this manual step.
Also, in the AKV ARM template, if I were to combine the mandatory field objectId with the optional field applicationId, the service principal shows up as a "compound identity". That does not fix the permissions issues in Azure Pipelines. I do not see a way of adding a service principal properly without doing it manually.
Firstly, please make sure the service connection is working correctly. Then refresh the page and try it again. Alternately you can also try in browser inprivate session.
Just as the message said "The specified Azure service connection needs to have "Get, List" secret management permissions on the selected key vault."
Basically, we need to click the "Authorize" button to enable Azure Pipelines to set these permissions for the specific service connection.
If that doesn't work, we can also manually set the permissions for the specific service connection.
Go to Project settings - > Service connections -> Select the
specific ARM service connection
Click Edit to popup the Update Authentication for xxx dialog
Click the "use the full version of the service connection dialog."
link, to get the Service principal client ID
Go to your key vault in Azure portal -> Access Policies -> Add a new
Access Policy -> Select a template (e.g Key&Secret Management) - >
Select Get, List for Secret permissions.
Click Select Principal -> Copy and paste the Service principal client ID
to search the user/application -> Select the searched
user/application
After that you can see the new APPLICATION access policy.
Try it again after successfully adding the application access policy.
UPDATE:
Generally in Azure DevOps we need to create a ARM service connection (the client which can access the azure sources) first before deploying an Azure Key Vault through an ARM template.
Actually when you select the Azure subscription then click Authorize in Azure resource group deployment task
, the ARM service connection is created automatically. You just need to check the AppID and get the ObjectID to use in the ARM template.
We can get the Service principal client ID (AppID) by following above steps. After that we can get ObjectId by the AppID with running the following command: (See Find service principal object ID using PowerShell for details.)
$(Get-AzureADServicePrincipal -Filter "AppId eq 'a89c3dee-f5bf-4ea1-a805-d4c729a4add3'").ObjectId
Then you can specific the ObjectId when deploying the Azure Key Vault through an ARM template.
I've created an ARM template that uploads an SSL certificate to Azure. I can run the ARM template from my local machine and it succeeds without any errors.
I then log into VSTS to set up a release template to perform the same task. I select the same template and provide the same parameters, but when I deploy the release it errors with a 401. The only information I can get by looking into the logs is that the request is denied because authorization failed.
I updated the permissions for the Service Principal account that VSTS is using to match my own (Owner, Contributor) of the resource group I'm deploying to and I get the same error. I then, temporarily, updated its permissions to Owner, Contributor of the entire subscription, and it still failed.
I'm beginning to suspect that this is an issue with Service Principal accounts in general, but I can't find any documentation to confirm or refute this claim.