We have some unit tests/integration tests running on Azure build/release pipelines. There are few tests that retrieve secrets from key vault and these are failing because the code is written for fetching secrets from keyvault using MSI and Azure app authentication features. Since pipelines are not enabled for MSI, the keyvault calls are failing and hence the tests are also failing. What is the alternative that exists for this scenario where pipelines can access keyvault successfuly?
Note: I have already gone through articles suggesting to use variable groups and azure keyvault tasks but not helpful in my scenario .Looking for alternatives.
Note: I have already gone through articles suggesting to use variable
groups and azure keyvault tasks but not helpful in my scenario.Looking
for alternatives.
You can try two directions:
Configure a self-hosted agent to run your pipeline in local environment. Of course the agent should be configured with your managed identity.
According to step5 from this blog:
AzureServiceTokenProvider will use the developer's security context to get a token to authenticate to Key Vault. This removes the need to create a service principal, and share it with the development team. It also prevents credentials from being checked in to source code. AzureServiceTokenProvider will use Azure CLI or Active Directory Integrated Authentication to authenticate to Azure AD to get a token. That token will be used to fetch the secret from Azure Key Vault.
You can use Azure Cli task to run your tests in command-line. Check this similar issue.
Related
I need to create Azure DevOps agent pool using Terraform.
In Terraform I'm using microsoft/azuredevops provider. And resource azuredevops_agent_pool
In conclusion, I have an error Error creating agent pool in Azure DevOps: Access denied. user needs Manage permissions to perform the action. For more information, contact the Azure DevOps Server administrator.
I have Administrator permissions in Azure Devops.
What can I try with this error?
UPD. I can create agent pool from Web UI azuredevops.
I am using authentication with PAT. PAT configured for FULL access.
UPD2. I understood that access on Project Level is other than access on Organization Level. So I have full access on Project Level but terraform is trying to create agent pull on Organization Level.
It's not possible at the moment. I face the same issue.
The issue is still opened since 2020.
https://github.com/Azure/terraform-azurerm-aci-devops-agent/issues/4
As mentioned on the README.md
Before running this module, you need to create an agent pool in your
Azure DevOps organization and a personal access token that it
authorized to manage this agent pool.
So it's not possible to automatically create an agent pool from the official Terraform azure DevOps provider
You might try to find a way through Azure DevOPS REST API:
https://learn.microsoft.com/en-us/rest/api/azure/devops/distributedtask/elasticpools/create?view=azure-devops-rest-7.1
You might also find this GitHub issue comment useful:
https://github.com/microsoft/terraform-provider-azuredevops/issues/204#issuecomment-962504540
Someone has already developed a way to do with Terraform using local-exec combined with azure cli/PowerShell/rest api
So I'm trying to add a new task, Azure File Copy, to my release pipeline. The file copy is pulling a single file from a new Azure Repository I created in Azure DevOps recently and putting it into a specific blob container. However, I seem to be running into an error
[error]AADSTS7000222: The provided client secret keys are expired. Visit the Azure Portal to create new keys for your app, or consider using certificate credentials for added security: https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-certificate-credentials
I tried looking for possible solutions for this, but considering this is a new repository, I'm not sure what I need to do. With my current existing app, I do have access to Microsoft Azure portal. With the link that's given in the error, it talks about updating the certificate, but it never had one to begin with.
Failing Azure File Copy when deploying a release to test environment with Azure DevOps
You could try to check if the service connection fails in Azure DevOps if you are using the Service Principal account to create the connection.
The service principal credential lifetime defaults to one year.
If yes, please create new credentials, and then update the Service Connection in Azure DevOps.
You could check this blog and this document for some more details.
It may be that the AZCOPY_SPA_CLIENT_SECRET environment variable on the machine running the task, is set to a key that has expired.
See: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10
I have a pipeline in Azure Devops building and pushing some images to DockerHub. These images are pushed to a production and development environment, but are also available for pulling for local development. For secrets in production and development in Azure we just use keystore and variable groups. However, we haven't found a good solution for injecting secrets when working locally.
For instance, in appsettings.json we have a ClientSecret parameter that is used for authenticating against Azure AD. How can we insert this parameter into appsettings.json during the release pipeline and not have it be visible to someone else later?
Some suggestions include using file transform in the pipeline, but this is not optimal as we don't really want to change any files. Another suggestion is using --build-arg in the pipeline, but these arguments become visible with docker history.
So how can I inject a secret into appsettings.json in a Docker image, and this secret should preferably not be visible anywhere at all?
As you have mentioned, with using File transforms a=nd variable substitution is a less configuration and quick approach.
Besides, you could also choose to use Azure Key Vault.
Azure Key Vault helps teams to securely store and manage sensitive
information such as keys, password, certificates, etc. in a
centralized storage which are safeguarded by industry-standard
algorithms, key lengths, and even hardware security modules. This
prevents information exposure through source code, a common mistake
that many developers make. Many developers leave sensitive information
such as database connection strings, passwords, private keys, etc. in
their source code which when gained by malicious users can result in
undesired consequences.
Access to a key vault requires proper authentication and authorization
and with RBAC, teams can have even fine granular control who has what
permissions over the sensitive data.
As for how to use Azure Key Vault in Azure DevOps, you could kindly refer below blog:
How to inject Azure Key Vault secrets in the Azure DevOps CI/CD
pipelines
Using secrets from Azure Key Vault in a pipeline
How to use docker image secret with Azure Key Vault, you could take a look at this link: Publishing a Single Image Docker Container with Secrets from VS2017 and Running it on Azure
More ways for your reference: 7 Ways to Deal with Application Secrets in Azure
I have several versions of the same web app running on the same subscription service.
I have 2 logical environments. Dev and UAT.
I have WebAppDev and WebAppUAT.
I have two key vaults KVDev and KVUAT.
How can I configure the correct web app to use the correct key vault?
What process assigns the web apps to key vaults?
Edit:
I had assumed that the key vault would act like the secrets do when developing.
This
Tutorial
seems to suggest that the key vault can be used as a configuration provider. However, the web app is not accessing the key vault values.
Get the Object ID from the identity blade of the web app.
Find your azure key vault and create a new access policy using the Object Id of web app.
Alternatively use the following in the powershell cli.
Set-AzKeyVaultAccessPolicy –VaultName -ObjectId "" -PermissionsToKeys backup,create,delete,get,import,list,restore -PermissionsToSecrets get,list,backup,restore,recover
Follow this tutorial and copy the context from the Program.cs in the sample code.
Tutorial
Taking a different angle, the current questions and comments tackle the authentication to KeyVault.
However, it sounds like a more fundamental problem and that you need to vary your configuration per environment.
i.e. WebAppDev needs to be configured to use a KVDev URL and WebAppUAT needs to use KVUAT URL.
Assuming you are using App Service Plans; this documentation provides a mechanism to store environment specific configuration along with guidance on how to use it for your programming language of choice, you will need to refer to the Application Specific configuration section.
https://learn.microsoft.com/en-us/azure/app-service/configure-common
Configuring in the portal will get you so far, but over time you will likely wish to contain the configuration in a release management pipeline so you don't need to configure things by hand. Azure DevOps Pipeline is one such tool for this:
https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/?toc=%2Fazure%2Fdevops%2Fpipelines%2Ftoc.json&bc=%2Fazure%2Fdevops%2Fboards%2Fpipelines%2Fbreadcrumb%2Ftoc.json&view=azure-devops
To get you started there is a specific deployment task which can aid in setting configuration for App Service Plans: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-rm-web-app-deployment?view=azure-devops
Over time I'd suggest rather than splitting configuration between a Release Pipeline and source control, instead having configuration which doesn't require deployment time modifications instead to stay source controlled in ARM templates, but that is an answer in it's own right so I won't confuse matters with too much detail.
If you want to access Azure key Vault by programming with SDK or REST API, then it would not be a problem. Because different key vaults have different DNS names. When you try to retrieve a secret or key in your code, you need to use its identify URL which contains its key vault DNS name. It will finally find the target key vault.
If you want to use key vault in web app with managed identity, you may refer to the tutorial: Use Azure Key Vault with an Azure web app in .NET
In that tutorial, you will enable the identity of a web app. And then you can assign access policy to that identity. In this way, the web app will be able to access the key vault with managed identity.
Let me give some details of my setup
I am building an asp.net core API app being hosted on Azure. I store my secret keys and stuff in azure keyvault. However, I have some AzureAddClientId and secret which is now stored in appsettings.json( to access key vault ). I have also committed appsettings.json to my git repo. However I know that is insecure. I use Azure DevOps for releases. So I'm thinking of doing the following. Please let me know your thoughts on this.
add appsettings.json to git ignore and share the file among developers.
add AzureAddClientId and AzureAADClientSecret to azure DevOps build pipeline as variables. ( Will devops automatically take the variables just as if they were in appsettings.json? )
Please have a look at using Managed Identities.
A common challenge when building cloud applications is how to manage the credentials in your code for authenticating to cloud services. Keeping the credentials secure is an important task. Ideally, the credentials never appear on developer workstations and aren't checked into source control. Azure Key Vault provides a way to securely store credentials, secrets, and other keys, but your code has to authenticate to Key Vault to retrieve them.
The managed identities for Azure resources feature in Azure Active Directory (Azure AD) solves this problem. The feature provides Azure services with an automatically managed identity in Azure AD. You can use the identity to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in your code.