I have a pipeline in Azure Devops building and pushing some images to DockerHub. These images are pushed to a production and development environment, but are also available for pulling for local development. For secrets in production and development in Azure we just use keystore and variable groups. However, we haven't found a good solution for injecting secrets when working locally.
For instance, in appsettings.json we have a ClientSecret parameter that is used for authenticating against Azure AD. How can we insert this parameter into appsettings.json during the release pipeline and not have it be visible to someone else later?
Some suggestions include using file transform in the pipeline, but this is not optimal as we don't really want to change any files. Another suggestion is using --build-arg in the pipeline, but these arguments become visible with docker history.
So how can I inject a secret into appsettings.json in a Docker image, and this secret should preferably not be visible anywhere at all?
As you have mentioned, with using File transforms a=nd variable substitution is a less configuration and quick approach.
Besides, you could also choose to use Azure Key Vault.
Azure Key Vault helps teams to securely store and manage sensitive
information such as keys, password, certificates, etc. in a
centralized storage which are safeguarded by industry-standard
algorithms, key lengths, and even hardware security modules. This
prevents information exposure through source code, a common mistake
that many developers make. Many developers leave sensitive information
such as database connection strings, passwords, private keys, etc. in
their source code which when gained by malicious users can result in
undesired consequences.
Access to a key vault requires proper authentication and authorization
and with RBAC, teams can have even fine granular control who has what
permissions over the sensitive data.
As for how to use Azure Key Vault in Azure DevOps, you could kindly refer below blog:
How to inject Azure Key Vault secrets in the Azure DevOps CI/CD
pipelines
Using secrets from Azure Key Vault in a pipeline
How to use docker image secret with Azure Key Vault, you could take a look at this link: Publishing a Single Image Docker Container with Secrets from VS2017 and Running it on Azure
More ways for your reference: 7 Ways to Deal with Application Secrets in Azure
Related
I have developed an Azure Function using visual Studio 2019. now when i deployed the Azure Function inside Azure >> all the settings (which contain some sensitive info) inside the local.settings.json, will be added inside the ""Azure Configuration" >> "App settings"". And as indicated by the page message that those values are stored encrypted, as follow:-
so does this mean that there is no need to take extra steps to secure the sensitive info inside the App setting by using Azure Key Vault ? as those are already secured?
They are secure, but users with permissions to the recourse can potentially access them with a role such as e.g. Contributor. Using a Key Vault would allow you to define access controls more precisely.
As example; Should somebody whose sole responsibility is to manage the resource have access to the secrets?
Also Key Vault allows for some other scenario's like being able to manage secrets centrally or share them across multiple resources (e.g. App Services across multiple regions). And maybe one that's often overlooked; monitoring. E.g. being able to tell 'who' accessed what key at what time.
Edit: link for adding key vault references to your app settings
It is considered a best practice to store any secrets your application might need in a Key Vault. This way, if someone has access to the application that needs to use these secrets in the Azure portal, they would still not have access to the actual secret.
Centralizing storage of application secrets in Azure Key Vault allows you to control their distribution. Key Vault greatly reduces the chances that secrets may be accidentally leaked. When application developers use Key Vault, they no longer need to store security information in their application. Not having to store security information in applications eliminates the need to make this information part of the code. For example, an application may need to connect to a database. Instead of storing the connection string in the app's code, you can store it securely in Key Vault.
Your applications can securely access the information they need by using URIs. These URIs allow the applications to retrieve specific versions of a secret. There's no need to write custom code to protect any of the secret information stored in Key Vault.
Source: About Azure Key Vault - Why use Azure Key Vault? - Centralize application secrets
TL;DR
Do you need to use Key Vault currently? Maybe not, although still depending on how many other people have access to the Subscription / Resource Group / Function App.
Would I advise you to put your secrets in Key Vault? Yes. Especially by doing so now will get you used to working with Key Vault and get you prepared for the future.
EDIT:
For referencing secrets from Key Vault, have a look at Key Vault references. This enables your application to get the secrets from Key Vault without the need for any code change!
For documentation on how to, please see Use Key Vault references for App Service and Azure Functions.
EDIT 2:
even if we store them inside Azure key vault they can be viewed
Yes, but then only if someone has the correct role assignment. If the secret is in Application Settings, anyone with access to the Function App inside of the Azure portal can also see the secrets.
Is there any point in using Azure Key Vault over App Configuration?
Yes, yes, I know - they are complimentary, key vault for secrets, app config for... well, app config.
But, considering they are both encrypted, basically for someone to see either a secret or a config value they'd have to have access to your azure portal (this is a low-level bad guy scenario).
The ONLY difference I see is that you can control permissions differently between the vault and config but apart from that if someone unauthorized has access to your portal you've got bigger problems.
So - why? and please only good and real arguments no "because you should" or "because person X said so", what benefits would I reap with key vault that I don't have with app config?
I appreciate your question. I'd re-phrase it to this:
Q) How are are Key Vault and App Configuration designed differently supporting different purposes? And where can I find a clear comparison table of features and benefits?
I also appreciate your aside:
please only good and real arguments no "because you should" or
"because person X said so", what benefits would I reap with key vault
that I don't have with app config?
Here is what I found to understand benefits as a contrast:
Article 1: What is Azure App Configuration?
App Configuration complements Azure Key Vault, which is used to store application secrets. App Configuration makes it easier to implement the following scenarios:
Centralize management and distribution of hierarchical configuration data for different environments and geographies
Dynamically change application settings without the need to redeploy or restart an application
Control feature availability in real-time
Article 2 : Key management with Key Vault
Key management with Key Vault
Without proper protection and management of the keys, encryption is rendered useless. Key Vault is the Microsoft-recommended solution for managing and controlling access to encryption keys used by cloud services. Permissions to access keys can be assigned to services or to users through Azure Active Directory accounts.
Key Vault relieves organizations of the need to configure, patch, and maintain hardware security modules (HSMs) and key management software. When you use Key Vault, you maintain control. Microsoft never sees your keys, and applications don’t have direct access to them. You can also import or generate keys in HSMs.
Article 3: Azure Key Vault recovery management with soft delete and purge protection
Soft delete and purge protection are two different key vault recovery features.
I think you should not neglect the fact that someone that have the configuration of your App Service can see the secrets. A developer of your company could have access a production App Service for bug investigation but it should not mean he has access to production secrets. The fact of having a single employee's laptop (with access to Azure Portal) hacked should not necessarily mean "access to every secret of your application".
But appart from (as you already know the above), what differences I see:
Better governance and acess monitoring : you have logs to see who try to access each secret, when and how, something you cannot do in an App Service
Better secret management: all secrets stored in one place, you only modify them in key vault once instead of on each app service configuration when some secrets are shared in (used by) multiple app services
Some advanced mechanism like recovery management and purge protection
Better development experience : when debugging an app, if your user has access to the keyvault he can just run the app that will load secrets from keyvault into configuration instead of manually copying secrets from azure app settings locally
1-Data stored in Azure Key Vault is encrypted (App Configuration is not)
2-If a person is associated with Contributor role, he/she can see the configurations in your App Configuration, on Key Vault, only allowed Principals.
3-You can rotate secrets stored in Azure Key Vault, and there won't be any downtime in your app (unless you're caching it on your App Service and need to restart it, in order to refresh the cache)
4-Azure Key Vault is the Microsoft recommended service to store Secrets, Keys and Certificates
I have configured a new .Net core application with CI/CD trough azure devops. I use the configuration tab in the application in azure to set settings - like token secrets, for example. However, every time the CI/CD deploys a new version, those settings are erased and I have to set them again.
Does anyone know how to stop this from happening?
I do not think that you can "stop" this from happening. The deployment of an application includes appication settings.
What you can do is to get your CI/CD pipeline to set the settings correctly. See: https://devblogs.microsoft.com/devops/managing-configuration-app-settings-for-multiple-environments-in-your-cd-pipeline/
Just want to through out another alternative to this is to have the secrets stored in a Key Vault and if deploying your App Service via an ARM template the secret values can be referenced to those stored in a key vault. Refer to this answer as a starting point.
Best practice would be the App Service is configured for Managed Service Identity and granted an access policy to a Key Vault where the secrets or connection strings are stored. This would lead to a more decoupled and reusable architecture. On App Service deployment the secrets will be grabbed from Key Vault and inserted into the App Settings. This type of deployment would work ideally with static keys.
If your keys could potentially changes there are methods to check Key Vault for a new password.
I will call out that this type of implementation will shift the secrets being wiped out from every time the app service is being deployed to every time the Key Vault is deployed.
I have several versions of the same web app running on the same subscription service.
I have 2 logical environments. Dev and UAT.
I have WebAppDev and WebAppUAT.
I have two key vaults KVDev and KVUAT.
How can I configure the correct web app to use the correct key vault?
What process assigns the web apps to key vaults?
Edit:
I had assumed that the key vault would act like the secrets do when developing.
This
Tutorial
seems to suggest that the key vault can be used as a configuration provider. However, the web app is not accessing the key vault values.
Get the Object ID from the identity blade of the web app.
Find your azure key vault and create a new access policy using the Object Id of web app.
Alternatively use the following in the powershell cli.
Set-AzKeyVaultAccessPolicy –VaultName -ObjectId "" -PermissionsToKeys backup,create,delete,get,import,list,restore -PermissionsToSecrets get,list,backup,restore,recover
Follow this tutorial and copy the context from the Program.cs in the sample code.
Tutorial
Taking a different angle, the current questions and comments tackle the authentication to KeyVault.
However, it sounds like a more fundamental problem and that you need to vary your configuration per environment.
i.e. WebAppDev needs to be configured to use a KVDev URL and WebAppUAT needs to use KVUAT URL.
Assuming you are using App Service Plans; this documentation provides a mechanism to store environment specific configuration along with guidance on how to use it for your programming language of choice, you will need to refer to the Application Specific configuration section.
https://learn.microsoft.com/en-us/azure/app-service/configure-common
Configuring in the portal will get you so far, but over time you will likely wish to contain the configuration in a release management pipeline so you don't need to configure things by hand. Azure DevOps Pipeline is one such tool for this:
https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/?toc=%2Fazure%2Fdevops%2Fpipelines%2Ftoc.json&bc=%2Fazure%2Fdevops%2Fboards%2Fpipelines%2Fbreadcrumb%2Ftoc.json&view=azure-devops
To get you started there is a specific deployment task which can aid in setting configuration for App Service Plans: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-rm-web-app-deployment?view=azure-devops
Over time I'd suggest rather than splitting configuration between a Release Pipeline and source control, instead having configuration which doesn't require deployment time modifications instead to stay source controlled in ARM templates, but that is an answer in it's own right so I won't confuse matters with too much detail.
If you want to access Azure key Vault by programming with SDK or REST API, then it would not be a problem. Because different key vaults have different DNS names. When you try to retrieve a secret or key in your code, you need to use its identify URL which contains its key vault DNS name. It will finally find the target key vault.
If you want to use key vault in web app with managed identity, you may refer to the tutorial: Use Azure Key Vault with an Azure web app in .NET
In that tutorial, you will enable the identity of a web app. And then you can assign access policy to that identity. In this way, the web app will be able to access the key vault with managed identity.
Let me give some details of my setup
I am building an asp.net core API app being hosted on Azure. I store my secret keys and stuff in azure keyvault. However, I have some AzureAddClientId and secret which is now stored in appsettings.json( to access key vault ). I have also committed appsettings.json to my git repo. However I know that is insecure. I use Azure DevOps for releases. So I'm thinking of doing the following. Please let me know your thoughts on this.
add appsettings.json to git ignore and share the file among developers.
add AzureAddClientId and AzureAADClientSecret to azure DevOps build pipeline as variables. ( Will devops automatically take the variables just as if they were in appsettings.json? )
Please have a look at using Managed Identities.
A common challenge when building cloud applications is how to manage the credentials in your code for authenticating to cloud services. Keeping the credentials secure is an important task. Ideally, the credentials never appear on developer workstations and aren't checked into source control. Azure Key Vault provides a way to securely store credentials, secrets, and other keys, but your code has to authenticate to Key Vault to retrieve them.
The managed identities for Azure resources feature in Azure Active Directory (Azure AD) solves this problem. The feature provides Azure services with an automatically managed identity in Azure AD. You can use the identity to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in your code.