I'm using the Hashicorp vault to store the application configuration in kv secrets. I need to perform an action based on the values getting change in the kv secrets. Is there is any way to monitor the field change or at least monitor any update to the kv secret?
Related
In Azure you can setup an App Config and a KeyVault. The point of the KeyVault being to store more sensitive data than your App Config and be able to regulate access to the config and vault separately.
So what is the benefit of using a keyvault reference in the app config?
You are basically allowing anyone with access to the app config to access certain values in your keyvault and are bypassing the additional layer of security the vault normally provides.
The additional layer being required auth to the vault to access those same values if they aren't referenced in the config.
I really don't understand what benefit keyvault references give you.
This blog article by Jan de Vries explains them in more detail: https://jan-v.nl/post/2021/using-key-vault-with-azure-app-configuration/.
The relevant part for your question:
As it happens, the code for accessing App Configuration doesn’t give your application permission to retrieve secrets from Key Vault.
The application retrieves them from Key Vault, not from App Configuration.
App Config only holds the reference, not the actual value.
Official docs also mention this:
Your application uses the App Configuration client provider to retrieve Key Vault references, just as it does for any other keys stored in App Configuration. In this case, the values stored in App Configuration are URIs that reference the values in the Key Vault. They are not Key Vault values or credentials. Because the client provider recognizes the keys as Key Vault references, it uses Key Vault to retrieve their values.
Your application is responsible for authenticating properly to both App Configuration and Key Vault. The two services don't communicate directly.
I suppose there are different approaches to using the KeyVault, but the way I tend to use it is as follows.
My application will have a set of secrets, which I store locally using the Secrets Manager, you would add the secret for your application:
dotnet user-secrets set "Movies:ServiceApiKey" "12345"
Your application can then read this setting using _moviesApiKey = Configuration["Movies:ServiceApiKey"]; as you'll see in the link above. Obviously, there's no way you can see this value in the code, but your application can read it from the Secrets Manager.
If you do forget the values, you can use the following command to retrieve them:
dotnet user-secrets list
KeyVault will work as your Secrets Manager within Azure. So, your application will need to have permission to access the KeyVault, and in my case I store the Vault name in the appsettings.json, and during the bootstrapping, I include the KeyVault configuration if running in Production mode i.e. on the Azure Server and not locally.
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.AddConsole();
logging.AddAzureWebAppDiagnostics();
})
.ConfigureAppConfiguration((context, config) =>
{
if (context.HostingEnvironment.IsProduction())
{
IConfigurationRoot builtConfig = config.Build();
ConfigurationBuilder keyVaultConfigBuilder = new ConfigurationBuilder();
keyVaultConfigBuilder.AddAzureKeyVault(builtConfig["VaultName"]);
IConfigurationRoot keyVaultConfig = keyVaultConfigBuilder.Build();
config.AddConfiguration(keyVaultConfig);
}
})
.UseStartup<Startup>();
Note, the check for context.HostingEnvironment.IsProduction(). Within the appsettings, I have:
"VaultName": "https://yourkvname.vault.azure.net/"
So, the only reference I have to the KeyVault from the application is the name, and that should be secure as only the application will have access to the keys/secrets.
One thing to note, you need to make sure that the names match both for your local secrets and the ones in the KeyVault. In my case, I am running on a Windows platform, so I needed to make a small change to the names using double dashes (--) in place of the colon (:), so...
Movies:ServiceApiKey
Becomes
Movies--ServiceApiKey
When working in Azure, storing secrets in Key Vault is a good idea. And to make it better, there’s the Key Vault Reference notation. This feature makes sure no one can read the secret(s) unless someone grants permission.
Speaking of secrets, they should never be directly stored in application settings of a Function App (same goes for App Services by the way). Why not ? Because secrets would be available to anyone who has access to the Function App in the Azure Portal. The right way is to use an Azure Key Vault which is the Azure component for securely storing and accessing secrets 🔒. Once your secrets are in the key vault, you have to grant the Key Vault access to the identity of your Function App and you can then reference the secrets you need directly in your application settings. These are called Key Vault references because an application setting does not contain directly the value of a secret but a reference to the secret which is stored in Key Vault. When running, your function will automatically have access to the secret and its value as an environment variable, as if it was a normal application setting.
Key Vault references work for both App Services and Function Apps and are particularly useful for existing applications that have their secrets stored in settings because securing the secrets with Azure Key Vault references does not require any code change.
Reference: https://www.techwatching.dev/posts/azure-functions-custom-configuration
https://www.sharepointeurope.com/using-key-vault-references-with-azure-app-configuration/
I'm attempting to generate 2 azurerm_key_vault_key: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/key_vault_key
I need both of them to have the same value(to be the same key exactly but in different keyvault). Is it possible to achieve that? I can't find anyway to explicitly define the key's value so I could generate it beforehand. Is that possible to have 2 keys like that?
You need to have 2 Keys in different Azure Key Vaults with same values.
Unfortunately it is not possible to import Key in Azure Key Vault with terraform, but you can use REST call to import Key into Azure Key Vault.
You can generate Keys beforehand and import it into both Azure KeyVaults.
https://learn.microsoft.com/en-us/rest/api/keyvault/importkey/importkey
I created Key Import feature request in Azure Terraform provider, please vote.
https://github.com/terraform-providers/terraform-provider-azurerm/issues/10301
We deploy a Linux App Service to Azure using terraform. The relevant configuration code is:
resource "azurerm_app_service" "webapp" {
app_settings = {
DOCKER_REGISTRY_SERVER_URL = "https://${local.ctx.AcrName}.azurecr.io"
DOCKER_REGISTRY_SERVER_USERNAME = data.azurerm_key_vault_secret.acr_admin_user.value
DOCKER_REGISTRY_SERVER_PASSWORD = data.azurerm_key_vault_secret.acr_admin_password.value
...
}
...
}
The problem is that terraform does not consider app_settings a secret and so it outputs in the clear the DOCKER_REGISTRY_SERVER_PASSWORD value in the Azure DevOps output (I obfuscated the actual values):
So, I am wondering - can docker running on an Azure Linux App Service host authenticate with the respective ACR without us having to pass the password in a way that makes it so obvious to every one who can inspect the pipeline output?
The following article seems relevant in general - https://docs.docker.com/engine/reference/commandline/login, but it is unclear how we can apply it in my context, if at all.
Also, according to https://feedback.azure.com/forums/169385-web-apps/suggestions/36145444-web-app-for-containers-acr-access-requires-admin#%7Btoggle_previous_statuses%7D Microsoft has started working on something relevant, but looks like this is still a work in progress (almost 5 months).
I'm afraid you must set the environment variables about DOCKER_REGISTRY_* to pull the images from the ACR, it's the only way to do that designed by Azure. But for the sensitive info about the password, it also provides a way to hide it. You can use the Key Vault to store the password in secret, and then get the password from the secret. Take a look at the document Use Key Vault references for App Service. So you can change the app_setting for the password like this:
DOCKER_REGISTRY_SERVER_PASSWORD = "#Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931)"
Or
DOCKER_REGISTRY_SERVER_PASSWORD = "#Microsoft.KeyVault(VaultName=myvault;SecretName=mysecret;SecretVersion=ec96f02080254f109c51a1f14cdb1931)"
Then it just shows the reference of the Key Vault, not the exact password.
Unfortunately Azure Web Apps do not support interacting with ACR using a managed identity, you must pass those Environment Variables to the App Service.
Terraform does not currently support applying a "sensitive" flag to arbitrary values. You can define outputs as sensitive, but it will not help with values you want to hide during the plan phase.
I would suggest checking out https://github.com/cloudposse/tfmask, using the TFMASK_RESOURCES_REGEX configuration to block the output you want to hide during your pipeline. If you're averse to adding dependencies, similar effect could be achieved by piping terraform apply through grep --invert-match "DOCKER_REGISTRY" instead.
#charles-xu has a good answer as well if you want to set up mappings between keyvault and your web app then push your tokens into kv secrets.
Now it's possible to use managed identity to pull images from ACR.
You may do the next:
go to your Container Registry page in the Azure portal
Open the tab Access Control (IAM)
The open Role assignments tab
Add role assignment AcrPull to your App Service or Function App
In the Deployment Center of your App Service choose Managed Identity for the Authentication setting.
Or you may use CLI by following the steps from the official documentation (link below):
https://learn.microsoft.com/en-us/azure/app-service/configure-custom-container?pivots=container-linux#use-managed-identity-to-pull-image-from-azure-container-registry
After you added role assignment DOCKER_REGISTRY_SERVER_URL, DOCKER_REGISTRY_SERVER_USERNAME and DOCKER_REGISTRY_SERVER_PASSWORD settings may be removed from App Service's App Settings.
I am trying store secrets in azure vault. I used azure sdk apis and I can successfully store/retrieve using those. I wanted to know if it's possible to categorise/group set of secrets under same tag and store them in some path.
I want to group some secrets used by one service, store them in one storage path. The same way for other services is separate storage paths. I couldn't find any way of doing that. Is that possible in azure vault?
In short: no, this is not possible.
Also: you cannot get secrets and their values in a list. If you want to get a list, you'll only get a SecretItem array and you have to call GetSecret on each secret you want to get the actual value for.
You could, however, implement something like this yourself by defining a template for the name of the secret that incorporates the name of the thing you would like to group on. Something like this:
$"{serviceName}-secrets-{secretName}"
This way, you can filter the list to only hold the secrets for the service you want to get them for and get their values.
I need a way to store hierarchical data in Azure Key Vaults so that I have a structure similar to:
AppName
/Prod
/Data
/Test
/Data
AppName2
/Prod
/Data
...
As far as I can tell I can only store a flat data structure. I am looking to be able to store data similar to Vault by HashiCorp which allows hierarchies.
For instance, in Vault by HashiCorp, I can get data using a 'path': "app/test/TestConnection" and I get the value at the endpoint of the path: TestConnection.
Any suggestion for alternatives would be fine or instruction on how to do what I need to do with Key Vault.
Thanks
Update
I tried some of the suggestions: MySettings--SomeSection--SecretThing, Multiple Vaults and neither works in the manner I need as described above. Not faulting the input but what I want to do just is not available in Key Vault.
#juunas Turns out that your suggestion may be the best solution. I only just discovered in another article that MySettings--SomeSection--Secret translates into something similar in .NET Core:
MySettings: {
SomeSection: "Secret"
}
Since my client wants to use Key Vault we are probably going to go with storing json structured data per a single secret per application.
Any other suggestions are welcome
Key Vault does not support hierarchies for secrets.
To emulate structure, you can do something similar what .NET Core does with its Key Vault configuration provider. You can specify a secret with a name like Settings--SomeCategory--SomeValue, and it'll correspond to the following JSON when loaded:
{
"Settings": {
"SomeCategory": {
"SomeValue": "value goes here"
}
}
}
So essentially you can use a separator to emulate the structure, similar also to how Azure Blob Storage emulates folders.
I would advice against mixing different environment secrets within the same key vault. Access cannot be restricted to some keys, as access is granted and denied on the Key Vault level only. You probably don't want the same persons/applications to be able to access all the different environments, but instead grant access to the production environment to a selected group of users and applications only, and vice versa.
As the Key Vault service by itself doesn't really cost anything, we at least have taken the approach to create one Key Vault per environment, i.e. dev, test and production. Within that key vault the secrets are "structured" by a prefix, i.e. AppName-Data and AppName2-Data. This gives the added benefit, that when moving from dev to test and to production, the references to the secrets don't need to be changed, as they have the same name in all the environments. Just the reference to the Key Vault needs to be changed, and all is set!