I need a way to store hierarchical data in Azure Key Vaults so that I have a structure similar to:
AppName
/Prod
/Data
/Test
/Data
AppName2
/Prod
/Data
...
As far as I can tell I can only store a flat data structure. I am looking to be able to store data similar to Vault by HashiCorp which allows hierarchies.
For instance, in Vault by HashiCorp, I can get data using a 'path': "app/test/TestConnection" and I get the value at the endpoint of the path: TestConnection.
Any suggestion for alternatives would be fine or instruction on how to do what I need to do with Key Vault.
Thanks
Update
I tried some of the suggestions: MySettings--SomeSection--SecretThing, Multiple Vaults and neither works in the manner I need as described above. Not faulting the input but what I want to do just is not available in Key Vault.
#juunas Turns out that your suggestion may be the best solution. I only just discovered in another article that MySettings--SomeSection--Secret translates into something similar in .NET Core:
MySettings: {
SomeSection: "Secret"
}
Since my client wants to use Key Vault we are probably going to go with storing json structured data per a single secret per application.
Any other suggestions are welcome
Key Vault does not support hierarchies for secrets.
To emulate structure, you can do something similar what .NET Core does with its Key Vault configuration provider. You can specify a secret with a name like Settings--SomeCategory--SomeValue, and it'll correspond to the following JSON when loaded:
{
"Settings": {
"SomeCategory": {
"SomeValue": "value goes here"
}
}
}
So essentially you can use a separator to emulate the structure, similar also to how Azure Blob Storage emulates folders.
I would advice against mixing different environment secrets within the same key vault. Access cannot be restricted to some keys, as access is granted and denied on the Key Vault level only. You probably don't want the same persons/applications to be able to access all the different environments, but instead grant access to the production environment to a selected group of users and applications only, and vice versa.
As the Key Vault service by itself doesn't really cost anything, we at least have taken the approach to create one Key Vault per environment, i.e. dev, test and production. Within that key vault the secrets are "structured" by a prefix, i.e. AppName-Data and AppName2-Data. This gives the added benefit, that when moving from dev to test and to production, the references to the secrets don't need to be changed, as they have the same name in all the environments. Just the reference to the Key Vault needs to be changed, and all is set!
Related
In Azure you can setup an App Config and a KeyVault. The point of the KeyVault being to store more sensitive data than your App Config and be able to regulate access to the config and vault separately.
So what is the benefit of using a keyvault reference in the app config?
You are basically allowing anyone with access to the app config to access certain values in your keyvault and are bypassing the additional layer of security the vault normally provides.
The additional layer being required auth to the vault to access those same values if they aren't referenced in the config.
I really don't understand what benefit keyvault references give you.
This blog article by Jan de Vries explains them in more detail: https://jan-v.nl/post/2021/using-key-vault-with-azure-app-configuration/.
The relevant part for your question:
As it happens, the code for accessing App Configuration doesn’t give your application permission to retrieve secrets from Key Vault.
The application retrieves them from Key Vault, not from App Configuration.
App Config only holds the reference, not the actual value.
Official docs also mention this:
Your application uses the App Configuration client provider to retrieve Key Vault references, just as it does for any other keys stored in App Configuration. In this case, the values stored in App Configuration are URIs that reference the values in the Key Vault. They are not Key Vault values or credentials. Because the client provider recognizes the keys as Key Vault references, it uses Key Vault to retrieve their values.
Your application is responsible for authenticating properly to both App Configuration and Key Vault. The two services don't communicate directly.
I suppose there are different approaches to using the KeyVault, but the way I tend to use it is as follows.
My application will have a set of secrets, which I store locally using the Secrets Manager, you would add the secret for your application:
dotnet user-secrets set "Movies:ServiceApiKey" "12345"
Your application can then read this setting using _moviesApiKey = Configuration["Movies:ServiceApiKey"]; as you'll see in the link above. Obviously, there's no way you can see this value in the code, but your application can read it from the Secrets Manager.
If you do forget the values, you can use the following command to retrieve them:
dotnet user-secrets list
KeyVault will work as your Secrets Manager within Azure. So, your application will need to have permission to access the KeyVault, and in my case I store the Vault name in the appsettings.json, and during the bootstrapping, I include the KeyVault configuration if running in Production mode i.e. on the Azure Server and not locally.
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.AddConsole();
logging.AddAzureWebAppDiagnostics();
})
.ConfigureAppConfiguration((context, config) =>
{
if (context.HostingEnvironment.IsProduction())
{
IConfigurationRoot builtConfig = config.Build();
ConfigurationBuilder keyVaultConfigBuilder = new ConfigurationBuilder();
keyVaultConfigBuilder.AddAzureKeyVault(builtConfig["VaultName"]);
IConfigurationRoot keyVaultConfig = keyVaultConfigBuilder.Build();
config.AddConfiguration(keyVaultConfig);
}
})
.UseStartup<Startup>();
Note, the check for context.HostingEnvironment.IsProduction(). Within the appsettings, I have:
"VaultName": "https://yourkvname.vault.azure.net/"
So, the only reference I have to the KeyVault from the application is the name, and that should be secure as only the application will have access to the keys/secrets.
One thing to note, you need to make sure that the names match both for your local secrets and the ones in the KeyVault. In my case, I am running on a Windows platform, so I needed to make a small change to the names using double dashes (--) in place of the colon (:), so...
Movies:ServiceApiKey
Becomes
Movies--ServiceApiKey
When working in Azure, storing secrets in Key Vault is a good idea. And to make it better, there’s the Key Vault Reference notation. This feature makes sure no one can read the secret(s) unless someone grants permission.
Speaking of secrets, they should never be directly stored in application settings of a Function App (same goes for App Services by the way). Why not ? Because secrets would be available to anyone who has access to the Function App in the Azure Portal. The right way is to use an Azure Key Vault which is the Azure component for securely storing and accessing secrets 🔒. Once your secrets are in the key vault, you have to grant the Key Vault access to the identity of your Function App and you can then reference the secrets you need directly in your application settings. These are called Key Vault references because an application setting does not contain directly the value of a secret but a reference to the secret which is stored in Key Vault. When running, your function will automatically have access to the secret and its value as an environment variable, as if it was a normal application setting.
Key Vault references work for both App Services and Function Apps and are particularly useful for existing applications that have their secrets stored in settings because securing the secrets with Azure Key Vault references does not require any code change.
Reference: https://www.techwatching.dev/posts/azure-functions-custom-configuration
https://www.sharepointeurope.com/using-key-vault-references-with-azure-app-configuration/
I am trying store secrets in azure vault. I used azure sdk apis and I can successfully store/retrieve using those. I wanted to know if it's possible to categorise/group set of secrets under same tag and store them in some path.
I want to group some secrets used by one service, store them in one storage path. The same way for other services is separate storage paths. I couldn't find any way of doing that. Is that possible in azure vault?
In short: no, this is not possible.
Also: you cannot get secrets and their values in a list. If you want to get a list, you'll only get a SecretItem array and you have to call GetSecret on each secret you want to get the actual value for.
You could, however, implement something like this yourself by defining a template for the name of the secret that incorporates the name of the thing you would like to group on. Something like this:
$"{serviceName}-secrets-{secretName}"
This way, you can filter the list to only hold the secrets for the service you want to get them for and get their values.
While developing an ARM template to deploy multiple app services and want to use the key vault to host multiple secret, and we need to include 110 references in the template.
While doing a POC, there is an error message:
The deployment has specified too many Key Vault parameter references.
The maximum of Key Vault parameter references is '30'
Is it possible to go beyond 30?
Your first option should be to consider why you need so many secrets and how you can come up with a better strategy.
Failing that, on the quick and dirty, you can wrap multiple secrets in the a string.
E.g.:
MySecret: value1|value2|value3|value4|value5
They you can use the ARM Template functions to pull out the value you need.
Look in to the array and split functions.
Hopefully you can keep related secrets together and organise them neatly.
I have a pod that runs containers which require access to sensitive information like API keys and DB passwords. Right now, these sensitive values are embedded in the controller definitions like so:
env:
- name: DB_PASSWORD
value: password
which are then available inside the Docker container as the $DB_PASSWORD environment variable. All fairly easy.
But reading their documentation on Secrets, they explicitly say that putting sensitive configuration values into your definition breaches best practice and is potentially a security issue. The only other strategy I can think of is the following:
create an OpenPGP key per user community or namespace
use crypt to set the configuration value into etcd (which is encrypted using the private key)
create a kubernetes secret containing the private key, like so
associate that secret with the container (meaning that the private key will be accessible as a volume mount), like so
when the container is launched, it will access the file inside the volume mount for the private key, and use it to decrypt the conf values returned from etcd
this can then be incorporated into confd, which populates local files according to a template definition (such as Apache or WordPress config files)
This seems fairly complicated, but more secure and flexible, since the values will no longer be static and stored in plaintext.
So my question, and I know it's not an entirely objective one, is whether this is completely necessary or not? Only admins will be able to view and execute the RC definitions in the first place; so if somebody's breached the kubernetes master, you have other problems to worry about. The only benefit I see is that there's no danger of secrets being committed to the filesystem in plaintext...
Are there any other ways to populate Docker containers with secret information in a secure way?
Unless you have many megabytes of config, this system sounds unnecessarily complex. The intended usage is for you to just put each config into a secret, and the pods needing the config can mount that secret as a volume.
You can then use any of a variety of mechanisms to pass that config to your task, e.g. if it's environment variables source secret/config.sh; ./mybinary is a simple way.
I don't think you gain any extra security by storing a private key as a secret.
I would personally resolve to user a remote keymanager that your software could access across the net over a HTTPS connection. For example Keywhiz or Vault would probably fit the bill.
I would host the keymanager on a separate isolated subnet, and configure firewall to only allow access to ip addresses which I expected to need the keys. Both KeyWhiz and Vault comes with an ACL mechanism, so you may not have to do anything with firewalls at all, but it does not hurt to consider it -- however the key here is to host the keymanager on a separate network, and possible even a separate hosting provider.
You local configuration file in the container would contain just the URL of the key service, and possible a credentials to retrieve the key from the keymanager -- the credentials would be useless to an attacker if he didn't match the ACL/IP addresses.
I am starting with Windows Azure. I have an Azure account with Microsoft and would like to use it from my Visual Studio project
In the Azure management portal, I can see the primary access key and secondary access key. However Visual Studio needs the account name and account key.
Are these related or am I missing something? If so, how can I get my account name and account key from the Azure management portal?
If you're referring to a Windows Azure Storage account, the storage account name would be the dns prefix you created (e.g. mystorage.blob.core.windows.net - the name would be mystorage).
You then get two keys - primary and secondary. Specify either. You have two so you can give one out to someone (such as giving the secondary key to a 3rd-party monitoring company) and revoke it by changing the key, without impacting you (assuming you're using the primary key for yourself).
The user interface changed a little bit but it's generally the same place:
Note that on the right side of the key you have a Connection String. There you can clearly see the name and the key as well.
Adding a new answer as Azure UI is updated.
So, after selecting the storage account(in my case its inAFlash) select Access keys option you will get the following screen shown in the screenshot
Here you see two key's Primary and secondary.
So, previously these were key1 and key2.
Reason for two keys?
it's to avoid downtime. let's say, if you had only one key and used it multiple places. Suddenly you had to share it to someone for some reason. So, you might need to regenerated this key(As you have already shared it) and update it in all places where you used it previously. So, to avoid this, we have secondary key, which you can share and regenerate it so that, primary key will be still valid.
You can check it by using Azure CLI:
az login
az storage account list -o table
az storage account keys list -n YourAccount
To extract the first key, you can use the following command:
az storage account keys list -n YourAccount -o json --query "[0].value"