How to access Azure Keyvault from docker container running locally? - azure

I have a docker image containing an ASP.NET Core app that uses Azure Key vault to access things like connection strings. When I run the image locally, I get this error:
Unhandled Exception: Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/[guid]. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/[guid]. Exception Message: Tried to get token using Managed Service Identity. Unable to connect to the Managed Service Identity (MSI) endpoint. Please check that you are running on an Azure resource that has MSI setup.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/[guid]. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Environment variable LOCALAPPDATA not set.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/[guid]. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. /bin/bash: az: No such file or directory
From what I understand, it first tries to get the access token as a managed service identity. As it's not running in the Azure cloud, it can't do this and tries to get it through visual studio connected service. As this won't be on the docker image, it tries using the Azure CLI, but this isn't installed on the docker image.
So I need to install the Azure CLI into the docker image. How is this done, given that the base image of the Dockerfile is FROM microsoft/dotnet:2.1-aspnetcore-runtime?
Is this base image an Alpine OS image, so do I need to look at installing Azure CLI with Alpine?
Assuming I have Azure CLI installed, is there a way to access Key vault without storing any credentials in Dockerfile source code or passing them to the container through plain text?
More generally, what is the best approach here.

My current solution is to use an environment variable with the access token.
Get the key and store in environment variable (after you did an az login and set the correct subscription):
$Env:ACCESS_TOKEN=(az account get-access-token --resource=https://vault.azure.net | ConvertFrom-Json).accessToken
The we add that environment variable in Visual Studio:
Change the code to:
config.AddEnvironmentVariables();
KeyVaultClient keyVaultClient;
var accessToken = Environment.GetEnvironmentVariable("ACCESS_TOKEN");
if (accessToken != null)
{
keyVaultClient = new KeyVaultClient(
async (string a, string r, string s) => accessToken);
}
else
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(
azureServiceTokenProvider.KeyVaultTokenCallback));
}
config.AddAzureKeyVault(
$"https://{builtConfig["KeyVaultName"]}.vault.azure.net/",
keyVaultClient,
new DefaultKeyVaultSecretManager());

Solution (not for production use)
A possible Solution to your Problem is to generate a Service Principal (SP) and grant this Service Principal access to the key vault (via RBAC or IAM). Microsoft Documentation on creating a SP
Using the credentials of the SP as client-id and client-secret (Random example) you can then log into the vault and retrieve the secrets.
Concerns
with this approach, you will introduce secrets into the code (propably the exact reason why you use the key vault). I suppose the local docker image is for development use only. Therefore I would recommend creating a Keyvault just for development (and access it using SP) while using a separate Kevault for Production where one of the established, secret-less authentication schemes is used.
You must make sure that the key vault allows access from outside the azure cloud (see the access policies on portal.azure.com)

In an attempt to simplify and automate E. Staal's answer, I came up with this:
Update your .gitignore file, by adding the following line to the bottom of it:
appsettings.local.json
Right click on the project in Solution Explorer, and click on Properties; in the Build Events tab, find the Pre-build event command line text box and add the following code:
cd /d "$(ProjectDir)"
if exist "appsettings.local.json" del "appsettings.local.json"
if "$(ConfigurationName)" == "Debug" (
az account get-access-token --resource=https://vault.azure.net > appsettings.local.json
)
In your launchSettings.json (or using the Visual Editor under project settings) configure the following values:
{
"profiles": {
// ...
"Docker": {
"commandName": "Docker",
"environmentVariables": {
"DOTNET_ENVIRONMENT": "Development",
"AZURE_TENANT_ID": "<YOUR-AZURE-TENANT-ID-HERE>"
}
}
}
}
In your Program.cs file find the CreateHostBuilder method and update the ConfigureAppConfiguration block accordingly -- here is mine as an example:
Host.CreateDefaultBuilder(args).ConfigureAppConfiguration
(
(ctx, cfg) =>
{
if (ctx.HostingEnvironment.IsDevelopment())
{
cfg.AddJsonFile("appsettings.local.json", true);
}
var builtConfig = cfg.Build();
var keyVault = builtConfig["KeyVault"];
if (!string.IsNullOrWhiteSpace(keyVault))
{
var accessToken = builtConfig["accessToken"];
cfg.AddAzureKeyVault
(
$"https://{keyVault}.vault.azure.net/",
new KeyVaultClient
(
string.IsNullOrWhiteSpace(accessToken)
? new KeyVaultClient.AuthenticationCallback
(
new AzureServiceTokenProvider().KeyVaultTokenCallback
)
: (x, y, z) => Task.FromResult(accessToken)
),
new DefaultKeyVaultSecretManager()
);
}
}
)
If this still doesn't work, verify that az login has been performed and that az account get-access-token --resource=https://vault.azure.net works correctly for you.

Although there's some time since you make this question, another option, suitable for production environments, would be using an x509 certificate.
Microsoft has this article explaining how to do this. You can use self-signed certificates or any other valid SSL certificate. That depends on your needs.

This is because your docker container is running as root user and the user registered in key vault is some other user (yourusername#yourcmpany.com)

Related

How to ask DefaultAzureCredential to use my user credential locally

I'm trying to develop a web app on an Azure VM that uses Azure Key Vault. Later this app will also be deployed to Azure. As far as I know, the most straight forward way to make the app work, both locally and deployed, with the key vault, is to use the DefaultAzureCredential class. The code would be like this:
string kvUri = "https://" + keyvaultName + ".vault.azure.net";
SecretClient client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
KeyVaultSecret secret = await client.GetSecretAsync(secretName);
At runtime, the provider will try different credential types in order.
This sounds exactly what I want:
When developing locally (on the Azure VM, though), I want to use my user credential (user identity added to the key vault's permission) without any configuration, since I have already logged into the Visual Studio using the same user credential.
Once deployed to Azure, I want to use the app registration credential (also added to the key vault's permission).
But when running the app locally, I'm getting a 403 error The user, group or application .... does not have secrets get permission on key vault ...
After looking up the object id in the error message, I realize it's the dev machine Azure VM's credential that the application uses, not my user credential.
Is there a way to change this behavior?
To prevent the Azure VM from getting a token, you can exclude the ManagedIdentityCredential in your Development environment and only enable it in a Non-Development environment.
if (environment.IsDevelopment())
{
var credentials = new DefaultAzureCredential(new DefaultAzureCredentialOptions
{
ExcludeManagedIdentityCredential = true,
ExcludeAzureCliCredential = true
});
}
else
{
var credentials = new DefaultAzureCredential(new DefaultAzureCredentialOptions
{
ExcludeVisualStudioCodeCredential = true,
ExcludeVisualStudioCredential = true
});
}
Once deployed to Azure, I want to use the app registration credential (also added to the key vault's permission).
An Azure App Service can use a managed identity as well. There is no need for a separate App Registration.
See https://learn.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme#key-concepts for more information.
Create and identity if you wish to use (default identity)
appservice -> select you application -> identity->enable it ->should give you a Id
and than add it to key Vault Access policy
alternatively app registration can be used with tenantId,clientId,secret to connect to keyvault

Azure Key Vault: The user, group, or app does not have secrets set permission on key vault

I am creating a script using Azure CLI that will automatically generate an App Registration (service principal), and then use that App Registration to create a secret that will be stored in Azure Key Vault.
However, I am getting the following error:
The user, group or application 'appid=04b07795-8ddb-461a-bbee-02f9e1bf7b46;oid=0ec2b0e8-daeb-46a8-b627-0d4f61f87157;numgroups=134;iss=https://sts.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47/' does not have secrets set permission on key vault 'asakeyabcfelaqpgsfnxcy;location=eastus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
Can anyone provide guidance on what this ID is and how to resolve this error? This is not my App Registration Object ID or App ID.
I think there're 2 points you're now concerning, one is you failed to add secret, another is the app id in the error message is not the one you registered.
I think you've followed the document to execute the cli command, so I just want to explain the second point. Pls allow me show you a code sample for a better explanation.
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
public async Task<IActionResult> PrivacyAsync()
{
var kvUri = "https://your_vault_name.vault.azure.net/";
var client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
_ = await client.SetSecretAsync("test0311", "hello");
return View();
}
When we want to add key vault secret to azure, we need to provide a credential so that our operations are authenticated. This is the DefaultAzureCredential() here, and it has several sources to get the authentication like screenshot below.
That means if someone sets the environment variables for authentication, then it will cover the information you entered when executing cli command, this may usually cause the issue that the app is different from what you set. I think you may follow this document to check all your configurations and try again, or you can directly add environment variables with the app you registered on your computer.
By the way, pls don't forget to add access policy in azure portal for the azure ad app you registered.

How to use Azure Key Vault in npmrc file?

I have a secret personal access token (only for building purposes) in my .npmrc file. As this secret is exposed, I thought of replacing this using Azure Key Vault. I haven't found any documentation around it. When I created the personal token before, I had given it only packaging/building access. How can I achieve this, please help me with this? Or is there any better way to include the personal access token in the .npmrc file?
Since you confirmed you are using Azure DevOps for your build, you don't need to maintain PAT in the .npmrc file. Just keep your npm registry URL there (I assume the private npm registry is also in the Azure DevOps) like below:
registry={your npm registry URL}
always-auth=false
Now, in the build pipeline, add npm Authenticate task before npm install.
- task: npmAuthenticate#0
inputs:
workingFile: <relative path to your .npmrc file>
Providing secrets to your resource can be done in many ways.
Some resources in Azure allow you to specify environment variables through the Azure CLI. Here's an example with the Azure container instances: link.
On Azure, once you have a Key Vault instance, you can use your Key Vault to provide secrets to your App Service and Azure Function instances. This is documented here: link, with a focus for Azure Resource Manager templates, which is specially useful for automated deployments.
Although the following is explained in the documentation link above, the general picture on how to use Key Vault secrets from other Azure resources requires the following:
Make a user assigned identity or Azure Active Directory application.
Grant access to this identity (or AAD app) by going to the Access Policies of your Key Vault (this can be done through the portal, of course), and giving your identity at least read access to your Key Vault.
After that, create a secret on your Key Vault, go to the secret details and copy the "Secret Identifier". This will be a URI similar to: https://myvault.vault.azure.net/secrets/mysecret/.
That's the URI you can use to bring Key Vault secrets to other resources.
You'll be able to access this secret from other resources by ensuring the resource has access to the same identity, and by providing the URI through a syntax similar to: #Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/).
For example, if you link an Azure Function to the same identity you granted read access to your Key Vault, you can provide a secret through environment variables by setting configuration properties in your resource. By going to the Azure Portal, locating your resource, then going to Configuration, then to Application settings, if you proceed to add the name of your environment variable, and as the value something similar to: #Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/), you'll be providing the expected environment variable with the expected secret value to your resource.
The final approach I can think of is by using the #azure/keyvault-secrets client. If using an NPM library to retrieve Key Vault secrets sounds interesting, this is the dependency for you. All the information needed to work with this library should be available on NPM: same link. But in any case, a sample using this client would look as follows:
const { DefaultAzureCredential } = require("#azure/identity");
const { SecretClient } = require("#azure/keyvault-secrets");
const credential = new DefaultAzureCredential();
const client = new SecretClient(`https://my-key-vault.vault.azure.net`, credential);
async function main() {
const secretName = "MySecretName";
const latestSecret = await client.getSecret(secretName);
console.log(`Latest version of the secret ${secretName}: `, latestSecret);
}
main();
You could use this library to load your secrets at any point while your service or program is running.
Please let me know if this information is useful for you. I'm here to help!

Exception while connecting to KeyVault from Azure VM

I am running my applictaion from Azure VM and trying to connect with KeyVault. But I am getting below exception
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/1e465dc8-5f36-4ab9-9a49-57cbfdcfdf9a. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/1e465dc8-5f36-4ab9-9a49-57cbfdcfdf9a. Exception Message: Tried to get token using Managed Service Identity. Unable to connect to the Managed Service Identity (MSI) endpoint. Please check that you are running on an Azure resource that has MSI setup.
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/1e465dc8-5f36-4ab9-9a49-57cbfdcfdf9a. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired.
Exception for Visual Studio token provider Microsoft.Asal.TokenService.exe : TS003: Error, TS001: This account 'username' needs re-authentication. Please go to Tools->Azure Services Authentication, and re-authenticate the account you want to use.
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/1e465dc8-5f36-4ab9-9a49-57cbfdcfdf9a. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. 'az' is not recognized as an internal or external command,
operable program or batch file.
I have checked the prerequisite such as -
1. created the KeyVault in the same resource group of the VM and added 2 secrets.
2. checked that the VM is registered in Active Directory and that it has a system assigned identity.
3. added access policy allowing read and list secrets to the VM.
Here is the code, What I am missing
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
var secret = keyVaultClient.GetSecretAsync($"https://vaultname.vault.azure.net/Secrets/connString").Result.Value;
Errors are indicating authentication issue, so 2 things to validate in order;
Confirm the VM can query Azure Metadata service
Invoke-RestMethod -Headers #{"Metadata"="true"} -URI "http://169.254.169.254/metadata/instance/compute/vmId?api-version=2017-08-01&format=text" -Method get`
If above query is successful then check the Identity API on the metadata service but if it fails then there is a communication issue between VM and Azure environment.
Confirm the VM can query the Identity API of Azure Metadata service
Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F' -Headers #{Metadata="true"}
If above query is successful then there is nothing wrong with MSI.
The problem was with the nuget version on Microsoft.Azure.Services.AppAuthentication. Version 1.0.3 solves this.
I'm using nuget package Azure.Identity version 1.3 and got the same issue TS003, TS001, so I tried to downgrade version to 1.2.2 and it works
In my case it was visual studio authentication issue, if your password has expired since you connected Azure stuff form visual studio, you need to re authenticate.

.NetCore 2.2 API fails to get token from AAD when using User Assigned Identity

We are unable to query a sql database in azure from an Azure App Service when using a user assigned managed identity (it works fine if we use a system assigned managed identity)
The application is a .net core 2.2 web api application.
We have a user assigned identity set up for an Azure App Service.
This identity has been set up as the ad sql admin by using the following command:
az sql server ad-admin create --resource-group iactests --server iactestsql --object-id -u iactestmanagedIdentity
The token is generated like this:
services.AddDbContext<SchoolContext>(options => options.UseSqlServer(new
SqlConnection
{
ConnectionString = configuration.GetConnectionString("SchoolContext"),
AccessToken = isDevelopmentEnvironment ? null : new AzureServiceTokenProvider().GetAccessTokenAsync("https://database.windows.net/").Result
}), ServiceLifetime.Scoped);
This is the error we get:
Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. MSI ResponseCode: BadRequest, Response:
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Visual Studio Token provider file not found at "D:\local\LocalAppData\.IdentityService\AzureServiceAuth\tokenprovider.json"
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. 'az' is not recognized as an internal or external command,
operable program or batch file.
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String authority, String resource, String scope)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthenticationResultAsync(String resource, String tenantId)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAccessTokenAsync(String resource, String tenantId)
--- End of inner exception stack trace ---
If we use a system assign identity and configure the sql ad admin to be said identity, it works fine
Any ideas?
Thanks in advance
The AppAuthentication library now supports specifying user-assigned identities for Azure VMs and App Services as of the 1.2.0-preview2 release.
To use a user-assigned identity, you will need to set an AppAuthentication connection string of the format:
RunAs=App;AppId={ClientId of user-assigned identity}
The AppAuthentication connection string can be set as an argument passed to the AzureServiceTokenProvider constructor or specified in the AzureServicesAuthConnectionString environment variable. For more information on AppAuthentication connection strings, see here.
It looks like AzureServiceTokenProvider does not support user assigned managed identities, at least at this point. AzureServiceTokenProvder is a wrapper over the local HTTP endpoint that provides tokens to the application.
I was looking into this, and appears that you must provide the clientId of the user assigned managed identity to the endpoint to get a token. And AzureServiceTokenProvider doesn't have a way to do that (at least that I could figure out).
User assigned managed identities adds the ability to have multiple User assigned managed identities for an application. So the API to get a token needs to specify which MSI you want, the system-assigned MSI, or one of the user-assigned MSIs. The way the HTTP endpoint does this is that it uses the system-assigned MSI unless you specify a clientId.
In any case, you can hit the token endpoint directly, and provide the clientId of the user-assigned MSI like this:
public async Task<String> GetToken(string resource, string clientId = null)
{
var endpoint = System.Environment.GetEnvironmentVariable("MSI_ENDPOINT", EnvironmentVariableTarget.Process);
var secret = System.Environment.GetEnvironmentVariable("MSI_SECRET", EnvironmentVariableTarget.Process);
if (string.IsNullOrEmpty(endpoint))
{
throw new InvalidOperationException("MSI_ENDPOINT environment variable not set");
}
if (string.IsNullOrEmpty(secret))
{
throw new InvalidOperationException("MSI_SECRET environment variable not set");
}
Uri uri;
if (clientId == null)
{
uri = new Uri($"{endpoint}?resource={resource}&api-version=2017-09-01");
}
else
{
uri = new Uri($"{endpoint}?resource={resource}&api-version=2017-09-01&clientid={clientId}");
}
// get token from MSI
var tokenRequest = new HttpRequestMessage()
{
RequestUri = uri,
Method = HttpMethod.Get
};
tokenRequest.Headers.Add("secret", secret);
var httpClient = new HttpClient();
var response = await httpClient.SendAsync(tokenRequest);
var body = await response.Content.ReadAsStringAsync();
var result = JObject.Parse(body);
string token = result["access_token"].ToString();
return token;
}

Resources