We're trying to access a secret from Azure keyvault.
The code tries to get an access token that will then be used to retrieve the secret from the keyvault.
private async Task<string> GetAccessToken(string authority, string resource, string scope)
{
...
return this.useGlobalMsiRunAs ?
await tokenProviderGlobal.Value.KeyVaultTokenCallback(authority, resource, scope) :
await tokenProvider.Value.KeyVaultTokenCallback(authority, resource, scope);
}
useGlobalMsiRunAs is set to false and the code calls tokenProvider.Value.KeyVaultTokenCallback(...).
The above call fails with Access token could not be acquired. The operation was canceled
---> Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net//xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. The operation was canceled.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net//xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Environment variable LOCALAPPDATA not set.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net//xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired.
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String resource, String authority, CancellationToken cancellationToken)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.<get_KeyVaultTokenCallback>b__12_0(String authority, String resource, String scope)
at abc.xyz.Common.Service.AzureKeyVaultAccess.GetAccessToken(String authority, String resource, String scope) in d:\dbs\el\manb\private\ClientCenter\MT\Source\Common\ClientCenter.Common.NetStandard\Service\AzureKeyVaultAccess.cs:line 472
at Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage response)
at Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretWithHttpMessagesAsync(String vaultBaseUrl, String secretName, String secretVersion, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretAsync(IKeyVaultClient operations, String vaultBaseUrl, String secretName, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
We are using a managed identity, so the answer in this question didn't help.
Can someone please give some leads as to what could be wrong here?
You can use either system-assigned or user-assigned managed identity for your AKS Cluster’s Agent pool
Once you assign the system-assigned or user-assigned managed identity for your AKS Cluster’s Agent pool, you can add the access policy in your key vault with read access to the secrets
Then you would be able to access Azure Key vault secrets from your AKS cluster
Your code can use a managed identity to request access tokens for services that support Azure AD authentication. Azure takes care of rolling the credentials that are used by the service instance
To access key vault using system-assigned managed identity, you can use DefaultAzureCredential() class
If you are using user-assigned managed identity, you can use ManagedIdentityCredential() class
Reference: c# - How to use user-assigned managed identity to access Key Vault for Function App Config in Azure - Stack Overflow
Related
I have implemented Teams bot and authenticating it using managed identity.But getting below error:
Tried to get token using Managed Service Identity. Access token could not be acquired. Received a non-retryable error. MSI ResponseCode: BadRequest, Response: {"error":"invalid_request","error_description":"Identity not found"}
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String resource, String authority, Boolean forceRefresh, CancellationToken cancellationToken)
at Microsoft.Bot.Connector.Authentication.ManagedIdentityAuthenticator.AcquireTokenAsync(Boolean forceRefresh)
at Microsoft.Bot.Connector.Authentication.Retry.Run[TResult](Func`1 task, Func`3 retryExceptionHandler)
I have created bot managed identity and using that identity while creating azure bot.I have bot code deployed on AKS(kubernetes) cluster.And added owner role to the managed identity of bot in AKS.
Based on documentation, connector service will take care of generating token based on values provided in appsettings.deve.json
What is the possible issue here?
I'm facing issue "HTTP Error 502.5 - ANCM Out-Of-Process Startup Failure" while deploying web api to azure app service. Actually I have 2 deployment slots Prod and Staging.
Earlier we deployed web api to prod slot and it is working fine.
Now we created staging slot and deployed same webapi to staging slot.
In staging slot we are getting issue.
When I check eventlog.xml in kudu, I found below errors.
Description: The process was terminated due to an unhandled
exception. Exception Info:
Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException:
Parameters: Connection String: [No connection string specified],
Resource: https://vault.azure.net, Exception Message: Tried the
following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified],
Resource: https://vault.azure.net, Exception Message: Tried to get
token using Managed Service Identity. Unable to connect to the Managed
Service Identity (MSI) endpoint. Please check that you are running on
an Azure resource that has MSI setup. Parameters: Connection String:
[No connection string specified], Resource: https://vault.azure.net,
Exception Message: Tried to get token using Visual Studio. Access
token could not be acquired. Visual Studio Token provider file not
found at
"D:\local\LocalAppData.IdentityService\AzureServiceAuth\tokenprovider.json"
Parameters: Connection String: [No connection string specified],
Resource: https://vault.azure.net, Exception Message: Tried to get
token using Azure CLI. Access token could not be acquired. 'az' is not
recognized as an internal or external command,operable program or
batch file.
at
Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAccessTokenAsyncImpl(String
authority, String resource, String scope) at
Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage
response) at
Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage
request, CancellationToken cancellationToken) at
Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String
vaultBaseUrl, Nullable1 maxresults, Dictionary2 customHeaders,
CancellationToken cancellationToken) at
Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient
operations, String vaultBaseUrl, Nullable1 maxresults,
CancellationToken cancellationToken) at
Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
at
Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList1
providers) at
Microsoft.Extensions.Configuration.ConfigurationBuilder.Build() at
Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException&
hostingStartupErrors) at
Microsoft.AspNetCore.Hosting.WebHostBuilder.Build() at
MDHvNextAPI.Program.Main(String[] args) in ...
According to your error message, you have a managed identity assigned to your production environment which is used to access Azure Key Vault in order to get some secret.
When creating a new deployment slot you need to create another managed identity and grant access to it on Key Vault.
In summary: The managed identity is not shared between deployment slots.
I have an Asp.Net Core 3.1 application that reads its configuration from a Key Vault using the following code:
var keyVaultEndpoint = builtConfig["ProductKeyVaultUri"];
if (!string.IsNullOrEmpty(keyVaultEndpoint))
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(
azureServiceTokenProvider.KeyVaultTokenCallback));
config.AddAzureKeyVault(keyVaultEndpoint, keyVaultClient,
new DefaultKeyVaultSecretManager());
}
It uses the most recent version of the Microsoft.Azure.Services.AppAuthentication package to date - 1.4.0
The application is deployed to an Azure App Service with a System Managed Identity (MI for short), which can read secrets from the relevant Key Vault. It works.
Indeed when I remove the System MI from the Key Vault access policy and restart the App Service, I get this:
Unhandled exception. Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: Access denied. Caller was not found on any access policy.
Caller: appid=6f215b10-33a1-4e5d-b3b7-20e8f3d3b587;oid=3d6af26c-af56-4cef-a832-41c2303a8cbe;numgroups=0;iss=https://sts.windows.net/2...b/
Vault: a...v;location=eastus2
at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at Gateway.Program.Main(String[] args)
/opt/startup/init_container.sh: line 20: 10 Aborted (core dumped) dotnet Gateway.dll
(I scrubbed the tenantId and the key vault name)
It gives us the AppId (6f215b10-33a1-4e5d-b3b7-20e8f3d3b587) and the ObjectId (3d6af26c-af56-4cef-a832-41c2303a8cbe) which indeed match the System MI.
So far so good.
Now, I replace the System MI with a User Assigned MI, which has the access to the secrets in the same KV. However, restarting the App Service does not yield any good. The web app fails to read the secrets from the Key Vault which aborts the start of the container. Here is what docker logs tell me:
Unhandled exception. Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. Received a non-retryable error. MSI ResponseCode: BadRequest, Response: {"statusCode":400,"message":"Unable to load requested managed identity.","correlationId":"1b0ee635-0805-4438-8ae8-747e9f6dd7c2"}
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Environment variable LOCALAPPDATA not set.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. /bin/bash: az: No such file or directory
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String resource, String authority, CancellationToken cancellationToken)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.<get_KeyVaultTokenCallback>b__8_0(String authority, String resource, String scope)
at Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage response)
at Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at Gateway.Program.Main(String[] args)
/opt/startup/init_container.sh: line 20: 10 Aborted (core dumped) dotnet Gateway.dll
From which I conclude that the docker must be told explicitly the name of the User Assigned MI. It kind of makes sense, because an App Service may have many User Assigned MIs, but only one System MI.
So, the question is - can we use User Assigned MI at all in this scenario?
The issue might be that you are not telling AzureServiceTokenProvider the id of the user-assigned MI.
Here in the docs you can see samples of the connection string syntax: https://learn.microsoft.com/en-us/azure/key-vault/service-to-service-authentication#connection-string-support.
For your case, specify a connection string like:
RunAs=App;AppId={ClientId of user-assigned identity}
And use that as a constructor argument for the token provider.
By default the provider tries only system-assigned MI.
We are unable to query a sql database in azure from an Azure App Service when using a user assigned managed identity (it works fine if we use a system assigned managed identity)
The application is a .net core 2.2 web api application.
We have a user assigned identity set up for an Azure App Service.
This identity has been set up as the ad sql admin by using the following command:
az sql server ad-admin create --resource-group iactests --server iactestsql --object-id -u iactestmanagedIdentity
The token is generated like this:
services.AddDbContext<SchoolContext>(options => options.UseSqlServer(new
SqlConnection
{
ConnectionString = configuration.GetConnectionString("SchoolContext"),
AccessToken = isDevelopmentEnvironment ? null : new AzureServiceTokenProvider().GetAccessTokenAsync("https://database.windows.net/").Result
}), ServiceLifetime.Scoped);
This is the error we get:
Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. MSI ResponseCode: BadRequest, Response:
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Visual Studio Token provider file not found at "D:\local\LocalAppData\.IdentityService\AzureServiceAuth\tokenprovider.json"
Parameters: Connection String: [No connection string specified], Resource: https://database.windows.net/, Authority: . Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. 'az' is not recognized as an internal or external command,
operable program or batch file.
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String authority, String resource, String scope)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthenticationResultAsync(String resource, String tenantId)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAccessTokenAsync(String resource, String tenantId)
--- End of inner exception stack trace ---
If we use a system assign identity and configure the sql ad admin to be said identity, it works fine
Any ideas?
Thanks in advance
The AppAuthentication library now supports specifying user-assigned identities for Azure VMs and App Services as of the 1.2.0-preview2 release.
To use a user-assigned identity, you will need to set an AppAuthentication connection string of the format:
RunAs=App;AppId={ClientId of user-assigned identity}
The AppAuthentication connection string can be set as an argument passed to the AzureServiceTokenProvider constructor or specified in the AzureServicesAuthConnectionString environment variable. For more information on AppAuthentication connection strings, see here.
It looks like AzureServiceTokenProvider does not support user assigned managed identities, at least at this point. AzureServiceTokenProvder is a wrapper over the local HTTP endpoint that provides tokens to the application.
I was looking into this, and appears that you must provide the clientId of the user assigned managed identity to the endpoint to get a token. And AzureServiceTokenProvider doesn't have a way to do that (at least that I could figure out).
User assigned managed identities adds the ability to have multiple User assigned managed identities for an application. So the API to get a token needs to specify which MSI you want, the system-assigned MSI, or one of the user-assigned MSIs. The way the HTTP endpoint does this is that it uses the system-assigned MSI unless you specify a clientId.
In any case, you can hit the token endpoint directly, and provide the clientId of the user-assigned MSI like this:
public async Task<String> GetToken(string resource, string clientId = null)
{
var endpoint = System.Environment.GetEnvironmentVariable("MSI_ENDPOINT", EnvironmentVariableTarget.Process);
var secret = System.Environment.GetEnvironmentVariable("MSI_SECRET", EnvironmentVariableTarget.Process);
if (string.IsNullOrEmpty(endpoint))
{
throw new InvalidOperationException("MSI_ENDPOINT environment variable not set");
}
if (string.IsNullOrEmpty(secret))
{
throw new InvalidOperationException("MSI_SECRET environment variable not set");
}
Uri uri;
if (clientId == null)
{
uri = new Uri($"{endpoint}?resource={resource}&api-version=2017-09-01");
}
else
{
uri = new Uri($"{endpoint}?resource={resource}&api-version=2017-09-01&clientid={clientId}");
}
// get token from MSI
var tokenRequest = new HttpRequestMessage()
{
RequestUri = uri,
Method = HttpMethod.Get
};
tokenRequest.Headers.Add("secret", secret);
var httpClient = new HttpClient();
var response = await httpClient.SendAsync(tokenRequest);
var body = await response.Content.ReadAsStringAsync();
var result = JObject.Parse(body);
string token = result["access_token"].ToString();
return token;
}
This is a cross post from git.
I have a console app using "Microsoft.Azure.Services.AppAuthentication" Version="1.2.0-preview" and "Microsoft.Azure.KeyVault" Version="3.0.2".
I have a user assigned managed identity on my Windows 2019 VM.
The managed identity has been given the contributor role assignment on my keyvault and read to the resource group it lives in.
I am using a user assigned managed identity as the intention is to run a similar app in a kubernetes pod (with aad-pod identity). I have a connection string environment variable set as AzureServicesAuthConnectionString RunAs=App;<clientId of my user assigned MI>;TenantId=<tenantId>
When I try to connect to the keyvault to retrieve a secret I get a Forbidden exception:
Unhandled Exception: System.AggregateException: One or more errors
occurred. (Operation returned an invalid status code 'Forbidden') --->
Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: Operation
returned an invalid status code 'Forbidden' at
Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretWithHttpMessagesAsync(String
vaultBaseUrl, String secretName, String secretVersion, Dictionary`2
customHeaders, CancellationToken cancellationToken) at
Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretAsync(IKeyVaultClient
operations, String secretIdentifier, CancellationToken
cancellationToken) at msiauth.AzureStuff.Run() in
C:\Users\aiadmin\src\azure\msiauth\tests.cs:line 50
--- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout,
CancellationToken cancellationToken) at
System.Threading.Tasks.Task.Wait() at msiauth.Program.Main(String[]
args) in C:\Users\aiadmin\src\azure\msiauth\Program.cs:line 10
When I try to create a Resource Group using "Microsoft.Azure.Management.ResourceManager" Version="2.0.0-preview" it works fine so the user assigned MI works for that.
I tried the exact same keyvault code but with a system assigned managed identity and it worked correctly so permissions are fine and the code works.
Has anybody been able to do this in .net core?
Reading or writing secrets from or to the KeyVault is an operation on the Key Vault itself and is allowed (or not) based on the Access Policies set.
The contributor role you are referring belongs to the Azure Resource Manager RBAC system and allows to manage the KeyVault (f.e. assign access policies).
Most likely the identity that does work has both an RBAC assignment and an access policy. The one that doesn't work will need an access policy (and doesn't need an RBAC assignment)
For more on access policies see: https://learn.microsoft.com/en-us/azure/key-vault/key-vault-get-started#authorize