ManagedIdentityCredential authentication unavailable when running from Visual Studio - azure

I recently configured our Azure App Service and Azure SQL Server to use managed identity access. The App Service has System assigned managed identity and has the correct assigned role as Owner for the SQL database. When I publish my code to the app service it works just fine and is able to access the database.
My issue is that I cannot connect to the database through my app when running it locally through Visual Studio. My Azure account is set as the Azure Active Directory admin on SQL Server and I'm able to open up the connection and run queries through Azure Data Studio when I'm signed in.
I'm using the same account through Visual Studio, even unlinked and relinked it under the Tools -> Options -> Azure Service Authentication / Account Selection. But when I try and run the app locally I get the error ManagedIdentityCredential authentication unavailable. Multiple attempts failed to obtain a token from the managed identity endpoint.
The error doesn't give me much to work with because I'm not sure how to see the full view of what is being tried in order to authenticate. I'm not using any code on my end to obtain an access token since I'm just letting the Microsoft SQLClient obtain it for me through my connection string.
Here is the full stacktrace of the error, anything that can point me in the right direction would be appreciated. My guess is it has something to do with Visual Studios probably needing to be allowed to get an access token on my behalf, but I figured since I'm signed into my azure account it might not matter.
at Microsoft.Data.ProviderBase.DbConnectionPool.CheckPoolBlockingPeriod(Exception e)
at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides)
at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternal(Boolean errorsExpected)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open(Boolean errorsExpected)
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.MoveNext()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source, Expression`1 predicate)
Azure.RequestFailedException: A socket operation was attempted to an unreachable network. (169.254.169.254:80)
---> System.Net.Http.HttpRequestException: A socket operation was attempted to an unreachable network. (169.254.169.254:80)
---> System.Net.Sockets.SocketException (10051): A socket operation was attempted to an unreachable network.
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
--- End of inner exception stack trace ---
at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Identity.ImdsManagedIdentitySource.<AuthenticateAsync>d__15.MoveNext()
at Azure.Identity.ManagedIdentityClient.<AuthenticateCoreAsync>d__17.MoveNext()
at Azure.Identity.ManagedIdentityClient.<AppTokenProviderImpl>d__18.MoveNext()
at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<SendTokenRequestToProviderAsync>d__4.MoveNext()
at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<FetchNewAccessTokenAsync>d__3.MoveNext()
at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<ExecuteAsync>d__2.MoveNext()
at Microsoft.Identity.Client.Internal.Requests.RequestBase.<RunAsync>d__12.MoveNext()
at Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.<ExecuteAsync>d__3.MoveNext()
at Azure.Identity.AbstractAcquireTokenParameterBuilderExtensions.<ExecuteAsync>d__0`1.MoveNext()
at Azure.Identity.MsalConfidentialClient.<AcquireTokenForClientCoreAsync>d__21.MoveNext()
at Azure.Identity.MsalConfidentialClient.<AcquireTokenForClientAsync>d__20.MoveNext()
at Azure.Identity.ManagedIdentityClient.<AuthenticateAsync>d__16.MoveNext()
at Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__16.MoveNext()
at
Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
at Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__16.MoveNext()
at Azure.Identity.ManagedIdentityCredential.<GetTokenAsync>d__14.MoveNext()
at Microsoft.Data.SqlClient.ActiveDirectoryAuthenticationProvider.<AcquireTokenAsync>d__17.MoveNext()
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.<>c__DisplayClass147_1.<<GetFedAuthToken>b__1>d.MoveNext()
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
Relevant connection DbContext information. Currently using version 7.0.3 of Microsoft.EntityFrameworkCore for the DbContext class
public class DbEntities: DbContext
{
public DbEntities() : base() { }
//All DbSet<...> classes here
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(Environment.GetEnvironmentVariable("DB_CONNECTION_STRING"));
}
}
Relevant AppSettings JSON file
{
"ConnectionStrings": {
"DB_CONNECTION_STRING": "Server=tcp:MY-SERVER.database.windows.net;Initial Catalog=development;Persist Security Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Authentication=\"Active Directory Managed Identity\";"
},
}

https://learn.microsoft.com/en-us/sql/connect/ado-net/sql/azure-active-directory-authentication?view=sql-server-ver16
By using Authentication=\"Active Directory Managed Identity\" you will tell your application to use only managed identity authentication. This can only be used if you are actually running as an Azure resource. You cannot use Managed Identity authentication with your personal account.
By using Active Directory Default as the value, the system will try multiple variants:
Authenticate with an Azure AD identity by using password-less and non-interactive mechanisms including Managed Identities, Visual Studio Code, Visual Studio, Azure CLI, etc.

Related

Login failed for user ''. while trying to access database using User Assigned ManagedIdentity [duplicate]

I am trying to use a User Assigned Managed Identity in one of our applications. I also read about the differences between System Assigned Managed Identity and User Assigned Managed Identity.
It is very clear to me that a System Assigned Managed Identity cannot be used locally as there you're assigning an identity to an Azure Resource.
However I am not clear if a User Assigned Managed Identity can be used locally. Only thing I could find is the following:
In my scenario, I would like to read some secrets from an Azure Key Vault. I have created a User Assigned Managed Identity and configured access policies on the Key Vault to give necessary permissions to this identity. Considering I am using this identity to access Azure Key Vault (which is an Azure resource), my expectation is that it should work regardless of the location (using JetBrains Rider as my IDE) from where my code is running.
However when I try to do something like:
var managedIdentityCredential = new ManagedIdentityCredential("managed-identity-id");
SecretClient secretClient = new(new Uri("https://mykeyvault.vault.azure.net/"), managedIdentityCredential);
KeyVaultSecret secret = await secretClient.GetSecretAsync(key);
I get the Azure.Identity.CredentialUnavailableException with ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found error message when I run the code locally:
Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found.
at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthenticateRequestAsync(HttpMessage message, Boolean async, AuthenticationChallenge challenge)
at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.ProcessCoreAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.HttpPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync[TResult](RequestMethod method, Func`1 resultFactory, CancellationToken cancellationToken, String[] path)
at Azure.Security.KeyVault.Certificates.CertificateClient.GetCertificateAsync(String certificateName, CancellationToken cancellationToken)
Any insights into this will be highly appreciated.
No. User managed identity is also not supported with ManagedIdentityCredential in the local environment.
You should use DefaultAzureCredential for the code to work in local environment.
See the Note tip here.
Note
The ManagedIdentityCredential works only in Azure environments of
services that support managed identity authentication. It doesn't work
in the local environment. Use DefaultAzureCredential for the code
to work in both local and Azure environments as it will fall back to a
few authentication options including managed identity.
In case you want to use a user-asigned managed identity with the
DefaultAzureCredential when deployed to Azure, specify the
clientId.

Can a "User Assigned Managed Identity" be used locally?

I am trying to use a User Assigned Managed Identity in one of our applications. I also read about the differences between System Assigned Managed Identity and User Assigned Managed Identity.
It is very clear to me that a System Assigned Managed Identity cannot be used locally as there you're assigning an identity to an Azure Resource.
However I am not clear if a User Assigned Managed Identity can be used locally. Only thing I could find is the following:
In my scenario, I would like to read some secrets from an Azure Key Vault. I have created a User Assigned Managed Identity and configured access policies on the Key Vault to give necessary permissions to this identity. Considering I am using this identity to access Azure Key Vault (which is an Azure resource), my expectation is that it should work regardless of the location (using JetBrains Rider as my IDE) from where my code is running.
However when I try to do something like:
var managedIdentityCredential = new ManagedIdentityCredential("managed-identity-id");
SecretClient secretClient = new(new Uri("https://mykeyvault.vault.azure.net/"), managedIdentityCredential);
KeyVaultSecret secret = await secretClient.GetSecretAsync(key);
I get the Azure.Identity.CredentialUnavailableException with ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found error message when I run the code locally:
Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found.
at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthenticateRequestAsync(HttpMessage message, Boolean async, AuthenticationChallenge challenge)
at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.ProcessCoreAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.HttpPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync[TResult](RequestMethod method, Func`1 resultFactory, CancellationToken cancellationToken, String[] path)
at Azure.Security.KeyVault.Certificates.CertificateClient.GetCertificateAsync(String certificateName, CancellationToken cancellationToken)
Any insights into this will be highly appreciated.
No. User managed identity is also not supported with ManagedIdentityCredential in the local environment.
You should use DefaultAzureCredential for the code to work in local environment.
See the Note tip here.
Note
The ManagedIdentityCredential works only in Azure environments of
services that support managed identity authentication. It doesn't work
in the local environment. Use DefaultAzureCredential for the code
to work in both local and Azure environments as it will fall back to a
few authentication options including managed identity.
In case you want to use a user-asigned managed identity with the
DefaultAzureCredential when deployed to Azure, specify the
clientId.

Does Azure App Service running Asp.Net Core 3.1 in a Linux docker container support User Assigned Managed Identity?

I have an Asp.Net Core 3.1 application that reads its configuration from a Key Vault using the following code:
var keyVaultEndpoint = builtConfig["ProductKeyVaultUri"];
if (!string.IsNullOrEmpty(keyVaultEndpoint))
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(
azureServiceTokenProvider.KeyVaultTokenCallback));
config.AddAzureKeyVault(keyVaultEndpoint, keyVaultClient,
new DefaultKeyVaultSecretManager());
}
It uses the most recent version of the Microsoft.Azure.Services.AppAuthentication package to date - 1.4.0
The application is deployed to an Azure App Service with a System Managed Identity (MI for short), which can read secrets from the relevant Key Vault. It works.
Indeed when I remove the System MI from the Key Vault access policy and restart the App Service, I get this:
Unhandled exception. Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: Access denied. Caller was not found on any access policy.
Caller: appid=6f215b10-33a1-4e5d-b3b7-20e8f3d3b587;oid=3d6af26c-af56-4cef-a832-41c2303a8cbe;numgroups=0;iss=https://sts.windows.net/2...b/
Vault: a...v;location=eastus2
at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at Gateway.Program.Main(String[] args)
/opt/startup/init_container.sh: line 20: 10 Aborted (core dumped) dotnet Gateway.dll
(I scrubbed the tenantId and the key vault name)
It gives us the AppId (6f215b10-33a1-4e5d-b3b7-20e8f3d3b587) and the ObjectId (3d6af26c-af56-4cef-a832-41c2303a8cbe) which indeed match the System MI.
So far so good.
Now, I replace the System MI with a User Assigned MI, which has the access to the secrets in the same KV. However, restarting the App Service does not yield any good. The web app fails to read the secrets from the Key Vault which aborts the start of the container. Here is what docker logs tell me:
Unhandled exception. Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException: Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. Received a non-retryable error. MSI ResponseCode: BadRequest, Response: {"statusCode":400,"message":"Unable to load requested managed identity.","correlationId":"1b0ee635-0805-4438-8ae8-747e9f6dd7c2"}
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Environment variable LOCALAPPDATA not set.
Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/2...b. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. /bin/bash: az: No such file or directory
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAuthResultAsyncImpl(String resource, String authority, CancellationToken cancellationToken)
at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.<get_KeyVaultTokenCallback>b__8_0(String authority, String resource, String scope)
at Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage response)
at Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at Gateway.Program.Main(String[] args)
/opt/startup/init_container.sh: line 20: 10 Aborted (core dumped) dotnet Gateway.dll
From which I conclude that the docker must be told explicitly the name of the User Assigned MI. It kind of makes sense, because an App Service may have many User Assigned MIs, but only one System MI.
So, the question is - can we use User Assigned MI at all in this scenario?
The issue might be that you are not telling AzureServiceTokenProvider the id of the user-assigned MI.
Here in the docs you can see samples of the connection string syntax: https://learn.microsoft.com/en-us/azure/key-vault/service-to-service-authentication#connection-string-support.
For your case, specify a connection string like:
RunAs=App;AppId={ClientId of user-assigned identity}
And use that as a constructor argument for the token provider.
By default the provider tries only system-assigned MI.

Certificate Store Access denied

I'm receiving a System.Security.Cryptography.CryptographicException Access is denied. when trying to create media service task or job. The app is running on an azure website instance. Everything works locally. Look like the app is unable to write to the certificate store. I have tried elevating the trust level for the app to full with no effect.
System.Security.Cryptography.CryptographicException: Access is denied.
at System.Security.Cryptography.X509Certificates.X509Store.Open(OpenFlags flags)
at Microsoft.WindowsAzure.MediaServices.Client.EncryptionUtils.SaveCertificateToStore(X509Certificate2 certToStore)
at Microsoft.WindowsAzure.MediaServices.Client.BaseContentKeyCollection.GetCertificateForProtectionKeyId(DataServiceContext dataContext, String protectionKeyId)
at Microsoft.WindowsAzure.MediaServices.Client.BaseContentKeyCollection.GetCertificateToEncryptContentKey(DataServiceContext dataContext, ContentKeyType contentKeyType)
at Microsoft.WindowsAzure.MediaServices.Client.JobData.ProtectTaskConfiguration(TaskData taskData, X509Certificate2& certToUse)
at Microsoft.WindowsAzure.MediaServices.Client.JobData.Submit()
at MediaServices.Encoding.Encode_video_h264(IAsset asset)
at Business.Background_tasks.Kickoff_encoding(IAsset asset, Guid id)
at Routing.Controllers.BlobsController.UploadBlock(Int32 id)
at lambda_method(Closure , ControllerBase , Object[] )
at System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters)
at System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters)
at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass15.<InvokeActionMethodWithFilters>b__12()
at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation)
at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass15.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14()
at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor, IDictionary`2 parameters)
at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
at System.Web.Mvc.Controller.ExecuteCore()
at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
at System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext)
at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
at System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f)
at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Edit
I have moved the application to a VM and still receive the same exception
Edit
Final solution found here http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/67216e79-c259-4685-95d0-3a09bab5c0c5/
When running on IIS7 - you can specify to load the user profile of the App Pool. This gives you access to the per user store of the app pool account.
This is what you would expect from Windows Azure Web Sites. You're running in a shared environment together with other Web Sites and for security reasons there are a many things you cannot do.
If you want full control of the machine where your web application is hosted (like full access to the certificate store, making changes in the registry, ...) you should deploy your application to a Web Role.

Windows SharePoint Services Search won't stop

Background: I had been asked to create a second Sharepoint application server to reprovision the one that is currently in the SharePoint Farm. I installed the os, SharePoint, and SP1. I then began the process of moving all the services on to the new server. (The central farm consists of 2 WFE and 1 App server, each on W2K8).
Problem: I had trouble with the SharePoint Configuration Wizard completing the upgrade. Believing that it was due to a problem with search I stopped both Search Services ( goodbye indexes). Afterward, I still had trouble moving them. I then decided to create a new SSP since my hours of web crawling provided evidence that this would help fix my problems. It nearly did. I got the Office SharePoint Server Search started. However Windows SharePoint Services Search is stuck in the "Stopping" status. So now what? the farm mostly works, has an indexer, but no WSS search even with a new Shared Service Provider.
Is there a way to force it stopped? I've tried psconfig, but maybe I don't have the right voodoo to kill the service completely.
I've included a stack trace from the logs that I beleive is part of the problem...
The error log stack trace:
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880i High System.Data.SqlClient.SqlException: Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'. at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject) at System.Data.SqlClient.SqlInternalCo...
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880k High at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command, CommandBehavior behavior) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.FetchId(QualifiedObjectName qName) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.GetObject(String name, Guid parentId, Type type) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.get_Farm() at Microsoft.SharePoint.Administration.SPFarm.FindLocal(SPFarm& farm, Boolean& isJoined) at Microsoft.SharePoint.Administration.SPFarm.get_Local() at Microsoft.SharePoint.Administration.SPServer.get_Local() at Microsoft.SharePoint.Administration.SPTimerStore.InitializeTimer(Int64& cacheVersion, Object& jobDefinitions, Int32& timerMode, String& serverName) at Microsoft.Share...
11/05/2009 10:28:58.89* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880i High ...nnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection) at System.Data...
11/05/2009 10:28:58.89* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880k High ...Point.Administration.SPNativeConfigurationProvider.InitializeTimer(Int64& cacheVersion, Object& jobDefinitions, Int32& timerMode, String& serverName)
11/05/2009 10:28:58.89* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880i High ....ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.SharePoint.Utilities.SqlSession.OpenConnection()
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880l High ConnectionString: 'Data Source=servernamedb;Initial Catalog=config_database;Integrated Security=True;Enlist=False' ConnectionState: Closed ConnectionTimeout: 15
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 880j High SqlError: 'Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.' Source: '.Net SqlClient Data Provider' Number: 18456 State: 1 Class: 14 Procedure: '' LineNumber: 65536 Server: 'servernamedb'
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Database 6y64 Critical SQL database login failed. Additional error information from SQL Server is included below. Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.
11/05/2009 10:28:58.89 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Topology 88bl Monitorable An exception occured while trying to acquire the local farm: System.Data.SqlClient.SqlException: Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'. at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnec...
11/05/2009 10:28:58.91* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Topology 88bl Monitorable ...tion owningObject) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectio...
11/05/2009 10:28:58.91* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Topology 88bl Monitorable ...nPool pool, DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.SharePoint.Utilities.SqlSes...
11/05/2009 10:28:58.91* OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Topology 88bl Monitorable ...sion.OpenConnection() at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command, CommandBehavior behavior) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.FetchId(QualifiedObjectName qName) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.GetObject(String name, Guid parentId, Type type) at Microsoft.SharePoint.Administration.SPConfigurationDatabase.get_Farm() at Microsoft.SharePoint.Administration.SPFarm.FindLocal(SPFarm& farm, Boolean& isJoined)
11/05/2009 10:28:58.91 OWSTIMER.EXE (0x08FC) 0x0A64 Windows SharePoint Services Timer 5utx Unexpected The timer service could not initialize its configuration, please check the configuration database. Will retry later.
It looks like the service is running under the 'NT AUTHORITY\ANONYMOUS LOGON, that can't be right?!?
To actually "kill" the process do the following:
1.Press Ctrl-Alt-Delete keys simultaneously, and then click on Task Manager. Alternatively, right click on taskbar and select Task Manager. You can also start Task Manager by manually starting “taskmgr” from Run command in Start Menu.
2.In the Windows Task Manager window, locate and select (highlight) the application or program that’s not responding. In your system is crawling and extremely slow in responding, you may also check out for services or processes that using too much CPU resources or memory resources.
3.Click on End Task.
4.A Task Manager Warning dialog saying “Terminating a process can cause undesired results including loss of data and system instability. The process will not be given the chance to save its state or data before it is terminated. Are you sure you want to terminate the process?” will appear. Confirm the process by clicking on OK or End Now.
From here
Here is something to try:
Find the SPSearch service in the W2K8 Services tab.
Right click the service and shut down. You might, as I did, receive an error stating that the handler is wrong. (BTW: make sure that you are logged into the server as the farm service account just to be safe)
Click on the Services button from here... you could have gone and managed the services directly from the management console also.
Goto SPSsearch>Properties>Log On tab. Make Logon as local system account.
Stop the service
Make Logon as your service account. Add the passwords
Start up type should be automatic. Then start the service here.
Go back to the Central Administration page.
Start the Service from here if it isn't started.
At this point you should have a working service both on the server and within the Farm Context. At least I did. Your milage may vary
Are you able to install sp2?
Stopping the search service actually works after sp2.

Resources