Azure Vaults Secrets Getting too many hits - azure

My Azure Keys Vaults Secrets contains my connection string. My app only connects to my Azure Vault during the app Startup.cs and then saves the connection string in a static class for use elsewhere.
I noticed a few days ago that Key Vaults Metrics was showing 2.45k hits during a 6 minute interval.
Here is the Starup.cs code:
public class Startup
{
var SecretUri = config["SecretUri"];
var TenantId = config["TenantId"];
var ClientSecret = config["ClientSecret"];
var ClientId = config["ClientId"];
var secretclient = new SecretClient(new Uri(SecretUri), new ClientSecretCredential(TenantId, ClientId, ClientSecret));
KeyVaultSecret keyv = secretclient.GetSecret("{myconnection_secretname}");
SecretServices.ConnectionString = keyv.Value;
}
From this point I use the SecretServices.ConnectionString anywhere else in the app where I need the connection string.
My question is there any way my app can hit the vault 2000 times in a few minutes or is something else happening?
Here is the graph from Azure Vaults Metrics Total API Hits:
This graph shows the sudden jump in the number of hits to the API Service.

It's possible that your app is hitting the Key Vault 2000 times in a few minutes. To get confirmation on this, you can have logging to your code to see how often the SecretServices.ConnectionString is being accessed.
Thanks # Tore Nestenius for the comment.
If the Key Vault Metrics show 2.45k hits, you may want to investigate other areas of your system to see if other components are accessing the Key Vault. For example, it's possible that some background jobs or scheduled tasks are hitting the Key Vault repeatedly.
You might also want to consider reducing the frequency with which your app accesses the Key Vault by caching the connection string in memory after retrieving it on startup. That way, your app would only need to retrieve the connection string from the Key Vault once, which would reduce the number of hits to the Key Vault.
If you are having too many connections, then you can use a memory cache and set the caching to a certain time.
Memory Cache:
MemoryCache memoryCache = MemoryCache.Default;
string mCachekey = VaultUrl + "_" +key;
if (!memoryCache.Contains(mCachekey))
{
try
{
using (var client = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(GetAccessTokenAsync),
new HttpClient()))
{
memoryCache.Add(mCachekey, await client.GetSecretAsync(VaultUrl, key), new CacheItemPolicy() { SlidingExpiration = TimeSpan.FromHours(1) });
}
}
catch (Exception ex)
{
throw new ApplicationException($"some exception {key}", ex);
}
return memoryCache[mCachekey] as string;
}
For more information, please see the below MS Docs.
Azure Key Vault service limits
Azure Key Vault throttling

Related

Is there a preset query for Azure Service Connections to retrieve certificate info with the azure-devops-api?

I am working on a front end application feature to get the details from all the key vault certificates and when they expire. The app uses the service connection to authenticate and the azure-devops-sdk and azure-devops-api to pull back data for api calls. I am currently getting data back on the contents within the key vault but it only lists the permissions for the certificates - https://learn.microsoft.com/en-us/rest/api/keyvault/getcertificate/getcertificates. I've tried appending /certificates on the end of the api call but it is not a valid endpoint.
I am aware of the azure rest api which returns this data - https://learn.microsoft.com/en-us/rest/api/keyvault/vaults/get but I am not sure how to implement this with the service connection.
Does anyone have any ideas on how I can gets this data back with the service connection?
export async function getCertInfoFromAzure(endpoint) {
requestInitialData = await getRequestInitialData();
const {project, settings, client} = requestInitialData;
const subscriptionId = endpoint.data.subscriptionId;
const apiVersion = providerMap['Microsoft.KeyVault'.toLowerCase()];
const resourceId = '/subscriptions/<subscription name> /resourceGroups/<resource group name>/providers/Microsoft.KeyVault/vaults/<key vault name>/';
const seRequest: ServiceEndpointRequest = createRequestObject(
`{{{endpoint.url}}}${resourceId}?api-version=${`2019-09-01`}`,
'jsonpath:$');
const response = await client.executeServiceEndpointRequest(seRequest, project.id, endpoint.id);
console.log('response', response);
console.log(JSON.parse(response.result));
console.log('seRequest', seRequest)
return {
response: JSON.parse(response.result[0]),
};
}

.Net Core 5.0 - Sql Azure + Always Encrypted + Managed Identity

I have a Azure SQL Db with encrypted columns (Always Encrypted with Azure KeyVault). I can access this db from SSMS and I can see the decrypted data.
I also have a web app made with .Net Core 5.0 which is deployed to Azure App Service. The app service has Managed Identity turned on and Key Vault that has enc/dec keys for that SQL Db has access policy setting to permit this app service to decrypt the data.
The web app works with managed identity as I can see that not encrypted data is retrieved without any issue.
Also, connection string does include Column Encryption Setting=enabled;. Here's the connection string:
Server=tcp:server.database.windows.net,1433;Database=somedb;Column Encryption Setting=enabled;
The problem is I can't find ANY samples with this kind of set up. I found some and I understand I need to register SqlColumnEncryptionAzureKeyVaultProvider. Here's my code to obtain SqlConnection:
internal static class AzureSqlConnection
{
private static bool _isInitialized;
private static void InitKeyVaultProvider(ILogger logger)
{
/*
* from here - https://github.com/dotnet/SqlClient/blob/master/release-notes/add-ons/AzureKeyVaultProvider/1.2/1.2.0.md
* and - https://github.com/dotnet/SqlClient/blob/master/doc/samples/AzureKeyVaultProviderExample.cs
*
*/
try
{
// Initialize AKV provider
SqlColumnEncryptionAzureKeyVaultProvider sqlColumnEncryptionAzureKeyVaultProvider =
new SqlColumnEncryptionAzureKeyVaultProvider(AzureActiveDirectoryAuthenticationCallback);
// Register AKV provider
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(
new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>(1, StringComparer.OrdinalIgnoreCase)
{
{SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, sqlColumnEncryptionAzureKeyVaultProvider}
});
_isInitialized = true;
}
catch (Exception ex)
{
logger.LogError(ex, "Could not register SqlColumnEncryptionAzureKeyVaultProvider");
throw;
}
}
internal static async Task<SqlConnection> GetSqlConnection(string connectionString, ILogger logger)
{
if (!_isInitialized) InitKeyVaultProvider(logger);
try
{
SqlConnection conn = new SqlConnection(connectionString);
/*
* This is Managed Identity (not Always Encrypted)
* https://learn.microsoft.com/en-us/azure/app-service/app-service-web-tutorial-connect-msi#modify-aspnet-core
*
*/
#if !DEBUG
conn.AccessToken = await new AzureServiceTokenProvider().GetAccessTokenAsync("https://database.windows.net/");
logger.LogInformation($"token: {conn.AccessToken}");
#endif
await conn.OpenAsync();
return conn;
}
catch (Exception ex)
{
logger.LogError(ex, "Could not establish a connection to SQL Server");
throw;
}
}
private static async Task<string> AzureActiveDirectoryAuthenticationCallback(string authority, string resource, string scope)
{
return await new AzureServiceTokenProvider().GetAccessTokenAsync("https://database.windows.net/");
//AuthenticationContext? authContext = new AuthenticationContext(authority);
//ClientCredential clientCred = new ClientCredential(s_clientId, s_clientSecret);
//AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
//if (result == null)
//{
// throw new InvalidOperationException($"Failed to retrieve an access token for {resource}");
//}
//return result.AccessToken;
}
}
This code does not throw any exceptions, and it works for non-encrypted queries. But for encrypted queries I get the following error:
Failed to decrypt a column encryption key. Invalid key store provider name: 'AZURE_KEY_VAULT'. A key store provider name must denote either a system key store provider or a registered custom key store provider. Valid system key store provider names are: 'MSSQL_CERTIFICATE_STORE', 'MSSQL_CNG_STORE', 'MSSQL_CSP_PROVIDER'. Valid (currently registered) custom key store provider names are: . Please verify key store provider information in column master key definitions in the database, and verify all custom key store providers used in your application are registered properly. Failed to decrypt a column encryption key. Invalid key store provider name: 'AZURE_KEY_VAULT'. A key store provider name must denote either a system key store provider or a registered custom key store provider. Valid system key store provider names are: 'MSSQL_CERTIFICATE_STORE', 'MSSQL_CNG_STORE', 'MSSQL_CSP_PROVIDER'. Valid (currently registered) custom key store provider names are: . Please verify key store provider information in column master key definitions in the database, and verify all custom key store providers used in your application are registered properly.
It seems like the key vault provider is not registered.
What should I do to make it work to query encrypted data?
packages used
<PackageReference Include="Microsoft.Azure.Services.AppAuthentication" Version="1.6.0" />
<PackageReference Include="Microsoft.Data.SqlClient" Version="2.1.0" />
<PackageReference Include="Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider" Version="1.2.0" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="5.0.0" />
I was able to use this code that uses provides a TokenCredential to the SqlColumnEncryption provider. DefaultAzureCredential returns a managed identity when deployed as an App Service:
SqlColumnEncryptionAzureKeyVaultProvider azureKeyVaultProvider = new SqlColumnEncryptionAzureKeyVaultProvider(new DefaultAzureCredential());
Dictionary<string, SqlColumnEncryptionKeyStoreProvider> providers = new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>
{
{ SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, azureKeyVaultProvider }
};
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(providers);
Call it from your startup.Configure method.
Turns out, it is impossible to read decrypted data in .NET 5 when MSI is used. There is a bug in MS packages and the app service is never authorized.
You have to use Service Principal. This works like a charm!
Update
I have to thank to MS engineers that offered a working solution:
public static async Task<string> KeyVaultAuthenticationCallback(string authority, string resource, string scope)
{
return await Task.Run(() => new ManagedIdentityCredential().GetToken(new TokenRequestContext(new string [] {"https://vault.azure.net/.default"})).Token);
/********************** Alternatively, to use User Assigned Managed Identity ****************/
// var clientId = {clientId_of_UserAssigned_Identity};
// return await Task.Run(() => new ManagedIdentityCredential(clientId).GetToken(new TokenRequestContext(new string [] {"https://vault.azure.net/.default"})).Token);
}
It's not a problem with .NET 5. You've taken the example authentication callback for Azure Key Vault and changed it to be specific to Azure SQL DB instead of the AKV resource. You need to adjust your callback to get a valid AKV token. This is just one way to get a token using the Azure.Core and Azure.Identity libraries:
private static async Task<string> AzureActiveDirectoryAuthenticationCallback(string authority, string resource, string scope)
{
return await Task.Run(() => new ManagedIdentityCredential().GetToken(new TokenRequestContext(new string [] {"https://vault.azure.net/.default"})).Token);
}
Answer for Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider v3
SqlColumnEncryptionAzureKeyVaultProvider azureKeyVaultProvider = new(new ManagedIdentityCredential());
Dictionary<string, SqlColumnEncryptionKeyStoreProvider> providers = new()
{
[SqlColumnEncryptionAzureKeyVaultProvider.ProviderName] = azureKeyVaultProvider
};
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(providers);
Taken from microsoft/sql-server-samples after almost losing my mind.
Apart from that, it's required to:
Assign app service managed identity Key Vault Crypto User role in Key Vault's IAM tab.
Grant app service managed identity DB user VIEW ANY COLUMN MASTER KEY DEFINITION and VIEW ANY COLUMN ENCRYPTION KEY DEFINITION permissions:
GRANT VIEW ANY COLUMN MASTER KEY DEFINITION to [app-service-user]
GRANT VIEW ANY COLUMN ENCRYPTION KEY DEFINITION to [app-service-user]
Allow app service access through Key Vault's firewall (i.e. adding app service outbound IPs to Key Vault's firewall rules).

Access Azure Key Vault secret in Azure Function

I'm building Azure Function in python triggered with Event Grid events, which should be able to gather secret from Kay Vault.
I added system-assigned managed identity to my Function App, and then I was able to pick my App in Key Vault access policies. I gave it permissions like below:
(I was trying different combinations at this one)
Also I provided new app setting with reference to mentioned key vault.
Unfortunetly when i try to check this value from code I'm unable to get this.
logging.info(os.environ)
When I add another app setting, just with plaintext it works great.
I will be grateful for any ideas what else can be done.
After hours of tests I found proper way to resolve this one.
In case of problems with Key Vault Reference make sure that App Function used for Azure Function is based on proper Hosting Plan.
Functions on 'Consumption Plan' are unaable to use Key Vault Reference. Same code on 'App Service Plan' works correctly.
https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
I couldn't figure out what you want to get with os.environ. I test with function It could work for me.
In function to retrieve key-valut if you already set it in appsettings, you could use Environment.GetEnvironmentVariable("secrest name", EnvironmentVariableTarget.Process) to implement it.
You can use the following helper to get the values
namespace AccessKeyVault
{
public static class GetKeyVaultValues
{
[FunctionName("GetKeyVaultValues")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
string linkKeyVaultUrl = $"https://keyVaultname.vault.azure.net/secrets/";
string keyvaultKey = $"KeyVaultKey";
var secretURL = linkKeyVaultUrl + keyvaultKey;
//Get token from managed service principal
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var kvClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
try
{
var clientIdRecord = await kvClient.GetSecretAsync(secretURL).ConfigureAwait(false);
string KeyvaultValue = clientIdRecord.Value;
return req.CreateErrorResponse(HttpStatusCode.OK, "Key vault secret value is : " + KeyvaultValue);
}
catch (System.Exception ex)
{
return req.CreateResponse(HttpStatusCode.BadRequest, "Key vault value request is not successfull");
}
}
}
}

Authenticate to multiple Azure services using Service Principle (.net Core)

I need to get access to Key Vault and Service Bus from code, using a Service Principle for authentication.
I can use the following code to access Service Bus, which works as expected - when I enable to Service Principle in the Access Policies I can pull the list of topics:
var credentials = SdkContext.AzureCredentialsFactory.FromServicePrincipal(APPID, APPSECRET, TENANTID, AzureEnvironment.AzureGlobalCloud);
var serviceBusManager = ServiceBusManager.Authenticate(credentials, SUBSCRIPTIONID);
var serviceBusNamespace = serviceBusManager.Namespaces.List().SingleOrDefault(n => n.Name == "SERVICEBUSNAMESPACE");
var topics = serviceBusNamespace.Topics.ListAsync().GetAwaiter().GetResult();
However, I also need to get some information from Key Vault and I was trying to establish a common way to authenticate.
METHOD 1
Similar to the above, I tried this code to access KeyVault:
var credentials = SdkContext.AzureCredentialsFactory.FromServicePrincipal(APPID, APPSECRET, TENANTID, AzureEnvironment.AzureGlobalCloud);
var kvManager = new KeyVaultClient(credentials);
var secret = kvManager.GetSecretAsync("https://VAULTNAMESPACE.vault.azure.net", "SECRETNAME").GetAwaiter().GetResult().Value;
I get the the following error:
Microsoft.Azure.KeyVault.Models.KeyVaultErrorException: 'Operation
returned an invalid status code 'Unauthorized''
METHOD 2
This code does work for Key Vault however (showing I have correct permissions):
string GetSecret()
{
var client = new KeyVaultClient(GetAccessToken);
var secret = client.GetSecretAsync("https://VAULTNAMESPACE.vault.azure.net", "SECRETNAME").GetAwaiter().GetResult();
return secret;
}
private static async Task<string> GetAccessToken(string authority, string resource, string scope)
{
var context = new AuthenticationContext("https://login.windows.net/" + tenantId);
var credential = new ClientCredential(appId, appSecret);
var tokenResult = await context.AcquireTokenAsync("https://vault.azure.net", credential);
return tokenResult.AccessToken;
}
But, again, it's a very KeyVault specific way to Authenticate and I was hoping to establish a common mechanism using SdkContext.AzureCredentialsFactory. Any reason why I'd be getting an Unauthorized exception with the code above connecting to Key Vault? (all is set up correctly in Azure).
Thanks for any tips!
When you use SdkContext.AzureCredentialsFactory.FromServicePrincipal to authenticate, it will use https://management.azure.com/ as its Resource Uri.
While Azure Key Vault has its own authorization system and its Resource Uri is https://vault.azure.net, so you may get the Unauthorized error message.
So, you could use Method2 to get access to Azure Key Vault with right Resource Uri.
For more details, you could refer to this article.

Is there any way to set up an alert on low disk space for Azure App Service

I'm running an Azure App Service on a Standard App Service Plan which allows usage of max 50 GB of file storage. The application uses quite a lot of disk space for image cache. Currently the consumption level lies at around 15 GB but if cache cleaning policy fails for some reason it will grow up to the top very fast.
Vertical autoscaling (scaling up) is not a common practice as it often requires some service downtime according to this Microsoft article:
https://learn.microsoft.com/en-us/azure/architecture/best-practices/auto-scaling
So the question is:
Is there any way to set up an alert on low disk space for Azure App Service?
I can't find anything related to disk space in the options available under Alerts tab.
Is there any way to set up an alert on low disk space for Azure App Service?
I can't find anything related to disk space in the options available under Alerts tab.
As far as I know, the alter tab doesn't contain the web app's quota selection. So I suggest you could write your own logic to set up an alert on low disk space for Azure App Service.
You could use azure web app's webjobs to run a background task to check your web app's usage.
I suggest you could use webjob timertrigger(you need install webjobs extension from nuget) to run a scheduled job. Then you could send a rest request to azure management api to get your web app current usage. You could send e-mail or something else according your web app current usage.
More details, you could refer to below code sample:
Notice: If you want to use rest api to get the current web app's usage, you need firstly create an Azure Active Directory application and service principal. After you generate the service principal, you could get the applicationid,access key and talentid. More details, you could refer to this article.
Code:
// Runs once every 5 minutes
public static void CronJob([TimerTrigger("0 */5 * * * *" ,UseMonitor =true)] TimerInfo timer,TextWriter log)
{
if (GetCurrentUsage() > 25)
{
// Here you could write your own code to do something when the file exceed the 25GB
log.WriteLine("fired");
}
}
private static double GetCurrentUsage()
{
double currentusage = 0;
string tenantId = "yourtenantId";
string clientId = "yourapplicationid";
string clientSecret = "yourkey";
string subscription = "subscriptionid";
string resourcegroup = "resourcegroupbane";
string webapp = "webappname";
string apiversion = "2015-08-01";
string authContextURL = "https://login.windows.net/" + tenantId;
var authenticationContext = new AuthenticationContext(authContextURL);
var credential = new ClientCredential(clientId, clientSecret);
var result = authenticationContext.AcquireTokenAsync(resource: "https://management.azure.com/", clientCredential: credential).Result;
if (result == null)
{
throw new InvalidOperationException("Failed to obtain the JWT token");
}
string token = result.AccessToken;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(string.Format("https://management.azure.com/subscriptions/{0}/resourceGroups/{1}/providers/Microsoft.Web/sites/{2}/usages?api-version={3}", subscription, resourcegroup, webapp, apiversion));
request.Method = "GET";
request.Headers["Authorization"] = "Bearer " + token;
request.ContentType = "application/json";
//Get the response
var httpResponse = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
string jsonResponse = streamReader.ReadToEnd();
dynamic ob = JsonConvert.DeserializeObject(jsonResponse);
dynamic re = ob.value.Children();
foreach (var item in re)
{
if (item.name.value == "FileSystemStorage")
{
currentusage = (double)item.currentValue / 1024 / 1024 / 1024;
}
}
}
return currentusage;
}
Result:

Resources