Can Azure Key Vault be used with Functions to store the connection string for queue triggers? - azure

I was able to use the Key Vault inside a function app as described here but when I tried to use the Key Vault to hold the connection string of a function with a queue trigger I have issues with the storage account connection string. The function seems to find the parameter I provide but either doesn't get the secret back or doesn't like the information when it throws the error No valid combination of account information found.
My function is defined as:
[FunctionName("ReadQueueForMessage")]
public static async Task Run([QueueTrigger("%AzureQueueTrigger%", Connection = "AzureWebJobsStorage")] string myQueueItem,
Binder binderinputblob,
ILogger log)
This works fine if I just define the connection string in my local.settings.json. What I'm trying to do is instead of just putting the connection string in the json file, I want to point the function to the Key Vault with this syntax:
"AzureWebJobsStorage": "#Microsoft.KeyVault(SecretUri=https://myappkeyvault.vault.azure.net/secrets/myapp-AzureWebJobsStorage-Queue/the-guid-of-secret)",
I did go to the Key Vault and update the access policies to include the function app so it can read/list the secrets. The documentation here shows making an update to the configuration after it was deployed to Azure. I'm trying to test first in Visual Studio. Maybe that is the issue? Or is it not possible at all to use the secret in this manner?

I'm trying to test first in Visual Studio.
For now using Azure Key Vault references with Azure Functions does not support to work on local, as confirmed by Azure Functions team. If you still want to test on local, you could implemented an incomplete local workaround like this issue.
I test on portal and it works well. You could refer to the following steps as below:
1.In VS Function.cs, then publish to azure:
public static void Run([QueueTrigger("queue", Connection = "AzureWebJobsStorage")]string myQueueItem, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
string connectionString = System.Environment.GetEnvironmentVariable("AzureWebJobsStorage");
log.Info($"The connection string is {connectionString}");
}
2.Set AzureWebJobsStorage on Appsettings setting on portal.
3.Then it will work fine.

With the Nuget packages Azure.Extensions.AspNetCore.Configuration.Secrets and Azure.Identity you can now use the KeyVault as a configuration provider, the same way as in ASP.NET Core. You need to add a class derived from FunctionsStartup to add the KeyVault as a configuration provider, see Add FunctionsStartup class with the KeyVault as a configuration provider.
If you add the AzureWebJobsStorage connection string as a secret to the KeyVault, you can remove it from the Configuration section of your Function App in Azure. Make sure to turn on System assigned in the Identity section and in the KeyVault add an Access Policy with the Secret permissions Get and List for your Function App.
When you run your Function App local for debugging, Azure.Identity automatically uses your Microsoft Account for access to the KeyVault, if it has at least Get and List access to the secrets.
Unfortunately, when you test local, the Function App does NOT read AzureWebJobsStorage from the configuration/KeyVault, but requires it to be stored in local.settings.json. To prevent storing keys on your local computer, you can set AzureWebJobsStorage to "UseDevelopmentStorage=true" in local.settings.json.
For detailed instructions see: Create Azure Function App with Dependency Injection and the Key Vault as Configuration Provider.
Example project: https://github.com/Forestbrook/FunctionWithKeyVaultAndDI

Related

Any way to concatenate two Application setting in Configuration - Azure Function app

In Azure Function App, I have added two application settings using the Configuration tab. The first Application setting is fetching the SAS token from the Azure Key vault using #Microsoft.KeyVault(SecretUri=##). The other application setting is the endpoint URL. Now I have to concatenate these two variables and use in connection parameter in HTTP and Queue Trigger.
For example, Below StorageConnectionAppSetting will be the key that will have concatenated value.
public static async Task Run([QueueTrigger("myqueue-items", Connection = "StorageConnectionAppSetting")] string queueItem, ILogger log)
Is there any way this concatenation can be done in the Application setting itself.
This isn't currently possible. Even if you were to customize configuration using DI, it won't work for triggers when deploying to the consumption or premium plans as mentioned at the end of the docs.

How do I allow ContinuousIntegration.exe to use a connection string in Azure Key Vault

I've got a Kentico Xperience (v13) instance in Azure and I want to run ContinuousIntegration.exe to populate my database up there with content from my CI xml files. The catch is that we're injecting the CMSConnectionString setting into the web app from Azure Key Vault (AKV) and the CI.exe isn't seeing it. Instead I get this error message:
CMS.DataEngine.ApplicationInitException: Cannot access the database specified by the 'CMSConnectionString' connection string. Please install the database externally and set a correct connection string.
Or maybe this error message:
Failed to execute the command.
Here's the relevant section from our web.config (that works for the website!):
<connectionStrings>
<!--Should be provided by Azure Key Vault-->
</connectionStrings>
How do I ensure that the executable gets access to the secrets in AKV?
It is possible to let ContinuousIntegration.exe know about a secure connection string with a small custom module that sets the connection string at startup. Here is the basic code of the module:
[assembly: AssemblyDiscoverable]
[assembly: RegisterModule(typeof(AzureConnectionStringModule))]
public class AzureConnectionStringModule : Module
{
public AzureConnectionStringModule()
: base(nameof(AzureConnectionStringModule))
{
}
protected override void OnPreInit()
{
base.OnPreInit();
var azureConnectionString = Environment.GetEnvironmentVariable("SQLAZURECONNSTR_CMSConnectionString");
if (string.IsNullOrWhiteSpace(azureConnectionString))
{
azureConnectionString = Environment.GetEnvironmentVariable("CMSConnectionString");
}
if (!string.IsNullOrWhiteSpace(azureConnectionString))
{
SettingsHelper.ConnectionStrings.SetConnectionString("CMSConnectionString", azureConnectionString);
}
}
}
From a fresh installation of Kentico Xperience 13, here are steps to configure this:
Follow the steps here to add Key Vault support to the admin app locally: https://learn.microsoft.com/en-us/azure/key-vault/general/vs-key-vault-add-connected-service.
Add the module above to the solution in a class library. Make sure the main project references the class library so that it is included during building.
Ensure that the ~\web.config does not have a connection string, or an app setting, named CMSConnectionString.
Deploy the app to an App Service.
In Azure, create an App Service configuration setting with name CMSConnectionString and value #Microsoft.KeyVault(VaultName=your-keyvault;SecretName=CMSConnectionString).
In the Key Vault, create a secret with name CMSConnectionString and value a connection string to an Azure SQL database. You may also need to follow https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references to create an access policy for my App Service.
At this point, the Kentico Xperience 13 admin should load with access to the database.
In the App Service portal, under Development Tools select Console.
In the console, run cd bin and then ContinuousIntegration.exe -r. This should produce a message about the repository not being configured, or output on the restore action.

Should Azure Functions app be restarted if a secret is updated in Key Vault?

I'm trying to understand the integration between Azure Functions and Key Vault. In the application settings file on the portal, if I reference a KV endpoint, does the Function runtime retrieve the result once and cache it locally or does it hit the endpoint every time the config key is referenced? In other words, I'm trying to understand if any changes in KV will require the Function app to be restarted or not.
Update:
Versions no longer required for Key Vault references in App Service and Azure Functions
If a version is not specified in the reference, then the app will use the latest version that exists in Key Vault. When newer versions become available, such as with a rotation event, the app will automatically update and begin using the latest version within one day. Any configuration changes made to the app will cause an immediate update to the latest versions of all referenced secrets.
If you use App Service Key Vault References, you will need to update the configuration value in the Azure Portal. Restarting will not change anything. This is because you are referencing an actual secret version. If you update the secret, you will get a new version.
Versions are currently required. When rotating secrets, you will need to update the version in your application configuration
See also Azure Function App use latest version of Key Vault Secret via Application Settings
For now, you don't need to restart your Azure function manually if you update a key value in Key Vault. The azure function will restart automatically for you to load all new values.
Azure functions load values defined in application settings at the start stage, if you use App Service Key Vault References in your Azure function, the key value will also be loaded from Key Vault at the start stage. If you modify application settings on Azure Portal, your Azure function will get restarted to reload all settings:
These are my test steps:
My code is simple, just get the key value from application settings:
#r "Newtonsoft.Json"
using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
public static async Task<IActionResult> Run(HttpRequest req, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string key = Environment.GetEnvironmentVariable("key");
return new OkObjectResult(key);
}
My key config in application settings:
As you can see I did not involve SecretVersion, so that I can get the latest value in KeyVault. If you involve SecretVersion in application settings, as #Alex AIT said, you should modify application settings with the latest SecretVersion as well.
Before I modify the key value:
Then I update the key4demo value in Key vault as: 78910
The Azure function reply 503 which means it is restarting :
After few seconds, my function replys me with the latest value in KV:
no, no need to restart function or anything. Key Vault lives a separate life.
Microsoft Docs
If a version is not specified in the reference, then the app will use the latest version that exists in the key vault. When newer versions become available, such as with a rotation event, the app will automatically update and begin using the latest version within 24 hours. The delay is because App Service caches the values of the key vault references and refetches it every 24 hours. Any configuration changes to the app that results in a site restart causes an immediate refetch of all referenced secrets.

Use Azure keyvault secrets offline when developing .NET core Azure functions

When I develop for Azure I usually start copying in some keyvault client code so only keyvault urls will be in my settings file, no secrets can ever end up my git repositories.
After starting to make Azure functions I realized that it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
The recommended approach seems to connect the app to keyvault directly in Azure when deployed, and just manage secrets locally in Secret Manager, like suggested in
this article
I am not developing alone, so while I am not adverse to using a tool like Secret Manager, I need to still have my offline secrets connected to the Azure keyvault! If others change anything.
Question: How do I manage secrets offline in a way that is synchronized with Azure keyvault?
it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
In short, it's possible.
Here are steps you could follow and refer to the detailed article.
1.Add a System Assigned Managed Identity to the Azure Function.
2.Go to the Access Control section of your Key Vault and click on Add a role assignment blade.
3.Go to your Key Vault and click on Access Policies and then click on Add service principal with secret GET permission.
4.When you use ServiceBusTrigger, you set ServiceBusConnectionString in Function ->Configuration ->Application settings.
public static void Run([ServiceBusTrigger(_topicName, _subscriptionName, Connection = "ServiceBusConnectionString")] string mySbMsg, ILogger log)
{ ....
}
5.Now you change the value of ServiceBusConnectionString to the Azure Key Vault reference with #Microsoft.KeyVault(SecretUri=Secret URI with version). Then you could run your function successfully with Key Vault.

How to map Azure Functions secrets from Key Vault automatically

I was wondering if it's possible to initialize the queue trigger or even the blob trigger off a connection string that is read from azure vault.
Right now, we have to set these data connection via environment settings via blade properties. However, I wanted to just use the service principal to retrieve the token for the azure key vault to get all these connection strings.
I'm trying to figure how to get this working in java.
Thanks,
Derek
This feature is tracked and in progress here:
Feature request: retrieve Azure Functions' secrets from Key Vault
Add binding to Key Vault
EDIT 28/11/2018: It is currently in preview
Simplifying security for serverless and web apps with Azure Functions and App Service
Former answer 07/10/2018
This solution won't work for Triggers using the consumption plan.
In the mean time I did some research about your problem and it is possible to read config from key vault if you use Azure Function v2.
I've created an Azure Functions v2 (.NET Standard) from Visual Studio.
It uses:
NETStandard.Library v2.0.3
Microsoft.NET.Sdk.Functions v1.0.22
Microsoft.Azure.WebJobs v3.0.0
Microsoft.Azure.WebJobs.Extensions.Storage v3.0.0
Because Azure Functions v2 uses ASP.NET core, I was able to reference this link to configure my functions app to use Azure Key Vault:
Azure Key Vault configuration provider in ASP.NET Core
I've added this nuget package:
Microsoft.Extensions.Configuration.AzureKeyVault
I've configured my app to use this nuget package:
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Linq;
[assembly: WebJobsStartup(typeof(FunctionApp1.WebJobsExtensionStartup), "A Web Jobs Extension Sample")]
namespace FunctionApp1
{
public class WebJobsExtensionStartup : IWebJobsStartup
{
public void Configure(IWebJobsBuilder builder)
{
// Get the existing configuration
var serviceProvider = builder.Services.BuildServiceProvider();
var existingConfig = serviceProvider.GetRequiredService<IConfiguration>();
// Create a new config based on the existing one and add kv
var configuration = new ConfigurationBuilder()
.AddConfiguration(existingConfig)
.AddAzureKeyVault($"https://{existingConfig["keyVaultName"]}.vault.azure.net/")
.Build();
// replace the existing configuration
builder.Services
.Replace(ServiceDescriptor.Singleton(typeof(IConfiguration), configuration));
}
}
}
My Azure functions uses MSI:
I've granted Read/List secrets permissions to the function app on my key vault:
I have a small queue triggered function:
public static class Function2
{
[FunctionName("Function2")]
public static void Run([QueueTrigger("%queueName%", Connection = "queueConnectionString")]string myQueueItem, ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
}
}
The queueName is defined in the local.settings.json file (App settings blade once deployed):
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"keyVaultName": "thomastestkv",
"queueName": "myqueue"
}
}
The queueConnectionString is configured in my keyvault:
Sourcing Application Settings from Key Vault
The Key Vault references feature makes it so that your app can work as if it were using App Settings as they have been, meaning no code changes are required. You can get all of the details from our Key Vault reference documentation, but I’ll outline the basics here.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
That’s it! No changes to your code required!
For this initial preview, you need to explicitly set a secret version, as we don’t yet have built-in rotation handling. This is something we look forward to making available as soon as we can.
User-assigned managed identities (public preview)
Our existing support for managed identities is called system-assigned. The idea is that the identity is created by the platform for a specific application and is tied to the lifecycle of the application. If you delete the application, the identity is removed from Azure Active Directory immediately.
Today we’re previewing user-assigned identities, which are created as their own Azure resource and then assigned to a given application. A user-assigned identity can also be assigned to multiple applications, and an application can have multiple user-assigned identities.
more details check this
Update
This is now GA
This was just released as preview a couple days ago.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
Using Keyvault integration within the function runtime
I just implemented it in Java following below two references.
https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
https://medium.com/statuscode/getting-key-vault-secrets-in-azure-functions-37620fd20a0b
in java use System.getenv("SECRET_KEY") to read the values from your app settings.
Happy to help if you need further assistance.
I have already given my answer in above, this answer is for #Matt Sanders's comment,
i just want to explain how MSI working in the Azure Environment,
"I have 2 user assigned identities, 1 has permissions to KeyVault, the other does not. How can you specify the correct user assigned an identity to use for retrieving the secret? I'm guessing this is not possible and User Assigned Identities are not supported even though they are listed in your answer. – Matt Sanders"
when you want to use Azure Manage Identity Service, your application must register in the Azure AD, for an example, lets say multiple users accessing your web application and, within your web application, you 'r trying to access vVault's secrets, In that case, vault doesnt care about the users that consume your application, it cares about the application,
please reffer below image,
I as showing the picture, only Azure Function added as an Identity to the vault, other applications are not,
so whoever using Azure function can access vault's secrets, according to this example only A and B can access secrets,

Resources