I'd like to use managed identities instead of username and password configured in the storage account connection. I only see the option to configure the queue trigger with a connection string, but can't add a managed identity to avoid secrets to be configured.
Is that possible at all?
I'm fairly certain it is not possible as of now, you can only use managed identities when the function runs to access resources, not for the trigger. I cannot dig up a proof for that right now, saw it on some GH issue.
This is now possible using the Microsoft Azure Function Extension Libraries, e.g. "Microsoft.Azure.WebJobs.Extensions.Storage.Blobs".
Exampleconnection configuration for managed identities:
"QueueSettings:StorageAccount": "",
"QueueSettings:StorageAccount__queueServiceUri": "https://mytestfa.queue.core.windows.net/",
"QueueSettings:StorageAccount__credential": "managedidentity"
And reference the connection in the function trigger like this:
[Function("ProcessUserData")]
public async Task ProcessUserData([QueueTrigger("%QueueSettings:UserDataQueue%", Connection = "QueueSettings:StorageAccount")] string queueItem, FunctionContext context)
{
var logger = context.GetLogger<QueueListener>();
...
}
Original announcement from the Microsoft DevBlog here:
https://devblogs.microsoft.com/azure-sdk/introducing-the-new-azure-function-extension-libraries-beta/
also reference here:
Azure Functions - use queue trigger with managed identity
Related
I need help in creating access to AZ a service bus without using thr Shared Access Key in an Azure Function.
When I use the Endpoint, SharedAccessKeyName, SharedAccessKey, QueueName and RuleName it works great!
I can create the connection string like this:
string connectionString = string.Format("Endpoint={0};SharedAccessKeyName={1};SharedAccessKey={2}", Endpoint, SharedAccessKeyName, SharedAccessKey);
Manager = new Azure.Messaging.ServiceBus.Administration.ServiceBusAdministrationClient(connectionString);
I was told that I can use the AZ Identity to accomplish this same thing without the SAS Key.
I have not been able to figure out how to do this. In fact, I do not even know where to begin.
You can make your app to run under a managed Identity, and then configure your Azure Service Bus resource to allow access for that identity.
See the documentation and the sample code here.
I was able to use the Key Vault inside a function app as described here but when I tried to use the Key Vault to hold the connection string of a function with a queue trigger I have issues with the storage account connection string. The function seems to find the parameter I provide but either doesn't get the secret back or doesn't like the information when it throws the error No valid combination of account information found.
My function is defined as:
[FunctionName("ReadQueueForMessage")]
public static async Task Run([QueueTrigger("%AzureQueueTrigger%", Connection = "AzureWebJobsStorage")] string myQueueItem,
Binder binderinputblob,
ILogger log)
This works fine if I just define the connection string in my local.settings.json. What I'm trying to do is instead of just putting the connection string in the json file, I want to point the function to the Key Vault with this syntax:
"AzureWebJobsStorage": "#Microsoft.KeyVault(SecretUri=https://myappkeyvault.vault.azure.net/secrets/myapp-AzureWebJobsStorage-Queue/the-guid-of-secret)",
I did go to the Key Vault and update the access policies to include the function app so it can read/list the secrets. The documentation here shows making an update to the configuration after it was deployed to Azure. I'm trying to test first in Visual Studio. Maybe that is the issue? Or is it not possible at all to use the secret in this manner?
I'm trying to test first in Visual Studio.
For now using Azure Key Vault references with Azure Functions does not support to work on local, as confirmed by Azure Functions team. If you still want to test on local, you could implemented an incomplete local workaround like this issue.
I test on portal and it works well. You could refer to the following steps as below：
1.In VS Function.cs, then publish to azure：
public static void Run([QueueTrigger("queue", Connection = "AzureWebJobsStorage")]string myQueueItem, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
string connectionString = System.Environment.GetEnvironmentVariable("AzureWebJobsStorage");
log.Info($"The connection string is {connectionString}");
}
2.Set AzureWebJobsStorage on Appsettings setting on portal.
3.Then it will work fine.
With the Nuget packages Azure.Extensions.AspNetCore.Configuration.Secrets and Azure.Identity you can now use the KeyVault as a configuration provider, the same way as in ASP.NET Core. You need to add a class derived from FunctionsStartup to add the KeyVault as a configuration provider, see Add FunctionsStartup class with the KeyVault as a configuration provider.
If you add the AzureWebJobsStorage connection string as a secret to the KeyVault, you can remove it from the Configuration section of your Function App in Azure. Make sure to turn on System assigned in the Identity section and in the KeyVault add an Access Policy with the Secret permissions Get and List for your Function App.
When you run your Function App local for debugging, Azure.Identity automatically uses your Microsoft Account for access to the KeyVault, if it has at least Get and List access to the secrets.
Unfortunately, when you test local, the Function App does NOT read AzureWebJobsStorage from the configuration/KeyVault, but requires it to be stored in local.settings.json. To prevent storing keys on your local computer, you can set AzureWebJobsStorage to "UseDevelopmentStorage=true" in local.settings.json.
For detailed instructions see: Create Azure Function App with Dependency Injection and the Key Vault as Configuration Provider.
Example project: https://github.com/Forestbrook/FunctionWithKeyVaultAndDI
When I develop for Azure I usually start copying in some keyvault client code so only keyvault urls will be in my settings file, no secrets can ever end up my git repositories.
After starting to make Azure functions I realized that it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
The recommended approach seems to connect the app to keyvault directly in Azure when deployed, and just manage secrets locally in Secret Manager, like suggested in
this article
I am not developing alone, so while I am not adverse to using a tool like Secret Manager, I need to still have my offline secrets connected to the Azure keyvault! If others change anything.
Question: How do I manage secrets offline in a way that is synchronized with Azure keyvault?
it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
In short, it's possible.
Here are steps you could follow and refer to the detailed article.
1.Add a System Assigned Managed Identity to the Azure Function.
2.Go to the Access Control section of your Key Vault and click on Add a role assignment blade.
3.Go to your Key Vault and click on Access Policies and then click on Add service principal with secret GET permission.
4.When you use ServiceBusTrigger, you set ServiceBusConnectionString in Function ->Configuration ->Application settings.
public static void Run([ServiceBusTrigger(_topicName, _subscriptionName, Connection = "ServiceBusConnectionString")] string mySbMsg, ILogger log)
{ ....
}
5.Now you change the value of ServiceBusConnectionString to the Azure Key Vault reference with #Microsoft.KeyVault(SecretUri=Secret URI with version). Then you could run your function successfully with Key Vault.
I was wondering if it's possible to initialize the queue trigger or even the blob trigger off a connection string that is read from azure vault.
Right now, we have to set these data connection via environment settings via blade properties. However, I wanted to just use the service principal to retrieve the token for the azure key vault to get all these connection strings.
I'm trying to figure how to get this working in java.
Thanks,
Derek
This feature is tracked and in progress here:
Feature request: retrieve Azure Functions' secrets from Key Vault
Add binding to Key Vault
EDIT 28/11/2018: It is currently in preview
Simplifying security for serverless and web apps with Azure Functions and App Service
Former answer 07/10/2018
This solution won't work for Triggers using the consumption plan.
In the mean time I did some research about your problem and it is possible to read config from key vault if you use Azure Function v2.
I've created an Azure Functions v2 (.NET Standard) from Visual Studio.
It uses:
NETStandard.Library v2.0.3
Microsoft.NET.Sdk.Functions v1.0.22
Microsoft.Azure.WebJobs v3.0.0
Microsoft.Azure.WebJobs.Extensions.Storage v3.0.0
Because Azure Functions v2 uses ASP.NET core, I was able to reference this link to configure my functions app to use Azure Key Vault:
Azure Key Vault configuration provider in ASP.NET Core
I've added this nuget package:
Microsoft.Extensions.Configuration.AzureKeyVault
I've configured my app to use this nuget package:
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Linq;
[assembly: WebJobsStartup(typeof(FunctionApp1.WebJobsExtensionStartup), "A Web Jobs Extension Sample")]
namespace FunctionApp1
{
public class WebJobsExtensionStartup : IWebJobsStartup
{
public void Configure(IWebJobsBuilder builder)
{
// Get the existing configuration
var serviceProvider = builder.Services.BuildServiceProvider();
var existingConfig = serviceProvider.GetRequiredService<IConfiguration>();
// Create a new config based on the existing one and add kv
var configuration = new ConfigurationBuilder()
.AddConfiguration(existingConfig)
.AddAzureKeyVault($"https://{existingConfig["keyVaultName"]}.vault.azure.net/")
.Build();
// replace the existing configuration
builder.Services
.Replace(ServiceDescriptor.Singleton(typeof(IConfiguration), configuration));
}
}
}
My Azure functions uses MSI:
I've granted Read/List secrets permissions to the function app on my key vault:
I have a small queue triggered function:
public static class Function2
{
[FunctionName("Function2")]
public static void Run([QueueTrigger("%queueName%", Connection = "queueConnectionString")]string myQueueItem, ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
}
}
The queueName is defined in the local.settings.json file (App settings blade once deployed):
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"keyVaultName": "thomastestkv",
"queueName": "myqueue"
}
}
The queueConnectionString is configured in my keyvault:
Sourcing Application Settings from Key Vault
The Key Vault references feature makes it so that your app can work as if it were using App Settings as they have been, meaning no code changes are required. You can get all of the details from our Key Vault reference documentation, but I’ll outline the basics here.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
That’s it! No changes to your code required!
For this initial preview, you need to explicitly set a secret version, as we don’t yet have built-in rotation handling. This is something we look forward to making available as soon as we can.
User-assigned managed identities (public preview)
Our existing support for managed identities is called system-assigned. The idea is that the identity is created by the platform for a specific application and is tied to the lifecycle of the application. If you delete the application, the identity is removed from Azure Active Directory immediately.
Today we’re previewing user-assigned identities, which are created as their own Azure resource and then assigned to a given application. A user-assigned identity can also be assigned to multiple applications, and an application can have multiple user-assigned identities.
more details check this
Update
This is now GA
This was just released as preview a couple days ago.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
Using Keyvault integration within the function runtime
I just implemented it in Java following below two references.
https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
https://medium.com/statuscode/getting-key-vault-secrets-in-azure-functions-37620fd20a0b
in java use System.getenv("SECRET_KEY") to read the values from your app settings.
Happy to help if you need further assistance.
I have already given my answer in above, this answer is for #Matt Sanders's comment,
i just want to explain how MSI working in the Azure Environment,
"I have 2 user assigned identities, 1 has permissions to KeyVault, the other does not. How can you specify the correct user assigned an identity to use for retrieving the secret? I'm guessing this is not possible and User Assigned Identities are not supported even though they are listed in your answer. – Matt Sanders"
when you want to use Azure Manage Identity Service, your application must register in the Azure AD, for an example, lets say multiple users accessing your web application and, within your web application, you 'r trying to access vVault's secrets, In that case, vault doesnt care about the users that consume your application, it cares about the application,
please reffer below image,
I as showing the picture, only Azure Function added as an Identity to the vault, other applications are not,
so whoever using Azure function can access vault's secrets, according to this example only A and B can access secrets,
So I'm fairly new to working with Azure and there are some things I can't quite wrap my head around. One of them being the Azure Storage Account.
My web jobs keeps stopping with the following error "Unhandled Exception: System.InvalidOperationException: The account credentials for '[account_name]' are incorrect." Understanding the error however is not the problem, at least that's what I think. The problem lies in understanding why I need an Azure Storage Account to overcome it.
Please read on as I try to take you through the steps taken thus far. Hopefuly the real question will become more clear to you.
In my efforts to deploy a WebJob on Azure we have created the following resources so far:
App Service Plan
App Service
SQL server
SQL database
I'm using the following code snippet to prevent my web job from exiting:
JobHostConfiguration config = new JobHostConfiguration();
config.DashboardConnectionString = null;
new JobHost(config).RunAndBlock();
To my understanding from other sources the Dashboard connection string is optional but the AzureWebJobsStorage connection string is required.
I tried setting the required connection string in portal using the configuration found here.
DefaultEndpointsProtocol=[http|https];AccountName=myAccountName;AccountKey=myAccountKey
Looking further I found this answer that clearly states where I would get the values needed, namely an/my missing Azure Storage Account.
So now for the actualy question: Why do I need an Azure Storage Account when I seemingly have all the resources I need place for the WebJob to run? What does it do? Is it a billing thing, cause I thought we had that defined in the App Service Plan. I've tried reading up on Azure Storage Accounts over here but I need a bit more help understanding how it relates to everything.
From the docs:
An Azure storage account provides resources for storing queue and blob data in the cloud.
It's also used by the WebJobs SDK to store logging data for the dashboard.
Refer to the getting started guide and documentation for further information
The answer to your question is "No", it is not mandatory to use Azure Storage when you are trying to setup and run a Azure web job.
If you are using JobHost or JobHostConfiguration then there is indeed a dependency for Storage accounts.
Sample code snippet is give below.
class Program
{
static void Main()
{
Functions.ExecuteTask();
}
}
public class Functions
{
[NoAutomaticTrigger]
public static void ExecuteTask()
{
// Execute your task here
}
}
The answer is no, you don't. You can have a WebJob run without being tied to an Azure Storage Account. Like Murray mentioned, your WebJob dashboard does use a storage account to log data but that's completely independent.