I've got a Kentico Xperience (v13) instance in Azure and I want to run ContinuousIntegration.exe to populate my database up there with content from my CI xml files. The catch is that we're injecting the CMSConnectionString setting into the web app from Azure Key Vault (AKV) and the CI.exe isn't seeing it. Instead I get this error message:
CMS.DataEngine.ApplicationInitException: Cannot access the database specified by the 'CMSConnectionString' connection string. Please install the database externally and set a correct connection string.
Or maybe this error message:
Failed to execute the command.
Here's the relevant section from our web.config (that works for the website!):
<connectionStrings>
<!--Should be provided by Azure Key Vault-->
</connectionStrings>
How do I ensure that the executable gets access to the secrets in AKV?
It is possible to let ContinuousIntegration.exe know about a secure connection string with a small custom module that sets the connection string at startup. Here is the basic code of the module:
[assembly: AssemblyDiscoverable]
[assembly: RegisterModule(typeof(AzureConnectionStringModule))]
public class AzureConnectionStringModule : Module
{
public AzureConnectionStringModule()
: base(nameof(AzureConnectionStringModule))
{
}
protected override void OnPreInit()
{
base.OnPreInit();
var azureConnectionString = Environment.GetEnvironmentVariable("SQLAZURECONNSTR_CMSConnectionString");
if (string.IsNullOrWhiteSpace(azureConnectionString))
{
azureConnectionString = Environment.GetEnvironmentVariable("CMSConnectionString");
}
if (!string.IsNullOrWhiteSpace(azureConnectionString))
{
SettingsHelper.ConnectionStrings.SetConnectionString("CMSConnectionString", azureConnectionString);
}
}
}
From a fresh installation of Kentico Xperience 13, here are steps to configure this:
Follow the steps here to add Key Vault support to the admin app locally: https://learn.microsoft.com/en-us/azure/key-vault/general/vs-key-vault-add-connected-service.
Add the module above to the solution in a class library. Make sure the main project references the class library so that it is included during building.
Ensure that the ~\web.config does not have a connection string, or an app setting, named CMSConnectionString.
Deploy the app to an App Service.
In Azure, create an App Service configuration setting with name CMSConnectionString and value #Microsoft.KeyVault(VaultName=your-keyvault;SecretName=CMSConnectionString).
In the Key Vault, create a secret with name CMSConnectionString and value a connection string to an Azure SQL database. You may also need to follow https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references to create an access policy for my App Service.
At this point, the Kentico Xperience 13 admin should load with access to the database.
In the App Service portal, under Development Tools select Console.
In the console, run cd bin and then ContinuousIntegration.exe -r. This should produce a message about the repository not being configured, or output on the restore action.
Related
I have followed the following for How to reference Key Vault connection strings from an Azure App Service.
https://davecallan.com/how-to-reference-key-vault-connection-strings-from-an-azure-app-
service/
The connection string has a name of "sqlconnection" in the App Service Configuration. I then deleted this entry from the Configuration Json file and restarted the Web app.
I then try to connect but it gives a database error with no details.
Any ideas on this ?
I tried to reproduce the same in my environment like below:
I created a key vault with Azure role-based access control (RBAC) like below:
Make sure to add Key Vault Administrator role in Access control like below:
Generate a key vault secret under setting. While generating secret of type as Manual name and set the value to the current SQL Database connection string stored in the Application web.config
Try to use Secret value connection string of your sql database like below:
Then you can able to see Secret Identifier like below:
In your web app -> under setting click configuration -> click new connection string
Type the name of the connection string and set the value #Microsoft.KeyVault(SecretUri=VALUE), where the VALUE
In Identity under setting -> make sure to add system assigned status as on like below:
Click on the "Role Assignments" button and then Add keyvault secrets user role like below:
As you are facing database error with no details you can check the Log4net errors under Portals_default\Logs.
Reference:
Connection strings and app settings not working properly on Azure App Service · Issue #4227 · dnnsoftware/Dnn.Platform · GitHub
Is there a special way to define a Key Value setting for ConnectionStrings in Azure App Configuration?
I have tried using:
ConnectionStrings:DatabaseKeyName
ConnectionStrings\DatabaseKeyName
Using the standard builder.Configuration.GetConnectionString("DatabaseKeyName") always results in a null value. Using builder.Configuration["ConnectionStrings:DatabaseKeyName"] also results in null, however if I use a keyname that does not start with ConnectionStrings (e.g. Test:ConnectionStrings:DatabaseKeyName it works as an app setting via builder.Configuration["Test:ConnectionStrings:DatabaseKeyName"]
The Null value for ConnectionStrings:DatabaseKeyName indicates there is some special handling for ConnectionStrings in Azure App Config, but I don't know where I am going wrong. The Microsoft example pages don't seem to cover ConnectionStrings (except via KeyVault).
Basically I do not want to have to change this:
services.AddDbContext<IciContext>(o =>
{
o.UseSqlServer(Configuration.GetConnectionString("DatabaseKeyName"));
});
To this:
services.AddDbContext<IciContext>(o =>
{
o.UseSqlServer(builder.Configuration["DatabaseKeyName"]);
});
Standard app config connection string setting I need to simulate from Azure App Config:
{
"ConnectionStrings": {
"DatabaseKeyName": "Data Source=localhost;Initial Catalog=xxxx;Integrated Security=True"
},
In my secrets file it is in this format (which does not work with Azure App Config):
{
"ConnectionStrings:DatabaseKeyName": "Server=xxxx;Database=xxxx;User ID=xxxx;Password=xxxx"
}
To get the Connection String from Azure App Configuration, please check the below process.
Install the NuGet Package Microsoft.Azure.AppConfiguration.AspNetCore latest version to add the AddAzureAppConfiguration and read the key values.
To read Azure App Configuration locally, we need to set the secret manager to store the connection string.
dotnet user-secrets init
The above command enables the secret storage and sets the secret ID in .csproj of your application.
In Program.cs, add the below code
var myAppConn= builder.Configuration.GetConnectionString("AppConfig");
Output:
As mentioned in the MSDoc, For the Apps deployed in Azure App Service it is recommended to store Connection String in Configuration Section => Application Settings => Connection Strings of the deployed App.
Is there a special way to define a Key Value setting for ConnectionStrings in Azure App Configuration?
In Azure App Configuration => *YourAppConfiguration* => click on Configuration explorer Under Operations => click on Create => Key-value
In Program.cs, add the below code
var myconn = builder.Configuration.GetConnectionString("AppConfig");
builder.Host.ConfigureAppConfiguration(builder =>
{
builder.AddAzureAppConfiguration(myconn);
})
.ConfigureServices(services =>
{
services.AddControllersWithViews();
});
In any of the cshtml file, add the below code
#using Microsoft.Extensions.Configuration
#inject IConfiguration Configuration
<h1>#Configuration["MyConnection"]</h1>
Output for Key-Value from AppConfiguration:
I was able to use the Key Vault inside a function app as described here but when I tried to use the Key Vault to hold the connection string of a function with a queue trigger I have issues with the storage account connection string. The function seems to find the parameter I provide but either doesn't get the secret back or doesn't like the information when it throws the error No valid combination of account information found.
My function is defined as:
[FunctionName("ReadQueueForMessage")]
public static async Task Run([QueueTrigger("%AzureQueueTrigger%", Connection = "AzureWebJobsStorage")] string myQueueItem,
Binder binderinputblob,
ILogger log)
This works fine if I just define the connection string in my local.settings.json. What I'm trying to do is instead of just putting the connection string in the json file, I want to point the function to the Key Vault with this syntax:
"AzureWebJobsStorage": "#Microsoft.KeyVault(SecretUri=https://myappkeyvault.vault.azure.net/secrets/myapp-AzureWebJobsStorage-Queue/the-guid-of-secret)",
I did go to the Key Vault and update the access policies to include the function app so it can read/list the secrets. The documentation here shows making an update to the configuration after it was deployed to Azure. I'm trying to test first in Visual Studio. Maybe that is the issue? Or is it not possible at all to use the secret in this manner?
I'm trying to test first in Visual Studio.
For now using Azure Key Vault references with Azure Functions does not support to work on local, as confirmed by Azure Functions team. If you still want to test on local, you could implemented an incomplete local workaround like this issue.
I test on portal and it works well. You could refer to the following steps as below：
1.In VS Function.cs, then publish to azure：
public static void Run([QueueTrigger("queue", Connection = "AzureWebJobsStorage")]string myQueueItem, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
string connectionString = System.Environment.GetEnvironmentVariable("AzureWebJobsStorage");
log.Info($"The connection string is {connectionString}");
}
2.Set AzureWebJobsStorage on Appsettings setting on portal.
3.Then it will work fine.
With the Nuget packages Azure.Extensions.AspNetCore.Configuration.Secrets and Azure.Identity you can now use the KeyVault as a configuration provider, the same way as in ASP.NET Core. You need to add a class derived from FunctionsStartup to add the KeyVault as a configuration provider, see Add FunctionsStartup class with the KeyVault as a configuration provider.
If you add the AzureWebJobsStorage connection string as a secret to the KeyVault, you can remove it from the Configuration section of your Function App in Azure. Make sure to turn on System assigned in the Identity section and in the KeyVault add an Access Policy with the Secret permissions Get and List for your Function App.
When you run your Function App local for debugging, Azure.Identity automatically uses your Microsoft Account for access to the KeyVault, if it has at least Get and List access to the secrets.
Unfortunately, when you test local, the Function App does NOT read AzureWebJobsStorage from the configuration/KeyVault, but requires it to be stored in local.settings.json. To prevent storing keys on your local computer, you can set AzureWebJobsStorage to "UseDevelopmentStorage=true" in local.settings.json.
For detailed instructions see: Create Azure Function App with Dependency Injection and the Key Vault as Configuration Provider.
Example project: https://github.com/Forestbrook/FunctionWithKeyVaultAndDI
I developed a cron trigger azure fuction who needs to search for soe data in my database.
Localy i can connect whit sql server, so i change the connection string in loca.settings.json to connect in azure sql and published the function, but the function cant connect with database.
I need to do something more than configure the local.settings.json?
The local.settings.json is only used for local testing. It's not even exported to azure.
You need to create a connection string in your application settings.
In Azure Functions - click Platform features and then Configuration.
Set the connection string
A function app hosts the execution of your functions in Azure. As a best security practice, store connection strings and other secrets in your function app settings. Using application settings prevents accidental disclosure of the connection string with your code. You can access app settings for your function app right from Visual Studio.
You must have previously published your app to Azure. If you haven't already done so, Publish your function app to Azure.
In Solution Explorer, right-click the function app project and choose Publish > Manage application settings.... Select Add setting, in New app setting name, type sqldb_connection, and select OK.
Application settings for the function app.
In the new sqldb_connection setting, paste the connection string you copied in the previous section into the Local field and replace {your_username} and {your_password} placeholders with real values. Select Insert value from local to copy the updated value into the Remote field, and then select OK.
Add SQL connection string setting.
The connection strings are stored encrypted in Azure (Remote). To prevent leaking secrets, the local.settings.json project file (Local) should be excluded from source control, such as by using a .gitignore file.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-scenario-database-table-cleanup
If you are using entity framework core to make a connection, Other Way of connection to SQL is by using dependency injection from .netcore library.
You can keep the connection string in Azure Key-vault or the config file from there you can read the same using azure function startup class. which need below code setup in your function app.
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
[assembly: FunctionsStartup(typeof( TEST.Startup))]
namespace TEST
{
internal class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
Contract.Requires(builder != null);
builder.Services.AddHttpClient();
var configBuilder = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
.AddAzureKeyVault($"https://XYZkv.vault.azure.net/");
var configuration = configBuilder.Build();
var conn = configuration["connectionString"];
builder.Services.AddDbContext<yourDBContext>(
options => options.UseSqlServer(configuration["connectionString"]));
}
}
}
after that where ever you are injecting this dbcontext, with context object you can do all CRUD operations by following microsoft's entity framework core library documentation.
Having just dealt with this beast (using a custom handler with Linux), I believe the simple way is to upgrade your App to premium-plan, allowing you to access the "Networking" page from "App Service plans". This should allow you to put both sql-server and app in the same virtual network, which probably makes it easier. (but what do I know?)
Instead, if you don't have the extra cash laying around, you can try what I did, and set up a private endpoint, and use the proxy connection setting for your database:
Create a virtual network
I used Address space: 10.1.0.0/16 (default I think)
Add subnet 10.1.0.0/24 with any name (adding a subnet is required)
Go to "Private link center" and create a private endpoint.
any name, resource-group you fancy
use resource type "Microsoft.Sql/Server" and you should be able to select your sql-server (which I assume you have created already) and also set target sub-resource to "sqlServer" (the only option)
In the next step your virtual network and submask should be auto-selected
set Private DNS integration to yes (or suffer later).
Update your firewall by going to Sql Databases, select your database and click "Set Server Firewall" from the overview tab.
Set Connection Policy to proxy. (You either do this, or upgrade to premium!)
Add existing virtual network (rule with any name)
Whitelist IPs
There probably is some other way, but the azure-cli makes it easy to get all possible IP's your app might use: az functionapp show --resource-group <group_name> --name <app_name> --query possibleOutboundIpAddresses
https://learn.microsoft.com/en-us/azure/app-service/overview-inbound-outbound-ips
whitelist them all! (copy paste exercise)
Find your FQDN from Private link center > Private Endpoints > DNS Configuration. It's probably something like yourdb.privatelink.database.windows.net
Update your app to use this url. You just update your sql server connection string and replace the domain, for example as ADO string: Server=tcp:yourdb.privatelink.database.windows.net,1433;Initial Catalog=somedbname;Persist Security Info=False;User ID=someuser;Password=abc123;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=True;Connection Timeout=30;
Also note that I at some point during all of this I switched to TrustServerCertificate=True and now I can't bother to figure out if it does a difference or not. So I left it as an exercise to the reader to find out.
So what we have done here...?
We have forced your function app to go outside the "azure-sphere" by connecting to the private endpoint. I think that if you bounce between azure-services directly, then you'll need some sort of authentication (like logging in to your DB using AD), and in my case, using custom handler and linux base for my app, I think that means you need some trust negotiation (kerberos perhaps?). I couldn't figure that out, so I came up with this instead.
I was wondering if it's possible to initialize the queue trigger or even the blob trigger off a connection string that is read from azure vault.
Right now, we have to set these data connection via environment settings via blade properties. However, I wanted to just use the service principal to retrieve the token for the azure key vault to get all these connection strings.
I'm trying to figure how to get this working in java.
Thanks,
Derek
This feature is tracked and in progress here:
Feature request: retrieve Azure Functions' secrets from Key Vault
Add binding to Key Vault
EDIT 28/11/2018: It is currently in preview
Simplifying security for serverless and web apps with Azure Functions and App Service
Former answer 07/10/2018
This solution won't work for Triggers using the consumption plan.
In the mean time I did some research about your problem and it is possible to read config from key vault if you use Azure Function v2.
I've created an Azure Functions v2 (.NET Standard) from Visual Studio.
It uses:
NETStandard.Library v2.0.3
Microsoft.NET.Sdk.Functions v1.0.22
Microsoft.Azure.WebJobs v3.0.0
Microsoft.Azure.WebJobs.Extensions.Storage v3.0.0
Because Azure Functions v2 uses ASP.NET core, I was able to reference this link to configure my functions app to use Azure Key Vault:
Azure Key Vault configuration provider in ASP.NET Core
I've added this nuget package:
Microsoft.Extensions.Configuration.AzureKeyVault
I've configured my app to use this nuget package:
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Linq;
[assembly: WebJobsStartup(typeof(FunctionApp1.WebJobsExtensionStartup), "A Web Jobs Extension Sample")]
namespace FunctionApp1
{
public class WebJobsExtensionStartup : IWebJobsStartup
{
public void Configure(IWebJobsBuilder builder)
{
// Get the existing configuration
var serviceProvider = builder.Services.BuildServiceProvider();
var existingConfig = serviceProvider.GetRequiredService<IConfiguration>();
// Create a new config based on the existing one and add kv
var configuration = new ConfigurationBuilder()
.AddConfiguration(existingConfig)
.AddAzureKeyVault($"https://{existingConfig["keyVaultName"]}.vault.azure.net/")
.Build();
// replace the existing configuration
builder.Services
.Replace(ServiceDescriptor.Singleton(typeof(IConfiguration), configuration));
}
}
}
My Azure functions uses MSI:
I've granted Read/List secrets permissions to the function app on my key vault:
I have a small queue triggered function:
public static class Function2
{
[FunctionName("Function2")]
public static void Run([QueueTrigger("%queueName%", Connection = "queueConnectionString")]string myQueueItem, ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
}
}
The queueName is defined in the local.settings.json file (App settings blade once deployed):
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"keyVaultName": "thomastestkv",
"queueName": "myqueue"
}
}
The queueConnectionString is configured in my keyvault:
Sourcing Application Settings from Key Vault
The Key Vault references feature makes it so that your app can work as if it were using App Settings as they have been, meaning no code changes are required. You can get all of the details from our Key Vault reference documentation, but I’ll outline the basics here.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
That’s it! No changes to your code required!
For this initial preview, you need to explicitly set a secret version, as we don’t yet have built-in rotation handling. This is something we look forward to making available as soon as we can.
User-assigned managed identities (public preview)
Our existing support for managed identities is called system-assigned. The idea is that the identity is created by the platform for a specific application and is tied to the lifecycle of the application. If you delete the application, the identity is removed from Azure Active Directory immediately.
Today we’re previewing user-assigned identities, which are created as their own Azure resource and then assigned to a given application. A user-assigned identity can also be assigned to multiple applications, and an application can have multiple user-assigned identities.
more details check this
Update
This is now GA
This was just released as preview a couple days ago.
This feature requires a system-assigned managed identity for your app. Later in this post I’ll be talking about user-assigned identities, but we’re keeping these previews separate for now.
You’ll then need to configure an access policy on your Key Vault which gives your application the GET permission for secrets. Learn how to configure an access policy.
Lastly, set the value of any application setting to a reference of the following format:
#Microsoft.KeyVault(SecretUri=secret_uri_with_version)
Where secret_uri_with_version is the full URI for a secret in Key Vault. For example, this would be something like: https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931
Using Keyvault integration within the function runtime
I just implemented it in Java following below two references.
https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references
https://medium.com/statuscode/getting-key-vault-secrets-in-azure-functions-37620fd20a0b
in java use System.getenv("SECRET_KEY") to read the values from your app settings.
Happy to help if you need further assistance.
I have already given my answer in above, this answer is for #Matt Sanders's comment,
i just want to explain how MSI working in the Azure Environment,
"I have 2 user assigned identities, 1 has permissions to KeyVault, the other does not. How can you specify the correct user assigned an identity to use for retrieving the secret? I'm guessing this is not possible and User Assigned Identities are not supported even though they are listed in your answer. – Matt Sanders"
when you want to use Azure Manage Identity Service, your application must register in the Azure AD, for an example, lets say multiple users accessing your web application and, within your web application, you 'r trying to access vVault's secrets, In that case, vault doesnt care about the users that consume your application, it cares about the application,
please reffer below image,
I as showing the picture, only Azure Function added as an Identity to the vault, other applications are not,
so whoever using Azure function can access vault's secrets, according to this example only A and B can access secrets,