I have some integration tests that get the database connection string from the web.config. My web code and my test code are all in one project and this has worked great until now.
When deployed on app harbor app harbor replaces the value in the web.config, but when in the Visual Studio unit test environment the value is not being pulled.
Is there a way to pull the value from web.config when doing a unit test?
Here is my code:
private static string GetMongoDbConnectionString()
{
string con = ConfigurationManager.AppSettings.Get("MONGOHQ_URL") ??
ConfigurationManager.AppSettings.Get("MONGOLAB_URI") ??
"mongodb://www.fromCSFile/test";
return con;
}
Here is my web.config
<appSettings>
<add key="MONGOLAB_URI" value="mongodb://www.fromweb.config/test"/>
I wasn't able to get it to work having app harbor inject the correct config settings, but I got close.
I added an app.config to my web project because my web project has my integration tests in it.
I added the environment app setting to app.config:
<appSettings>
<add key="Environment" value="localconfig"/>
I noticed when that code ran on app harbor the environment value was Test:27017.
I wrote this code:
private static string GetMongoDbConnectionString()
{
string con = ConfigurationManager.AppSettings.Get("MONGOHQ_URL") ??
ConfigurationManager.AppSettings.Get("MONGOLAB_URI");
string env = ConfigurationManager.AppSettings.Get("Environment");
if (env.StartsWith("Test", StringComparison.OrdinalIgnoreCase))
{
con = "mongodb://xxxxxxxxx";
}
return con;
}
Where the xxxxxxxxx is the value I want to use at integration test time.
Related
EDIT: This question has been significantly restructured, now I've figured out a bit more of the problem and this should clarify things.
I am following this tutorial: https://learn.microsoft.com/en-us/azure/app-service/tutorial-dotnetcore-sqldb-app
I've deployed my own multiproject app, which works, but I can't get the connection string working properly. For some reason, it only works if I hardcode the connection string into the OnConfiguring method of my DBContext class. Otherwise, it throws an error.
Like so:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder
.UseSqlServer(
"Nasty hard coded azure connection string",
providerOptions => { providerOptions.EnableRetryOnFailure(); });
}
However, obviously, i want to get the connection string from a configuration file or environment variable.
Prior to deploying, I had the following. An extension method for IServiceColleciton which sets up the connection string:
public static void ConfigureSqlContext(this IServiceCollection services,
IConfiguration configuration) =>
services.AddDbContext<PeakedDbContext>(opts =>
opts.UseSqlServer(configuration.GetConnectionString("defaultConnection")));
then this method is called in program.cs. A pretty normal setup.
And I also set up an IDesignTimeDBContextFactory like so:
public class RepositoryContextFactory : IDesignTimeDbContextFactory<PeakedDbContext>
{
public PeakedDbContext CreateDbContext(string[] args)
{
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json")
.Build();
var builder = new DbContextOptionsBuilder<PeakedDbContext>()
.UseSqlServer(configuration.GetConnectionString("defaultConnection"));
return new PeakedDbContext(builder.Options);
}
}
Both my appsettings.json AND the Azure App Service configuration have the same name "defaultConnection".
As far as I can tell this is the approach recommended here: https://learn.microsoft.com/en-us/ef/core/cli/dbcontext-creation?tabs=dotnet-core-cli
I have also tried adding an empty constructor for my DBContext. (Not sure how this affects things as I have other DIs on my DBContext constructor. My DBContext consructors are getting a bit out of hand:
public PeakedDbContext()
{
}
public PeakedDbContext(DbContextOptions options) : base(options)
{
}
public PeakedDbContext(DbContextOptions options, ICurrentUserService currentUserService) : base(options)
{
_currentUserService = currentUserService;
}
According the the second link, above, I shouldn't need OnConfiguring method in my DBContext... and even if I do, what is the correct way to pass access to configuration.GetConnectionString, instead of hardcoding the connection string? Should I just add yet another DBContext constructor with the config injected? However, it ONLY works if I have the onconfiguring method. Neither the contextfactory nor the extension method setup are being used by azure app service.
Shouldn't it use the designtime factory or the hostconfiguration extension method I've set up above? What is the right way to use _configuration.GetConnectionString("defaultConnection") so that it works both locally and on Azure Deployment?
Update:
Still no luck. I tried adding the database connection string as an environment variable on azure like so:
and then updating all my reference to getconnection string - in program.cs, IDesignFactory and OnConfiguring - like so:
Environment.GetEnvironmentVariable("PeakedDbConn")
This continues to work locally. But When deploying to Azure it claims the connection string in null... so it's not seeing this variable. Nor can I find any code that will access the defaultConnection from the image. Which is strange, because it accesses the SECRET variable just fine.
I have followed the same code which you have provided with few changes.
Check the below steps to get the Connection string from appsettings.json and override the value if Azure App Connection String has been set.
As you are using .NET Core 6, I have set all the Configurations in Program.cs itself.
My Program.cs
builder.Services.AddDbContext<MyDatabaseContext>(options =>
options.UseSqlServer(builder.Configuration.GetConnectionString("MyDbConnection")));
builder.Configuration.AddEnvironmentVariables();
Same Connection String name must exist in both Local and Azure App Connection Strings.
My appsettings.json
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"ConnectionStrings": {
"MyDbConnection": "Dummy Connection String"
}
}
Azure App Connection String:
To check if we are getting the Connection String Value, I have written code in Controller.
private readonly IConfiguration Configuration;
public HomeController(ILogger<HomeController> logger,IConfiguration config)
{
_logger = logger;
Configuration = config;
}
public IActionResult Index()
{
var myconnStr = Configuration.GetConnectionString("MyDbConnection");
ViewBag.myconnStr = myconnStr;
return View();
}
My .csproj file :
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="7.0.2" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="7.0.2" />
<PackageReference Include="Microsoft.Extensions.Configuration.AzureAppConfiguration" Version="5.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="7.0.0" />
</ItemGroup>
</Project>
Local Output:
Deployed App Output:
Ok so it turned out to be GitHub actions that was the problem. The tutorial I followed in the first link doesn't mention this, perhaps because it's a single project api... not exactly sure.
The tutorial uses github actions to build and deploy the app, but for me, during the build it fails saying there is no connection string. This is because GitHub build process doesn't have access to either your local or the azure environment variable.
So I had to go into setting on my github repo and on the left click on secrets and variables < actions.
Click create a new Repository Secret, give it the same name as your environment variable i.e. PEAKEDDBCONN for me. Then, give it a value. I just used my local host string, but I guess you could type 'monkeynuts' in here if you wanted, it just needs to not be null.
Then you need to add a line to your workflow yaml file, the same one the tutorial talks about, to tell it about the environment variable. I added it like so:
jobs:
build:
runs-on: ubuntu-latest
env:
PEAKEDDBCONN: ${{ secrets.PEAKEDDBCONN }}
Then it all builds nicely and works.
For anybody who it helps, I decided to blog this whole setup:
https://garyfrewin.hashnode.dev/setting-up-an-entity-framework-core-web-api-on-azure-with-sqlserver-a-step-by-step-guide-for-hobby-projects
I developed a Azure Function in Visual Studio and it works when published in Azure (since a year ago).
Now I made a template from that Azure Function, in Visual Studio, and I changed some details, but it's mainly the same. When I test it locally, it works fine.
But when I publish it in Azure and try to test it, I get this error:
The ConnectionString property has not been initialized
I usually write the Connection String in the Application Settings (and it works well for the older Azure functions).
Here is how the function gets the value of the Connection String:
var repo = new GranularRepository(ConfigurationManager.AppSettings["BoConnectionString"]);
I also tried:
var repo = new AvgDeliveryTime_GranularRepository(Environment.GetEnvironmentVariable("BodbConnectionString"));
According to your description, I could not distinguish your function runtime.
For v1:
You could use both ConfigurationManager.AppSettings and System.Environment.GetEnvironmentVariable to get your connection string in Azure.
var a = ConfigurationManager.AppSettings["BoConnectionString"];
var b = System.Environment.GetEnvironmentVariable("BoConnectionString");
For v2:
You could use System.Environment.GetEnvironmentVariable and ConfigurationBuilder to get it. Add ExecutionContext parameter, which is used to locate function app directory.
var a= System.Environment.GetEnvironmentVariable("BoConnectionString");
/////////////
public static void Run(...,ExecutionContext context)
{
var config = new ConfigurationBuilder()
.SetBasePath(context.FunctionAppDirectory)
.AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();
// Get Connection strings
var connParameter= "MySqlAzureConnection";
string connectionString = config.GetConnectionString($"{connParameter}");
}
Both of v1 and v2 you set the connection string in application settings.
For more details, you could refer to this issue.
I fixed it by creating a new Azure Function project.
So my conclusion is that making a template from another Azure Function and using that as the new project, in Visual Studio, is creating issues.
As described in this article: https://azure.microsoft.com/en-us/blog/windows-azure-web-sites-how-application-strings-and-connection-strings-work/, Azure Web Apps/Web Sites/Web Jobs can take their configuration settings (appSettings, connectionString) from environment variables instead of app.config/web.config.
For example, if an environment variable named "APPSETTING_appSettingKey" exists, it will override the following setting from app.config/web.config:
<appSettings>
<add key="appSettingKey" value="defaultValue" />
</appSettings>
This works fine once the application is deployed in Azure, but I would like to use the same method when testing locally.
I tried to emulate this in a local command line:
> set APPSETTING_appSettingKey=overridedValue
> MyWebJob.exe
The web job accesses this setting using:
ConfigurationManager.AppSettings["appSettingKey"]
When running in Azure, it reads the value "overridedValue" as expected, but locally it reads the value "defaultValue" from the app.config file.
Should I expect this to work, or is this implemented only under an Azure environment?
I could obviously create an abstraction over ConfigurationManager that emulates this, but this wouldn't work when calling code that needs a connection string name instead of a connection string value. Also, I want to use the same method regardless of the environment to simplify management of settings.
There are 3 reasons why I need this:
1) I don't like the idea of deploying to production a web.config file that references connection strings, etc for a developement environment, because there's a risk of an error that would cause the development settings (in web.config) to be used in production (production web app connecting to development database, etc), for example if an environment variable is named incorrectly (after renaming the setting in web.config but forgetting to rename it in environment variables)
2) I'm trying to setup development environments where each developer has his own isolated cloud resources (storage account, databases,...). Currently, everyone has to manually edit his .config files to reference the correct resources, and be careful when checking-in or merging changes to these files.
3) A solution can have multiple projects that need to duplicate the same settings (main web app, web jobs, integration test projects,...). This causes a lot of work to ensure updated settings are replicated across all files.
This would be simplified if there was an environment-independent .config file without any actual configuration, each developer would configure a set of environment variables once and be able to use them for all parts of a solution.
Yes, this special transformation of environment variables into config values is done via a component that is specific to Azure WebApps and won't be in play locally.
Generally people are fine with the local behavior this produces - locally you are reading from config settings as usual, but in Azure you're reading from secure settings that were configured via the App Settings portal blade (so these settings aren't in your source code).
You could write an abstraction over this if you wish, E.g. the WebJobs SDK actually does this internally (code here).
When I am developing locally and want to consistantly use Environment.GetEnvironmentVariable. In my static class Main I have the following code:
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
Environment.SetEnvironmentVariable("UseDevelopmentSettings", "true");
}
Then in my static class Functions I add a static constructor and in there I call the static method below:
static void AddAppSettingsToEnvironmentVariables()
{
String useDevelopmentSettings = Environment.GetEnvironmentVariable("UseDevelopmentSettings"); ;
if (!(String.IsNullOrEmpty(useDevelopmentSettings)))
{
foreach (String key in ConfigurationManager.AppSettings.AllKeys)
{
Environment.SetEnvironmentVariable(key, ConfigurationManager.AppSettings[key]);
}
}
}
The code is small enough that I can simply comment it out before I test in Azure.
If you want to test the application with the value that will be used in Azure portal AppSettings/Connection String. I would recommend use HostingEnvironment.IsDevelopmentEnvironment. To ensure it will work, please change the <compilation debug="true" targetFramework="4.5.2" /> to <compilation debug="false" targetFramework="4.5.2" />. set the value with the same value in Azure portal if (HostingEnvironment.IsDevelopmentEnvironment == false). I have try with a simple project, hope it helps:
public ActionResult Index()
{
if (HostingEnvironment.IsDevelopmentEnvironment == true)
{
ViewBag.Message = "Is development.";
}
else
{
ViewBag.Message = "Azure environment.";
}
return View();
}
Here is the result:
I recently deployed an ASP.Net Web API project to our Azure App Service test slot but started receiving an error when making requests to the API endpoints. Through remote debugging, it became clear that the app was extracting my dev connection strings from the deployed web.config file.
The connection strings are supposed to come from the Application Settings we set up via the Azure Portal - and, in previous deployments, they were - but that's not the case.
Why would this happen and what can be done to ensure the correct behaviour occurs? We absolutely don't want our production database secrets being put into GIT via the web.config...
I recently experienced the same problem and fixed it:
In Azure App Services, the machine-wide web.config file is located at D:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\web.config.
This file differs to a normal machine-wide web.config file because it has this extra element:
<system.web>
...
<compilation>
<assemblies>
<add assembly="EnvSettings, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
...
</assemblies>
</compilation>
</system.web>
The EnvSettings.dll assembly is located inside D:\Program Files\IIS\Microsoft Web Hosting Framework (unfortunately this directory is access-controlled and I can't get into it).
But EnvSettings.dll is mirrored in the GAC, so I was able to copy it from there.
Inside EnvSettings.dll is an [assembly: PreApplicationStartMethod] attribute that runs a method in EnvSettings.dll which copiesthe APPSETTING_ and database connection-string settings from Environment Variables into the .NET Framework ConfigurationManager.AppSettings and ConfigurationManager.ConnectionStrings collections.
Note this copying only happens once (during application startup), and only in the first AppDomain - so if you have other AppDomain instances in your application they won't see the updated ConfigurationManager.
Therefore, if you see that your Azure Portal configuration settings for your App Service are not being used when you dump your ConfigurationManager, then the following is likely happening:
You used <clear /> in your <compilation><assemblies> element, which stops EnvSettings.dll from being loaded at all.
In which case you need to either add back the <add assembly="EnvSettings... element from above to your own web.config, or find some other way to load it.
I don't recommend saving EnvSettings.dll locally and adding an assembly reference to your project, as EnvSettings.dll is part of the Microsoft Web Hosting Framework.
Or you have code that is clearing or resetting ConfigurationManager after EnvSettings populates it for you.
Or something else is going on that I have no idea about!
As an alternative to having EnvSettings.dll copy your settings over, another option is to copy the environment-variables over yourself - and as you control the code that does this it means you can call it whenever you need to (e.g. if you ever reset them).
Here's the code I used:
public static class AzureAppSettingsConfigurationLoader
{
public static void Apply()
{
foreach( DictionaryEntry environmentVariable in Environment.GetEnvironmentVariables() )
{
String name = (String)environmentVariable.Key;
String value = (String)environmentVariable.Value;
if( name.StartsWith( "APPSETTING_", StringComparison.OrdinalIgnoreCase ) )
{
String appSettingName = name.Substring( "APPSETTING_".Length );
ConfigurationManager.AppSettings[ appSettingName ] = value;
}
else if( name.StartsWith( "SQLAZURECONNSTR_", StringComparison.OrdinalIgnoreCase ) )
{
String csName = name.Substring( "SQLAZURECONNSTR_".Length );
ConfigurationManager.ConnectionStrings.Add( new ConnectionStringSettings( csName, value, providerName: ""System.Data.SqlClient" ) );
}
}
}
}
See my sample here: http://mvc5appsettings.azurewebsites.net/
// My web.config also has a "HERO_TEXT" key in
// that reads "Value from web.config"
string hero = ConfigurationManager.AppSettings["HERO_TEXT"];
Wiki page on App Settings for .NET:
https://github.com/projectkudu/kudu/wiki/Managing-settings-and-secrets
As already mentioned here, make sure you have that App Setting in the right slot.
As I know, the settings in Azure portal will override existing setting in Web.config. So If you want to ignore the Azure Application settings in portal and use Web.config instead. I am afraid you need to configure the settings in web.config, and remove the same key/pair in Azure portal.
I've got an azure webjob, that has some appsettings for api keys etc.
I've also got a bunch of PRODUCTION azure app settings (specified in the portal), that should override my webjob config appsettings.. But they don't.
For my website, they work as expected, and all is fine. For the webjobs however, they just get completely ignored, and my app settings from the config are used instead.
Is this a bug in azure? All the docs suggest that this should work.
EDIT
I found this blog all about using CloudConfigurationManager.GetSetting so I've implemented that and it still won't work - still using the settings that are defined in web job's app settings file :(
Thanks
So it looks like I've found a bug!
It's because my AppSettings were referenced from another file, like this (in app.config):
<AppSettings file="appsettings.config"/>
This basically breaks azure's config management.
Use
"D:\home\site\wwwroot\" to create an absolute path to files inside your website.
inside your WebJob to point to any file inside your wwwwroot directory. Using the Absolute path with D:\home\site\wwwroot\appsettings.config should fix you website.
Did you set your settings only in the Web.config file or did you set them in the portal or using the VS "Website Settings" tooling? You need to set the WebJobs-related settings at that level, not just in the Web.config. Once you do that, restart your site, and the settings should be there.
I had a similar problem but for me it was that we set the "is_in_place" settings.job value to true. I guess when it's in place, it doesn't update the config file with the settings. We didn't have a strict requirement that it run in place, so removing that setting fixed the problem for us.
I have a similar problem with several web jobs sharing a common settings file. It would be inconvenient to duplicate the settings in each job.
Someone mentioned using configSource= instead of file= in the app.config to reference the external config settings file. I tried this, and it appears that it now works as expected. The settings are being taken from the Portal App Settings instead of the file now.
Fortunately, the settings for the web jobs all come from the external file. The use of file= allowed me to use additional per-app settings other than just those in the file, but fortunately I don't need to do that.
A workaround to this problem is to place the settings inline in the AppSettings tag of the Web.config like this.
<AppSettings>
<add key="host" value="someHost" />
</AppSettings>
Azure App Service Application Settings are persisted in the Environment Variables. Go to the Azure Portal add your application settings or connection string values. Then, you can check out the environment variables of your app service at https://sitename.scm.azurewebsites.net
If you want your webjobs to share those settings, in your webjob project, create a appsettings.json:
{
"APPSETTING_AzureWebJobsDashboard": "",
"APPSETTING_AzureWebJobsStorage": "",
"SQLAZURECONNSTR_xxx": ""
}
In your Main() method
private static void Main(string[] args)
{
var builder = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables();
var configurations = builder.Build();
var services = new ServiceCollection()
.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(configurations["SQLAZURECONNSTR_xxx"]),
ServiceLifetime.Transient)
.BuildServiceProvider();
var host = new JobHost(new JobHostConfiguration
{
DashboardConnectionString = configurations["APPSETTING_AzureWebJobsDashboard"],
StorageConnectionString = configurations["APPSETTING_AzureWebJobsStorage"]
});
host.RunAndBlock();
}
This is simpler than trying to figure out the path to the website config on the app service