I'm working against Azure websites where the portal can be used to over-ride values in the web.config's AppSetting and ConnectionString sections. But other sections still require a transform or hard-coding of the web.config. I need to apply the same control over values found at /configuration/system.net/mailSettings/smtp
Ideally, I'd like to store a string in an AppSetting value and then, when the site boots, it parses this string and over-writes the initial contents of my SMTP section with parsed values found in AppSettings.
Big picture objective: eliminate reliance on hard-coded smtp values of the web.config and instead, apply those values from the portal's listing of AppSettings. Possible?
thx
I'm not too familiar with System.Net.Mail, but you can always read and initialize your own config. You can set the following AppSettings in the portal
smtp.host = <hostName>
smtp.port = <port>
smtp.userName = <userName>
smtp.password = <password>
smtp.defaultCredentials = <true | false>
and then have something like that in your code.
public SmtpClient GetSmtpClient()
{
Func<string, string> config = s => ConfigurationManager.AppSettings[s] ?? System.Environment.GetEnvironmentVariable(s);
return new SmtpClient
{
Host = config("smtp.host"),
Port = int.Parse(config("smtp.port")), // handle parsing errors
Credentials = new NetworkCredential(config("smtp.userName"), config("smtp.userName")),
UseDefaultCredentials = bool.Parse(config("smtp.defaultCredentials")), //handle parsing errors
};
}
then when debugging locally you can rely on Environment Variables and when deploying to Azure you use the AppSettings.
Related
I'm using Pulumi to deploy several Azure ressources, which works fine.
I'm deploying a TopicAuthorizationRule and I need to manipulate the connection string in order to have it working with an Azure Function Trigger.
const myPolicy = new azure.eventhub.TopicAuthorizationRule(...);
const myPolicyConnectionString = myPolicy.primaryConnectionString.get();
const goodConnectionString = myPolicyConnectionString .substr(0, myPolicyConnectionString .lastIndexOf(';EntityPath'));
And I have this error: Cannot call '.get' during update or preview
How can I do this string manipulation in order to set it in AppSettings?
Connection string value is unknown yet at the time of preview, so you can't use it directly. It's contained in a value of type Output<T> which is going to be resolved at update time.
You can transform the values of Output<T> by using apply function:
const goodConnectionString =
myPolicy.primaryConnectionString.apply(s => s.substr(0, s.lastIndexOf(';EntityPath'));
which can then be used to assign AppSettings (without calling get explicitly).
Github link for reproduction.
I have an ASP.NET Core (RC1) application that works fine locally. The issue I'm having is that my connection string is not being picked up by my Azure app. I've asked similar questions to this, but I've narrowed down the issue on my end in this app. Note, it requires an app on Azure to reproduce it.
Here's the issue I'm seeing.
First, my configuration is setup as such:
public Startup()
{
var builder = new ConfigurationBuilder()
.AddJsonFile("config.json")
.AddEnvironmentVariables();
mConfiguration = builder.Build();
}
And EF7 is setup here:
public void ConfigureServices(IServiceCollection services)
{
services.AddEntityFramework()
.AddSqlServer()
.AddDbContext<FooDbContext>(options =>
{
// I'm assuming it's failing here.
// I'm not sure how to debug it running on Azure.
// All the developer exception page shows is:
// 500 Internal Server Error "An error occurred while starting the application."
options.UseSqlServer(mConfiguration["Data:ConnectionStringTest:ConnectionString"]);
});
services.AddScoped<IFooDataService, FooSqlDataService>();
}
My config.json has:
{
"Data": {
"ConnectionStringTest": {
"ConnectionString": "Data Source=(localdb)\\mssqllocaldb;Initial Catalog=ConnectionStringTest"
}
}
}
And this should be overridden by the connection string I've setup in Azure:
When going to the Kudu SCM and looking at the environment variables on the Azure web app instance, I see the following:
SQLAZURECONNSTR_Data:ConnectionStringTest:ConnectionString = my_connection_string_here
I am assuming this is the class that is being used under the hood when my environment variable is used at runtime: EnvironmentVariablesConfigurationProvider
Ok here's what I found, and this feels awkward.
It seems that you need to use Data:{my_connection_string_key}:ConnectionString everywhere EXCEPT in Azure. This environment variable converter will construct the proper connection string using this format automatically if the connection string is prefixed with SQLAZURECONNSTR_.
This means when you setup your connection string in Azure, you need to omit EVERYTHING except the key to your connection string. Do not insert Data: or :ConnectionString... simply use {connection_string_key} (refer to the above format) instead. If you include the entire format in your Azure key/value pair, the EnvironmentVariablesConfigurationProvider will add another Data: and :ConnectionString around it, resulting in something like Data:Data:{my_connection_string_key}:ConnectionString:ConnectionString.
In ConfigureServices(...), use the format that ASP expects:
... options.UseSqlServer(mConfiguration["Data:ConnectionStringTest:ConnectionString"]);
You can therefore locally use this for your local JSON, for testing in development/fallback:
{
"Data": {
"ConnectionStringTest": {
"ConnectionString": "Data Source=(localdb)\\mssqllocaldb;Initial Catalog=ConnectionStringTest"
}
}
}
Just make sure your Azure connection string has the middle part of that format (ConnectionStringTest using this example).
This will make your environment variable in Azure look like this in raw format:
SQLAZURECONNSTR_ConnectionStringTest = {insert connection string here}
And the EnvironmentVariablesConfigurationProvider will strip off the Azure prefix string, and wrap your key in the hardcoded format: Data:{0}:ConnectionString
Experiment Results
To augment your excellent answer, I did a local experiment to confirm that the SQLAZURECONNSTR_connection_string_key environmental variable becomes this configuration:
mConfiguration["Data:connection_string_key:ConnectionString"]
Local Experiment
A local environmental variable emulates an Azure SQL Database connection string named connection_string_key.
PS> $env:SQLAZURECONNSTR_connection_string_key = "an azure conn string"
The following code dumps all the environmental variables and configuration sections to the page.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
app.Run(async (context) =>
{
await context.Response.WriteAsync("# Environmental Variables \r\n");
await DumpAllEnvVariables(context, Environment.GetEnvironmentVariables());
await context.Response.WriteAsync("# Configuration Sections \r\n");
await DumpAllConfigItems(context, mConfiguration.GetChildren());
});
}
private async Task DumpAllEnvVariables(HttpContext context, IDictionary envVariables)
{
foreach (var envVar in envVariables.Cast<DictionaryEntry>())
{
await context.Response.WriteAsync($"{envVar.Key}"); // : {envVar.Value}
await context.Response.WriteAsync($"\r\n");
}
}
private async Task DumpAllConfigItems(HttpContext context,
IEnumerable<IConfigurationSection> sections, string prefix = "")
{
foreach (var section in sections)
{
await context.Response.WriteAsync($"{prefix}{section.Key}"); // : {envVar.Value}
await context.Response.WriteAsync($"\r\n");
if(section.GetChildren().Any())
{
await DumpAllConfigItems(context, section.GetChildren(), prefix + " ");
}
}
}
How do I configure the Azure Caching Service in code? Note I'm talking specifically about the new Caching Service, currently in preview, that uses v2+ of the caching libraries. I am not talking about the older Azure Shared Caching service, which is very similar.
In my particular case I want to configure caching parameters in the service configuration (.cscfg) file, rather than in web.config. But there are probably other reasons as well.
This seems to do the trick.
//Utility function
private static SecureString MakeSecureString(string s)
{
SecureString secureString = new SecureString();
foreach (char c in s)
{
secureString.AppendChar(c);
}
secureString.MakeReadOnly();
return secureString;
}
//Populate these values from whereever you want, perhaps read from config
string host = "foo.cache.windows.net"
string rawAuthToken = "abcdefgblahblahblahfoobar==";
//Create the SecureString that we need for the security config
SecureString authToken = MakeSecureString(rawAuthToken);
DataCacheFactoryConfiguration cacheConfig = new DataCacheFactoryConfiguration();
cacheConfig.AutoDiscoverProperty = new DataCacheAutoDiscoverProperty(true, host);
cacheConfig.SecurityProperties = new DataCacheSecurity(authToken);
var cacheFactory = new DataCacheFactory(cacheConfig);
//Get a cache as normal
DataCache cache = cacheFactory.GetCache("default");
I am using below code to create queue, using SharedSecretTokenProvider. However I am not able to supply correct values of managerName & managerKey value form windows azure portal.
This results in Http 401 Unauthorized exception. How do I resolve this error?
const string queueName = "thequeue";
var tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
ConfigurationManager.AppSettings["managerName"],
ConfigurationManager.AppSettings["managerKey"]);
Uri uri = ServiceBusEnvironment.CreateServiceUri("http", "MyNamespace" , string.Empty);
NamespaceManager namespaceManager = new NamespaceManager(uri, tokenProvider);
QueueDescription qd = namespaceManager.CreateQueue(new QueueDescription(queueName)
{
DefaultMessageTimeToLive = TimeSpan.FromMinutes(15),
DuplicateDetectionHistoryTimeWindow = TimeSpan.FromMinutes(10),
LockDuration = TimeSpan.FromMinutes(2),
EnableBatchedOperations = true,
EnableDeadLetteringOnMessageExpiration = true,
RequiresDuplicateDetection = true
});
I had tried this a couple of times with your code before I realized the problem. You are using the SharedSecretTokenProvider which will go to ACS thinking it has an issuer and a key. Since you are trying to use the SAS you'll want to use a CreateSharedAccessSignatureTokenProvider instead.
Swap that out and provide the key and keyName and you should be good.
Also, Viperguynaz is correct, you should use the "sb" instead of http as well. It was failing before it reached there because the token provider was correctly declining you access since it didn't understand the key and keyname you were passing for what it thought was the issuer and key that ACS uses.
Start with ServiceBusEnvironment.CreateServiceUri Method. Note that Service Bus endpoint URIs must always use the “sb://” protocol; for example sb://contoso.servicebus.windows.net/helloservicebus.
Uri address = ServiceBusEnvironment.CreateServiceUri("sb", "contoso", "helloservicebus");
Get your URI inputs set correctly and you should be in business.
Is there anywhere in the service runtime that would tell me if I'm currently running on 'Staging' or 'Production'? Manually modifying the config to and from production seems a bit cumbersome.
You should really not change your configurations when you're based upon if you're in Prod or Staging. Staging area is not designed to be a "QA" environment but only a holding-area before production is deployed.
When you upload a new deployment, current deployment slot where you upload your package to is destroyed and is down for 10-15minutes while upload and start of VM's is happening. If you upload straight into production, that's 15 minutes of production downtime. Thus, Staging area was invented: you upload to staging, test the stuff, and click "Swap" button and your Staging environment magically becomes Production (virtual IP swap).
Thus, your staging should really be 100% the same as your production.
What I think you're looking for is QA/testing environment? You should open up a new service for Testing environment with its own Prod/Staging. In this case, you will want to maintain multiple configuration file sets, one set per deployment environment (Production, Testing, etc.)
There are many ways to manage configuration-hell that occurs, especially with Azure that has on top of .config files, its own *.cscfg files. The way I prefer to do it with Azure project is as follows:
Setup a small Config project, create folders there that match Deployment types. Inside each folder setup sets of *.config & *.cscfg files that match to particular deployment environment: Debug, Test, Release... these are setup in Visual Studio as well , as build target types. I have a small xcopy command that occurs during every compile of the Config project that copies all the files from Build Target folder of Config project into root folder of the Config project.
Then every other project in the solution, LINKS to the .config or .cscfg file from the root folder of the Config project.
Voila, my configs magically adapt to every build configuration automatically. I also use .config transformations to manage debugging information for Release vs. non-Release build targets.
If you've read all this and still want to get at the Production vs. Staging status at runtime, then:
Get deploymentId from RoleEnvironment.DeploymentId
Then use Management API with a proper X509 certificate to get at the Azure structure of your Service and call the GetDeployments method (it's rest api but there is an abstraction library).
Hope this helps
Edit: blog post as requested about the setup of configuration strings and switching between environments # http://blog.paraleap.com/blog/post/Managing-environments-in-a-distributed-Azure-or-other-cloud-based-NET-solution
Sometimes I wish people would just answer the question.. not explain ethics or best practices...
Microsoft has posted a code sample doing exactly this here: https://code.msdn.microsoft.com/windowsazure/CSAzureDeploymentSlot-1ce0e3b5
protected void Page_Load(object sender, EventArgs e)
{
// You basic information of the Deployment of Azure application.
string deploymentId = RoleEnvironment.DeploymentId;
string subscriptionID = "<Your subscription ID>";
string thrumbnail = "<Your certificate thumbnail print>";
string hostedServiceName = "<Your hosted service name>";
string productionString = string.Format(
"https://management.core.windows.net/{0}/services/hostedservices/{1}/deploymentslots/{2}",
subscriptionID, hostedServiceName, "Production");
Uri requestUri = new Uri(productionString);
// Add client certificate.
X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
store.Open(OpenFlags.OpenExistingOnly);
X509Certificate2Collection collection = store.Certificates.Find(
X509FindType.FindByThumbprint, thrumbnail, false);
store.Close();
if (collection.Count != 0)
{
X509Certificate2 certificate = collection[0];
HttpWebRequest httpRequest = (HttpWebRequest)HttpWebRequest.Create(requestUri);
httpRequest.ClientCertificates.Add(certificate);
httpRequest.Headers.Add("x-ms-version", "2011-10-01");
httpRequest.KeepAlive = false;
HttpWebResponse httpResponse = httpRequest.GetResponse() as HttpWebResponse;
// Get response stream from Management API.
Stream stream = httpResponse.GetResponseStream();
string result = string.Empty;
using (StreamReader reader = new StreamReader(stream))
{
result = reader.ReadToEnd();
}
if (result == null || result.Trim() == string.Empty)
{
return;
}
XDocument document = XDocument.Parse(result);
string serverID = string.Empty;
var list = from item
in document.Descendants(XName.Get("PrivateID",
"http://schemas.microsoft.com/windowsazure"))
select item;
serverID = list.First().Value;
Response.Write("Check Production: ");
Response.Write("DeploymentID : " + deploymentId
+ " ServerID :" + serverID);
if (deploymentId.Equals(serverID))
lbStatus.Text = "Production";
else
{
// If the application not in Production slot, try to check Staging slot.
string stagingString = string.Format(
"https://management.core.windows.net/{0}/services/hostedservices/{1}/deploymentslots/{2}",
subscriptionID, hostedServiceName, "Staging");
Uri stagingUri = new Uri(stagingString);
httpRequest = (HttpWebRequest)HttpWebRequest.Create(stagingUri);
httpRequest.ClientCertificates.Add(certificate);
httpRequest.Headers.Add("x-ms-version", "2011-10-01");
httpRequest.KeepAlive = false;
httpResponse = httpRequest.GetResponse() as HttpWebResponse;
stream = httpResponse.GetResponseStream();
result = string.Empty;
using (StreamReader reader = new StreamReader(stream))
{
result = reader.ReadToEnd();
}
if (result == null || result.Trim() == string.Empty)
{
return;
}
document = XDocument.Parse(result);
serverID = string.Empty;
list = from item
in document.Descendants(XName.Get("PrivateID",
"http://schemas.microsoft.com/windowsazure"))
select item;
serverID = list.First().Value;
Response.Write(" Check Staging:");
Response.Write(" DeploymentID : " + deploymentId
+ " ServerID :" + serverID);
if (deploymentId.Equals(serverID))
{
lbStatus.Text = "Staging";
}
else
{
lbStatus.Text = "Do not find this id";
}
}
httpResponse.Close();
stream.Close();
}
}
Staging is a temporary deployment slot used mainly for no-downtime upgrades and ability to roll back an upgrade.
It is advised not to couple your system (either in code or in config) with such Azure specifics.
Since Windows Azure Management Libraries and thanks to #GuaravMantri answer to another question you can do it like this :
using System;
using System.Linq;
using System.Security.Cryptography.X509Certificates;
using Microsoft.Azure;
using Microsoft.WindowsAzure.Management.Compute;
using Microsoft.WindowsAzure.Management.Compute.Models;
namespace Configuration
{
public class DeploymentSlotTypeHelper
{
static string subscriptionId = "<subscription-id>";
static string managementCertContents = "<Base64 Encoded Management Certificate String from Publish Setting File>";// copy-paste it
static string cloudServiceName = "<your cloud service name>"; // lowercase
static string ns = "http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration";
public DeploymentSlot GetSlotType()
{
var managementCertificate = new X509Certificate2(Convert.FromBase64String(managementCertContents));
var credentials = new CertificateCloudCredentials(subscriptionId, managementCertificate);
var computeManagementClient = new ComputeManagementClient(credentials);
var response = computeManagementClient.HostedServices.GetDetailed(cloudServiceName);
return response.Deployments.FirstOrDefault(d => d.DeploymentSlot == DeploymentSlot.Production) == null ? DeploymentSlot.Staging : DeploymentSlot.Production;
}
}
}
An easy way to solve this problem is setting at your instances an key to identify which environment it is running.
1) Set at your production slot:
Set it Settings >> Application settings >> App settings
And create a key named SLOT_NAME and value "production". IMPORTANT: check Slot setting.
2) Set at your staging slot:
Set it Settings >> Application settings >> App settings
And create a key named SLOT_NAME and value "staging". IMPORTANT: check Slot setting.
Access from your application the variable and identify which environment the application is running. In Java you can access:
String slotName = System.getenv("APPSETTING_SLOT_NAME");
Here are 4 points to consider
VIP swap only makes sense when your service faces the outside world. AKA, when it exposes an API and reacts to requests.
If all your service does is pull messages from a queue and process them, then your services is proactive and VIP swap is not a good solution for you.
If your service is both reactive and proactive, you may want to reconsider your design. Perhaps split the service into 2 different services.
Eric's suggestion of modifying the cscfg files pre- and post- VIP swap is good if the proactive part of your service can take a short down time (Because you first configure both Staging and Production to not pull messages, then perform VIP Swap, and then update Production's configuration to start pulling messages).