My node.js application runs nicely on my own machine, however, after pushing it up to an Azure WebSite using Git, I'm getting request validation problems.
Some of my requests are trapped by the IIS validation process. The fix is to add this to the web.config:
<httpRuntime requestPathInvalidCharacters=""/>
I downloaded the web.config from the site\wwwroot folder of my website using FTP, added it to the repo, added the new element in the XML, and pushed it back up to Azure.
My web.config file seems to have been overridden by the default.
Is there a way of changing this behaviour? Or another way of turning off request validation? Or am I just being thick?
UPDATE
I take it all back, it seems to be express which is blocking the request. However, the question still stands as 'how do I customise my web.config?'
Could you try setting a custom validation type? Like this:
<httpRuntime requestPathInvalidCharacters=""
requestValidationType="MyValidator, MyAssembly"/>
And the class:
public class MyValidator : RequestValidator
{
protected override bool IsValidRequestString(HttpContext context, string value, RequestValidationSource requestValidationSource, string collectionKey, out int validationFailureIndex)
{
validationFailureIndex = 0;
return true;
}
}
Related
As described in various other related questions here, I am also expecting long lasting (30 seconds) first call after (re)deploying a web role with pretty large EF6 Model and a plenty of referenced nuget-packages. After trying out different proposed solutions with preloadEnabled and serviceAutoStartProviders I am still consufed and decided to reopen this topic in hope, that somebody has come upon a better solution in the meantime.
The main goal for me is to make a web role answering the first request almost as fast as subsequent calls as soon as the role quits its busy state on a fresh deployment and gets accessible by load balancer and external clients. Unfortunately I experienced following problems with proposed solutions so far:
preloadEnabled:
I do add Application Initialization module via PKGMGR.EXE /iu:IIS-ApplicationInit in a startup task. So far so good.
When I then try to execute %windir%\system32\inetsrv\appcmd set site "MySiteName" -applicationDefaults.preloadEnabled:true it fails as at the time of execution of a startup script there are still no websites created in IIS on a fresh deployment.
If I try to set the preloadEnabled-setting via ServerManager-class in my Application_Start method instead, I cannot understand, how this code is intended to be executed before a first external call on the web role is made, as the preloadEnabled setting is defaulted to false after a fresh web role deploy and thus the Application_Start method in my understanding does not get a chance to be executed by the Application Initialization module?
serviceAutostartProviders:
here we need to put a name of our AutostartProvider implementing the IProcessHostPreloadClient interface in applicationhost.config i.e by using either appcmd script or the ServerManager class, BUT:
serviceAutostartProvider is like preloadEnabled a site related setting, so we have the same problem here as with %windir%\system32\inetsrv\appcmd set site "MySiteName" -applicationDefaults.preloadEnabled:true in 1.2 - at execution time of startup script after a fresh deployment the websites are not yet created in IIS and the script fails to execute properly
Another possibility would be to include applicationhost.config into the deployment package, but I did not find any solution to do this for a web role.
So how did you guys managed to ensure, that preloading assemblies and some initialization code (like filling memory caches) is run before the role gets it's first hit from outside?
We start to gain some traffic now and get approx. 1-2 requests per second on our WebApi, and thus a 30 second delay for preloading "visible" to clients after each update deployment is becoming a major issue now.
We are scheduling update deploys at low traffic times, but what if I need to make an urgent hotfix deploy?
Perhaps the best way to accomplish this is to use deployment slots. Deploy updates to your staging slot first. Before a switch from a staging slot to a production slot takes place, Kudu will hit the root of the staging slot with a request in order to warm up the application. After the request to the staging slot's root returns, the IP switch occurs and your slots are swapped.
However, sometimes you need to warm up other pages or services to get the app ready to handle traffic, and hitting the root with a warmup request is insufficient. You can update your web.config so that Kudu will hit additional pages and endpoints before the IP switch occurs and the slots are swapped.
These warmup request URLs should be in the tag. Here's an example:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.webServer>
<applicationInitialization>
<add initializationPage="/pagetowarmup1" />
<add initializationPage="/pagetowarmup2" />
<add initializationPage="/pagetowarmup3" />
</applicationInitialization>
</system.webServer>
</configuration>
You can read the Kudu docs on this issue here.
OK. Now I got it.
The point is to set preloadEnabled-property not inside of the Application_Start method in Global.asax (as it will not be hit before a first request to the Role anyway), but inside RoleEntryPoint.OnStart.
The difference is that RoleEntryPoint.OnStart is called directly after deployment package is extracted and everything is set up for starting the role. At this moment the azure instance is still in it's busy state and is not yet available from outside as long as some code inside RoleEntryPoint.OnStart is being executed.
So here is the code I came up so far for warming up the instance before it gets its first call from outside
public class WebRole : RoleEntryPoint
{
public override bool OnStart()
{
// set preloadEnabled of each Site in this deployment to true
using (var serverManager = new ServerManager())
{
foreach (var mainSite in serverManager.Sites)
{
var mainApplication = mainSite.Applications["/"];
mainApplication["preloadEnabled"] = true;
}
serverManager.CommitChanges();
}
// call my warmup-code which will preload caches and do some other time consuming preparation
var localuri = new Uri(string.Format("https://{0}/api/warmup", RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["Endpoint1"].IPEndpoint));
try
{
var request = (HttpWebRequest)WebRequest.Create(localuri);
request.Method = "GET";
var response = request.GetResponse();
}
catch { }
// send this thread to sleep in order to give the warmup-request a chance to complete BEFORE the Role will get started
// and becomes available to the outside world
System.Threading.Thread.Sleep(60000);
return base.OnStart();
}
}
As described in this article: https://azure.microsoft.com/en-us/blog/windows-azure-web-sites-how-application-strings-and-connection-strings-work/, Azure Web Apps/Web Sites/Web Jobs can take their configuration settings (appSettings, connectionString) from environment variables instead of app.config/web.config.
For example, if an environment variable named "APPSETTING_appSettingKey" exists, it will override the following setting from app.config/web.config:
<appSettings>
<add key="appSettingKey" value="defaultValue" />
</appSettings>
This works fine once the application is deployed in Azure, but I would like to use the same method when testing locally.
I tried to emulate this in a local command line:
> set APPSETTING_appSettingKey=overridedValue
> MyWebJob.exe
The web job accesses this setting using:
ConfigurationManager.AppSettings["appSettingKey"]
When running in Azure, it reads the value "overridedValue" as expected, but locally it reads the value "defaultValue" from the app.config file.
Should I expect this to work, or is this implemented only under an Azure environment?
I could obviously create an abstraction over ConfigurationManager that emulates this, but this wouldn't work when calling code that needs a connection string name instead of a connection string value. Also, I want to use the same method regardless of the environment to simplify management of settings.
There are 3 reasons why I need this:
1) I don't like the idea of deploying to production a web.config file that references connection strings, etc for a developement environment, because there's a risk of an error that would cause the development settings (in web.config) to be used in production (production web app connecting to development database, etc), for example if an environment variable is named incorrectly (after renaming the setting in web.config but forgetting to rename it in environment variables)
2) I'm trying to setup development environments where each developer has his own isolated cloud resources (storage account, databases,...). Currently, everyone has to manually edit his .config files to reference the correct resources, and be careful when checking-in or merging changes to these files.
3) A solution can have multiple projects that need to duplicate the same settings (main web app, web jobs, integration test projects,...). This causes a lot of work to ensure updated settings are replicated across all files.
This would be simplified if there was an environment-independent .config file without any actual configuration, each developer would configure a set of environment variables once and be able to use them for all parts of a solution.
Yes, this special transformation of environment variables into config values is done via a component that is specific to Azure WebApps and won't be in play locally.
Generally people are fine with the local behavior this produces - locally you are reading from config settings as usual, but in Azure you're reading from secure settings that were configured via the App Settings portal blade (so these settings aren't in your source code).
You could write an abstraction over this if you wish, E.g. the WebJobs SDK actually does this internally (code here).
When I am developing locally and want to consistantly use Environment.GetEnvironmentVariable. In my static class Main I have the following code:
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
Environment.SetEnvironmentVariable("UseDevelopmentSettings", "true");
}
Then in my static class Functions I add a static constructor and in there I call the static method below:
static void AddAppSettingsToEnvironmentVariables()
{
String useDevelopmentSettings = Environment.GetEnvironmentVariable("UseDevelopmentSettings"); ;
if (!(String.IsNullOrEmpty(useDevelopmentSettings)))
{
foreach (String key in ConfigurationManager.AppSettings.AllKeys)
{
Environment.SetEnvironmentVariable(key, ConfigurationManager.AppSettings[key]);
}
}
}
The code is small enough that I can simply comment it out before I test in Azure.
If you want to test the application with the value that will be used in Azure portal AppSettings/Connection String. I would recommend use HostingEnvironment.IsDevelopmentEnvironment. To ensure it will work, please change the <compilation debug="true" targetFramework="4.5.2" /> to <compilation debug="false" targetFramework="4.5.2" />. set the value with the same value in Azure portal if (HostingEnvironment.IsDevelopmentEnvironment == false). I have try with a simple project, hope it helps:
public ActionResult Index()
{
if (HostingEnvironment.IsDevelopmentEnvironment == true)
{
ViewBag.Message = "Is development.";
}
else
{
ViewBag.Message = "Azure environment.";
}
return View();
}
Here is the result:
I recently deployed an ASP.Net Web API project to our Azure App Service test slot but started receiving an error when making requests to the API endpoints. Through remote debugging, it became clear that the app was extracting my dev connection strings from the deployed web.config file.
The connection strings are supposed to come from the Application Settings we set up via the Azure Portal - and, in previous deployments, they were - but that's not the case.
Why would this happen and what can be done to ensure the correct behaviour occurs? We absolutely don't want our production database secrets being put into GIT via the web.config...
I recently experienced the same problem and fixed it:
In Azure App Services, the machine-wide web.config file is located at D:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\web.config.
This file differs to a normal machine-wide web.config file because it has this extra element:
<system.web>
...
<compilation>
<assemblies>
<add assembly="EnvSettings, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
...
</assemblies>
</compilation>
</system.web>
The EnvSettings.dll assembly is located inside D:\Program Files\IIS\Microsoft Web Hosting Framework (unfortunately this directory is access-controlled and I can't get into it).
But EnvSettings.dll is mirrored in the GAC, so I was able to copy it from there.
Inside EnvSettings.dll is an [assembly: PreApplicationStartMethod] attribute that runs a method in EnvSettings.dll which copiesthe APPSETTING_ and database connection-string settings from Environment Variables into the .NET Framework ConfigurationManager.AppSettings and ConfigurationManager.ConnectionStrings collections.
Note this copying only happens once (during application startup), and only in the first AppDomain - so if you have other AppDomain instances in your application they won't see the updated ConfigurationManager.
Therefore, if you see that your Azure Portal configuration settings for your App Service are not being used when you dump your ConfigurationManager, then the following is likely happening:
You used <clear /> in your <compilation><assemblies> element, which stops EnvSettings.dll from being loaded at all.
In which case you need to either add back the <add assembly="EnvSettings... element from above to your own web.config, or find some other way to load it.
I don't recommend saving EnvSettings.dll locally and adding an assembly reference to your project, as EnvSettings.dll is part of the Microsoft Web Hosting Framework.
Or you have code that is clearing or resetting ConfigurationManager after EnvSettings populates it for you.
Or something else is going on that I have no idea about!
As an alternative to having EnvSettings.dll copy your settings over, another option is to copy the environment-variables over yourself - and as you control the code that does this it means you can call it whenever you need to (e.g. if you ever reset them).
Here's the code I used:
public static class AzureAppSettingsConfigurationLoader
{
public static void Apply()
{
foreach( DictionaryEntry environmentVariable in Environment.GetEnvironmentVariables() )
{
String name = (String)environmentVariable.Key;
String value = (String)environmentVariable.Value;
if( name.StartsWith( "APPSETTING_", StringComparison.OrdinalIgnoreCase ) )
{
String appSettingName = name.Substring( "APPSETTING_".Length );
ConfigurationManager.AppSettings[ appSettingName ] = value;
}
else if( name.StartsWith( "SQLAZURECONNSTR_", StringComparison.OrdinalIgnoreCase ) )
{
String csName = name.Substring( "SQLAZURECONNSTR_".Length );
ConfigurationManager.ConnectionStrings.Add( new ConnectionStringSettings( csName, value, providerName: ""System.Data.SqlClient" ) );
}
}
}
}
See my sample here: http://mvc5appsettings.azurewebsites.net/
// My web.config also has a "HERO_TEXT" key in
// that reads "Value from web.config"
string hero = ConfigurationManager.AppSettings["HERO_TEXT"];
Wiki page on App Settings for .NET:
https://github.com/projectkudu/kudu/wiki/Managing-settings-and-secrets
As already mentioned here, make sure you have that App Setting in the right slot.
As I know, the settings in Azure portal will override existing setting in Web.config. So If you want to ignore the Azure Application settings in portal and use Web.config instead. I am afraid you need to configure the settings in web.config, and remove the same key/pair in Azure portal.
namespace StorageRoleMVC4.Controllers
{
public class SearchController : ApiController
{
public Dictionary<string, string> Get([FromUri] string searchString, [FromUri] string searchObject)
{
var searchHelper = new SearchStorageHelper();
var objectList = searchHelper.Retrieve(searchString, searchObject);
return objectList;
}
}
}
Is there anything about this controller that makes it unreachable once it's deployed (to an Azure web role)? I just get a 404 error when I try to reach it. It works great on the local emulator.
The last 2 times I've deployed my project, all the controllers in my web service have returned 404 errors for several hours, until the project seems to fix itself. I'm not sure why, but it might be related.
UPDATE
There is a WARNING in the event log on the web role VM after I publish:
The application '/' belonging to site '1273337584' has an invalid AppPoolId 'ea7a2e15-9390-49e1-a16b-67ff1cdb7dcb' set. Therefore, the application will be ignored.
This is the id of my site, but the AppPoolId is not correct. Changing the app pool turns the 404 into a 502.
Also, after publishing, the World Wide Web Publishing Service is turned off. When I turn it on and do an IIS reset, after the reset it's turned off again.
When I reboot the web role VM, most of the controllers work again, and the World Wide Web Publishing Service is turned on. But still, this SearchController doesn't work. Or any other new controllers I've created since this problem started happening.
Well, after a ridiculous amount of unsuccessful troubleshooting, I just rolled back to an earlier version of the code and found that it didn't read the web service when I deployed it. So I started over with that version and re-built the delta.
This involved removing a few web.config entries and removing some libraries, creating a few classes and referencing them in the global.asax (I think that's where it was) in order to override te Authorize attribute...
If anyone has a better answer, I will switch the answer to what you post.
I have an ASP.NET MVC 4 site running perfectly well in an Azure WebRole. The ASP.NET MVC project was started on its own, after which I added an Azure Cloud Service project to the solution and added the ASP.NET project/site as one of the 'roles' of the service (so it shows up in the 'Roles' folder).
My problem is that I would like to have working a WebRole.cs file within the ASP.NET MVC project, but no matter what I've tried to do, it appears that when deployed, it just never gets called. OnStart and the override of Run (which I know, must never leave the loop) -- these just apparently never get called.
But if you startup a new CloudService project and add, at that time from the start, an ASP.NET MVC project, it automatically has a WebRole.cs file in it, so my guess is that I need to configure something somewhere for the WebRole.cs (actually speaking, the WebRole class, which inherits RoleEntryPoint) to get called. What might that be?
using System;
using System.Web;
//using Microsoft.WindowsAzure.StorageClient;
using System.Diagnostics;
using System.Threading;
namespace Us.WebUI
{
public class WebRole : Microsoft.WindowsAzure.ServiceRuntime.RoleEntryPoint
{
public override bool OnStart()
{
return true; //return base.OnStart(); // CALL THIS???
}
public override void Run()
{
while (true) {
Thread.Sleep(TimeSpan.FromSeconds(30));
try {
EmailFuncs.SendEmailToUs("An email from our WebRole?????", "Email me this, email me that.");
}
catch { }
}
}
}
}
UPDATE: Thanks, the question has been answered. But I will add: On doing this, while it clearly was working (fully deployed and in emulator), that suddenly I was having problems doing a full publish of the site. After a azure publish took 3 hours:
Verifying storage account 'xyz'... > Uploading Package... > - Updating... [stayed here for 3 hours], it failed with this error: The server encountered an internal error. Please retry the request. So one thing I was wondering is, did I need to override OnStop in WebRole.cs?
UPDATE 2: Those previous problems were fixed, and had nothing to do with this issue. Actually, I've learned this: If you ever have any warnings generated in your build, Azure often will not work with them even when they don't cause problems locally or in other hosts. Since then, I've been much more studious to tackling build warnings (but critical to this is turning off with warning codes the many warning types you want to ignore!).
Adding a class to your Web Project which inherits from RoleEntryPoint is sufficient, it should just work. Did you try setting a breakpoint in the emulator?
What you might be experiencing is that EmailFuncs.SendEmailToUs requires info from the app/web.config and that this info is not available. You need to know that your WebRole class runs in a different process (not your web application), meaning it's not using your web.config. If you want the WebRole.cs to read info from the configuration file, you'll need to add these settings in WaIISHost.exe.config