I'm trying to do what might be a simple thing, but i just can't find the answer.
I know how to delete cache on the application level (for a website).
But i need to CLEAR ALL THE CACHE of my azure appfabric cache on webrole start.
EDIT : What if it can be done ? Considering a severe misfunction in Azure Asp.net shared caching or am I dumb ?
This this the scenario i'm stuck into :
I have a a WebRole which have few different websites
Consider all my websites have a default.aspx file on each root with a 60 minutes caching, this is plain regular <%# OutputCache Duration="3600" VaryByParam="*" %>, no fancy code
I browse them, consequently several cached pages are stored in appfabric caches for 60 minutes
Consider uner the hood in this deployment cache depencies are created upon a path like 'E:\sitesroot\xx'
In between, my Webrole reset or I upload a new deployment
Now consider my websites in newly created instances have this path : 'F:\sitesroot\xx'
You get it, heres the result I have :
Directory 'E:\sitesroot\12' does not exist. Failed to start monitoring file changes.
System.Web.HttpException (0x80070003): Directory 'E:\sitesroot\12' does not exist. Failed to start monitoring file changes.
at System.Web.FileChangesMonitor.FindDirectoryMonitor(String dir, Boolean addIfNotFound, Boolean throwOnError)
at System.Web.FileChangesMonitor.StartMonitoringPath(String alias, FileChangeEventHandler callback, FileAttributesData& fad)
at System.Web.Caching.CacheDependency.Init(Boolean isPublic, String[] filenamesArg, String[] cachekeysArg, CacheDependency dependency, DateTime utcStart)
at System.Web.Caching.CacheDependency..ctor(Int32 dummy, String[] filenames)
at System.Web.Caching.OutputCache.HasDependencyChanged(Boolean isFragment, String depKey, String[] fileDeps, String kernelKey, String oceKey, String providerName)
at System.Web.Caching.OutputCache.Get(String key)
at System.Web.Caching.OutputCacheModule.OnEnter(Object source, EventArgs eventArgs)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
It took me some times to understand that this issue MIGHT BE because the current appfacbric cache relied on a cachedepency with a hard path "E:\sitesroot\". What if this path change, has it DOES.
So what can I do ? I'm lost here, I tried to clear appcache on Global.asax Application_Start, but this does not seem to work (and i don't understand why it does not).
Here's my code to clear cache on each websites Global.asax Application_Start
Public Shared Sub cacheClear()
Dim keys As New List(Of String)()
' retrieve application Cache enumerator
Dim enumerator As IDictionaryEnumerator = HttpRuntime.Cache.GetEnumerator()
' copy all keys that currently exist in Cache
While enumerator.MoveNext()
keys.Add(enumerator.Key.ToString())
End While
' delete every key from cache
For i As Integer = 0 To keys.Count - 1
HttpRuntime.Cache.Remove(keys(i))
Next
End Sub
This should do the trick, it does not seems to.
There certainly a ridiculous point i'm missing.
EDIT
I ended dropping completely asp.net outputcache provider for appfabric and writing a custom cache with appfabric. Works great, few line of codes and cachedependancy on azure made easy.
I'm not 100% sure, but I don't think this can be done.
Related
I have a WebJob that needs to create a JWT token to talk with an external service. The following code works when I run the WebJob on my local machine:
public static string SignES256(byte[] p8Certificate, object header, object payload)
{
var headerString = JsonConvert.SerializeObject(header);
var payloadString = JsonConvert.SerializeObject(payload);
CngKey key = CngKey.Import(p8Certificate, CngKeyBlobFormat.Pkcs8PrivateBlob);
using (ECDsaCng dsa = new ECDsaCng(key))
{
dsa.HashAlgorithm = CngAlgorithm.Sha256;
var unsignedJwtData = Base64UrlEncoder.Encode(Encoding.UTF8.GetBytes(headerString)) + "." + Base64UrlEncoder.Encode(Encoding.UTF8.GetBytes(payloadString));
var signature = dsa.SignData(Encoding.UTF8.GetBytes(unsignedJwtData));
return unsignedJwtData + "." + Base64UrlEncoder.Encode(signature);
}
}
However, when I deploy my WebJob to Azure, I get the following exception:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: NotificationFunctions.QueueOperation ---> System.Security.Cryptography.CryptographicException: The system cannot find the file specified. at System.Security.Cryptography.NCryptNative.ImportKey(SafeNCryptProviderHandle provider, Byte[] keyBlob, String format) at System.Security.Cryptography.CngKey.Import(Byte[] keyBlob, CngKeyBlobFormat format, CngProvider provider)
It says it can't find a specified file, but the parameters I am passing in are not looking at a file location, they are in memory. From what I have gathered, there may be some kind of cryptography setting I need to enable to be able to use the CngKey.Import method, but I can't find any settings in the Azure portal to configure related to this.
I have also tried using JwtSecurityTokenHandler, but it doesn't seem to handle the ES256 hashing algorithm I need to use (even though it is referenced in the JwtAlgorithms class as ECDSA_SHA256).
Any suggestions would be appreciated!
UPDATE
It appears that CngKey.Import may actually be trying to store the certificate somewhere that is not accessible on Azure. I don't need it stored, so if there is a better way to access the certificate in memory or convert it to a different kind of certificate that would be easier to use that would work.
UPDATE 2
This issue might be related to Azure Web Apps IIS setting not loading the user profile as mentioned here. I have enabled this by setting WEBSITE_LOAD_USER_PROFILE = 1 in the Azure portal app settings. I have tried with this update when running the code both via the WebJob and the Web App in Azure but I still receive the same error.
I used a decompiler to take a look under the hood at what the CngKey.Import method was actually doing. It looks like it tries to insert the certificate I am using into the "Microsoft Software Key Storage Provider". I don't actually need this, just need to read the value of the certificate but it doesn't look like that is possible.
Once I realized a certificate is getting inserted into a store somewhere one the machine, I started thinking about how bad of a think that would be from a security standpoint if your Azure Web App was running in a shared environment, like it does for the Free and Shared tiers. Sure enough, my VM was on the Shared tier. Scaling it up to the Basic tier resolved this issue.
As described in various other related questions here, I am also expecting long lasting (30 seconds) first call after (re)deploying a web role with pretty large EF6 Model and a plenty of referenced nuget-packages. After trying out different proposed solutions with preloadEnabled and serviceAutoStartProviders I am still consufed and decided to reopen this topic in hope, that somebody has come upon a better solution in the meantime.
The main goal for me is to make a web role answering the first request almost as fast as subsequent calls as soon as the role quits its busy state on a fresh deployment and gets accessible by load balancer and external clients. Unfortunately I experienced following problems with proposed solutions so far:
preloadEnabled:
I do add Application Initialization module via PKGMGR.EXE /iu:IIS-ApplicationInit in a startup task. So far so good.
When I then try to execute %windir%\system32\inetsrv\appcmd set site "MySiteName" -applicationDefaults.preloadEnabled:true it fails as at the time of execution of a startup script there are still no websites created in IIS on a fresh deployment.
If I try to set the preloadEnabled-setting via ServerManager-class in my Application_Start method instead, I cannot understand, how this code is intended to be executed before a first external call on the web role is made, as the preloadEnabled setting is defaulted to false after a fresh web role deploy and thus the Application_Start method in my understanding does not get a chance to be executed by the Application Initialization module?
serviceAutostartProviders:
here we need to put a name of our AutostartProvider implementing the IProcessHostPreloadClient interface in applicationhost.config i.e by using either appcmd script or the ServerManager class, BUT:
serviceAutostartProvider is like preloadEnabled a site related setting, so we have the same problem here as with %windir%\system32\inetsrv\appcmd set site "MySiteName" -applicationDefaults.preloadEnabled:true in 1.2 - at execution time of startup script after a fresh deployment the websites are not yet created in IIS and the script fails to execute properly
Another possibility would be to include applicationhost.config into the deployment package, but I did not find any solution to do this for a web role.
So how did you guys managed to ensure, that preloading assemblies and some initialization code (like filling memory caches) is run before the role gets it's first hit from outside?
We start to gain some traffic now and get approx. 1-2 requests per second on our WebApi, and thus a 30 second delay for preloading "visible" to clients after each update deployment is becoming a major issue now.
We are scheduling update deploys at low traffic times, but what if I need to make an urgent hotfix deploy?
Perhaps the best way to accomplish this is to use deployment slots. Deploy updates to your staging slot first. Before a switch from a staging slot to a production slot takes place, Kudu will hit the root of the staging slot with a request in order to warm up the application. After the request to the staging slot's root returns, the IP switch occurs and your slots are swapped.
However, sometimes you need to warm up other pages or services to get the app ready to handle traffic, and hitting the root with a warmup request is insufficient. You can update your web.config so that Kudu will hit additional pages and endpoints before the IP switch occurs and the slots are swapped.
These warmup request URLs should be in the tag. Here's an example:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.webServer>
<applicationInitialization>
<add initializationPage="/pagetowarmup1" />
<add initializationPage="/pagetowarmup2" />
<add initializationPage="/pagetowarmup3" />
</applicationInitialization>
</system.webServer>
</configuration>
You can read the Kudu docs on this issue here.
OK. Now I got it.
The point is to set preloadEnabled-property not inside of the Application_Start method in Global.asax (as it will not be hit before a first request to the Role anyway), but inside RoleEntryPoint.OnStart.
The difference is that RoleEntryPoint.OnStart is called directly after deployment package is extracted and everything is set up for starting the role. At this moment the azure instance is still in it's busy state and is not yet available from outside as long as some code inside RoleEntryPoint.OnStart is being executed.
So here is the code I came up so far for warming up the instance before it gets its first call from outside
public class WebRole : RoleEntryPoint
{
public override bool OnStart()
{
// set preloadEnabled of each Site in this deployment to true
using (var serverManager = new ServerManager())
{
foreach (var mainSite in serverManager.Sites)
{
var mainApplication = mainSite.Applications["/"];
mainApplication["preloadEnabled"] = true;
}
serverManager.CommitChanges();
}
// call my warmup-code which will preload caches and do some other time consuming preparation
var localuri = new Uri(string.Format("https://{0}/api/warmup", RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["Endpoint1"].IPEndpoint));
try
{
var request = (HttpWebRequest)WebRequest.Create(localuri);
request.Method = "GET";
var response = request.GetResponse();
}
catch { }
// send this thread to sleep in order to give the warmup-request a chance to complete BEFORE the Role will get started
// and becomes available to the outside world
System.Threading.Thread.Sleep(60000);
return base.OnStart();
}
}
We have an Azure WebApp with WEBSITE_LOCAL_CACHE_OPTION = Always set in order to reduce static file latency.
We have made a file change there and would like to force the instances to reload the cache.
How do I force a file cache refresh?
The only way to refresh the cache is to restart the website.
Azure Web apps WEBSITE_LOCAL_CACHE_OPTION=Always requires stop and start of the site
I know it's an old subject, but maybe it will be helpfull for someone :
If you are using C# you can add a button inside your application to remove the cache :
IDictionaryEnumerator enumerator = HttpContext.Current.Cache.GetEnumerator();
while (enumerator.MoveNext())
{
HttpContext.Current.Cache.Remove((string)enumerator.Key);
}
namespace StorageRoleMVC4.Controllers
{
public class SearchController : ApiController
{
public Dictionary<string, string> Get([FromUri] string searchString, [FromUri] string searchObject)
{
var searchHelper = new SearchStorageHelper();
var objectList = searchHelper.Retrieve(searchString, searchObject);
return objectList;
}
}
}
Is there anything about this controller that makes it unreachable once it's deployed (to an Azure web role)? I just get a 404 error when I try to reach it. It works great on the local emulator.
The last 2 times I've deployed my project, all the controllers in my web service have returned 404 errors for several hours, until the project seems to fix itself. I'm not sure why, but it might be related.
UPDATE
There is a WARNING in the event log on the web role VM after I publish:
The application '/' belonging to site '1273337584' has an invalid AppPoolId 'ea7a2e15-9390-49e1-a16b-67ff1cdb7dcb' set. Therefore, the application will be ignored.
This is the id of my site, but the AppPoolId is not correct. Changing the app pool turns the 404 into a 502.
Also, after publishing, the World Wide Web Publishing Service is turned off. When I turn it on and do an IIS reset, after the reset it's turned off again.
When I reboot the web role VM, most of the controllers work again, and the World Wide Web Publishing Service is turned on. But still, this SearchController doesn't work. Or any other new controllers I've created since this problem started happening.
Well, after a ridiculous amount of unsuccessful troubleshooting, I just rolled back to an earlier version of the code and found that it didn't read the web service when I deployed it. So I started over with that version and re-built the delta.
This involved removing a few web.config entries and removing some libraries, creating a few classes and referencing them in the global.asax (I think that's where it was) in order to override te Authorize attribute...
If anyone has a better answer, I will switch the answer to what you post.
I use local storage on Windows Azure to store temporary files. In there I call an .exe file to make a conversion of several other files in same local storage folder. Problem is I always get the exception "Access to the path XYZ.exe is denied.".
I should mention the following:
- I am using a worker role
- set in the service definition file
and tried to add permission to the folder I am accessing:
public static void AddPermission(string absoluteFolderPath)
{
DirectoryInfo myDirectoryInfo = new DirectoryInfo(absoluteFolderPath);
DirectorySecurity myDirectorySecurity = myDirectoryInfo.GetAccessControl();
myDirectorySecurity.AddAccessRule(new FileSystemAccessRule(
"NETWORK SERVICE",
FileSystemRights.FullControl,
AccessControlType.Allow));
myDirectoryInfo.SetAccessControl(myDirectorySecurity);
}
UPDATE:
I tried with this code now:
public static void FixPermissions()
{
var tempDirectory = RoleEnvironment.GetLocalResource("localStorage").RootPath;
Helper.addPermission(tempDirectory);
var dir = new DirectoryInfo(tempDirectory);
foreach (var d in dir.GetDirectories())
Helper.addPermission(d.FullName);
}
private static void addPermission(string path)
{
FileSystemAccessRule everyoneFileSystemAccessRule = new FileSystemAccessRule("Everyone",
FileSystemRights.FullControl,
InheritanceFlags.ContainerInherit | InheritanceFlags.ObjectInherit,
PropagationFlags.None, AccessControlType.Allow);
DirectoryInfo directoryInfo = new DirectoryInfo(path);
DirectorySecurity directorySecurity = directoryInfo.GetAccessControl();
directorySecurity.AddAccessRule(everyoneFileSystemAccessRule);
directoryInfo.SetAccessControl(directorySecurity);
}
I get a really strange behaviour of the page. I still get the errors but sometimes some files gets converted by the ffmpeg.exe file.
Can someone help me out here??
Thanks a lot.
SOLUTION:
So seems the problem was that I ran the .exe file within local storage and therefore had the given security issues. Putting the .exe into the application and referring directly solved my issue.
Thx for your help.
By default your worker role will most likely not be running with sufficient privilege to allow changes to the access control lists on Azure folders.
There's two possible options:
Best: run a script at startup to set the permissions. Details are on MSDN here: http://msdn.microsoft.com/en-us/library/gg456327.aspx. You'll want to set executionContext="elevated".
The best way to write the script itself is through Powershell. An example is here: http://weblogs.thinktecture.com/cweyer/2011/01/fixing-windows-azure-sdk-13-full-iis-diagnostics-and-tracing-bug-with-a-startup-task-a-grain-of-salt.html. Alternatively, write a console application to do the same thing.
Easiest, but much less secure: set the security in your OnStart method, and run your whole worker role elevated: in your service definition file include
<WebRole name="WebApplication2">
<Runtime executionContext="elevated" />
<Sites>
However, I'd really not recommend that as it's a terrible security hole for something that's running in the public cloud.