Since most of CDNs update the cache data only after it reaches an expiry, what if the origin server is updated before that. Does the CDN serve the old data until updated upon expiry?
Generally they have a callback method for you to let them know if a content changed and needs to be updated/re-cached. So it won't serve the old content until the next refresh.
You can see the documentation for Cloudflare-Purge Cache and Azure.
Yes, the CDN will serve the older content even when the origin is updated. This is because no one really told the CDN that there is an updated asset at the origin.
There are a couple of ways you can make sure the CDN always has the latest asset cached. The easiest way is to purge the cache each time there is a new/updated asset. This can be built into your CI/CD pipeline so that there is an auto-purge each time you deploy a new version
The other way I've seen a lot of people do is to have a version as part of the URL itself. So that when the asset is updated, it would have a new version as part of the object name. So when the browser makes a request, it is automatically requesting the new object.
Hopefully, this helps.
Related
We have an interesting behaviour that we are trying to understand/workaround.
We have an Azure Function running on a consumption host that has nothing but proxies on it.
One of these proxies points to an Azure CDN endpoint, that in turn points to an Azure Storage Account.
From time to time we update the file in the storage account and purge the CDN endpoint.
Requesting the file directly from the CDN returns the (correct) new file data.
However, the Function Proxy url continues to return the (incorrect) old file data.
Browser caches are disabled/cleared and all that normal stuff.
We can see the Last-Modified headers are different, so clearly the proxy is returning the old file.
Further, adding a querystring to the proxy URL - anything you like (we used ?v=1) would return the (correct) new file. Removing the querystring gets us back to the old file again.
Is this behaviour intentional? I have read UserVoice requests where people wanted caching added to Functions and it was explicitly declined due to the number of alternatives available. I see no other explanation for this behaviour, though.
Does anyone know how to disable/defeat proxy response caching?
Is there a way for certain users to force a bypass of CloudFront's cache so that they can always request new versions of files from the source, but the new files that they request are then available on the cache to other normal-permission users?
For example, if I had a sports score app that cached score information for 5 minutes, but I wanted to add a premium tier that allows subscribers to download up-to-date data whenever they use the app. I'd still want the data that the premium users download to be cached on CloudFront so that regular users don't waste API calls into the servers.
I was thinking maybe there would be a way to do this in the request headers, but I couldn't find any documentation on it.
Say I have a website with 2 files, which are statically hosted on S3:
index.html
script.js
I have a scheme where these files are updated via a git pull from a remote origin every time I push. This got me thinking, though, that there's the possibility for a request to be made to the server as a git pull is happening to update the files. This seems like it could create two problems that would cause page errors:
A partially-updated index.html or script.js is returned.
An old index.html is returned to the client. The files are then updated on the server. The client then makes a request for script.js, which returns the new version.
What is a good practice for mitigating these two issues?
I imagine the first issue won't be a problem, assuming filesystem operations are atomic and the files are updated in a single go. However, I haven't seen anything that addresses the second issue, which seems more difficult to address.
One way of addressing #2 is to use the cache-busting method of programmatically adding a hash to the script.js filename on compile, e.g.
<script src="script.js"> becomes <script src="script-79b1264ad3bc303fd.js">
Now, when script-[whatever].js is requested via a non-matching index.html, the client gets a 404 rather than the wrong script.
As for how you'd go about deploying this, it depends on your build pipeline. Are you using Grunt/Gulp/etc?
I have two asp.net core applications which share a number of static assets (images, css, fonts etc) and as a temporary solution I’m keeping copies in both projects. I want to streamline this and obviously keep only one copy of the asset and make it accessible to both projects.
I was thinking about solutions to this and wondered if it was possible to create a new project in the solution and somehow replicate the content to the Azure CDN for example upon git commit and then just reference the CDN URL’s in the source. Obviously I dont want to manually manage this content and have to manually upload the files to a CDN/Storage endpoint every time I change my CSS or add an image.
However, I can’t see if this is possible and if so how to set this up?
The app itself is hosted in Azure so this either needs to be implemented as a visual studio fix (e.g. the old ‘add link’ no longer seems to be available?) or in Azure.
Any suggestions on how to achieve this would be welcomed.
Thanks
I use PhysicalFileProvider to share static assets between multiple ASP.NET core web projects and it works really well. I also posted this same answer here: https://stackoverflow.com/a/52048104/188740, as it applies there as well.
To do this, I created a new folder in src. I called it assets so my Solution Explorer looks something like this:
src/assets
src/Foo.Web1
src/Foo.Web2
test/Foo.Tests
I put all of the shared static assets (e.g. css, JavaScript, images) in /src/assets. Now I map /src/assets to the wwwroot/assets folders in Foo.Web1 and Foo.Web2 projects using PhysicalFileProvider. In Startup.cs's Configure method, I add this:
if (Env.IsDevelopment())
{
// app is IApplicationBuilder and Env is IHostingEnvironment
app.UseStaticFiles(new StaticFileOptions
{
FileProvider = new PhysicalFileProvider(Regex.Replace(Env.ContentRootPath, $"{Env.ApplicationName}$", "assets"))),
RequestPath = new PathString("/assets"),
OnPrepareResponse = ctx =>
{
// Optionally manipulate the response as you'd like. for example, I'm setting a no cache directive here for dev.
ctx.Context.Response.Headers.Append("Cache-Control", "no-cache, no-store, must-revalidate");
}
});
}
Now any request that comes to /assets will be served from the /src/assets folder that can be shared by Foo.Web1 and Foo.Web2. All changes in /src/assets will be reflected in real-time. This work great in dev, but when it comes to production, I don't do this. Instead, I copy the files from /src/assets to Foo.Web1/wwwroot/assets and Foo.Web2/wwwroot/assets in my deployment script so this remapping doesn't need to happen.
Have a look at WebEssentials.AspNetCore.CdnTagHelpers. That's a nuget which makes it easy to serve .net core static resources via CDN. Seems it will solve your problem
The easiest way is to publish your static content to a storage account (blobs), and then connect the CDN to the storage account; you then to change any url references in your project to use the CDN endpoint instead of just the normal relative route.
The bit you want to automate is at build/release time - you can do this with VSTS adding a command task to use AZCOPY to put all your static files in the right place in the storage account; you can do this in your post build task as well if you're not using a CI/CD process for dev/test.
If you need to purge the CDN you can do that via a REST API call, like
POST /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Cdn/profiles/{profileName}/endpoints/{endpointName}/purge?api-version=2016-10-02
See CDN docs here
Its a good route to go down, not enough people realise how easy CDN in azure is and how much it improves your site's feel of snappiness.
[EDIT] - forgot about the AzureFileCopy vsts task, see GitHub project
The solution proposed by Russell Young will definitely work, but perhaps an easier way is to use a custom origin in the CDN and point the origin URL to the original location of the static assets. That way there is no need manage the assets in 2 locations.
Even if the image is changed, overwritten, modified, IIS still serves the cached copy.
I am trying to upload an image from a webcam taken every 15 seconds. The image makes it onto the server but when I refresh the browser with the image FROM the server it does not refresh.
IIS caches the file apparently for more than 2 minutes. I want this to be in real-time. Tried disabling caching everywhere I could think of. No luck.
Embed your image as follows:
<*ImageTag src="WebCamImage.aspx?data={auto-generated guid}" ... >
*ImageTag = img (spam filter won't let me post it)
And create a page (WebCamImage.aspx) that streams the static image file back to the browser while ignoring the "data" request parameter, which is only used to avoid any caching (make sure to set the response content type to "image/jpeg" or whatever is adequate in the #page header).
Are you sure that the image is cached on the server and not on the client. Have you tried requesting the same image from a different client?
If this IS server side caching then this article has all the answers for you:
http://blogs.msdn.com/david.wang/archive/2005/07/07/HOWTO-Use-Kernel-Response-Cache-with-IIS-6.aspx
You are most likely "affected" by the kernel-mode caching.
See that scavenger time?
Scavenger - 120 seconds by default and controlled by the registry key HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\UriScavengerPeriod
That is probably what you experience (2min caching)
Try turning kernel-mode caching off to see if it makes a difference (performance may suffer but it will be no worse than IIS5)