I created an ad hoc application to test the azure cache features to study for the az204.
It is a simple node application in app service that renders a large image using lorem picsum.
<img src="https://picsum.photos/2000" style="width: 100%;">
I created a Standard Azure CDN Profile and added the Endpoint.
Then Set the global rule to be override with Always Cache expiration 30 min.
Expected Result:
From this moment I expected my application to cache the image, witch means that when I reload the page accessing from the CDN's url I should get the same image as before for at least 30 minute.
Actual Result:
But the actual result is that when I load the page it is loading always a different image as it would without a cdn.
I also tried by creating a new rule for image type jpeg override 30 min, but It didn't worked.
How can I return a cached image from lorem picsum using the Frontdoor Azure Standard Cache CDN?
I tried to reproduce the same in my environment, If you are trying to access Azure CDN caching rules to cache expiration duration the condition passes though the storage blob files.
I have created front door CDN profile endpoint and added caching behavior override like below:
This cache rule applies for your storage container file like below:
In storage container -> files. This files we can access through url and Override and Set if missing caching behavior.
https://lorem.azureedge.net/container1/lorem pic 1.jpg
Set TTL as 86400 or 1-day cache control Cachecontrol="public,max-age=86400"
Changing the cache duration while accessing the url lorem picsum is not possible
When you check front door designer these are the different cache behaviors which can be implemented at different edge.
References:
Azure Storage Azure Front Door Azure CDN
Integrate Caching And CDNs Within Solutions by Thiago Vivas
Related
I encounter the problem reported in X-cache hit after Azure CDN Purge , that if I set a long Always Cache expiration / TTL of for example 1 year, the local browser will not fetch new content from the Azure CDN EVEN one does a manual Azure CDN purge, because the browser just takes the local cache.
In our web setup, we would like the Azure CDN to cache everthing UNTIL our next deployment, thats why we set a long global "Always Cache expiration" of 1 year. We assumed that the a manual CDN purge will force the browser to refresh, but it didnt because of the local browser cache where the TTL 1 year is locked. Now users still get old apps after our deployments.
Can this be solved with additional directives?
When we set "Last-Modified" : "<date>" as global directive after each deployment we get an error. ETag does not seem to be supported by standard MS CDN plan.
{"ErrorMessage":"Invalid RulesEngineConfig: (Header \"Last-Modified\" is not allowed to be modified by the rules engine)."}
What are the exact rule settings in Azure CDN to solve this use case and control that the browser gets new content only after we deployed?
The browser is doing what you instructed it to do given the caching rules.
What you could do is configure the assets with unique filenames or a cache fingerprint and a very long cache TTL. But the HTML page with a shorter TTL. So when you update assets, users will get a fresh HTML page and pull the latest assets.
What we already have in place
Azure Blob Storage with static files, ir our case images
Azure CDN in front of the Blob Storage
ASP.NET Core Service
uploading images to the blob storage
saving meta data about the logical collection of images, called Asset
let's call this the media service
How we use that
Let's say we have another service called the product service.
Each product has a product picture associated with it.
We create a new asset in the media service and get back an AssetId.
The product service now stores this AssetId in his local database
What we're looking for
As long as we operate with the assets meta data in the back end life's good.
But at the end of the day we want to visualize the pictures in a WebUI. To do that we need to have the Url pointing back to the image on the blob storage / CDN.
Right now the back end actively resolves the Urls before the data flows to the WebUI. But there must be better way of doing this.
One feasible approach that comes to mind is to have a simple resolver middleware
/images/{assetid}/{assetType} -> Redirect to CDN Url
but then each call flows over this middleware instead of going to the CDN in first place before falling back to a resolver to get the correct Url.
What do I miss? What's the best practice any way to solve such basic thing in the micro-service area?
Thanks
I would not use a middleware component to redirect the URLs, because as you mentioned, the client will always contact first your middleware component, and there is an additional unnecessary call and overhead. The nature of the CDN is to serve the assets in a global distributed network, so your assets will be indexed to the CDN edges and it will be served from the closest edge to the client. You should be able to use the WebUI component and use the CDN Urls (https://.azureedge.net/*). In case you need to use redirect rules, you can use the CDN features to perform redirect rules. Note: Depending on the CDN service you are using, you may need to upgrade CDN pricing tier to Premium. Eg. Verizon you need to use Premium tier to have ability for redirect rules.
I'm using Azure CLI to purge the contents from Azure CDN endpoint. I got a reference from Microsoft Docs: https://learn.microsoft.com/en-us/cli/azure/cdn/endpoint?view=azure-cli-latest
https://learn.microsoft.com/en-us/azure/cdn/cdn-purge-endpoint
I use the following command to refresh specific png file as shown below:
az cdn endpoint purge -g cdnRG --profile-name cdnprofile2 --content-paths "/img/cdn.png" --name cdnprofileendpoint2
Command executed successfully, though surprised that content is not refreshing or sometime it takes time.
is it the acceptable pattern?
Kindly advise.
Since purging an Azure CDN endpoint only clears the cached content on the CDN edge servers. Any downstream caches, such as proxy servers and local browser caches, may still hold a cached copy of the file. You can force a downstream client to request the latest version of your file by giving it a unique name every time you update it, or by taking advantage of query string caching.
I suggest purging the same path contents in the Azure portal comparing to purge it with Azure CLI command. You also try to purge CDN endpoint with Azure Powershell.
The important thing is that the CDN provider takes influence on the purging time.
Purge requests take approximately 10 minutes to process with Azure CDN
from Microsoft, approximately 2 minutes with Azure CDN from Verizon
(standard and premium), and approximately 10 seconds with Azure CDN
from Akamai. Azure CDN has a limit of 50 concurrent purge requests at
any given time at the profile level.
I've created a cdn service and set the origin url to a azure website which runs iis-node that serves a NodeJS web application, when trying to access static resources the cdn always return 404 while the same pathname works when accessing the resource through the origin url,
anything that i might be missing here?
thanks!
Turns out it takes at least an hour until the content is being populated, during that time the CDN return 404.
Based on the article
A content delivery network (CDN) is a distributed network of servers
that can efficiently deliver web content to users. CDNs store cached
content on edge servers in point-of-presence (POP) locations that are
close to end users, to minimize latency.
Azure Content Delivery Network (CDN) offers developers a global
solution for rapidly delivering high-bandwidth content to users by
caching their content at strategically placed physical nodes across
the world
What you can do though to fasten the process is to just log in to Azure portal, go to the
CDN profiles section, pick your CDN, And Click on Purge on the top menu.
It should automatically update the content of the cache memory for your endpoint/s and you shouldn't wait hours.
Hope it helps. Cheers
In case anyone else comes across this thread - I built an Azure CDN (Verizon Standard) and it took almost 24 hours for the images/files to pull up using the Endpoint Hostname.
Unfortunately, in my case I thought I was doing something wrong so I kept changing it which probably prolonged it. When I finally realized I did everything correct I decided to wait it out.
I had the same issue but the cause for me was that the Azure CDN only serves content under the /CDN path. So if you are trying to access anything on http://yourdomain.com/here.jpg you'll get a 404 (unless you have a URL rewwrite in place)
I am using azure blob storage to store images in a public container and embedding them in a public website. Everything works fine, blobs are publicly available on xxxxx.blob.core.windows.net the instant i upload them. I wanted to use Azure CDN for their edge caching infrastructure and set up one at xxxxx.vo.msecnd.net.
But now, when i point my images to the CDN, it returns 404 for a good 15 mins or so, then it starts serving. It's mentioned on their documentation that we should not use CDN for high violatile or frequently changing blobs, but a simple CMS with image upload feature for a public site warrants a CDN isn't it?
I am in exactly the same situation at the moment for product images that are uploaded to my e-commerce site. I prefer to use Azure CDN on top of Azure blob storage for all of the obvious reasons but cannot wait 15 minutes for the image to be available.
For now I have resolved to store the blob storage URL initially but then later rewrite it to use the CDN domain via an Azure WebJob running once daily. It seems like an unnecessary amount of extra work but I haven't yet found a better option and really want to use the Azure CDN.
What I'm doing right now... for website related images and files I upload manually before deployment (https://abc.blob.core.windows.net/cdn) and If website User upload an image or file using my website, Internally I upload that file on blob storage (separate container not CDN) using CloudBlobClient
CDN is used for static content delivery, but in your case you need dynamic content delivery via CDN. You could use Cloud Service + CDN. This makes Dynamic contents delivered from CDN using ASP.net Caching concepts.
Please refer this link for more details: Using the Windows Azure Content Delivery Network (CDN)
CDN enables a user to fetch content from a CDN-POP that is geographically closest to the user thus allowing lower read latencies.
Without a CDN, every request would reach the origin server (in your case Azure Storage). The low latency offered by CDN is achieved when there are cache hits. On a cache miss, a CDN-POP will fetch the content from the origin server reducing the latency benefit offered by CDN. Cache hits are usually dependent on whether the content is static (results in cache hits) or dynamic (results in cache miss) and its popularity (hot objects result in cache hit).
Your choice of using a CDN or not depends on a) whether your files are static or dynamic, if dynamic then the benefit of using a CDN is lower b) whether low latency is important to your application and c) Request rate : With low number of requests your files are likely to be cached-out so a CDN may not be that useful and d) Whether you have high scalability requirements. Note, Azure storage has the following scalability limits. If your application exceeds the scalability limits of azure storage then it is recommended to use a CDN