We have an interesting behaviour that we are trying to understand/workaround.
We have an Azure Function running on a consumption host that has nothing but proxies on it.
One of these proxies points to an Azure CDN endpoint, that in turn points to an Azure Storage Account.
From time to time we update the file in the storage account and purge the CDN endpoint.
Requesting the file directly from the CDN returns the (correct) new file data.
However, the Function Proxy url continues to return the (incorrect) old file data.
Browser caches are disabled/cleared and all that normal stuff.
We can see the Last-Modified headers are different, so clearly the proxy is returning the old file.
Further, adding a querystring to the proxy URL - anything you like (we used ?v=1) would return the (correct) new file. Removing the querystring gets us back to the old file again.
Is this behaviour intentional? I have read UserVoice requests where people wanted caching added to Functions and it was explicitly declined due to the number of alternatives available. I see no other explanation for this behaviour, though.
Does anyone know how to disable/defeat proxy response caching?
Related
Since most of CDNs update the cache data only after it reaches an expiry, what if the origin server is updated before that. Does the CDN serve the old data until updated upon expiry?
Generally they have a callback method for you to let them know if a content changed and needs to be updated/re-cached. So it won't serve the old content until the next refresh.
You can see the documentation for Cloudflare-Purge Cache and Azure.
Yes, the CDN will serve the older content even when the origin is updated. This is because no one really told the CDN that there is an updated asset at the origin.
There are a couple of ways you can make sure the CDN always has the latest asset cached. The easiest way is to purge the cache each time there is a new/updated asset. This can be built into your CI/CD pipeline so that there is an auto-purge each time you deploy a new version
The other way I've seen a lot of people do is to have a version as part of the URL itself. So that when the asset is updated, it would have a new version as part of the object name. So when the browser makes a request, it is automatically requesting the new object.
Hopefully, this helps.
I have a load of images stored as blobs in an Azure container.
I am attempting to get the browser (Chrome) to cache these images to save bandwidth. I have read many posts stating that this can be achieved by setting the Cache-Control response header. I am using the Microsoft Azure Storage Explorer to modify these headers, e.g:
public, max-age=7776000
When loading this image (https://XXXXXXXX.blob.core.windows.net/XXXXXXXX/Themes/summer.jpg) this is what I see using Google Chrome's Developer Tools:
It doesn't make any difference to the caching of the image. I have tried many different permutations of the allowed CacheControl attributes but I don't see any caching going on at all. The status is always 200, but I was expecting 304 for a cache hit. Is this correct?
Whichever CacheControl string I provide is always displayed in Chrome's results; it just doesn't seem to make any difference to the caching aspect. I've tried variations of public, private, max-age, s-maxage, must-revalidate. And just to be complete, no-cache and no-store. No differences were observed.
The above image takes 900ms+ to load for me. However, when saved locally, the same image takes 19ms. I would expect if the browser was caching the image then it's timing would be equivalent to the local time.
Other posts suggest that an Azure CDN be used. However, I don't want to go down this route as the site that uses these images would not need that.
Am I missing a setting in Azure to allow caching? Loading the images directly in the browser, or within a web page makes no difference either.
Can anyone provide assistance? Let me know if any other information is required.
The CacheControl settings should take effect.
When using chrome, please make sure you didn't select the option "disable-cache".
I can see the expected behavior when set max-age=xx for CacheControl in chrome.
I uploaded a 295437KB file to Azure private Blob. I connected Azure Verizon Premium CDN via an app service that streams it from the Blob. The file returned is truncated, at different lengths, less than the full file length. Several 10s of MB shorter.
I have checked the file size on the Blob (correct) and also tested the call that retrieves it from the App Service (correct).
So it appears to be on the CDN side. Is there some timeout or request limit I can set on the CDN to alleviate this?
Here is an example of a CDN call that truncates the file:
https://holojem-prod-files-cdn.azureedge.net/artifacts/11/283/332/0008%20Watch%20This%20Video.mp4?DYiNiOt7Q_9xGaZhscklXmcn0tlpDU649hQUD2n7WzgxfirhVQyzwch2-szLjDmUjAshEfe2ZsQ6ejEDR46QvHVKf5WneWFAz1vOQppOPfcBq3KCS11mZ3LpnfFGEzR9RtnsvKyvVSadMXuFy8cLPLYiy4S2boiJ0S-YhQdODqFY7_MbeiJB
And here is the underlying API (mine) that the CDN points to:
I get the full video if I hit that. It is 295,437 KB.
http://holojem-prod-cdn-api.azurewebsites.net/artifacts/11/283/332/0008%20Watch%20This%20Video.mp4?DYiNiOt7Q_9xGaZhscklXmcn0tlpDU649hQUD2n7WzgxfirhVQyzwch2-szLjDmUjAshEfe2ZsQ6ejEDR46QvHVKf5WneWFAz1vOQppOPfcBq3KCS11mZ3LpnfFGEzR9RtnsvKyvVSadMXuFy8cLPLYiy4S2boiJ0S-YhQdODqFY7_MbeiJB
Interestingly, the results are not consistent. When I hit the origin directly a second time from Postman, I got a file of 260,276 KB
When I downloaded from the origin in Chrome, I got 260,744 the first time and 262,144 KB the second time.
The origin is an ASPNET Core Web API
According to your CDN url, I found the CDN have compressed the file when I downloaded it.
You could run fiddler to catch the request as below:
According to this article : To check whether your files are being returned compressed, you need to use a tool like Fiddler or your browser's developer tools. Check the HTTP response headers returned with your cached CDN content. If there is a header named Content-Encoding with a value of gzip, bzip2, or deflate, your content is compressed.
So I suggest you could firstly check the compress setting in your azure portal.
More details, you could refer to this article.
Update:
According to your two url, I have download both video. I found the website's size is a little more than the CDN's video.
The result is as below:
I have also compare the difference between these two file by using mediainfo --fullscan.
Just the Overall bit rate not the same.
One is 17.7 Mbps, another oen is 17.6 Mbps. There are both two minutes.
So I guess may be something wrong with your website to get the blob stream code.I suggest you could recheck it. If you still face the same issue, I suggest you could post some relevant code and the blob video url for us to reproduce the issue.
I have written a Cygwin app that uploads (using the REST API PUT operation) Block Blobs to my Azure storage account, and it works well for different size blobs when using HTTP. However, use of SSL (i.e. PUT using HTTPS) fails for Blobs greater than 5.5MB. Blobs less than 5.5MB upload correctly. Anything greater and I find that the TCP session (as seen by Wireshark) reports a dwindling window size that goes to 0 once the aforementioned number of bytes are transferred. The failure is repeatable and consistent. As a point of reference, PUT operations against my Google/AWS/HP cloud storage accounts work fine when using HTTPS for various object sizes, which suggests the problem is not my client but specific to the HTTPS implementation on the MSAZURE storage servers.
If I upload the 5.5MB blob as two separate uploads of 4MB and 1.5MB followed by a PUT Block List, the operation succeeds as long as the two uploads used separate HTTPS sessions. Notice the emphasis on separate. That same operation fails if I attempt to maintain an HTTPS session across both uploads.
Any ideas on why I might be seeing this odd behavior with MS Azure? Same PUT operation with HTTPS works ok with AWS/Google/HP cloud storage servers.
Thank you for reporting this and we apologize for the inconvenience. We have managed to recreate the issue and have filed a bug. Unfortunately we cannot share a timeline for the fix at this time, but we will respond to this forum when the fix has been deployed. In the meantime, a plausible workaround (and a recommended best practice) is to break large uploads into smaller chunks (using the Put Block and Put Block List APIs), thus enabling the client to parallelize the upload.
This bug has now been fixed and the operation should now complete as expected.
Searched Google and so - no luck.
Just got this message in Azure for 3 CDN endpoints.
There seems no way to know what is going on without MS support. It is a test account and I do not recall setting this. I have been through similar obfuscated MS error messages only to discover that Azure had crashed.
What does it mean?
This isn't really a direct answer, but could help with the general problem of "what happens if the CDN goes down?".
There is a recent development called the "Progressive Web App".
Basically unless served by localhost, everything has to be over https, but script is cached as a local application in your browser.
When your app makes requests to the registered domain, these are intercepted by a callback you put in your serviceWorker.js, so you can cache even application data locally, and sync the local data occasionally with the server (or on receive events if you're using webSockets).
Since the Service Worker intercepts REST calls to the registered domain, this in theory makes it fairly easy to add to just about any framework.
https://developers.google.com/web/fundamentals/getting-started/codelabs/your-first-pwapp/
Sometimes there is a (global) problem with the CDN. It happend before.
You can check the azure CDN status on this page: https://azure.microsoft.com/en-us/status/
At this moment everything looks good, you still have problems?