I have enabled pagespeed module and find that for some resources (image, js and css) that are re-written by pagespeed the cache is set to the default 5 minutes. Few other resources (image, js and css) re-written by pagespeed has Cache-Control: max-age=31536000.
I explicitly give set by ExpiresDefault to 1 year for all my static resources in .htaccess.
The response i get has this:
Cache-Control:max-age=300,private
I am expecting:
Cache-Control:max-age=31536000,private
Suggestions and pointers are appreciated.
mod_pagespeed only serves responses with Cache-Control:max-age=300,private if the Hash in the URL doesn't match the content. This can happen normally when A) the contents of the resource changed recently and so there are a mixture of requests for both old and new URLs for some time or B) the rewriting does not finish in time when serving the resource.
This is most likely to happen if the resource request goes to a different server than the HTML request. You can try flushing the cache and see if this clears up.
I tested my site using yslow and I got Grade B in Configure Entity tags.
I tried below condition in .htaccess and my site's Etags are removed, but not from JS included by CDN like validate.min.js
Header unset Pragma
FileETag None
Header unset ETag
Here is the image,
How to configure etags from Validate plugin from CDN.
It can be possible duplicate of How to off Etag with htaccess? except that here I am getting problem with js included by CDN.
I believe the answer is: you can't. Configurations like the ETags can only be controlled by the host, in this cast the CDN.
I think it's safe to not worry about this for your site. Loading that JS from a CDN is already a win, and this CDN is correctly supporting top performance rules like minification, gzip compression, and future expiration dates.
I'm trying to set up my IIS server as an origin server for a CDN. I have solved some issues already for example that IIS doesn't give gziped content to proxies (if they have the via header) and also that frequentHitThreshold problem.
My CDN supplier pointed out that another problem with IIS is that it doesn't return a "Vary" header if the client doesn't request the content gziped. According to them the problem is that if for some reason the first client that request the content doesn't want the content gziped the CDN then doesn't request a new version of the file since the Vary header doesn't indicate that it should return two different files depending on "Accept-Encoding".
My only solution so far is to add "Vary: Accept-Encoding" as a custom header but since IIS automatically add this vary header when gziped is requested so i end up with multiple values like "Vary: Accept-Encoding, Accept-Encoding".
Anyone have any solution to this? Or can confirm that it's a real issue.
This is a real issue. IIS gzip module overwrites existing Vary headers. Please vote on this MS Connect issue. Related article here.
This issue is now addressed by an official patch to IIS. To download and further info, visit http://support.microsoft.com/kb/2877816
Erez Benari, IIS PM
I use Amazon Cloudfront to host all my site's images and videos, to serve them faster to my users which are pretty scattered across the globe. I also apply pretty aggressive forward caching to the elements hosted on Cloudfront, setting Cache-Controlto public, max-age=7776000.
I've recently discovered to my annoyance that third party sites are hotlinking to my Cloudfront server to display images on their own pages, without authorization.
I've configured .htaccessto prevent hotlinking on my own server, but haven't found a way of doing this on Cloudfront, which doesn't seem to support the feature natively. And, annoyingly, Amazon's Bucket Policies, which could be used to prevent hotlinking, have effect only on S3, they have no effect on CloudFront distributions [link]. If you want to take advantage of the policies you have to serve your content from S3 directly.
Scouring my server logs for hotlinkers and manually changing the file names isn't really a realistic option, although I've been doing this to end the most blatant offenses.
You can forward the Referer header to your origin
Go to CloudFront settings
Edit Distributions settings for a distribution
Go to the Behaviors tab and edit or create a behavior
Set Forward Headers to Whitelist
Add Referer as a whitelisted header
Save the settings in the bottom right corner
Make sure to handle the Referer header on your origin as well.
We had numerous hotlinking issues. In the end we created css sprites for many of our images. Either adding white space to the bottom/sides or combining images together.
We displayed them correctly on our pages using CSS, but any hotlinks would show the images incorrectly unless they copied the CSS/HTML as well.
We've found that they don't bother (or don't know how).
The official approach is to use signed urls for your media. For each media piece that you want to distribute, you can generate a specially crafted url that works in a given constraint of time and source IPs.
One approach for static pages, is to generate temporary urls for the medias included in that page, that are valid for 2x the duration as the page's caching time. Let's say your page's caching time is 1 day. Every 2 days, the links would be invalidated, which obligates the hotlinkers to update their urls. It's not foolproof, as they can build tools to get the new urls automatically but it should prevent most people.
If your page is dynamic, you don't need to worry to trash your page's cache so you can simply generate urls that are only working for the requester's IP.
As of Oct. 2015, you can use AWS WAF to restrict access to Cloudfront files. Here's an article from AWS that announces WAF and explains what you can do with it. Here's an article that helped me setup my first ACL to restrict access based on the referrer.
Basically, I created a new ACL with a default action of DENY. I added a rule that checks the end of the referer header string for my domain name (lowercase). If it passes that rule, it ALLOWS access.
After assigning my ACL to my Cloudfront distribution, I tried to load one of my data files directly in Chrome and I got this error:
As far as I know, there is currently no solution, but I have a few possibly relevant, possibly irrelevant suggestions...
First: Numerous people have asked this on the Cloudfront support forums. See here and here, for example.
Clearly AWS benefits from hotlinking: the more hits, the more they charge us for! I think we (Cloudfront users) need to start some sort of heavily orchestrated campaign to get them to offer referer checking as a feature.
Another temporary solution I've thought of is changing the CNAME I use to send traffic to cloudfront/s3. So let's say you currently send all your images to:
cdn.blahblahblah.com (which redirects to some cloudfront/s3 bucket)
You could change it to cdn2.blahblahblah.com and delete the DNS entry for cdn.blahblahblah.com
As a DNS change, that would knock out all the people currently hotlinking before their traffic got anywhere near your server: the DNS entry would simply fail to look up. You'd have to keep changing the cdn CNAME to make this effective (say once a month?), but it would work.
It's actually a bigger problem than it seems because it means people can scrape entire copies of your website's pages (including the images) much more easily - so it's not just the images you lose and not just that you're paying to serve those images. Search engines sometimes conclude your pages are the copies and the copies are the originals... and bang goes your traffic.
I am thinking of abandoning Cloudfront in favor of a strategically positioned, super-fast dedicated server (serving all content to the entire world from one place) to give me much more control over such things.
Anyway, I hope someone else has a better answer!
This question mentioned image and video files.
Referer checking cannot be used to protect multimedia resources from hotlinking because some mobile browsers do not send referer header when requesting for an audio or video file played using HTML5.
I am sure of that about Safari and Chrome on iPhone and Safari on Android.
Too bad! Thank you, Apple and Google.
How about using Signed cookies ? Create signed cookie using custom policy which also supports various kind of restrictions you want to set and also it is wildcard.
Our IIS server has Dynamic and Static HTML Compression enabled, but when I browse to our website and view the Response Headers in Fiddler, I only see the "Content-Encoding: gzip" header for one resource (a flash file).
Why would the other response types not have this header? Does it mean that compression is NOT working for the other responses?
The only way to be 100% sure that compression is active is to compare the size of the downloaded resource against the original file on the server. The network tab of the Firebug extension can help you here.
It looks like our company network was actually stripping out the Content-Encoding header. (I have no idea why). When I browse from home the gzipping seems to work fine. This post on StackExchange.com helped me figure it out.