I was really surprised that I've had this much trouble finding this answer anywhere, but I can't.
With the built-in VCL in vcl_recv, etc., in Varnish 4.0+, does Varnish cache 404 responses by default?
Only the following status codes will be cached by default:
200: OK
203: Non-Authoritative Information
300: Multiple Choices
301: Moved Permanently
302: Moved Temporarily
304: Not modified
307: Temporary Redirect
410: Gone
404: Not Found
Source: http://book.varnish-software.com/4.0/chapters/VCL_Basics.html
Related
I tried to exclude robots.txt form varnish cache by using the following lines of code in default.vcl
if(req.url ~ "^/robots\.txt$") {
return(pass);
}
Now Network tab in dev tools, it shows a Age: 0 and X-Cache:MISS. But,for some reason varnish does not exclude the file from being cached. I even deleted the file from its location. But still its loading the url https://www.example.com/robots.txt
I also purged varnish cache using following commands
curl -X PURGE www.example.com/robots.txt
and
varnishadm "ban req.http.host == www.example.com && req.url ~ ^/robots.txt"
and
varnishadm "ban req.http.host ~ www.example.com && req.url ~ ^/robots.txt"
It shows the 200 Purged message, but still no luck.
Can anyone help me out ?
"I even deleted the file from its location. But still its loading the url https://www.example.com/robots.txt" -- may be your browser is caching it.
Sending PURGE request to varnish will only remove the object from the cache, not from the backend, so if you PURGE something from varnish and then send a GET request to it, it will serve the object to you for sure.
If you want it to be lost forever, you need to remove it from your backend.
I am using W3 Total Cache with Amazon cloudfront. I have in my htaccess file:
# BEGIN W3TC CDN
<FilesMatch "\.(ttf|ttc|otf|eot|woff|font.css)$">
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
</FilesMatch>
# END W3TC CDN
But still getting error:
Font from origin 'https://example.cloudfront.net' has been blocked from loading by Cross-Origin Resource Sharing policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://www.example.com' is therefore not allowed access.
Why is this happening?
Found the solution in this link: https://www.naschenweng.info/2014/09/23/wordpress-w3-total-cache-cloudfront-font-cors-issue/.
You need to change the CloudFront distribution's behaviors settings:
Change “Forward Headers” from “None” to “Whitelist”
Add “Origin” to the “Whitelist Headers”
Make sure that “Use Origin Cache Headers” is checked
Then invalidate the cached fonts.
Wrestling with this for days, and think I finally fixed it. Here are some things to check:
The webserver config should add the proper header. Apache syntax is listed in the question. Here's Nginx syntax that I used:
location ~* \.(eot|ttf|woff|woff2)$ {
add_header Access-Control-Allow-Origin '*';
}
Within W3TC > Performance > CDN > Custom File List, I added the following to upload the actual font files:
{plugins_dir}/*.ttf
{plugins_dir}/*.woff
While you're there, set the Theme file types to upload to the following. Per #Yao's link, the default separators are inconsistent (should all be semicolons, not commas)
*.css;*.js;*.gif;*.png;*.jpg;*.ico;*.ttf;*.otf;*.woff;*.less
In S3 > Permissions > CORS Configuration, change the default
<AllowedHeader>Authorization</AllowedHeader>
to:
<AllowedHeader>*</AllowedHeader>
You should start seeing the necessary Access-Control-Allow-Origin header in the response.
In CloudFront > Distribution > Behaviors, make the following changes:
Change Allowed HTTP Methods to GET, HEAD, OPTIONS (you need OPTIONS)
Change Forward Headers to Whitelist
Under Whitelist Headers, Add >> Origin
To test:
curl -I -s -X GET -H "Origin: www.example.com" https://abcdefg543210.cloudfront.net/wp-content/path/to/foo.ttf
This should give you back the following header:
Access-Control-Allow-Origin: *
X-Cache: Miss from cloudfront
I found this blog post to be pretty helpful:
http://blog.celingest.com/en/2014/10/02/tutorial-using-cors-with-cloudfront-and-s3/
basicly It's because the font isn't set to be shared outside of the domain that you are on, so you can just use it as a resourse for, in this case, https://example.cloudfront.net
This can be changed in the webserver settings though.
The problem may not always be with the origin settings on Nginx or Apache at your web server end.
You will also need to enable CORS on your S3 AWS account for this to work correctly.
I was using http and recently installed ssl certificate and now images are no longer loading.
But i figured it out that it's the following line in my .htaccess.
What does this line do and how do i modify it to to allow https?
RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]
Do i really need this line?
Thanks.
Rule says return FORBIDDEN HTTP 403 to the image file requests. You may safely comment out/remove the directive.
I have a simple 404 redirect in a .htaccess file which works fine for Http requests:-
ErrorDocument 404 /apps/handle_error.php
The handle_error.php returns an appropriate PNG image depending on the URL path. However, when I replace the request with Https, it returns the default 404 error page and never does the redirect.
I know my SSL certificate is installed correctly as I get valid content returned for normal static pages. Is there something I am missing in my htaccess file regarding 404 error redirects for Https requests?
(A similar question was asked here but with no clear resolution)
After much soul searching I found the answer, for my specific case anyway.
My Apache2, mod_ssl and mod_rewrite was not configured correctly to work together. The default AllowOverride setting in my SSL config was set to "None" and mod_rewrite requires a minimum of AllowOverride of "FileInfo".
(To be fair, I got my answer from ServerFault)
I want to send 410 status code if there is any ? on the URL. Because the site is only works with SEO enabled URLs (Like www.domain.com/seo-enabled-urls).
With the StatckOverflow community support did this 301 redirection that works perfectly:
RewriteBase /
RewriteRule ^romance-package-two\.html$ http://www.domain.com/wedding-champagne [QSA,NC,R=301,L]
In the same way I want to add 410 message code that will tell Google (like search engines) permanently deleted so we wont have problem with SEO unknown or not found URLs.
If the URL looks like this
www.domain.com/seo-enabled-urls?param1=val1¶m2=val2
I want to keep the same URL but just with that 410 code like this
RewriteCond %{query_string} ^(.*&)?$ [NC]
RewriteRule ^1$ same url code [QSA,NC,R=410,L]
Please help me.
You can't redirect with a HTTP 410 code -- 410 means that the requested page is permanently gone, and that there's no replacement:
410 Gone
The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. This response is cacheable unless indicated otherwise.
-- http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.11
The corresponding definition for HTTP 301 indicates that it's already exactly what you wanted:
301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.
-- http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.2
The accepted answer is no longer correct at least as far back as Apache 2.0, where mod_rewrite supports a 410 GONE target:
'gone|G' (force URL to be gone)
This forces the current URL to be gone - it immediately sends back a HTTP response of 410 (GONE). Use this flag to mark pages which no longer exist as gone.
(see http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewriterule)
That means that a rule such as you had above should simply be:
RewriteCond %{query_string} ^(.*&)?$ [NC]
RewriteRule ^1$ - [G]