Remove s-maxage from Cache-Control header using Varnish - varnish

We're currently using the s-maxage directive in the Cache-Control header from our origin to control the TTL in Varnish. However, I'd like to remove it from the response before delivery, so that no other caches in the request chain act on it.
I'm currently looking at the header VMOD, to remove s-maxage from the header, but leave the rest of it intact. I believe this could be achieved with something like this:
sub vcl_deliver {
header.regsub(resp, "s-maxage=[0-9]+,?\s?", "")
}
As a newcomer to Varnish, I wanted to sanity-check this approach and make sure there isn't a better way to tackle it?
Appreciate any support or advice.

Replace header at delivery time
The following VCL snippet will strip off the s-maxage attribute from the Cache-Control header before it is sent to the client.
sub vcl_deliver {
set resp.http.cache-control = regsub(resp.http.cache-control,
"(,\s*s-maxage=[0-9]+\s*$)|(\s*s-maxage=[0-9]+\s*,)","");
}
Replace header at storage time
It is also possible to strip off this attribute from the Cache-Control header before it gets stored into a cache object. In that case, you'll use the beresp.http.cache-control variable inside vcl_backend_response.
sub vcl_backend_response {
set beresp.http.cache-control = regsub(beresp.http.cache-control,
"(,\s*s-maxage=[0-9]+\s*$)|(\s*s-maxage=[0-9]+\s*,)","");
}
Using vmod_headerplus
If you're using Varnish Enterprise, you can use the vmod_headerplus module to easily delete header attributes:
vcl 4.1;
import headerplus;
sub vcl_deliver {
headerplus.init(resp);
headerplus.attr_delete("Cache-Control","s-maxage",",");
headerplus.write();
}
vcl 4.1;
import headerplus;
sub vcl_backend_response {
headerplus.init(beresp);
headerplus.attr_delete("Cache-Control","s-maxage",",");
headerplus.write();
}
Although Varnish Enterprise is the commercial version of Varnish Cache, you can still use it without upfront license payments if you use it on AWS, Azure or GCP.
Varnish Enterprise on AWS
Varnish Enterprise on Azure
Varnish Enterprise on GCP

Related

Varnish how to cache for mobile and desktop site

I want to know how to cache for mobile and desktop site. I have mobile and desktop site whose root is written in nginx. Mobile/Desktop is served to the user based on the user-agent whenever the user visits the site, so in this scenario how to cache for mobile and cache for desktop site so that when the user visits the website, Get the right content from the cache.
Please help to write VCL for mobile and desktop cache in Varnish.
You can download https://github.com/varnishcache/varnish-devicedetect/blob/master/devicedetect.vcl and include this file in your main VCL file. By calling the devicedetect subroutine in your main VCL file.
This subroutine will set a X-UA-Device header that contains the device type, which you can then vary on.
Here's an example:
vcl 4.1;
backend default {
.port = "8080";
}
include "devicedetect.vcl";
sub vcl_recv {
call devicedetect;
if(req.http.X-UA-Device ~ "^(mobile|tablet)\-.+$") {
set req.http.X-UA-Device = "mobile";
} else {
set req.http.X-UA-Device = "desktop";
}
}
sub vcl_hash {
hash_data(req.http.X-UA-Device);
}

Is there any way to cache request with auth headers in varnish

Is there any way to cache request with auth headers in varnish?
I want to ignore the auth headers while caching the request
There are various ways to approach this, depending on the importance of auth headers.
1. You don't care about auth
If you don't care about the auth part and if you want to risk serving cached content to unauthorized users, you can just use the following VCL code:
sub vcl_recv {
unset req.http.Authorization;
}
2. Ignore authorization to some extent
It is also possible to care about auth a bit, but not too much.
The following VCL snippet will allow caching even if there is an Authorization header:
sub vcl_recv {
if(req.http.Authorization) {
return(hash);
}
}
The consequence of this is that the initial cache miss will pass through to the backend and will be processed there. Potential unauthorized access will be handled there.
But as soon as the has been dealt with, the object is stored in the cache and the next requests will get cached content regardless of the authorization status of that request.
3. Perform auth on the edge
It is also possible to handle the auth part in Varnish while caching the content.
The following VCL code will handle this:
sub vcl_recv {
if(req.http.Authorization != "Basic YWRtaW46c2VjcmV0") {
return (synth(401, "Restricted"));
}
unset req.http.Authorization;
}
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
}
}
This code will actively inspect the content of the Authorization header and will ensure the username admin is used with password secret.
The YWRtaW46c2VjcmV0 string is nothing more than a base64 encoding of admin:secret.
4. Use vmod_basicauth
A more advanced and flexible way to terminate auth on the edge is by using https://git.gnu.org.ua/vmod-basicauth.git/. This VMOD can be compiled from source and can be downloaded from ftp://download.gnu.org.ua/release/vmod-basicauth.
Assuming the credentials are stored in /var/www/.htpasswd, you can leverage this VMOD to match the Authorization header to the content of the .htpasswd file.
Here's the VCL:
vcl 4.1;
import basicauth;
sub vcl_recv {
if (!basicauth.match("/var/www/.htpasswd",req.http.Authorization)) {
return (synth(401, "Restricted"));
}
unset req.http.Authorization;
}
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
}
}
This is entirely possible but also extremely dangerous: Varnish would return the same cached (authorized) content to all requests.
Example:
User A requests resource Z with proper authentication. Varnish relays the request to backend, caches the response and returns the resource.
User B requests resource Z with proper authentication. They will get the cached resource Z even if Z contains user A's content.
User X requests resource Z with invalid authentication. They will too get the cached resource anyway since the backend is bypassed.
Having said that, you can override Varnish's built-in VCL. Details are documented but the main idea is:
Copy default vcl_recv VCL (for your version) from source and add it to the end of your vcl_recv.
Remove the safeguards from vcl_recv: Just remove vcl_req_authorization which disables caching:
sub vcl_req_authorization {
if (req.http.Authorization) {
# Not cacheable by default.
return (pass);
}
}
In your vcl file issue a return statement at the end so built-in vcl is not used.

Change cache control directive for stale responses in Varnish

This is my first time experimenting with Varnish.
We noticed our CDN not doing any request coalescing, so as an experiment I'm allowed to try out Varnish.
However I can't seem to figure out how to modify the response headers when stale content is served.
Image the following vcl and all backend responses have max-age=60, public, s-maxage=600 as the cache-control directive.
sub vcl_backend_response {
set beresp.grace = 3600s;
return (deliver);
}
When I:
visit /foo
do a varnish soft purge and a subsequent CDN (hard) purge on /foo
visit /foo again
When visiting /foo for the second time I'll immediately get a (stale) response. However I don't want it to have the same cache-control directive anymore, because then my CDN will continue to serve this stale response for another 10 minutes.
Is it possible to change the cache-control directive for stale content to max-age=0, public, s-maxage=10?
Here's the VCL code you need to modify the Cache-Control header for stale content:
vcl 4.1;
sub vcl_deliver {
//set resp.http.x-ttl = obj.ttl;
//set resp.http.x-grace = obj.grace;
if(obj.ttl <= 0s && obj.grace > 0s ) {
set resp.http.Cache-Control = "max-age=0, public, s-maxage=10";
}
}
If you want to debug the process, just uncomment the 2 lines and look for the x-ttl and x-grace response headers to see the remaining values for both timers.
Although you can control the staleness by setting beresp.grace in VCL, you can also set the staleness through the stale-while-revalidate directive in the Cache-Control header.
Here's an example:
Cache-Control: max-age=60, public, s-maxage=600, stale-while-revalidate=3600
This will allow the browser to cache for a minute, allow intermediary servers to cache for 10 minutes and set an allowed staleness of an hour while revalidating outdated content.

Set Varnish headers conditionally based on client.ip

How would I use client.ip as a conditional in setting headers in the fetch section of a Varnish 3.0 VCL? I have some troubleshooting headers that I like to set to solve caching issues, however, I don't want them publicly visible. I'd love to be able to whitelist the headers for my ip address only.
Is there any way to access client.ip in _fetch?
You can best set all troubleshooting headers in you _recv without any conditions and remove them in you vcl deliver. this way you dont need to add the same ip check on every conditional header
if you want to use an ip range you can use the following code
acl debug {
"your ip adress1";
"you ip adress 2";
}
in you vcl_recv
if (!client.ip ~ debug) {
set req.http.x-debug = "debug";
}
in you vcl_deliver
if(!req.htt.x-debug){
remove resp.http.debugheader1;
remove resp.http.debugheader2;
}

Trigger.IO CORS Setup

I'm trying to get CORS working on my trigger.io app:
I've got the following setup in my .htaccess
Header set Access-Control-Allow-Headers: "Accept,Origin,Content-Type,X-Requested-With"
Header set Access-Control-Allow-Methods "GET,PUT,POST,DELETE,OPTIONS"
Header set Access-Control-Allow-Credentials: "true"
Header set Access-Control-Allow-Origin "http://localhost:3000,content://io.trigger.forge99d5a0b8621e11e28cc2123139286d0c"
Running the trigger App in the web (localhost:3000) works fine.
But when I deploy it to an (android) device I see the following error in the debug output:
[ERROR] XMLHttpRequest cannot load {link}http://mydevtest.lan/api/auth/currentuser.{/link} Origin content://io.trigger.forge99d5a0b8621e11e28cc2123139286d0c is not allowed by Access-Control-Allow-Origin. -- From line 1 of null
I'm fearing that setting content:// in the Access-Control-Allow-Origin header is not legal.
The Access-Control-Allow-Origin header as you have it is invalid. Valid values are either '*', or a space separated list of origins. One of the following should work:
Header set Access-Control-Allow-Origin "*"
or
Header set Access-Control-Allow-Origin "http://localhost:3000 content://io.trigger.forge99d5a0b8621e11e28cc2123139286d0c"
Note that I've never tested the latter form (with multiple origins). While the CORS spec allows it, browsers may not yet support it.
One other thing you could do is read in the value of the Origin header, validate it on your server (i.e. manually check that the value equals either "http://localhost:3000" or "content://io.trigger.forge99d5a0b8621e11e28cc2123139286d0c"), and then echo only that value in the Access-Control-Allow-Origin response header. However this requires a little more work since it introduces some server-side conditional processing.
I also fear that content:// is not allowed in CORS, could you try setting Access-Control-Allow_origin to *, if that works then that is probably the problem.
A better solution would be to avoid doing XHR requests and use forge.request.ajax which will make the request from native code and avoid any cross domain restrictions. You can find the documentation for that here http://docs.trigger.io/en/v1.4/modules/request.html#modules-request

Resources