Make varnish serve request from cache (cookies are cleaned)

Make varnish serve request from cache (cookies are cleaned) - varnish

HTML
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-54516992-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-54516992-1');
</script>
</head>
<body>
test
</body>
</html>
Backend. This is Python + Django. Please, don't be afraid. I just set a cookie here (render_to_response) and then I need a place to stop at a breakpoint (below in the code it is shown by a comment where it is).
class HomeView(TemplateView):
template_name = "home/home.html"
def render_to_response(self, context, **response_kwargs):
response = super(TemplateView, self).render_to_response(context, **response_kwargs)
response.set_cookie('sort_on', 'title')
return response
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
return context # Breakpoint.
varnishd For learning purposes I just clean all cookies.
$ varnishd -V
varnishd (varnish-6.0.6 revision 29a1a8243dbef3d973aec28dc90403188c1dc8e7)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2019 Varnish Software AS
VCL
vcl 4.1;
backend default {
.host = "127.0.0.1";
.port = "8080";
}
sub vcl_recv {
call remove_proprietary_cookies;
if (req.http.Cookie) {
return (pass);
}
}
sub remove_proprietary_cookies{
set req.http.Cookie = regsuball(req.http.Cookie, ".*", "");
}
Then I reload the page a couple of times for the cookie to be definitely there.
In Chrome:
document.cookie
"sort_on=title; _ga=GA1.1.988164671.1586849704; _gid=GA1.1.995393496.1586849704; _gat_gtag_UA_54516992_1=1"
Picture of cookies (duplicates of the text above):
Ok. We've checked that Cookie is set. Now let's check that the cookie is properly cleaned.
I stop at the braakpoint and check cookie. The value is {}.
Well. All cookies seems to be cleaned.
The problem:
When reloading I constantly come to the breakpoint. This means that Varnish passes the request to the backend. In other words if (req.http.Cookie) not working as I expect. I expect that at the previous step I have removed cookies. Then I check if cookies exist. And there shouldn't be any.
Could you help me:
Understand what is going on here.
Organize everything so that I should definitely remove cookies if I have removed them incorrectly.
Make Varnish serve this request from cache without passing it to the backend.
==============ADDED ON 16th of APRIL=================
+++++++++++++++++++++++++++++++++++++++++++++++++++++
I have upgraded Varnish to 6.4:
michael#michael:~$ varnishd -V
varnishd (varnish-6.4.0 revision 13f137934ec1cf14af66baf7896311115ee35598)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2020 Varnish Software AS
What we are going to test:
Set Google Analytics cookies and a cookie at the backend.
Let Varnish remove all cookies and cache the response.
Check that request is passed to the backend only once.
Then I organized nginx behind Varnish (only to log requests). Nginx listens at 8090. This is log config:
log_format main '[$time_local] $http_cookie';
Cookies:
In HTML I organize Google Analytics' tracking.
At the backend I set cookie called "sort_on" with value "title".
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
If we don't cut cookies, the log looks this (this is just for COMPARISON):
127.0.0.1 - - [16/Apr/2020:08:11:05 +0300] "GET / HTTP/1.1" sort_on=title; _ga=GA1.1.236170900.1587013419; _gid=GA1.1.2033785209.1587013419 200 334 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36" "127.0.0.1"
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
varnish.ctl
vcl 4.1;
import cookie;
backend default {
.host = "127.0.0.1";
.port = "8090";
}
sub vcl_recv {
unset req.http.Cookie;
if (req.http.Cookie) {
return (pass);
}
}
Nginx passes requests to the backend. Backend is at port 8080.
In short what listens where:
Varnish - 8000
Nginx - 8090
Backend - 8080
Start varnishd:
michael#michael:~$ sudo varnishd -f /home/michael/PycharmProjects/varnish/varnish.vcl -a localhost:8000
Open Chrome, load the page several times:
Nginx's access log:
127.0.0.1 - - [16/Apr/2020:08:12:49 +0300] "GET / HTTP/1.1" - 200 334 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36" "127.0.0.1"
127.0.0.1 - - [16/Apr/2020:08:13:21 +0300] "GET / HTTP/1.1" - 200 334 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36" "127.0.0.1"
We can see that cookie is absent. It is Ok. Cookies are cut off.
Problem: Requests are inevitably passed to the backend. I always stop at the breakpoint.
Will you be so kind as to give me a piece of advice on how to cope with the problem?

Varnish doesn't look at the contents of the Cookie request header, but rather checks if the header is present.
What you need to do is to check if the Cookie header is empty, and if so, just remove the entire thing.
Just add this after your regsub statement:
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
Make the process a bit more flexible
In reality you'll probably will not remove all cookies, but just the ones that are not essential to your application.
You can use this statement to remove all cookies, except that ones you need for your application:
sub vcl_recv {
if (req.http.Cookie) {
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie,
";(SESS[a-z0-9]+|NO_CACHE)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
else {
return (pass);
}
}
}
This snippet will remove all cookies, expect that ones that match the SESS[a-z0-9]+|NO_CACHE regex.
Use vmod_cookie
There is a cleaner way to approach cookie substitution in Varnish and it involves using the vmod_cookie module in Varnish. You can find it here: https://github.com/varnishcache/varnish-cache/tree/master/lib/libvmod_cookie
If you upgrade to Varnish version 6.4, the vmod_cookie will be part of the core Varnish installation.
This is the equivalent using vmod_cookie:
vcl 4.1;
import cookie;
sub vcl_recv {
cookie.parse(req.http.cookie);
cookie.keep_re("SESS[a-z0-9]+,NO_CACHE");
set req.http.cookie = cookie.get_string();
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
}

Related

Caching a specific URL path in Varnish fails

I've set up some VCL to only cache the /js/ folder.
sub vcl_backend_response {
if (bereq.url ~ "^/js/.$") {
unset beresp.http.set-cookie;
set beresp.http.cache-control = "max-age = 2592000";
set beresp.ttl = 1y;
}else {
set beresp.http.cache-control = "max-age = 0";
set beresp.ttl = 0s;
}
}
When I check the cache status of items in the js folder after reloading a few times, nothing has been cached and the cache control header shows 0.
# curl -I localhost:6081/js/themes.js
HTTP/1.1 200 OK
Content-Type: application/javascript
Etag: W/"1655132873"
Last-Modified: Mon, 13 Jun 2022 15:07:53 GMT
Accept-Ranges: bytes
X-Content-Type-Options: nosniff
Content-Length: 656
Date: Thu, 11 Aug 2022 19:13:50 GMT
cache-control: max-age = 0
Vary: Accept-Encoding
X-Varnish: 5
Age: 0
Via: 1.1 varnish (Varnish/6.5)
Connection: keep-alive
Any idea on how to fix this?

The vcl_backend_response subroutine is called prior to storing an object in the cache. The VCL logic you provided will at least ensure that the Javascript ends up in the cache.
However, I'm not too sure that your ^/js/.$" regular expression will match the right URLs. Maybe ^/js/.*$ would be a better match.
Built-in VCL for vcl_backend_response
There may also be other factors that prevent the object from being stored in te cache. If you take a look at the following tutorial about Varnish's built-in VCL, you'll see what logic is applied: https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/#11-vcl_backend_response
If the response contains a Cache-Control: no-cache or a Cache-Control: no-store or a Cache-Control: private header, Varnish will still not store the object in the cache.
To prevent this from happening, you can actually call return(deliver). This wil bypass any other built-in VCL logic.
Here's what your VCL code would look like:
sub vcl_backend_response {
if (bereq.url ~ "^/js/.*$") {
unset beresp.http.set-cookie;
set beresp.http.cache-control = "max-age = 2592000";
set beresp.ttl = 1y;
return(deliver);
}else {
set beresp.http.cache-control = "max-age = 0";
set beresp.ttl = 120s;
set beresp.uncacheable = true;
return(deliver);
}
}
However, built-in VCL logic mostly makes sense, so I'd be very cautious when bypassing it.
You probably noticed that I removed set beresp.ttl=0 from the VCL example and replaced it with set beresp.uncacheable=true. That's because you should never set the TTL to zero. For reasons that are not relevant to this question, they will make Varnish perform really poorly and will cause increased backend fetches to your backend.
Built-in VCL for vcl_recv
While we discussed how to force objects to be stored in the cache through vcl_backend_response logic, that doesn't mean the object will be served from the cache. That's where the built-in VCL for vcl_recv comes into play.
See https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/#1-vcl_recv for a tutorial about the vcl_recv built-in VCL.
If a request contains a Cookie or an Authorization header, Varnish will not serve the object from cache, despite it being static content.
To bypass this behavior, you could also add the following code:
sub vcl_recv {
if (req.url ~ "^/js/.*$") {
unset req.http.Cookie;
unset req.http.Authorization;
return(hash);
}
}
This will ensure all content from the /js/ folder will be served from the cache.
VCL template for caching static data
If it's your goal to cache static data regardless of the folder it is stored in, you can use the following VCL template:
sub vcl_recv {
if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset req.http.Cookie;
unset req.http.Authorization
return(hash);
}
}
sub vcl_backend_response {
if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset beresp.http.Set-Cookie;
set beresp.ttl = 1d;
}
}
It comes from the following tutorial: https://www.varnish-software.com/developers/tutorials/example-vcl-template/#13-caching-static-content.
You can still adjust it by increasing the TTL or by prefixing the specific folder in the regex.

Varnish 6 missing requests for same URL coming from different browsers

This is how my varnish.vcl looks like.
vcl 4.0;
import directors;
import std;
backend client {
.host = "service1";
.port = "80";
}
sub vcl_recv {
std.log("varnish log info:" + req.http.host);
# caching pages in client
set req.backend_hint = client;
# If request is from conent or for pages remove headers and cache
if ((req.url ~ "/content/") || (req.url ~ "/cms/api/") || req.url ~ "\.(png|gif|jpg|jpeg|json|ico)$" || (req.url ~ "/_nuxt/") ) {
unset req.http.Cookie;
std.log("Cachable request");
}
# If request is not from above do not cache and pass to Backend.
else
{
std.log("Non cachable request");
return (pass);
}
}
sub vcl_backend_response {
if ((bereq.url ~ "/content/") || (bereq.url ~ "/cms/api/") || bereq.url ~ "\.(png|gif|jpg|jpeg|json|ico)$" || (bereq.url ~ "/_nuxt/") )
{
unset beresp.http.set-cookie;
set beresp.http.cache-control = "public, max-age=259200";
set beresp.ttl = 12h;
return (deliver);
}
}
# Add some debug info headers when delivering the content:
# X-Cache: if content was served from Varnish or not
# X-Cache-Hits: Number of times the cached page was served
sub vcl_deliver {
# Was a HIT or a MISS?
if ( obj.hits > 0 )
{
set resp.http.X-Cache-Varnish = "HIT";
}
else
{
set resp.http.X-Cache-Varnish = "MISS";
}
# And add the number of hits in the header:
set resp.http.X-Cache-Hits = obj.hits;
}
If I am hitting a page from same browser netwrok tab showing
X-Cache-Varnish = "HIT";
X-Cache-Hits = ;
Lets say if I hot from chrome 10 times this is what I get
X-Cache-Varnish = "HIT";
X-Cache-Hits = 9;
9 because first was a miss and rest 9 were served from cache.
If I try incognito window or a different browser it gets its own count starting from 0. I think somehow I am still caching cookies. I could not identify what I am missing.
Ideally, I want to delete all cookies for specific paths. but somehow unset does not seem to be working for me.

If you really want to make sure these requests are cached, make sure you do a return(hash); in your if-statement.
If you don't return, the built-in VCL will take over, and continue executing its standard behavior.
Apart from that, it's unclear whether or not your backend sets a Vary header which might affect your hit rate.
Instead of guessing, I suggest we use the logs to figure out it.
Run the following command to track your requests:
varnishlog -g request -q "ReqUrl ~ '^/content/'"
This statement's VSL Query expression assumes the URL starts with /content. Please adjust accordingly.
Please send me an extract of varnishlog for 1 specific URL, but also for both situations:
The one that hits the cache on a regular browser tab
The one that results in a cache miss in incognito mode or from a different browser
The logs will give more context and explain what happened.

Can't PURGE entire domain in Varnish, but can PURGE individual pages, do I have a misconfiguration?

I'm assuming that I must have made a mistake in my Varnish configuration. (I'm running version 6.0.7.) Here is the relevant section:
####----SECTION THREE: PURGE RULES----####
# Access Control List to define which IPs can purge content
acl purge {
"localhost";
"127.0.0.1";
"<my home IP address>";
"::1";
}
####----SECTION FOUR: PROCESSING RULES----####
sub vcl_recv {
# Do not allow purging from non-approved IPs
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
return (purge);
}
# Allow banning regexes
if (req.method == "BAN") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send BAN requests."));
}
ban("req.http.host == " + req.http.host + " && req.url ~ ^" + req.url);
return (synth(200, "Ban added"));
}
# Allow purging regexes
if (req.method == "PURGE") {
if (req.http.X-Purge-Method == "regex") {
ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);
return (synth(200, "Banned"));
} else {
return (purge);
}
(N.B.: In my setup, Varnish is running on port 6081, with HAProxy in front of it.)
Using this setup, from either my home IP address or the server Varnish is running on, I am able to clear individual pages from the Varnish cache by running, e.g.:
curl -i -X PURGE https://example.com/page/
I can also clear the entire domain from the Varnish cache by running, e.g.:
curl -i -X BAN https://example.com
Likewise, I can clear individual pages from the Varnish cache by running, e.g.:
curl -i -X PURGE http://<IP of Varnish server>:6081/page/ -H "Host: example.com”
I can also clear the entire domain from the Varnish cache by running, e.g.:
curl -i -X BAN http://<IP of Varnish server>:6081 -H "Host: example.com”
But I have noticed that I cannot clear the entire domain by running either:
curl -i -X PURGE https://example.com
or:
curl -i -X PURGE http://<IP of Varnish server>:6081 -H "Host: example.com”
To clear the entire domain, I have to use BAN instead. I do not know if this is a problem or a misconfiguration, but it seems to be, because although I have read that PURGE is not used to clear everything, the Wordpress plugin I am using seems to be trying to, by sending a PURGE request to domain.com/.* when I hit the “Clear ALL Varnish Cache” button.
This, frankly, confuses me. Because it doesn’t work either in the plug-in itself or if I try it from the command line. What am I missing? Is the plugin just out of date? Did Varnish used to be able to clear the entire cache this way? I'd appreciate any pointers. Thanks!

Here's a cleaned up version of the purging logic in your VCL:
acl purge {
"localhost";
"127.0.0.1";
"::1";
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
if (req.http.X-Purge-Method == "regex") {
ban("obj.http.x-url ~ " + req.url + " && obj.http.x-host ~ " + req.http.host);
return (synth(200, "Banned"));
}
return (purge);
}
}
sub vcl_backend_response {
set beresp.http.x-url = bereq.url;
set beresp.http.x-host = bereq.http.host;
}
sub vcl_deliver {
unset resp.http.x-url;
unset resp.http.x-host;
}
Purging
The PURGE HTTP method is what triggers the cache invalidation. Without any headers return(purge); will be called, which will invalidate the exact URL that was used.
If the X-Purge-method request header is added, and is set to regex, a regular expression match will occur. Invalidating multiple objects using a regular expression cannot be done with return(purge);, and requires banning.
Bans
The ban() function adds a ban expression to the ban list. Expressions on this list are matched to all the objects that are stored in cache. The matching ones are removed.
You can consult the ban list using the following command on your Varnish server:
varnishadm ban.list
Lurker-friendly bans
The ban lurker, a special thread that processes the ban list, doesn't have access to the request context. If you want this thread to remove the objects from cache, based on request parameters like the URL or the hostname, you need to apply som trickery.
As you can see, the pattern I suggest, matches on obj.http.x-url & obj.http.x-host. These are set in vcl_backend_response via 2 custom response headers.
If you don't do this, the ban lurker cannot match the objects, and the responsibility will shift to the next request. Even without these so-called lurker-friendly bans, banning will still work, but it is not as efficient.
Testing it
The first example will invalidate http://example.com/page from the cache
curl -i -XPURGE -H "Host: example.com" "http://localhost:6081/page/"
The next example will invalidate all objects whose URL starts with http://example.com/page:
curl -i -XPURGE -H "Host: example.com" -H "X-Purge-Method: regex" "http://localhost:6081/page/"
The last example will invalidate all objects from cache for the example.com domain:
curl -i -XPURGE -H "Host: example.com" -H "X-Purge-Method: regex" "http://localhost:6081/.*"
Here's the ban list item for that last example:
root#varnish:/etc/varnish# varnishadm ban.list
Present bans:
1606393598.913178 0 - obj.http.x-url ~ /.* && obj.http.x-host ~ example.com
1606393484.785268 3 C
As you can see, it matches all URLs for the example.com host. When you perform an HTTP request after that, you'll see the Age: 0 response header. This indicates that this response has been stored in cache for zero seconds. This means the ban was successful.

My varnish config doesn't seem to be working properly

I'm very new on varnish and I've a business on my hands recently. It's a local magazine website http caching (Tech Stack is Javascript + PHP). I'm trying to use varnish 4 for caching the website. What they want me to do is; any new articles should be appeared on FE immediately, any deleted articles should be erased from the FE immediately, any changes on website's current appereance should be applied directly (changing articles' current locations, they can be dragged anywhere on the website based on articles' popularity change.) and finally any changes on existing articles should be applied to website immediately. As you see on the config below, in sub vcl_recv block I tried to use return(purge) for POST requests, because new articles and article changes is applied via POST request. But it doesn't work at all. When I try create a new dummy content or make changes on existing articles, it's not purging the cache and showing the fresh content even if POST request is successful. Also, on the BE side, I tried to use if (beresp.status == 404) for deleted articles, but it doesn't work too. When I delete the dummy article I created, it's not being deleted too, I'm still seein the stale content. How should I change my config to get all these things done? Thank you.
my varnish config is ;
import directors;
import std;
backend server1 {
.host = "<some ip>";
.port = "<some port>";
}
sub vcl_init {
new bar = directors.round_robin();
bar.add_backend(server1);
}
sub vcl_recv {
set req.backend_hint = bar.backend();
if (req.http.Cookie == "") {
unset req.http.Cookie;
}
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") {
unset req.http.cookie;
}
if (req.url ~ "\.*") {
unset req.http.cookie;
}
if (req.method == "POST") {
return(purge);
}
}
sub vcl_deliver {
# A bit of debugging info.
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
}
else {
set resp.http.X-Cache = "MISS";
}
}
sub vcl_backend_response {
set beresp.grace = 1h;
set beresp.ttl = 120s;
if (bereq.url ~ "\.*") {
unset beresp.http.Set-Cookie;
unset beresp.http.Cache-Control;
}
if (bereq.method == "POST") {
return(abandon);
}
if (beresp.status == 404) {
return(abandon);
}
return (deliver);
}

No need to use the director if you only have one backend. Varnish will automatically select the backend you declared if there's only 1 backend.
Purging content
The POST purge call you're doing is not ideal. Please have a look at the following page to learn more about content invalidation in Varnish: https://varnish-cache.org/docs/6.0/users-guide/purging.html#http-purging
The snippet on that page contains an ACL to protect your platform from unauthorized purges.
It's important to know that you'll need to create a hook into your CMS or your MVC controller, that does the purge call.Here's a simple example using curl in PHP:
$curl = curl_init("http://your.varnish.cache/url-to-purge");
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "PURGE");
curl_exec($curl);
As you can see, this is an HTTP request done in cURL that uses the custom PURGE HTTP request method. This call needs to be executed in your good right after the changes are stored in the database. This post-publishing hook will ensure that Varnish clears this specific object from cache.
VCL cleanup
The statement below doesn't look like a reliable way to remove cookies, because the expression will remove cookies from all pages dat contain a dot:
if (req.url ~ "\.*") {
unset req.http.cookie;
}
The same applies to the following statement coming from the vcl_backend_response hook:
if (bereq.url ~ "\.*") {
unset beresp.http.Set-Cookie;
unset beresp.http.Cache-Control;
}
I assume some pages do actually need cookies to properly function. An admin panel for example, or the CMS, or maybe even a header that indicates whether or not you're logged in.
The best way forward is to define a blacklist or whitelist of URL patterns that can or cannot be cached.
Here's an example:
if(req.url !~ "^/(admin|user)" {
unset req.http.Cookie;
}
The example above will only keep cookies for pages that start with /admin or /user. There are other ways as well.
Conclusion
I hope the purging part is clear. If not, please take a closer look at https://varnish-cache.org/docs/6.0/users-guide/purging.html#http-purging.
In regards to the VCL cleanup: purging can only work if the right things are stored in cache. Dealing with cookies can be tricky in Varnish.
Just try to define under what circumstances cookies should be kept for specific pages. Otherwise, you can just remove the cookies.
Hope that helps. Good luck.
Thijs

TOO_MANY_REDIRECTS error when iFraming Kibana dashboard using cookies

I'm attempting to embed password protected Kibana dashboards inside an iFrame to my Node powered Express application, based on information in this thread. Kibana is protected using X-Pack and requires users to login in order to see their visualised data.
This currently requires the user to log in twice, once to login into the application and again to access Kibana dashboards, which is not the goal.
Following information in this thread, I implemented some code that makes a pre-flight POST request to https://elk-stack.my.domain:5601/api/security/v1/login to obtain a cookie 🍪
This client side request...
function preFlightKibanaAuth () {
...
$.ajax({
type: 'POST',
url: '/kibana-auth',
datatype: 'json',
success: function (response) {
if (response && response.authenticated) {
$('iframe#kibana-dashboard').prop('src', 'https://elk-stack.my.domain:5601/s/spacename/app/kibana#/dashboards?_g=()')
}
},
error: function (err) {
console.log(err)
}
})
}
Is routed to this route...
router
.route('/kibana-auth')
.post((req, res, next) => {
...
if (authorised) {
...
authenticateKibana(req)
.then(cookie => {
if (cookie && cookie.name && cookie.value) {
res.set('Set-Cookie', `${cookie.name}=${cookie.value}; Domain=my.domain; Path=/; Secure; HttpOnly`)
res.send({ 'authenticated': true })
} else {
res.send({ 'authenticated': false })
}
})
.catch((err) => {
logger.error(err)
res.send({ 'authenticated': false })
})
}
...
})
Which makes it's way to this function, where the cookie is actually obtained and parsed...
authenticateKibana () {
return new Promise((resolve, reject) => {
...
request({
method: 'POST',
uri: `https://elk-stack.my.domain:5601/api/security/v1/login`,
headers: {
'kibana-version': '6.5.4',
'kibana-xsrf': 'xsrftoken',
},
type: 'JSON',
form: {
password: 'validPassword',
username: 'validUsername'
}
}, function (error, res, body) {
if (!error) {
let cookies = cookieParser.parse(res)
cookies.forEach(function (cookie) {
if (cookie.name.startsWith('kibana')) {
// Got Kibana Cookie
return resolve(cookie)
}
})
}
...
})
})
}
This works great and I can successfully authenticate with Kibana, obtain the cookie and set in the clients browser (see below screenshot).
The issue I'm seeing is when the src of the iFrame is updated in the success callback of the preFlightKibanaAuth() request. I can see the authenticated Kibana dashboard load in briefly (so the cookie is allowing the client to view their authenticated dashboards), however, I then see multiple GET requests to /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana that results in a TOO_MANY_REDIRECTS error.
I've found the below comment in the GitHub issues page, which I think maybe the issue I'm having in some way because I'm seeing this in the logs (see bottom): "message":"Found 2 auth sessions when we were only expecting 1.". I just can't figure it out!
Usually what causes this is having multiple cookies for the same
"domain" and "name", but with different values for "path". If you open
the developer tools in Chrome, then click on the Application tab, then
expand the Cookies section, and click on the domain, do you have
multiple cookies with the name "sid"? If so, you can fix this issue by
clearing all of them.
I changed the cookie name from "sid" to "kibana" but don't have two of them visible in Applications tab, just the one I set following the call to /kibana-auth.
The iFrame then loads in the https://elk-stack.my.domain:5601/s/spacename/app/kibana#/dashboards?_g=() and the issue arises. Clearing my cookies just resulted in fetching and setting another one (if we don't already have one), which is what is required, so this didn't solve the problem.
When I send the Set-Cookie header back to the client, I am setting the Domain to the main domain: my.domain, which ends up as .my.domain. The Kibana instance is on a subdomain: elk-stack.my.domain and if I login to the Kibana front end, I can see that the Domain of the cookie it returns is set to elk-stack.my.domain. I'm not sure that should matter though.
Can anyone please shed any light on this or point me in the direction?
Thanks in advance
Here's a glimpse at the logging info from /var/log/kibana/kibana.stdout when a request is made. There's a bit of junk in there still but you can still see what's happening.
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /api/security/v1/login."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Request has been authenticated via header."}
{"type":"response","#timestamp":"2019-02-12T19:47:44Z","tags":[],"pid":7857,"method":"post","statusCode":204,"req":{"url":"/api/security/v1/login","method":"post","headers":{"kibana-version":"6.5.4","kbn-xsrf":"6.5.4","host":"10.30.10.30:5601","content-type":"application/
x-www-form-urlencoded","content-length":"35","connection":"close"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102"},"res":{"statusCode":204,"responseTime":109,"contentLength":9},"message":"POST /api/security/v1/login 204 109ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:44Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":3,"contentLength":9},"message":"GET /app/kibana 302 3ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"\"getConnections\" has been called."}
{"type":"ops","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"os":{"load":[0.2568359375,0.31640625,0.3173828125],"mem":{"total":33567580160,"free":346796032},"uptime":1585351},"proc":{"uptime":33636.577,"mem":{"rss":322772992,"heapTotal":225566720,"heapUsed":184707176,"external":2052484},"delay":6.417333126068115},"load":{"requests":{"5601":{"total":2,"disconnects":0,"statusCodes":{"204":1,"302":1}}},"concurrents":{"5601":1},"responseTimes":{"5601":{"avg":56,"max":109}},"sockets":{"http":{"total":0},"https":{"total":0}}},"message":"memory: 176.2MB uptime: 9:20:37 load: [0.26 0.32 0.32] delay: 6.417"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","monitoring-ui","kibana-monitoring"],"pid":7857,"message":"Received Kibana Ops event data"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","monitoring-ui","kibana-monitoring"],"pid":7857,"message":"Received Kibana Ops event data"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
The below then repeats over and over...
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /app/kibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["plugin","debug"],"pid":7857,"message":"Checking Elasticsearch version"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /app/kibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["plugin","debug"],"pid":7857,"message":"Checking Elasticsearch version"}
Kibana Version: 6.5.4
Elasticsearch: 6.5.4

At first, I thought this all turned out to be a mismatch in the Cookie attributes, alas, it wasn't!
Received some info from the Elastic team...
The cookie which Kibana replies with generally sets the httpOnly flag,
and the secure flag (when hosted over https), in addition to the
domain. If any of the settings differ for the cookie which you're
trying to force Kibana to use, you'll see 2 cookies being submitted
and behaviour similar to what you're seeing.
Thought I was setting the cookie with different attributes, but wasn't... ended up using a plugin to get this off the ground: https://readonlyrest.com/

Develop Reference

node.js excel linux python-3.x azure haskell apache-spark rust .htaccess string