Is it possible to let clients with certain IP's pass thru to the backend and not cache using varnish? I don't see this in any of the example configs.
i think better way is described here https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-October/021278.html
if you have list of IP's you should create an acl list:
acl passem {
"192.168.55.0/24";
}
and then in vcl.recv you should
if (client.ip ~ passem) {
return(pass);
}
I received this answer from the mailing list.
Yes, you can:
if (client.ip == IP)
{
return(pass);
}
Related
Is there any way to cache request with auth headers in varnish?
I want to ignore the auth headers while caching the request
There are various ways to approach this, depending on the importance of auth headers.
1. You don't care about auth
If you don't care about the auth part and if you want to risk serving cached content to unauthorized users, you can just use the following VCL code:
sub vcl_recv {
unset req.http.Authorization;
}
2. Ignore authorization to some extent
It is also possible to care about auth a bit, but not too much.
The following VCL snippet will allow caching even if there is an Authorization header:
sub vcl_recv {
if(req.http.Authorization) {
return(hash);
}
}
The consequence of this is that the initial cache miss will pass through to the backend and will be processed there. Potential unauthorized access will be handled there.
But as soon as the has been dealt with, the object is stored in the cache and the next requests will get cached content regardless of the authorization status of that request.
3. Perform auth on the edge
It is also possible to handle the auth part in Varnish while caching the content.
The following VCL code will handle this:
sub vcl_recv {
if(req.http.Authorization != "Basic YWRtaW46c2VjcmV0") {
return (synth(401, "Restricted"));
}
unset req.http.Authorization;
}
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
}
}
This code will actively inspect the content of the Authorization header and will ensure the username admin is used with password secret.
The YWRtaW46c2VjcmV0 string is nothing more than a base64 encoding of admin:secret.
4. Use vmod_basicauth
A more advanced and flexible way to terminate auth on the edge is by using https://git.gnu.org.ua/vmod-basicauth.git/. This VMOD can be compiled from source and can be downloaded from ftp://download.gnu.org.ua/release/vmod-basicauth.
Assuming the credentials are stored in /var/www/.htpasswd, you can leverage this VMOD to match the Authorization header to the content of the .htpasswd file.
Here's the VCL:
vcl 4.1;
import basicauth;
sub vcl_recv {
if (!basicauth.match("/var/www/.htpasswd",req.http.Authorization)) {
return (synth(401, "Restricted"));
}
unset req.http.Authorization;
}
sub vcl_synth {
if (resp.status == 401) {
set resp.http.WWW-Authenticate = {"Basic realm="Restricted area""};
}
}
This is entirely possible but also extremely dangerous: Varnish would return the same cached (authorized) content to all requests.
Example:
User A requests resource Z with proper authentication. Varnish relays the request to backend, caches the response and returns the resource.
User B requests resource Z with proper authentication. They will get the cached resource Z even if Z contains user A's content.
User X requests resource Z with invalid authentication. They will too get the cached resource anyway since the backend is bypassed.
Having said that, you can override Varnish's built-in VCL. Details are documented but the main idea is:
Copy default vcl_recv VCL (for your version) from source and add it to the end of your vcl_recv.
Remove the safeguards from vcl_recv: Just remove vcl_req_authorization which disables caching:
sub vcl_req_authorization {
if (req.http.Authorization) {
# Not cacheable by default.
return (pass);
}
}
In your vcl file issue a return statement at the end so built-in vcl is not used.
Logstash has an open port where everyone can send in data.
Anonymous data messes everything up
All data from all customers in one pool is messed up too.
So I read and tried https://www.elastic.co/guide/en/x-pack/5.6/logstash.html (also for Version 6 and 7) but this does not seem to be it. It rather authenticates against elasticsearch than in front of logstash. What I like to have is some sort of this:
input {
# One port to rule them all - possible?
tcp {
port => 5000
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
user => elastic
password => nope
document_id => "%{[#metadata][fingerprint]}"
# Here comes the user prefix again.
index => "%{[user]}-%{[host]}-%{+YYYY.MM.dd}"
}
if [user] == "foo" {
# Also put things in IRC
}
if [user] == "qux" {
# Forward somewhere else
}
}
In the end we like to have the data separated available when using Elastic or Kibana which may be no big deal when I read the documentation. But I also think that there should be some auth in front of logstash. Correct me if I am wrong.
If not:
How to prevent anonymous data?
How to distinguish them well?
Since you can't change the original message, you will need to filter based on the host that sent that message, you can use the host field for that, as it seems you are already doing on your output.
To prevent anonymous data you can use SSL, so each machine sending to your logstash will need to have the certificate files, or you can configure a firewall on the logstash machine and configure it to allow connections from your costumers only.
The x-pack security feature that you mentioned is a paid feature, but its focus is on putting a security layer on elasticsearch/kibana, there is no auth in front of logstash, to do that you need firewall rules and/or ssl certificates.
Its seems that purging doesn't work when using Cloudflare(Cloudflare returns 403 Forbidden).
This what i got when i searched for a solution online:
"The problem is that when you are using cloudflare, varinsh does not get the original IP of the sender. Instead it gets the IP of the cloudflare. So purging can not be done. We need to tell the varnish the original IP of the sender."
Add these following lines inside vcl_recv
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.method == "PURGE" || req.url == "/purge") {
# Replace these IP with your IP
if ( req.http.X-Forwarded-For !~ "(209.152.41.21|105.45.120.37)") {
return(synth(405, "This IP is not allowed to send PURGE requests."));
}
ban("req.url ~ /");
return (purge);
}
I tried this solution but it didn't work.
This question is old but could still use an answer. :-) The quote you found is partially correct. Here's why:
Your setup is likely similar to the below:
------ --------- ------------ ------
| WP | <- | Varnish | <- | CloudFlare | <--- | User |
------ --------- ------------ ------
There are two ways that the purge can happen:
User -> CloudFlare -> Varnish, or
User -> CloudFlare -> Varnish -> WP -> WP plugin -> Varnish.
The second situation can successfully cause a purge. If you have a plugin which triggers a cache purge/invalidate, it will come from a predictable IP address. In fact, if you run varnish on the same server as WP, the IP address will be [127.0.0.1]. There's a nice implementation for this situation.
The problem with the first situation (purging directly from CloudFlare) is that CloudFlare has many IP addresses which you would have to keep up to date. But more importantly, there's nothing that would prevent a bad actor from also creating a service using CloudFlare which would also be allowed to send a purge request to your server, basically rendering this security worthless.
Since X-Forwarded-For is basically just a list of all of the previous IP addresses along the way, this would also be easy to fake and bypass.
Since filtering on IP address is not useful directly through CloudFlare, you could alternatively use a token/secret that you send as part of the purge request. (i.e. only allow a special URL like:
if (req.url == "/purge-719179c7-6226-4b87-9503-1b6d54d5fea5") {... with some other Guid of course). One could argue that this is still not secure, but perhaps better than allowing all CloudFlare IPs.
TL;DR: How to detect local website users?
I have a self-hosted website running in the student-building I live in. In this website I would like a page for and links to certain local applications, like the webremote of the RPi running Kodi, an FTP, a page of instructions etc.
I don't want those to be visible to random internet users, so is there any way for a website to detect whether the user is accessing the website from inside the local network? Preferably in JavaScript, but PHP would also be fine.
in PHP it can be done in several ways. The simplest way is to do a simple IP check.
if ($_SERVER['REMOTE_ADDR'] = "10.1.0.25") { // Internet IP or IP rangehere
// Show links for internal users
} else {
// Show stuff for all other users
}
or for range of IPs assuming 192.168.1.x addresses
if ($_SERVER['REMOTE_ADDR'] >= "192.168.1.1" && $_SERVER['REMOTE_ADDR'] <= "192.168.1.1") {
// Internal Info
} else {
// External Info
}
How would I use client.ip as a conditional in setting headers in the fetch section of a Varnish 3.0 VCL? I have some troubleshooting headers that I like to set to solve caching issues, however, I don't want them publicly visible. I'd love to be able to whitelist the headers for my ip address only.
Is there any way to access client.ip in _fetch?
You can best set all troubleshooting headers in you _recv without any conditions and remove them in you vcl deliver. this way you dont need to add the same ip check on every conditional header
if you want to use an ip range you can use the following code
acl debug {
"your ip adress1";
"you ip adress 2";
}
in you vcl_recv
if (!client.ip ~ debug) {
set req.http.x-debug = "debug";
}
in you vcl_deliver
if(!req.htt.x-debug){
remove resp.http.debugheader1;
remove resp.http.debugheader2;
}