In vcl_recv I'm trying to send a 403 to requests that contain the following characters in the url: ",',<,>,(, and )
if(req.url ~ "[\'\<\>()].*\.html" ) {
return (synth(403, "Forbidden"));
}
everything works except the double quote "
I tried regex like:
"[\"\'\<\>()].*\.html"
"[\\"\'\<\>()].*\.html"
"[%22\'\<\>()].*\.html"
"[x22\'\<\>()].*\.html"
All of them do not compile with "varnishd -C -f default.vcl"
I'm currently on varnish-4.1.1
Does anyone know how to escape the " correctly?
How about:
if (req.url ~ "[\x27<>()\x22]") {
return (synth(403, "Forbidden"));
}
Regex test
%22 will give you a double quote within your quoted string in Varnish VCL
Related
I want to invalidate a request in Varnish from a java backend with HTTP headers.
till now I am able to achieve cache arequest which does not have query param in it.
Let's say I have a request: localhost:8090/api/data/abc?fields=test,test1
what headers do I need to set in this case for varnish to cache it.
I am able to ban a request which is like : localhost:8090/api/data/abc
by using this headers for this request:
request: localhost:8090/api/data/abc
headers:
responseHeaders.set("x-host", "localhost:8080");
responseHeaders.set("x-url", "/api/data/abc");
VCL code
You can use the VCL example from the banning tutorial on the Varnish Developer Portal, which looks like this:
vcl 4.1;
acl purge {
"localhost";
"192.168.55.0"/24;
}
sub vcl_recv {
if (req.method == "BAN") {
if (!client.ip ~ purge) {
return (synth(405));
}
if (!req.http.x-invalidate-pattern) {
return (purge);
}
ban("obj.http.x-url ~ " + req.http.x-invalidate-pattern
+ " && obj.http.x-host == " + req.http.host);
return (synth(200,"Ban added"));
}
}
sub vcl_backend_response {
set beresp.http.x-url = bereq.url;
set beresp.http.x-host = bereq.http.host;
}
sub vcl_deliver {
unset resp.http.x-url;
unset resp.http.x-host;
}
Keep in mind that you need to add your backend definition and customize the values of the ACL.
Removing objects that match a specific query string pattern
Let's say the cache contains 4 objects identified by the following URL:
/?a=1
/?a=2
/?a=3
/?a=4
Imagine we want to remove the first 3. Assuming the value of the a query string parameter is a number, we can create the following HTTP request:
curl -XBAN -H"x-invalidate-pattern: ^/\?a=[1-3]+" http://localhost
As a result /?a=1, /?a=2 and /?a=3 will be removed from the cache, whereas /?a=4 is still stored in the cache.
Conclusion
The query string parameter is part of the URL. As long as you can match it in a regular expression, you can remove specific objects from the cache.
I am new to VCL rules.
I want to exclude special pages from being cached by varnish cache.
What I exactly want to do is exclude all urls from being cached that include a specific query string "query=(number between 1 and 100)"
This code works only for one specific query.
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=100") {
return(pass);
}
}
I just want to be sure this rule should work for the whole range from 1-100, right ?
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=[0-9]") {
return(pass);
}
}
or do i have to do it like this ?
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=1||query=2||...||query=99||query=100") {
return(pass);
}
}
I don't know if Varnish supports curly braces, if it does, you should do something like:
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=([0-9]{1,2}|100)") {
return(pass);
}
}
By the way, this regular expression matches "query=990". I don't know how your url is composed but you should add something to avoid that (if you really need to).
For example, if there are other params:
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=([0-9]{1,2}|100)&") {
return(pass);
}
}
Or in case it's the last param in the url:
sub vcl_recv {
# don't cache these special pages
if (req.url ~ "query=([0-9]{1,2}|100)$") {
return(pass);
}
}
I have in Apache something like this
RewriteCond %{REMOTE_HOST} !^11\.22\.33\.[12]\d\d$ #As example allows from 100 till 200
RewriteCond %{REMOTE_HOST} !^127\.0\.0\.1$
RewriteCond %{TIME} <20140113090000
RewriteRule ^/access-on-monday/ http://www.mysite.com/ [NC,L,R=302]
which works perfect.
I need same for Varnish.
Because server behind LoadBalancer it gets X-Forwarded-For header with real client IP. I still check %{REMOTE_HOST} in Apache because rpaf_module is installed.
I added next in Varnish with helping ipcast vmod
import ipcast;
acl office {
"localhost";
"11.22.33.100"/24; //Let's think that it matches with 11.22.33.100 - 11.22.33.200
}
sub vcl_recv {
if (req.http.X-Forwarded-For !~ ",") {
set req.http.xff = req.http.X-Forwarded-For;
} else {
set req.http.xff = regsub(req.http.X-Forwarded-For, "^[^,]+.?.?(.*)$", "\1");
}
if (ipcast.clientip(req.http.xff) != 0) {
error 400 "Bad request";
}
if (!client.ip ~ office) {
set req.http.X-Redir-Url = "http://" + req.http.Host + "/";
error 751 "Found";
}
}
Then in vcl_error I make redirect but it doesn't matter here.
My question is it possible to make timebased access like in Apache?
You are limited with simple VCL but Varnish allow to do much more than simple statements.
You can enhance and drive your workhorse with inline-C or VMODs, and do this job in C.
For example, if you want to do add timebased access from :
backend server_available_in_2014 {
.host="127.0.0.1";
.port="8080";
}
sub vcl_recv {
set req.backend = server_available_in_2014; # IT MUST BE AVAILABLE ONLY in 2014
}
You can convert your date 201401010000 into an UNIX timestamp 1389617122, and write simple inline-C :
backend server_available_in_2014 {
.host="127.0.0.1";
.port="8080";
}
C{
double TIM_real(void);
}C
sub vcl_recv {
C{
if (TIM_real() > 1389617122.0) {
VRT_l_req_backend(sp, VGCDIR(_server_available_in_2014));
}
}C
}
TIM_real() is returning a current timestamp (look at varnish/lib/libvarnish/time.c) and VRT_l_req_backend statement is exactly the same as set req.backend = server_available_in_2014;, but written in C instead of VCL.
If you want more tweaks, you can compile your VCL into C by executing the following command: varnishd -f default.vcl -C
So I'm trying to read a JSON file at a seperate url and based off what what is in the JSON file I would like for the program to do something. But the way I'm currently trying to do it is not working.
<?php
$homepage = file_get_contents('http://direct.cyberkitsune.net/canibuycubeworld/status.json');
//echo $homepage;
if($homepage contains 'f'){
echo 'true';
}else{
echo 'Something went terribly wrong!";
}
?>
This is the error that I get
Parse error: syntax error, unexpected T_STRING in /home/a1285224/public_html/cubeworld.php on line 4
When I searched up the "file_get_contents" under the php manual it says that it should read the entire file into a string, so I'm a little confused why I'm getting a string error.
Thanks~
In stead of 'contains', see the strpos function. Also, you must close single quotes with single quotes, and double quotes with double quotes.
i.e.
$homepage = file_get_contents("http://direct.cyberkitsune.net/canibuycubeworld/status.json");
// echo $homepage;
if (strpos($homepage, "f") !== false)
{
echo "true";
}
else
{
echo "Somethingwentterriblywrong!";
}
Based on what you are trying to do, also see PHP's JSON library.
To get the JSON data, you can use json_decode.
To dump all vars:
$json = json_decode($homepage, true);
var_dump($json);
To get all the other data:
printf("Time: %s<br>Site: %s<br>Reg: %s<br>Shop: %s", $json['time'], $json['site'], $json['reg'], $json['shop']);
I'm using Varnish 2.0.6 and I'm having trouble with finding good documentation to write the vcl_hash function.
I need to remove a few parameters from the URL of my API before caching. In particular a userid that is passed to track analytics but not to change the results.
URL: /api/browse?node=123&userid=3432432564363
I wrote this but it's not cleat to me if the vcl_hash function needs to end with 'hash' or 'return(hash)' or NOTHING and if I need to handle all the cases or just my special case. It's not clear to me if I'm overwriting method or I'm extending it.
I have:
sub vcl_hash {
if (req.url ~ "^/api/browse") {
set req.hash += regsuball(req.url,"&userid=([A-z0-9]+)","");
}
hash;
}
Is it missing something?
I tested a few things, and this one seems to work:
sub vcl_hash {
if (req.url ~ "^/api/browse") {
set req.hash += regsuball(req.url,"&userid=([A-z0-9]+)","");
} else {
set req.hash += req.url;
}
set req.hash += req.http.host;
hash;
}
So it looks like you also have to handle the default case when you rewrite vcl_hash.
The following is a general solution that works for me (starting from varnish v4), to remove several unwanted parameters.
The list of parameters can be extended easily, as long as the value-regex matches: The value regex matches all URL-safe characters, so it should match for all URL-encoded parameters.
sub vcl_hash {
# conditional replacement is faster then evaluating regexes all the time
if (req.method == "GET" || req.method == "HEAD") {
hash_data(regsuball(req.url, "(userid|sid|uid)=[%.-_~A-z0-9]+&?", ""));
}
else {
hash_data(req.url);
}
hash_data(req.http.host);
return (lookup);
}