Varnish returns 404 for valid URLs - varnish

Question

I have a Varnish setup to cache some specific URLs (mostly XML outputs) for a period of time like 5-10 seconds. DC guys set it up and created the config file but unfortunately it just doesn't work.
I'd be glad if you can take a look to the configuration and help me to spot the problem.
Here is my default.vcl
backend default {
.host = "127.0.0.1";
.port = "80";
}
sub vcl_recv {
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
if(req.http.host == "foo.example1.com"){
if(req.url ~ "^/somedir\?somevar=anothervar&andanother=.*"){
return (lookup);
}
}else if(req.http.host == "bar.example2.com"){
if(req.url ~ "^/foo/bar\?somevar=anothervar&andanother=.*"){
return (lookup);
}
} else {
return (pass);
}
# Currently not reached.
return (lookup);
}
sub vcl_pipe {
# Note that only the first request to the backend will have
# X-Forwarded-For set. If you use X-Forwarded-For and want to
# have it set for all requests, make sure to have:
# set bereq.http.connection = "close";
# here. It is not set by default as it might break some broken web
# applications, like IIS with NTLM authentication.
return (pipe);
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return (hash);
}
sub vcl_miss {
return (fetch);
}
sub vcl_fetch {
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Vary == "*") {
/*
* Mark as "Hit-For-Pass" for the next 2 minutes
*/
set beresp.ttl = 120 s;
return (hit_for_pass);
}
set beresp.ttl = 5 s;
return (deliver);
}
And the log output when I tried to call the URL from localhost with
curl --verbose --header 'Host: foo.example1.com' "http://127.0.0.1:6081/somedir?somevar=anothervar&andanother=123456" is;
14 BackendClose b default
14 BackendOpen b default 127.0.0.1 60502 127.0.0.1 80
14 TxRequest b GET
14 TxURL b /somedir?somevar=anothervar&andanother=123456
14 TxProtocol b HTTP/1.1
14 TxHeader b User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
14 TxHeader b Accept: */*
14 TxHeader b Host: foo.example1.com
14 TxHeader b X-Forwarded-For: 127.0.0.1
14 TxHeader b X-Varnish: 1405453128
14 TxHeader b Accept-Encoding: gzip
14 RxProtocol b HTTP/1.1
14 RxStatus b 404
14 RxResponse b Not Found
14 RxHeader b Date: Thu, 07 Nov 2013 17:07:50 GMT
14 RxHeader b Server: Apache/2.2.15 (Red Hat)
14 RxHeader b Content-Length: 295
14 RxHeader b Content-Type: text/html; charset=iso-8859-1
14 Fetch_Body b 4(length) cls 0 mklen 1
14 Length b 295
14 BackendReuse b default
12 SessionOpen c 127.0.0.1 48771 :6081
12 ReqStart c 127.0.0.1 48771 1405453128
12 RxRequest c GET
12 RxURL c /somedir?somevar=anothervar&andanother=123456
12 RxProtocol c HTTP/1.1
12 RxHeader c User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
12 RxHeader c Accept: */*
12 RxHeader c Host: foo.example1.com
12 VCL_call c recv lookup
12 VCL_call c hash
12 Hash c /somedir?somevar=anothervar&andanother=123456
12 Hash c foo.example1.com
12 VCL_return c hash
12 VCL_call c miss fetch
12 Backend c 14 default default
12 TTL c 1405453128 RFC 120 -1 -1 1383844070 0 1383844070 0 0
12 VCL_call c fetch
12 TTL c 1405453128 VCL 5 -1 -1 1383844070 -0
12 VCL_return c deliver
12 ObjProtocol c HTTP/1.1
12 ObjResponse c Not Found
12 ObjHeader c Date: Thu, 07 Nov 2013 17:07:50 GMT
12 ObjHeader c Server: Apache/2.2.15 (Red Hat)
12 ObjHeader c Content-Type: text/html; charset=iso-8859-1
12 VCL_call c deliver deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 404
12 TxResponse c Not Found
12 TxHeader c Server: Apache/2.2.15 (Red Hat)
12 TxHeader c Content-Type: text/html; charset=iso-8859-1
12 TxHeader c Content-Length: 295
12 TxHeader c Accept-Ranges: bytes
12 TxHeader c Date: Thu, 07 Nov 2013 17:07:50 GMT
12 TxHeader c X-Varnish: 1405453128
12 TxHeader c Age: 0
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: keep-alive
12 Length c 295
12 ReqEnd c 1405453128 1383844070.464804649 1383844070.465569973 0.000082016 0.000713110 0.000052214
12 SessionClose c EOF
12 StatSess c 127.0.0.1 48771 0 1 1 0 0 1 257 295

Answer 1

This varnishlog is very clear.
Your backend is responding with a 404 for the URL you requested. You will find the same return code in your backend access logs.

Answer 2

Also same problem here, but turning off varnish and going back to apache to listen to port 80, all pages work like a charm.once you get varnish to respond to port 80 all goes down.So i think it is not the backend problem...