Error 503 backend fetch failed while i am trying to download - varnish

Question

I have a haproxy-varnish setup for my ASPNET Core Web application and my application sending request via haproxy.conf file to the varnish. I can download all downloadable files except one url /Payroll/Export/EmployeeAllDataExport?.... File format is .xlsx. Tranfer-encoding header added manually by someone else so maybe that change refers the problem but i couldn't resolve until now.
I am sharing varnishlog -g request -q "RespStatus == 503" output, default.vcl file and releated lines in haproxy.cfg
P.S: If i disable varnish request from haproxy i can download the file but i need to do it with varnish and there is a problem for only one url not in an entire application.
varnishlog output:
* << Request >> 1384717
- Begin req 1384716 rxreq
- Timestamp Start: 1658919403.185251 0.000000 0.000000
- Timestamp Req: 1658919403.185251 0.000000 0.000000
- VCL_use boot
- ReqStart $IP1 43974 a0
- ReqMethod GET
- ReqURL /Payroll/Report/EmployeeAllDataExport?1658919403106
- ReqProtocol HTTP/1.1
- ReqHeader Host: $URL1
- ReqHeader sec-ch-ua: ".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"
- ReqHeader sec-ch-ua-mobile: ?0
- ReqHeader sec-ch-ua-platform: "Windows"
- ReqHeader Upgrade-Insecure-Requests: 1
- ReqHeader User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
- ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
- ReqHeader Sec-Fetch-Site: same-origin
- ReqHeader Sec-Fetch-Mode: navigate
- ReqHeader Sec-Fetch-User: ?1
- ReqHeader Sec-Fetch-Dest: document
- ReqHeader Referer: https://$URL1/Payroll/Report/EmployeeAllData?sitemap=63f8e1f5-a873-4cb5-8a5c-16b79863da33
- ReqHeader Accept-Encoding: gzip, deflate, br
- ReqHeader Accept-Language: tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-Port: 443
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-For: $IP2
- ReqHeader Connection: close
- ReqUnset X-Forwarded-For: $IP2
- ReqHeader X-Forwarded-For: $IP2, $IP1
- VCL_call RECV
- ReqUnset Host: $URL1
- ReqHeader Host: $URL1
- ReqURL /Payroll/Report/EmployeeAllDataExport?1658919403106
- ReqUnset Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqUnset Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqUnset Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqUnset Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqUnset Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
- ReqHeader Surrogate-Capability: key=ESI/1.0
- VCL_return hash
- ReqUnset Accept-Encoding: gzip, deflate, br
- ReqHeader Accept-Encoding: gzip
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 1384718 fetch
- Timestamp Fetch: 1658919404.932211 1.746959 1.746959
- RespProtocol HTTP/1.1
- RespStatus 503
- RespReason Backend fetch failed
- RespHeader Date: Wed, 27 Jul 2022 10:56:44 GMT
- RespHeader Server: Varnish
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- RespHeader X-Varnish: 1384717
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.2)
- VCL_call DELIVER
- RespHeader X-Cache: MISS
- RespHeader X-Cache-Hits: 0
- RespUnset Server: Varnish
- RespUnset X-Varnish: 1384717
- RespUnset Via: 1.1 varnish (Varnish/6.2)
- VCL_return deliver
- Timestamp Process: 1658919404.932258 1.747006 0.000047
- Filters
- RespHeader Content-Length: 284
- RespHeader Connection: close
- Timestamp Resp: 1658919404.932344 1.747092 0.000086
- ReqAcct 5695 0 5695 210 284 494
- End
** << BeReq >> 1384718
-- Begin bereq 1384717 fetch
-- VCL_use boot
-- Timestamp Start: 1658919403.185407 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Payroll/Report/EmployeeAllDataExport?1658919403106
-- BereqProtocol HTTP/1.1
-- BereqHeader sec-ch-ua: ".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"
-- BereqHeader sec-ch-ua-mobile: ?0
-- BereqHeader sec-ch-ua-platform: "Windows"
-- BereqHeader Upgrade-Insecure-Requests: 1
-- BereqHeader User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
-- BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
-- BereqHeader Sec-Fetch-Site: same-origin
-- BereqHeader Sec-Fetch-Mode: navigate
-- BereqHeader Sec-Fetch-User: ?1
-- BereqHeader Sec-Fetch-Dest: document
-- BereqHeader Referer: https://$URL1/Payroll/Report/EmployeeAllData?sitemap=63f8e1f5-a873-4cb5-8a5c-16b79863da33
-- BereqHeader Accept-Language: tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7
-- BereqHeader X-Forwarded-Proto: https
-- BereqHeader X-Forwarded-Port: 443
-- BereqHeader X-Forwarded-Proto: https
-- BereqHeader X-Forwarded-For: $IP1, $IP2
-- BereqHeader Host: $URL1
-- BereqHeader Cookie: CurrentLanguage=tr; .AspNetCore.Antiforgery.9TtSrW0hzOs=CfDJ8E7lWAHTLKBOsOUJGWpm85n139G92-PEfuGVynC-Lh4dd-5XOYoILAhHGAD6DW-my0vgWvin4bBKCLT-GFQonAQCvkTZV8EuOB8s6RPOunsRFPpgucKPwKQSmF7Syx0tg8TubZs_Xw6qTxh9gVq03lo; Hcm=CfDJ8E7lWAHTLKBOsOUJGWpm85lmo
-- BereqHeader Surrogate-Capability: key=ESI/1.0
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 1384718
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 31 haproxy $IP1 8585 $IP3 52722
-- BackendStart $IP1 8585
-- Timestamp Bereq: 1658919403.185494 0.000087 0.000087
-- Timestamp Beresp: 1658919404.931989 1.746582 1.746495
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Connection: close
-- BerespHeader Date: Wed, 27 Jul 2022 10:56:44 GMT
-- BerespHeader Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
-- BerespHeader Server: Kestrel
-- BerespHeader Transfer-Encoding: identity
-- BerespHeader Strict-Transport-Security: max-age=2592000
-- BerespHeader content-disposition: attachment; filename="$Filename.xlsx"
-- BackendClose 31 haproxy
-- Error Body cannot be fetched
-- Timestamp Error: 1658919404.932043 1.746636 0.000054
-- BerespProtocol HTTP/1.1
-- BerespStatus 503
-- BerespReason Service Unavailable
-- BerespReason Backend fetch failed
-- BerespHeader Date: Wed, 27 Jul 2022 10:56:44 GMT
-- BerespHeader Server: Varnish
-- VCL_call BACKEND_ERROR
-- BerespHeader Content-Type: text/html; charset=utf-8
-- BerespHeader Retry-After: 5
-- VCL_return deliver
-- Storage malloc Transient
-- Length 284
-- BereqAcct 5731 0 5731 327 0 327
-- End
default.vcl
vcl 4.1;
# Based on: https://github.com/mattiasgeniar/varnish-6.0-configuration-templates/blob/master/default.vcl
import std;
import directors;
backend haproxy { # Define one backend
.host = "10.10.0.117"; # IP or Hostname of backend
.port = "8585"; # Port Apache or whatever is listening
.max_connections = 300; # That's it
.first_byte_timeout = 300s; # How long to wait before we receive a first byte from our backend?
.connect_timeout = 300s; # How long to wait for a backend connection?
.between_bytes_timeout = 300s; # How long to wait between bytes received from our backend?
}
acl purge {
"localhost";
"127.0.0.1";
"::1";
}
sub vcl_init {
new vdir = directors.round_robin();
vdir.add_backend(haproxy);
}
sub vcl_recv {
#if (req.url ~ "^/Payroll/Report") {
# return(pipe);
#}
set req.backend_hint = vdir.backend(); # send all traffic to the vdir director
if (req.http.Host) {
set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
}
unset req.http.proxy;
set req.url = std.querysort(req.url);
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
return (purge);
}
# Only deal with "normal" types
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "PATCH" &&
req.method != "DELETE" ){
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
# Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-websockets.html)
if (req.http.Upgrade ~ "(?i)websocket") {
return (pipe);
}
# Only cache GET or HEAD requests. This makes sure the POST requests are always passed.
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Strip hash, server doesn't need it.
if (req.url ~ "\#") {
set req.url = regsub(req.url, "\#.*$", "");
}
# Strip a trailing ? if it exists
if (req.url ~ "\?$") {
set req.url = regsub(req.url, "\?$", "");
}
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
if (req.url ~ "^[^?]*\.(7z|avi|bz2|flac|flv|gz|mka|mkv|mov|mp3|mp4|mpeg|mpg|ogg|ogm|opus|rar|tar|tgz|tbz|txz|wav|webm|xz|zip)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
# Remove all cookies for static files
# A valid discussion could be held on this line: do you really need to cache static files that don't cause load? Only if you have memory left.
# Sure, there's disk I/O, but chances are your OS will already have these files in their buffers (thus memory).
# Before you blindly enable this, have a read here: https://ma.ttias.be/stop-caching-static-files/
if (req.url ~ "^[^?]*\.(7z|apng|webp|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|ogg|ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
#if (req.url ~ "^/Payroll/Report") {
# unset req.http.Cookie;
# return (hash);
#}
# Send Surrogate-Capability headers to announce ESI support to backend
set req.http.Surrogate-Capability = "key=ESI/1.0";
if (req.http.Authorization) {
# Not cacheable by default
return (pass);
}
if (req.url ~ "(jpg|jpeg|png|gif|tiff|bmp|apng|webp)$") {
return (pipe);
}
return (hash);
}
sub vcl_pipe {
# Called upon entering pipe mode.
# In this mode, the request is passed on to the backend, and any further data from both the client
# and backend is passed on unaltered until either end closes the connection. Basically, Varnish will
# degrade into a simple TCP proxy, shuffling bytes back and forth. For a connection in pipe mode,
# no other VCL subroutine will ever get called after vcl_pipe.
# Note that only the first request to the backend will have
# X-Forwarded-For set. If you use X-Forwarded-For and want to
# have it set for all requests, make sure to have:
# set bereq.http.connection = "close";
# here. It is not set by default as it might break some broken web
# applications, like IIS with NTLM authentication.
# set bereq.http.Connection = "Close";
# Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-websockets.html)
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
}
return (pipe);
}
sub vcl_pass {
# Called upon entering pass mode. In this mode, the request is passed on to the backend, and the
# backend's response is passed on to the client, but is not entered into the cache. Subsequent
# requests submitted over the same client connection are handled normally.
# return (pass);
}
# The data on which the hashing will take place
sub vcl_hash {
# Called after vcl_recv to create a hash value for the request. This is used as a key
# to look up the object in Varnish.
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# hash cookies for requests that have them
if (req.http.Cookie) {
hash_data(req.http.Cookie);
}
# Cache the HTTP vs HTTPs separately
if (req.http.X-Forwarded-Proto) {
hash_data(req.http.X-Forwarded-Proto);
}
}
sub vcl_hit {
# Called when a cache lookup is successful.
if (obj.ttl >= 0s) {
# A pure unadultered hit, deliver it
return (deliver);
}
# https://www.varnish-cache.org/docs/trunk/users-guide/vcl-grace.html
# When several clients are requesting the same page Varnish will send one request to the backend and place the others
# on hold while fetching one copy from the backend. In some products this is called request coalescing and Varnish does
# this automatically.
# If you are serving thousands of hits per second the queue of waiting requests can get huge. There are two potential
# problems - one is a thundering herd problem - suddenly releasing a thousand threads to serve content might send the
# load sky high. Secondly - nobody likes to wait. To deal with this we can instruct Varnish to keep the objects in cache
# beyond their TTL and to serve the waiting requests somewhat stale content.
# if (!std.healthy(req.backend_hint) && (obj.ttl + obj.grace > 0s)) {
# return (deliver);
# } else {
# return (miss);
# }
# We have no fresh fish. Lets look at the stale ones.
if (std.healthy(req.backend_hint)) {
# Backend is healthy. Limit age to 10s.
if (obj.ttl + 10s > 0s) {
#set req.http.grace = "normal(limited)";
return (deliver);
}
} else {
# backend is sick - use full grace
if (obj.ttl + obj.grace > 0s) {
#set req.http.grace = "full";
return (deliver);
}
}
}
sub vcl_miss {
# Called after a cache lookup if the requested document was not found in the cache. Its purpose
# is to decide whether or not to attempt to retrieve the document from the backend, and which
# backend to use.
return (fetch);
}
# Handle the HTTP request coming from our backend
sub vcl_backend_response {
# Called after the response headers has been successfully retrieved from the backend.
# Pause ESI request and remove Surrogate-Control header
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
# Enable cache for all static files
# The same argument as the static caches from above: monitor your cache size, if you get data nuked out of it, consider giving up the static file cache.
# Before you blindly enable this, have a read here: https://ma.ttias.be/stop-caching-static-files/
if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|otf|ogg|ogm|opus|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset beresp.http.set-cookie;
}
#if (bereq.url ~ "^/Payroll/Report") {
# unset beresp.http.set-cookie;
#}
# Large static files are delivered directly to the end-user without
# waiting for Varnish to fully read the file first.
# Varnish 4 fully supports Streaming, so use streaming here to avoid locking.
if (bereq.url ~ "^[^?]*\.(7z|avi|bz2|flac|flv|gz|mka|mkv|mov|mp3|mp4|mpeg|mpg|ogg|ogm|opus|rar|tar|tgz|tbz|txz|wav|webm|xz|zip)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize blocks (128k by default) if the backend doesn't send a Content-Length header, so only enable it for big objects
}
# Sometimes, a 301 or 302 redirect formed via Apache's mod_rewrite can mess with the HTTP port that is being passed along.
# This often happens with simple rewrite rules in a scenario where Varnish runs on :80 and Apache on :8080 on the same box.
# A redirect can then often redirect the end-user to a URL on :8080, where it should be :80.
# This may need finetuning on your setup.
#
# To prevent accidental replace, we only filter the 301/302 redirects for now.
if (beresp.status == 301 || beresp.status == 302) {
set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
}
# Set 2min cache if unset for static files
if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
set beresp.ttl = 120s; # Important, you shouldn't rely on this, SET YOUR HEADERS in the backend
set beresp.uncacheable = true;
return (deliver);
}
# Don't cache 50x responses
if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504) {
return (abandon);
}
# Allow stale content, in case the backend goes down.
# make Varnish keep all objects for 6 hours beyond their TTL
set beresp.grace = 6h;
return (deliver);
}
# The routine when we deliver the HTTP request to the user
# Last chance to modify headers that are sent to the client
sub vcl_deliver {
# Called before a cached object is delivered to the client.
if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
# Please note that obj.hits behaviour changed in 4.0, now it counts per objecthead, not per object
# and obj.hits may not be reset in some cases where bans are in use. See bug 1492 for details.
# So take hits with a grain of salt
set resp.http.X-Cache-Hits = obj.hits;
# Remove some headers: PHP version
unset resp.http.X-Powered-By;
# Remove some headers: Apache version & OS
unset resp.http.Server;
unset resp.http.X-Drupal-Cache;
unset resp.http.X-Varnish;
unset resp.http.Via;
unset resp.http.Link;
unset resp.http.X-Generator;
return (deliver);
}
sub vcl_purge {
# Only handle actual PURGE HTTP methods, everything else is discarded
if (req.method == "PURGE") {
# restart request
set req.http.X-Purge = "Yes";
return(restart);
}
}
sub vcl_synth {
if (resp.status == 720) {
# We use this special error status 720 to force redirects with 301 (permanent) redirects
# To use this, call the following from anywhere in vcl_recv: return (synth(720, "http://host/new.html"));
set resp.http.Location = resp.reason;
set resp.status = 301;
return (deliver);
} elseif (resp.status == 721) {
# And we use error status 721 to force redirects with a 302 (temporary) redirect
# To use this, call the following from anywhere in vcl_recv: return (synth(720, "http://host/new.html"));
set resp.http.Location = resp.reason;
set resp.status = 302;
return (deliver);
}
return (deliver);
}
sub vcl_fini {
# Called when VCL is discarded only after all requests have exited the VCL.
# Typically used to clean up VMODs.
return (ok);
}
P.S: please consider i tried if(req.url ~ $URL){return (pass);} or pipe and if(bereq.url ~ $URL){unset beresp.http.set-cookie;}. Those lines which i tried now command line at the default.vcl file.
haproxy.cfg
frontend HTTP
bind *:80
redirect scheme https if { hdr(Host) -i $URL1 } !{ ssl_fc }
frontend returning-varnish # returning traffic from varnish
bind :8585
#acl $Backend1 hdr(host) -i $URL1
#use_backend $Backend1 if $Backend1
frontend HTTPS
bind *:443 ssl crt $Directory1 crt $Directory2
# http-request lua.cors
# http-response lua.cors "GET,PUT,POST" "localhost:1907"
# http-response lua.cors "GET,PUT,POST" "*"
http-response add-header Access-Control-Allow-Origin %[capture.req.hdr(0)] if { capture.req.hdr(0) -m found }
rspadd Content-Security-Policy:\ upgrade-insecure-requests
reqadd X-Forwarded-Proto:\ https
#rspadd Strict-Transport-Security:\ max-age=31536000
http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; always;"
maxconn 100000
redirect prefix https://$URL2 if { hdr(host) -i $URL4 }
redirect prefix https://$URL3 if { hdr(host) -i $URL4 }
#use_backend Varnish if $Backend1
use_backend $Backend1 if $Backend1
backend Varnish
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server varnish1 $IP2:6081 check
backend $Backend1
mode http
balance leastconn
server $Server $Application_Ip:31717 check inter 1s rise 2 fall 2
P.S: Because of the varnish is not working while I want to download file, I made those lines as a command line at haproxy.cfg file as well.

Answer 1

Based on the logs I'm assuming the response headers of your backend response are not properly formatted.
The VSL error is the following:
-- Error Body cannot be fetched
Your backend response doesn't include a Content-Length header, which is fine. But if you don't return a buffered response, Varnish looks at the Transfer-Encoding header to see whether or not you'r using Chunked Transfer Encoding.
This is the header your application is returning:
-- BerespHeader Transfer-Encoding: identity
When we take a look at the code, specifically this line https://github.com/varnishcache/varnish-cache/blob/55753f98b1b392766bd4baed7eb684c006225f57/bin/varnishd/http1/cache_http1_proto.c#L319, you can see that Varnish throws the error because it doesn't recognize the transfer encoding.
Please set a Transfer-Encoding: chunked header and Varnish will have no issue with this response and will stream the content through based on how your backend flushes the various chunks of content.