Ubuntu 16.04.2
varnish-4.1.1
I stuck here:
https://varnish-cache.org/docs/4.1/tutorial/starting_varnish.html
The very first change in configuration in the whole book. It said: change host to www.varnish-cache.org and reload.
/etc/varnish/default.vcl
vcl 4.0;
backend default {
.host = "www.varnish-cache.org";
.port = "80";
}
I executed:
sudo service varnish restart
sudo service varnish reload
But anyway I constantly have "Error 503 Backend fetch failed".
I have tried:
$ sudo varnishd -d -f default.vcl
Error:
Failed to create vcl_boot/vgc.so: Permission deniedVCL compilation failed
It seems that compilation fails. Could you help me here?
It's a somewhat broken tutorial for a few reasons:
They ask you to point backend to a DNS name. The proper way is to specify IP in backend definitions
Whatever you specify (DNS or IP) it will end up passing Host header of the site you access Varnish with and ask backend server to deliver site with that hostname.
So why you're getting an error as per tutorial:
You access, e.g. http://localhost/ (or whatever hostname you access your Varnish with)
Then Varnish talks to HTTP server at varnish-cache.org and asks for http://localhost.
Obviously the varnish-cache.org server has no idea about that one and most likely (as per their configuration will issue a redirect / error / etc.) thus the error that you see.
It is best to point it to your own web server instead and do it like this:
vcl 4.0;
backend default {
.host = "127.0.0.1";
.port = "8080";
}
The above assumes that you run a web server (nginx or Apache, etc.) at the same machine with Varnish and you made it run at port 8080.
I built and installed squid 3.5.23 as follows:
./configure --prefix=/usr/local/squid
make all
make install
Here is the default squid.conf used by the version. I made minimal modifications to to the file to make my setup anonymous:
forwarded_for delete
request_header_access Via deny all
request_header_access Cache-Control deny all
After I got the (remote) proxy server running, I confirmed that I could configure my (local) browser to send traffic through it. I then took it to the next step, and had my router send all traffic originating from my local network to my proxy server:
iptables -t nat -A PREROUTING -s 192.168.11.0/24 -d ! 192.168.11.0/24 -p tcp -j DNAT --to-destination 100.200.30.40:3128
However, all of my requests came back with a 400 from squid (BAD REQUEST). On investigating further, I discovered that the request headers were using relative urls (my browser is smart enough to always use absolute urls if it knows it is talking to a proxy server).
I know HTTP 1.1 headers are required to have a Host header, which squid can use to determine the original destination of packets it receives. How do I configure the proxy server to use that header? I am looking for the squid 3.5 equivalent of httpd_accel_uses_host_header on
Running squid in accelerator mode fixed this:
http_port 3128 accel
I have magento website in Linux server (Varnish cache), some of the product details page shows error as
Error 503 Backend fetch failed Guru Meditation: XID: 98757
My website IP is 52.163.xxx.xx
Please find the below details and help me to fix this issue.
/etc/default/varnish
DAEMON_OPTS="-a :8080 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"
/etc/varnish/default.vcl
backend default{
.host = "127.0.0.1";
.port = "8080";
}
sudo service varnish restart
Stopping HTTP accelerator varnishd No /usr/sbin/varnishd found running; none killed.
[fail]
Starting HTTP accelerator varnishd [fail]
bind(): Address already in use
bind(): Address already in use
Error: Failed to open (any) accept sockets.
As I understand it, you are running varnish and backend webserver (say nginx or apache) on the very same linux machine, right?
First of all, try to run this command:
sudo netstat -anp | grep LISTEN | grep 8080
And see what process is bound on port 8080 and on which ip.
First part of your question suggests varnish is running, just not be able to connect to backend.
But the second part tells me you are not able to start varnish.
So please make it clear and perhaps attach output from the command above.
Let's continue with second part, i.e. varnish not able to start.
I guess you have backend server running on 8080, be it nginx, apache, whatever.
Your varnish backend config confirms it after all.
Check that web server is bound on 127.0.0.1 and not on 0.0.0.0 not to allow public traffic to connect directly do backend web server.
If this is the case, you have to change listening ip:port of varnish to non-colliding combination.
You can either:
change port to something else as 8080, let's say 80
change port of backend web server to something else if you need 8080 to be public
double check your backend web server is listening on localhost only and bind varnish to your public ip instead of 0.0.0.0 (default, means all machine's ips)
You can do the last option by changing main varnish configuration to:
DAEMON_OPTS="-a 52.163.xxx.xx:8080 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"
This scenario has one important drawback. If you somehow come to new public ip, you have to change it in main varnish configuration too. If this is something you can encode into automation recipe, it shouldn't be problem. But if you manage it by hand, be sure you have really good documenting practice or you'll be hunting ghost bugs in future. :)
One mistake is having both Varnish and your backend server running on the same port 8080. You have 2 options to solve this:
Most straightforward and simple. Adjust Varnish DAEMON_OPTS to listen on port 80.
It may still work on the same ports, provided that you make Varnish and your backend server listen on different interfaces:
Varnish would normally listen on external interface. Thus, adjust your Varnish listen parameter to be bound to specific IP: DAEMON_OPTS="-a 52.163.xxx.xx:8080 ...
Bind your backend server (Apache, Nginx, whatever) to listen only on the loopback interface, 127.0.0.1.
Your VCL is "empty" and you should be using corresponding plugin for Magento which will ensure that Varnish caches things, by generating correct VCL file for you:
Magento 1.x: Turpentine plugin
Magento 2.x: .. is able to generate VCL from admin backend of your Magento installation.
I have only one public ip address so use Varnish as a reverse proxy for multiple servers. Here is the configuration.
1st physical server Varnish/Apache - port 80, port 8080, ip address 10.0.0.40
2nd physical server 3 Drupal Vhosts - port 80, ip address 10.0.0.30
3rd physical server 2 Non Drupal Vhosts - port 80, ip address 10.0.0.31
In /etc/sysconfig/varnish,
DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -u varnish -g varnish \ -S /etc/varnish/secret \ -s file,/var/lib/varnish/varnish_storage.bin,1G"
In default vcl,
backend default { .host = "127.0.0.1"; .port = "8080"; }
Reverse proxy is working ok and I can see Varnish cache working by checking http header. However I am not sure above configuration is correct or optimal, especially only one backend definition on default vcl file. Any advice?
I suggest the following approach:
NGINX > VARNISH > APACHE
Nginx: to handle SSL termination easily and also you can use it to cache the static content. As far as I know that Nginx is better than Varnish in caching the static content also Varnish is not supposed to cache the static content.
Varnish: will receive requests from Nginx and pass it to Apache.
Apache: will act as a load balancer which will send the requests to the backend servers (Drupal/Non-drupal)
Check the following resources:
1- HTTPS Everywhere With Nginx, Varnish And Apache
2- Simple load balancing with Apache
If my answer is not clear enough let me know.
I've just installed a Ubuntu 12.04 server and nginx 1.2.7, removed default from sites-enabled and added my own file into sites-available and symlink at sites-enabled. Then restarted nginx.
Problem: However going to the URL does not load the site. netstat -nlp | grep nginx and netstat -nlp | grep 80 both returns no results! lsof -i :80 also returns nothing. A dig from another server returns the correct ip address so it shouldn't be a DNS problem. I was able to connect to apache which I have now stopped its service. nginx logs also show nothing.
How should I troubleshoot this problem?
/etc/nginx/site-available/mysite.com
server {
listen 80;
server_name www.mysite.com mysite.com *.mysite.com;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
root /var/www/mysite/public;
index index.php index.html;
location / {
try_files $uri $uri/ /index.php?$args ;
}
location ~ \.php$ {
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_read_timeout 300;
}
}
I had this same problem, the solution was that I had not symlinked my siteconf file correctly. Try running vim /etc/nginx/sites-enabled/mysite.com—can you get to it? I was getting "Permission Denied."
If not run:
rm /etc/nginx/sites-enabled/mysite.com
ln -s /etc/nginx/sites-available/mysite.com /etc/nginx/sites-enabled/mysite.com
If your logs are silent on the issue, you may not be including the sites-enabled directory. One simple way to tell that the site is being loaded is to set the error/access log path within your server block to a unique path, reload nginx, and check if the files are created.
Ensure the following include directive exists within the http context in /etc/nginx/nginx.conf.
http {
...
include /etc/nginx/sites-enabled/*;
}
I've found it helpful to approach debugging nginx with the following steps:
1... Make sure nginx is running.
ps aux | grep nginx
2... Check for processes already bound to the port in question.
lsof -n -i:80
3... Make sure nginx has been reloaded.
sudo nginx -t
sudo nginx -s reload
On Mac, brew services restart nginx is not sufficient to reload nginx.
4... Try creating simple responses manually to make sure your location path isn't messed up. This is especially helpful when problems arise while using proxy_pass to forward requests to other running apps.
location / {
add_header Content-Type text/html;
return 200 'Here I am!';
}
I ran into the same problem, I got a Failed to load resource: net::ERR_CONNECTION_REFUSED error when connecting over HTTP, but fine over HTTPS. Ran netstat -tulpn and saw nginx not binding to port 80 for IPv4. Done everything described here. Turned out to be something very stupid:
Make sure the sites-available file with the default_server is actually enabled.
Hope this saved some other poor idiot out there some time.
You are probably binding nginx to port 80 twice. Is that your full config file? Don't you have another statement listening to port 80?
A semi-colon ; missing in /etc/nginx/nginx.conf for exemple on the line before include /etc/nginx/servers-enabled/*; can just bypass this intruction and nginx -t check will be successful anyway.
So just check that all instructions in /etc/nginx/nginx.conf are ended with a semi-colon ;.
I had faced the same problem over the server, here I am listing the how I had solved it :
Step 1 :: Installing the Ngnix
sudo apt update
sudo apt install nginx
Step 2 – Adjusting the Firewall
sudo ufw app list
You should get a listing of the application profiles:
Output
Available applications:
Nginx Full
Nginx HTTP
Nginx HTTPS
OpenSSH
As you can see, there are three profiles available for Nginx:
Nginx Full: This profile opens both port 80 (normal, unencrypted web traffic) and port 443 (TLS/SSL encrypted traffic)
Nginx HTTP: This profile opens only port 80 (normal, unencrypted web traffic)
Nginx HTTPS: This profile opens only port 443 (TLS/SSL encrypted traffic)
Since I haven’t configured SSL for our server yet in this guide, we will only need to allow traffic on port 80.You can enable this by typing:
sudo ufw allow 'Nginx HTTP'
You can verify the change by typing:
sudo ufw status
Step 3 – Checking your Web Server
systemctl status nginx
Now Check port 80 , It worked for me hope will work for you as well.
Have you checked if your nginx binary really exists? please check if
#whereis nginx
outputs the binary path and check this path with your init script from /etc/init.d/nginx. e.g.
DAEMON=/usr/sbin/nginx
(In my init script "test -x $DAEMON || exit 0" is invoked and in any case this script returned nothing - my binary was completely missing)
While we all think we don't make silly mistakes, we do.
So, if you are looking into NGINX issues and all signs are showing it should work then you should take a step away from the files and look downstream.
System Firewall, Hardware Firewall, Nat router/firewall.
For myself this issue was my router, I run a home lab and so I can access services behind my router from afar I use NGINX to reverse proxy as my router only handles incoming based on IP and doesn't do any handling of hostnames, I'm sure this is all fairly normal.
In any case my issue cropped up as I was securing my network a few days ago, removing some port forwarding that isnt needed any longer and I accidentally removed port 80.
Yes it was as simple as forwarding that port again to NGINX and all was fixed.
I will now walk away with my head hung in extreme shame though I leave this answer to show my gratitude to the people in this thread that lead me to find my own error.
So thank you.
In my case those network command's outputs showed nginx was correctly binding to port 80, yet the ports weren't externally accessible or visible with nmap.
While I suspected a firewall, it turns out that old iptables rules on the machine were redirecting traffic from those ports and conflicting with nginx. Use sudo iptables-save to view all currently applicable rules.
I am facing the same issue. Just reload the nginx help me
sudo nginx -t
If you got error then just delete the log.txt file
then,
sudo nginx -s reload