Trying to deploy my node js api onto digital ocean using ngnix i can log into my server through ssh fine, Its only when i try to access my server from a browser i get a 502 error i run logs on the command line and this is what is logged to the console
2022/03/28 16:14:37 [error] 1312008#1312008: *20 connect() failed (111: Connection refused) while connecting to upstream, client: 192.53.170.243, server: telebotstools.io, request: "GET / HTTP/1.1", upstream: "http://[::1]:5000/", host: "188.166.14.11"
2022/03/28 16:30:25 [error] 1399533#1399533: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 71.183.157.83, server: telebotstools.io, request: "GET / HTTP/1.1", upstream: "http://[::1]:8080/", host: "telebotstools.io"
2022/03/28 16:30:25 [error] 1399533#1399533: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 71.183.157.83, server: telebotstools.io, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "telebotstools.io"
2022/03/28 16:35:41 [error] 1399533#1399533: *4 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 71.183.157.83, server: telebotstools.io, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "telebotstools.io"
2022/03/28 16:36:41 [error] 1399533#1399533: *4 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 71.183.157.83, server: telebotstools.io, request: "GET / HTTP/1.1", upstream: "http://[::1]:8080/", host: "telebotstools.io"
I've tried to troubleshoot the problem with no luck
I have restarted the server checked logs rewritten my config file multiple times with no luck
i followed this url to the T and i still cant seem to trouble shoot the problem
server {
listen 80 default_server;
listen [::]:80 default_server;
# include snippets/snakeoil.conf;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name telebotstools.io www.telebotstools.io;
location / {
proxy_pass http://localhost:8080; #whatever port your app runs on
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
Thanks in advance
Related
About a month ago I configured a Digital Ocean Droplet to forward all requests to mydomain.com to Webflow (a no-code site-builder) and any requests to mydomain.com/api/v1 to the Node.js backend running on the same Droplet.
Everything was working, but today I went to the site and got a 502 Bad Gateway Nginx error, and I'm not sure why. Whenever I try and connect, I get these errors:
2022/10/16 19:52:44 [error] 1571#1571: *7 SSL_do_handshake() failed (SSL: error:0A000438:SSL routines::tlsv1 alert internal error:SSL alert number 80) while SSL handshaking to upstream, client: ipAddress, server: mydomain.com, request: "GET / HTTP/1.1", upstream: "https://ipAddress:443/", host: "mydomain.com"
2022/10/16 19:52:45 [error] 1571#1571: *7 SSL_do_handshake() failed (SSL: error:0A000438:SSL routines::tlsv1 alert internal error:SSL alert number 80) while SSL handshaking to upstream, client: ipAddress, server: mydomain.com, request: "GET / HTTP/1.1", upstream: "https://ipAddress:443/", host: "mydomain.com"
2022/10/16 19:52:45 [error] 1571#1571: *7 SSL_do_handshake() failed (SSL: error:0A000438:SSL routines::tlsv1 alert internal error:SSL alert number 80) while SSL handshaking to upstream, client: 162.229.177.82, server: mydomain.com, request: "GET / HTTP/1.1", upstream: "https://ipAddress:443/", host: "mydomain.com"
2022/10/16 19:52:45 [error] 1571#1571: *7 no live upstreams while connecting to upstream, client: ipAddress, server: mydomain.com, request: "GET /favicon.ico HTTP/1.1", upstream: "https://webflow/favicon.ico", host: "mydomain.com", referrer: "https://example.com/"
For privacy I've changed any IP addresses to "ipAddress" and the host to "mydomain.com". What do these errors mean, and what are some potential fixes?
If it helps, my Nginx sites-available file looks like this:
upstream webflow {
server proxy-ssl.webflow.com:443;
}
resolver 8.8.8.8 8.8.4.4;
server {
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name mydomain.com www.mydomain.com;
location / {
proxy_pass https://webflow;
proxy_ssl_server_name on;
proxy_ssl_name $host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
}
location /api/v1/ {
proxy_pass http://dropletIp:3001;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
My nginx.conf works as expected locally without error, but when moving to this App Service environment I get the errors outlined below.
I am developing a React app built on TypeScript using the Azure App Service multi-container (preview) app in Web App for Containers, but running into some issues with NGINX. The main errors I get are some error logs saying this when I try to run the App Service:
"connect() failed (111: Connection refused)"
"no live upstreams while connecting to upstream"
My WEBSITES_PORT under App Service > Settings > Configuration is set to 80. I have also tried to set it to 80:80. In both cases I get the same error logs below. Setting WEBSITES_PORT to 3001 and removing nginx from the list of services in the container settings file results in the App Service deploying successfully.
Let me know if there are other files I can provide in addition to the ones below.
My container settings found under App Service > Settings > Container Settings pointing to my private Azure Container Registry that stores all of my application images. The structure is very similar to the docker compose file I use for local deployment.
version: '3.3'
services:
mysite:
image: "reactapp.azurecr.io/my_site_img"
ports:
- "3001:3001"
nginx:
image: "reactapp.azurecr.io/nginx"
ports:
- "80:80"
An nginx.conf that controls the routing in my nginx image.
upstream my_site_proxy {
server localhost:3001;
}
server {
listen 0.0.0.0:80;
server_name localhost;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://my_site_proxy/;
proxy_redirect off;
}
}
The Error Log file that is generated when I try to run my Azure App Service with the above configuration.
2020-07-13T01:22:52.929149550Z 2020/07/13 01:22:52 [error] 27#27: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /robots1234.txt HTTP/1.1", upstream: "http://127.0.0.1:3001/robots1234.txt", host: "127.0.0.1:4548"
2020-07-13T01:22:52.929653182Z 2020/07/13 01:22:52 [warn] 27#27: *1 upstream server temporarily disabled while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /robots1234.txt HTTP/1.1", upstream: "http://127.0.0.1:3001/robots1234.txt", host: "127.0.0.1:4548"
2020-07-13T01:22:52.930048306Z 2020/07/13 01:22:52 [error] 27#27: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /robots1234.txt HTTP/1.1", upstream: "http://127.0.0.1:3001/robots1234.txt", host: "127.0.0.1:4548"
2020-07-13T01:22:52.930060507Z 2020/07/13 01:22:52 [warn] 27#27: *1 upstream server temporarily disabled while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /robots1234.txt HTTP/1.1", upstream: "http://127.0.0.1:3001/robots1234.txt", host: "127.0.0.1:4548"
2020-07-13T01:22:52.936363702Z 172.16.7.1 - - [13/Jul/2020:01:22:52 +0000] "GET /robots1234.txt HTTP/1.1" 502 157 "-" "-" "-"
2020-07-13T01:22:53.004840493Z 2020/07/13 01:22:53 [error] 27#27: *1 no live upstreams while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /robots933456.txt HTTP/1.1", upstream: "http://my_site_proxy /robots933456.txt", host: "127.0.0.1:4548"
2020-07-13T01:22:53.005790052Z 172.16.7.1 - - [13/Jul/2020:01:22:53 +0000] "GET /robots933456.txt HTTP/1.1" 502 157 "-" "-" "-"
2020-07-13T01:22:53.024544427Z 2020/07/13 01:22:53 [error] 27#27: *4 no live upstreams while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://my_site_proxy /", host: "mysite.azurewebsites.net", referrer: "https://portal.azure.com/"
2020-07-13T01:22:53.025501687Z 172.16.7.1 - - [13/Jul/2020:01:22:53 +0000] "GET / HTTP/1.1" 502 559 "https://portal.azure.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36" "198.8.81.196:62138"
2020-07-13T01:22:53.152345935Z 2020/07/13 01:22:53 [error] 27#27: *5 no live upstreams while connecting to upstream, client: 172.16.7.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "http://my_site_proxy /favicon.ico", host: "mysite.azurewebsites.net", referrer: "https://mysite.azurewebsites.net/"
2020-07-13T01:22:53.153395901Z 172.16.7.1 - - [13/Jul/2020:01:22:53 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://mysite.azurewebsites.net/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36" "198.8.81.196:62138"
You need to change your nginx upstream configuration to this:
upstream my_site_proxy {
server mysite:3001;
}
You should connect to mysite, which is the name of your app container. Docker will resolve this DNS name to the IP address of the app container. You would only connect to localhost if you were running nginx and your app inside the same container (which is not best practice.)
I am setting up a new server which is running a Flask application. I am running gunicorn to run my flask application(app.py) which is stored in directory "/home/abc/application"
My gunicorn.conf file which is stored at "/etc/tmpfiles.d/gunicorn.conf" in an AWS EC2 instance running AWS linux image. The config for the file is - "d /run/gunicorn 0755 abc abc -"
My Gunicorn service file looks like -
located /etc/systemd/system/gunicorn.service
The file looks as follows -
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ec2-user
Group=ec2-user
WorkingDirectory=/home/abc/application
ExecStart=/usr/local/bin/gunicorn --bind 127.0.0.1:8080 --timeout 60 --log-level=debug wsgi
Restart=always
[Install]
WantedBy=multi-user.target
My /etc/nginx/nginx.conf file looks as follows -
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
# Settings for a TLS enabled server.
server {
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
server_name _;
root /usr/share/nginx/html;
ssl_certificate "/etc/pki/tls/certs/ssl-bundle.crt";
ssl_certificate_key "/etc/pki/tls/private/server.key";
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 10m;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
}
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
#
# location / {
# }
#
# error_page 404 /404.html;
# location = /40x.html {
# }
#
# error_page 500 502 503 504 /50x.html;
# location = /50x.html {
# }
# }
}
}
After configuring the above I have enabled the service and run them. The website does not come up and I get - 504 Gateway Time-out message
The error logs for ngnix shows -
2019/04/18 10:22:58 [error] 2417#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 10:48:42 [error] 2417#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET /home HTTP/1.1", upstream: "http://127.0.0.1:8000/home", host: "xyz.com"
2019/04/18 11:11:31 [error] 11938#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 11:15:13 [error] 11938#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 11:49:22 [error] 20052#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 13:58:39 [error] 19953#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 14:03:42 [error] 20764#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:06:13 [error] 21307#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:10:31 [error] 21307#0: *3 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:16:39 [error] 21307#0: *5 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "a.b.c.d"
2019/04/18 14:17:39 [error] 21307#0: *7 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "a.b.c.d"
2019/04/18 15:46:16 [error] 21307#0: *9 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:01:25 [error] 21307#0: *11 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:08:27 [error] 12330#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:28:23 [error] 12330#0: *3 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:54:22 [error] 12330#0: *5 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
The logs for gunicorn service as shown by the command sudo journalctl -u gunicorn.service | tail -n 25 are-
`Apr 19 07:08:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:08:14 -0400] [5762] [INFO] Booting worker with pid: 5762
Apr 19 07:08:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:08:14,702 - console - DEBUG - Debug CONSOLE
Apr 19 07:09:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:09:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:5762)
Apr 19 07:09:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:09:14 -0400] [5944] [INFO] Booting worker with pid: 5944
Apr 19 07:09:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:09:15,070 - console - DEBUG - Debug CONSOLE
Apr 19 07:10:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:10:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:5944)
Apr 19 07:10:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:10:14 -0400] [6135] [INFO] Booting worker with pid: 6135
Apr 19 07:10:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:10:15,361 - console - DEBUG - Debug CONSOLE
Apr 19 07:11:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:11:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:6135)
Apr 19 07:11:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:11:14 -0400] [6317] [INFO] Booting worker with pid: 6317
Apr 19 07:11:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:11:15,454 - console - DEBUG - Debug CONSOLE`
I did execute the following commands to give the ec2-user access to the directory -
sudo chown ec2-user.ec2-user abc/ -R
I have tried a lot of tutorials but I cannot make it work. Can you please help me what I am doing wrong? I am not using a virtual environment and how to get the website running on local host at port 8080 to run via proxy on the host server.
I fond the answer. Even though the settings were correct, the environment variables were not being read directly from the script. So I had to declare the environment variables inside the app.py script. This is not a problem when you run the application directly as system variables are read by the application.
Two places where this can be done is either in the gunicorn environment file or inside the script. (depending on the use-case)
I have nginx server installed and I use spawn-fcgi to create a socket but for some reason the socket refuses the connection from nginx.
This is my config:
location /sympa {
gzip off;
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/sympa/wwsympa.socket;
# If you changed wwsympa_url in sympa.conf, change this regex too!
fastcgi_split_path_info ^(/sympa)(.*)$;
fastcgi_param SCRIPT_FILENAME /usr/lib/cgi-bin/sympa/wwsympa.fcgi;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
Error:
2018/02/10 15:21:37 [error] 4405#4405: *5 connect() to unix:/var/run/sympa/wwsympa.socket failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: 127.0.0.1, request: "GET /sympa HTTP/1.1", upstream: "fastcgi://unix:/var/run/sympa/wwsympa.socket:", host: "127.0.0.1"
Any suggestions?
Quick breakdown -
Dev environment works properly. SSL Cert on local server, nginx configured for proxy pass on 443 traffic.
location / {
proxy_pass http://localhost:3000;
proxy_set_header X-Real-IP $remote_addr;
#proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
for some reason, version 1.1 never works (always tosses an error)
Again, Dev environment works properly.
Production environment is giving me the following error:
WebSocket connection to 'wss://website.com/sockjs/421/fin4cg38/websocket' failed: Error during WebSocket handshake: Unexpected response code: 400
I use a load balancer which has the SSL cert on it and we enforce SSL through our application.
Now, I checked the configuration of NGINX and I have it as follows:
server {
listen 80;
server_name www.website.com;
error_log /var/log/nginx/sites-error.log;
return 301 https://website.com$request_uri;
}
server {
listen 80 default_server;
server_name _;
location / {
proxy_pass http://localhost:3000;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
The way its set up is anything on 80 goes to 443, etc etc.
If I uncomment the proxy_set_headers, our sign up breaks and the chrome console doesn't show any errors.
error log has this:
2014/07/13 18:15:16 [error] 16175#0: *93 connect() failed (111: Connection refused) while connecting to upstream, client: 10.189.245.5, server: _, request: "POST /sockjs/421/wq2dqb2w/xhr_send HTTP/1.1", upstream: "http://[::1]:3000/sockjs/421/wq2dqb2w/xhr_send", host: "website.com", referrer: "https://website.com/signIn"
2014/07/13 18:15:42 [error] 16487#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 10.189.245.5, server: _, request: "GET /sockjs/120/95m6dozn/websocket HTTP/1.1", upstream: "http://[::1]:3000/sockjs/120/95m6dozn/websocket", host: "website.com"
2014/07/13 18:16:12 [error] 16487#0: *12 connect() failed (111: Connection refused) while connecting to upstream, client: 10.189.245.5, server: _, request: "GET /f1be3f98162c975fdc9524a0a222f9a02cc6dcde.js HTTP/1.1", upstream: "http://[::1]:3000/f1be3f98162c975fdc9524a0a222f9a02cc6dcde.js", host: "website.com", referrer: "https://website.com/signIn"
I'm not really sure what's going on. Any help would be awesome.
Thanks!