nginx responses 504/502 gateway timeout error at times - node.js

I faced with a problem.
I have 6 express apps running on Node and use Nginx as a reverse proxy, all of them running without a problem for months. But recently, when I'm trying to navigate to the inner page of any of site it returned me 502 or 504 nginx error.
When I try to run apps on ngrok or locally they work correctly but on the production server, I got 504/502 error.
Nginx log says
2019/04/10 16:38:12 [error] 1362#1362: *245 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 37.9.113.120, server: my.server, request: "GET /videos/videoId HTTP/1.1", upstream: "http://127.0.0.1:3000/videos/videoId", host: "www.my.host"
I tried to increment timeouts
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
But it didn't help (
Here is my server configuration.
server {
listen x.x.x.x:443 http2;
ssl on;
server_name www.myservername.com;
...(ssl conf here)
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
I dig on StackOverflow similar topics but didn't find a solution. The most strange thing, in this case, is, after some time inner page is available but after I do loading tests and send about 100 requests on production server it stops working for about half or hour
Thanks for the help in advance.

You're listening on 443, using ssl, you must specify your certificates/key :
server {
listen 443 ssl;
server_name www.example.com;
ssl_certificate /var/lib/nginx/ssl/serverssl.crt;
ssl_certificate_key /var/lib/nginx/ssl/serverssl.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_set_header Host example.com;
proxy_pass http://localhost:8192/;
}
}

Related

TLS connection refused with nothing on logs

I've been attempting to install an SSL certificate for the past several hours to no success. There are no errors on the nginx log, but everytime I try to access my website through https I just get a connection refused.
I've tried everything I can find on Google and SO results:
I've disabled the Ubuntu firewall
I've run netstat and there's an active listen on port 443
I've verified the sha256 checksum of the certificate, private key and certificate requests
service nginx restart + service nginx status shows everything ok
I feel like I'm arriving at the limit of my experience with nginx (which isn't much) and Linux, so I turn to SO for help.
Here's a redacted copy of my nginx.conf, which proxies the requests to a Node.JS server:
server {
listen my.ip.address:80;
listen my.ip.address:443 ssl;
ssl_certificate /etc/ssl/my-server.com.crt;
ssl_certificate_key /etc/ssl/my-server.com.key;
server_name my-server.com;
client_max_body_size 700m;
root "/var/www/my-website/";
access_log "/var/www/my-website/logs/access_log";
error_log "/var/www/my-website/logs/error_log";
location / {
proxy_connect_timeout 300s;
proxy_send_timeout 600;
proxy_read_timeout 600;
proxy_buffer_size 64k;
proxy_buffers 16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_pass_header Set-Cookie;
proxy_redirect off;
proxy_hide_header Vary;
proxy_set_header Accept-Encoding '';
proxy_pass http://my.ip.address:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Accel-Internal /internal-nginx-static-location;
proxy_no_cache $arg_aid;
access_log off;
}
location /internal-nginx-static-location/ {
alias /var/www/my-website/;
add_header Cache-Control no-cache;
add_header X-Powered-By nginx;
internal;
}
}
I'm running nginx 1.10.3 on Ubuntu 16.04.3 LTS
Add below line before "ssl_certificate xxx" line and try
ssl on;
Updated:
Can you try with this simple config.
upstream myapp {
server http://my.ip.address:8080;
}
server {
listen 443;
server_name mywebsite;
access_log /var/log/nginx/app_access.log main;
error_log /var/log/nginx/app_error.log;
ssl on;
ssl_certificate /etc/ssl/certs/nginx.crt;
ssl_certificate_key /etc/ssl/certs/nginx.key;
location / {
proxy_pass http://myapp;
}
}
The problem was caused by a nano .conf.save backup on the nginx /sites-enabled folder which had a misconfigured https server declaration - I was not aware that nginx loaded files other than .conf.
'Connection refused' means that nothing was listening at the target IP and port you tried to connect to.
It has nothing to do with HTTPS, TLS, SSL, or certificates whatsoever.

vue with nginx issue

Hi I am running node+express+vue
Here is my nginx configuration
server {
listen 443;
server_name mydomain.me;
ssl on;
ssl_certificate /root/mydomain.me.cert;
ssl_certificate_key /root/mydomain.me.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
location / {
proxy_pass http://localhost:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
}
}
With the above configuration I can access my site using https://mydomain.me but in Chrome's console there are some error appears.
GET https://localhost:8080/sockjs-node/info?t=1517580112789 net::ERR_CONNECTION_REFUSED
client?d420:175 [WDS] Disconnected!
sockjs.js?3600:1601 GET https://localhost:8080/sockjs-node/info?t=1517580112789 net::ERR_CONNECTION_REFUSED
Would you please help me why localhost:8080 is still appears? and how to fix this
The websocket is trying to directly connect to 8080 through nginx. You need to proxy that connection,as well.
Don't forget to change WebSocket URL when you initialise your connection:
var sock = new SockJS('https://mydomain.me');

nodejs nginx 502 gateway error

I am trying to use a nodejs app behind an nginx reverse proxy to handle the ssl
I have my app running on localhost:2000. I can confirm this as working with a curl command.
This is my nginx setup:
# the IP(s) on which your node server is running. I chose port 3000.
upstream dreamingoftech.uk {
server 127.0.0.1:2000;
keepalive 16;
}
# the nginx server instance
server {
listen 0.0.0.0:80;
server_name dreamingoftech.uk;
return 301 https://$host$request_uri;
}
#HTTPS
server {
listen 443 ssl http2;
server_name dreamingoftech.uk;
access_log /var/log/nginx/dreamingoftech.log;
error_log /var/log/nginx/dreamingoftech.error.log debug;
ssl_certificate /etc/letsencrypt/live/dreamingoftech.uk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dreamingoftech.uk/privkey.pem;
include snippets/ssl-params.conf;
# pass the request to the node.js server with the correct headers and much more can be added, see nginx config options
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://dreamingoftech.uk/;
proxy_redirect off;
#proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "";
proxy_ssl_session_reuse off;
proxy_cache_bypass $http_upgrade;
}
}
if I now curl https://dreamingoftech.uk, it takes a while but I do get the webpage delivered. albeit with the message:
curl: (18) transfer closed with 1 bytes remaining to read
However when viewed from a browser I get a 502 gateway error.
I have checked the error log and this is the result: ERROR LOG
I can't understand why the reverse proxy is adding such a time delay into the process. Any ideas would be greatly appreciated.
PS: in the upstream config I have tried localhost instead of 127.0.0.1 to no avail
I have almost the same configuration. Can you try the following
You can redirect all http to https
server {
listen 80;
return 301 https://$host$request_uri;
}
or for a specific site like this
server {
server_name dreamingoftech.uk;
return 301 https://dreamingoftech.uk$request_uri;
}
but choose only one for your case
and then you make sure you node server is running on http mode and not https.
Also you mentioned that you run node on port 3000, then use port 3000 and not 2000 as I can see in your config.
After you confirm the above redirect all packets into localhost like this
server {
listen 443;
server_name dreamingoftech.uk;
ssl_certificate /etc/letsencrypt/live/dreamingoftech.uk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dreamingoftech.uk/privkey.pem;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
proxy_pass http://localhost:3000;
proxy_read_timeout 90s;
proxy_redirect http://localhost:3000 https://dreamingoftech.uk;
}
}
Create a file and sum the above code put it in sites-available with a name like dreamingoftech.uk and the use ln -s to create a softlink into sites-enabled. go to your nginx.conf and make sure you include folder sites-enabled
Then must restart nginx to check if it works
#Stamos Thanks for your reply. I tried that but unfortunately it didn't work. I decided to try the most basic node app I could still using the basic modules I am using.
I tried this and it worked straight away.
The problem is with my app therefore. I will spend time rebuilding and testing step by step until I find the issue,
Thanks for your time!

Socket.io lost connection after few minutes but only with NGINX proxy enabled

I have a problem with socket.io. When I start my Nodejs App Sockets works correctly but after few minutes the connection to websocket is closed and after reconnecting Socket.io fires emit again.
I'm using NGINX Proxy and I have noticed that bypassing NGINX the problem is solved, which configuration I need to edit? I think that the problem is my nginx configuration.
This is my NGINX default config:
server {
listen 80; #listen for all the HTTP requests
server_name example.com www.example.com;
return 301 https://www.example.com$request_uri; }
server {
server_name example.com;
listen 443 ssl http2;
#Optimize Webserver work
#client_max_body_size 16M;
keepalive_timeout 20;
ssl on;
ssl_certificate /root/social/ssl/cert.pem;
ssl_certificate_key /root/social/ssl/key.pem;
location / {
proxy_pass http://localhost:5430;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
upstream io_nodes {
server 127.0.0.1:5430;
keepalive 20;
}
Please help
You should add another parameter:
proxy_read_timeout 96000;
The default value is 60s. You will get a message "lost connect" after 60s idles with the default value.

upstream timed out error in nginx+socket.io+proxy+ssl

I think I almost out of idea. Been trying all solution on other answer and all blogpost, but still out of luck.
Here's what I'm trying to do. I have nodejs server with socket.io running in certain port and want proxy pass from the port 80 with the nginx. The condition is my app still work, and successfully sent the request via socket.io and the other client received it. But the thing is I keep getting "Upstream timed out" error, and there is a lot of them.
This is my current nginx setting after a lot changes:
upstream node_server.com {
server 127.0.0.1:3002 max_fails=0 fail_timeout=10s;
keepalive 512;
}
server {
listen 443;
client_max_body_size 16M;
keepalive_timeout 10;
ssl on;
ssl_certificate my.crt;
ssl_certificate_key my.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
ssl_prefer_server_ciphers on;
server_name my_domain.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_buffers 8 32k;
proxy_buffer_size 64k;
proxy_pass https://node_server.com;
proxy_redirect off;
proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
tcp_nodelay on; # not necessary
}
}
And this is the error message I got (there's a lot of them):
[error] 8131#0: *4599 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 112.215.64.131, server: my_domain.com, request: "GET /socket.io/?EIO=3&sid=ZRsV9eAnEItl90-vAH-j&transport=polling&user=2104655266f3db0f2a HTTP/1.1", upstream: "https://127.0.0.1:3002/socket.io/?EIO=3&sid=ZRsV9eAnEItl90-vAH-j&transport=polling&user=2104655266f3db0f2a", host: "my_domain.com"
Even tough my app still work right now, I want to get rid all of the error.
Really appreciate any help.
Change it
proxy_pass https://node_server.com; to proxy_pass http://node_server.com;
And you should access:
https://my_domain.com
`
Should hope it help !

Resources