Nginx - Node.js error with upstream - node.js

Have a Frontend server which accesses a backend api all written in node.js.
Currently have Nginx with Phusion Passenger configured to launch the node app
Everything has been running fine when all environments (dev.qa.prod) experienced this similar issue which crashed our Frontend servers. Restarting Nginx allowed the application to work.
The Errors look like this
[error] 25833#0: *14050 upstream prematurely closed connection while reading response header from upstream, client: 10.0.0.183, server: 54.148.10.11, request: "POST /api/course/54e113e8d98e579c1a790bbd/step/54e113e8d98e579c1a790bbe HTTP/1.1", upstream: "https://54.201.58.163:443/api/course/54e113e8d98e579c1a790bbd/step/54e113e8d98e579c1a790bbe", host: "dev.****.net", referrer: "https://dev.***.net/08f35b9a752554df591279a88babad96fd7e88021084d0396ef7bda16798eaa5743bfc4880015294dd3199482dd9fc564bbf6a681b6881eb5d91b369059a1643"
Nginx Configuration under sites-available/default:
server {
server_name 10.0.0.134;
listen 80;
root /home/ubuntu/web/portal;
passenger_enabled on;
passenger_set_cgi_param _PASSENGER_NODE_CONTROL_SERVER 1;
location /api {
access_log off;
proxy_pass https://dev-api.***.net;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300;
}}

Related

Heath degrading of Elastic Beanstalk

I have set up my node.js server using elastic beanstalk. Recently I have seen that my server health is degrading to sever from ok state and I keep getting emails for that.
While checking the logs in the error. logs of Nginx I found out that most of the errors are occurring because of
2022/04/23 18:34:14 [error] 3620#0: *14 upstream prematurely closed connection while reading response header from upstream, client: 172.31.54.200, server: , request: "POST /parse/functions/processCloverRequest HTTP/1.1", upstream: "http://155.0.0.1:8081/parse/functions/processCloverRequest", host: "server-i9cck-env.us-east-1.elasticbeanstalk.com"
I have also handled each error in the application and increased the read and connect timeout of Nginx.
location / {
proxy_pass http://nodejs;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
}
But still I am getting the emails and the health of my instances degrades. Is there a way to resolve this issue?

Running nginx to serve files and act as a reverse proxy for Node app on same domain

I am currently trying to run Nginx as a reverse proxy for a small Node application and serve up files for the core of a site.
E.g.
/ Statically served files for root of website
/app/ Node app running on port 3000 with Nginx reverse proxy
server {
listen 80;
listen [::]:80;
server_name example.com www.example.com;
root /var/www/example.com/html;
index index.html index.htm;
# Set path for access_logs
access_log /var/log/nginx/access.example.com.log combined;
# Set path for error logs
error_log /var/log/nginx/error.example.com.log notice;
# If set to on, Nginx will issue log messages for every operation
# performed by the rewrite engine at the notice error level
# Default value off
rewrite_log on;
# Settings for main website
location / {
try_files $uri $uri/ =404;
}
# Settings for Node app service
location /app/ {
# Header settings for application behind proxy
proxy_set_header Host $host;
# proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Proxy pass settings
proxy_pass http://127.0.0.1:3000/;
# Proxy redirect settings
proxy_redirect off;
# HTTP version settings
proxy_http_version 1.1;
# Response buffering from proxied server default 1024m
proxy_max_temp_file_size 0;
# Proxy cache bypass define conditions under the response will not be taken from cache
proxy_cache_bypass $http_upgrade;
}
}
This appeared to work at first glance, but what I have found over time is that I am being served 502 errors constantly on the Node app route. This applies to both the app itself, as well as static assets included in the app.
I've tried using various different variations of the above config, but nothing I can find seems to fix the issue. I had read of issues with SELinux, but this is currently not on the server in question.
Few additional bits of information;
Server: Ubuntu 18.04.3
Nginx: nginx/1.17.5
2020/02/09 18:18:07 [error] 8611#8611: *44 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: x, server: example.com, request: "GET /app/assets/images/image.png HTTP/1.1", upstream: "http://127.0.0.1:3000/assets/images/image.png", host: "example.com", referrer: "http://example.com/overlay/"
2020/02/09 18:18:08 [error] 8611#8611: *46 connect() failed (111: Connection refused) while connecting to upstream, client: x, server: example.com, request: "GET /app/ HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "example.com"
Has anyone encountered similar issues, or knows what it is that I've done wrong?
Thanks in advance!
It's may be because of your node router.It's better to share nodes code too.
Anyway try put your main router and static route like app.use('/app', mainRouter); and see it make any sense?

When using Nginx as a reversed proxy, do I need to do routing for pure websocket application in expressjs?

Hard to write a proper title.
I have a pure websocket application which is based on an express server. In development mode the express server does some simple routing to access the html, js and png files. Basically it is a one page app that only handle websocket protocol. But in production mode, I delegate all that to Nginx so I don't do any routing in express production. I was expecting Nginx will find all these files on its "root" directory. I get a "Cannot Get /" error however.
TL;DR version:
When I use Nginx to serve static contents for a one page app, do I need to do anything in express routing? If so, how?
UPD (my Nginx settings, can find another more detail question here):
upstream upstream_project {
server 127.0.0.1:8888;
keepalive 64;
}
server {
listen 0.0.0.0:80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/local/share/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name ws_server;
location / {
try_files $uri $uri/ index.html;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://upstream_project;
proxy_read_timeout 240s;
}
}
UPDATED:
I have found in Nginx log that the client is trying to access the server via ipv6. See this:
kevent() reported that connect() failed (61: Connection refused) while
connecting to upstream, client: ::1, server: ws_server, request: "GET /
HTTP/1.1", upstream: "http://127.0.0.1:8888/", host: "localhost"
I remove the ipv6 server listen line (listen [::]:80 default_server ipv6only=on;) and have to change the try_files line to use file name explicitly.
try_files $uri $uri/app.js ;
Then I can get my app working. But I don't understand. Why do I have to do this?
I still can't access the static png file from the subdirectory of "root" folder. Any help would be appreciated.
NGINX supports WebSocket by allowing a tunnel to be set up between a client and a backend server. For NGINX to send the Upgrade request from the client to the backend server, the Upgrade and Connection headers must be set explicitly, as in this example:
location /wsapp/ {
proxy_pass http://wsbackend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
Take a look on this NGINX guide for more details and examples.

Howto debug when nginx gives 502 bad gateway?

On landing.example.com:10000 have I a webserver that works fine, which is a Docker container that exposes port 10000. Its IP is 172.17.0.2.
What I would like is having a nginx reverse proxy on port 80, and send the visitor to different Docker containers depending on the URL they visit.
server {
listen 80;
server_name landing.example.com;
location / {
proxy_pass http://172.17.0.2:10000/;
}
access_log /landing-access.log;
error_log /landing-error.log info;
}
When I do this, I get 502 Bad Gateway and the log says
2016/04/14 16:58:16 [error] 413#413: *84 connect()
failed (111: Connection refused) while connecting to upstream, client:
xxx.xxx.xxx.xxx, server: landing.example.com, request: "GET / HTTP/1.1",
upstream: "http://172.17.0.2:10000/", host: "landing.example.com"
try this:
upstream my_server {
server 172.17.0.2:10000;
}
server {
listen 80;
server_name landing.example.com;
location / {
proxy_pass http://my_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_redirect http:// $scheme://;
}
}
Here you define the upstream server (your server by IP or hostname)
and make sure to forward the headers too so the server answering knowns who to answer to.

Connection refused on OPTIONS header in Node/Angular/Express App with nginx as reverse proxy

I'm trying to get an express app running on a Digital Ocean Ubuntu 12.04 droplet with nginx as a reverse proxy, but I'm always getting an OPTIONS net::ERR_CONNECTION_REFUSED error when I try to POST a form. I've tried a bunch of different nginx configurations and cors settings and it still happens.
Using Node v0.12, Express v4 and latest nginx.
server {
listen 80;
server_name thepalette.tk;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:3000;
proxy_redirect off;
}
}
Edit:
I've found the nginx error logs and it says the following:
2015/03/05 05:45:15 [error] 6079#0: *1 connect() failed (111: Connection refused)
while connecting to upstream,
client: x.x.x.x,
server: thepalette.tk,
request: "GET /register HTTP/1.1",
upstream: "http://x.x.x.x:3000/register",
host: "thepalette.tk"
use cors middleware for express in your app.js.
For example:
var cors = require("cors");
app.use(cors({
origin: true,
credentials: true
}));

Resources