TLS connection refused with nothing on logs - linux

I've been attempting to install an SSL certificate for the past several hours to no success. There are no errors on the nginx log, but everytime I try to access my website through https I just get a connection refused.
I've tried everything I can find on Google and SO results:
I've disabled the Ubuntu firewall
I've run netstat and there's an active listen on port 443
I've verified the sha256 checksum of the certificate, private key and certificate requests
service nginx restart + service nginx status shows everything ok
I feel like I'm arriving at the limit of my experience with nginx (which isn't much) and Linux, so I turn to SO for help.
Here's a redacted copy of my nginx.conf, which proxies the requests to a Node.JS server:
server {
listen my.ip.address:80;
listen my.ip.address:443 ssl;
ssl_certificate /etc/ssl/my-server.com.crt;
ssl_certificate_key /etc/ssl/my-server.com.key;
server_name my-server.com;
client_max_body_size 700m;
root "/var/www/my-website/";
access_log "/var/www/my-website/logs/access_log";
error_log "/var/www/my-website/logs/error_log";
location / {
proxy_connect_timeout 300s;
proxy_send_timeout 600;
proxy_read_timeout 600;
proxy_buffer_size 64k;
proxy_buffers 16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_pass_header Set-Cookie;
proxy_redirect off;
proxy_hide_header Vary;
proxy_set_header Accept-Encoding '';
proxy_pass http://my.ip.address:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Accel-Internal /internal-nginx-static-location;
proxy_no_cache $arg_aid;
access_log off;
}
location /internal-nginx-static-location/ {
alias /var/www/my-website/;
add_header Cache-Control no-cache;
add_header X-Powered-By nginx;
internal;
}
}
I'm running nginx 1.10.3 on Ubuntu 16.04.3 LTS

Add below line before "ssl_certificate xxx" line and try
ssl on;
Updated:
Can you try with this simple config.
upstream myapp {
server http://my.ip.address:8080;
}
server {
listen 443;
server_name mywebsite;
access_log /var/log/nginx/app_access.log main;
error_log /var/log/nginx/app_error.log;
ssl on;
ssl_certificate /etc/ssl/certs/nginx.crt;
ssl_certificate_key /etc/ssl/certs/nginx.key;
location / {
proxy_pass http://myapp;
}
}

The problem was caused by a nano .conf.save backup on the nginx /sites-enabled folder which had a misconfigured https server declaration - I was not aware that nginx loaded files other than .conf.

'Connection refused' means that nothing was listening at the target IP and port you tried to connect to.
It has nothing to do with HTTPS, TLS, SSL, or certificates whatsoever.

Related

can Nginx randomly stop working by certain requests?

I'm currently having issues with my website. Sometimes, after a fresh restart of nginx service the url of my website works just fine in the browser, It redirects successfully to the .NET Core webapp running on Kestrel.If I type the IP of my vps it also works just fine. But suddenly and randomly nginx stops serving the website and the browser just shows err_connection_closed.
Some technical information:
Kestrel is running on localhost:5000, Nginx TCP ports are managed by ufw and opened for: 80 and 443.
I'm using: Ubuntu 16.04, nginx and a .NET Core 3.1 web app. Steps were followed as next url Host and Deploy using Linux and Kestrel
Something that I have noticed in syslog file is that some IPs are blocked by ufw, but I'm not sure why they are coming from China, Mongolia or even Poland, as the initial marketing campaign is currently located for Mexico.
Other log file that I searched in was /var/log/nginx/access.log Here, some IPs try to request random urls in my website like GET /Telerik.Web.UI.WebResource.axd?type=rau HTTP/1.1" 404 0 "-" or even like "GET /phpmyadmin/ HTTP/1.1" 301 178 "-" which is absolutely not me because I'm using PostgreSQL. Although, I have to say that I've seen that after this requests are randomly made, the nginx stops working but I'm not 100% sure if this is accurate, as seen in the title, this is very random.
Some config files for nginx:
/etc/nginx/sites-available/default
# Default server configuration
#
server {
listen 80;
server_name keecheeapp.com *.keecheeapp.com;
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
/etc/nginx/proxy_conf
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffers 32 4k;
/etc/nginx/nginx.conf
#other directives
events {
worker_connections 768;
# multi_accept on;
}
http {
include /etc/nginx/proxy.conf;
limit_req_zone $binary_remote_addr zone=one:10m rate=5r/s;
server_tokens off;
sendfile on;
keepalive_timeout 29; # Adjust to the lowest possible value that makes sense for your use case.
client_body_timeout 10; client_header_timeout 10; send_timeout 10;
upstream keecheeapp{
server localhost:5000;
}
server {
listen *:80;
add_header Strict-Transport-Security max-age=15768000;
return 301 https://$host$request_uri;
}
server {
listen *:443 ssl;
server_name keecheeapp.com;
ssl_certificate /etc/ssl/certs/keecheeapp.com-concat-certs.crt;
ssl_certificate_key /etc/ssl/certs/private_new.key;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
#Redirects all traffic
location / {
proxy_pass http://www.keecheeapp.com;
limit_req zone=one burst=10 nodelay;
}
}
}
There are several issues with your Nginx configuration:
In the file /etc/nginx/nginx.conf
The combination of limit_req_zone $binary_remote_addr zone=one:10m rate=5r/s; and limit_req zone=one burst=10 nodelay; will limit the request processing rate per client to 5 requests/second. If you send too many requests per second then you will get error messages from Nginx. So if you want to keep the limit feature, try to increase the existing value to, for example, rate=50r/s and burst=100. If you want to disable this feature, delete or comment out those lines. You can learn more about this feature here.
The value http://www.keecheeapp.com for the proxy_pass directive is wrong . The correct value is keecheeapp as defined by the upstream keecheeapp {...} block. So change proxy_pass http://www.keecheeapp.com; to proxy_pass http://keecheeapp;
The server block in the file /etc/nginx/sites-available/default instructs Nginx to serve your website using HTTP.
The following server block in the file /etc/nginx/nginx.conf instructs Nginx to serve your website using HTTPS.
server {
listen *:443 ssl;
server_name keecheeapp.com;
...
}
So your website is accessible over both HTTP and HTTPS. It's not a good idea. You should redirect all HTTP requests to HTTPS as follows:
Delete or comment out the server block in in the file /etc/nginx/sites-available/default
Modify the following server block in the file /etc/nginx/nginx.conf
server {
listen *:80;
add_header Strict-Transport-Security max-age=15768000;
return 301 https://$host$request_uri;
}
To:
server {
listen *:80;
server_name keecheeapp.com *.keecheeapp.com;
add_header Strict-Transport-Security max-age=15768000;
return 301 https://$host$request_uri;
}
With your given configuration, Nginx is passing all requests to Kestrel, including static file requests (image, JS, CSS, etc.). This is unrealistic. Let Nginx handle static files, and Kestrel handles dynamic requests. Please change the following configuration block:
#Redirects all traffic
location / {
proxy_pass http://www.keecheeapp.com;
limit_req zone=one burst=10 nodelay;
}
To:
root /path/to/your/static/folder;
# Serve static file requests
location / {
try_files $uri $uri/ #kestrel;
}
# Pass dynamic requests to Kestrel
location #kestrel {
proxy_pass http://keecheeapp;
limit_req zone=one burst=10 nodelay;
}
Change /path/to/your/static/folder to the actual folder on your server.
After editing, don't forget to test Nginx configuration with sudo nginx -t, then reload it with sudo systemctl reload nginx.service.

nginx responses 504/502 gateway timeout error at times

I faced with a problem.
I have 6 express apps running on Node and use Nginx as a reverse proxy, all of them running without a problem for months. But recently, when I'm trying to navigate to the inner page of any of site it returned me 502 or 504 nginx error.
When I try to run apps on ngrok or locally they work correctly but on the production server, I got 504/502 error.
Nginx log says
2019/04/10 16:38:12 [error] 1362#1362: *245 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 37.9.113.120, server: my.server, request: "GET /videos/videoId HTTP/1.1", upstream: "http://127.0.0.1:3000/videos/videoId", host: "www.my.host"
I tried to increment timeouts
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
But it didn't help (
Here is my server configuration.
server {
listen x.x.x.x:443 http2;
ssl on;
server_name www.myservername.com;
...(ssl conf here)
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
I dig on StackOverflow similar topics but didn't find a solution. The most strange thing, in this case, is, after some time inner page is available but after I do loading tests and send about 100 requests on production server it stops working for about half or hour
Thanks for the help in advance.
You're listening on 443, using ssl, you must specify your certificates/key :
server {
listen 443 ssl;
server_name www.example.com;
ssl_certificate /var/lib/nginx/ssl/serverssl.crt;
ssl_certificate_key /var/lib/nginx/ssl/serverssl.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_set_header Host example.com;
proxy_pass http://localhost:8192/;
}
}

nodejs nginx 502 gateway error

I am trying to use a nodejs app behind an nginx reverse proxy to handle the ssl
I have my app running on localhost:2000. I can confirm this as working with a curl command.
This is my nginx setup:
# the IP(s) on which your node server is running. I chose port 3000.
upstream dreamingoftech.uk {
server 127.0.0.1:2000;
keepalive 16;
}
# the nginx server instance
server {
listen 0.0.0.0:80;
server_name dreamingoftech.uk;
return 301 https://$host$request_uri;
}
#HTTPS
server {
listen 443 ssl http2;
server_name dreamingoftech.uk;
access_log /var/log/nginx/dreamingoftech.log;
error_log /var/log/nginx/dreamingoftech.error.log debug;
ssl_certificate /etc/letsencrypt/live/dreamingoftech.uk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dreamingoftech.uk/privkey.pem;
include snippets/ssl-params.conf;
# pass the request to the node.js server with the correct headers and much more can be added, see nginx config options
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://dreamingoftech.uk/;
proxy_redirect off;
#proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "";
proxy_ssl_session_reuse off;
proxy_cache_bypass $http_upgrade;
}
}
if I now curl https://dreamingoftech.uk, it takes a while but I do get the webpage delivered. albeit with the message:
curl: (18) transfer closed with 1 bytes remaining to read
However when viewed from a browser I get a 502 gateway error.
I have checked the error log and this is the result: ERROR LOG
I can't understand why the reverse proxy is adding such a time delay into the process. Any ideas would be greatly appreciated.
PS: in the upstream config I have tried localhost instead of 127.0.0.1 to no avail
I have almost the same configuration. Can you try the following
You can redirect all http to https
server {
listen 80;
return 301 https://$host$request_uri;
}
or for a specific site like this
server {
server_name dreamingoftech.uk;
return 301 https://dreamingoftech.uk$request_uri;
}
but choose only one for your case
and then you make sure you node server is running on http mode and not https.
Also you mentioned that you run node on port 3000, then use port 3000 and not 2000 as I can see in your config.
After you confirm the above redirect all packets into localhost like this
server {
listen 443;
server_name dreamingoftech.uk;
ssl_certificate /etc/letsencrypt/live/dreamingoftech.uk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dreamingoftech.uk/privkey.pem;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
proxy_pass http://localhost:3000;
proxy_read_timeout 90s;
proxy_redirect http://localhost:3000 https://dreamingoftech.uk;
}
}
Create a file and sum the above code put it in sites-available with a name like dreamingoftech.uk and the use ln -s to create a softlink into sites-enabled. go to your nginx.conf and make sure you include folder sites-enabled
Then must restart nginx to check if it works
#Stamos Thanks for your reply. I tried that but unfortunately it didn't work. I decided to try the most basic node app I could still using the basic modules I am using.
I tried this and it worked straight away.
The problem is with my app therefore. I will spend time rebuilding and testing step by step until I find the issue,
Thanks for your time!

Socket.io with Sails.js/Node.js and NGINX on SSL: bad gateway

I am cross posting this from Serverfault because it seems the Sails team monitors Stackoverflow.
I just started venturing in NGINX and SSL.
Using Ubuntu 16.04.
I am running a Sails server on the standard 1337 port and just set up NGINX with SSL (using letsencrypt). Port 80 is redirected to 443 and upstream goes to Sails.
I also have a Tomcat server listening on 8080 and use NGINX to redirect the same way.
Everything works fine: I can browse both servers on https without special ports on browser.
I have set up socket.io to use websockets protocol only (no polling). This is set on the server and on the browser client.
However, socket.io (sails.io) throws a 502 error no the browser. (polling gave an error too)
Here is my NGINX sites-available for the Sails server:
upstream sails {
server 127.0.0.1:1337 fail_timeout=0;
}
server {
listen 80;
listen [::]:80;
server_name mysails.server.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443;
listen [::]:443 ssl http2;
server_name mysails.server.com;
include snippets/ssl-mysails.server.conf;
include snippers/ssl-params.conf;
large_client_header_buffers 8 32k;
location / {
proxy_pass http://sails/;
proxy_http_version 1.1;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header Port $server_port;
proxy_set_header X-Real-IP $remot_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass_request_headers on;
}
location /socket.io/ {
proxy_pass http://sails/;
proxy_http_version 1.1;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header Port $server_port;
proxy_set_header X-Real-IP $remot_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass_request_headers on;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffers 8 32k;
proxy_buffer_size 64k;
}
}
The snippets/ssl-mysails.server.conf and snippers/ssl-params.conf files contain:
ssl_certificate /path/to/letsencrypt/fullchain.pem;
ssl_certificate_key /path/to/letsencrypt/privkey.pem;
and
# from https://cipherli.st/
# and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable preloading HSTS for now. You can use the commented out header line that includes
# the "preload" directive if you understand the implications.
#add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
Anyone has any clue on what's going on...?
** UPDATE **
I have added a lot of information about logs and behaviours on the Serverfault question
https://serverfault.com/questions/829100/socket-io-with-sails-js-node-js-and-nginx-on-ssl-bad-gateway
So the issue is a bad configuration on the NGINX sites-avalable conf file.
location /socket.io/ {
proxy_pass http://sails/;
...
}
should be
location /socket.io/ {
proxy_pass http://sails/socket.io/;
...
}
Pretty basic stuff: the "location" is not forwarded to the proxy_pass (why would it be, right?) -- So you need to make sure the socket requests are redirected to the exact socket endpoint.

2 way SSL between nginx and backend server

I have a backend server on Node.js and I am trying to setup 2 way SSL between Nginx and this backend server.
But I get an error as:
2015/11/02 06:51:02 [error] 12840#12840: *266 upstream SSL certificate does not match "myLocalMachine" while SSL handshaking to upstream,
and this is when I set proxy_ssl_verify on. If its off then it works fine. Following is my Nginx setup:
upstream myLocalMachine {
server MyPublicIP:8888;
}
server {
listen 8222 ssl;
proxy_cache two;
ssl_certificate /etc/nginx/ssl/server-cert.pem;
ssl_certificate_key /etc/nginx/ssl/server-key.pem;
ssl_client_certificate /etc/nginx/ssl/client-cert.pem;
ssl_verify_client on;
location / {
proxy_ssl_session_reuse off;
proxy_ssl_verify on;
proxy_ssl_trusted_certificate /etc/nginx/ssl/backend-server-cert.pem;
proxy_ssl_certificate /etc/nginx/ssl/server-cert.pem;
proxy_ssl_certificate_key /etc/nginx/ssl/server-key.pem;
proxy_ssl_password_file /etc/nginx/ssl/pwd.pass;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass https://myLocalMachine;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_cache_valid any 1m;
proxy_cache_min_uses 1;
#proxy_cache_bypass $cookie_nocache $arg_nocache$arg_comment;
proxy_cache_methods GET HEAD POST;
proxy_cache_key "$request_body";
}
}
Solution:
I had used url in certificates lets say: example.com but I was using some
custom name as myLocalMachine in upstream and proxy_pass which I replaced it with url.
Use url in upstream block & proxy_pass as below
upstream example.com {
# ip & ports are for examples
server 11.11.11.11:2222;
}
proxy_pass https://example.com;

Resources