How to fix "NGNIX 504 Gateway Time-out" - python-3.x

Question

I am setting up a new server which is running a Flask application. I am running gunicorn to run my flask application(app.py) which is stored in directory "/home/abc/application"
My gunicorn.conf file which is stored at "/etc/tmpfiles.d/gunicorn.conf" in an AWS EC2 instance running AWS linux image. The config for the file is - "d /run/gunicorn 0755 abc abc -"
My Gunicorn service file looks like -
located /etc/systemd/system/gunicorn.service
The file looks as follows -
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ec2-user
Group=ec2-user
WorkingDirectory=/home/abc/application
ExecStart=/usr/local/bin/gunicorn --bind 127.0.0.1:8080 --timeout 60 --log-level=debug wsgi
Restart=always
[Install]
WantedBy=multi-user.target
My /etc/nginx/nginx.conf file looks as follows -
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
# Settings for a TLS enabled server.
server {
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
server_name _;
root /usr/share/nginx/html;
ssl_certificate "/etc/pki/tls/certs/ssl-bundle.crt";
ssl_certificate_key "/etc/pki/tls/private/server.key";
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 10m;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
}
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
#
# location / {
# }
#
# error_page 404 /404.html;
# location = /40x.html {
# }
#
# error_page 500 502 503 504 /50x.html;
# location = /50x.html {
# }
# }
}
}
After configuring the above I have enabled the service and run them. The website does not come up and I get - 504 Gateway Time-out message
The error logs for ngnix shows -
2019/04/18 10:22:58 [error] 2417#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 10:48:42 [error] 2417#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET /home HTTP/1.1", upstream: "http://127.0.0.1:8000/home", host: "xyz.com"
2019/04/18 11:11:31 [error] 11938#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 11:15:13 [error] 11938#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 11:49:22 [error] 20052#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 13:58:39 [error] 19953#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "xyz.com"
2019/04/18 14:03:42 [error] 20764#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:06:13 [error] 21307#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:10:31 [error] 21307#0: *3 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 14:16:39 [error] 21307#0: *5 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "a.b.c.d"
2019/04/18 14:17:39 [error] 21307#0: *7 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "a.b.c.d"
2019/04/18 15:46:16 [error] 21307#0: *9 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:01:25 [error] 21307#0: *11 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:08:27 [error] 12330#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:28:23 [error] 12330#0: *3 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
2019/04/18 16:54:22 [error] 12330#0: *5 upstream timed out (110: Connection timed out) while reading response header from upstream, client: y.a.b.c, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "xyz.com"
The logs for gunicorn service as shown by the command sudo journalctl -u gunicorn.service | tail -n 25 are-
`Apr 19 07:08:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:08:14 -0400] [5762] [INFO] Booting worker with pid: 5762
Apr 19 07:08:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:08:14,702 - console - DEBUG - Debug CONSOLE
Apr 19 07:09:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:09:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:5762)
Apr 19 07:09:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:09:14 -0400] [5944] [INFO] Booting worker with pid: 5944
Apr 19 07:09:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:09:15,070 - console - DEBUG - Debug CONSOLE
Apr 19 07:10:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:10:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:5944)
Apr 19 07:10:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:10:14 -0400] [6135] [INFO] Booting worker with pid: 6135
Apr 19 07:10:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:10:15,361 - console - DEBUG - Debug CONSOLE
Apr 19 07:11:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:11:14 -0400] [21683] [CRITICAL] WORKER TIMEOUT (pid:6135)
Apr 19 07:11:14 ip-x.x.x.x.ec2.internal gunicorn[21683]: [2019-04-19 07:11:14 -0400] [6317] [INFO] Booting worker with pid: 6317
Apr 19 07:11:15 ip-x.x.x.x.ec2.internal gunicorn[21683]: HI 2019-04-19 07:11:15,454 - console - DEBUG - Debug CONSOLE`
I did execute the following commands to give the ec2-user access to the directory -
sudo chown ec2-user.ec2-user abc/ -R
I have tried a lot of tutorials but I cannot make it work. Can you please help me what I am doing wrong? I am not using a virtual environment and how to get the website running on local host at port 8080 to run via proxy on the host server.

Answer 1

I fond the answer. Even though the settings were correct, the environment variables were not being read directly from the script. So I had to declare the environment variables inside the app.py script. This is not a problem when you run the application directly as system variables are read by the application.
Two places where this can be done is either in the gunicorn environment file or inside the script. (depending on the use-case)