How to configure NGINX server which download any files in derectory - linux

I am trying to configure NGINX server on Linux which downloads any files from the directory.
But the problem I am facing is when the file is a text file or the file name contains any special character like spaces and ( "()"#"&" ) then the browser will not entertain and gives me nothing.
How do I solve this issue?
my Configuration is as fallows
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
#default_type application/*;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
index index.html index.htm;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name localhost;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;

To download files rather than display them in the browser, the MIME type should be set to application/octet-stream. Which is often declared as the default type anyway.
Normally, nginx uses the file extension to determine which MIME type to use, but this can be turned off in a location by specifying the types directive with an empty set.
Filenames with strange characters can be accessed using %-encoded characters in the URL.
As an experiment, you might want to turn on autoindex and turn off index to browse the files. This will also show you that your difficult filenames can be downloaded too.
location / {
index not_a_file;
autoindex on;
types {}
}
Note: I am not aware of a mechanism to turn off index, other than setting it to an unlikely name. It is possible that the default index.html may not collide with the contents of your download directories.
See this document for more.

Related

Nodejs showing 413 Payload Too Large on nginx server

I'm running this code on my Amazon AWS server without load balancer. It's a simple server I setup. I'm trying to run a code that crawls for data, written in nodejs. Currently, it's showing the error shown below when I upload a lot of data to crawl:
Request Method: POST
Status Code: 413 Payload Too Large
After many of the suggestions I read here on StackOverflow, I added client_max_body_size 500M; on http, server, location and restarted the server but it doesn't have any effect on it.
Here's the nginx.conf file:
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
client_max_body_size 500M;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name example.com;
root /usr/share/nginx/html/crawler;
client_max_body_size 500M;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
#try_files $uri /index.html;
proxy_pass http://127.0.0.1:4200/;
client_max_body_size 500M;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
Where am I going wrong?
I would suggest you to consider those points :
What's the size of your POST request? It might be bigger than 500MB, therefor you could raise the limit even further, technically, you'll only need to put it on the server (Even tho chunking the post data would be way better than raising the limit)
Are you using Express? Maybe did you put up some limits on the request size with Express.
Kind regards

Is it dangerous opening port 3000 of the server?

I want to deploy my Angular + NodeJS application. My NodeJS application runs on http://localhost:3000 on the server. And my Angular application tries to send it's requests to the server with this prefix address: http://server.ip.address:3000. I opened the port 3000 of the server with the following commands to help my program works and it works fine by now.
irewall-cmd --zone=public --add-port=3000/tcp --permanent
firewall-cmd --reload
But I am not sure if I did a good job or not?
My Angular app runs on nginx and my NodeJS app runs on PM2. I also tried to setting a reverse proxy as you can see below inside etc/nginx/nginx.conf, but it didn't work and just opening port 3000 worked for me!
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /demo/stock-front9/dist/strategy;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
#proxy_pass http://localhost:3000;
#proxy_http_version 1.1;
# First attempt to serve request as file, then
# as directory, then redirect to index(angular) if no file found.
try_files $uri $uri/ /index.html;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
What is the best way to deploy Angular + NodeJS application and how can I do it?
You can deploy the app by just assigning port to process.env.PORT, and put the whole angular build in a public/src folder and give the public folder path in node server file.
You can take reference here https://github.com/Ris-gupta/chat-application
There's no best way but there some best practices. Opening port directly on a server is not good solution. I would suggest you to use docker and publish your application inside container with NGINX. Also you can deploy your backend server in same way.

NGINX not show default page on Amazon EC2 Instance

I installed nginx on Fedora. But I don't know why I cannot get the default nginx page by requesting server IP through browser. My request is down by timeout.
But nginx is running.
$ sudo service nginx status
nginx (pid 20372) is running...
My default generated config is
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
index index.html index.htm;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name localhost;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
}
# redirect server error pages to the static page /40x.html
#
error_page 404 /404.html;
location = /40x.html {
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# Settings for a TLS enabled server.
#
# server {
# listen 443 ssl;
# listen [::]:443 ssl;
# server_name localhost;
# root /usr/share/nginx/html;
#
# ssl_certificate "/etc/pki/nginx/server.crt";
# ssl_certificate_key "/etc/pki/nginx/private/server.key";
# # It is *strongly* recommended to generate unique DH parameters
# # Generate them with: openssl dhparam -out /etc/pki/nginx/dhparams.pem 2048
# #ssl_dhparam "/etc/pki/nginx/dhparams.pem";
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# ssl_ciphers HIGH:SEED:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!RSAPSK:!aDH:!aECDH:!EDH-DSS-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!SRP;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
#
# location / {
# }
#
# error_page 404 /404.html;
# location = /40x.html {
# }
#
# error_page 500 502 503 504 /50x.html;
# location = /50x.html {
# }
# }
}
I have no idea what is happening. Also /var/log/nginx/access.log and /var/log/nginx/access.log are empty. Help please...
On Amazon EC2, you will need to open up the firewall to allow incoming HTTP connections from your browser to the instance.
Login to the Amazon Web Console
Go to EC2
Find your instance
Click on its Security Group
Click Inbound Tab
Click Edit
Add Rule -> HTTP, set the Source field to My Ip
The changes will go into effect immediately.
Please note that if you are accessing your instance from a non fixed IP (coffee shop wifi), you will need to change the Source IP address everytime you connect and get assigned a new IP address. So if it works, and then after a while, it seems to hang, that may be why.
If you are also serving HTTPS, you will to add a specific rule also.

Nginx caching when serving static content

I'm serving a directory statically with Nginx, using the following site config, and works well:
server {
listen 80;
server_name spa.me.com;
access_log /var/log/nginx/spa-access.log;
error_log /var/log/nginx/spa-error.log;
location / {
autoindex on;
root /home/ubuntu/spa/dist/;
}
}
Now I want to add caching feature to this server. So I've read some posts and:
Declare a proxy_cache_path.
Use it from my server.
Add a X-Cached header to tell the client the cache status.
So now my config is as follows:
proxy_cache_path /var/cache/nginx/spa levels=1:2 keys_zone=spa:10m inactive=15m;
server {
listen 80;
server_name spa.mediasmart.mobi;
access_log /var/log/nginx/spa-access.log;
error_log /var/log/nginx/spa-error.log;
location / {
proxy_cache spa;
add_header X-Cached $upstream_cache_status;
autoindex on;
root /home/ubuntu/spa/dist/;
}
}
For some reason, the X-Cache header is never sent, so I'm not able to know (using curl, for instance) if a resource is recovered from the cache or not. Furthermore, the cache directory is empty, so it seems no caching feature has been really enabled.
I've also tried to add the $upstream_cache_status to my logs file, creating a custom format and using it:
...
log_format spa_log_format '$remote_addr - $upstream_cache_status [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"';
...
server {
...
access_log /var/log/nginx/spa-access.log spa_log_format;
...
}
But $upstream_cache_status never has a value. Always an empty string is printed on its place in the log.
What I'm doing wrong? Thanks

EPIPE Error with ExpressJS, nginx proxy server

I am running multiple ExpressJS Node apps through an Nginx proxy server, and am getting an EPIPE Error thrown whenever my users try to download a file. This does not happen on my local setup (which is identical to the server's except for the proxy server), so I figure it has something to do with my Nginx configuration.
Here are my Nginx configs:
/etc/nginx/nginx.conf
user www www;
worker_processes 1;
error_log /home/alex/logs/error.log;
pid /var/run/nginx.pid;
worker_rlimit_nofile 8192;
events {
worker_connections 4096; ## Default: 1024
}
http {
include mime.types;
index index.html index.htm index.php;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /home/alex/logs/access.log main;
sendfile on;
tcp_nopush on;
gzip on;
server_names_hash_bucket_size 128; # this seems to be required for some vhosts
include /etc/nginx/conf.d/*.conf;
}
/etc/nginx/conf.d/default.conf
server {
listen 80;
server_name example.com;
# log access and stuff
access_log /home/alex/logs/example-site.log main;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
# Proxy to the NodeJS server
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://127.0.0.1:8999;
}
# redirect server error pages to their HTML
include /etc/nginx/errpages.conf;
}
The ExpressJS server is sending the download using the following code:
app.get('/citrite/p/:patch', function(req, res)
{
if(set.citFiles.indexOf(req.params.patch) == -1)
{
res.send(mbuild.get404());
}
else
{
track.incrViewcount(req.params.patch, 'citrite');
res.download(set.citDir + '/' + req.params.patch, files.doneSaving);
}
});
That code and everything else works fine on my local git repo, but when I push from there and pull on the server-side, the site kicks and screams - it times out on the user's end and gives me an EPIPE error in the console. I am running Node.js version 4.2.1 and ExpressJS version 4.13.3.
I figured out what the problem was: apparently, having sendfile set to on is what was causing the downloads to stall, and turning that off (specifically, removing the directive in the config) fixed the issue. Not exactly sure why this would interfere, but getting rid of the setting cleared things up.

Resources