Nginx changing path - python-3.x

I have been searching thoruhg nginx documentation and posts online but I can't find the answer to this.
I am running a python application and I want one of the paths altered slightly. I can't do this in python for various reasons.
I want the input url:
/public/web/apidocs***
(where * could be anything - including nothing)
to be passed to the python app as
/apidocs*
This is my configuration:
server {
listen 80;
server_name localhost; ##ignored if there is only one server block
charset utf-8;
client_max_body_size 75M;
location = /frontend/webfrontendConnectionData {
try_files $uri #yourapplication;
}
location /public/web/frontend {
alias /frontend/;
autoindex off;
}
location /public/web/adminfrontend {
alias /adminfrontend/;
autoindex off;
}
location ^(/public/web)(/apidocs.*)$ {
try_files $2 #yourapplication;
}
location / {
try_files $uri #yourapplication;
}
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/app/uwsgi.sock;
}
}
Update based on answer so far:
Thank you for pointing out the mistake or missing out the ~ in the config.
The config has changed as follows:
location ~ ^(/public/web)(/apidocs.*)$ {
try_files $2 #yourapplication;
}
location / {
try_files $uri #yourapplication;
}
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/app/uwsgi.sock;
}
Unfortunately it still doesn't work as expected:
wget 127.0.0.1:80/apidocs -> Goes to python App and works as expected
wget 127.0.0.1:80/apidocs/ -> Goes to python App and works as expected
wget 127.0.0.1:80/apidocs/swagger.json -> Goes to python App and works as expected
wget 127.0.0.1:80/public/web/apidocs -> FAILS I want this to give the same response as wget 127.0.0.1:80/apidocs
wget 127.0.0.1:80/public/web/apidocs/ -> FAILS I want this to give the same response as wget 127.0.0.1:80/apidocs/
wget 127.0.0.1:80/public/web/apidocs/swagger.json -> FAILS I want this to give the same response as wget 127.0.0.1:80/apidocs/swagger.json
All the failed responses give me:
robert#ansiblerunner:~/t$ wget 127.0.0.1:80/public/web/apidocs/swagger.json
--2019-04-08 09:50:39-- http://127.0.0.1/public/web/apidocs/swagger.json
Connecting to 127.0.0.1:80... connected.
HTTP request sent, awaiting response... 404 NOT FOUND
2019-04-08 09:50:39 ERROR 404: NOT FOUND.
Can anyone suggest the correct syntax for the rule I want.
Update #2
After reading more nginx documentation I found out that it doesn't take the location rules in order and use the first one it hits, rather it has a complex longest match algorithm. I was worried the "location /" block was then overriding all the paths so I have altered the config so it is like the following:
location /api/public {
try_files $uri #yourapplication;
}
location /api/authed {
try_files $uri #yourapplication;
}
location ~* ^(/public/web)(/apidocs.*)$ {
try_files $2 #yourapplication;
}
#location / {
# try_files $uri #yourapplication;
#}
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/app/uwsgi.sock;
}
/api/public and /api/authed both work as normal.
it's the /public/web/apidocs that doesn't. The difference is that I need nginx to change the path provided to the python app in this case. This is why I have a regexp with two parts and it passing $2 rather than $uri. I don't think it's doing what I expect which is to pass /apidoc to the app.

I don't see a regex, there, as it lacks a tilde.
You probably want a clause that starts with:
location ~ ^(/public/web)(/apidocs.*)$ {
You didn't mention which docs you were reading.
Consulting a tutorial,
Understanding Nginx Server and Location Block Selection Algorithms,
we see the syntax is:
location optional_modifier location_match {
and we optionally can specify:
~: If a tilde modifier is present, this location will be interpreted as a case-sensitive regular expression match.
I cannot explain your reported 404 symptom,
as the location / prefix clause should have absorbed the GET request
and at least given #yourapplication an opportunity to log the raw URL.

I have an answer
I was completely off base with my understanding of what try_files did not work as I expected. I found an article saying if I use it with an underscore it won't try files and will just use it's fallback.
I also found out that I needed to use rewrite to change the path.
Getting rid of the root location may have not been needed but I prefer not to have it in any case.
My final config is as follows:
server {
listen 80;
server_name localhost; ##ignored if there is only one server block
charset utf-8;
client_max_body_size 75M;
#location = /frontend/webfrontendConnectionData {
# try_files $uri #yourapplication;
#}
location /public/web/frontend {
alias /frontend/;
autoindex off;
}
location /public/web/adminfrontend {
alias /adminfrontend/;
autoindex off;
}
# try_files _ #xxx means don't look at any file go to xxx block
location /api/public {
try_files _ #yourapplication;
}
location /api/authed {
try_files _ #yourapplication;
}
location /public/web/apidocs {
rewrite ^/public/web(/apidocs.*)$ /$1? break;
try_files _ #yourapplication;
}
#location / {
# try_files _ #yourapplication;
#}
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/app/uwsgi.sock;
}
}
Thanks to J_H for comments and answers.

Related

Nginx v1.14.2 -- too many redirects

In my limited experience with Nginx, I get too many redirects when I deploy many WEB projects on the one domain.
I want to use one 'location' to match multiple items.
File path: /mnt/vue/web and /mnt/vue/admin and /mnt/page/web
When I visit https://${domain}/page/single, it is wrong. It redirects to https://${domain}/page/single/index.html/index.html/index.html/...
When I visit https://${domain}/vue/web, it is ok.
I've tried multiple variations of this, but none of them seem to work. Any ideas? Thanks in advance.
Here is my Nginx conf:
server {
listen 443 http2;
server_name ${domain};
ssl on;
ssl_certificate /etc/nginx/conf.d/cert/web/web.pem;
ssl_certificate_key /etc/nginx/conf.d/cert/web/web.key;
ssl_session_timeout 5m;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
# wechat verify
root /mnt/wechat;
# for vue h5-wechat
location ~ /(.*)/web {
root /mnt;
try_files $uri $uri/ /$1/web/index.html =404;
}
# for vue admin-web
location ~ /(.*)/admin {
root /mnt;
try_files $uri $uri/ /$1/admin/index.html =404;
}
# for api
location ~ /(.*)/api/(.*)$ {
proxy_pass http://172.16.184.254:$1;
rewrite /(.*)/api/(.*)$ /$2 break;
}
# for single pages !!!!! Where the problem occurred.
location ~ /(.*)/single/ {
alias /mnt/$1/web/;
index index.html;
}
# for cache
location ~ /(css|img|js|fonts|image)/ {
add_header Cache-Control "public, max-age=300";
}
location /files {
rewrite /files/(.*)$ /$1 break;
proxy_pass https://172.16.92.152;
}
}
server {
listen 80;
server_name ${domain};
return 301 https://$server_name$request_uri;
#rewrite ^(.*)$ https://$host$1 permanent;
}
Using alias within a regular expression location block, you need to capture the entire URI. See this document for details.
For example:
location ~ ^/(.*)/single/(.*)$ {
alias /mnt/$1/web/$2;
index index.html;
}

Update file with data from keyboard using sed

Sed is drawing me nuts !!
I'm trying to append a file with a input data from keyboard. This data needs to be put in the line after some text matches criteria.
My file:
location ~* ^/(index\.php/admin|admin) {
allow 10.10.1.25;
try_files $uri $uri/ /index.php?$args;
location ~* \.php$ { try_files /dummy #proxy; }
deny all;
}
That is the part of my script where I'm collecting data from keyboard and run sed:
echo "Type IP you want to whitelist > "
read ip
sed -i '/allow 10.10.1.25;/i allow ${ip};' test.txt
My file now looks like:
location ~* ^/(index\.php/admin|admin) {
allow $ip
allow 10.10.1.25;
try_files $uri $uri/ /index.php?$args;
location ~* \.php$ { try_files /dummy #proxy; }
deny all;
}
What I'm doing wrong ? I want to value from this variable was in the file not $ip. How my sed command should looks like if I wanted this to be inserted after this line:
location ~* ^/(index\.php/admin|admin) {
Many thanks for your help

nginx: how to avoid forced downloads in all wav files?

This is my nginx site config file:
server {
listen 81;
root /path/;
index index.php index.html index.htm;
location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$ {
access_log off;
expires 30d;
}
location /path/to/sounds {
add_header Content-Disposition "inline";
}
location / {
if (!-e $request_filename) {
rewrite ^(.+)$ /index.php?q=$1 last;
}
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass 127.0.0.1:9001;
fastcgi_index index.php;
include fastcgi_params;
}
}
I have several wav files in a specific folder: 0001.wav, 0002.wav, a.wav, b.wav...
A page should play these files but in some cases (files starting with letters, such as a.wav, b.wav) Chrome is downloading them. This is very strange.
For the a.wav file, the Chrome console log says:
Resource interpreted as Document but transferred with MIME type application/octet-stream:
while the same console log says:
Resource interpreted as Document but transferred with MIME type audio/x-wav
I have Apache in the same server. It works fine for all files:
Resource interpreted as Document but transferred with MIME type audio/x-wav
I have already added in /etc/nginx/mime.types:
audio/x-wav wav;
In addition, I also replaced
location /path/to/sounds {
to
location ~ \.wav$ {
What is wrong here?
SOLVED:
I added to my mime.types file:
audio/x-wav wav;
In my nginx config:
location ~ \.wav$ {
add_header Content-Disposition "inline";
}

Is there a Nginx environment variable equivalent to Apaches' {ENV:REDIRECT_STATUS}?

When migrating from Apache to Nginx certain rules from the .htaccess file have to be 'translated' to Nginx the configuration file.
One problem I can't seem to solve, an example is the easiest way to explain:
The request http://www.domain.com/nginx is internally rewritten by Apache to index.php?option=com_content&view=article&id=145
Now I would like to block direct requests to index.php?option=com_content so the page is only available thru http://www.domain.com/nginx in order to avoid duplicate content. In Apache this was achieved by using these .htaccess rules:
# Check if it's the first pass to prevent a loop. In case of first pass, the environment variable contains nothing
# If http://www.domain.com/nginx already internally has been rewritten to index.php?option=com_content&view=article&id=145 {ENV:REDIRECT_STATUS} contains '200' and the request is allowed to be processed
RewriteCond %{ENV:REDIRECT_STATUS} ^$
# Check if the query string contains requests for the page
RewriteCond %{QUERY_STRING} ^index.php?option=com_content&view=article&id=145 [NC]
# If conditions apply, reject request
RewriteRule .* 404 [L]
In Nginx, is there such an environment variable that I may use?
Or should I approach this in a totally different way?
EDIT 1:
In real life, it is not only that one page but a dynamic Joomla site with lots of pages. I tested the above which works, but the intention is to block ALL requests on index.php?option_content&view=article&id=*
EDIT 2:
This is the working NGINX configuration file:
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html/domainname;
index index.php index.html index.htm;
server_name domainname.com;
server_name localhost;
location / {
try_files $uri $uri/ /index.php?$args;
}
# deny running scripts inside writable directories
location ~* /(images|cache|media|logs|tmp)/.*\.(php|pl|py|jsp|asp|sh|cgi)$ {
return 403;
error_page 403 /403_error.html;
}
## give 404 header & redirect to custom errorpage without changing URL ##
error_page 404 = /404_custom.php; #global error page, script handles header
error_page 500 502 503 504 /50x.html;
location =/index.php {
set $arg_set "${arg_option}___${arg_view}___${arg_id}";
if ($arg_set ~* "^(((\w|-)+?)___){2}((\w|-)+?)$") {
return 404;
}
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
}
The Apache approach will not work here, but there are many other ways you can go about this problem, depending on how many such rules you are going to implement and some other conditions. In general case, I would use something like this:
map "${arg_option}___${arg_view}___${arg_id}" $show404 {
default 0;
# Put here the argument value sets of the pages
# you want to hide - one set per line
"com_content___article___145" 1;
}
server {
...
location /nginx {
rewrite ^.*$ /index.php?option=com_content&view=article&id=145 break;
proxy_pass ...
}
location =/index.php {
if ($show404) {
return 404;
}
proxy_pass ...;
}
...
}
EDIT:
If you want to block all requests to index.php where the arguments "option", "view" and "id" are present, no matter what their values are, you could use something like this:
location =/index.php {
set $arg_set "${arg_option}___${arg_view}___${arg_id}";
if ($arg_set ~* "^(((\w|-)+?)___){2}((\w|-)+?)$") {
return 404;
}
proxy_pass ...
}
In case some values of these arguments should be checked, just modify the regular expression to suit your purpuse:
location =/index.php {
set $arg_set "${arg_option}___${arg_view}___${arg_id}";
if ($arg_set ~* "^com_content___article___(\d+)$") {
return 404;
}
proxy_pass ...
}
Also, in your situation map can be used to simplify the configuration, so that you wouldn't have to add another location for every article, instead encapsulating all the rewrite rules in one map block like this:
map "$request_uri" $real_args {
default "";
"~*^/nginx" option=com_content&view=article&id=145;
"~*^/some_article" option=com_content&view=news&id=123;
"~*^/another_article" option=com_content&view=article&id=515;
}
server {
...
location / {
if ($real_args) {
rewrite ^.*$ /index.php?$real_args break;
}
proxy_pass ...
}
location =/index.php {
# See above
}
...
}
EDIT 2:
For one or two exceptions you could improve your regular expression, using a negative look-ahead:
if ($arg_set ~* "^(((\w|-)+?)___){2}((?!175$)(\w|-)+?)$") {
return 404;
}
But if you expect to have a lot of such URLs, you'll have to introduce map into your configuration after all. Otherwise your regular expression will get too complicated and unmanageable. The configuration in this case would look like this:
map "${arg_option}___${arg_view}___${arg_id}" $exception {
default 0;
"com_content___article___175" 1;
"com_content___news___188" 1;
"something___else___211" 1;
}
server {
...
location =/index.php {
set $arg_set "${arg_option}___${arg_view}___${arg_id}";
if ($exception) {
break;
}
if ($arg_set ~* "^(((\w|-)+?)___){2}((\w|-)+?)$") {
return 404;
}
proxy_pass ...;
}
...
}
This might seem somewhat counterintuitive but that's just how "if" works in Nginx. The second "if" will not be evaluated if Nginx meets break in the first "if" block.

How to set variable value based on a subrequest

I am trying to set the value of a variable based on the result if the user is correctly logged in. I will use this variable for conditional ssi. I am using the auth_request module to authorise the user. This authorisation happens for all pages. The problem I am facing is that for 401/403 errors, NGINX passes 401 to the client. What I want to do is to show some pages anyway (even if the authorization fails), but set the variable (to the status of subrequest) for conditional ssi.
Config File
location / {
set $auth_status 100; #Default Value
error_page 401 403 $show_anyway;
error_page 500 = /auth_fallback;
auth_request /auth_public;
auth_request_set $auth_status $upstream_status;
auth_request_set $show_anyway $request_uri;
}
location = /auth_public
{
proxy_pass http://localhost:8081;
proxy_pass_request_body off;
#return 200; #Tried this, but sub request doesn't fire.
}
error_page 401 403 =200 ... can help. I tested the following config
ssi on;
location / {
set $auth_status 100;
auth_request /auth.php;
auth_request_set $auth_status $upstream_status;
error_page 401 403 =200 #process;
}
location #process {
try_files $uri =404;
}
location = /auth.php {
fastcgi_pass 127.0.0.1:9001;
fastcgi_index index.php;
include fastcgi_params;
}
Note about try_files inside #process - $uri/ droped for prevent internal redirect by index directive (if index file found). If internal redirect pass to location /, last error code will be used.

Resources