What is causing this 301 redirect? [closed] - linux

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 11 years ago.
The community reviewed whether to reopen this question 8 months ago and left it closed:
Original close reason(s) were not resolved
I have a problem with my server redirecting http://www.mylesgray.com:8080/ -> http://www.mylesgray.com/.
Here are my Nginx default and fastcgi_params config files:
https://gist.github.com/1745271
https://gist.github.com/1745313
This is rather a nusance as I am trying to run a benchmark of Nginx w/ caching vs Varnish w/ caching on top of Nginx to see if there is any performance benefit of one over the other.
As such I have straight Nginx w/ caching listening on port 8080 and varnish on port 80 which forwards any non-cached requests to Nginx on localhost:8080, so obviously what I want to do is run an ab benchmark on http://www.mylesgray.com:8080/ and on http://www.mylesgray.com/ to see the difference.
Here are the results of curl -I on various addresses.
# curl -I http://www.mylesgray.com:8080
HTTP/1.1 301 Moved Permanently
Server: nginx/0.7.65
Date: Sun, 05 Feb 2012 12:07:34 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/5.3.2-1ubuntu4.7ppa5~lucid1
X-Pingback: http://www.mylesgray.com/xmlrpc.php
Location: http://www.mylesgray.com/
# curl -I http://mylesgray.com
HTTP/1.1 301 Moved Permanently
Server: nginx/0.7.65
Content-Type: text/html; charset=UTF-8
X-Powered-By: PHP/5.3.2-1ubuntu4.7ppa5~lucid1
X-Pingback: http://www.mylesgray.com/xmlrpc.php
Location: http://www.mylesgray.com/
Content-Length: 0
Date: Sun, 05 Feb 2012 12:15:51 GMT
X-Varnish: 1419774165 1419774163
Age: 15
Via: 1.1 varnish
Connection: keep-alive
# curl -I http://mylesgray.com:8080
HTTP/1.1 301 Moved Permanently
Server: nginx/0.7.65
Date: Sun, 05 Feb 2012 12:16:08 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/5.3.2-1ubuntu4.7ppa5~lucid1
X-Pingback: http://www.mylesgray.com/xmlrpc.php
Location: http://www.mylesgray.com/
Then running curl -I http://www.mylesgray.com gives:
# curl -I http://www.mylesgray.com
HTTP/1.1 200 OK
Server: nginx/0.7.65
Content-Type: text/html; charset=UTF-8
X-Powered-By: PHP/5.3.2-1ubuntu4.7ppa5~lucid1
X-Pingback: http://www.mylesgray.com/xmlrpc.php
Content-Length: 5132
Date: Sun, 05 Feb 2012 12:07:29 GMT
X-Varnish: 1419774133 1419774124
Age: 30
Via: 1.1 varnish
Connection: keep-alive
So as you can see 80 is served by Varnish and 8080 by Nginx but I cannot find anywhere anything that does a 301 redirect, not in nginx.conf or in the sites-enabled/default file and I don't believe it is caused by Wordpress itself but an very much open to correction.
Please help, this is driving me nuts!
Myles

You should add a '/' at then end of your URLs. Furthermore if you run ab http://foo.com it will return you a "ab: invalid URL" error. If you do "ab -t 10 http://example.com/" everything will work fine. You should always use '/' in your URLs otherwize your webserver will try to redirect the page to the home page automatically for you which generates an undesirable extra load on the server and some extra bytes on the wire.
You web server told you what it did:
'/' is missing and something is incorrect with the port numer:
# curl -I http://www.mylesgray.com:8080
HTTP/1.1 301 Moved Permanently
[...]
======> Location: http://www.mylesgray.com/
'www' and '/' are missing:
# curl -I http://mylesgray.com
HTTP/1.1 301 Moved Permanently
[...]
=======> Location: http://www.mylesgray.com/
[...]
'/' and 'www' are missing:
# curl -I http://mylesgray.com:8080
HTTP/1.1 301 Moved Permanently
[...]
========> Location: http://www.mylesgray.com/
'hope that helps :)

The presence of an X-Powered-By: PHP header means that wordpress is issuing the 301. It's due to wordpress forcing www.mylesgray.com. When you use a nonstandard port, user agents will generally include the port in the Host: header. Try adding
fastcgi_param HTTP_HOST $host;
with the rest of your fastcgi_param directives (or alog with your "include fastcgi_params;") and it should fix this.

Related

Azure CDN - Images Respond 404 to CURL

We have a vendor who sends us photos that are hosted on Azure Edge. These photos are available and I can download them, but if we do a CURL request we get a 404 roughly 4 out of 5 times. If we do a HEAD request to get the filesize, we get a 404 about 7 out of 10 times. On our production server, we get a 404 100% of the time. Any idea how we might work around this or if there's another way to check these files without the vendor having to fix their issue?
Sample file:
curl -I http://tdrvehicles2.azureedge.net/photos/202008/1419/1850/f253435f-86b1-4cc4-b95c-7756addddad4.jpg
HTTP/1.1 404 Not Found
Pragma: no-cache
Content-Length: 0
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Cache-Control: max-age=31536000
Expires: Thu, 19 Aug 2021 14:12:54 GMT
Date: Wed, 19 Aug 2020 14:12:54 GMT
Connection: keep-alive```

302 redirect loop... with a twist

I have this situation that utterly baffles me:
The setup
a newly setup server (centos 7, apache, mysql, nothing fancy) that hosts a simple php app that I need to interact with from my main app on another server. This service is setup to run on service-name.domain.tld while the main app is on domain.tld (just mentioning in case it make any difference).
The problem
For some reason when I try to access the service app from the main server I get an endless loop of 302 redirects.
If I do a curl -D - http://service-name.domain.tld from the main server I get:
HTTP/1.1 302 Found
Date: Wed, 03 Aug 2016 12:30:26 GMT
Server: Apache
Location: http://service-name.domain.tld
Content-Length: 218
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
</body></html>
If I do the same command from my computer, the result is
HTTP/1.1 200 OK
Date: Wed, 03 Aug 2016 12:31:39 GMT
Server: Apache/2.4.6 (CentOS) PHP/5.6.24
X-Powered-By: PHP/5.6.24
Cache-Control: no-cache
Content-Length: 30
Content-Type: text/html; charset=UTF-8
Welcome.
As it should be. Same happens for any request I make (even static files). Also I should specify same situation happens for wget and get_file_contents in php
I'm really lost right now, not knowing where to go from here. So any kind of direction is greatly appreciated.

url excluded by still returning via varnish

I have excluded the url like this from my varnish cache
if (req.url ~ "^/folder_name/") {
return (pass);
}
but still when i access curl -I http://ip/folder_name i see the below response
HTTP/1.1 301 Moved Permanently
Date: Mon, 04 Jul 2016 08:48:46 GMT
Server: Apache/2.2.15 (CentOS)
Location: http://ip/folder_name/
Content-Length: 319
Content-Type: text/html; charset=iso-8859-1
X-Varnish: 294958
Age: 0
Via: 1.1 varnish-v4
Connection: keep-alive
Can anyone please tell me what am i doing wrong, i need to exclude
"all urls that start with /folder_name" from being cached in varnisg
Your regex ends in a slash and "http://ip/folder_name does" not - could this be it?

Data attached to POST request is gone during redirection to GET with CURL

I wrote followed command to send POST with JSON data to server. The server must redirect my request and send GET with the same data:
curl -L -i -XPOST \
-d 'id=105' \
-d 'json={"orderBy":0,"maxResults":50}' http://mysite.com/ctlClient/
I get response:
HTTP/1.1 302 Found
Date: Thu, 04 Jul 2013 13:12:08 GMT
Server: Apache
X-Powered-By: PHP/5.3.19
Set-Cookie: PHPSESSID=1hn0g8d7gtfl4nghjvab63btmk2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
location: http://mysite.com/fwf/online/
Content-Length: 0
Connection: close
Content-Type: text/html
HTTP/1.1 200 OK
Date: Thu, 04 Jul 2013 13:12:08 GMT
Server: Apache
X-Powered-By: PHP/5.3.19
Set-Cookie: PHPSESSID=16akc7kdcoet71ipjflk9o9cnm5; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 1
Connection: close
Content-Type: text/html
From access-log I see:
"POST /ctlClient/ HTTP/1.1" 302 - "-" "Apache-HttpClient/4.1 (java 1.5)"
"GET /fwf/online/ HTTP/1.1" 200 1 "-" "Apache-HttpClient/4.1 (java 1.5)"
So far so good,
The problem is that GET doesn't receives my data added to post. Sounds like during redirect my data dismissed somehow. From Android client it works, therefore its not Server side problem.
What I need to do to pass POST data to GET request?
Thank you very much,
[EDIT]
#nif offerd to upgrade CURL, i did , to 7.28.0.
Still get the same problem
[INFO]
1st time i go to http://mysite.com/ctlClient/index.php where:
case 105: // id=105
session_unset();
session_start();
foreach($_POST as $key => $value){$_SESSION[$key] = $value;}
ctlGotoSameDomain("/fwf/online/"); // <- aka redirect
return true;
after redirect i go to /fwf/online/index.php and there my request is empty:
public function __construct() {
$this->json = isset($_SESSION['json']) ? $_SESSION['json'] : null;
msqLogFile("fwf/post", Array('post' => 'Request: '.$this->json));
}
http://mysite.com/ctlClient/index.php get 2 parameters properly: id and json
From curl's manpage:
When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.
Edit
I did some research and found out, that it might be a problem with your curl version. Newer version will honour the -XPOST option and will POST to the redirected location as well. But older versions had an own option for this, i.e. --post301 and --post302. According to their manpage:
--post301
Tells curl to respect RFC 2616/10.3.2 and not convert POST requests into GET
requests when following a 301 redirection. The non-RFC behaviour is ubiquitous
in web browsers, so curl does the conversion by default to maintain
consistency. However, a server may require a POST to remain a POST after such
a redirection. This option is meaningful only when using -L, --location
(Added in 7.17.1)
--post302
Tells curl to respect RFC 2616/10.3.2 and not convert POST requests into GET
requests when following a 302 redirection. The non-RFC behaviour is ubiquitous
in web browsers, so curl does the conversion by default to maintain
consistency. However, a server may require a POST to remain a POST after such
a redirection. This option is meaningful only when using -L, --location
(Added in 7.19.1)
References:
Following redirects with curl
HTTP RFC
I need to add -b to my script to enable the cookies. CURL by default doesn't use them and this issue caused to session ID change. Therefore no data transferred.
curl -b -L -i -X POST \
-d 'id=105' \
-d 'json={"orderBy":0,"maxResults":50}' http://mysite.com/ctlClient/
Now its working

why can't get this page in linux with wget/telnet?

this URL www.jinfuwu.com can be access in windows browser,windows telnet,
but in my ubuntu server, i can't get this page:
telnet (ubuntu):
root#ubuntu:~# telnet www.jinfuwu.com 80
Trying 121.199.111.176...
Connected to www.jinfuwu.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.jinfuwu.com
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Sun, 05 Dec 2010 01:34:33 GMT
Accept-Ranges: bytes
ETag: "f671fd911c94cb1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
X-UA-Compatible: IE=EmulateIE7
Date: Sun, 05 Dec 2010 10:03:21 GMT
Content-Length: 1214Connection closed by foreign host.
wget (ubuntu):
root#ubuntu:~# wget http://www.jinfuwu.com
--18:10:29-- http://www.jinfuwu.com/
=> `index.html.2'
Resolving www.jinfuwu.com... 121.199.111.176
Connecting to www.jinfuwu.com|121.199.111.176|:80... connected.
HTTP request sent, awaiting response...
Read error (Connection reset by peer) in headers.
Retrying.
....
but in my windows ,i using telnet command, i can get the page
telnet (windows7):
run:
telnet www.jinfuwu.com 80
paste:
GET / HTTP/1.1
Host: www.jinfuwu.com
and press doubles Enter,i can see the page HTML code.
google it:
site:jinfuwu.com
google can access this site
can you tell me why?
btw: also www.joytg.com,same question
thanks a lot :)
Did some further digging for you and found the root cause is due to misconfigured routers. You can read about it all here.
The workaround that article mentions is to:
echo 0 > /proc/sys/net/ipv4/tcp_default_win_scale
However, this file has changed and on newer setups you need to instead:
echo 0 > /proc/sys/net/ipv4/tcp_window_scaling
You will need to be root when running that though.
$ wget http://www.jinfuwu.com
--2010-12-05 12:58:39-- http://www.jinfuwu.com/
Resolving www.jinfuwu.com... 121.199.111.176
Connecting to www.jinfuwu.com|121.199.111.176|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12145 (12K) [text/html]
Saving to: `index.html'
100%[====================================================>] 12,145 5.19K/s in 2.3s
2010-12-05 12:58:43 (5.19 KB/s) - `index.html' saved [12145/12145]
FWIW, I can get the page just fine using wget or curl from MacPorts.

Resources