We have a vendor who sends us photos that are hosted on Azure Edge. These photos are available and I can download them, but if we do a CURL request we get a 404 roughly 4 out of 5 times. If we do a HEAD request to get the filesize, we get a 404 about 7 out of 10 times. On our production server, we get a 404 100% of the time. Any idea how we might work around this or if there's another way to check these files without the vendor having to fix their issue?
Sample file:
curl -I http://tdrvehicles2.azureedge.net/photos/202008/1419/1850/f253435f-86b1-4cc4-b95c-7756addddad4.jpg
HTTP/1.1 404 Not Found
Pragma: no-cache
Content-Length: 0
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Cache-Control: max-age=31536000
Expires: Thu, 19 Aug 2021 14:12:54 GMT
Date: Wed, 19 Aug 2020 14:12:54 GMT
Connection: keep-alive```
> 14.195.188.230 - - [18/Mar/2017:16:43:11 +0000] "(004026579154BP05000004026579154111213V0000.0000N00000.0000E000.0000000000.0010000000L0000021C)" 400 173 "-" "-" "-"
This is the error message that i received
HTTP/1.1 400 Bad Request
Server: nginx/1.10.1
Date: Sun, 19 Mar 2017 02:19:35 GMT
Content-Type: text/html
Content-Length: 173
Connection: close
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/1.10.1</center>
</body>
</html>
This is the log that i can see in my nginx access log, I need this data in my node server.
(004026579154BP05000004026579154111213V0000.0000N00000.0000E000.0000000000.0010000000L0000021C)
1) I am using elastic Beanstalk, i don't know how can i pass the above value to my node server module? If its possible to get that value as http or https request inside my node express module.
2) If i have to run a net server, than in which port should i listen for tcp, and how nginx will know about that port, for http server port, i use process.env.port
I have this situation that utterly baffles me:
The setup
a newly setup server (centos 7, apache, mysql, nothing fancy) that hosts a simple php app that I need to interact with from my main app on another server. This service is setup to run on service-name.domain.tld while the main app is on domain.tld (just mentioning in case it make any difference).
The problem
For some reason when I try to access the service app from the main server I get an endless loop of 302 redirects.
If I do a curl -D - http://service-name.domain.tld from the main server I get:
HTTP/1.1 302 Found
Date: Wed, 03 Aug 2016 12:30:26 GMT
Server: Apache
Location: http://service-name.domain.tld
Content-Length: 218
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
</body></html>
If I do the same command from my computer, the result is
HTTP/1.1 200 OK
Date: Wed, 03 Aug 2016 12:31:39 GMT
Server: Apache/2.4.6 (CentOS) PHP/5.6.24
X-Powered-By: PHP/5.6.24
Cache-Control: no-cache
Content-Length: 30
Content-Type: text/html; charset=UTF-8
Welcome.
As it should be. Same happens for any request I make (even static files). Also I should specify same situation happens for wget and get_file_contents in php
I'm really lost right now, not knowing where to go from here. So any kind of direction is greatly appreciated.
I use Privoxy or Proxomitron to inject custom Javascript tags into websites which then load scripts from a local python server (on localhost:8888):
... <script type="text/javascript"
src="http://localhost:8888/tweakscript.js"
></script></body>
Some of these script tags are huge third party javascript libraries which are also stored on my computer. They never change but are reloaded each time. I want to cache them.
i tried these headers, without success:
HTTP/1.1 200 OK
Content-type: text/javascript; charset=UTF-8
Vary: Accept-Encoding
Date: Sun, 04 Oct 2015 03:24:14 GMT # the current date
Last-Modified: Fri, 02 Oct 2015 06:34:40 GMT # never changes
Expires: Fri, 01 Apr 2016 03:24:14 GMT # one month in future
Cache-Control: public, max-age=15552000 # cache for one year
Access-Control-Allow-Origin: * # Content Security Policy
Access-Control-Allow-Methods: GET, POST, OPTIONS
Connection: close
(javascript code here)
How can i make the webbrowser cache these files?
OK, i found the solution:
When requesting files, the browser adds an If-Modified-Since request header, eg:
If-Modified-Since: Tue, 28 Jul 2015 22:48:42 GMT
If the server then sends as response ...
HTTP/1.1 304 Not Modified
... then the browser will load the file from cache.
I'm attempting to embed an HTML5 audio element pointing to MP3 or OGG data served by a PHP file . When I view the page in Safari, the controls appear, but the UI says "Live Broadcast." When I click play, the audio starts as expected. Once it ends, however, I can't start it playing again by clicking play. Even using the JS API on the audio element and setting currentTime to 0 fails with an index error exception.
I suspected the headers from the PHP script were the problem, particularly missing a content length. But that's not the case. The response headers include a proper Content- Length to indicate the audio has finite size. Furthermore, everything works as expected in Firefox 3.5+. I can click play on the audio element multiple times to hear the sound replay.
If I remove the PHP script from the equation and serve up a static copy of the MP3 file, everything works fine in Safari.
Does this mean Safari is treating audio src URLs with query parameters differently than URLs that don't have them? Anyone have any luck getting this to work?
My simple example page is:
<!DOCTYPE html>
<html>
<head></head>
<body>
<audio controls autobuffer>
<source src="say.php?text=this%20is%20a%20test&format=.ogg" />
<source src="say.php?text=this%20is%20a%20test&format=.mp3" />
</audio>
</body>
</html>
HTTP Headers from PHP script:
HTTP/1.x 200 OK
Date: Sun, 03 Jan 2010 15:39:34 GMT
Server: Apache
X-Powered-By: PHP/5.2.10
Content-Length: 8993
Keep-Alive: timeout=2, max=98
Connection: Keep-Alive
Content-Type: audio/mpeg
HTTP Headers from direct file access:
HTTP/1.x 200 OK
Date: Sun, 03 Jan 2010 20:06:59 GMT
Server: Apache
Last-Modified: Sun, 03 Jan 2010 03:20:02 GMT
Etag: "a404b-c3f-47c3a14937c80"
Accept-Ranges: bytes
Content-Length: 8993
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Type: audio/mpeg
I tried hard-coding the Accept-Ranges header into the script too, but no luck.
Can you post the headers sent by the server both with and without the PHP script? I'm wondering if the PHP script is sending a different Content-Type than serving the files normally.
It would also be a good idea to specify the type attribute on the source elements, so the browser does not have to download both clips to determine if it can play them.
I cannot reproduce your problem. I have tried to recreate the problem in Safari 4.0.4, and the current WebKit nightly, with the following test page. I am simply using mod_rewrite to dispatch to different formats based on a parameter instead of PHP, but I don't think that should make a difference, unless somehow PHP is modifying the file.
<!DOCTYPE html>
<title>Auido test</title>
<audio controls autobuffer>
<source src="gnossienne-no-1?foo=bar&format=.mp4">
<source src="gnossienne-no-1?foo=bar&format=.ogg">
</audio>
Can you try my example out and let me know if it works for you?
edit Ah. After poking around at it a bit more, it appears that the problem is due to an odd way that the <audio> element in Safari behaves in attempting to determine the size of the content.
Here's an excerpt from a packet capture of Safari upon encountering an <audio> element pointing to a file served directly from Apache. As you can see, it first tries to fetch the first two bytes of the media, presumably so it can get a Content-Length back, and possibly other headers. It then tries to fetch the whole thing. Then, inexplicably, it tries to fetch the first two bytes again, but passes appropriate caching headers to get a "304 Not Modified" response. And finally, still inexplicably, it fetches the last 3440 bytes of the file all over again. It does all of these in separate TCP connections, which adds considerable overhead, in addition to the overhead of fetching the data a couple of times.
GET /stackoverflow/audio-test/say-noid3?foo=bar&format=.mp3 HTTP/1.1
Host: ephemera.continuation.org
Range: bytes=0-1
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
Accept-Encoding: identity
Cookie: [redacted]
HTTP/1.1 206 Partial Content
Date: Tue, 05 Jan 2010 02:12:48 GMT
Server: Apache
Last-Modified: Tue, 05 Jan 2010 02:02:08 GMT
ETag: "b2a80ad-11f6-47c6139aaa800"
Accept-Ranges: bytes
Content-Length: 2
Content-Range: bytes 0-1/4598
Connection: close
Content-Type: audio/mpeg
# 2 bytes of data
GET /stackoverflow/audio-test/say-noid3?foo=bar&format=.mp3 HTTP/1.1
Host: ephemera.continuation.org
Range: bytes=0-4597
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
Accept-Encoding: identity
Cookie: [redacted]
HTTP/1.1 206 Partial Content
Date: Tue, 05 Jan 2010 02:12:48 GMT
Server: Apache
Last-Modified: Tue, 05 Jan 2010 02:02:08 GMT
ETag: "b2a80ad-11f6-47c6139aaa800"
Accept-Ranges: bytes
Content-Length: 4598
Content-Range: bytes 0-4597/4598
Connection: close
Content-Type: audio/mpeg
# 4598 bytes of data
GET /stackoverflow/audio-test/say-noid3?foo=bar&format=.mp3 HTTP/1.1
Host: ephemera.continuation.org
Range: bytes=0-1
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
Accept-Encoding: identity
Cookie: [redacted]
If-None-Match: "b2a80ad-11f6-47c6139aaa800"
If-Modified-Since: Tue, 05 Jan 2010 02:02:08 GMT
HTTP/1.1 304 Not Modified
Date: Tue, 05 Jan 2010 02:12:49 GMT
Server: Apache
Connection: close
ETag: "b2a80ad-11f6-47c6139aaa800"
# no data
GET /stackoverflow/audio-test/say-noid3?foo=bar&format=.mp3 HTTP/1.1
Host: ephemera.continuation.org
Range: bytes=1158-4597
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
Accept-Encoding: identity
Cookie: [redacted]
HTTP/1.1 206 Partial Content
Date: Tue, 05 Jan 2010 02:12:49 GMT
Server: Apache
Last-Modified: Tue, 05 Jan 2010 02:02:08 GMT
ETag: "b2a80ad-11f6-47c6139aaa800"
Accept-Ranges: bytes
Content-Length: 3440
Content-Range: bytes 1158-4597/4598
Connection: close
Content-Type: audio/mpeg
# 3440 bytes of data
Anyhow, on to how it deals with the output of your PHP script. Here, Safari again tries to download the first two bytes, but your script ignores the Range request and returns the whole thing. Apparently, WebKit doesn't like that, and so it tries again, without the Range request. Again, your script sends the full contents. Safari now tries once more, adding an Icy-Metadata header, which indicates it thinks that it's downloading a stream and wants streaming metadata to be sent. It finally accepts the output of that, and the <audio> element can play.
GET /say.php?text=this%20is%20a%20test&format=.mp3 HTTP/1.1
Host: tts.mindtrove.info
Range: bytes=0-1
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
Accept-Encoding: identity
HTTP/1.1 200 OK
Date: Tue, 05 Jan 2010 02:14:28 GMT
Server: Apache
X-Powered-By: PHP/5.2.10
Content-Length: 4598
Connection: close
Content-Type: audio/mpeg
# 4598 bytes of data
GET /say.php?text=this%20is%20a%20test&format=.mp3 HTTP/1.1
Host: tts.mindtrove.info
Connection: close
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Accept: */*
HTTP/1.1 200 OK
Date: Tue, 05 Jan 2010 02:14:28 GMT
Server: Apache
X-Powered-By: PHP/5.2.10
Content-Length: 4598
Connection: close
Content-Type: audio/mpeg
# 4598 bytes of data
GET /say.php?text=this%20is%20a%20test&format=.mp3 HTTP/1.1
Host: tts.mindtrove.info
Accept: */*
User-Agent: Apple Mac OS X v10.6.2 CoreMedia v1.0.0.10C540
Icy-Metadata: 1
Connection: close
HTTP/1.1 200 OK
Date: Tue, 05 Jan 2010 02:14:28 GMT
Server: Apache
X-Powered-By: PHP/5.2.10
Content-Length: 4598
Connection: close
Content-Type: audio/mpeg
# 4598 bytes of data
In summary, it appears that Safari (or more accurately, QuickTime, which Safari uses to handle all media and media downloading) has a completely braindamaged approach to downloading media. Something in the way you send your data back, probably the fact that you don't respond to Range requests, makes it think that you are sending streaming media, causing it to download the content repeatedly (though even when confronted with a server that does respond to a Range request, it still does several more requests than it really needs to).
My advice would be to try to respond appropriately to Range requests; when serving up media, browsers will likely use them to try to minimize bandwidth, by only buffering as much as they need to be able to play through (although have the autobuffer attribute which indicates that you would like them to buffer the whole thing, browsers may ignore that). I would recommend using X-Sendfile to let Apache deal with serving the file, caching, and range requests, but you appear to be on Dreamhost, which doesn't have mod_xsendfile installed, so you're going to have to roll your own Range handling.
Old topic but still valid in 2019. We finally found a solution... Below PHP script sample will consider Safari's "Range" header request.
$bytes_total = strlen($data);
if (isset($_SERVER['HTTP_RANGE'])) {
$byte_range = explode('-',trim(str_ireplace('bytes=','',$_SERVER['HTTP_RANGE'])));
$byte_from = $byte_range[0];
$byte_to = ($byte_range[1]+1);
$data = substr($data,$byte_from,$byte_to);
header('HTTP/1.1 206 Partial Content');
}
else {
$byte_from = 0;
$byte_to = $bytes_total;
}
$length = strlen($data);
header('Content-type: '.$content_type);
header('Accept-Ranges: bytes');
header('Content-length: ' . $length);
header('Content-Range: bytes '.$byte_from.'-'.$byte_to.'/'.$bytes_total);
header('Content-Transfer-Encoding: binary');
print($data);
For the record, while both Pochang and Chris are correct that you need the Content-Range header to fix this problem in Safari, Chrome requires one extra header that must be included for setting currentTime to work properly:
header( 'Accept-Ranges: bytes');
Note that you don't actually have to respond correctly to the request's Range header, you just have to include this in the response.
I got the same problem.
The key point is the Content-Range header.
Everything works fine after I add it to the mp3-output php.
Pochang is correct. Including a Content-Range header in the PHP response will cause Safari to behave properly. It also allows seeking (media.currentTime = 0;) without the dreaded INDEX_SIZE_ERR in Safari, though not in Chrome.
The PHP code for the header is:
$len = strlen( $data );
$shortlen = $len - 1;
header( 'Content-Range: bytes 0-'.$shortlen.'/'.$len);
And in 2020 it is still actual question.
Simply adding Content-Range header does not work.
Bellow is my implementation(based on some answers here).
$content_length = strlen($media_total);
$total_bytes = $content_length;
$content_length_1 = $content_length - 1;
if (isset($_SERVER['HTTP_RANGE'])) {
$byte_range = explode('-',trim(str_ireplace('bytes=','',$_SERVER['HTTP_RANGE'])));
$byte_from = $byte_range[0];
$byte_to = intval($byte_range[1]);
$byte_to = $byte_to == 0 ? $content_length_1 : $byte_to;
$media_total = substr($media_total,$byte_from,$byte_to);
$content_length = strlen($media_total);
header('HTTP/1.1 206 Partial Content');
}
else {
$byte_from = 0;
$byte_to = $content_length_1;
}
$content_range = 'bytes '.$byte_from.'-' . $byte_to . '/' . $total_bytes;
header('Accept-Ranges: bytes');
header("Content-Range: ".$content_range);
header("Content-type: ".$type);
header("Content-length: ".$content_length);
header('Content-Transfer-Encoding: binary');
echo $media_total;
exit;
Original question here: Timing problem for generated audio in some browsers