We are running into an issue that seems unique to AVPlayer. We've built a new architecture for our HLS streams which makes use of relative URLs. We can play these channels on a number of other players just fine, but when trying to build using AVPlayer, the channel gets 400 errors requesting either child manifests/segments with relative URLs.
Using curl, we are able to get a 200 success by getting to a url like: something.com/segmentfolder/../../abcd1234/videosegment_01.mp4
Curl is smart enough to get rid of the ../../ and create a valid path so the actual request (which can be seen using curl -v) looks something like: something.com/abcd1234/videosegment_01.mp4
Great. But AVPlayer doesn't do this. So it makes the request as is, which leads to a 400 error and it can't download the segment.
We were able to simulate this problem with Swift playground fairly easily with this code:
let hlsurl = URL(string: "website.com/segmentfolder/../../abc1234/videosegment.mp4")!
print(hlsurl)
print(hlsurl.standardized)
The first print shows the URL as is. Trying to GET that URL leads to a 400. The second print properly handles it by adding .standardized to the URL. This leads to a 200. The problem is, this only works for the top level/initial manifest, it doesn't work for all the child manifests and segments.
let url = URL(string: "website.com/segmentfolder/../../abc1234/videosegment.mp4")!
let task = URLSession.shared.dataTask(with: url.standardized) {(data, response, error) in
guard let data = data else { return }
print(String(data: data, encoding: .utf8)!)
}
So question, does AVPlayer support relative URLs? If so, why can't it handle the ../../ in the URL path like other players and curl can? Is there a special way to get it to trigger standardizing ALL URL requests?
Any help would be appreciated.
Related
this is kinda weird (node repl v8.15.0):
let URL = require('url').URL
let {pathname} = new URL('https://my.domain.com/e30%3D/with%3F')
console.log(pathname) // logs '/e30%3D/with%3F' <-- this looks right
then in my CloudFlare worker (using the service-worker-mock):
let URL = require('url').URL
let {pathname} = new URL('https://my.domain.com/e30%3D/with%3F')
console.log(pathname) // logs '/e30=/abc%21%3Fdef' <-- `=` unquoted in path?
I'm guessing it's probably a different version of URL? Anyway I can control that?
Your expectation that the two URL implementations parse the same way is actually correct, if you open the inspector and run the second code it should encode the way you expect. Unfortunately, as Harris points out, the Workers URL implementation is buggy and difficult to fix. I'd recommend using some sort of URL polyfill in your code to encode URLs properly.
As the title says, I'm trying to intercept script requests from the user's page, make a GET request to the script url from the background, add a bit of functionality and send it back to the user.
A few caveats:
I don't want to do this with every script request
I still have to guarantee that the script tags are executed in the original order
So far I came with two solutions, none of which work properly. The basic code:
chrome.webRequest.onBeforeRequest.addListener(
function handleRequest(request) {
// First I make the get request for the script myself SYNCHRONOUSLY,
// because the webRequest API cannot handle async.
const syncRequest = new XMLHttpRequest();
syncRequest.open('GET', request.url, false);
syncRequest.send(null);
const code = syncRequest.responseText;
},
{ urls: ['<all_urls>'] },
['blocking'],
);
Now once we have the code, there are two approaches that I've tried to insert it back into the page.
I send the code through a port to a content script, that will add it to the page inside a <script></script> tag. Along with the code, I also send an index to keep sure the scripts are inserted back into the page in the correct order. This works fine for my dummy website, but it breaks on bigger apps, like youtube, where it fails to load the image of most videos. Any tips on why this happens?
I return a redirect to a data url:
if (condition) return { cancel: false }
else return { redirectUrl: 'data:application/javascript; charset=utf-8,'.concat(alteredCode) };
This second options breaks the code formatting, sometimes removing the space, sometimes cutting it short. I'm not sure on the reason behind this behavior, it might have something to do with data url spec.
I'm stuck. I've researched pretty much every related answer on this website and couldn't find anything. Any help or information is greatly appreciated!
Thanks for your time!!!
I have a simple utility that i use to size image on the fly via url params.
Having some troubles with the ruby image libraries (cmyk to rvb is, how to say… "unavailable"), i gave it a shot via nodejs, which solved the issue.
Basically, if the image does not exists, node or ruby transforms it. Otherwise when the image has already been requested/transformed, the ruby or node processes aren't touched, the image is returned statically
The ruby works perfectly, a bit slow if lot of transforms are requested at once, but very stable, it always go through whatever the amount (i see the images arriving one the page one after another)
With node, it works also perfectly, but when a large amount of images are requested, for a single page load, the first images is transformed, then all the others requests returns the very same image (the last transformed one). If I refresh the page, the first images (already transformed) is returned right away, the second one is returned correctly transformed, but then all the other images returned are the same as the one just newly transformed. and it goes on the same for every refresh. not optimal , basically the resquests are "merged" at some point and all return the same image. for reason i don't understand
(When using 'large amount', i mean more than 1)
The ruby version :
get "/:commands/*" do |commands,remote_path|
path = "./public/#{commands}/#{remote_path}"
root_domain = request.host.split(/\./).last(2).join(".")
url = "https://storage.googleapis.com/thebucket/store/#{remote_path}"
img = Dragonfly.app.fetch_url(url)
resized_img = img.thumb(commands).to_response(env)
return resized_img
end
The node js version :
app.get('/:transform/:id', function(req,res,next){
parser.parse(req.params,function(resized_img){
// the transform are done via lovell/sharp
// parser.parse parse the params, write the file,
// return the file path
// then :
fs.readFileSync(resized_img, function(error,data){
res.write(data)
res.end()
})
})
})
Feels like I'm missing here a crucial point in node. I expected the same behaviour with node and ruby, but obviously the same pattern transposed in the node area just does not work as expected. Node is not waiting for a request to process, rather processes those somehow in an order that is not clear to me
I also understand that i'm not putting the right words to describe the issue, hoping that it might speak to some experienced users, let them provide clarifiactions to get a better understanding of what happens behind the node scenes
I'm developing a web crawler in nodejs. I've created a unique list of the urls in the website crawle body. But some of them have extensions like jpg,mp3, mpeg ... I want to avoid crawling those who have extensions. Is there any simple way to do that?
Two options stick out.
1) Use path to check every URL
As stated in comments, you can use path.extname to check for a file extension. Thus, this:
var test = "http://example.com/images/banner.jpg"
path.extname(test); // '.jpg'
This would work, but this feels like you'll wind up having to create a list of file types you can crawl or you must avoid. That's work.
Side note -- be careful using path. Typically, url is your best tool for parsing links because path is aimed at files/directories, not urls. On some systems (Windows), using path to manipulate a url can result in drama because of the slashes involved. Fair warning!
2) Get the HEAD for each link & see if content-type is set to text/html
You may have reasons to avoid making more network calls. If so, this isn't an option. But if it is OK to make additional calls, you could grab the HEAD for each link and check the MIME type stored in content-type.
Something like this:
var headersOptions = {
method: "HEAD",
host: "http://example.com",
path: "/articles/content.html"
};
var req = http.request(headersOptions, function (res) {
// you will probably need to also do things like check
// HTTP status codes so you handle 404s, 301s, and so on
if (res.headers['content-type'].indexOf("text/html") > -1) {
// do something like queue the link up to be crawled
// or parse the link or put it in a database or whatever
}
});
req.end();
One benefit is that you only grab the HEAD, so even if the file is a gigantic video or something, it won't clog things up. You get the HEAD, see the content-type is a video or whatever, then move along because you aren't interested in that type.
Second, you don't have to keep track of file names because you're using a standard MIME type to differentiate html from other data formats.
I had this working on kohana 2, but in kohana 3 it doesn't.
To serve an image stored as BLOB in a database, I did the following:
1- A controller to which I request what image do I want. I connects to the database, using a model of course, and serve the image using a view.
$prod = ORM::factory('product',$idx);
$img = new View('image');
$img->pic = $prod->getImage();
2-The model has a little trick to get this working:
public function getImage()
{
return imagecreatefromstring($this->image);
}
image is the blob column where I store the picture I want to serve.
3- In the view:
I set the content-type header and then serve the image
header('content-type: image/png; charset=UTF-8');
imagepng($pic);
This worked in Kohana 2, but in KO3 it doesn't,
I'm trying to use $response->send_file(), but I'm getting lost
First, you should never send headers with header() unless you're hacking the fw. Ko3.1 nicely separates Request from Response and the latter is the one responsible for response headers / everything else (both of them are written pretty much following the RFC 2616).
Secondly, there is absolutelly no need for a view file in this case, Response::$_body is what the current response object returns.
Response::send_file() returns the response as download, I suppose that's not what you're trying to accomplish?
So, you need something like this (modify to your own needs):
public function action_image($id)
{
$image = ORM::factory('product', $id);
if ( ! $image->loaded()) // ... 404 ?
$this->response
->headers('Content-Type','image/png')
->body($image->image)
->check_cache(NULL, $this->request); // 304 ?
}