I have a large repo of mp3s on my LAMP server (I think it's a Debian VPS now) and currently I use a crude flash based mp3 player that "streams" the mp3s directly from my server. I am implementing an HTML5 player but I feel this is a similar to my flash integration (and this is slightly besides the point of this question)
This question is about how I should start molding my delivery to limit bandwidth - connection speed isn't entirely an issue (although should be reasonable) but my bandwidth costs are flying now and I need to take measures to limit stream quality (transcode down to 128)
Ideally I'd like to avoid RTMP so I can use my shiny new soundmanager2(html5) plugin for mobile etc, and as such I've been looking into this article here - http://www.metabrew.com/article/transcoding-http-mp3-streaming-proxy-in-bash which details how to create a proxy to on-the-fly transcode bitrate and serve new files.
I am fairly new to this arena and am open to all solutions! Thanks!
P.S I am open to 3rd party services - affordable ones at least :)
Possible Solutions (on SO)
reduce bandwidth streaming mp3s php
Related
I am using Azure Blob Storage to store a video. I would like a user to be able to scroll the video to any point in time and play it from there.
For short videos, there is no problem because the whole video loads and you can do that, but for larger videos, it does not seem to work out of the box. And in some sense it makes sense - files by default do not have the functionality to be downloadable from the middle. But all decent video streaming websites offer this functionality. I must be missing some video concepts, would appreciate, if someone linked me to some articles explaining how things like this are done. Bonus points if the solution is using Microsoft Azure.
Large video example (28 secs, 126MB):
https://www.w3schools.com/code/tryit.asp?filename=GP328W3SEY77
Small video example (10 secs, 1MB):
https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_video
Video streaming servers or cloud services are usually dedicated specialised servers and their functionality can be quite complex.
A video 'file' typically consists of one or more video and audio tracks in a 'container' like MP4. The container will have header information and pointers to the track info.
Simple HTTP streaming of an mp4 file is possible if your server supports range requests, i.e. downloading parts of the file at a time, and if the header information is at the start of the video file - in mp4 is it usually at the end by default but can be moved to the start.
More sophisticated streaming servers, including most/all of the popular commercial services, use a dedicated streaming protocol, typically ABR HLS or DASH these days. These provide chunked multiple different bit rate versions of the video and allow the client switch between bit rates for each chunk it downloads - see more info here: https://stackoverflow.com/a/42365034/334402
The thumbnails you see when you scroll along a video timeline are actually usually a separate track in the video file or stream. They are a set of images at timepoints and the entire set of thumbnails can be downloaded quickly at the start of playback to give a view of what a particular part of the video will look like if the user wants to jump to it. When the user actually jumps to that section of the video, the client requests from the server the chunks or section of the file corresponding to that thumbnail.
Azure CDN actually provides some nice functionality:
video starts to play instantly (browser doesn't need to wait for the video to fully load)
you can scroll back/forward in time.
(Obviously, this in addition to the standard CDN functionality of multiple PoP, caching, etc.)
The above CDN setup was tested on Standard Microsoft, but Verizon and Akamai seem to be offering similar functionality.
I'd like to either 1) Stream my camera+mic or 2) Stream my desktop+mic (or OBS) to many people (1 to many) watching from a web browser. Since I'd like to communicate with people in real time low latency is highly preferred (1 second or less would be great).
What's the standard way of doing this today? I have no idea what twitch uses but from playing with rtmp (using the nginx rtmp module) there seems to be a 4+second delay (or maybe I configured it poorly) and doesn't seem to have native browser support. People watching will use their desktop computers but some may be watching from their phones.
Am I to use webrtc somehow (does it support 1 to many? what software do I use)? Or HLS (which seems to want recordings, what config do I want for a live video)? What are my options?
This is an extremely difficult question. If you only need a hand full of viewers (around 10 or less), WebRTC the way to go. If there is a lot of viewers, and you can spend a lot of money, WebRTC is still a good choice, but the bandwidth cost can add up. If you don’t have a lot of money, but need a lot of viewers, you need cheeper bandwidth, this means using an HTTP CDN, and HLS or DASH which adds latency, but is cheeper to operate. If you have 100+ million dollars, you can build your own WebRTC cdn and have the best of both.
So to answer your question, there is no standard way. It depends on your circumstance, use cases and budget.
I have 2 web hosts from two different hosting company. One is for hosting my web page and the secondary where I upload the videos (mp4 format). At the moment I'm using http://www.longtailvideo.com/players/jw-flv-player/ it because I can use HTML5 and if the client doesn't supports HTML5 it falls back to regular FLV video player.
The videos I receive has .avi or .mpeg extensions. I'm using Miro Video Convertor to convert videos to .mp4 than I upload them to my secondary web host. From there I can easily access the mp4 formatted video via URL. After everything is finished I just copy and paste the URL link to my HTML document, something like this:
<video
src="http://66.55.XXX.XXX/university/students/video1.mp4"
width="640px"
height="480px"
id="vidi"
</video>
I already made my research about video streaming, but... I don't understand or.. am I doing it right? If I just copy paste the link it means I'm streaming the video from web host #1 to web host #2 ? Is it right?
Also, the videos has 1280 x 1024 HD quality and I know if Videos has a higher quality, the buffer, the load time last longer. This is why I re-size videos to 640 x 480 and also to be compatible with HTML5.
How many bandwidth am I using? And a client? If one of person (student) is viewing the video how many bandwidth is he using? I payed for a web host for unlimited storing, because I upload 10, 12 GB of data every week.
I'm very worried for the load or buffer time. Currently the web page is used for ~30, 40 people, but what if the whole year or university will be using the web page? What am I supposed to do?
Am I doing the streaming right? This is why I opted for 2 different web hosts, to have more bandwidth.
Sorry for long post and for my english.
Thank you !
If I just copy paste the link it means I'm streaming the video from web host #1 to web host #2 ? Is it right?
Firstly, it looks like the media file is being served up via plain HTTP, no logic. So I'd not call it "streaming" but rather "progressive download". (It's a marketing ploy by hosting companies--if it is video, it must be streaming, right? Ah...no.)
Secondly, no: the video will not go from 66.55.XXX.XXX to the web server that is hosting your website. Rather it will go straight from to 66.55.XXX.XXX to the web browser.
Also, the videos has 1280 x 1024 HD quality and I know if Videos has a higher quality, the buffer, the load time last longer. This is why I re-size videos to 640 x 480 and also to be compatible with HTML5.
Resizing the video to reduce the bandwidth means you'll need to transcode the video to a smaller size. Setting the width and height attributres on the <video> tag will only change the displayed size. These two attributes have no impact on the bit rate coming from the server, and hence no impact on the buffer or load times.
How many bandwidth am I using? And a client? If one of person (student) is viewing the video how many bandwidth is he using?
There are two terms you need to be aware of here:
Traffic: The number of bytes sent (volume)
Bandwidth: The rate at which the bytes are sent (rate)
It's an important distinction. Again, many hosting companies mix up these concepts in the name of marketing. Be careful.
How does this impact your situation? Think about it like this: If you have a 1GB video on hosted, and it is viewed 10 times, that is 10GB of traffic. The bandwidth depends on the server sending the file, the client's network connection speed, and a network in between. As a rule of thumb you don't need to worry about this, except for two points:
The bit rate of the video needs to be smaller than the bit rate of the network connection between client and server. If not you'll have buffering during video play back.
Your hosting company may (probably!) restricts how many concurrent users can view a video at once. if 100 people download the video at once, each at an average of 2mbps, that is 200mpbs of bandwidth!
Unless you have more than 10-100 viewers a day, I would not worry about bandwidth too much.
A simple way to calculate the bandwidth of your video is:
bit rate = (bytes * 8) / (time in seconds)
Silly example: 800s long 1GB video (rounded for clarity)
bit rate = (1,000,000,000 bytes * 8 bits per byte) / (800 seconds)
bit rate = (8,000,000,000 bits) / (800 seconds)
bit rate = 10,000,000 bits per second
bit rate = 10,000 kilobits per second
bit rate = 10 megabits per second
I payed for a web host for unlimited storing, because I upload 10, 12 GB of data every week.
"Unlimited storage", ah maybe. If you upload enough data at some point someone is going to take notice and tell you that your ToC have been violated. The hosting market is evil that way.
I'm very worried for the load or buffer time. Currently the web page is used for ~30, 40 people, but what if the whole year or university will be using the web page? What am I supposed to do?
At once? Or per day? Is that people watching one video? Or videos viewed? If you have 10-25 concurrent then you probably should be at least mildly worried about the hosting company.
Frankly, the web/video hosting market is full of bait-n-switch tactics, opaque pricing, gangster ToCs, and convoluted marketing speak. You would probably be better served by using a service like Amazon's AWS. Specifically, use Amazon S3 to store your videos and use Amazon CloudFront to stream the videos to the clients. All of this has three distinct advantages to shady hosting companies:
Fair and transparent pricing (including an online calculator)
Pay for what you use (and not more)
Effectively unlimited storage and bandwidth (AWS has terabits of bandwidth and exabytes of storage)
I highly recommend AWS for small but non-trivial projects like you seem to have.
And go full size HD! It is a much more compelling experience for your viewers.
Good luck!
I'm attempting to create a simple video conversion app which accepts any video format and then formats it into the desired format "automagically" all while attempting to keep aspect ratio and sound quality consistent. I do a ton of video converting and it would be handy for me to just shoot it off to a server to do all the converting while I work on other projects.
After some research it looks like FFMPEG is my best bet ...
Anyone have any thoughts? Is there a open source project that has already done this / code that I might have missed or looked over?
I donot recommend hosting such a service yourself. I think you should be able to find 'off-the-shelf' cloud provider with 'instant' transcoding machines on-demand.
You are not looking for application hosting, such as GAE, you are looking for P aaS (Platform as a Service), such as Azure, AWS EC2; Here is a good comparison:
https://softwareengineering.stackexchange.com/questions/64727/windows-azure-vs-amazon-ec2-vs-google-app-engine (note the answer by a GAE developer).
Basically, PaaS should be able to deliver you computing units (where you are specifically expected to utilize the full CPU (etc) potential of the instance - after that, you just terminate the instance; you pay by the hour, and you scale up as required; You can equally good run 1 small instance for a month, or run 60 large instances for a few hours and work out the financial equation).
I do remember seeing pre-existing cloud images precisely for doing video transcoding - don't remember where exactly, but look around at rightscale.com (which works with Rackspace and EC2 IIRC, perhaps more these days).
ffmpeg is easily the best choice for this. I use ffmpeg in my Python DLNA Digital Media Server to convert video files of any kind to the type expected by my television.
I've had a play with VLC's dummy interface, and mencoder, and neither came close to ffmpeg's usability.
Sorry if the question is confused, as I'm confused myself. I'm working around these requirements:
I'm building a public website where I need to display video.
I need to control what the player looks like
I'm the sole publisher of the video, meaning it can't be on YouTube for example
I need as much protection as possible in terms of protecting the content from being downloaded
So, I've read around StackOverflow and the web, and found lots of suggestions, like numerous flash players, Streaming servers, DRM protocols, services like Panda etc etc.
The problem is I don't understand how everything fits together.
For example, what makes my video content secure?
Is it the player on the client? is it the server that hosts the content? is it the streaming process? who hosts the streaming servers and what difference does this make?
Bearing in mind this is otherwise a very simple site, and is not a business venture.
if you were working around my requirements, what would you do? Could you explain step by step at a high level?
EDIT:
Just based on a couple of answers, I'm not saying no one can ever download my content. And I realize this kind of thing is expensive.
I'm just asking, if you had my requirements, what would you do? And could you explain it to me so i understand?
thanks again
Edit:
Thanks again for all the feedback, I can't vote anyone up as I'm a new user, but your answers have been very helpful.
The one thing I will say, is that my only request was to attempt security, that is 'make it difficult' for most users...that is common in software security.
Some of the suggestions have been just to not even try.
My question was really based around the fact that I know nothing about video deployment on the web, apart form the basic embedded swf flv combo.
Anyway, your info has been very useful though. I'll try a simple "real" streaming service (as opposed to HTTP streaming).
Any other recommendations would be awesome
cheers
"For example, what makes my video content secure? " Nothing.
"Is it the player on the client?" Neither. Anyone can write a client and retain the video content. Remember this. Anyone can write a client. This client can absorb and save your video. Nothing can stop this. Nothing.
"is it the server that hosts the content?" No. Server is only one piece of security. You have to secure the protocol. And the client. And anyone can write a client and retain the video content.
"is it the streaming process?" No. Protocol is only one piece of security. You have to secure the server, the protocol and the client. And anyone can write a client and retain the video content.
"who hosts the streaming servers and what difference does this make?" You host the streaming video servers. Otherwise, you might as well use YouTube.
Edit
"The problem is I don't understand how everything fits together."
"For example, what makes my video content secure?"
These are unrelated. You keep mentioning security, AND not knowing how "everything" fits together.
Here's a suggestion: stop mentioning security -- edit your question to eliminate all references to security and see if you get more useful answers.
Many companies sell streaming media servers. You put HTML in your page that references the streaming media site.
Example. Apple sells Quicktime media server. Read http://developer.apple.com/documentation/QuickTime/Conceptual/QTScripting_HTML/QTScripting_HTML_Document/chapter_1000_section_1.html for lots of information on how to present video from quicktime.
Before you go too far worrying about setting up these secure streaming protocol client server whatevers, make sure you weigh up the cost of your time getting this going, versus the cost of someone downloading your video.
Just to be clear: if your server is sending to a client, then they can copy (download) it. There's no way around it.
Response to your comment:
What I'd probably try doing if you wanted to try to avoid users downloading the files is this (I'll assume you're using FLV files, since they're the de facto standard on the web these days):
Put the FLV files in a non web-accessible directory.
Have a player.swf file request the file via a script on your site, eg: video.php?file=myVideo.flv
The video.php can then perform whatever security checks you'd like: for example, require logins, check the referrer, etc.
If the security checks are ok, then pass through the appropriate video file. If not, then perhaps have a short back-up video which is an ad for your site or something, saying "to watch this video, please come to mysite.com!"
Mostly video streaming sites like Hulu achieve a kind of poor-man's security by using RTMP to transfer the video data. You would need special server software to serve video via RTMP, for example Adobe Flash Media Server or WebORB.
RTMP is a proprietary protocol, so this is a case of security through obscurity; it's non-trivial to download a copy of the video (you can't just grab the file from a URL), but there are programs out there that are capable intercepting the stream and keeping a copy.
2.I need to control what the player looks like
Download and customise a free player like OSFLV.
4.I need as much protection as possible in terms of protecting the content from being downloaded
Forget it.
DRM for FLV exists, but you'll have to pay Adobe a load of money for Flash Media Server and Flash Media Rights Management Server, you'll lose client compatibility and ease of deployment, and in the end it's still breakable. Big old waste of time.
Accept that some people will download your videos, and put a big watermark on them so at least when they do you're getting free advertising.