I'm developing a video chat-like application using Flash RTMFP and Stratus. So far, I'm having good success. I can build from source, tweak settings, and get video and audio in both directions.
There's one glaring problem I haven't been able to solve, however -- when using a client on a Linux machine, the video received by the other end looks very poor. It's blocky and pixellated, almost as if it's rendering 160x120 in a much larger frame. When sending from a Mac (my other dev machine), the video looks quite good.
I've tried modifying all the settings I can think of -- frame rate, "quality", size, audio settings -- with no discernible improvement. I've tried running it as a local file and from a remote server. The network where I'm working is extremely fast, so that shouldn't be an issue.
Is there anything else I can try? Any suggestions or ideas are greatly appreciated.
Many thanks!
Bad camera or bad camera driver?
Stratus does not change video encoding, it simply is another variation of the RTMFP protocol for transferring exactly the same compressed stream.
One way you can check whether Stratus indeed plays any role in this is to try to stream the same stuff through Adobe Flash Media Server, the development version is free from adobe.com.
I have done Stratus applications, and have not experienced any degradation of video quality compared to Flash Media Server solution. In fact when the camera quality is set to 100, you won't notice the difference between raw camera video and compressed stream when using loopback mode. Apart from possibly limited framerate, if you specify bandwidth (the three are intimately related - bandwidth, framerate, quality, as per documentation of Camera.setQuality or Camera.setMode)
Related
My primary intention is to setup a VoIP session between 2 users A & B; Here the raw audio / video media bytes are fetched from A's browser are played in B's browser and vice versa.
The reason is that, when the user C & D are added into this call, we need not have to create a P2P mesh network which limits the performance.
Tried recording media with getUserMedia() and playback, but it is not real time. It also gives a bad user experience. (However, haven't experimented yet with videos of small chunks as 200 ms)
Is there any approach where I can get the raw bytes of the media and play it on other browser? Currently I have a server in between which can connect to both peers if required.
Any online examples or libraries are welcome.
Have already asked 2 questions in this regard with 100-100 bounties, but not much of use:
How to use libsrtp or similar library to decrypt/encrypt the WebRTC data stream?
How to integrate part of WebRTC as a static / dynamic library with the existing C++ code?
Related: How to stream, live video playing on my browser to browser of another user?
If i understand you well is you're looking on how to have more than two users on the session right? without using mesh topology
thats possible and configurable as well by means that some maybe active speaker or everyone is active speaker not only receiver whatever configuration you choose but to me it seems that you're asking for video conferencing
there are couple of tools for this the best one i might recommend is mediasoup its a SFU as selective fowarding unit mediasoup
I don't know if I understand correctly, but it is not likely that you will get raw video data and play it on the browser, it will just kill your bandwith and performance because the raw data is huge.
You need to use the compressed data ( media codec ex.H264 ) and you need a protocol to send and receive it. If you are looking for sub-second latency than webrtc is your best choice in here already. If you have a server in between, distribute your media through that server instead of Mesh. Check this out for webrtc network topologies:
https://antmedia.io/webrtc-servers/
I am trying to build a website and mobile app (iOS, Android) for the internet radio station.
Website users broadcast their music or radio and mobile users will just listen radio stations and chat with other listeners.
I searched a week and make a prototype with Wowza engine (using HLS and RTMP) and SHOUTcast server on Amazon EC2.
Using HLS has a delay with 5 seconds, but RTMP and SHOUTcast has 2 second delay.
With this result I think I should choose RTMP or SHOUTcast.
But I am not sure RTMP and SHOUTcast are the best protocol. :(
What protocol should I choose?
Do I need to provide a various protocol to cover all platform?
This is a very broad question. Let's start with the distribution protocol.
Streaming Protocol
HLS has the advantage of allowing users to get the stream in the bitrate that is best for their connection. Clients can scale up/down seamlessly without stopping playback. This is particularly important for video, but for audio even mobile clients are capable of playing 128kbit streams in most areas. If you intend to have a variety of bitrates available and want to change quality mid-stream, then HLS is a good protocol for you.
The downside of HLS is compatibility. iOS supports it, but that's about it. Android has HLS support but it is still buggy. (Maybe in another year or two once all the Android 3.0 folks are gone, this won't be as much of an issue.) JWPlayer has some hacks to make HLS work in Flash for desktop users.
I wouldn't bother with RTMP unless you're only concerned with Flash users.
Pure progressive streaming with HTTP is the route I almost always choose to go. Everything can play it. (Even my Palm Pilot's default media player from 12 years ago.) It's simple to implement and well understood.
SHOUTcast is effectively HTTP, but a poorly implemented version that has compatibility issues, particularly on mobile devices. It has a non-standard status line in its response which breaks a lot of clients. Icecast is a good alternative, and is what I would recommend for production use today. As another option, I have created my own streaming service called AudioPump which is HTTP as well, and has been specifically built to fix compatibility with oddball mobile clients, such as native Android players on old hardware. It isn't generally available yet, but you can contact me at brad#audiopump.co if you want to try it.
Latency
You mentioned a latency of 2 seconds being desirable. If you're getting 2-second latency with SHOUTcast, something is wrong. You don't want latency that low, particularly if you're streaming to mobile clients. I usually start with a 20-second buffer at a minimum, which is flushed to the client as fast as it can receive it. This enables immediate starting of the stream playback (as it fills up the client-side buffer so it can begin decoding) while providing some protection against buffer underruns due to network conditions. It's not uncommon for mobile users to walk around the corner of a building and lose their nice signal quality. You want your stream to survive that as best as possible, so if you have already sent the data to cover the drop out, the user doesn't have to know or care that their connection became mediocre for a short period of time.
If you do require low latency, you're looking at the wrong technology entirely. For low latency, check out WebRTC.
You certainly can tweak your traditional internet radio setup to reduce latency, but rarely is that a good idea.
Codec
Codec choice is what will dictate your compatibility more than anything else. MP3 is easily the most compatible, and AAC isn't far behind. If you go with AAC, you get better quality audio for a given bitrate. Most folks use this to reduce their bandwidth bill.
There are licensing fees with MP3, and there may be with AAC depending on what you're using for a codec. Check with a lawyer. I am not one, and the licensing is extremely complicated.
Other codecs include Vorbis and Opus. If you can use Opus, do so as the licensing is wide open and you get good quality for the bandwidth. Client compatibility here though is the killer of Opus. (Maybe in a few years it will be better.) Vorbis is a mediocre codec, but is free and clear.
On the extreme end, I have some stations doing their streaming in FLAC. This is lossless audio quality, but you're paying for 8x the bandwidth as you would with a medium quality MP3 station. FLAC over HTTP streaming compatibility is not code at the moment, but it works alright in VLC.
It is very common to support multiple codecs for your streams. Depending on your budget, if you can't do that, you're best off with MP3.
Finally on encoding, don't go from a lossy codec to another lossy codec if you can help it. Try to get the output stream as close to the input as possible. If you re-encode audio, you lose quality every time.
Recording from Browser
You mentioned users streaming from a browser. I built something like this a couple years ago with the Web Audio API where the audio is captured and then encoded and sent off to Icecast/SHOUTcast servers. Check it out here: http://demo.audiopump.co:3000/ A brief explanation of how it works is here: https://stackoverflow.com/a/20850467/362536
Anyway, I hope this helps you get started.
Streaming straight audio/mpeg (mp3 packets) has worked everywhere I've tried.
If you are developing an APP then go with AAC, if you are simply playing via web browser then you need a HTML5 Implimentation which is MP3. All custom protocols like RTMP or SHOUTcast requires additional UI to be built. There are some third party players available in open source world. You can either use them or stick to HTML5 MP3/OGG as most people now days are using chrome browser or other HTML5 complaint browsers.
I'm evaluating building an application which, simplifying the requirements, records from a microphone equipped small computer (eg: a Raspberry PI) and streams the digitalized sound over wireless connection in almost realtime to a server on the same LAN (No Internet involved). Ideally, the server application would record different streams from various wifi microphones and mix them together..
I'm currently looking into obtain a pretty good quality out of this, comparable somehow to a 128Kb stereo MP3.
At this point, I'm still evaluating options here, so I'm also looking to see your opinion on the feasibility of this.. if you think it's doable, what libraries, APIs, protocols would you use? Consider that this will be likely deployed on Linux based embedded computers (for the wifi mic part) and Linux based servers.
Thanks for your help.
I listen often Shoutcast on the iPad. This sounds pretty good to me. I do not know the kb/s rate there, I think they stream mp3. So I do not think this would be a big issue if you can live with the quality loss which comes with mp3. The bigger issue might be, how good your wireless connection is. When your network is pretty busy, there are more errors and lower speed. It also depends on the wireless standard and the hardware you are using. You may think about buffering, too.
I'm currently on edge with what container I should use for the videos I put on my website.
I recently started uploaded videos of game play/walkthroughs and saw the need for a container that could hold HD video without limitations on file size, codecs (AAC or AVC), or resolution (in the future I want to be able to support 5K video) and 5.1 Dolby digital and up audio. Of course I don't expect the 5K to be efficient at being streamed, I just want it to be available.
This is where the confusion started.
I currently use the .flv container because people state it is all around better. Less resource consumptive, widely used, and supports the common codecs. The problem with this is simple. It cannot support the HD content I want to show: 5.1 dolby audio and limitless file size.
MP4 is everything I need, but I heard that it can be slow to respond, pseudostreaming modules are not widely accepted by browsers, and I don't have time to change containers everytime someone wants to update to .mp5, 6, 12, etc.
That's where I am including .mkv as the container. .MKV also supports everything I want (HD, 3D), all codecs, universal, and limitless file attributes. THE ONLY problem is that it cannot be streamed.
I know this is a programmers site, but may be in the future, being that we can only advance web connections, I or someone else could program a module for apache .mkv streaming. I'm don't know where an apache module source is, so I cannot do it at this time.
I leaning between .flv and .mkv. I'm not really concerned about .mp4 because if I want to be future-proof I need .mkv, if I'm not concerned about the future or updates I should stay with .flv.
What do you all think. Would it really be so difficult to program a .mkv streaming module?
Excluding web streaming, which of the 3 would be all around better. Video quality (AAC AVC), file size limits, universal, web support, etc.
Thanks,
You can use the window media streaming platform. After that they will look after your every problem. However ,MP4 with h264 video and aac audio and streamed/played with flash is also good.
I have a large amount of audio stored on my web server in a very custom format that can't be replayed by anything other than my own application. That application is a Win32 app that can connect to my web server and stream and replay that audio.
I'd really like to be able to do the streaming and replaying from within a browser, but don't know where to start. Ideally I'd like the technology to be cross-platform (unlike my current Win32 app) and cross-browser (IE 6 and above and Firefox).
My current thoughts are to look at things like:
Flash, but doesn't that only replay mp3 audio?
Java, are VMs freely available still?
Converting the audio to a WAV file on the web server and then using someone else's plugin to replay that file. I'd rather keep the conversion off the web server for performance reasons, but is still an option.
Writing my own custom plugin to do the complete stream and replay operation.
Any guidance would be most useful.
Please note that the audio is not music and that simply converting to another audio format is not trivial. The audio that is stored also changes frequently (every minute) would need constant conversion.
Why are you using a proprietary music format? I'd probably not even bother downloading a program to listen to it.
I would suggest you convert it to mp3 and then use flash.
Building your own plugin would probably be hard, there are so many different platforms you'd have to cater for, something like flash is written for them already.
Apart from converting server-side: Implement a decoder for your format in ActionScript or Java. Then you can write a Flash movie or Java applet that plays it. Both languages/runtimes should be fast enough to decode in realtime unless your format is very complex. Flash would be the more accessible of the two, since nearly everyone has the plugin installed. (It's possible that playing a raw sound buffer isn't supported by older Flash versions than 10, I'm no expert on that.) The Java plugin is definitely free, but you'd require the users to install it.
I'd go with converting the audio to WAV (or MP3) on the server. Writing your own cross-platform browser component would be a lot of work, thanks to the different ways the major OSes handle their audio APIs.
Try taking a look at shoutcast.
Basically its a server app that will stream music to any client that connects to it through a browser (effectively your own radio station). I've never used it myself but should be straight forward.
Another idea is winamp remote. Again you install the app on the server but this time you can browse your music collection on their website and play individual songs.