I am building an application which collects speech via microphone as wav files. These recordings need to be streamed to a server and saved (as wav files, I know they are big but they have to be wav). I also need to stream audio (these can be mp3) from the server to the web application to be played for the user. I have no idea how to implement this, but I would like to use a Java EE application because I am familiar with Java and it's easier to maintain than Flex (we are having trouble with old Flex code at work). My concerns are:
How do I buffer the transmission so that users hear the whole file without breaks? Transferring the whole file and then playing it is fine, too, but knowing how to do this would be nice.
How do I verify transmissions to the server? Can I send in packets and verify/resend per packet?
Are there existing APIs for this (please!) or do I have to write this all by hand?
As I commented on your question, it is unclear whether you have already decided upon which components exist in the topology. In particular, it is unclear whether you already have a server process in charge of storing those audio files. Therefore, I will have to make a few assumptions in my answer. Feel free to comment, and I'll try my best to adjust.
The only way to ensure that an audio file is played (by the end user) without network-induced breaks is to have the end-user (or an application running at the end-user's side, such as some JavaScript code) play the audio stream after it was downloaded in its entirety. Unless you do that, you can only reduce the risk of breaks; you cannot eliminate it. Even the most sophisticated buffering algorithm cannot cope with a network outage 99.99% into buffering the entire stream. As I am not sure whether you have a client-side application involved in this, I can't advise how to force the client-side to download the entire file rather than playing it "as it comes"; in the simplest case, you might be able to suffice with using the Content-Disposition header: http://en.wikipedia.org/wiki/MIME#Content-Disposition
The answer to this question, again, depends on how you architect the solution. In general, though, as long as you use standard stream API's (such as Java IO), I wouldn't worry too much about verifying the content for errors. Error-correction is already provided lower in the networking stack (for example, your operating system's networking driver).
Apache Commons' File-Upload might be useful - again, depending on your architecture: http://commons.apache.org/fileupload/
Related
I'm used to working with user-uploaded files to the same server, and transferring my own files to a remote server. But not transferring user-uploaded files to a remote server.
I'm looking for the best (industry) practice for selecting a transfer protocol in this regard.
My application is running Django on a Linux Server and the files live on a Windows Server.
Does it not matter which protocol I choose as long as it's secure (FTPS, SFTP, HTTPS)? Or is one better than the other in terms of performance/security specifically in regards to user-uploaded files?
Please do not link to questions that explain the differences of protocols, I am asking specifically in the context of user-uploaded files.
As long as you choose a standard protocol that provides (mutual) authentication, encryption and message authentication, there is not much difference security-wise. If all of this is provided by a layer of TLS in your chosen protocol (like in all of your examples), you can't make a big mistake on a design level (but implementation is key, many security bugs are bugs of implementation, and not design flaws). Such protocols might differ in the supported list of algorithms for different purposes though.
Performance-wise there can be quite significant differences, it depends on what you want to optimize for. If you choose HTTPS, you won't be able to keep a connection open for a long time, and would most probably have to bear the overhead of the whole connection setup with authentication and everything, for every transmitted file. (Well, you can actually keep a https connection open, but that would be quite a custom implementation for such file uploads.) Choosing FTPS/SFTP you will be able to keep a connection open and transmit as many files as you want, but would probably have to have more complex error handling logic (sometimes connections terminate without the underlying sockets knowing about it for a while and so on). So in short I think HTTPS would be more resilient, but secure FTP would be more performant for many small files.
It's also an architecture question, by using HTTPS, you would be able to implement all of this in your application code, while something like FTP would mean dependence on external components, which might be important from an operational point of view (think about how this will actually be deployed and whether there is already a devops function to manage proper operations).
Ultimately it's just a design decision you have to make, the above is just a few things that came to mind without knowing all the circumstances, and not at all a comprehensive list of things to consider.
For sometime now I've been trying to do something that I never thought that it would be that hard: audio streaming. My objective is simple; a simple web app through which a certain someone can click a button and live-stream his own voice to other people using this app. It's an online classroom of sorts. Here's the details:
A broadcast/lecture is scheduled for a certain date and time (done)
A user logs-in as a teacher/instructor to a simple interface where he can click "start broadcasting" (done)
When the instructor clicks "broadcast" his voice is streamed to other users. Other student-type users can also log in and start listening to THE BROADCAST this teacher started. (and here is the trick!)
The broadcast itself should be automatically stored to a local file in the process. So that students can go back to it anytime.
Of course I spent so many hours googling and stackoverflow-ing this problem, and here is what I could understand so far:
If the starting point is the browser, I must use the GetUserMedia API, the result is raw PCM data that I can download, send to server or stream to others. (simple)
Offering the broadcast to the listeners (students) will be done via HTML5's Audio API. (simple)
WebRTC cannot help me here, because it's a p2p thing, there cannot be a server middling in the process, and I NEED TO KEEP A COPY OF THE LECTURE LOCALLY. (Here's a working example)
I can use tools like Binary.js to stream the audio binary data to the students, but this requires a file to be present already on the desk.
I need to convert the PCM data to a format like MP3 or OGG in the process, and not use WAV because it's much expensive bandwidth-wise.
I feel like it should be straight forward, but I cannot get it to work, I cannot piece all of this together and offer a stable and good experience for the user.
So again, I would love to know how to do the following:
Break the GetUserMedia raw data into packets and convert it to mp3, stream it to the server, where a script (NodJS probably) can store it locally and stream it whoever tuned-in, in real time.
I am open to whatever tool you recommend, I know that NodeJS will be present in the solution, and I am happy to use it. If the streaming could be done via a 3rd-party tool, I have no problem with that.
Thanks you in advance.
I see your comment about WebRTC, but I think you should investigate it more.
Like what you see here in this (old) post: http://servicelab.org/2013/07/24/streaming-audio-between-browsers-with-webrtc-and-webaudio/
Otherwise, you might have to go for a third party solution, like https://www.crowdcast.io/
(Even if you find a video-only solution, you can use a static picture or so for the video)
Event broadcasting is a good business for many companies. If it was that easy, there wouldn't be only few and well known competitors in the market.
In my project, I need to be able to compress small audio files so that it can be transferred easily from server to the client. I found We Chat uses audio compression closest to my requirement. But I couldn't find ways of doing it
I'm able to write code for image compression by reducing resolution and clarity. What should I try here ? combine two frames into one etc. Streaming is definitely possible if two frames are mixed into one. I want to know if there are any well developed libraries to do the same. I'm easy on the format until client side is able to read it. I also want to know if streaming is possible in compressed format.
I run a wiki that deals with the specifics of all kinds of multimedia technology. You might be interested in the Audio Codecs category, currently cataloging about 150 different audio codecs.
I can't tell from your question if you control both the client and server, or if you only control the server. If you control both, try a simple IMA ADPCM variant in order to get started. If you don't control the client, then of course your will need to code towards whatever the client can support.
There are many more codecs, of course, and lots of them have existing open source implementations that you may wish to leverage.
I'm trying to play an FLV file located on a remote server ('crossdomain.xml' does not exists in the process) in 3 ways:
From a browser using an SWF player located on some server
From VLC, pointing to the remote file.
Downloading the remote file and swf player - playing it locally
Guess what?
Didn't play the flv
Played like a charm
Played like a charm
Conclusion: Flash's Cross Domain security is useless.
Please tell me where I'm wrong or perhaps I'm just helping someone understand that this security is useless.
I wasn't going to write my own answer, because I felt like #jpea had already written the most important things. But it seems like the idea and use of the crossdomain.xml files is still unclear. So here it is:
Cross-site scripting does not refer to accessing media content from other servers, but to an attack method used for roughly 80% of all internet security violations. It can happen in many different ways, but always involves injecting foreign code into a web page (or plug-in content) to make the client behave in a way that was not intended. It might result in an attack on the server later, but the initial problem is always related to vulnerabilities on the client side.
Crossdomain-policy files are the Flash implementation of the so-called "same-origin-policy", an important part in preventing cross-site scripting. Essentially, it is meant to ensure that any content loaded by an SWF must be within the same domain (as opposed to "on the same server") as the original content.
What does this mean, in practice? It means, for example, that an attacker is not allowed to load your original SWF into an (invisible) enclosing SWF hosted on a different server, and monitor all incoming and outgoing traffic, or capture keyboard events, to steal passwords and such: Violating the crossdomain-policy will cause a security error that stops execution of all ActionScript.
It does not, however prevent FLV files from being played in some other way - and that is absolutely not what it is intended to do.
Admittedly, there are (more or less easy) ways to get around crossdomain-policy files, for example by using a proxy to channel the SWFs URL requests, so using them will not result in "real" security. But as part of a multi-level security strategy, they do help to raise the bar for attackers.
crossdomain.xml is meant as a security measure for the Flash player plugin. An FLV alone isn't the security risk, the player is. In instance #2, you didn't use the Flash player. Instance #3, it's uses the same security that Flash uses in it's IDE (to allow debugging). Instance #1 worked exactly as intended.
crossdomain.xml isn't meant as a DRM sort of security, or to not allow downloading of files. It's meant to disallow unintended domains from using your FLV/F4V from another server (better known as cross site scripting).
Sorry if the question is confused, as I'm confused myself. I'm working around these requirements:
I'm building a public website where I need to display video.
I need to control what the player looks like
I'm the sole publisher of the video, meaning it can't be on YouTube for example
I need as much protection as possible in terms of protecting the content from being downloaded
So, I've read around StackOverflow and the web, and found lots of suggestions, like numerous flash players, Streaming servers, DRM protocols, services like Panda etc etc.
The problem is I don't understand how everything fits together.
For example, what makes my video content secure?
Is it the player on the client? is it the server that hosts the content? is it the streaming process? who hosts the streaming servers and what difference does this make?
Bearing in mind this is otherwise a very simple site, and is not a business venture.
if you were working around my requirements, what would you do? Could you explain step by step at a high level?
EDIT:
Just based on a couple of answers, I'm not saying no one can ever download my content. And I realize this kind of thing is expensive.
I'm just asking, if you had my requirements, what would you do? And could you explain it to me so i understand?
thanks again
Edit:
Thanks again for all the feedback, I can't vote anyone up as I'm a new user, but your answers have been very helpful.
The one thing I will say, is that my only request was to attempt security, that is 'make it difficult' for most users...that is common in software security.
Some of the suggestions have been just to not even try.
My question was really based around the fact that I know nothing about video deployment on the web, apart form the basic embedded swf flv combo.
Anyway, your info has been very useful though. I'll try a simple "real" streaming service (as opposed to HTTP streaming).
Any other recommendations would be awesome
cheers
"For example, what makes my video content secure? " Nothing.
"Is it the player on the client?" Neither. Anyone can write a client and retain the video content. Remember this. Anyone can write a client. This client can absorb and save your video. Nothing can stop this. Nothing.
"is it the server that hosts the content?" No. Server is only one piece of security. You have to secure the protocol. And the client. And anyone can write a client and retain the video content.
"is it the streaming process?" No. Protocol is only one piece of security. You have to secure the server, the protocol and the client. And anyone can write a client and retain the video content.
"who hosts the streaming servers and what difference does this make?" You host the streaming video servers. Otherwise, you might as well use YouTube.
Edit
"The problem is I don't understand how everything fits together."
"For example, what makes my video content secure?"
These are unrelated. You keep mentioning security, AND not knowing how "everything" fits together.
Here's a suggestion: stop mentioning security -- edit your question to eliminate all references to security and see if you get more useful answers.
Many companies sell streaming media servers. You put HTML in your page that references the streaming media site.
Example. Apple sells Quicktime media server. Read http://developer.apple.com/documentation/QuickTime/Conceptual/QTScripting_HTML/QTScripting_HTML_Document/chapter_1000_section_1.html for lots of information on how to present video from quicktime.
Before you go too far worrying about setting up these secure streaming protocol client server whatevers, make sure you weigh up the cost of your time getting this going, versus the cost of someone downloading your video.
Just to be clear: if your server is sending to a client, then they can copy (download) it. There's no way around it.
Response to your comment:
What I'd probably try doing if you wanted to try to avoid users downloading the files is this (I'll assume you're using FLV files, since they're the de facto standard on the web these days):
Put the FLV files in a non web-accessible directory.
Have a player.swf file request the file via a script on your site, eg: video.php?file=myVideo.flv
The video.php can then perform whatever security checks you'd like: for example, require logins, check the referrer, etc.
If the security checks are ok, then pass through the appropriate video file. If not, then perhaps have a short back-up video which is an ad for your site or something, saying "to watch this video, please come to mysite.com!"
Mostly video streaming sites like Hulu achieve a kind of poor-man's security by using RTMP to transfer the video data. You would need special server software to serve video via RTMP, for example Adobe Flash Media Server or WebORB.
RTMP is a proprietary protocol, so this is a case of security through obscurity; it's non-trivial to download a copy of the video (you can't just grab the file from a URL), but there are programs out there that are capable intercepting the stream and keeping a copy.
2.I need to control what the player looks like
Download and customise a free player like OSFLV.
4.I need as much protection as possible in terms of protecting the content from being downloaded
Forget it.
DRM for FLV exists, but you'll have to pay Adobe a load of money for Flash Media Server and Flash Media Rights Management Server, you'll lose client compatibility and ease of deployment, and in the end it's still breakable. Big old waste of time.
Accept that some people will download your videos, and put a big watermark on them so at least when they do you're getting free advertising.