In short, I have a site where on the client side the user has a "beat maker" app. The user can turn on / turn off noises, background beats, etc, to essentially create their own custom "song" based on the pre-defined noises, tones and tunes that I have on the client side.
I need to somehow translate the beat they're making (in HTML5 canvas) to my server-side (currently Node.JS) and spit out an MP3 of their creation.
Basically I have to somehow have my server-side backend gracefully concatenate + overlap + mix various smaller MP3/wav files into one MP3 file that matches the beat that they created on the client side. I then have to return that MP3 to the client side for download.
Anyone able to point me in the right direction?
As far as my research indicates, this isn't easily accomplished or feasible at all (I.E. within realistic budget / time constraints of the project) due to the complexity of the problem at hand.
This is possible, and there are some audio libraries for JavaScript, but I would take a different approach.
The Web Audio API is very solid these days. You can have your user make all the adjustments client-side, and then generate the audio file right there in the user's browser. If you need to get a copy server-side, you can upload the raw PCM to your server (bandwidth intensive), or send the parameters to the server and re-generate the file.
Now unfortunately, PhantomJS doesn't support Web Audio. To generate a perfect server-side copy, I would execute Chrome with a special page that renders the audio and then uploads to the local server. This guarantees that the sound output you get is the exact same as that of the client, and leaves all of the heavy lifting to the Web Audio API already implemented in the browser.
You won't find much off-the-shelf for a project like this, but with a little creativity I think you will find that this isn't too difficult.
Related
I have a 3d widget that contains a web browser. I need to replicate the web browser content on the client side. I need whatever changes that are made to the web browser on the server side to be synced with all the clients. For example, if a youtube video is loaded on the server, I want all the clients to be able to watch it through the same replicated web browser.
I've been working on a replication project with Web Browsers and Video Streaming in general for the last couple of weeks. As far as I'm aware, you can't actually "replicate" it because the Web Browser Plugin (the normal one that UE has without needing to download it from the Marketplace) is pretty much an unfinished prototype that lacks a lot of control. Rather, you can replicate it but it won't work, so to say. It just means it does the functions it's supposed to do instead of anything being shared across clients and the server.
What you can do instead is actually Execute Javascript commands from inside your Web Browser Widget, which you call from your Actor Blueprint or Character Blueprint with Multicast Events.
Just to make it simpler, what you can do is send a "request" to the server if the Actor you're using the event from isn't a Server (UE4 has an "isServer" boolean you can call from pretty much anywhere), and then actually execute that with a Multicast Event if it is a Server.
Another difficult part about it is the fact that you can't really achieve Sound Attenuation with Browsers unless you brute-force it with Sphere/Box Collisions and the Youtube API which lets you change the volume of the Youtube video.
For more information about using Multicasts properly, I suggest you to look at this blog. Didn't understand a lick until I read this.
I am using Azure Blob Storage to store a video. I would like a user to be able to scroll the video to any point in time and play it from there.
For short videos, there is no problem because the whole video loads and you can do that, but for larger videos, it does not seem to work out of the box. And in some sense it makes sense - files by default do not have the functionality to be downloadable from the middle. But all decent video streaming websites offer this functionality. I must be missing some video concepts, would appreciate, if someone linked me to some articles explaining how things like this are done. Bonus points if the solution is using Microsoft Azure.
Large video example (28 secs, 126MB):
https://www.w3schools.com/code/tryit.asp?filename=GP328W3SEY77
Small video example (10 secs, 1MB):
https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_video
Video streaming servers or cloud services are usually dedicated specialised servers and their functionality can be quite complex.
A video 'file' typically consists of one or more video and audio tracks in a 'container' like MP4. The container will have header information and pointers to the track info.
Simple HTTP streaming of an mp4 file is possible if your server supports range requests, i.e. downloading parts of the file at a time, and if the header information is at the start of the video file - in mp4 is it usually at the end by default but can be moved to the start.
More sophisticated streaming servers, including most/all of the popular commercial services, use a dedicated streaming protocol, typically ABR HLS or DASH these days. These provide chunked multiple different bit rate versions of the video and allow the client switch between bit rates for each chunk it downloads - see more info here: https://stackoverflow.com/a/42365034/334402
The thumbnails you see when you scroll along a video timeline are actually usually a separate track in the video file or stream. They are a set of images at timepoints and the entire set of thumbnails can be downloaded quickly at the start of playback to give a view of what a particular part of the video will look like if the user wants to jump to it. When the user actually jumps to that section of the video, the client requests from the server the chunks or section of the file corresponding to that thumbnail.
Azure CDN actually provides some nice functionality:
video starts to play instantly (browser doesn't need to wait for the video to fully load)
you can scroll back/forward in time.
(Obviously, this in addition to the standard CDN functionality of multiple PoP, caching, etc.)
The above CDN setup was tested on Standard Microsoft, but Verizon and Akamai seem to be offering similar functionality.
So I have some time on my hands and thought I would make myself a little jukebox / radio type app.
It would be fairly simple, just a collection of MP3's on the server, one is chosen at random, it plays, on completion, the next one is chosen and plays. The front of this would just be a super simple page that has a player and displays the metadata.
I don't really have any experience with server programming but I'm going to look in to Node, seems like it would be good for this. I've already written a little script in Python that chooses a song from a selection and plays it (using VLC at the moment) so it should be simple to port it to Node / js.
Just wondering if someone could point me in the right direction for how to link the "player" with the "playlist".
Looking in to it, I can only find solutions involving a client and server using shoutCast or ICEcast or similar - so the playlist streams audio to a shoutcast server, and the website is just a player looking at the shoutCast URL - that seems unnecessary for me, as the streaming and the site would be the same thing.
New to a lot of this :) but I have time at the moment so happy to get stuck in!
Thanks in advance
I've built something similar and opted to do almost all the work client-side. There are several advantages:
Less infrastructure needed (and less to go wrong)
You can re-use normal HTTP CDNs for serving static files
More flexibility for client changes later (like sending different content to different clients, for A/B testing)
Potential for offline playback
No need for SHOUTcast or Icecast. All you need to do is publish the playlist and associated files to a web server. Client side, you can read this playlist, randomly pick an entry, and then load it via a simple new Audio().
Now, when your audio object fires its ended event, set the src property to the URL of the next item in the playlist. (This detail is important or Safari will stop playing audio, as they assume the user didn't want to continue.)
I am searching for the same. I think you are right using node.
I am building an application which collects speech via microphone as wav files. These recordings need to be streamed to a server and saved (as wav files, I know they are big but they have to be wav). I also need to stream audio (these can be mp3) from the server to the web application to be played for the user. I have no idea how to implement this, but I would like to use a Java EE application because I am familiar with Java and it's easier to maintain than Flex (we are having trouble with old Flex code at work). My concerns are:
How do I buffer the transmission so that users hear the whole file without breaks? Transferring the whole file and then playing it is fine, too, but knowing how to do this would be nice.
How do I verify transmissions to the server? Can I send in packets and verify/resend per packet?
Are there existing APIs for this (please!) or do I have to write this all by hand?
As I commented on your question, it is unclear whether you have already decided upon which components exist in the topology. In particular, it is unclear whether you already have a server process in charge of storing those audio files. Therefore, I will have to make a few assumptions in my answer. Feel free to comment, and I'll try my best to adjust.
The only way to ensure that an audio file is played (by the end user) without network-induced breaks is to have the end-user (or an application running at the end-user's side, such as some JavaScript code) play the audio stream after it was downloaded in its entirety. Unless you do that, you can only reduce the risk of breaks; you cannot eliminate it. Even the most sophisticated buffering algorithm cannot cope with a network outage 99.99% into buffering the entire stream. As I am not sure whether you have a client-side application involved in this, I can't advise how to force the client-side to download the entire file rather than playing it "as it comes"; in the simplest case, you might be able to suffice with using the Content-Disposition header: http://en.wikipedia.org/wiki/MIME#Content-Disposition
The answer to this question, again, depends on how you architect the solution. In general, though, as long as you use standard stream API's (such as Java IO), I wouldn't worry too much about verifying the content for errors. Error-correction is already provided lower in the networking stack (for example, your operating system's networking driver).
Apache Commons' File-Upload might be useful - again, depending on your architecture: http://commons.apache.org/fileupload/
Sorry if the question is confused, as I'm confused myself. I'm working around these requirements:
I'm building a public website where I need to display video.
I need to control what the player looks like
I'm the sole publisher of the video, meaning it can't be on YouTube for example
I need as much protection as possible in terms of protecting the content from being downloaded
So, I've read around StackOverflow and the web, and found lots of suggestions, like numerous flash players, Streaming servers, DRM protocols, services like Panda etc etc.
The problem is I don't understand how everything fits together.
For example, what makes my video content secure?
Is it the player on the client? is it the server that hosts the content? is it the streaming process? who hosts the streaming servers and what difference does this make?
Bearing in mind this is otherwise a very simple site, and is not a business venture.
if you were working around my requirements, what would you do? Could you explain step by step at a high level?
EDIT:
Just based on a couple of answers, I'm not saying no one can ever download my content. And I realize this kind of thing is expensive.
I'm just asking, if you had my requirements, what would you do? And could you explain it to me so i understand?
thanks again
Edit:
Thanks again for all the feedback, I can't vote anyone up as I'm a new user, but your answers have been very helpful.
The one thing I will say, is that my only request was to attempt security, that is 'make it difficult' for most users...that is common in software security.
Some of the suggestions have been just to not even try.
My question was really based around the fact that I know nothing about video deployment on the web, apart form the basic embedded swf flv combo.
Anyway, your info has been very useful though. I'll try a simple "real" streaming service (as opposed to HTTP streaming).
Any other recommendations would be awesome
cheers
"For example, what makes my video content secure? " Nothing.
"Is it the player on the client?" Neither. Anyone can write a client and retain the video content. Remember this. Anyone can write a client. This client can absorb and save your video. Nothing can stop this. Nothing.
"is it the server that hosts the content?" No. Server is only one piece of security. You have to secure the protocol. And the client. And anyone can write a client and retain the video content.
"is it the streaming process?" No. Protocol is only one piece of security. You have to secure the server, the protocol and the client. And anyone can write a client and retain the video content.
"who hosts the streaming servers and what difference does this make?" You host the streaming video servers. Otherwise, you might as well use YouTube.
Edit
"The problem is I don't understand how everything fits together."
"For example, what makes my video content secure?"
These are unrelated. You keep mentioning security, AND not knowing how "everything" fits together.
Here's a suggestion: stop mentioning security -- edit your question to eliminate all references to security and see if you get more useful answers.
Many companies sell streaming media servers. You put HTML in your page that references the streaming media site.
Example. Apple sells Quicktime media server. Read http://developer.apple.com/documentation/QuickTime/Conceptual/QTScripting_HTML/QTScripting_HTML_Document/chapter_1000_section_1.html for lots of information on how to present video from quicktime.
Before you go too far worrying about setting up these secure streaming protocol client server whatevers, make sure you weigh up the cost of your time getting this going, versus the cost of someone downloading your video.
Just to be clear: if your server is sending to a client, then they can copy (download) it. There's no way around it.
Response to your comment:
What I'd probably try doing if you wanted to try to avoid users downloading the files is this (I'll assume you're using FLV files, since they're the de facto standard on the web these days):
Put the FLV files in a non web-accessible directory.
Have a player.swf file request the file via a script on your site, eg: video.php?file=myVideo.flv
The video.php can then perform whatever security checks you'd like: for example, require logins, check the referrer, etc.
If the security checks are ok, then pass through the appropriate video file. If not, then perhaps have a short back-up video which is an ad for your site or something, saying "to watch this video, please come to mysite.com!"
Mostly video streaming sites like Hulu achieve a kind of poor-man's security by using RTMP to transfer the video data. You would need special server software to serve video via RTMP, for example Adobe Flash Media Server or WebORB.
RTMP is a proprietary protocol, so this is a case of security through obscurity; it's non-trivial to download a copy of the video (you can't just grab the file from a URL), but there are programs out there that are capable intercepting the stream and keeping a copy.
2.I need to control what the player looks like
Download and customise a free player like OSFLV.
4.I need as much protection as possible in terms of protecting the content from being downloaded
Forget it.
DRM for FLV exists, but you'll have to pay Adobe a load of money for Flash Media Server and Flash Media Rights Management Server, you'll lose client compatibility and ease of deployment, and in the end it's still breakable. Big old waste of time.
Accept that some people will download your videos, and put a big watermark on them so at least when they do you're getting free advertising.