I'm looking for best approach for my application.
I have video upload functionality. Front-end will send upload/video request with attached video file, then my Back-End will handle this request, will reduce the size and quality of the video (using fluent-ffmpeg
), then will create thumbnail image, based on the first frame of the video, then will upload video and his thumbnail image to the AWS S3 bucket, and in finally will return the compressed video and thumbnail to the front-end.
The problem that i have, all of those (back-end) tasks for compressing, creating thumbnail and uploading are much time consuming and sometimes (depends on the video size and duration) my nginx server will return 504 Gateway Time-out, which is normal. The question is:
How to handle this case. Should i use web sockets to notify the front-end for progression for processing the video, or i don't need to wait until all of those actions is completed.
My goal is to have functionality, where i can upload a video and show some progress bar for video processing and the user can be able to "play" with the application, not to be required to waiting until video is processed successfully
Seems like this is an architectural problem. Here is one of the solution that I prefer.
Use queue and store progress in some key value db. You may be unfamiliar with queue so I would recommend you to check some queue related tutorials. As you are using amazon, sqs might be interesting to you. In Rails you can check sidekiq. Laravel has laravel horizon.
While each queue is progress design app so it can report it. Like 50% 60% etc.
Process Thumbnails etc on queue too.
And if you want to scale you can simply increase the number of queue. I think that's how other also handle it.
Related
I was creating an application in react for uploading video and using a REST API to send that to the server and store in S3. I also wanted the simple audio version of the video for some other tasks and I am confused as to what might be the better way:
Creating audio file on the fly when it is needed using node-ffmpeg package and not store it anywhere
Start converting the video file to audio on the browser client only, and posting that to the server for storage along with the video.
Just post the video to the server and use queue system for creating a new task for video conversion to audio and then save that to the S3 storage.
The second method seems to be saving some compute power on the server but it might be a problem if the video upload completes, audio conversion is still going on and the client disconnects.
Would appreciate some help, thanks.
I need some help.
What is the best way to set up LIVE STREAMING over the web from my WEBCAM to the server and back to multiple users?
Essentially I'm trying to create a group video chat application that can support many users.
I don't want it to be peer to peer webRTC.
I actually managed to make it work with getUserMedia() -> mediaRecorder -> ondataavailable -> pass blob chunks to node.js via SOCKET.IO -> socket.io sends back blob chunks to other connected users -> append those chunks to a sourceBuffer that's connected to a mediaSource that's set as the source URL on a
And it actually worked! BUT it's so slow and laggy and resource intensive. As these chunks get passed like 20 per second and it's slowing the page a lot. I don't think you're supposed to pass that many blobs to the sourceBuffer so quickly. Just for a test I tried saving mediaRecordings every 3 seconds (so it's not that resource intensive) and passing those webm blobs to the sourceBuffer but for some reason only the first webm loads, and the other ones don't get added or start playing.
It just can't work for a production app this way.
What's the "RIGHT" way to do this?
How to pass a video stream from webcam to a Node.js server properly?
And how to stream this live stream back to the web from the Node.js server so that we can have a group video chat?
I'm a bit lost. Please help.
Do I use HLS? RecordRTC?
Do I stream from Node.js via http or via socket.io?
There are services that already let you do that easily like vonage video api tokbox but those seem to be very expensive?
I want to run the video streaming through my own Node.js server that I control.
What's the best way to do this?
Please help.
Thank you
Essentially I'm trying to create a group video chat application that can support many users.
I don't want it to be peer to peer webRTC.
Video chat requires low latency, and therefore requires usage of WebRTC. Remember that one of the "peers" can actually be a server.
And it actually worked! BUT it's so slow and laggy and resource intensive.
Video encoding/decoding is resource intensive no matter how you do it. If by "slow" and "laggy" you mean high latency, then yes, recording chunks, sending chunks, decoding chunks, will have higher latency by its very nature. Additionally, what you're describing won't drop frames or dynamically adjust the encoding, so if a connection can't keep up, it's just going to buffer until it can. This is a different sort of tradeoff than what you want.
Again, for a video chat, realtime-ness is more important than quality and reliability. If that means discarding frames, resampling audio stupid-fast to catch up, encoding at low bitrates, even temporarily dropping streams entirely for a few seconds, that's what needs to happen. This is what the entire WebRTC stack does.
As these chunks get passed like 20 per second and it's slowing the page a lot. I don't think you're supposed to pass that many blobs to the sourceBuffer so quickly.
No, this is unlikely your problem. The receiving end probably just can't keep up with decoding all these streams.
Do I use HLS?
Not for anyone actively participating in the chat... people who require low latency. For everyone else, yes you can utilize HLS and DASH to give you a more affordable way to distribute your stream over existing CDNs. See this answer: https://stackoverflow.com/a/37475943/362536 Basically, scrutinize your requirements and determine if everyone is actually participating. If they aren't, move them to a cheaper streaming method than WebRTC.
RecordRTC?
No, this is irrelevant to your project and frankly I don't know why people keep using this library for anything. Maybe they have some specific use case for it I don't know about, but browsers have had built-in MediaRecorder for years.
There are services that already let you do that easily like vonage video api tokbox but those seem to be very expensive?
This is an expensive thing to do. I think you'll find that using an existing service that already has the infrastructure ready to go is going to be cheaper than doing it yourself in most cases.
I have a node.js server that's receiving WEBM blob binary data small packets through socket.io from a Webpage!
(navigator.mediaDevices.getUserMedia -> stream -> mediaRecorder.ondataavailable -> DATA . I'm sending that DATA back to the server. So that includes timestamp and binary data).
How do I stream those back on a http request in a never ending live stream that can be consumed by a HTML webpage simply by adding the URL in the VIDEO tag?
Like this:
<video src=".../video" autoplay></video>
I want to create a live video stream that and basically stream back my Webcam to an html page but I'm a bit lost how do I do that. Please help. Thanks
Edit: I'm using express.js to serve the app.
I just am not sure what I need to do on the Server with the coming webm binary blobs to serve it properly to be consumed by an html page on an endpoint /video
Please help :)
After many failed attempts I was finally able to build what I was trying to:
Live video streaming through socket.io.
So what I was doing was:
Start getUserMedia to start the web camera
Start a mediaRecorder set to record intervals of 100 ms
On each available chunk emit an event through socket.io to the server with the blob converted to base64 string
Server sends back base64 converted 100ms video chunk back to all connected sockets.
Webpage gets the chunk and uses mediaSource and sourceBuffer to add the chunk to the buffer
Attach the media source to a video element and VOILA :) the video would play SMOOTHLY. As long as you attach each chunk in order and you don't skip chunks (in which case it stops playing)
And IT WORKED! BUT was unusable.. :(
The problem is the mediaRecorder process is CPU intensive and the page cpu usage was jumping to 15% and the whole process was TOO SLOW.
There was 2.5 seconds latency on the video stream passing through socket.io and virtually the same EVEN if DON'T send the blobs through socket.io but render them on the same page.
Sooo I found out this works but DOESN'T work for a sustainable video chat service. It's just not designed for it. For recording a webcam video to playback later, mediaRecorder can work but not for live streaming.
I guess for live streaming there's no way around WebRTC, you MUST use WebRTC to send the video stream to either a peer or a server to send to other peers. DO NOT TRY to build a live video chat service with mediaRecorder. You're only gonna waste your time. I did that for you :) so you don't have to. Just look into webRTC. You may have to use a TURN server. Twilio provide STUN, TURN servers but it costs money. BUT you can run your own TURN server with Coturn and other services but I'm yet to look into that.
Thanks. Hope that helps someone.
I'm writing an audio streaming server - similar to Icecast, and I'm running into a problem with streaming audio files. Proxying audio works fine (an audio source connects and sends audio in real time, which is then transmitted to clients over HTTP), but when I try to stream an audio file it goes by to quickly - clients end up with the entire audio file within their local buffer. I want them to only have a few 10s of seconds in their local buffer.
Essentially, how can I slow down the sending of an audio file over HTTP?
The files are all MP3. I've managed to get it pretty much working by experimenting with hardcoded thread delays etc... but that's not a sustainable solution.
If you're sticking with http you could use chunked transfer encoding and delay sending the packets/chunks. This would indeed be something similar to hardcoded thread::sleep but you could use an event loop to determine when to send the next chunk instead of pausing the thread.
You might run into timing issues though, maybe your sleep logic is causing longer delays than the runtime of the song. YouTube has similar logic to what you're talking about. It looks like they break videos into multiple http requests and the frontend client requests a new chunk when the buffer is too small. Breaking the file into multiple http body requests and then reassembling them at the client might have the characteristics you're looking for.
You could simply implement the http Range header and allow the client to only request a specific Range of the mp3 file. https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
The easiest method (by far) would be to have the client request chunks of the audio file on demand. std::net::TcpStream (which is what you said you're using) doesn't have a method to throttle the transfer rate, so you don't have many options to limit streaming backend short of using hard-coded thread delays.
As an example, you can have your client store a segment of audio, and when the user listening to the audio reaches a certain point before the end of the segment (or skips ahead), the client makes a request to the server to fetch the relevant segment.
This is similar to how real-world streaming services (like Youtube) work, because as you said, it would be a bad idea to store the entire file client-side.
I have a NodeJS server that is serving out web requests, users can also upload images to my server. I am currently processing these images on the same server. This was really a stop gap solution until I could do something better as the image processing eats up a lot of CPU, and I don't want to do it on my web server.
Type of processing is very simple:
Auto orient the image, POST the image to the processing server and
have it return an oriented image.
Create thumbnails, POST the image to the processing server and have
it return a thumbnail.
I would like to be able to use this image processing server for many other servers to hit. I also want it to run with little memory/cpu. I.E. if I have more CPU/memory available then great but I don't want it to crash if deployed on a server with limited resources.
The way I currently handle this is I have a config file the specifies how many images I can process concurrently. If I have very limited resources I will only process 1 image at a time while queuing the rest. This works fine right now cause the web server and image processing server is one in the same.
My standalone image processing server will have to 'queue' requests such that it can limit the amount of images it is processing at a time.
I don't really want to store the images on the processing server. Mainly because my web server knows the data structure (which images belong with which content). So I really want to just make a request to a server and get back the processed image.
Some questions.
Is a NodeJS implementation a good solution?
Should this be a RESTful http api. I am wondering about this because I am not sure I can pull it off given the requirement of only processing a certain amount of images at a time. If I get 15 requests at the same time I am not sure what to do here, I don't want to process the first while the others wait for a response (processed image).
Could use sockets to connect my web server with my image processing server to avoid question #2. This isn't as 'featureful' as it may be harder to use with other servers. But does get me out of the request/response problem I have in #2 above. IE web server could push image to be processed on socket and processing server could just answer back on the socket whenever it is done.
Is there some opensource solution that will run on linux/unix that will suit my needs. This seems like a pretty common problem, should be fun to implement but would hate to reinvent the wheel if I could use/contribute to another product. I have seen Apace::ImageMagick, but I cannot POST images they have to reside on the server already. There are also some windows only solutions that I cannot use.
Please help...
Why not break down the process into discrete steps.
User uploads image. Gets some kind of confirmation message that image has been successfully uploaded and is being processed.
Place request for image processing in queue.
Perform image processing.
Inform user that image processing is complete.
This fits your requirements in that users can upload many images in a short period of time without overloading the system (they simply go into the queue), image processing can happen on a separate server without moving the files)