I want to create an online classes type of site. I want the tutor to broadcast to all students and if a student has a question they can broadcast to ask the whole class. Meaning only a max of 2 people will be broadcasting. I would want to use webRTC but connecting like 30 people would give to much overhead. Is there a way of broadcasting 2 signals to 30 users using webRTC where the 30 remain dumb clients while using SOCKET IO for signalling?
Came across RTMP when doing my research and would like to ask if the tutor and the student (with the question) could "stream" their sessions to the other students. Where both can communicate with Webrtc after which streams are broadcasted to the others.
Can it be done ? Can it be done using REACT, SOCKET IO, WEBRTC and or RTMP ?
One option would be to send the stream to some users, then let those users retransmit to others. This can be done with webrtc scalable broadcasting. There will be more latency the more users come in between though.
A more used solution in an SFU. With this solution the sender will only need to send once stream to the server and the server handles all the retransmission to the other users. So by having a more powerful server you can easily scale your application for more users. There are several ways to implement this:
Janus-gateway
Kurento
Mediasoup
Here is a simple example project of how videoconferencing is implemented with mediasoup.
Related
I would like to use webRTC but instead of p2p would like to broadcast my audio/video feed to nodejs in realtime. I can encode the video to 125 kbps and 10-12 frames per second for smooth transmission. The idea is nodejs will receive this feed, save it and broadcast it on the same time as a realtime session/webinar. I can connect p2p but I am not sure how to
send feed to nodejs instead of peer
on nodejs how to receive feed
The WeRTC protocol suite is complex enough that an implementation from scratch for a selective forwarding unit SFU is likely to take at least a year by a team of experts. It requires handling a variety of networking protocols including datagrams (UDP) and TCP. And it may require transcoding between video and audio codecs.
The good news is that browser endpoints are now excellent. And open-source server implementations are good enough to get to a minimum viable product.
I am trying to build a basic conference call system based on plain RTP.
_____
RTP IN #1 ______ | | _______ MIX RTP receiver #1
|______| MIX |_____|
______| | RTP | |_______ MIX RTP receiver #2
RTP IN #2 |_____|
I am creating RTP streams on Android via the AudioStream class and using a server written in Node.js to receive them.
The naive approach I've been using is that the server receives the UDP packets and forwards them to the participants of the conversation. This works perfectly as long as there are two participants, and it's basically the same as if the two were sending their RTP stream to each other.
I would like this to work with multiple participants, but forwarding the RDP packets as they arrive to the server doesn't seem to work, probably for obvious reasons. With more than two participants, the result of delivering the packets coming in from different sources to each of the participants (excluding the sender of such packet) results in a completely broken audio.
Without changing the topology of the network (star rather than mesh) I presume that the server will need to take care of carrying out some operations on the packets in order to extract a unique output RTP stream containing the mixed input RTP streams.
I'm just not sure how to go about doing this.
In your case I know two options:
MCU or Multipoint Control Unit
Or RTP simulcast
MCU Control Unit
This is middle box (network element) that gets several RTP streams and generate one or more RTP streams.
You can implement it by yourself but it is not trivial because you need to deal with:
Stream decoding (and therefore you need jitter buffer and codecs implementation)
Stream mixing - so you need some synchronisation between streams (collect some data from source 1 and source 2, mix them and send to destination 3)
Also there are several project that can do it for you (like Asterisk, FreeSWITCH etc), you can try to write some integration level with them. I haven't heard anything about something on Node.js
Simulcast
This is pretty new technology and their specifications available only in IETF drafts. Core idea here is to send several RTP streams inside one RTP stream simultaneously.
When destination receives several RTP streams it needs to do exactly the same as MCU does - decode all streams and mix them together but in this case destination may use hardware audio mixer to do that.
Main cons for this approach is bandwidth to the client device. If you have N participants you need:
either send all N streams to all other
or select streams based on some metadata like voice activity or audio level
First one is not efficient, second is very tricky.
The options suggested by Dimtry's answer were not feasible in my case because:
The middle box solution is difficult to implement, requires too many resources or requires to rely on an external piece of software, which I didn't want to have to rely on, especially because Android RTP stack should work out of the box with basic support from a server component, especially for hole punching
The simulcast solution cannot be used because the Android RTP package cannot handle that and ad far as my understanding goes it's only capable of handling simple RTP streams
Other options I've been evaluating:
SIP
Android supports it but it's more of a high level feature and I wanted to build the solution into my own custom application, without relying on additional abstractions introduced by a high level protocol such as SIP. Also, this felt just too complex to set up, and conferencing doesn't even seem to be a core feature but rather an extension
WebRTC
This is supposed to be the de-facto standard for peer 2 peer voice and video conferencing but looking through code examples it just looks too difficult to set up. Also requires support from servers for hole punching.
Our solution
Even though I had, and still have, little experience on this I thought there must be a way to make it work using plain RTP and some support from a simple server component.
The server component is necessary for hole punching, otherwise getting the clients to talk to each other is really tricky.
So what we ended up doing for conference calling is have the caller act as the mixer and the server component as the middle-man to deliver RTP packets to the participants.
In practice:
whenever a N-user call is started, we instantiate N-1 simple UDP broadcast servers, listening on N-1 different ports
We send those N-1 ports to the initiator of the call via a signaling mechanism built on socket.io and 1 port to each of the remaining participants
The server component listening on those ports will simply act as a relay: whenever it receives a UDP packet containing the RTP data it will forward it to all the connected clients (the sockets it has seen thus far) except the sender
The initiator of the call will receive and send data to the other participants, mixing it via the Android AudioGroup class
The participants will send data only to the initiator of the call, and they will receive the mixed audio (together with the caller's own voice and the other participants' voices) on the server port that has been assigned to them
This allows for a very simple implementation, both on the client and on the server side, with minimal signaling work required. It's certainly not a bullet proof conferencing solution, but given the simplicity and feature completeness (especially regarding common network issues like NAT traversal, which using a server aid is basically a non-issue) is in my opinion better than writing lots of code which requires many resources for mixing server-side, relying on external software like SIP servers, or using protocols like WebRTC which basically achieve the same with lots more effort implementation wise.
Wanted to clear a few questions about websocket.
Is it possible to stream videos from server to client and client to server at the same time...something like video calling?
Can the server stream two videos to a single client at a time?
Regarding the first question, yes you can. There are already wrappers that simplify that task such as BinaryJS.
As per your second question, it would require a little extra configuration. Once a bidirectional link between client and server is established, the client will treat every incoming message as part of the same stream. Separating or multiplexing two videos in the same stream would have to carry another mark to help the client separate it.
It would be a better idea to open a new connection (with the same server) to stream a second video.
I am writing some NodeJS an application for some clients (mobile, single page app...). The application is game and quite social - users can have friends and game needs to sharing events between them in realtime. So I decided to use Socket.io.
How to effectively share events between connected sockets of user's friends? One user can be connected with more then one client (so he has more connected sockets).
Is it good idea to use socket.io "rooms" where each user has one room and when some event occurs, it is emited to each room of my friends?
How to effectively handle this?
For better notion you can imagine it the same way as facebook wall (or right panel of friends activity).
How would you handle it?
You could try solving this task with socket.io rooms but i think that would kill your application because of the overhead, for example
1000 users => equals => 1000 rooms ( 1 room for each user )
thats 1000 connection without anyone particularly listening to friend activity.
So the approach above already seems like not scalable.
I would try publish/subscribe pattern which is not very difficult to implement, you can use a micro library or Backbone with .listenTo or even socket.io with Redis pub/sub.
A user would have the option to listen or stop listening to a friend events.
One user would not get a ton of notification for simple friend event.
and the list goes on depending on your application behavior.
Some questions that have interesting information.
What should I be using? Socket.io rooms or Redis pub-sub?
Node.js, Socket.io, Redis pub/sub high volume, low latency difficulties
I am writing a multiplayer real time game for the browser with the server as a master instance and the clients as input devices and slaves to show the graphics.
I have to send out changes in the game world very often and very fast and it doesn't matter if some of the data sometimes gets lost on the way because a couple of milliseconds later there will be the next update anyway.
Right now I am using Socket.io to talk between the server and the clients but this uses TCP which makes the update come in unnecessary late sometimes.
I know that there is WebRTC with data channels where I would be able to send my updates through wit UDP which would be very awesome and exactly what I want. And it even seems to be implemented in Firefox and Chrome already https://stackoverflow.com/a/12864512/63779
What I now need is some Node library which would allow me to use data channels to send my data (for now just JSON strings) with help of UDP to the clients which are browsers. On the browser I would be able to use webkitRTCPeerConnection() but I have no idea how to start something like that on the Node server. Any suggestions? If there is no Node module for that, would it be possible to write something in some other language and just send the data via Unix domain sockets or something?