I am developing multiparty conferencing in nodejs using kurento (kurento-utils-js on client side and kurento-client package on server side)
When someone speaks (either local or on remote stream), I want to show the audio level on user interface (UI) to just show that he/she is speaking.
You can use hark API provided by kurentoUtils. Tweak the threshold between [-100, 0] to see which values works best for you. -50 works for me.
const speechEvent =kurentoUtils.WebRtcPeer.hark(stream, { threshold: -50 });
speechEvent.on('speaking', () => {
/* do something on the UI */
});
speechEvent.on('stopped_speaking', () => {
/* do something on the UI */
});
Related
I have an animation written in Javascript. I am new to nodejs (no knowledge) and I have been finding it difficult to stream the animation in real-time to users connected to the site.
I read about socket.io and Websockets but I do not have a good approach. Currently, the animation starts with a function call and writes to a canvas.
I need to know how to stream this animation from the server-side to the client so that multiple connected users can see the same scene of the animation at the same time. A functional explanation with code will also be appreciated.
Without knowing what type of animations are used, how they're built and work, I would suggest doing the animation client-side and just send some sort of synchronization command from server to all clients using socket.io .
I also would suggest putting all users (which all should see the same animation) in to one room (see rooms).
Now you can send a synchronized command to all users, e.g. to start, continue, stop and reset the animation. On the other hand, on the server-side you could track which of the clients in the room have already loaded the animation, started, stopped and so on.
// server
io.to('some-room').emit('load', animationid);
/* ... */
socket.on('loaded', () => {
if (allClientsLoaded) {
io.to('some-room').emit('start');
}
});
socket.on('started', () => {
});
// client
socket.on('load', async (animationid) => {
await loadAnimation(animationid);
socket.emit('loaded');
});
socket.on('start', () => {
startAnimation();
});
I am creating a browser game with node.js (backend api) and angular (frontend). My goal is to implement an in-game chat to allow communication between players on the same map. The chat is not an essential part of the game, so messages don't need to be instant (few seconds of latency should be ok). It is just a cool feature to talk some times together.
A good solution should be to implement socket.io to have real-time communication. But as chat is not an essential component and is the only thing which will require websockets, i'm wondering if there is not an alternative to avoid server overload with sockets handling.
I thinked about polling every 2 or 3 seconds my REST API to ask for new messages, but it may overload server the same way... What are your recommandations?
Thank you for your advices
There's a pretty cool package called signalhub. It has a nodejs server component and stuff you can use in your users' browsers. It uses a not-so-well-known application of the http (https) protocol called EventSource. EventSource basically opens persistent http (https) connections to a web server.
It's a reliable and lightweight setup. (The README talks about WebRTC signalling, but it's useful for much more than that.)
On the server side, a simple but effective server setup might look like this:
module.exports = function makeHubServer (port) {
const signalhubServer = require('signalhub/server')
const hub = signalhubServer({ maxBroadcasts: 0 })
hub.on('subscribe', function (channel) {
/* you can, but don't have to, keep track of subscriptions here. */
})
hub.on('publish', function (channel, message) {
/* you can, but don't have to, keep track of messages here. */
})
hub.listen(port, null, function () {
const addr = hub.address()
})
return hub
}
In a browser you can do this sort of thing. It user GET to open a persistent EventSource to receive messages. And, when it's time to send a message, it POSTs it.
And, Chromium's devtools Network tab knows all about EventSource connections.
const hub = signalhub('appname', [hubUrl])
...
/* to receive */
hub.subscribe('a-channel-name')
.on('data', message => {
/* Here's a payload */
console.log (message)
})
...
/* to send */
hub.broadcast('a-channel-name', message)
I made a test test bot for discord.js using Node.js / ffmpeg, to play a radio station in a channel on discord and there are no problems there, however i was thinking about how it would stream the same station to multiple servers/channels efficiently.
For example, if I am playing it on "Discord server one":
var voiceChannel = message.member.voiceChannel;
voiceChannel.join().then(connection => {
console.log("joined channel");
const dispatcher = connection.playArbitraryInput('http://philae.shoutca.st:8950/live', { volume: 0.5 });
dispatcher.on("end", end => {
console.log("left channel");
voiceChannel.leave();
});
}).catch(err => console.log(err));
However if i want to play it on another server that my bot is a member of, do i need to create the audio stream again?
If so that kinda sux. My bot would have to encode the stream seperatly for each instance?
I would like to know if there is any way i can reuse the audio stream and restream it so that if I had my bot in 100 channels its only one download instance of the stream and 100 uploads
I haope i am making sense but let me make some fake code to try help explain:
lets say instead i could do something like this:
let cachedAudioStream = cacheArbitraryInput('http://philae.shoutca.st:8950/live', {});
then for every outgoing instance:
connection1.playArbitraryInput(cachedAudioStream, { volume: 0.5 });
connection2.playArbitraryInput(cachedAudioStream, { volume: 0.8 });
connection3.playArbitraryInput(cachedAudioStream, { volume: 1 });
and so on
Thanks
Since the stream is managed by Discord.js (you only give a link, and Discord.js uses FFMPEG to handle it), you can't do it without modifying the Discord.js code.
Your question is relevant but I don't think it's feasible, at least if you uses Discord.js. You can still open an issue on their Github repository to ask them.
discord.js has a 'Voice Broadcast' for stuff like radio bots. Yes, you do have to connect the call to the broadcast but that's all.
I have the MediaStream object returned from getUserMedia and can show it in my own screen.
The thing is, I don't know how to *send / pipe / stream/ * that MediaStream from Point A through NodeJS using socket.io to Point B.
My code right now is:
// Cámara
if (navigator.getUserMedia) {
navigator.getUserMedia({audio: true, video: true}, function(stream) {
video.src = URL.createObjectURL(stream) || window.URL.createObjectURL(stream);
webcamstream = stream;
}, onVideoFail);
} else {
alert ('failed');
}
});
function onVideoFail(e) {
console.log('webcam fail!', e);
};
I need the way to make this stream being sent constantly to other user using NodeJS.
The comment made in the answer for Audio and video conference with NodeJS are still valid
if you were to send the streams through socket.io instead of a peer connection, in any case that would be the raw video content (bytes). You would lose:
the streaming part (RTP/RTCP), and corresponding packet loss cancellation
the bandwidth adaptation
the encryption (DTLS)
the media engine (jitter correction, …)
over webRTC.
why not implement an SFU in node.js? Use node.js/socket.io for the signaling (i.e. the initial handshake) but also use node.js as a peer connection endpoint which relays/routes the media stream to (an)other(s) peer(s)? You would have an intermediate server like you seem to want, and all the advantages of webRTC.
another solution is to use an MCU, google webrtc+mcu and you will find many.
this might be a little late, but there's now a library called wrtc/node-webrtc. use that! https://github.com/node-webrtc/node-webrtc
I am trying to understand how node.js will fit in my scenario.
Currently, a mobile device sends gps coordinates to a Rails backend thru POST. The mobile device sends it every minute.
POST http://127.0.0.1:3000/location/
My RubyOnRails web app can display the history of locations on a map, no problem with this.
But my experience in nodejs is still in its infancy and I would like to understand how to display a map for real time GPS updates as they come in.
Use Socket.IO.
Most tutorials show some kind of chat application. In your case the communication is only uni-directional. And your device is not connected through WebSockets, but only POSTs new coordinates without a channel back (device doesn't subscribe to events from server).
Your web-page (showing a Google Maps control) connects to your Node.js server through socket.io and gets new coordinates pushed from your server (socket.emit). You have to remember somehow which "new-coordinate" events from devices have to be published to which listening web-clients.
And of course you need an initial list of recent coordinates for your web-page if you want to show not only new coordinates of a device. You could deliver them over sockets as well, but as you will need some kind of authentication maybe a REST call is clearer for first population of devices GPS-track.
So when a device updates its location, you only have to find connected listeners for that device and emit("new-coordinate", {lat: ..., lng: ..., deviceId: ...} events to the maps. In the web-page, you receive the events and handle them like this:
<script src="/socket.io/socket.io.js" type="text/javascript"></script>
<script type="text/javascript">
$(function () {
if (typeof io === 'undefined') {
return; // socket.io not loaded
}
var socket = io.connect();
socket.on('new-coordinate', function (data) {
console.log('arrived', data);
$(document).trigger('new-coordinate', data);
// or just directly update your map ...
});
});
</script>
Edit
As your web-page and the new-coordinate POSTs are delivered through RoR, you have to trigger the events from Ruby to node.js server-side. Basically you could call your node.js app from Ruby via REST, but there are other options like Redis pub/sub or dnode ruby.
(comment) I'd migrate the RoR app to node.js ;-) (/comment)