WebRTC - How to establish a peer connection after offers and answers - node.js

I have a node.js running which the users will connect to. The offer and answer will be generated and sent through node.js.
I'm trying to establish a peer connection and send over a camera stream. I tried my code without using ICE candidates as the computers where in the same subnet. I tried to implement ICE afterwards. I'm not sure if i've done it right though or if it's even needed if the computers are on the same subnet.
var localStream;
//Connect to signaling server
var signalingChannel = io.connect('http://85.134.54.193:8001');
console.log("Connect to signaling server");
var servers = null;
var video1;
var video2;
var audio1;
var audio2;
var cfg = {"iceServers":[{"url":"stun:stun.l.google.com:19302"}]};//{ "iceServers": [{ "url": "stun:stun.l.google.com:19302" }] };
var con = { 'optional': [{'DtlsSrtpKeyAgreement': true}, {'RtpDataChannels': true }] };
var peerConnection;
//Runs after the page has been loaded
window.onload=function(){
//Gets ID for the video element which will display the local stream
video1 = document.getElementById("audio1");
//Gets ID for the video element which will display the remote stream
video2 = document.getElementById("audio2");
audio1 = document.getElementById("audio1");
audio2 = document.getElementById("audio2");
}
//Start button function
function caller(){
peerConnection = new webkitRTCPeerConnection(cfg);
navigator.webkitGetUserMedia({'audio':true, video:true}, function (stream) {
console.log("Got local audio", stream);
video1.src = window.webkitURL.createObjectURL(stream)
peerConnection.addStream(stream);
},
function ( err ) {
console.log( 'error: ', err );
});
console.log("Calling");
//Create Offer
peerConnection.createOffer(function (offerDesc) {
console.log("Created local offer", offerDesc.sdp);
peerConnection.setLocalDescription(offerDesc);
}, function () { console.warn("Couldn't create offer"); });
//ICE Candidates Generator
peerConnection.onicecandidate = function(evt) {
//When The Ice Gathering is complete
if (evt.target.iceGatheringState == "complete") {
//Create a new offer with ICE candidates
peerConnection.createOffer(function(offer) {
console.log("Offer with ICE candidates: " + offer.sdp);
signalingChannel.emit('offer', JSON.stringify(offer));
console.log("offer sent");
signalingChannel.on('answer', function(data){
console.log("Receive answer");
//The answer is set as the remote description for the offerer
peerConnection.setRemoteDescription(new RTCSessionDescription(JSON.parse(data)));
console.log("Set remote desc");
peerConnection.onaddstream = gotRemoteStream;
console.log("Add remote stream to peer connection");
});
});
}
}
}
function answerer(){
peerConnection = new webkitRTCPeerConnection(cfg);
navigator.webkitGetUserMedia({'audio':true, video:true}, function (stream) {
console.log("Got local audio", stream);
video1.src = window.webkitURL.createObjectURL(stream)
peerConnection.addStream(stream);
},
function ( err ) {
console.log( 'error: ', err );
});
console.log("Answering");
//Listen for offer
signalingChannel.on('offer', function(data){
console.log("Offer Received");
//Set the remote description from caller's local description
peerConnection.setRemoteDescription(new RTCSessionDescription(JSON.parse(data)));
//Generate answer after getting the remote description
peerConnection.createAnswer(function(sessionDescription) {
//Set local description
peerConnection.setLocalDescription(sessionDescription);
//The local desc will be the answer sent back to offerer
signalingChannel.emit('answer', JSON.stringify(sessionDescription));
console.log("Answer sent");
});
});
}
function gotRemoteStream(event){
video2.src = window.webkitURL.createObjectURL(event.stream);
}

Here is a sequence of events I have working today (Feb 2014) in Chrome. This is for a simplified case where peer 1 will stream video to peer 2.
Set up some way for the peers to exchange messages. (The variance in how people accomplish this is what makes different WebRTC code samples so incommensurable, sadly. But mentally, and in your code organization, try to separate this logic out from the rest.)
On each side, set up message handlers for the important signalling messages. You can set them up and leave them up. There are 3 core messages to handle & send:
an ice candidate sent from the other side ==> call addIceCandidate with it
an offer message ==> SetRemoteDescription with it, then make an answer & send it
an answer message ===> SetRemoteDescription with it
On each side, create a new peerconnection object and attach event handlers to it for important events: onicecandidate, onremovestream, onaddstream, etc.
ice candidate ===> send it to other side
stream added ===> attach it to a video element so you can see it
When both peers are present and all the handlers are in place, peer 1 gets a trigger message of some kind to start video capture (using the getUserMedia call)
Once getUserMedia succeeds, we have a stream. Call addStream on the peer 1's peer connection object.
Then -- and only then -- peer 1 makes an offer
Due to the handlers we set up in step 2, peer 2 gets this and sends an answer
Concurrently with this (and somewhat obscurely), the peer connection object starts producing ice candidates. They get sent back and forth between the two peers and handled (steps 2 & 3 above)
Streaming starts by itself, opaquely, as a result of 2 conditions:
offer/answer exchange
ice candidates received, exchanged, and added
When I want to change the stream, I go back to step 3 and set up a new peer connection object and do the whole offer/answer again.

Why do you wait for ICE to complete before creating an answer? what about doing them simultaneously? That might help, as it is just meant to work simultaneously. If you can post your logs after this when it would still not work we can try debugging it even further. If you want to see an audio-only example of this (it sends both music-audio and microphone-audio) check here, and the github source. Server made with node.js and ws plugin. The audio connection works with webRTC.

Related

How does a ws websocket tell if connection event corresponds to an existing client in the clients list?

I am using the ws module to implement a WebSocket server in NodeJS. On the client-side, I request the connection using
webSocket = new WebSocket(url);
On the server-side, I have code that handles the 'connect' event, in which I print out the number of clients using
console.log("Total number of clients = " + wsServer.clients.size);
If I open the client-side in different tabs (or browsers), the number of clients is incremented for each new connection (as expected).
If I refresh a page, the webSocket = new WebSocket(url); code is called again and on the server the code handling the 'connect' event (see below) is also called again. However, in this case, the number of clients is not incremented. This is nice behaviour as it maintains the number of connections one wants, but I cannot see how this is done. I want to be able to test if this is an existing connection as I have a chat room running that says 'so-and-so' has joined when a new connection is made. However, I don't want this to happen every time a user refreshes their page.
Here is the server-side event-handler:
// On connection
wsServer.on("connection", function (ws, req) {
let { query } = url.parse(req.url, true);
ws.userName = ("name" in query) ? query.name : null;
ws.roomCode = ("roomCode" in query) ? query.roomCode : null;
ws.userPIN = ("PIN" in query) ? query.PIN : null;
console.log(ws.userName + " joined room " + ws.roomCode);
console.log("Total number of clients = " + wsServer.clients.size);
let data = {
userName: "Server",
message: ws.userName + " joined the room."
};
let dataStr = JSON.stringify(data);
// Loop through each client
wsServer.clients.forEach((client) => {
// Check if client is ready and in same room
if ((client.readyState === WebSocket.OPEN) && (client.roomCode == ws.roomCode)) {
client.send(dataStr);
}
});
// On message event
ws.on("message", function (msg) {
// Loop through each client
wsServer.clients.forEach((client) => {
// Check if client is ready and in same room
if ((client.readyState === WebSocket.OPEN) && (client.roomCode == ws.roomCode)) {
client.send(msg.toString());
}
});
});
});
As one can see, I do modify the client by adding name and room code fields to it but these are not present when the 'connect' event fires, implying the object is being created from scratch. However, no extra client is being added to the clients list, so what I would like to understand is:
how does the ws package know this is an existing connection?
how can I test for this?
Any advice would be gratefully received!
On investigation, it turns out the when the user refreshes the page, the WebSocket connection is closed and then re-opened. Hence there is no testing for an existing client, the existing client is just deleted on the server and the new connection added - hence the number of clients does not appear to change.

Sending Messages from Leaf Device Downstream device not being handled by IoT Edge running at Transparent Gateway

I have followed all the instruction for setting up a "Downstream Device" to send messages through IoT Edge running in Transparent Gateway. I believe my routing rules are correct, but my Function module is not receiving any of the Messages through the message flow.
These are the instruction I've followed:
https://learn.microsoft.com/en-us/azure/iot-edge/how-to-create-transparent-gateway-linux
I am using 2 Linxu VMs (ubuntu 16.04.5).
IoT Edge Transparent Gateway VM is configured with all the certs properly setup, configured and verified. I've been able to using the openssl tool from the
openssl s_client -connect {my-gateway-machine-name-dns-name}.centralus.cloudapp.azure.com:8883 -CAfile /certs/certs/azure-iot-test-only.root.ca.cert.pem -showcerts
Downstream device running on Linux VM with Certs installed and verified. My connection string is as follows:
HostName={IoTHubName}.azure-devices.net;DeviceId=TC51_EdgeDownStreamDevice01;SharedAccessKey={My-Shared-Access-Key}=GatewayHostName={my-gateway-machine-name-dns-name}.centralus.cloudapp.azure.com
a. I have verified I get a successful verification of the SSL cert using the openssl tool.
b. I'm using the the following in my downstream device for my connection using the NodeJS SDK
var client = DeviceClient.fromConnectionString(connectionString, Mqtt);
c. I can see the messages showing up at the Azure IoT Hub in the Cloud, but I can't get my module running on the IoT Edge Transparent Gateway to be hit.
Here are my routing rules configured for the edgeHub as specified in "Routing messages from downstream devices" in the sample doc page.
This is what the example docs show:
{ "routes":{ "sensorToAIInsightsInput1":"FROM /messages/* WHERE NOT IS_DEFINED($connectionModuleId) INTO BrokeredEndpoint(\"/modules/ai_insights/inputs/input1\")", "AIInsightsToIoTHub":"FROM /messages/modules/ai_insights/outputs/output1 INTO $upstream" } }
This is what my routing configuration is set to:
"routes": {
"downstreamBatterySensorToBatteryDataFunctionInput1": "FROM /* WHERE NOT IS_DEFINED($connectionModuleId) INTO BrokeredEndpoint(\"/modules/BatteryDataFunctionModule/inputs/input1\")",
"BatteryDataFunctionModuleToIoTHub": "FROM /messages/modules/BatteryDataFunctionModule/outputs/* INTO $upstream"
}
** Note that I've used by "FROM /* WHERE NOT IS_DEFINED" and "FROM /messages/* WHERE NOT IS_DEFINED"
My module on the IoT Edge is setup as a Function. When I use the out of the box example where the simulator device is another module running on the IoT Edge, then my function is hit correctly. Its only when I'm trying to use a "Downstream Device" that the module is not being triggered.
I have enabled "Debug Logging for the IoT Edge Service" running on my Transparent Gateway.
This is the basic Run method for the Function module:
#r "Microsoft.Azure.Devices.Client"
#r "Newtonsoft.Json"
using System.IO;
using Microsoft.Azure.Devices.Client;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
// Filter messages based on the temperature value in the body of the message and the temperature threshold value.
public static async Task Run(Message messageReceived, IAsyncCollector<Message> output, TraceWriter log)
{
How can I figure out how to get my Module running in IoT Edge to be hit/triggered from a Downstream device?
So, you say you are seeing messages show up in IoT Hub, but not in Edge... A couple of things:
you posted this as your connection string in your node app:
HostName={IoTHubName}.azure-devices.net;DeviceId=TC51_EdgeDownStreamDevice01;SharedAccessKey={My-Shared-Access-Key}=GatewayHostName={my-gateway-machine-name-dns-name}.centralus.cloudapp.azure.com
Did you copy/paste this exactly? the reason I ask is that, between the shared access key and the word "GatewayHostName", you have an equals sign and not a semi-colon..
it should be:
HostName={IoTHubName}.azure-devices.net;DeviceId=TC51_EdgeDownStreamDevice01;SharedAccessKey={My-Shared-Access-Key};GatewayHostName={my-gateway-machine-name-dns-name}.centralus.cloudapp.azure.com
(note the ';' before GatewayHostName… if you really did have an equals sign there instead of a semicolon, there's no telling what kind of chaos that would cause :-)
Secondly, in your route, you call your module BatteryDataFunctionModule.. just want to make sure that module name is exact, including being case-sensitive. You probably know that, but don't want to assume..
Finally, if the two things above check out, can you add an addition debugging route that sends the 'incoming data' to IoTHub as well..
"FROM /* WHERE NOT IS_DEFINED($connectionModuleId) INTO $upstream"
so we can make sure the messages are actually making it through iot edge.
There are 2 problems that needed to be addressed to get the Downstream Device to communication
Thanks to #Steve-Busby-Msft I needed to have a semi-colon (;) at the end of the SharedAccessKey and before the GatewayHostName
you posted this as your connection string in your node app: HostName={IoTHubName}.azure-devices.net;DeviceId=TC51_EdgeDownStreamDevice01;SharedAccessKey={My-Shared-Access-Key}=GatewayHostName={my-gateway-machine-name-dns-name}.centralus.cloudapp.azure.com
The NodeJS application Downstream Device also has to load up the cert correctly at the 'Application level'.
Notice the section of code for
var edge_ca_cert_path = '[Path to Edge CA certificate]';
Node JS Downstream Application
'use strict';
var fs = require('fs');
var Protocol = require('azure-iot-device-mqtt').Mqtt;
// Uncomment one of these transports and then change it in fromConnectionString to test other transports
// var Protocol = require('azure-iot-device-http').Http;
// var Protocol = require('azure-iot-device-amqp').Amqp;
var Client = require('azure-iot-device').Client;
var Message = require('azure-iot-device').Message;
// 1) Obtain the connection string for your downstream device and to it
// append this string GatewayHostName=<edge device hostname>;
// 2) The edge device hostname is the hostname set in the config.yaml of the Edge device
// to which this sample will connect to.
//
// The resulting string should look like the following
// "HostName=<iothub_host_name>;DeviceId=<device_id>;SharedAccessKey=<device_key>;GatewayHostName=<edge device hostname>"
var connectionString = '[Downstream device IoT Edge connection string]';
// Path to the Edge "owner" root CA certificate
var edge_ca_cert_path = '[Path to Edge CA certificate]';
// fromConnectionString must specify a transport constructor, coming from any transport package.
var client = Client.fromConnectionString(connectionString, Protocol);
var connectCallback = function (err) {
if (err) {
console.error('Could not connect: ' + err.message);
} else {
console.log('Client connected');
client.on('message', function (msg) {
console.log('Id: ' + msg.messageId + ' Body: ' + msg.data);
// When using MQTT the following line is a no-op.
client.complete(msg, printResultFor('completed'));
// The AMQP and HTTP transports also have the notion of completing, rejecting or abandoning the message.
// When completing a message, the service that sent the C2D message is notified that the message has been processed.
// When rejecting a message, the service that sent the C2D message is notified that the message won't be processed by the device. the method to use is client.reject(msg, callback).
// When abandoning the message, IoT Hub will immediately try to resend it. The method to use is client.abandon(msg, callback).
// MQTT is simpler: it accepts the message by default, and doesn't support rejecting or abandoning a message.
});
// Create a message and send it to the IoT Hub every second
var sendInterval = setInterval(function () {
var windSpeed = 10 + (Math.random() * 4); // range: [10, 14]
var temperature = 20 + (Math.random() * 10); // range: [20, 30]
var humidity = 60 + (Math.random() * 20); // range: [60, 80]
var data = JSON.stringify({ deviceId: 'myFirstDownstreamDevice', windSpeed: windSpeed, temperature: temperature, humidity: humidity });
var message = new Message(data);
message.properties.add('temperatureAlert', (temperature > 28) ? 'true' : 'false');
console.log('Sending message: ' + message.getData());
client.sendEvent(message, printResultFor('send'));
}, 2000);
client.on('error', function (err) {
console.error(err.message);
});
client.on('disconnect', function () {
clearInterval(sendInterval);
client.removeAllListeners();
client.open(connectCallback);
});
}
};
// Provide the Azure IoT device client via setOptions with the X509
// Edge root CA certificate that was used to setup the Edge runtime
var options = {
ca : fs.readFileSync(edge_ca_cert_path, 'utf-8'),
};
client.setOptions(options, function(err) {
if (err) {
console.log('SetOptions Error: ' + err);
} else {
client.open(connectCallback);
}
});

Node.JS node-gcm-Service only sends to one device

I am using node-gcm-service package to send message from my server to the registered devices. When there is one device, the message is successfully delivered, when there are two devices, only the first device (deviceIds[0]) receives the message. Both devices receive messages if they are the only device to which the message is sent or either device is the first device in the deviceIds array. Any ideas what might be going on? Below is the relevant code snippet.
Thank You,
Gary
function(cb) {
var gcmSender = new gcm.Sender();
gcmSender.setAPIKey("api-key");
var gcmMessage = new gcm.Message({
collapse_key: "floomit",
data: {
message:"new_photo",
user:user,
stream:streams[0].name
},
delay_while_idle:true,
time_to_live:34,
dry_run:false
});
gcmSender.sendMessage(
gcmMessage.toString(),
deviceIds,
true,
cb
);
}
Try this snippet hope it'll solve your issue...
i tried with following code it's work fine with all devices.
var gcm = require('node-gcm-service');
function (cb){
var gcmSender = new gcm.Sender();
gcmSender.setAPIKey("google_api_key");
var gcmMessage = new gcm.Message({
collapse_key: "floomit",
data: {
message:"new_photo",
user:user,
stream:streams[0].name
},
delay_while_idle:true,
time_to_live:34,
dry_run:false
});
var registrationIds = [];
registrationIds.push('xxxxxxxxxx1');
registrationIds.push('xxxxxxxxxxx2');
console.info(registrationIds);
for(var i=0;i<registrationIds.length;i++){
gcmSender.sendMessage(gcmMessage.toString(), registrationIds[i], true, function(err, data) {
if (!err) {
// do something
console.info("data",JSON.stringify(data));
} else {
// handle error
console.info("error",JSON.stringify(err));
}
});
}
}
The issue seems to be with the GCM service itself. For 2 devices, the 2nd device eventually received the message, not sure how many hours later. When I had 3 devices, 2 of them received the notification right away, the 3rd one, hasn't received it yet, after more than 30 minutes.

how can I make private chat rooms with sockjs?

I am trying to make a chat system where only two users are able to talk to each other at a time ( much like facebook's chat )
I've tried multiplexing, using mongoDB's _id as the name so every channel is unique.
The problem I'm facing is that I cannot direct a message to a single client connection.
this is the client side code that first sends the message
$scope.sendMessage = function() {
specificChannel.send(message)
$scope.messageText = '';
};
this is the server side receiving the message
specificChannel.on('connection', function (conn) {
conn.on('data', function(message){
conn.write('message')
}
}
When I send a message, to any channel, every channel still receives the message.
How can I make it so that each client only listens to the messages sent to a specific channel?
It appeared that SockJS doesn't support "private" channels. I used the following solution for a similar issue:
var channel_id = 'my-very-private-channel'
var connection = new SockJS('/pubsub', '')
connection.onopen = function(){
connection.send({'method': 'set-channel', 'data': {'channel': channel_id}})
}
Backend solution is specific for every technology stack so I can't give a universal solution here. General idea is the following:
1) Parse the message in "on_message" function to find the requested "method name"
2) If the method is "set-channel" -> set the "self.channel" to this value
3) Broadcast further messages to subscribers with the same channel (I'm using Redis for that, but it also depends on your platform)
Hope it helps!

Pusher binding to events regardless of channel

I am attempting to listen to a particular event type regardless of the channel it was triggered in. My understanding of the docs (http://pusher.com/docs/client_api_guide/client_events#bind-events/lang=js) was that I can do so by calling the bind method on the pusher instance rather than on a channel instance. Here is my code:
var pusher = new Pusher('MYSECRETAPPKEY', {'encrypted':true}); // Replace with your app key
var eventName = 'new-comment';
var callback = function(data) {
// add comment into page
console.log(data);
};
pusher.bind(eventName, callback);
I then used the Event Creator tool in my account portal to generate an event. I used a random channel name, set the Event to "new-comment" and just put in some random piece of text into the Event Data. But, I am getting nothing appearing in my Console.
I am using https://d3dy5gmtp8yhk7.cloudfront.net/2.1/pusher.min.js, and performing this test in the latest Chrome.
What am I missing?
Thanks!
Shaheeb R.
Pusher will only send events to the client if that client has subscribed to the channel. So, the first thing you need to do is subscribe the channel. Binding to the event on the client:
pusher.bind('event_name', function( data ) {
// handle update
} );
This is also known as "global event binding".
I've tested this using this code and it does work:
http://jsbin.com/AROvEDO/1/edit
For completeness, here's the code:
var pusher = new Pusher('APP_KEY');
var channel = pusher.subscribe('test_channel');
pusher.bind('my_event', function(data) {
alert(data.message);
});

Resources