I have a node.js script which allows a client to connect and receive some realtime data from an external script.
I have just upgraded node.js & socket.io to the current versions (from <0.9) and am trying to get to grips with what happens when a client quits, times out or disconnects from the server.
Here is my current node.js script;
var options = {
allowUpgrades: true,
pingTimeout: 50000,
pingInterval: 25000,
cookie: 'k1'
};
var io = require('socket.io')(8002, options);
cp = require('child_process');
var tail = cp.spawn('test-scripts/k1.rb');
//On connection do the code below//
io.on('connection', function(socket) {
console.log('************ new client connected ****************', io.engine.clientsCount);
//Read from mongodb//
var connection_string = '127.0.0.1:27017/k1-test';
var mongojs = require('mongojs');
var db = mongojs(connection_string, ['k1']);
var k1 = db.collection('k1');
db.k1.find({}, {'_id': 0, "data.time":0}).forEach(function(err, doc) {
if (err) throw err;
if (doc) { socket.emit('k1', doc); }
});
//Run Ruby script & Listen to STDOUT//
tail.stdout.on('data', function(chunk) {
var closer = chunk.toString()
var sampArray = closer.split('\n');
for (var i = 0; i < sampArray.length; i++) {
try {
var newObj = JSON.parse(sampArray[i]);
// DO SOCKET //
socket.emit('k1', newObj);
} catch (err) {}
}
});
socket.on('disconnect', function(){
console.log('****************** user disconnected *******************', socket.id, io.engine.clientsCount);
socket.disconnect();
});
});
In the old version of socket.io when a client exits I get the following logged in debug;
info - transport end (undefined)
debug - set close timeout for client Owb_B6I0ZEIXf6vOF_b-
debug - cleared close timeout for client Owb_B6I0ZEIXf6vOF_b-
debug - cleared heartbeat interval for client Owb_B6I0ZEIXf6vOF_b-
debug - discarding transport
then everything goes quite and all is well.
With the new (1.3.7) version of socket.io when a client exits I get the following logged in debug;
socket.io:client client close with reason transport close +2s
socket.io:socket closing socket - reason transport close +1ms
socket.io:client ignoring remove for -0BK2XTmK98svWTNAAAA +1ms
****************** user disconnected ******************* -0BK2XTmK98svWTNAAAA
note the line socket.io:client ignoring remove for -0BK2XTmK98svWTNAAAA
but after that and with no other clients connected to the server I'm still seeing it trying to write data to a client that already left. (in the example below this is what I get after I've had 2 clients connected, both of which have since disconnected.
socket.io:client ignoring packet write {"type":2,"data":["k1",{"item":"switch2","datapoint":{"type":"SWITCH","state":"0"}}],"nsp":"/"} +1ms
socket.io:client ignoring packet write {"type":2,"data":["k1",{"item":"switch2","datapoint":{"type":"SWITCH","state":"0"}}],"nsp":"/"} +3ms
I'm trying to stop this apparently new behaviour so that once a client has disconnected and the server is idle its not still trying to send data out.
I've been playing about with socket.disconnect and delete socket["id"] but I'm still left with the same thing.
I tried with io.close() which sort of worked - it booted any clients who where actually connected and made them re-connect but still left the server sitting there trying to send updates to the client that had left.
Am I missing something obvious, or has there been a change in the way this is done with the new version of socket.io? There is nothing in the migration doc about this. The only other result I found was this bug report from June 2014 which has been marked as closed. From my reading of it - it appears to be the same problem I'm having but with the current version.
Update: I've done some more testing and added io.engine.clientsCount to both instances of console.log to track what it's doing. It appears when I connect 1 client it gives me 1 (as expected) and when I close that client it changes to 0 (as expected) this leads me to believe that the client connection has been closed and engine.io know this. So why am I still seeing all the 'ignoring packet write' lines and more with every client who has disconnected.
Update 2: I've updated the code above to include the parser section and the DB section - this represents the full node script as there was a thought that I may need to clean up my own clients. I have tried adding the following code to the script in the hope it would but alas not :(
In the connection event I added clients[socket.id] = socket; and the disconnection event I added delete clients[socket.id]; but it didn't change anything (that I could see)
Update 3: Answer thanks to #robertklep It was an 'event handler leak' that I was actually looking for. Having found that I also found this post.
My guess is that the newer socket.io is just showing you (by way of debug messages) a situation that was already happening in the old socket.io, where it just wasn't being logged.
I think the main issue is this setup:
var tail = cp.spawn('test-scripts/k1.rb');
io.on('connection', function(socket) {
...
tail.stdout.on('data', function(chunk) { ... });
...
});
This adds a new handler for each incoming connection. However, these won't miraculously disappear once the socket is disconnected, so they keep on trying to push new data through the socket (whether it's disconnected or not). It's basically an event handler leak, as they aren't getting cleaned up.
To clean up the handlers, you need to keep a reference to the handler function and remove it as a listener in the disconnect event handler:
var handler = function(chunk) { ... }:
tail.stdout.on('data', handler)
socket.on('disconnect', function() {
tail.stdout.removeListener('data', handler);
});
There's also a (slight) chance that you will get ignored packet writes from your MongoDB code, if the socket is closed before the forEach() has finished, but that may be acceptable (since the amount of data is finite).
PS: eventually, you should consider moving the processing code (what handler is doing) to outside the socket code, as it's now being run for each connected socket. You can create a separate event emitter instance that will emit the processed data, and subscribe to that from each new socket connection (and unsubscribe again when they disconnect), so they only have to pass the processed data to the clients.
This is most probably due to your connection is established via polling transport, which is sooo painful for developer. The reason is that this transport uses timeout to determine if the client is here or not.
The behavior you see is due to the client has left but next polling session opening moment has not come yet, and due to it server still thinks that client "it out there".
I have tried to "fight" this problem in many ways (like adding a custom onbeforeunload event on client side to force disconnect) but they all just do not work in 100% cases when polling is used as transport.
Related
I need the socket connection to establish before I can send data from the route to the user (otherwise it is not possible).
In app.js file I have socket connection logic:
app.use(function(req, res, next)
{
req.sio = sio;
next();
});
sio.on('connection',
function(soc)
{
console.log('socket connected');
soc.on('disconnect', function(reason)
{
console.log('socket disconnected');
});
// and more about socket connection here...
});
In index.js file I have route.post logic:
router.post('/route1', function(req, res, next) // user is moved from index.js to route1.js if he fills the form
{
var fromInput = req.body.form_name;
console.log('DATA passed from INDEX.JS: ' + formInput);
if ((formInput !== '') && (formInput !== null) && (formInput !== undefined))
{
function render()
{
//// first we render the page, so the javascript (with socket.io notes) can be read it and then the browser know that socket connection should be established
return new Promise(function(resolve, reject)
{
resolve(res.render('route1'));
});
}
// I need to pass some data AFTER the socked connection is established - cause I move to a different page path - using a **router.post** and cause of that socket connection is disconnected - so I need to wait till its usable again. For simplicity let suppose socket connection is established after 2 seconds (it is a simple check for req.soc.connected):
var soc = false;
setTimeout(function()
{
soc = true; // after 2 sec soc is true (connection is established)
}, 2000);
// Now I want to create an interval that will monitor IF socket connection is established every 100ms (so checking won't happen to often - it is not "resource hungry"). If socket connection is not ready the function should call it self (recursion) if the socket connection is established it (function) should fire a promise.
var arr = [];
arr.push(exe(100, data));
function exe(delay, d)
{
d = data;
return new Promise(function(resolve)
{
if (d === false)
{
setTimeout(function()
{
console.log('wait another ' + delay + ' [ms] - ' + d);
return resolve(exe(delay, d));
}, delay);
}
else
{
console.log('socket connected!');
return resolve(d);
}
});
}
render().then(function()
{
return Promise.all(arr).then(function(arr)
{
console.log(arr);
});
}).then(function()
{
console.log('ALL DONE!');
});
}
});
Comment are in code. If something isn't clear let me know.
#jfriend00
1 - true,
2 - true,
3 - I call render() immediately - so page is loaded and client make a socket connection, then the rest of the code should execute and send the data.
yes I did use POST with a form. There could be socket connection between the server and index page - not a problem I can create one, but I dunno what for.
"or there could be a socket.io connection created in the response to the POST when the browser renders and processes that." I'm trying that one :) I have data in this router.post I want to sent with help of sockets - but first I need to make a connection.
as I understand it... user did use form, so path is changed (socket connection is broken), then I'm in router.post I render the page FIRST - so the browser can read it's JS and make a socket connection, BUT you want to say that my response is not finished? So the browser say - ok you want me to render a page, but what now - cause we are NOT finish yet?!
So I will never establish a socket connection, cause I did not properly response? And cause of this I will not be able to send the data (later code in router.post) cause socket connection is never established cause I did not response properly? Cause my tests show me otherwise - it is working just fine.
you are right - code should works now.
till socket connection is established.
yea, good catch. I will make some kind of database - redis with express session I guess.
So again step by step.
User did fill the form so he is redirect from index.js to route1.js (so it does not make a difference if there is a socket connection BEFORE filling the form or not cause the connection is lost). We are in process of redirecting him (router.post) so I thought I will render the route1 page immediately, so the JS from it can be read by browser, and socket connection can be established (which take time - and IF its possible). So I wait with the data I want to sent to the user (in router.post for example... the form input or whatever) TILL the connection is established, and the send it to the user, with help of socket.io.
The thing is that socket io connection is lost when you change page (path). So I thought (and it could be wrong cause I'm newb) then I wait till it is established, and then send the data. I hope it does make sense.
This structure can never work. Here's what it looks like you're trying to do:
Express server receives a POST request.
Then, you try to wait for a socket.io connection to appear before you process the POST and send a response.
Finally, when you think you've found a socket.io connection, you then call your render() function to "presumably" send a response.
Without really understanding what you're' trying to accompilsh, there are a number of things wrong with the current code:
A POST request comes from either an Ajax call or a form POST. There is no socket.io connection associated directly with either one of those. There could have been a socket.io connection when the page loaded BEFORE the POST request was sent or there could be a socket.io connection created in the response to the POST when the browser renders and processes that.
Even if there was a socket.io connection created when the browser processes the POST response, you're trying to wait for the socket.io connection BEFORE you send the response so you're waiting for something that won't happen until you're done waiting (essentially a deadlock - A won't finish until B finishes, but B can't start until A finishes).
This structure render().then(waitUntil(100, d)) isn't correct. You MUST pass .then() a function reference. You are passing it a promise (the return value form calling waitUntil(...)). This is the least of your problems though because the overall structure of what you're trying to do is wrong.
The whole implementation of waitUntil() is confused and I can't even tell what it's trying to actually wait for.
This is a server that can field lots of connections from lots of clients. You can't just wait for the "next" socket.io connection and assume that connection is from the client you just got a request for. The only way to associate a socket.io connection with an http request is to use some identifying characteristic in both (usually a cookie) and then in the http request, you get the cookie and look up the cookie to see if you currently have a socket.io connection that matches that cookie. This is something that express-socket.io-session helps with.
Unfortunately, you don't describe what you're really trying to accomplish here so I can't point you to a good solution. All, I can really say here is that this scheme will not work. If you want further help with the actual problem, please edit your question to include the problem description in words (not your coding issues). Show the exact sequence of events you want to happen and explain what you're trying to accomplish and why.
EDIT: I see that I'm getting ping timeout and transport error reasons in my handler for disconnect on the server. This makes it difficult to maintain state in my server (I'm trying to keep track of which users are connected in a chat-like setup(. I was reading that it may be related to background tabs in Chrome (which I'm running). Does anyone have any experience with these 'spurious' disconnect events?
I'm new to Socket.io and am having some trouble understanding the connection and disconnection process.
As I understand it, the server receives the connection event once when a client connects, and one registers all the handlers for that client in the callback on on.('connection'). Is that true?
I want to maintain an of connected users, so I add a user to that array on the connection handler.
Should I then listen for the disconnect event to know when to remove a user from that array? Can I be guaranteed that that event will only be fired once?
It's a bit confusing, because on the client side, there is the connect event, which apparently can be fired multiple times -- the documentation says
// note: you should register event handlers outside of connect,
// so they are not registered again on reconnection
which is a different paradigm than on the server side, where all the handlers are registered inside the connection handler. But if the client-side connect event can fire on re-connection, what is the reconnect event for? (The docs says this event is "Fired upon a successful reconnection.")
In general I'm confused about the process of connection, disconnection and re-connection and how this relates to events, whether it happens "randomly" due to connection issues or only under the programmer's control, and how many times one should anticipate receiving each of these events -- once only for server, multiple times for client?
Thanks for any help!
I'm new to Socket.io and am having some trouble understanding the
connection and disconnection process.
Welcome to the wonderful world of Node.js + Socket.io It's super powerful!
As I understand it, the server receives the connection event once when
a client connects, and one registers all the handlers for that client
in the callback on on.('connection'). Is that true?
Correct. Take a look at this example of my code:
Server-side
var clients = []; /* stores all sockets on the fly */
io.on('connection', function (socket) {
clients[socket.id] = socket; /* keeps an array of sockets currently connected */
socket.on('disconnect', function (data) {
console.log(socket.id + " disconnected");
delete clients[socket.id];
});
});
Client-side
socket = io.connect(YOUR_SOCKET_URI, { transports: ['websocket'] } );
socket_delegates();
socket_delegates = function() {
// Socket events
socket.on('connect', function(data) {
/* handle on connect events */
});
socket.on('disconnect', function () {
/* handle disconnect events - possibly reconnect? */
});
socket.on('reconnect', function () {
/* handle reconnect events */
});
socket.on('reconnect_error', function () {
/* handle reconnect error events - possible retry? */
});
}
Should I then listen for the disconnect event to know when to remove a
user from that array? Can I be guaranteed that that event will only be
fired once?
Yes. You will see in the above server code that we listen for disconnect and then do what we need to.
Nothing should be random. You should have code in place to handle the connect, disconnect on the server side and code to handle the connect, disconnect and reconnect on the client side.
I am currently writing an Electron app, for which in the main process I want to talk to a server over websockets.
Part of my main process' code base depends on the user's socket connection status. I require the socket.io-client library
const socket = require('socket.io-client')(host);
which gives me access to the variable socket.connected, which is true or false according to whether a connection is established to the server.
Thus, what I would like is to attach an event emitter to this variable. I have made it to work with polling
var events = require('events').EventEmitter;
var event = new events();
// Emits successfully every 200ms 'Connection: true|false'
event.on('status', function(status){
console.log('Connection: ', status);
});
setInterval(function() {
let status = socket.connected;
event.emit('status', status);
}, 200);
but was wondering whether this truly is the way to implement it. To me it seems strange to have to resort to polling in an async framework like nodejs. On the other, I could not find other ways to implement. Best-case scenario would be to attach somehow an event emitter directly to socket.connected, but was unable to find how to do that. Could anybody advise me on a better way to implement?
Thanks
You can get notified of the completion of a client connection with the connect event:
socket.on('connect', function() {
// client socket is now connected to the server
});
socket.on('disconnect', function() {
// client socket is now disconnected from the server
});
Documentation for client events here: http://socket.io/docs/client-api/#manager(url:string,-opts:object). There are other events in that doc if you also want to see other things like a reconnect.
I can't figure out one problem I got.
I'm using the Net module on my Node.JS server which is used to listen to client connections.
The client do connect to the server correctly and the connection remains available to read/write data. So far, so good. But when the client unexpectedly disconnects (ed. when internet falls away at client side) I want to fire an event server side.
In socket.io it would be done with the 'disconnect' event, but this event doesn't seem to exist for the Net module. How is it possible to do?
I've searched on Google/StackOverflow and in the Net documentation (https://nodejs.org/api/net.html) but I couldn't find anything usefull. I'm sry if I did mis something.
Here is a code snippet I got:
var net = require('net');
var server = net.createServer(function(connection) {
console.log('client connected');
connection.wildcard = false;//Connection must be initialised with a configuration stored in the database
connection.bidirectional = true;//When piped this connection will be configured as bidirectional
connection.setKeepAlive(true, 500);
connection.setTimeout(3000);
connection.on('close', function (){
console.log('Socket is closed');
});
connection.on('error', function (err) {
console.log('An error happened in connection' + err.stack);
});
connection.on('end', function () {
console.log('Socket did disconnect');
});
connection.on('timeout', function () {
console.log('Socket did timeout');
connection.end();
});
connection.on('data', function (data) {
//Handling incoming data
});
});
serverUmrs.listen(40000, function () {
console.log('server is listening');
});
All the events(close, end, error, timeout) don't fire when I disconnect the client(by pulling out the UTP cable).
Thanks in advance!
EDIT:
I did add a timeout event in the code here above but the only thing that happens is that the socket does timeout after 3 seconds everytime the client does connect again. Isn't KeepAlive enough to make the socket not Idle? How is it possible to make the socket not idle without to much overhead. It may be possible that there are more than 10,000 connections at the same time which must remain alive as long as they are connected (ie respond to the keepalive message).
Update:
I think the KeepAlive is not related with the Idle state of socket, sort of.
Here is my test, I remove the following code in your example.
//connection.setKeepAlive(true, 500);
Then test this server with one client connect to it var nc localhost 40000. If there is no message sending to server after 3 seconds, the server logs as below
Socket did timeout
Socket did disconnect
Socket is closed
The timeout event is triggered without KeepAlive setting.
Do further investigation, refer to the Node.js code
function onread(nread, buffer) {
//...
self._unrefTimer();
We know timeout event is triggered by onread() operation of socket. Namely, if there is no read operation after 3 seconds, the timeout event will be emitted. To be more precisely, not only onread but also write successfully will call _unrefTimer().
In summary, when the write or read operation on the socket, it is NOT idle.
Actually, the close event is used to detect the client connection is alive or not, also mentioned in this SO question.
Emitted when the server closes. Note that if connections exist, this event is not emitted until all connections are ended.
However, in your case
disconnect the client(by pulling out the UTP cable).
The timeout event should be used to detective the connection inactivity. This is only to notify that the socket has been idle. The user must manually close the connection. Please refer to this question.
In TCP connection, end event fire when the client sends 'FIN' message to the server.
If the client side is not sending 'FIN' message that event is not firing.
For example, in your situation,
But when the client unexpectedly disconnects (ed. when internet falls away at client side) I want to fire an event server side.
There may not be a 'FIN' message because internet is gone.
So you should handle this situation in timeout without using keepAlive. If there is no data coming data, you should end or destroy the socket.
EDIT: I did add a timeout event in the code here above but the only
thing that happens is that the socket does timeout after 3 seconds
everytime the client does connect again. Isn't KeepAlive enough to
make the socket not Idle? How is it possible to make the socket not
idle without to much overhead. It may be possible that there are more
than 10,000 connections at the same time which must remain alive as
long as they are connected (ie respond to the keepalive message).
For your edit, your devices should send to the server some heartbeat message between a time period. So that, server understands that that device is alive and that timeout event will not fire because you get some data. If there is no heartbeat message such cases you cannot handle this problem.
I need to send data to nodejs server via socket.io when the user closes the browser tab .
I tried doing :
var data={};
window.onbeforeunload = function() {
// i have a object to be sent
data.data1='abcd';
data.data2=1234;
socket.emit("senddata",data);
}
This code works when the user navigates around clicking links on the site but doesnot work when the user closes the browser tab
I also tried configuring the socket io on server side as below .. thinking the error may be due to socket connection being closed before emitting data:
var io = require('socket.io').listen(app.listen(port));
io.configure(function () {
io.set('close timeout',12000);
});
It also didnt work most of the time.
I also tried this on client side:
var socket = require('socket.io').listen(80, {
"sync disconnect on unload":false
});
It also did not work
I had tried receiving data like this
var io = require('socket.io').listen(app.listen(port));
io.sockets.on('connection', function (socket) {
socket.on('senddata', function (data) {
// data processing
});
});
please ..help me with this problem..thanks in advance..
When user connects - register on server side the time it happened.
Socket.IO has heart beating and pinging functionality. So just subscribe to disconnect event on server side and once it happens - means client disconnected - and you have on server time when client had connection. So that way you have timespan of client session.
Do not trust client to tell you any time or important data, as client can simply 'lie' - which leads to hacks/cheats/bugs on your server-side.
There is no reliable way to send data just before disconnect from client-side at all. There is nothing in Socket.IO for that, nor in just one of transports (WebSockets). As well as there is too many different scenarios of disconnection (power off, force close, ethernet cable out, wifi lose, etc).