I'm facing a problem with a tcp server for IoT devices, I'm using a stream approach implementing a transform class to process incoming data (the device class).
this.server = net.createServer(async function onConnection(socket) {
const device = new gateway.DeviceClass(gateway);
socket.setTimeout(0);
socket
.pipe(device)
then on close event I disconnect the stream processor and destroy the socket
socket.once('close', ()=>{
this.socket.unpipe();
this.socket.removeAllListeners();
this.socket.destroy();
this.socket.unref();
})
The problem is that sometime the client is still sending data? and I got a closed stream write error. Also sometime it seems I'm missing data that is transmitted but never received?
from the documentation close event is emitted "Emitted once the socket is fully closed. The argument hadError is a boolean which says if the socket was closed due to a transmission error." so I expect no more data is sent over it.
Related
I have a React.js client app and a Node.js server app and the Node.js app receives json data in real time via socket.io from another microservice. The JSON data is sent very often and this breaks the client app. For example:
I stop the server but the client still receives data
If I try refreshing the browser, it takes a lot of time to refresh
It also used to disconnect and reconnect the sockets (I fixed this by increasing the pingTimeout but that did not solve the other problems)
I also used maxHttpBufferSize and updateTimeout by increasing them but that does not really help. Decreasing the maxHttpBufferSize stops the messages from being received but I want them to be received just in a manner which does not break my client application.
Any advices on what I can do to improve my situation?
EDIT:
It could also work if I do not send all messages but skip every second or so but I am not sure how to achieve this?
Backpressure can be implemented with acknowledgements:
the client notifies the server when it has successfully handled the packet
socket.on("my-event", (data, cb) => {
// do something with the data
// then call the callback
cb();
});
the server must wait for the acknowledgement before sending more packets
io.on("connection", (socket) => {
socket.emit("my-event", data, () => {
// the client has acknowledged the packet, we can continue
});
})
Reference: https://socket.io/docs/v4/emitting-events/#acknowledgements
Note: using volatile packets won't work here, because the amount of data buffered on the server is not taken in account
Reference: https://github.com/websockets/ws/blob/master/doc/ws.md#websocketbufferedamount
I have a smartcard reader. It sends the read data to the process through an event emitter called emitter (which extends from Nodejs' native events module). And I am emitting the received String data to the frontend using Socket.io:
emitter.on('cardReceived', (studentRfid) => {
socket.emit('attended', studentRfid);
console.log('Attended!');
});
The problem is: when the card is read, the log Attended! is immediately printed on the screen. But the data comes to the frontend with a delay (about 5 seconds).
Initially, I thought the problem was with the network/socket.
However, I have tried to emit some data as fast as in intervals of 10ms to the frontend (outside of the 'on' listener), surprisingly it did not cause any delays. The data transfer was perfectly in sync:
setInterval(() => {
socket.emit('message', 'some message');
}, 10 );
So the problem is not with Socket.io or network (it is super fast as you can see above).
What may cause these long delays?
I am very eager to integrate Socket.io to my node.js project, but I have some confusions on how to properly use socket.io. I have been looking through documentations and tutorials, but I have not been able to understand some concepts on how socket.io works.
The scenario I have in mind is the following:
There are multiple clients C1, C2, ..., Cn
Clients emit request to the server R1,...,Rn
Server receives request, does data processing
When data-processing is complete, Server emits response to clients Rs1, .., Rs2
The confusion I have in this scenario is that, when the server has finished data processing it emits the response in the following way:
// server listens for request from client
socket.on('request_from_client', function(data){
// user data and request_type is stored in the data variable
var user = data.user.id
var action = data.action
// server does data processing
do_some_action(..., function(rData){
// when the processing is completed, the response data is emitted as a response_event
// The problem is here, how to make sure that the response data goes to the right client
socket.emit('response_to_client', rData)
})
})
But here I have NOT defined which client I am sending the response to!
How does socket.io handle this ?
How does socket.io make sure that: response Rs1 is sent to C1 ?
What is making sure that: response Rs1 is not sent to C2 ?
I hope I have well explained my doubts.
The instance of the socket object corresponds to a client connection. So every message you emit from that instance is send to the client that opened that socket connection. Remember that upon the connection event you get (through the onDone callback) the socket connection object. This event triggers everytime a client connects to the socket.io server.
If you want to send a message to all clients you can use
io.sockets.emit("message-to-all-clients")
and if you want to send an event to every client apart the one that emits the event
socket.broadcast.emit("message-to-all-other-clients");
I have NodeJs handling incoming tcp connections from GPRS devices.
My question is, should I use socket.pipe(socket) at the scope of net.createServer(...) ?
Is calling this pipe() in this form that allow duplex communication, i.e. gprs->node and node->gprs? Or can I avoid calling this method?
You didn't share your code, but generally speaking
pipe() is not necessary for achieving duplex communication
as tcp socket are by nature bi-directional.
net.createServer(function (gprsSocket) {
gprsSocket.on('data', function (data) {
// incoming data
// every time the GPRS is writing data - this event is emited
})
// outgoing data
// call this whenever you want to send data to
// the GPRS regardless to incoming data
gprsSocket.write('hello\n')
})
Answering your question - no, its not necessary.
In-fact calling socket.pipe(socket) will send all data received back to the GPRS.
and is basically doing something like this (though is not exactly the same)
gprsSocket.on('data', function (data) {
// echo the data back to the gprs
gprsSocket.write(data);
})
pipe() is used when you want to redirect one stream to another
// redirect all data to stdout
gprsSocket.pipe(process.stdout)
I can't figure out one problem I got.
I'm using the Net module on my Node.JS server which is used to listen to client connections.
The client do connect to the server correctly and the connection remains available to read/write data. So far, so good. But when the client unexpectedly disconnects (ed. when internet falls away at client side) I want to fire an event server side.
In socket.io it would be done with the 'disconnect' event, but this event doesn't seem to exist for the Net module. How is it possible to do?
I've searched on Google/StackOverflow and in the Net documentation (https://nodejs.org/api/net.html) but I couldn't find anything usefull. I'm sry if I did mis something.
Here is a code snippet I got:
var net = require('net');
var server = net.createServer(function(connection) {
console.log('client connected');
connection.wildcard = false;//Connection must be initialised with a configuration stored in the database
connection.bidirectional = true;//When piped this connection will be configured as bidirectional
connection.setKeepAlive(true, 500);
connection.setTimeout(3000);
connection.on('close', function (){
console.log('Socket is closed');
});
connection.on('error', function (err) {
console.log('An error happened in connection' + err.stack);
});
connection.on('end', function () {
console.log('Socket did disconnect');
});
connection.on('timeout', function () {
console.log('Socket did timeout');
connection.end();
});
connection.on('data', function (data) {
//Handling incoming data
});
});
serverUmrs.listen(40000, function () {
console.log('server is listening');
});
All the events(close, end, error, timeout) don't fire when I disconnect the client(by pulling out the UTP cable).
Thanks in advance!
EDIT:
I did add a timeout event in the code here above but the only thing that happens is that the socket does timeout after 3 seconds everytime the client does connect again. Isn't KeepAlive enough to make the socket not Idle? How is it possible to make the socket not idle without to much overhead. It may be possible that there are more than 10,000 connections at the same time which must remain alive as long as they are connected (ie respond to the keepalive message).
Update:
I think the KeepAlive is not related with the Idle state of socket, sort of.
Here is my test, I remove the following code in your example.
//connection.setKeepAlive(true, 500);
Then test this server with one client connect to it var nc localhost 40000. If there is no message sending to server after 3 seconds, the server logs as below
Socket did timeout
Socket did disconnect
Socket is closed
The timeout event is triggered without KeepAlive setting.
Do further investigation, refer to the Node.js code
function onread(nread, buffer) {
//...
self._unrefTimer();
We know timeout event is triggered by onread() operation of socket. Namely, if there is no read operation after 3 seconds, the timeout event will be emitted. To be more precisely, not only onread but also write successfully will call _unrefTimer().
In summary, when the write or read operation on the socket, it is NOT idle.
Actually, the close event is used to detect the client connection is alive or not, also mentioned in this SO question.
Emitted when the server closes. Note that if connections exist, this event is not emitted until all connections are ended.
However, in your case
disconnect the client(by pulling out the UTP cable).
The timeout event should be used to detective the connection inactivity. This is only to notify that the socket has been idle. The user must manually close the connection. Please refer to this question.
In TCP connection, end event fire when the client sends 'FIN' message to the server.
If the client side is not sending 'FIN' message that event is not firing.
For example, in your situation,
But when the client unexpectedly disconnects (ed. when internet falls away at client side) I want to fire an event server side.
There may not be a 'FIN' message because internet is gone.
So you should handle this situation in timeout without using keepAlive. If there is no data coming data, you should end or destroy the socket.
EDIT: I did add a timeout event in the code here above but the only
thing that happens is that the socket does timeout after 3 seconds
everytime the client does connect again. Isn't KeepAlive enough to
make the socket not Idle? How is it possible to make the socket not
idle without to much overhead. It may be possible that there are more
than 10,000 connections at the same time which must remain alive as
long as they are connected (ie respond to the keepalive message).
For your edit, your devices should send to the server some heartbeat message between a time period. So that, server understands that that device is alive and that timeout event will not fire because you get some data. If there is no heartbeat message such cases you cannot handle this problem.