I'm developing a simple website, where the client and server communicate over web sockets. I'm using nodejs and the socket.io library for the socket communication
Specifically, my server works as a middleware between an mqtt broker and my client. So on one hand, my server connects with the mqtt broker to consume messages and on the other hand delivers these messages to the connected clients over web sockets. I'm using the node mqtt library for the mqtt communication.
My codebase is fairly large, so to give you a feeling of how my code looks like, I will show this example, which should be straightforward to understand:
const io = require("socket.io")(port);
handleRequests(io) {
io.on("connection", (socket) => {
logger.info(`New client connected: ${socket.id}`);
this.clients[socket.id] = { // track clients and subscribed topics
topic: '',
};
this.numberOfUsers++;
io.sockets.emit("onUser", this.numberOfUsers);
this.handleChange(socket);
this.addToSubscribedClients(socket);
this.removeFromSubscribedClients(socket);
this.handleDisconnect(socket);
this.sendMqttMessageToClient(socket);
});
}
This is my "main" function, where as you can see, I'm initializing an io object and using it later by passing it to the handleRequests function. Each time a new client connects, I'm calling the callback function where I call the five other functions and passing the socket object as a parameter, which should be fine I guess. I'm passing the socket object as a parameter because I need it to later call socket.emit in order to send back message to a specific client, since the socket object is unique for each client.
This works great until more than ~ 30 clients are connected. I'm trying to debug this for 2 weeks now and can't figure out why this is happening. I'm testing this by opening multiple tabs in my browser. I start with one client and then increase the number of clients/tabs. At some points, I notice that some clients receive no values from the server but other clients still do, which is incorrect since all clients should receive the values in real time.
I noticed that the clients, which are not receiving values have other ids than the ones stored on the server. I tested this with a simple console.log() on both clients and server. How this is happening? I'm very positive that I'm sending the ids correctly since there are other clients, which still receive values from the server. My guess is that the server is somehow disconnecting some clients automatically, because if a client reconnects then a new id will be assigned to it automatically.
Does anyone have any idea why this is happening? and why it works fine with the first ~30 clients and starts to occur when many clients are connected? This issue is very hard to debug since the code works fine for a small number of clients and no errors are thrown when the bug occurs, so I'm hoping that someone had this before.
Edit
Now I just found that i can print a reason for socket disconnection. When I do that, ping timeout is printed, which I don't understand because when I have one single or few clients connected then this error does not happen.
Related
I have a dasboard that is making socket request every five seconds, sometimes, some users start getting data from other user socket request, but at the begging everything is working fine.
I have tried with sticky-session, diferrent socket instance, personalized socket event names.
if someone unsderstand my problem and i have a solution, i would be grateful.
Sockets are, by definition, separate from each other. I suspect the issue is that you're emitting to a namespace rather than to a particular socket.
io.of('someNamespace').emit('data');
vs
io.of('someNamespace').on('connection', (socket) => {
socket.emit('data');
});
In the first example we're sending data to all sockets in the namespace. In the second we're only sending data to a particular socket. The difference is in where you're emitting the data.
I'm trying to have 2 servers communicate with each other, I'm pretty new to websockets so its kind of confusing. Also, just to put it out there, i'm not trying to do this: websocket communication between servers;
My goal here is to basically use a socket to read data from another server (if this is possible?) I'll try to easily explain more below;
We'll assume there is a website called https://www.test.com (going to this website returns an object)
With a normal HTTP request, you would just do:
$.get('https://www.test.com').success(function (r) {
console.log(r)
})
And this would return r, which is an object thats something like this {test:'1'};
Now from what I understand with websockets, is that you cannot return data from them because you don't actually 'request' data, you just send data through said socket.
Since I know what test.com returns, and I know all of the headers that i'm going to need, is it possible to just open a socket with test.com and wait for that data to be changed without requesting it?
I understand how client-server communication works with socketio/websockets im just not sure if its possible to do server-server communication.
If anyone has any links to documentation or anything trying to help explain, it would be much appreciated, I just want to learn how this works. (or if its even possible)
Yes, I you can do what (assuming I understood your needs correctly). You can establish a websocket connection between two servers and then either side can just send data to the other. That will trigger an event at the other server and it will receive the sent data as part of that event. You can do this operation either direction from serverA to serverB or vice versa or both.
In node.js, everything is event driven. So, you would establish the webSocket connection and then just set up an event handler to be triggered when data arrives. The other server can then just send new data whenever it has updated data to send. This is referred to as the "push" model. So, rather than serverA asking serverB is it has any new data, you establish the webSocket connection and serverB just sends new data to serverA whenever that new data is available. Done correctly, this is both more efficient and more timely (as there is no polling interval and no cycles wasted asking for data when there is nothing new).
The identical model can be used between servers or client to server. The only difference with the client/server model is that the webSocket must be initially established client to server. With the server to server model, either server can initiate the connection.
You can think of a webSocket connection like establishing a phone call. Once the phone call is established, either side can just say something and the other end hears what they're saying. The webSocket connection is similar. Once its established, either side can just send some data to the other end and the other end will receive it. It's an open pipeline ready to have data sent either way. In node.js, when data arrives on that pipeline, it triggers and event so the listener will get that event and see the data that was sent.
Node create a unique url and bind socket.io to it.
var io = require("socket.io").listen(server, {path: req.originalUrl});
When a client connects also binds his socket.io-client to that url
var socket = io('192.168.1.101:3000', {path: window.location.pathname});
I don't have problems and everything works great.
When a client performs a particular action, server do
socket.broadcast.emit("foo"); //I made console.log here and it prints
client-side:
socket.on("foo", () => console.log("okay"));
The problem is that client-side "foo" event is almost never fired. Sometimes it is fired but only in particular occations. For example it happened that a socket.io-client auto-reconnect to server and then the event is fired.
I don't know if the problem is related to this, because for example socket.emit works, but when another client connects I always get this
error
socket.broadcast.emit("foo");
broadcasts a message to all connected clients EXCEPT the one specified by socket. If you want to broadcast to all connected clients, then use:
io.broadcast.emit("foo");
You will also have to make sure that your clients are correctly connected and not regularly losing and re-establishing their connections (you can see if that is happening by logging the connection and disconnection events on the server). If they are losing their connections somehow and then reconnecting, then which ones would get the broadcast message would happen to depend upon which ones were not in the middle of a temporarily lost connection.
I have a node server accepting websocket connections from the clients. Each client can broadcast a message to all of the other clients.
UPDATE: I am using https://github.com/websockets/ws as my library of choice.
At the moment, the server has an array with all of the connections. Each connection has a tabId. When one of the client emits a message, I go through all of the connections and check: if the connection's tabId doesn't match, I send the message to the client.
For loading issues, I am facing the problem of having to have more than one server. So, there will be say two servers, each one with a number of clients.
How do I make sure that a message gets broadcast to all of the websocket clients, and not only the ones connected to the same server?
One possible solution I thought is to have the connections stored on a database, where each record has the tabId and the serverId. However, even a simple broadcast gets tricky as messages to "local" sockets are easy to broadcast (the socket is local and available) whereas messages to "remote" sockets are tricky, and would imply intra-server communication.
Is there a good pattern to solve this? Surely, this is something that people face every day.
You could use a messagequeue like RabbitMQ.
When a client logs in to your server, create a consumer which listens to a queue which will receive messages directed to that particular client. And when the clients are sending messages, just use a publisher to publish them to the recipients queue.
This way it doesn't matter and you don't need to know on which nodes the clients are on, or if they jump from a node to another.
I have a solution where an application is emitting messages using socket.io-emitter and a socket server is handling the messages.
The socket server receives messages from the client browser without an issue however it does not pickup any messages from the other application which are sent over the redis adapter.
I've debugged the redis adapter and can see that the messages are being received and I can also see that they are associated with the correct namespace. They just appear to not be firing the socket.on() event
The server code has more going on however basically boils down to the following
io.adapter(redisIO({host: redisHost, port: redisPort}));
io.of('/mynamespace').on('connection',
function(socket) {
// This message never gets fired
socket.on('other-server-message',dosomething);
// This message works fine
socket.on('message-from-browser-client',dosomethingelse);
}
);
There isn't much documentation around so any help would be great
I eventually realised what the issue was. I was misunderstanding what the socket.io-emitter was doing.
I was sending a message from the emitter and then trying to capture it on the server and then push it out to the clients. The emitter is actually broadcasting directly to the clients in the web browser as if it was just another server socket (which is what I wanted).
Soon as i updated the client code to check for the message it worked perfectly