socket.io+redis+expressjs cluster - get socket object in expressjs request - node.js

Question based on this answer: https://stackoverflow.com/a/18650183/4478897
I tried to find this solution but nothing seems to work in the way that I need.
Clustering expressjs and socket.io we can share sessions using redis and send io messages inside io world (io.sockets.on('connection',...). The problem is if we want to send the message (or use a simple socket.join/leave) inside the expressjs world (route.get/post).
If we are not using clusters we can atach the client socket object to the express request object (or simply export the io object) and then use it at any time on any GET/POST route.
At the other hand, if we are clustering and use the mentioned method to get the socket object inside the expressjs world, sometimes the socket object is undefined because the socket object for this client is initialized at other worker.
Some example flow:
Client connects to http://localhost and worker 1 handles this request.
After the page is loaded, the client connects to socket.io. Worker 2 handles this connection.
Client do a POST and again worker 1 or worker X handles this request.
In this case when the client do the POST, only the worker 2 knows the socket object for this client. So this will get an undefined socket object.
So, the question:
How can we get the client socket object from any worker to reuse it on expressjs request object.
Maybe my code is wrong but is almost like the link to the answer mentioned above.
NOTEs
Don't want to use some kind of proxy.
Don't want to migrate to other libraries (expressio, sockjs...)
Sorry for my English :)
Using last nodejs, socket.io, expressjs, socket.io-redis, redis... versions
Don't hesitate to ask something!
UPDATE 1
Possible solution but still need to test it. Dont know if this is a really good: solution.
UPDATE 3: Working code on my own answer
UPDATE 2
Like update 1 but using https://nodejs.org/dist/latest-v5.x/docs/api/cluster.html#cluster_event_message

remoteJoin and remoteLeave methods were added in socket.io-redis 3.0.0:
io.adapter.remoteJoin('<my-id>', 'room1', function (err) {
if (err) { /* unknown id */ }
// success
});
io.adapter.remoteLeave('<my-id>', 'room1', function (err) {
if (err) { /* unknown id */ }
// success
});
Note: The implementation looks a lot (hopefully?) like the answer above.

Well finally tried the code and it works (with some misspells modifications and other things) but i'm sure that needs to be a better code somewhere. So i'm open to more answers!
This code is part of my socket.io module when authorize the client socket and some other stuff...
var redis = require("redis");
var redisPub = redis.createClient();
var redisSub = redis.createClient();
var PubSubChannel = "clusterChannel";
// Function that checks if this worker knows the socket object of this socketId.
// If not, publish the message to all the other sockets (workers)
io.socketDo = function (type, socketId, roomName) {
if (typeof io.sockets.connected[socketId] != "undefined") {
if (type === "join") {
return io.sockets.connected[socketId].join(roomName);
}
if (type === "leave") {
return io.sockets.connected[socketId].leave(roomName);
}
} else {
redisPub.publish(
PubSubChannel,
JSON.stringify({
type: type,
socketId: '' + socketId,
roomName: roomName
})
);
}
};
// Subscribe to some channel
redisSub.subscribe(PubSubChannel);
// When this worker receive a message from channel "PubSubChannel" checks
// if it have the socket object for this socketId and do the operation
redisSub.on("message", function (channel, data) {
data = JSON.parse(data);
var type = data.type;
var socketId = data.socketId;
var roomName = data.roomName;
if ((type === "join" || type === "leave") && channel == PubSubChannel){
if (typeof io.sockets.connected[socketId] != "undefined") {
if (type === "join") {
return io.sockets.connected[socketId].join(roomName);
}
if (type === "leave") {
return io.sockets.connected[socketId].leave(roomName);
}
}
}
});
Then just simply export the module and attach it to your expressjs request => req.io = io
// req.session.socketId value is fetched on "io.sockets.on('connection', function(socket) {"
// by express to socket.io using redis shared sessions
app.get('/', function (req, res) {
req.io.socketDo('join', req.session.socketId, 'someRoomToJoin');
// IT WORKS!
req.io.sockets.in('someRoomToJoin').emit('text');
req.io.socketDo('leave', req.session.socketId, 'someRoomToLeave');
res.send('Hello World!');
});

Related

Socket.Io not emitting immediately after first emit (order important)

Environment:
Backend
node:latest
socket.io | 4.5.2
Frontend
React Native | 0.70.4
socket.io-client | 4.6.0
both Android and iOS
Here is my NodeJs entry file:
const numCPUs = cpus().length
if (cluster.isPrimary) {
const app = express()
const httpServer = http.createServer(app)
setupMaster(httpServer, { loadBalancingMethod: 'least-connection' })
setupPrimary()
for (let i = 0; i < numCPUs; i++) {
cluster.fork()
}
cluster.on('exit', (worker) => {
cluster.fork()
})
} else {
const app = express()
const httpServer = http.createServer(app)
const io = new Server(httpServer, { maxHttpBufferSize: 1e8 })
io.adapter(createAdapter())
setupWorker(io)
API.Socket.init(io, process.pid)
middlewares.forEach((middleware: any) => app.use(middleware))
routes.forEach((route) => app.use(route.path, route.handler))
httpServer.listen(CONFIG.PORT, () => {})
}
I have a simple chat application.
When user A sends message to user B, new chat message and notification is recorded in database. Now that chat message and notification* should be sent to the B user. There are 2 socket emit-functions for that:
sendNewNotification(
notification: BE.Entities.TNotification,
toUser: string,
) {
this.io
?.to(toUser)
.volatile.emit(ECustomEvents.NewNotification, notification)
}
sendPrivateMessage(
toUser: string | Array<string>,
chatMessage: BE.Entities.TChatMessage,
sourceUser: BE.Entities.TUser,
) {
this.io
?.to(toUser)
.volatile.emit(ECustomEvents.PrivateMessage, chatMessage, sourceUser)
}
If I do it like this, the targetUser is not going to receive the event with the newChatMessage however he will receive the savedNotification
API.Socket.sendPrivateMessage(targetUserId, newChatMessage, userToPass)
API.Socket.sendNewNotification(savedNotification, targetUserId)
Now, if I switch these lines:
API.Socket.sendNewNotification(savedNotification, targetUserId)
API.Socket.sendPrivateMessage(targetUserId, newChatMessage, userToPass)
the behavior would be as expected: the target user B will receive both saved notification and new chat message
How is that possible? What could be wrong?
Thank you mates in advance!
With the current information, I'm not so sure the order matters but perhaps that it's a side-effect / coincidence. Are you checking anywhere to make sure the server-side socket is ready before the client emits?
Consider this super simple WebSocket chat sandbox:
One of the issues I noticed when writing this is when the server WebSocket is not ready, I could not emit from the client to the server. To make sure the server is ready, I sent a ping from the server to the client notifying the client that the server is ready:
wss.on("connection", async function connection(client, request) {
console.log("user connected", Date.now());
client.send(JSON.stringify({ ready: true }));
...
});
I also notice you are usingg the volatile.emit which according to the documentation:
Volatile events
Volatile events are events that will not be sent if the underlying connection is not ready (a bit like UDP, in terms of reliability).
This can be interesting for example if you need to send the position of the characters in an online game (as only the latest values are useful).
socket.volatile.emit("hello", "might or might not be received");
The Socket.IO docs have a similar listener which lets you know when the server is ready.
If you prevent the client from emitting until the server is ready, you can avoid this issue. You also should not need to use the volatile.emit for something that must be delivered.

How to handle multiple incoming connections in Node.js?

I create a server with Node.js:
var net = require('net');
var PORT = 8181;
var server = net.createServer(
function(socket) {
console.log(this.address());
socket.on('data', function(data) {
var msg = data.toString().replace(/\n$/, '');
console.log('got: ' + msg);
});
process.stdin.on('readable',
function() {
var chunk = process.stdin.read();
if (chunk !== null) {
socket.write(chunk);
}
}
)
socket.write('heyyo\n');
}
)
Now, when multiple connections are coming in, this server sends out the typed in line only to the first connection.
I have two questions:
what is a standard way to handle this, i.e. to store the incoming sockets into an array?
exactly what happens that causes the readable event not to reach the other connections' callback function?
I would highly recommend using a library like socket.io. It makes handling connect/disconnect as well as placing sockets in rooms very simple. Additionally you can get the full list of available rooms and connected sockets through the adapter class it offers. A functional example is available in the docs.

Save Data on Socket in Socket.IO

I want to save some data on the socket, server side, so whenever the client emits any data to the server, I want that data to be available!
One use case can be storing a token on the socket. When the client is connecting for the first time, it will emit the token, if it has one, or it will show the login page and then the login data will be sent to the server. Whichever one it is, I want to store the token on the server, so that every request after that doesn't need to specify the token.
Later, I'll use RedisStore, so all the data will be accessible all the servers running the app.
My only question is, where do I store the data on the socket so it's associated with that client?
on http://socket.io/#how-to-use
scroll to: Storing data associated to a client
use socket.set and socket.get to set and get data asynchronously
I'm suffering from the same question and guessing what's going on with an example code from socket.io on version 4.x
In the example, They use middleware(use function to register a middleware)
namespace.use((socket, next) => {
// get data from client
const sessionID = socket.handshake.auth.sessionID;
const {userId, username} = yourFunction();
// set socket specific data
socket.sessionID = sessionID;
socket.userID = session.userID;
socket.username = session.username;
next();
});
Middlewares are executed when a socket is connected with a server.
and you can use the data afterward
note - Socket.IO reference tells use socket.data for this purpose
namespace.on('connection', socket => {
socket.emit("join", `${socket.username} has been joined`);
})
If you use multiple servers, then you have to keep in mind that the data is only valid for the server
On multiple server environment, You need a single source of data which will be used by socket servers.
namespace.use(async (socket: Socket & { sessionID?: string, userID?: string, username?: string }, next) => {
const sessionID = socket.handshake.auth.sessionID; // [socket.handshake][4]
// or other [socket related attributes][4]
if (sessionID) {
// you have to implement a function to save and retrive session info
const session = await someFunctionToRetrieveSession(sessionID);
if (session) {
socket.sessionID = sessionID;
socket.userID = session.userID;
socket.username = session.username;
return next();
}
}
const username = socket.handshake.auth.username;
if (!username) {
return next(new Error("invalid username"));
}
socket.sessionID = randomId();
socket.userID = randomId();
socket.username = username;
next();
});
and one more thing as I understood the namespace.use function is called only for the namespace if your client use other namespace then default then default('/') use function will not be called.
//client side
io("/chat");
...
//server side
io.use() // == io.of('/').use() will not be called
io.of('/chat').use() // only will be called
Thanksfully the author of the example implemented a sessionStorage using redis
refer to this example code
with this info, I guess socket.io server saves sockets' info in memory and set a property of a socket will be saved and when the socket comes later the server retrives the socket and it's related data. but because it happens on memory so you can't share the info among other servers that's why you have to find a way to share the data with other servers(eg. redis)
You can save the data on the global variables when you dont want to use any database
var globalVariable = {};
io.sockets.on("connection", function (socket) {
socket.on("save-client-data", function (clientData) {
var clientId = clientData.clientId;
globalVariable[clientId] = JSON.parse(clientHandshakeData);
});
socket.on("get-client-data", function (clientId) {
var clientData = globalVariable[clientId];
socket.emit("get-client-data", JSON.stringify(clientData));
});
});
This worked for my scenario, however I'm not aware of the performance implications.

webSocketServer node.js how to differentiate clients

I am trying to use sockets with node.js, I succeded but I don't know how to differentiate clients in my code.
The part concerning sockets is this:
var WebSocketServer = require('ws').Server,
wss = new WebSocketServer({port: 8080});
wss.on('connection', function(ws) {
ws.on('message', function(message) {
console.log('received: %s', message);
ws.send(message);
});
ws.send('something');
});
This code works fine with my client js.
But I would like to send a message to a particular user or all users having sockets open on my server.
In my case I send a message as a client and I receive a response but the others user show nothing.
I would like for example user1 sends a message to the server via webSocket and I send a notification to user2 who has his socket open.
In nodejs you can directly modify the ws client and add custom attributes for each client separately. Also you have a global variable wss.clients that can be used anywhere. Please try the following code with at least two clients connected:
var WebSocketServer = require('ws').Server;
var wss = new WebSocketServer({
server: httpsServer
});
wss.getUniqueID = function () {
function s4() {
return Math.floor((1 + Math.random()) * 0x10000).toString(16).substring(1);
}
return s4() + s4() + '-' + s4();
};
wss.on('connection', function connection(ws, req) {
ws.id = wss.getUniqueID();
wss.clients.forEach(function each(client) {
console.log('Client.ID: ' + client.id);
});
});
You can also pass parameters directly in the client connection URL:
https://myhost:8080?myCustomParam=1111&myCustomID=2222
In the connection function you can get these parameters and assign them directly to your ws client:
wss.on('connection', function connection(ws, req) {
const parameters = url.parse(req.url, true);
ws.uid = wss.getUniqueID();
ws.chatRoom = {uid: parameters.query.myCustomID};
ws.hereMyCustomParameter = parameters.query.myCustomParam;
}
You can simply assign users ID to an array CLIENTS[], this will contain all users. You can directly send message to all users as given below:
var WebSocketServer = require('ws').Server,
wss = new WebSocketServer({port: 8080}),
CLIENTS=[];
wss.on('connection', function(ws) {
CLIENTS.push(ws);
ws.on('message', function(message) {
console.log('received: %s', message);
sendAll(message);
});
ws.send("NEW USER JOINED");
});
function sendAll (message) {
for (var i=0; i<CLIENTS.length; i++) {
CLIENTS[i].send("Message: " + message);
}
}
you can use request header 'sec-websocket-key'
wss.on('connection', (ws, req) => {
ws.id = req.headers['sec-websocket-key'];
//statements...
});
This code snippet in Worlize server really helped me a lot. Even though you're using ws, the code should be easily adaptable. I've selected the important parts here:
// initialization
var connections = {};
var connectionIDCounter = 0;
// when handling a new connection
connection.id = connectionIDCounter ++;
connections[connection.id] = connection;
// in your case you would rewrite these 2 lines as
ws.id = connectionIDCounter ++;
connections[ws.id] = ws;
// when a connection is closed
delete connections[connection.id];
// in your case you would rewrite this line as
delete connections[ws.id];
Now you can easily create a broadcast() and sendToConnectionId() function as shown in the linked code.
Hope that helps.
It depends which websocket you are using. For example, the fastest one, found here: https://github.com/websockets/ws is able to do a broadcast via this method:
var WebSocketServer = require('ws').Server,
wss = new WebSocketServer({host:'xxxx',port:xxxx}),
users = [];
wss.broadcast = function broadcast(data) {
wss.clients.forEach(function each(client) {
client.send(data);
});
};
Then later in your code you can use wss.broadcast(message) to send to all. For sending a PM to an individual user I do the following:
(1) In my message that I send to the server I include a username
(2) Then, in onMessage I save the websocket in the array with that username, then retrieve it by username later:
wss.on('connection', function(ws) {
ws.on('message', function(message) {
users[message.userName] = ws;
(3) To send to a particular user you can then do users[userName].send(message);
I'm using fd from the ws object. It should be unique per client.
var clientID = ws._socket._handle.fd;
I get a different number when I open a new browser tab.
The first ws had 11, the next had 12.
You can check the connection object. It has built-in identification for every connected client; you can find it here:
let id=ws._ultron.id;
console.log(id);
One possible solution here could be appending the deviceId in front of the user id, so we get to separate multiple users with same user id but on different devices.
ws://xxxxxxx:9000/userID/<<deviceId>>
By clients if you mean the open connections, then you can use ws.upgradeReq.headers['sec-websocket-key'] as the identifier. And keep all socket objects in an array.
But if you want to identify your user then you'll need to add user specific data to socket object.
If someone here is maybe using koa-websocket library, server instance of WebSocket is attached to ctx along side the request. That makes it really easy to manipulate the wss.clients Set (set of sessions in ws). For example pass parameters through URL and add it to Websocket instance something like this:
const wss = ctx.app.ws.server
const { userId } = ctx.request.query
try{
ctx.websocket.uid = userId
}catch(err){
console.log(err)
}
Use a global counter variable and assign its value for every new connection:
const wss = new WebSocket.Server({server});
let count_clients = 0;
wss.on('connection', function connection(ws){
ws.id=count_clients++;
console.log(`new connection, ws.id=${ws.id}, ${ws._socket.remoteAddress}:${ws._socket.remotePort} #clients=${wss.clients.size}`);
ws.on('close', req => {console.log(`disconnected, ws.id=${ws.id}, ${ws._socket.remoteAddress}:${ws._socket.remotePort} #clients=${wss.clients.size}`);});
...
Here is what I did:
* on connect, server generate an unique id (e.g uuid) for the connection,
* save it in memory, (e.g as key of map),
* send back to client in response,
*
*
* client save the id, on each request will also send the id as part of request data,
* then server identify the client by id, on receive further request,
*
* server maintain client, e.g cleanup on close/error,
*
I've impl the idea, it works well to identify the client.
And, I also achieved group/topic broadcast based on the idea, which need the server to maintain extra info.
There are a lot of interesting answers that do the job, however they mostly seem unclean, that is if you don't mind mutating the ws object. I did it this way because I'm using TypeScript and you can't arbitrarily add properties to objects.
import WebSocket from 'ws'
declare module 'ws' {
interface WebSocket {
id: any
key: string
}
}
The id doesn't have to be type any can be number or string depending on how you ID your connections. I haven't flushed out the system yet but for now when a connection is made, I just assign a random number.
const socketConnection = (socket: WebSocket.WebSocket): void => {
socket.id = Math.random()
console.log(socket.id)
const msg = JSON.stringify({ res: `[open] Welcome to the WebSocket server!` })
socket.send(msg)
}
This can be modified at any point so once I authenticate the connection I plan on assigning a relative ID here and might even add in a key property if I want to do some more fancy stuff.
How this works is explained in the Module Augmentation section of the documentation.
TypeScript: Module Augmentation
You can check that it's still assigned by looking over multiple messages in the onmessage event.
const socketMessage = (socket: WebSocket.WebSocket): void => {
socket.on('message', async (message: WebSocket.RawData) => {
console.log(socket.id)
console.log(socket.key)
})
}
Oh and a note, I made this module declaration in the document where I setup my socket. But the modification does populate across documents. For example in the AuthController I started prototyping I use it this way.
export default class AuthController {
public static connections: DLinkedList = new DLinkedList()
static async validate(request: { id: string, socket: WebSocket.WebSocket }): Promise<void> {
console.log('test', request.socket.id)
this.connections.add(request.socket, request.id)
request.socket.send(JSON.stringify({ res: true }))
console.log(this.connections.size())
}
static getSocket(id: string): WebSocket.WebSocket {
return this.connections.getAtKey(id).data
}
static removeSocket(socket: WebSocket.WebSocket) {
}
}
You can also do this in pure JS just by directly modifying the WebSocket object prototype. Some of the answers here talk about it. I haven't done it myself but the principle is similar.
Add a method to an existing class in typescript?
Hope this is useful.

Node JS Express JS Client / Server game

I'm writing a two player card game, (say it's regular poker for simplicity) in Node.js and Express js. There's a couple of things I'm having trouble with. First, how do I make sure that there are only 2 players that can access an instance of the game, and is it possible to have them reconnect if they lose the connection? Second, how do send a message from the server to the client? I can send it within the "socket.on" listener call, but within the normal scope of the program I can't get it to work.
var socket = io.listen(app);
socket.on('connection', function(client){
player++;
if(player <= 2) {
var messageout = "player " + player + " connected";
client.broadcast(messageout);
client.on('message', function(data){console.log(data); })
client.on('disconnect', function(){console.log('disconnected');})
}
else {
socket.end;
}
});
I'm having trouble conceptually what's going on here and how to approach the problem. For example, do I do it all with sockets? Or do I return a web page with the updated state of the game (cards, bets, etc.) every turn?
First, how do I make sure that there
are only 2 players that can access an
instance of the game?
Create an array of instance objects.
When a new player joins either create a new instance and set them as player1, or add them to an existing instance as player two.
var instances = [];
function Instance () {
return {
name = 'game' + instances.length + 1,
gameVariables = defaults,
player1 = null,
player2 = null,
player1UUID = UUID(),
player2UUID = UUID()
}
}
Is it possible to have them reconnect
if they lose the connection?
If you send each player a UUID when they initially connect you can have them use it to authenticate when they reconnect.
How do I send a message from the
server to the client?
client.send({ gameState: gameState() });
of if you've saved the client into an object: instances[ 'game1' ].player1.send( data );
I do it all with sockets?
I would deal with all dynamic interactions with web sockets.
Do I return a web page with the updated state of the game (cards, bets, etc.) every turn?
I wouldn't send html over web sockets. Instead send json and use a client side template to render it.
Try now.
// server
var nowjs = require("now")
var everyone = nowjs.initialize(app);
everyone.connected(function() {
if (users.length < 2) {
users.push(this);
}
});
everyone.now.sendMessage = function(message, cb) {
gameLogic(message);
users.forEach(function(user) {
user.update(gameLogic.getState());
});
};
//client
$("#action").click(function() {
now.sendMessage($(this).data("action"));
});
now.update = function(data) {
// update UI
};

Resources