Pool Websocket Connections - NodeJS - node.js

I'm looking to build a node app that will accomplish the following:
Open several websocket connections, almost as if each of them were a
thread
Allow each websocket to have a unique/dynamic URL
Create a pool of websocket connections in an object based off some kind of DB query (so I can dynamically add/remove connections)
I've decided to use the ws library (https://github.com/websockets/ws) since its the fastest and least bloated option available. I currently have the following function, which only supports a single ws connection:
chat.prototype.connect = function() {
var self = this;
self.ws = new ws(url);
self.ws.on('message', function(data, flags) {
var message = JSON.parse(data);
self.handle(message);
});
};
This code listens to a single websocket URL and passes the message(s) to my handler to process the message. Instead, I want to make this function listen to multiple (potentially hundreds) of websocket URL's.
Does anyone have some ideas on how to accomplish this?

Say that you have the list of url's you need to connect to stored in an instance property called urls. You could set up the connections like this:
chat.prototype.connect = function() {
urls.forEach(this.connectOne.bind(this));
};
chat.prototype.connectOne = function(url) {
var handle = this.handle.bind(this);
var conn = this.connections[url] = new ws(url);
conn.on('message', function(data, flags) {
var message = JSON.parse(data);
handle(message);
});
};
To implement adding new connections, periodically query your database and check if each URL is already present in this.connections; if not, you can use this.connectOne() to add it. You'd do something similar to remove a connection.

Related

Websocket + Redis: multiple channels, specific subscriptions/publishing

I'm new to websockets, and am wondering how best to go about this.
My scenario: I have a server that handles different classes of users. For this example, let's say the classes are "mice", "cats", and "dogs"
Each of those classes should have their own channels to listen to for changes e.g. "mice-feed", "cat-feed", and "dog-feed"
My question is: after the server authenticates and determines the class of the current user, what's the best way to have them subscribed to a specific channel, or channel(s), so that when I broadcast messages to said channel(s), I can make sure that only members of particular classes get them (as against everyone currently connected to that server)?
My current code setup looks like this:
var ws = require('ws');
var redis = require('redis');
/* LOCATION 1 */
// prep redis, for websocket channels
var pub = redis.createClient();
var sub = redis.createClient();
// subscribe to our channels
sub.subscribe('mice-feed');
sub.subscribe('cat-feed');
sub.subscribe('dog-feed');
// declare the server
const wsServer = new ws.Server({
noServer: true,
path: "/",
});
/* ... removing some code for brevity... */
wsServer.on("connection", function connection(websocketConnection, connectionRequest) {
/* LOCATION 2 */
});
Do I put the redis declarations in LOCATION 1 (where it currently is), or in LOCATION 2 (when a successful connection is established)? Or neither of the above?
(also: I know it's possible to do this on the websocket end directly i.e. iterate through every client and ws.send if some criterion is matched, but iteration can become costly, and I'm wondering if I can do it on a redis-channel wide operation instead)
If I were building this, my first approach would be this:
// connect to Redis
const client = createClient();
client.on('error', (err) => console.log('Redis Client Error', err));
await client.connect();
// declare the server
const wsServer = new ws.Server(...elided...);
// handle connection
wsServer.on('connection', async (websocketConnection, connectionRequest) => {
const sub = client.duplicate()
// figure out the feed
const feed = 'animal-feed';
await sub.subscribe(feed, message => {
...do stuff...
});
});
It's pretty straightforward but would result in ever user having a dedicated connect to Redis. That may or may not matter depending on how many users you anticipate having.

Multiple connections, but single websocket.on("message") event emitter

I have a single device establishing two WebSocket connections to the Einaros/ws WebSocket server. Whenever the second WebSocket connection sends a message to the server, only the first websocket.on("message") event emitter responds. There is no way to differentiate which WebSocket the message is coming from because there seems to be only a single websocket.on("message") event emitter object.
How can I differentiate from which WebSocket connection the message is being received from without passing an ID from the client side?
I apologize if I am overlooking something simple, I am a node.js and coding novice. From the code below it looks like there should be separate event emitter objects created for each WebSocket connection so that the server knows which connection the message is coming from. My code looks like this:
var connections = new Map();
var idCounter = 0;
wss.on("connection", function connection(ws) {
var connectionID = idCounter++;
connections.set(connectionID, ws);
var session = connections.get(connectionID);
session.on("message", function incoming(message) {
session.send(message);
}
}
--- Update ---
I have performed another test. With the code below "objectTest" contains the unique WebSocket connection distinguished by 'sec-websocket-key' printed to the console. However "this.send(message);" and "console.log(this);" both refer to the first established WebSocket connection even while "objectTestMap" contains the second "objectTest" that is unique.
var connections = new Map();
var idCounter = 0;
wss.on("connection", function connection(ws) {
var connectionID = idCounter++;
connections.set(connectionID, ws);
var session = connections.get(connectionID);
var sendThis = String(connectionID);
session.send(sendThis);
var objectTestMap = new Map();
var objectTest = session.on("message", function incoming(message) {
this.send(message);
console.log(this);
});
objectTestMap.set(connectionID, objectTest);
console.log(objectTestMap.get(connectionID));
});
Their was an error on my client application that was connecting to the server. No problems with WS and the above code works as it should.

Node JS TCP Proxy: Reuse socket in callback function

I'm trying to implement a TCP proxy in Node JS. I only have some experience with Javascript so I met a lot of problems along the way. I've done a lot of searching for this one but had no luck.
The problem occurs when browser sends a CONNECT request for HTTPS. My proxy will parse the host name and port, and then create a new socket that connects to the server. If all these steps went well, I will start forwarding message.
Part of my code looks like this:
var net = require('net');
var server = net.createServer(function(clientSock) {
clientSock.on('data', function(clientData) {
var host = // get from data
var port = // get from data
if (data is a CONNECT request) {
// Create a new socket to server
var serverSock = new net.Socket();
serverSock.connect(port, host, function() {
serverSock.write(clientData);
clientSock.write('HTTP/1.1 200 OK\r\n');
}
serverSock.on('data', function(serverData) {
clientSock.write(serverData);
}
}
}
}
Since the CONNECT request needs both client socket and server socket open until one side closes the connection, the code above doesn't have this behavior. Every time I receive some data from client, I will create a new socket to server and the old one is closed.
Is there a way to store the server socket as a global variable so that the data event handler can reuse it? Or is there any other way to solve this?
Thanks a lot!!!!
You can just move the variable up to a higher scope so it survives across multiple events and then you can test to see if its value is already there:
var net = require('net');
var server = net.createServer(function(clientSock) {
var serverSock;
clientSock.on('data', function(clientData) {
var host = // get from data
var port = // get from data
if (data is a CONNECT request) {
// Create a new socket to server
if (!serverSock) {
serverSock = new net.Socket();
serverSock.connect(port, host, function() {
serverSock.write(clientData);
clientSock.write('HTTP/1.1 200 OK\r\n');
}
serverSock.on('data', function(serverData) {
clientSock.write(serverData);
}
} else {
serverSock.write(clientData);
}
}
}
}

Saving open sockets (or pointers to sockets) Node JS

I have a socket based server that uses Einaros WS on Node JS. I think my question applies regardless of the socket library choice. Currently, whenever I get a new connection I store each socket object in an array like this:
var WebSocketServer = require("ws").Server;
...
var server = require('http').createServer(app);
...
var wss = new WebSocketServer({server: server});
var clients = [];
wss.on("connection", function(ws) {
console.log("websocket connection open");
ws.on("message", function(message) {
{
message = JSON.parse(message);
switch (message.type) {
case "START":
{
ws.user_id = message.user_id;
clients[ws.user_id]=ws;
}
...
This means for each open connection I am storing what I believe to be a fairly large socket object. Is there a way to store an identifier or pointer to this object instead of storing the entire socket in an array this way? How do systems that can handle a very large number of open connections store / remember open sockets?
The "ws" variable is just a reference to the websocket object, not a copy of the structure, so it is no more expensive than storing any other object (or primitive) in an array.
It is better to store socket IDs in REDIS as it will help when you will scale your application and your socket servers will be on different machines.

Limiting number of connections in Node.js

I recently learned node.js, and I'm having fun rewriting perl scripts into non-blocking scripts. I am currently writing a script to go out and connect to some hosts and pull some data. I'm pulling the connection data from a database, and would like to limit it to x connections (probably 50) at a time, with new connections starting up when one is complete.
Here's the code I'm playing with:
var net = require('net');
var mysql = require('mysql');
var mysql_conn = mysql.createConnection({
//connection info...
});
var connect_hash ={};
function make_connection (id, ip, port, username, password, cb) {
var conn = net.createConnection(port, ip);
var completeData = '';
connect_hash[id] = {};
connect_hash.id.ip = ip;
// stuff happens here..
}
mysql_conn.query("select id, IP, Port, Username, Password from Hosts",
function (err, ne_records, fields){
if (err) throw err;
ne_records.forEach(function(host){
make_connection(host.id, host.IP, host.Port, host.Username, host.Password, function(attempt) {
delete connection_hash[id];
db_save (attempt);
});
});
});
As written right now, it'll just open connections to every host in the table and do stuff on them. I'd really like it to open a specified number at a time, and only start new connections when the old ones are done and deleted out of connection_hash. Is there a way to do this?
This is simple – keep a queue of hosts to be connected to, and only open n connections at a time. When a connection completes (or fails), start the next one in the queue.
You might want to take a look at how the HTTP Agent class is implemented. It does what you're trying to accomplish for HTTP requests.

Resources