Okay so multithreading nodeJS isn't much problem from what I've been reading. Just deploy several identical apps and use nginx as reverse proxy and load balancer of all the apps.
But actually native cluster module works pretty well too, I found.
However, what if I have socket.io with the nodeJS app? I have tried the same strategy with nodeJS + socket.IO; however, it obviously did not work because every socket event emitted will be more or less evenly distributed and sockets other than the one that made the connection would have no idea where the request came from.
So best method I can think of right now is to separate nodeJS server and socket.IO server. Scale nodeJS server horizontally (multiple identical apps) but just have one socket.IO server. Although I believe it would be enough for the purpose of our solution, I still need to look out for future. Has anyone succeeded in horizontally scaling Socket.IO? So multiple threads?
The guidelines on the socket.io website use Redis with a package called socket.io-redis
https://socket.io/docs/using-multiple-nodes/
Looks like is just acts like a single pool for the connections, and each node instance connects to that.
Putting your socket server on a separate service (micro-service) is a probably fine, the downside is needing to manage communications between the two instances.
Related
I currently have a Node server running that works with MongoDB. It handles some HTTP requests, but it largely used WebSockets. Basically, the server connects multiple users to rooms with WebSockets.
My server currently has around 12k WebSockets open and it's almost crippling my single threaded server, and now I'm not sure how to convert it over.
The server holds HashMap variables for the connected users and rooms. When a user does an action, the server often references those HashMap variables. So, I'm not sure how to use clusters in this. I thought maybe creating a thread for every WebSocket message, but I'm not sure if this is the right approach, and it would not be able to access the HashMaps for the other users
Does anyone have any ideas on what to do?
Thank you.
You can look at the socket.io-redis adapter for architectural ideas or you can just decide to use socket.io and the Redis adapter.
They move the equivalent of your hashmap to a separate process redis in-memory database so all clustered processes can get access to it.
The socket.io-redis adapter also supports higher-level functions so that you can emit to every socket in a room with one call and the adapter finds where everyone in the room is connected, contacts that specific cluster server, and has it send the message to them.
I thought maybe creating a thread for every WebSocket message, but I'm not sure if this is the right approach, and it would not be able to access the HashMaps for the other users
Threads in node.js are not lightweight things (each has its own V8 instance) so you will not want a nodejs thread for every WebSocket connection. You could group a certain number of WebSocket connections on a web worker, but at that point, it is likely easier to use clustering because nodejs will handle the distribution across the clusters for you automatically whereas you'll have to do that yourself for your own web worker pool.
I am writing real time connection web server with WebSocket(ws) library in Node js(Express).so when I am running server on pm2 cluster mode than websocket not working properly.Is there any way to make share data between pm2 clusters with websocket.
From what I understand, that's kind of the point of cluster mode. It wouldn't be able to keep the connection alive across multiple "nodes" running processes. Straight from the PM2 docs:
Be sure your application is stateless meaning that no local data is stored in the process, for example sessions/websocket connections, session-memory and related. Use Redis, Mongo or other databases to share states between processes.
If you could pull out your websocket into its own instance and avoid side effects from a pm2 cluster--that's probably the most performant, although maybe not feasible in your set up. Best of luck!
You can switch the library to socket.io they already solve this issue by using redis adapter for socket.io Socket.IO Redis
By running socket.io with the socket.io-redis adapter you can run multiple socket.io instances in different processes or servers that can all broadcast and emit events to and from each other.
My current setup is running multiple node instances using PM2 to manage the instances and act as a load balancer.
I would like to implement some functionality using websockets.
The first issue that came to mind is sharing the sockets among X node instances.
My understanding is that if I boot up a websocket-server within a node env only that env will have access to the sockets connected to it.
I do not want to load up web sockets for each instance for each user as that seems like a waste of resources.
Currently I am playing around with the websocket package on npm but I am in no way tied to this if there is a better alternative.
I would like the sockets to more or less push data one-way from server to client and avoid anything coming from the client to the server.
My solution so far is spin up another node instance that solely acts as a websocket server.
This would allow a user to make all requests as normal to the usual instances but make a websocket connection to the separate node instance dedicated to sockets.
The servee could then fire off messages to the dedicated socket server anytime something is updated to send data back to the appropriate clients.
I am not sure this is the best option and I am trying to see if there are other recommended ways of managing websockets across multiple node instances yet still allow me to spin up/down node instances as required.
I'd recommend you avoid a complex setup and just get socket.io working across multiple nodes, thus distributing the load; If you want to avoid data coming from the client to the server, just don't listen to incoming events on your server.
Socket.io supports multiple nodes, under the following conditions:
You have sticky sessions enabled. This ensures requests connect back to the process from which they originated from.
You use a special adapter called socket.io-redis & a small Redis instance as a central point of storage - it keeps track of namespaces/rooms and connected sockets across your cluster of nodes.
Here's an example:
// setup io as usual
const io = require('socket.io')(3000)
// Set a redisAdapter as an adapter.
const redisAdapter = require('socket.io-redis')
io.adapter(redisAdapter({ host: 'localhost', port: 6379 }))
From then on, it's business as usual:
io.emit('hello', 'to all clients')
You can read more here: Socket.IO - Using Multiple Nodes.
This is a bit of a stretch, but I hope someone can help.
I'm a PHP/iOS developer who's been working on an app that has a messaging component. Front end is Obj-C, backend is PHP/MySQL currently. As I've gone further into development, I'm feeling the shortcomings of polling and I've been looking for a more realtime solution and, sure enough, I've found the answer in web sockets. PHP doesn't play too well in this domain, but I've been able to get things working locally by using Laravel + Redis + Node.js.
Next I needed to find a suitable host for the real world app deployment and this is where I'm running into my first major obstacle (or perceived obstacle?)
Heroku appears to have very low limits on the number of Redis connections allowable:
Link: https://elements.heroku.com/addons/heroku-redis
Free plan: 20 connections
$120/month: 400 connections
$1450/month: 5000 connections
The problem is, if this app does well and gains the kind of traction I want, a LOT of people will be using it at the same time all across the country and these limits have me worried. These prices seem a bit ridiculous or I'm not looking at it correctly.
So my question is, does maintaining an open web socket (one user) mean that one of the Redis connections is used? Or am I looking at this completely wrong? Trying to decide if I need to just stick to polling or if there is a cost-efficient solution to this. I do want to stick to Laravel/Redis if possible because I am not too familiar with JS and I feel that my backend will be much less secure if I try to go down that route at this point.
Proper design will use 2 Redis connections per server (or per Heroku Dyno):
One connection will be used to Subscribe (to listen) to the app's channel(s). This connection cannot be used for other functions, so...
A second connection is used for all other Redis features, such as Database use and Publishing to the app's channel(s).
I don't know if you're into Ruby, but I'm the author of the Plezi Http(REST)/Websocket framework and had to manage a solution for Plezi's scaling capabilities over Redis (which is an automated feature, you just tell Plezi the Redis server's address and you're good to go).
If you want to look over Plezi's Redis code, you will notice there are two connections and that each server registers to two channels - a global channel and a private channel: one used for application wide events and the other one allows messages to be routed to specific connections based on the server they belong to (avoiding workload on unrelated servers).
Good luck!
I am looking to build an web application using node.js and possibly socket.io but I am having a lots of confusion regarding whether to use socket.io or go with plain http. In the app the node.js server will be basically an api server which serves json to the javascript client or may be mobile clients too. The web app will also has chat messeneger for its users, this is where socket.io comes in.
I am not sure whether to use socket.io for the whole app or only for the chat part. Although my app itself could benefit from socket.io but its nothing that I think can't be done using plain http and client making more requests to the server.
I have read at several places that sometimes socket.io can be difficult to scale for more users.
Socket.io often crashes and specially creates probs when there are firewall in clients system.
More importantly.....I checked out socket.io user list and did not find many users, so was curious to know what kind of platform is more know chat network like facebook messenger, google talk etc are built upon, Are any built using http-ajax and continues querying to the server.
Please help me out in solving this question. Some might argue that this is a opinion based question. But what actually I am trying to figure out the implementation of socket.io and its limitation.
I would suggest serving your API over HTTP and leave the real-time business to Socket.io. If you are adverse to using Websockets, like #GeoPheonix stated, you can choose from a variety of transport methods using both socket.io and sockjs (https://github.com/sockjs/sockjs-node).
As far as scaling is concerned, I deployed a socket.io based real-time analytics/tracking service for a very large application with ana average of 400+ concurrent connections with no visible performance impact, but this may depend on the implementation and hardware.
Socket.io is faster than plain http. I recommend you to use it for all since you have to have a chat in first place.
In my case, real-time Texas Hold'em-like game can receive up to 2500 concurrent with one node process. However, if you change transport from websocket to xhr-polling, It can receive like more 10x than pure websocket. Your application is just chat so, I guess a little slow down wouldn't be a problem. If you sure you will exceed this number, yes, scale socket.io is a pain.
This problem will happen only if you open socket.io for port other than 80 and 443. If you have frontend web server with other language already, you can still use socket.io on another subdomain to be able to run on port 80 without conflict with your main frontend web server. Socket.io support cross-domain without a problem.
Have you used trello.com? If not, try it :). It's best for task management or even some Agile thing. They used socket.io. https://c9.io/ is another one. It's online IDE with google doc-like collaborative. One thing to note is xhr-polling trasport in socket.io is the same with http-ajax with long-polling (Better than general ajax). You can read more info at:
http://book.mixu.net/node/ch13.html