We have a node.js server that is primarily used with socket.io for browser inter connectivity in a web application.
We want to have a high availability solution which would theoretically consist of two node.js servers, one as a primary server and the other as a backup should the primary fail. The solution would allow that if or when the primary node.js server goes down the backup would take over to provide seamless functionality without interruption.
Is there a solution that allows socket.io to maintain the array of client connections over multiple servers without duplication of clients or of messages sent?
Is there another paradigm we should be considering for HA and node.js?
There is no way to have a webSocket auto fail-over without any interruption to a new server when the one it is currently connected to goes down. The webSockets that were connected to the server that went down will die. That's just how TCP sockets work.
Fortunately with socket.io, the client will quickly realize that the connection has been lost (within seconds) and the clients will try to reconnect fairly quickly. If your backup server is immediately in place (e.g. hot standby) to handle the incoming socket.io connections, then the reconnect will be fairly seamless from the client point of view. It will appear to just be a momentary network interruption from the client's point of view.
On the server, however, you need to not only have a backup, but you have to be able to restore any state that was present for each connection. If the connections are just pipes for delivering notifications and are stateless, then this is fairly easy since your backup server that receives the reconnects will immediately be in business.
If your socket.io connections are stateful on the server-side, then you will need a way to restore/access that state when the backup server takes over. One way of doing this is by keeping the state in a redis server that is separate from your web server (though you will then need a backup/high availability plan for the redis server too).
SocketIO in primary and backup server can be connected to a redis server. This will maintain the sessions in the primary server and can be used by backup server, The clients should once again connect to the new server(when primary fails).
SoketIO- Redis
HA - proxy is used for load balancing between multiple node.js instances. The usage of HA proxy will depend on how you are going to deal with failure of primary server. If you have any method to automatically switch primary server, then HA-proxy will not be much useful, else you can configure HA-Proxy to forward request to backup server if the primary server is unreachable.
Other options similar to HA-Proxy are:
node-http-proxy
Nginx
Related
My team and I are working on a digital signage platform.
We have ~ 2000 Raspberry Pi around the world connected to a Nodejs server using Socket IO. The Raspberries are initiating the connection.
We would like to be able to scale horizontally our application on multiple servers but we have a problem that we can’t figure out.
Basically, the application stores the sockets of the connected Raspberry in an array.
We have an external program that calls the API within the server, this results by the server searching which sockets will be "impacted" by the API call and send them the informations.
After lots of search, we assume that we have to stores the sockets (or their ID) elsewhere (Redis ?), to make the application stateless. Then, any server can respond to a API call and look the sockets in a central place.
Unfortunately, we can’t find any detailed example on how to do that.
Can you please help us ?
Thanks
(You can't store sockets from multiple server instances in a shared datastore like redis: they only make sense in the context of the server where they were initiated).
You will need a cluster of node.js servers to handle this. There are various ways to make a cluster. They all involve directing incoming connections from your RPis to a "generic" hostname, for example server.example.com. Behind that server.example.com hostname will be multiple node.js servers.
Each incoming connection from each RPi connects to just one of those multiple servers. (You know this, I believe.) This means one node.js server in your cluster "owns" each individual RPi.
(Telling you how to rig up a cluster of node.js servers is beyond the scope of this answer. Hints: round-robin DNS or a reverse-proxy nginx front end.)
Then, you want to route -- to fan out -- the incoming data from each API call to each server in the cluster, so the server can route it to the RPis it owns.
Here's a good way to handle that:
Set up a redis cache or other shared data store. It can be very small.
When each node.js server starts, have it register itself as active. That is, have it place its own specific address for handling API calls into the shared server. The specific address is probably of the form 12.34.56.78:3000: that is, an IP address and port.
Have each server update that address every so often, once a minute or so, to show it is still alive.
When an API call arrives at server.example.com, it will come to a more-or-less randomly chosen node.js server instance.
Get that server to read the list of server addresses from the redis cache
Get that server to repeat the API call to all servers except itself. Add a parameter like repeated=yes to the repeated API calls.
Then, each server looks at its list of connected sockets and does what your application requires.
On server shutdown, have the server unregister itself -- remove its address from redis -- if possible.
In other words, build a way of fanning out the API calls to all active node.js servers in your cluster.
If this must scale up to a very large number (more than a hundred or so) node.js servers, or to many hundreds of API calls a minute, you probably should investigate using message queuing software.
SECURE YOUR REDIS server from random cybercreeps on the internet.
I am working on a nodejs app with Socket.io and I did a test in a single process using PM 2 and it was no errors. Then I move to our production environment(We use Google Cloud Compute Instance).
I run 3 app processes and a iOS client connects to the server.
By the way the iOS client doesn't keep the socket connection. It doesn't send disconnect to the server. But it's disconnected and reconnect to the server. It happens continuously.
I am not sure why the server disconnects the client.
If you have any hint or answer for this, I would appreciate you.
That's probably because requests end up on a different machine rather than the one they originated from.
Straight from Socket.io Docs: Using Multiple Nodes:
If you plan to distribute the load of connections among different processes or machines, you have to make sure that requests associated with a particular session id connect to the process that originated them.
What you need to do:
Enable session affinity, a.k.a sticky sessions.
If you want to work with rooms/namespaces you also need to use a centralised memory store to keep track of namespace information, such as the Redis/Redis Adapter.
But I'd advise you to read the documentation piece I posted, things might have changed a bit since the last time I've implemented something like this.
By default, the socket.io client "tests" out the connection to its server with a couple http requests. If you have multiple server requests and those initial http requests don't go to the exact same server each time, then the socket.io connect will never get established properly and will not switch over to webSocket and it will keep attempting to use http polling.
There are two ways to fix this.
You can configure your clients to just assume the webSocket protocol will work. This will initiate the connection with one and only one http connection which will then be immediately upgraded to the webSocket protocol (with socket.io running on top of that). In socket.io, this is a transport option specified with the initial connection.
You can configure your server infrastructure to be sticky so that a request from a given client always goes back to the exact same server. There are lots of ways to do this depending upon your server architecture and how the load balancing is done between your servers.
If your servers are keeping any client state local to the server (and not in a shared database that all servers access), then you will need even a dropped connection and reconnect to go back to the same server and you will need sticky connections as your only solution. You can read more about sticky sessions on the socket.io website here.
Thanks for your replies.
I finally figured out the issue. The issue was caused by TTL of backend service in Google Cloud Load Balancer. The default TTL was 30 seconds and it made each socket connection tried to disconnect and reconnect.
So I updated the value to 3600s and then I could keep the connection.
I'm making a nodejs application that will act a server for other sites in different countries as the data being transmitted will be business related data. I would like to know how I can safely/securely send this data.
I am currently using socket.io to act as my main server (Master) on other sites there are (Slave) servers that handle the data from the master server.
I have got this working in a local environment but want to deploy this in the other sites.
I have tried to Google this to see if anyone else has done this but came across socket.io sessions but I don't know if this will fit with (Server->Server) connections.
Any help or experience would be grateful.
For server-server communication where you control both ends of the communication you can use WebSocket over HTTPS, you can use TCP over SSH tunnel or any other encrypted tunnel. You can use a PubSub service, a queue service etc. There are a lot of ways you can do it. Just make sure that the communication is encrypted either natively by the protocols you use or with VPN or tunnels that connect your servers in remote locations.
Socket.io is usually used as a replacement for WebSocket where there is no native support in the browser. It is rarely used for server to server communication. See this answer for more details:
Differences between socket.io and websockets
If you want a higher level framework with focus on real-time data then see ActionHero:
https://www.actionherojs.com/
For other options of sending real-time data between servers you can use some shared resource like a Redis database or some pub/sub service like Faye or Kafka, or a queue service like ZeroMQ or RabbitMQ. This is what is usually done to make things like that work across multiple instances of the server or multiple locations. You could also use a CouchDB changes feed, or a similar feature of RethinkDB to make sure that all of your instances get all the data as soon as it is posted by any one of them. See:
http://docs.couchdb.org/en/2.0.0/api/database/changes.html
https://rethinkdb.com/docs/changefeeds/javascript/
https://redis.io/topics/pubsub
https://faye.jcoglan.com/
https://kafka.apache.org/
Everything that uses HTTP is easy to encrypt with HTTPS. Everything else can be encrypted with a tunnel or VPN.
Good tools that can add encryption for protocols that are not encrypted themselves (like e.g. the Redis protocol) are:
http://www.tarsnap.com/spiped.html
https://www.stunnel.org/index.html
https://openvpn.net/
https://forwardhq.com/help/ssh-tunneling-how-to
See also:
https://en.wikipedia.org/wiki/Tunneling_protocol
Note that some hosting services may give you preconfigured tunnels or internal network interfaces that pass data encrypted between your servers located in different data centers of that provider. Some providers give you tools and tutorials to that easily as well.
Using any technique, is it possible to keep the websocket connected when restarting Node.js server?
No, not directly. By design, an OS cleans up any resources owned by a process when that process shuts-down. This is how the OS prevents resource leaks over time as processes start up and shut down. So, when your server process shuts down, any sockets it still has open will get shut down by the OS.
The usual design solution for this is to code clients that automatically reconnect if they lose their webSocket connection when the client didn't intend to lose the connection. This type of auto-reconnect behavior is built into socket.io (a layer that sits on top of webSocket) for this exact reason.
If you insert a proxy in front of your server, configured so that clients connect to the proxy and then the proxy then connects through to your server, then it might be possible to teach the proxy to auto-reconnect to the server in a way that the clients would not know anything had happened (as long as they didn't try to send messages while the server was down). Of course, then you have the same restart issue with the proxy.
In some operating systems, it is possible to transfer ownership of TCP sockets from one process on the same host to another process. So, I could conceive of a scheme (which I have not tried) where you could fire up a temporary process, transfer all the webSocket sockets from your server to this temporary process, then restart your server, then after the new server instance comes up, transfer the sockets back and then kill the temporary process.
Since there are multiple other reasons why a webSocket might be unintentionally disconnected, I think the client-side reconnect is a solution that covers both the server restart and many other potential things that could happen in your system and for which code has already been written.
I have spun 3 node instances using pm2. They are all running a websocket server using these ports: (9300, 9301, and 9302).
My main server acts as a nginx load balancer. The nginx upstream block:
upstream websocket {
least_conn;
server 127.0.0.1:9300;
server 127.0.0.1:9301;
server 127.0.0.1:9302;
}
After 10 players have connected, they are distributed in round-robin fashion. I am also utilizing Redis for Pub/Sub for all the node instances.
I am curious if it's possible for a connected player that is on instance 9300, switch to 9302 while not losing their connection?
The reasoning is because my game is instance based. I have "games" if you will, that players can create or join. If I can get the connected players onto the same node instance for their games, I would reduce all the extra Pub/Sub signals and achieve better latency. (Or so I think, but just curious if this is possible)
I am curious if it's possible for a connected player that is on
instance 9300, switch to 9302 while not losing their connection?
No, it is not possible. A TCP socket is a connection between two specific endpoints and it cannot be moved from one endpoint to another after it is established. There are very good security reasons why this is prohibited (so connections can't be hijaacked).
The usual way around this problem is for the server to tell the client to reconnect and give it instructions for how to connect to the particular server you want it connected to (e.g. connect to a specific port or specific hostname or some other means that your load balancer might use).