How web servers serve multiply requests simultaneously. I think that sockets are used during the communication betwen client and server. But what if two clients try to connect one socket at one time? This socket will be used and second client could not be connected?
Servers can accept and service multiple client connections at the same time, usually by using multiple threads, overlapped/asynchronous I/O, or even forking multiple processes. But in any case, multiple clients are serviced in parallel to each other. A new client connection is refused only if the server has run out of resources to accept more clients.
Related
My team and I are working on a digital signage platform.
We have ~ 2000 Raspberry Pi around the world connected to a Nodejs server using Socket IO. The Raspberries are initiating the connection.
We would like to be able to scale horizontally our application on multiple servers but we have a problem that we can’t figure out.
Basically, the application stores the sockets of the connected Raspberry in an array.
We have an external program that calls the API within the server, this results by the server searching which sockets will be "impacted" by the API call and send them the informations.
After lots of search, we assume that we have to stores the sockets (or their ID) elsewhere (Redis ?), to make the application stateless. Then, any server can respond to a API call and look the sockets in a central place.
Unfortunately, we can’t find any detailed example on how to do that.
Can you please help us ?
Thanks
(You can't store sockets from multiple server instances in a shared datastore like redis: they only make sense in the context of the server where they were initiated).
You will need a cluster of node.js servers to handle this. There are various ways to make a cluster. They all involve directing incoming connections from your RPis to a "generic" hostname, for example server.example.com. Behind that server.example.com hostname will be multiple node.js servers.
Each incoming connection from each RPi connects to just one of those multiple servers. (You know this, I believe.) This means one node.js server in your cluster "owns" each individual RPi.
(Telling you how to rig up a cluster of node.js servers is beyond the scope of this answer. Hints: round-robin DNS or a reverse-proxy nginx front end.)
Then, you want to route -- to fan out -- the incoming data from each API call to each server in the cluster, so the server can route it to the RPis it owns.
Here's a good way to handle that:
Set up a redis cache or other shared data store. It can be very small.
When each node.js server starts, have it register itself as active. That is, have it place its own specific address for handling API calls into the shared server. The specific address is probably of the form 12.34.56.78:3000: that is, an IP address and port.
Have each server update that address every so often, once a minute or so, to show it is still alive.
When an API call arrives at server.example.com, it will come to a more-or-less randomly chosen node.js server instance.
Get that server to read the list of server addresses from the redis cache
Get that server to repeat the API call to all servers except itself. Add a parameter like repeated=yes to the repeated API calls.
Then, each server looks at its list of connected sockets and does what your application requires.
On server shutdown, have the server unregister itself -- remove its address from redis -- if possible.
In other words, build a way of fanning out the API calls to all active node.js servers in your cluster.
If this must scale up to a very large number (more than a hundred or so) node.js servers, or to many hundreds of API calls a minute, you probably should investigate using message queuing software.
SECURE YOUR REDIS server from random cybercreeps on the internet.
I am working on a nodejs app with Socket.io and I did a test in a single process using PM 2 and it was no errors. Then I move to our production environment(We use Google Cloud Compute Instance).
I run 3 app processes and a iOS client connects to the server.
By the way the iOS client doesn't keep the socket connection. It doesn't send disconnect to the server. But it's disconnected and reconnect to the server. It happens continuously.
I am not sure why the server disconnects the client.
If you have any hint or answer for this, I would appreciate you.
That's probably because requests end up on a different machine rather than the one they originated from.
Straight from Socket.io Docs: Using Multiple Nodes:
If you plan to distribute the load of connections among different processes or machines, you have to make sure that requests associated with a particular session id connect to the process that originated them.
What you need to do:
Enable session affinity, a.k.a sticky sessions.
If you want to work with rooms/namespaces you also need to use a centralised memory store to keep track of namespace information, such as the Redis/Redis Adapter.
But I'd advise you to read the documentation piece I posted, things might have changed a bit since the last time I've implemented something like this.
By default, the socket.io client "tests" out the connection to its server with a couple http requests. If you have multiple server requests and those initial http requests don't go to the exact same server each time, then the socket.io connect will never get established properly and will not switch over to webSocket and it will keep attempting to use http polling.
There are two ways to fix this.
You can configure your clients to just assume the webSocket protocol will work. This will initiate the connection with one and only one http connection which will then be immediately upgraded to the webSocket protocol (with socket.io running on top of that). In socket.io, this is a transport option specified with the initial connection.
You can configure your server infrastructure to be sticky so that a request from a given client always goes back to the exact same server. There are lots of ways to do this depending upon your server architecture and how the load balancing is done between your servers.
If your servers are keeping any client state local to the server (and not in a shared database that all servers access), then you will need even a dropped connection and reconnect to go back to the same server and you will need sticky connections as your only solution. You can read more about sticky sessions on the socket.io website here.
Thanks for your replies.
I finally figured out the issue. The issue was caused by TTL of backend service in Google Cloud Load Balancer. The default TTL was 30 seconds and it made each socket connection tried to disconnect and reconnect.
So I updated the value to 3600s and then I could keep the connection.
I have spun 3 node instances using pm2. They are all running a websocket server using these ports: (9300, 9301, and 9302).
My main server acts as a nginx load balancer. The nginx upstream block:
upstream websocket {
least_conn;
server 127.0.0.1:9300;
server 127.0.0.1:9301;
server 127.0.0.1:9302;
}
After 10 players have connected, they are distributed in round-robin fashion. I am also utilizing Redis for Pub/Sub for all the node instances.
I am curious if it's possible for a connected player that is on instance 9300, switch to 9302 while not losing their connection?
The reasoning is because my game is instance based. I have "games" if you will, that players can create or join. If I can get the connected players onto the same node instance for their games, I would reduce all the extra Pub/Sub signals and achieve better latency. (Or so I think, but just curious if this is possible)
I am curious if it's possible for a connected player that is on
instance 9300, switch to 9302 while not losing their connection?
No, it is not possible. A TCP socket is a connection between two specific endpoints and it cannot be moved from one endpoint to another after it is established. There are very good security reasons why this is prohibited (so connections can't be hijaacked).
The usual way around this problem is for the server to tell the client to reconnect and give it instructions for how to connect to the particular server you want it connected to (e.g. connect to a specific port or specific hostname or some other means that your load balancer might use).
I am trying to design a client server model for chat application.Do I need to have two different socket with different port to send and recieve.Actually I have 3 nodes.1 server and 2 client(or slaves).Message passed by client 1 for client 2 will always routed via server.What kind of port and number of socket i should use.Please exaplain I doing all these things in C++ using winsock and pthread API's.
Typically you use TCP (SOCK_STREAM). You need just one socket to for incoming connections. bind(2) it to a specific port. You can accept(2) any number of connections on it.
Several threads can wait (accept(2)) on the same socket. The OS will dispatch one connection to just one thread ;)
For each outgoing connection you have to create a new socket but you don't have to bind it. The OS will choose an appropriate IP/port itself.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I was reading a lot about nodejs but still not clear about following :
With TCP protocol client and server agree on one port and then can maintain a connection. Server knows IP address of client and hence can send back messages. If we use nodejs then multiple clients can connect to same nodejs server on same port. How this can be possible? How multiple connections can be established on same port by same server.
If client is behind NAT then its IP can be dynamic, so how can nodejs server can send data to client.
What will be resource utilization in maintaining persistent connections on server and client ?
What happend when nodejs server crashes? How can client initiate connection again ?
If there is network problem on client side and it terminates and initiate connection after every 5 mins ..then is there a way this scenario can be handled?
With TCP protocol client and server agree on one port and then can maintain a connection.Server knows IP address of client and hence can send back messages.If we use nodejs then multiple clients can connect to same nodejs server on same port. How this can be possible? How multiple connections can be established on same port by same server.
There is no limitation to the number of connections which can be maintained on a single port (although, in practice, there may be operating system or hardware limitations). Of course, only a single process can listen on a port, but that has nothing to do with connections. That's not node-specific, all TCP servers work like that.
If client is behind NAT then its IP can be dynamic , so how can nodejs server can send data to client.
You're essentially asking how NAT works. Again, there's nothing node-specific in this case. The NAT server simply alters packet headers as necessary, and maintains translation tables for routing, just like with any connection.
What will be resource utilization in maintaining persistent connections on server and client
The overhead is really quite minimal for just the connection itself. A little bit of memory, but almost insignificant in the big picture. If you're storing additional associated data with each connection, that may be different. Node.js handles large number of concurrent connections very well, but if you're concerned, you can always search for benchmark tests, or write your own.
What happend when nodejs server crashes?How can client initiate connection again?
Sockets emit both close and error events. Simply listen for them, and attempt to reconnect afterwards, probably with a back-off delay.
If there is network problem on client side and it terminates and initiate connection after every 5 mins ..then is there a way this scenario can be handled?
Not entirely sure what you're asking here. The client simply needs to reconnect as stated in the previous question/answer. If you're associating certain data with a client socket, and you want a grace period where the client has a chance to reconnect before that data is freed, then you'll need to set a timeout which will free those resources after a certain amount of time. Then, in the connection listener, or perhaps on an authentication event, you'll want to analyse the new connection to see if it matches a recently disconnected client.
I would definitely recommend looking at socket.io, especially if your use case is web-based, although it can be used for more than just browser/server connections. It will do a lot of the things you seem to be concerned about (reconnection, resource association, disconnect grace period, etc) more or less automatically.
TCP connection is kept continously open. Saving is achieved by client not having to continously refresh from server in a long pooling manner in order to check if there are any new messages on the server like in case of AJAX. Creating a new connection every couple of seconds for client to refresh from server is heavy on the server, proxies and routers. In case of Node.js connection is kept open, but it is not active until the client or server have something to send. Found good article here http://www.html5rocks.com/en/tutorials/websockets/basics/
Imagine having a 1000 chat clients and each of them asks server every 3 seconds if any new messages arrived. That results in 2000 requests and responses per minute on the server. In case of Node.js server will send message to client only when there is a message to send, while all 1000 connections will be idle in the meantime, but will be kept open.
TCP connections are always initiated on the same port like port 80, but communication is maintaines on different ports assigned to each connection when it is open. So you would still need to keep connections continously open, but you will not have to send pooling messages continously like you needed before.