I have nodejs app running on AWS EC2.
I would like to scale it up by creating more instances of it.
I don't quite understand how to do it on the networking side.
Lets say I create another instance and it's listening to a different port.
Should I change the client side to request from two different ports? I believe this could lead to race conditions on the DB
Am I suppose to listen on one port on the EC2 machine and direct the request to one of the instances? In that case the port will be busy until the instance is done with the request instead of processing requests in parallel with the other instance
Does anyone has some pointers or maybe can point me to some documents about this subject?
At the basic level, you'll have multiple instances of your Node.js application, connecting to a common database backend. (Your database should be clustered as well, but that's a topic for another post.)
If you're following best practices already, one request will be one response, and it won't matter if subsequent requests land on a different application server. That is, the client could hit any of them at any time and there won't be any side effects. If not, you'll need some sort of server pinning (usually done via cookie or similar methods) to ensure clients always land on the same application server.
The simplest way to load balance is to have your clients connect to a host name, and that hostname resolved round-robin style to several hosts. This will spread the traffic around, but not necessarily correctly. For instance, a particular command or issue on one host could mean it can only handle 5 requests, while the other servers can handle 5000. Most cloud providers have management of load balancing. AWS certainly does.
Since you're already on AWS, I'd recommend you deploy your application via Elastic Beanstalk. It automates the spin-up/tear-down as-configured. You could certainly roll your own solution, but it's probably just easier to use what's already there. The load balancer is configured with Beanstalk.
Related
I am looking for a way to cluster WebSocket servers - written in node - so that a proper load balancing and a client request will be served by appropriate node instance. In case of WebSocket, the connection is stateful and I believe a node cluster could help. I want the connection/state information to be shared so that any node instance could serve the request than the client does not need to keep a track of the specific node instance. The reason for this thought process is to ensure that the node instances can be killed and replaced by new instances without bothering about the overheads of state management.
I have a setup where we use multiple instances with load balancers in AWS ECS, deployed by CI/CD pipelines. The number of frontend and backend servers varies between 2 and 8 each depending on bursts and current deployments. If one server crashes, a new one will take its place.
We use socket.io with the Redis adapter to share the websocket state between all connected instances via the in-memory db Redis. This ensures that even if the clients are connected to different instances, they all receive the events.
This is more like a design question but I have no idea where to start.
Suppose I have a realtime Node.js app that runs on multiple servers. When a user logs in she doesn't know which server she will be assigned to. She will just login, do something and logout and that's it. A user won't be interacting with other users on a different server, nor will her details be stored on another server.
In the backend I assume the Node.js server will put the user's login details to some queue and then when there is space it will assign this user to an available server (A server that has the lowest ping value or is not full). Because there is a limit number of users on one physical server when the users try to login to a "full" server it will direct her to another available server.
I am using ws module of node.js. Is there any service available for this purpose or do I have to build my own? How difficult would that be?
I am not sure how websocket fits into this question. Ignoring it. I guess your actual question is about load balancing... Let me try paraphasing it.
Q: Does NodeJS has any load balancing feature that I can leverage?
Yes and it is called cluster in NodeJS. Instead of the traditional one node process listening on a single port, this module allows you to spawn a group of node processes and have them all binded to the same port.
This means is that all the user know is only the service's endpoint. He sends a request to it and 1 of the available server in the group will serve him whenever possible.
Alternatively using Nginx, the web server, as your load balancer is also a very popular approach to this problem.
References:
Cluster API: https://nodejs.org/api/cluster.html
Nginx as load balancer: http://nginx.org/en/docs/http/load_balancing.html
P.S
I guess the key word for googling solutions to your problem is load balancer.
Out of the 2 solutions I would recommend going the Nginx way as it is a much scalable approach
Example:
Your Node process could possibly be spread across multiple hosts (horizontal scaling). The former solution is more for vertical scaling, taking advantages of multi-cores machine.
We need to deploy multiple Node/Express apps on a set of servers (behind load balancer). These apps are fully independent of each other in terms of functionality. I'll first explain how I am thinking of doing this and then I am looking for input in terms of best practices, if there are any red flags in my design etc.
Here is the setup I am thinking about:
Frontend server behind load balancer will have node-http-proxy running and it'll accept incoming requests on port 80. This reverse proxy will route the requests to appropriate node apps that are running on different ports on this server. for ex:
var http = require('http'),
httpProxy = require('http-proxy');
var options = {
router: {
'myapphost.com/MyFirstApp': 'myapphost.com:3000',
'myapphost.com/MySecondApp': 'myapphost.com:3001'
}
}
// ...and then pass them in when you create your proxy.
var proxyServer = httpProxy.createServer(options).listen(80);
Each of the node app will be running on node cluster using something like Cluster2 to take advantage of multi-core systems.
My questions:
Is this right design and strategy?
Some of our applications need to be stateful. What is the best way to do the state management in this kind of a setup? Using external session storage such as Redis is the right approach? Or pinning a session to a given frontend machine and using in-memory session storage?
UPDATE:
Since I posted this question, after talking to couple of people, there is one more approach that has come up--
I can use Nginx as a reverse proxy and load balancer in front of my frontend machines. Each frontend machine will serve only one app. There can be one more back up machine for that app. (depending upon requirement). So if I have three apps, I'll have three separate machines each one serving different app. All the requests will be received by Nginx on port 80, Nginx reverse proxy will route the request to the right frontend machine. Each machine will have Node cluster to take advantage of multi-core system. Advantage of this approach is-- deployment becomes lot easier for each app. Also, we can scale each app separately.
Please share your thoughts on this approach as well.
This is a right approach, as well you mentioned redis, or you could use any other session storages like connect-mongo or any other session storage.
If you are using Load Balancer, I guess you have multiple instances of same servers? If this is the case, then you need to analyze session performance and how much it will be used, and then decide if you need to shard session storage/database, or one single machine that every balanced instance will be talking to.
You already think the right way, why wont you try and hack some of it, and see if it fits your needs?
As well you need to think about static media, possibly to store it separately (S3 + CloudFront?).
As well as how you are going to deliver updates to instances and restart them while still keeping application logic consistency.
This structure as well enables possibilities for AB testing (you can load balance to specific instances with "testing version" of application. While majority will be balanced across main instances.
It all depends on your scale as well, and sometimes you really don't need that much initially, just be ready to improve and scale in the future.
When coming to deploy Node.JS/Meteor for large scale application a single CPU will not be sufficient. We also would like to have it on multiple servers for redundancy.
What is the recommended setup for such deployment ? how does the load balancing works ? will this support the push data technology for cross servers clients (one client connects to server 1, 2nd client connects to server 2 and we would like an update in client one to be seen in client 2 and vice versa).
Thanks Roni.
At the moment you just need to use a proxy between them. The paid galaxy solution should help but details are scarce at the moment as the product isn't out yet.
You can't simply proxy (normally using nginx, etc) between two servers as each server will store the user's state (i.e their login state) during the DDP Session (the raw wire protocol meteor uses to transmit data).
There is one way you could do it at the moment. Get meteorite and install a package called meteor-cluster.
The package should help you relay data between instances and relay data between the instances via Redis. A youtube video also shows this and how to set it up.
An alternative solution is to use Hipache to manage the load balancing. You can use multiple workers (backends) for one frontent, like so:
$ redis-cli rpush frontend:www.yourdomain.com http://address.of.server.2
$ redis-cli rpush frontend:www.yourdomain.com http://address.of.server.2
There is more information on how to do this in the git page I linked to above, there is a single config file to edit and the rest is done for you. You may also want to have a dedicated server for MongoDB.
folks. I wander whether there is any high-level API for servers' communication in Node.js framework? For example, I have several servers, where my application runs and I want to control loading of this servers. Sometimes, if some server is overloaded I want to redirect some connection requests to another(more free one). Are there any functions which could help me? Or I have to implement my own functionality?
Try looking at cluster. This allows you to control multiple node proccess and scale nicely.
Alternatively just set up TCP sockets and pass messages around over TCP or pass messages around over a database like redis.
You should be able to pipe HTTP connection down streams. You have one HTTP server as a load balancer and this server just passes messages on to your other servers and passes them back.
You're looking for what's called a load balancer. There are many off-the-shelf solutions, nginx being one of the standards today (and VERY quick/easy to set up).
I don't know of a node-native solution, but it's not that hard to write one. In general, however, load balancers don't actually monitor server load, they monitor whether a server is live or not and distribute traffic relatively equally.
As for your communications question, no -- there's no standardized API to communicate to/from node.js servers. Again, however, not hard to set up -- Assuming you're already hosting HTTP (using express, or native), just listen for specific requests, perhaps to /comm/ or whatever you deem appropriate and pass JSON back-and-forth.
Not too sure for Nodejs but I've heard ppl using Capistrano with Nodejs