I've been a ruby/php web application developer for quite some time and I'm used to the idea of horizontal scaling of server instances to handle more requests. Horizontal scaling - meaning separate instances of an application sitting behind a load-balancer that share nothing and are unaware of each other.
The main question I have is, since Node.js and it's emphasis on evented-io allows for a single box running a node.js server to handle 'thousands' of simultaneous requests - is load-balancing/horizontal scaling used to scale nodejs applications? Is scaling a node app limited to vertical scaling (throwing more RAM/Processing power at the problem)?
My second question has to do with node.js horizontal scaling and websockets. I've seen quite a few Node.js 'chat' tutorials out there that make use of websockets.
(favorite: http://martinsikora.com/nodejs-and-websocket-simple-chat-tutorial)
Since websockets effectively keep an open line of communication open between a browser and a server, would a horizontally scaled architecture typical of the PHP/Ruby world cause a chat application like the one explained in the link to break - as new websocket connection requests would be assigned to different processes/servers and there would be no one central resource tracking all connected clients?
Node.js supports horizontal scaling in much the way you describe via the built-in cluster module.
Regarding your second question about the use of websockets/socket.io in this environment, you have to use something like Redis to store shared state across multiple instances of your application as described here.
Node.js's cluster functionality is limited to single server with multiple processor. Mainly it leverages number of processors in server. I think the question if more about the scenario when we want to scale horizontally with multiple servers with a Load balancer facade.
If you have node.js instances spread across multi servers(horizontal scaling), it will serve the same purpose, you need to program it properly to support this type of setup.
Related
I've started to port a web app backend to Google App Engine for scaling. But I'm completely new to GAE and just reading into the concepts. Steep learning curve.
I'm 95% certain that at some point many millions or at another point at least hundreds of thousands of users will start using the web app through a GUI app that I'm writing. And they will be globals users, so at some point in the future I'm expecting a relatively stable flow of connection requests.
The GAE Standard Environment comes to mind for scaling.
But I also want the GUI app to react when user related data changes in the backend. Which suggests web sockets, which aren't supported in the Standard Environment, but in the Flexible Environment.
Here's my idea: The main backend happens in a Standard app, but the GUI listens to update notifications from a Flexible app through web sockets. The Standard app calls the Flexible app after noteworthy data changes have occurred, and the Flexible app notifies the GUI.
But is that even possible? Because sibling Flexible instances aren't aware of each other (or are they?), how can I trigger the persistent connections held by the Flexible instance with an incoming call from the Standard app to send out a notification?
(The same question goes for the case where I have only one Flexible app and no Standard app, because the situation is kind of the same.)
I'm assuming that the Flexible app can access the same datastore that the Standard app can. Didn't look this one up.
Please also comment on whether the Standard app is even a good idea at all in this case and I should just go with Flexible. These are really new concepts to me.
Also: Are there limits to number of persistent connections held by a Flexible app? Or will it simply start another instance if a limit is reached?
Which of the two environments end up cheaper in the long run?
Many thanks.
You can only have one App engine instance per project however you can have multiple flex services or standard services inside of an instance.
Whether if standard is a good idea it depends up to your arquitecture, I'm pretty sure you've looked at the comparison chart, from experience is that if your app can work okay with all the restrictions (code runtimes, no availability to do background process, no SSH debugging, among others) I will definitely go for standard since it has a very good performance when working with spikes of traffic deploys new services in just seconds, keep in mind that automatic scaling is needed for the best performance result.
There are multiple ways to connect between flex or standard services one would be to just send an HTTP request from one service to another, but some other options with GCP services like Pub/Sub.
In the standard environment, you can also pass requests between
services and from services to external endpoints using the URL Fetch
API.
Additionally, services in the standard environment that reside within
the same GCP project can also use one of the App Engine APIs for the
following tasks:
Share a single memcache instance.
Collaborate by assigning work
between services through Task Queues.
Regarding Data Store you can access the same datastore from different services here is a quickstart for flex and the quickstart for standard
Which of the two environments end up cheaper in the long run?
Standard pricing is based on instance hours
Flexible pricing is based on usage of vCPU, memory, and persistent disks
If your service run very hight performance process on short periods of time probably standard will be chepear, however if you run low performance process on long periods of time, flex will be chepear, but again it depends on each use case.
I am trying to create back-end with nodes and it will have three clients.
- Mobile App
- Web App
- Admin Panel
I heard that the node is a single thread app. To have maximum performance is it better to have three node servers or just one that connects all? I am using MERN stack.
The question doesn't have a single answer and could possibly be considered opinion based.
However, the ultimate choice of your architecture - wheter you will have a single app that serves the backend for multiple frontends or multiple separate apps for each frontend - doesn't really depend on performace considerations and is completely independent on the fact that node is single threaded or not.
It's because, regarding scalability and performance, you will possibly use the cluster module to scale your app to multiple processor cores and then, you will have a farm of servers processing incoming requests. This holds regardless of your architecture.
If I were you, I'd have a single application supporting multiple frontends from the single backend then. Assuming you can correctly control the access to the backend (so that for example, web users are not able to forge requests and interfere with the mobile app), a single app could possibly be easier to deploy.
Suppose we're building an SPA (Single Page Application) which depends on a couple of independent back-end systems. Basically a Microservices-esque implementation of SignalR-hubs.
Each of these back-end systems expose a separate SignalR endpoint with a separate set of hubs;
The separation of these hubs across different back-end systems is intentional in design and cannot be unified into a single one;
Basically, a single browser session to the app will have to maintain a separate SignalR-connection for each endpoint on which we depend.
From what we have gathered, this approach will not scale due to the fact - at this time of writing - most browsers seem to implement an arbitrary limit of concurrent websocket connections.
This leads us to the following questions:
Are we correct in our assumption that this will not scale in the browser?
If our assumption is correct, what are our options? Will we need to implement a single encompassing SignalR-endpoint?
I want to create Sails.js (Node.js) server app, which will provide API for single-page-app. This server will consist of multiple modules:
user management
forum
chat
admin GUI
content management
payment gateway
...
All these modules will share one database. The server must be able to handle as many requests and web sockets as possible. Clean architecture and performance are my primary goals.
My questions:
Should I create multiple servers running on multiple ports? I mean, one server for content management module. Another server for forum management module.
Or is it better to create only one big universal server, which will consists of multiple separate modules (hooks in Sails.js) and runs on one port? Will performance of the server decrease in this case ?
I was thinking about vertical scaling one big universal server, running on single port with pm2. Or is it better to scale Node.js horizontaly and split server to multiple smaller servers ?
Im new to Node.js so I appreciate any advice.
I think it really boils down to the scale of the project.
For very simple things there's no real reason to scale past a single but reliable server is there?
However for broader projects that have a back-end that is resource intensive and a lot of users and traffic, you may a want to split the back / front end aspects depending on the requirements.
In which case you might have a single server (or more) dealing with the specific administrative requests or routines then have the client / user API running through a load balancer and spread across multiple servers in multiple regions or break it down further into an auto scaling group so as to accommodate for fluctuations in traffic.
It would be worthwhile to note too that this is really suited for higher volumes of traffic or resource usage as you're dedicating the server infrastructure for this purpose, for smaller applications where there is infrequent usage then breaking things down into micro services from the start and getting billed for the runtime rather than dedicated infrastructure utilization might make more sense to me. You could take a look at AWS API Gateway and Lambda services for some more information on that (I am not affiliated to AWS in any way, I just appreciate what they have managed to put together there).
I want to scale my Node.js Socket application vertically and horizontally and I haven´t found a sophisticated solution yet.
My application has two use-cases:
Broadcast messages from one user to all others
Push messages from one user to a subset of users
On one hand, I´ve read that I need Redis for both cases together with socket.io-redis
On the other hand, I´ve watched this video and read this SO answer where it says that Redis isn´t reliable and it´s not guaranteed that the published messages will arrive, so you should only use it for clustering/vertical scaling
Microsoft Azures solution to use ServiceBus is out of question, because I don´t want to use Azure.
Instead of Redis, the guy recommends using RabbitMQ for horizontal scaling.
For the vertical scaling there is also socket.io-clusterhub, an IPC for node processes, but it seems to work only on Socket.io <= v0.9.0
Then there is this guy, who has implemented his own method to pass messages to other nodes via HTTP requests, which makes somehow sense. But why HTTP requests if you could also establish direct socket connections between servers, push the message to all servers simultaneously and overcome the delay of going from one server to another?
As a conclusion I thought maybe I could go with Redis on EACH server, just for the exchange of messages when clustering my application on multiple processes, together with RabbitMQ as a S2S communication solution.
But it seems a bit like an overkill to have one Redis per Server and another central RabbitMQ.
Is there any known shorter/better solution to scale Socket.io reliably in both directions?
EDIT:
I´ve tried using a single Redis Server for multiple Node.js Servers, where each of them uses Clustering via sticky-session over all cores. While the Clustering at its own works like a charm with redis, there seems to be a problem when using multiple servers. Messages won´t arrive at the other nodes.
I'd say Kafka is a good fit for the horizontal scaling. It is a fairly sophisticated way of distributing a huge amount of events across servers (which at the end is what you want). This is a good read about it: https://engineering.linkedin.com/kafka/running-kafka-scale
Regarding the vertical scale, instead of socket.io-clusterhub I would use something called PM2 (https://github.com/Unitech/pm2) which allows you to resize the scale of the apps in every computer dynamically as well as controlling the logs and reporting to keymetrics.io (if you are using it).
If you need any snippets ask me and I will edit the answer but in the PM2 github there are quite few.