If I were writing an application using the MEAN stack, and the database is optimized sufficiently enough to almost never be the bottleneck, can Node.js itself become a bottleneck due to the site traffic and/or number of concurrent users? This is purely from the perspective of Node.js being an asynchronous, single threaded event loop. One of the first tenets of Node.js development is to avoid writing code that performs CPU intensive tasks.
Say, if I had to post-process the data returned from MongoDB and that was even moderately CPU intensive, it sounds like that should be handled by a service layer sitting in between Node.js and MongoDB, that is not pounding the same CPU dedicated to Node.js. Techniques such as process.nextTick() are harder to comprehend and more importantly to realize when to use them.
Forgive me for this borderline rant, but I really do want to have a better idea of Node.js' strengths and weaknesses.
Related
Let's assume we are stress testing 2 servers that perform database read/write operations without caching, and make network calls. One is a Node.js server, and the other is an ASP.Net Core one that utilizes Tasks. Assume that both perform the same operations, receive the same requests/second, and run on machines with equal processing power.
I've read that Node.js is very performant on I/O operations, but I can't get my head around it. Sure, the awaited db and network calls run in a non-blocking way, however each of these calls are handled by an individual thread from a limited thread pool. This doesn't sound performant to me at all. I would be forced to implement a caching mechanism to mitigate this, which is something I don't really like. So either Node.js is not the greatest choice for these kinds of operations, or I have incorrect knowledge.
As for ASP.NET Core, I don't know the internals of it, but I'm pretty sure it doesn't have the thread limitation issues Node.js has, so it should have shorter response times by logic. Yet I still can't know for sure if this is the case due to resource consumption and context switching cost concerns.
So which of these 2 would theoretically have shorter response times, and why exactly?
Why this question?
I’m learning things about js performances and web rendering. This post was very useful.
If you follow some links you will land here and read this:
User code in Node.js runs in a single thread, so for compute operations (as opposed to I/O), you can execute them concurrently, but not in parallel.
So I’ve read things about concurrency and parallelism in nodeJS. What I’ve learned is that nodeJS is:
Parallel for I/O bound tasks since it’s handled by libuv
Concurrent for CPU bound tasks
This explains why renderToString is a slow operation since it is CPU bound. But it seems that there is a way to enable parallelism of CPU bound tasks in nodejs: clustering.
The question
This is why I’m here. Do you know why renderToString isn’t clustered (don’t know if this is valid English)?
Maybe it’s too complicated?
Maybe it just can’t be done in parallel?
Maybe it doesn't improve performances for some reason?
I would like to understand why. Because after those readings, I tend to think that nodeJS is very performant when it comes to dealing with I/O, but it seems to also be performant for CPU bound tasks since you can create clusters. Nevertheless it doesn’t seems to be trivial and it’s a choice to consider in some specific cases.
So this leads us to one bonus question: what are the limitations/drawbacks of nodejs’s clusters ? (Excepting the fact that it seems to be complicated to setup and to maintain on large projects?)
It would not make sense to place the abstraction at this level.
It is not hard to run a renderToString() as a cluster. For instance, you can easily use the worker-farm library.
The problem is this becomes hard to use in a beneficial way, because the "store" of data built for each incoming request must be in scope for the entire component tree that renderToString() works on.
Perhaps though with the experimental worker threads node.js library, we might get some multi-threaded renderToString. But, the work on SSR (Server Side Rendering) of React is not nearly as active as the client.
Maybe with the work to allow React to suspend a tree its rendering, and start it up, we'll eventually have a thread that can continue rendering while primary thread acts on incoming request/action.
Can you please explain what is the I/O Scaling Problem in Node.js Performance?
I am reading the book "Basarat Ali Syed - Beginning Node.js-Apress" but the explanation is not enough to I/O Scaling.
A server typically has a mix of computational things to do and I/O things to do (getting data from somewhere like a database or a disk or another server). In today's modern servers with pretty fast multi-core processors, it is more common for a given server request to be limited by I/O than by CPU.
So, if you're going to scale a server to be able to handle lots of requests and to handle them with good performance, you have to find a way to be able to most efficiently handle lots of I/O requests because that's probably what your server is limited by. This would be the "I/O scaling problem". How to scale your server and code architecture to handle lots of I/O requests very efficiently.
It so happens that the node.js single-threaded architecture with asynchronous I/O is very efficient at doing lots of I/O and can be more efficient than other server architectures that use multiple threads and blocking I/O calls.
If you go to your Table of Contents in that book, you will see the following:
Understanding Node.js Performance
The I/O Scaling Problem
Traditional Web Servers Using a Process Per Request
Traditional Web Servers Using a Thread Pool
The Nginx way
Node.js Performance Secret
I don't have the book myself, but I would presume that "The I/O Scaling Problem" section of the book describes it for you. And, then you can read about the node.js performance secret for how it handles this. The servers that use a process or a thread per request take more system resources to have lots of requests in flight at the same time (which is one key to handle lots of requests). The node.js non-blocking I/O model, on the other hand, is very efficient at handling lots and lots of in-flight requests.
I have been developing in Node.js recently and have a good idea of what is going on as far as the event loop. Given that I had experience with javascript, Node made sense for me to use, but I wonder, has anyone ever stopped using a multithreading system and went to async for PERFORMANCE? Or made a choice to go with async instead of multithreading for performance?
What are real-life examples of async non-blocking I/O triumphing over multithreading in the real world?
Regardless you do multithreading or not, all modern operating system I/O operations are inherently async. So saying that "async is faster" is not actually true. However, async programming allows you to conserve threads with almost the same performance. That's important on a server because number of threads you can get is finite (limited by available memory).
So with async you don't necessarily get better performance but better scalability on I/O heavy tasks. Async could even perform worse on CPU-intensive tasks since you don't benefit of parallelizing work between server cores. I/O on the other hand, works with interrupts and DMA which does not involve CPU. That's how you can get good-enough performance because you can keep executing until hardware notifies you of I/O completion by signaling an interrupt.
Edit: I just figured out that I did not answer your actual question. But as long as you know how async benefits you, you may not need real world examples on that. Just know that:
communicating with database is I/O
reading from or writing to disk is I/O
sending/receiving packets over network is I/O
the rest is CPU
Depending on if you use I/O or CPU more async could give you great scalability or worse performance. Typically web applications are I/O intensive, which benefits from async programming.
It is easier to code, because you do not need to manage consistencies/data exchange between multiple processes/threads.
There is no real advantage for this model when creating just a simple web application, but when having a server managing multiple connections at the same time, where the data that is passed is somehow tied together.
Perfomance-wise there are some advantages because having multiple threads/processes tend to use way more memory than just a single thread.
Simple math: Having one thread handling one connection and each thread needs 1MB in RAM it is very easy to figure out how much connection you could on one computer. With node.js you have one process handling multiple connections and it takes for example 20MB in RAM and 4kb per new connection. I think you get the idea.
Async I/O is not faster, but it helps you consume fewer resources while doing the same thing multple times at the same time.
Trying to grasp some basics of Redis I came across an interesting blog post .
The author states:
Redis is single-threaded with epoll/kqueue and scale indefinitely in terms of I/O concurrency.
I surely misunderstand the whole threading thing, because I find this statement puzzling. If a program is single-threaded, how does it do anything concurrently? Why it is so great that Redis operations are atomic, if the server is single-threaded anyway?
Could anybody please shed some light on the issue?
Well it depends on how you define concurrency.
In server-side software, concurrency and parallelism are often considered as different concepts. In a server, supporting concurrent I/Os means the server is able to serve several clients by executing several flows corresponding to those clients with only one computation unit. In this context, parallelism would mean the server is able to perform several things at the same time (with multiple computation units), which is different.
For instance a bartender is able to look after several customers while he can only prepare one beverage at a time. So he can provide concurrency without parallelism.
This question has been debated here:
What is the difference between concurrency and parallelism?
See also this presentation from Rob Pike.
A single-threaded program can definitely provide concurrency at the I/O level by using an I/O (de)multiplexing mechanism and an event loop (which is what Redis does).
Parallelism has a cost: with the multiple sockets/multiple cores you can find on modern hardware, synchronization between threads is extremely expensive. On the other hand, the bottleneck of an efficient storage engine like Redis is very often the network, well before the CPU. Isolated event loops (which require no synchronization) are therefore seen as a good design to build efficient, scalable, servers.
The fact that Redis operations are atomic is simply a consequence of the single-threaded event loop. The interesting point is atomicity is provided at no extra cost (it does not require synchronization). It can be exploited by the user to implement optimistic locking and other patterns without paying for the synchronization overhead.
OK, Redis is single-threaded at user-level, OTOH, all asynchronous I/O is supported by kernel thread pools and/or split-level drivers.
'Concurrent', to some, includes distributing network events to socket state-machines. It's single-threaded, runs on one core, (at user level), so I would not refer to this as concurrent. Others differ..
'scale indefinitely in terms of I/O concurrency' is just being economical with the truth. They may get more belief if they said 'can scale better than one-thread-per-client, providing the clients don't ask for much', though they may then feel obliged to add 'blown away on heavy loading by other async solutions that use all cores at user level'.