Nodejs can not have a built-in thread API like java and .net
do. If threads are added, the nature of the language itself will
change. It’s not possible to add threads as a new set of available
classes or functions.
Nodejs 10.x added worker threads as an experiment and now stable since 12.x. I have gone through the few blogs but did not understand much maybe due to lack of knowledge. How are they different than the threads.
Worker threads in Javascript are somewhat analogous to WebWorkers in the browser. They do not share direct access to any variables with the main thread or with each other and the only way they communicate with the main thread is via messaging. This messaging is synchronized through the event loop. This avoids all the classic race conditions that multiple threads have trying to access the same variables because two separate threads can't access the same variables in node.js. Each thread has its own set of variables and the only way to influence another thread's variables is to send it a message and ask it to modify its own variables. Since that message is synchronized through that thread's event queue, there's no risk of classic race conditions in accessing variables.
Java threads, on the other hand, are similar to C++ or native threads in that they share access to the same variables and the threads are freely timesliced so right in the middle of functionA running in threadA, execution could be interrupted and functionB running in threadB could run. Since both can freely access the same variables, there are all sorts of race conditions possible unless one manually uses thread synchronization tools (such as mutexes) to coordinate and protect all access to shared variables. This type of programming is often the source of very hard to find and next-to-impossible to reliably reproduce concurrency bugs. While powerful and useful for some system-level things or more real-time-ish code, it's very easy for anyone but a very senior and experienced developer to make costly concurrency mistakes. And, it's very hard to devise a test that will tell you if it's really stable under all types of load or not.
node.js attempts to avoid the classic concurrency bugs by separating the threads into their own variable space and forcing all communication between them to be synchronized via the event queue. This means that threadA/functionA is never arbitrarily interrupted and some other code in your process changes some shared variables it was accessing while it wasn't looking.
node.js also has a backstop that it can run a child_process that can be written in any language and can use native threads if needed or one can actually hook native code and real system level threads right into node.js using the add-on SDK (and it communicates with node.js Javascript through the SDK interface). And, in fact, a number of node.js built-in libraries do exactly this to surface functionality that requires that level of access to the nodejs environment. For example, the implementation of file access uses a pool of native threads to carry out file operations.
So, with all that said, there are still some types of race conditions that can occur and this has to do with access to outside resources. For example if two threads or processes are both trying to do their own thing and write to the same file, they can clearly conflict with each other and create problems.
So, using Workers in node.js still has to be aware of concurrency issues when accessing outside resources. node.js protects the local variable environment for each Worker, but can't do anything about contention among outside resources. In that regard, node.js Workers have the same issues as Java threads and the programmer has to code for that (exclusive file access, file locks, separate files for each Worker, using a database to manage the concurrency for storage, etc...).
It comes under the node js architecture. whenever a req reaches the node it is passed on to "EVENT QUE" then to "Event Loop" . Here the event-loop checks whether the request is 'blocking io or non-blocking io'. (blocking io - the operations which takes time to complete eg:fetching a data from someother place ) . Then Event-loop passes the blocking io to THREAD POOL. Thread pool is a collection of WORKER THREADS. This blocking io gets attached to one of the worker-threads and it begins to perform its operation(eg: fetching data from database) after the completion it is send back to event loop and later to Execution.
Related
I have a question, Node.js uses libuv inside of u core, to manage its event loop and by default works whit 4 threads and process queue whit limit of 1024 process.
Process queue limit
Threads by default
So, because most programmers say it's single thread?
By default, node.js only uses ONE thread to run your Javascript. Thus your Javascript runs as single threaded. No two pieces of your Javascript are ever running at the same time. This is a critical design element in Javascript and is why it does not generally have concurrency problems with access to shared variables.
The event driven system works by doing this:
Fetch event from event queue.
Run the Javascript callback associated with the event.
Run that Javascript until it returns control back to the system.
Fetch the next event from the event queue and go back to step 2.
If no event in the event queue, go to sleep until an event is added to the queue, then go to step 1.
In this way, you can see that a given piece of Javascript runs until it returns control back to the system and then, and only then, can another piece of Javascript run. That's where the notion of "single threaded" comes from. One piece of Javascript running at a time. It vastly simplifies concurrency issues and, when combined with the non-blocking I/O model, it makes a very efficient system, even when lots of operations are "in flight" (though only one is actually running at a time).
Yes, node.js has some threads inside of libuv that are used for things like implementing file system access. But those are only for native code inside the library and do NOT make your Javascript multi-threaded in any way.
Now, recent versions of node.js do have Worker Threads which allow you to actually run multiple threads of Javascript, but each thread is a very separate environment and you must communicate with other threads via messages without the direct sharing of variables. This is relatively new to nodejs version 10.5 (though it's similar in concept to WebWorkers in the browser. These Worker Threads are not used at all unless you specifically engage them with custom programming designed to take advantage of them and live within their specific rules of operation.
Node.js maintains an event loop but then it also has by default four threads for the complicated requests. How this is single threaded when there are more threads available in the thread pool?
Also, the threads assigned by the event loop for the complicated task are the dedicated threads then how it's different from other multithreading concepts?
In the context to which you're referring, "single threaded" means that your Javascript runs as a single thread. No two pieces of Javascript are ever running at the same time either literally or time sliced (note: as of 2020 node.js does now have WorkerThreads, but those are something different from this original discussion). This massively simplifies Javascript development because there is no need to do thread synchronization for Javascript variables which are shared between different pieces of Javascript because only one piece of Javascript can ever be running at the same time.
All that said, node.js does use threads internal to its implementation. The default four threads you mention are used in a thread pool for disk I/O. Because disk I/O is normally a synchronous operation at the OS level that blocks the calling thread and node.js has a design where all I/O operations should be offered as asynchronous operations, the node.js designers decided to fulfill the asynchronous interface by using a pool of threads in order to implement (in native code), the fs module disk I/O interface (yes there are non-blocking disk I/O operations in some operating systems, but the node.js designers decided not to use them). This all happens under the covers in native code and does not affect the fact that your Javascript runs only in a single thread.
Here's a summary of how a disk I/O call works in node.js. Let's assume there's already an open file handle.
Javascript code calls fs.write() on an existing file handle.
fs module packages the arguments to the function and then calls native code.
Native code gets a thread from the thread pool and initiates the OS call to write data to that file
Native code returns from the function
fs module returns from the fs.write() call
Javascript continues to execute (whatever statements came after the fs.write() call
Some time later the native code fs.write() call on a thread finishes. It obtains a mutex protecting the event loop and inserts an event in the event queue.
When the Javascript engine is done executing whatever stream of Javascript it was running, it checks the event queue to see if there are any other events to run.
When it finds an event in the event queue, it removes it from the event queue and executes the callback associated with that event, starting a new stream of running Javascript.
Because a new event is never acted upon until the current stream of Javascript is done executing, this is where Javascript gets is event-driven, single threaded nature even though native code threads may be used to implement some library functions. Those threads are used to make a blocking operation into a non-blocking operation, but do not affect the single threaded-ness of Javascript execution itself.
The key here is that node.js is event driven. Every new operation that triggers some Javascript to run is serialized through the event queue and the next event is not serviced until the current stream of Javascript has finished executing.
In the node.js architecture the only way to get two pieces of Javascript to run independently and at the same time is to use a separate node.js process for each. Then, they will run as two completely separate operations and the OS will manage them separately. If your computer has at least two cores, then they can literally run at the same time, each on their own core. If your computer has only one core, they will essentially be in their own process thread and the OS will time slice them (sharing the one CPU between them).
I will tell it in a clear and simple way and clear the confusion :
Node Event Loop is SINGLE-THREADED But THE Other processes are not.
The confusion came from c++, which Node uses underline ( NodeJs is about 30% js + 70% c++ ).So, By default, The JS part of NodeJs is single-threaded BUT it uses a thread pool of c++. So, We have a single thread JS which is the event loop of NodeJs + 4 threads of c++ if needed for asynchronous I/O operations.
It is also important to know that The event loop is like a traffic organizer, Every request go through the loop ( which is single-thread ) then the loop organizes them to the pool threads if I/O processes are needed, so if you have a high computational app that does like heavy lifting image-processing, video-editing, audio-processing or 3d-graphics ..etc, which is not needed for most apps,So NodeJs will be a bottleneck for that high load computational app and the traffic organizer will be unhappy.
While NodeJS shine for I/O bound apps ( most apps ) Like apps dealing with databases and filesystem.
Again: By default, NodeJs uses a 4 thread pool (PLUS one thread for the event loop itself ). so by default (total of 5) because of the underlying c++ system.
As a general idea, The CPU could contain one or more cores, it depends on your server(money).
Each core could have threads. Watch your activity Monitor discover how many threads you are using.
Each process has multiple threads.
The multi-threading of Node is due to that node depends on V8 and libuv ( C Library ).
So Long story short:-
Node is single-threaded for the event loop itself but there are many operations that are done outside the event Loop, Like crypto and file system (fs ). if you have two calls for crypto then each of them will reach each THREAD ( imagine 3 calls to crypto and 1 for fs, These calls will be distributed one for each thread from the 4 thread pool )
Finally: It is very easy to increase the default number of threads of the C-Library libuv thread pool which is 4 by default by changing the value of process.env.uv_threadpool_size. and also you could use clustering ( PM2 recommended ) to like clone the event-loop, like have multiple event-loops in case the single-threaded one is not enough for your high load app.
So nobody illustrates that thread pool is a c++ thing that’s nodeJs control mostly not the developer, which still asking How it’s single-thread while having a thread-pool !!
Hope that simplifies that advanced topic.
By default, the execution of your JavaScript code runs on a single thread.
However, node.js tries to make most long-running calls async. For some that just involves doing async OS calls, but for some others node.js will execute the call itself on a secondary thread, while continuing to run other JS code. Once the async call terminated, the Js callback or Promise handler will run.
For async processing, Node.js was created explicitly as an experiment. It is believed that more performance and scalability can be achieved by doing async processing on a single thread under typical web loads than the typical thread based implementation.
As far as I know, all IO requests and other asynchronous tasks are done by libuv in nodejs.
I want to know if libuv is using threading. If it is, is it using all available core or not?
First of all, what is libuv. As mentioned in the documentation, it's a multi-platform support library with a focus on asynchronous I/O.
libuv doesn't use thread for asynchronous tasks, but for those that aren't asynchronous by nature.
As an example, it doesn't use threads to deal with sockets, it uses threads to make synchronous fs calls asynchronous.
When threads are involved, libuv uses a thread pool the size of which you can change at compile-time using UV_THREADPOOL_SIZE.
node.js is provided with a precompiled version of libuv and thus a fixed UV_THREADPOOL_SIZE parameter.
It goes without saying that it has nothing to do with the number of cores of your chip.
I'm tempted to affirm that you can safely ignore the topic, for libuv and thus node.js don't use threads intensively for their purposes (unless you are using them in a really perverse way or if you are running an high number of libuv work requests).
Feel free to run an instance of node.js per core if you need as most of the users do.
The design overview section of libuv is also clear enough about this point:
The I/O (or event) loop is the central part of libuv. It establishes the content for all I/O operations, and it’s meant to be tied to a single thread. One can run multiple event loops as long as each runs in a different thread.
The libuv module has a responsibility that is relevant for some particular functions in the standard library. for SOME standard library function calls, the node C++ side and libuv decide to do expensive calculations outside of the event loop entirely.They make something called a thread pool that thread pool is a series of four threads that can be used for running computationally intensive tasks such as hashing functions.
By default libuv creates four threads in this thread pool. Thread Pool in the picture is organized by the Libuv So that means that in addition to that thread used for the event loop there are four other threads that can be used to offload expensive calculations that need to occur inside of our application. Many of the functions include in the node standard library will automatically make use of this thread pool.
Network (Network IO) is responsible for api requests, File system (File IO) is fs module. so node.js single thread delegates those heavy work to the libuv
If you have too many function calls, It will use all of the cores. CPU cores do not actually speed up the processing function calls, they just allow for some amount of concurrency inside of the work that you are doing.
From here:
A single instance of Node.js runs in a single thread. To take
advantage of multi-core systems the user will sometimes want to launch
a cluster of Node.js processes to handle the load.
The cluster module allows easy creation of child processes that all
share server ports.
Multiple processes could be better than multithreading in some cases. Some people even think theads are evil. Maybe node.js is designed in such a way to take advantage of processes better than threads.
The following two diagrams are my understanding on how threads work in a event-driven web server (like Node.js + JavaScript) compared to a non-event driven web server (like IIS + C#)
From the diagram is easy to tell that on a traditional web server the number of threads used to perform 3 long running operations is larger than on a event-driven web server (3 vs 1.)
I think I got the "traditional web server" counts correct (3) but I wonder about the event-driven one (1). Here are my questions:
Is it correct to assume that only one thread was used in the event-driven scenario? That can't be correct, something must have been created to handle the I/O tasks. Right?
How did the evented server handled the I/O? Let's say that the I/O was to read from a database. I suspect that the web server had to create a thread to hand off the job of connecting to the database? Right?
If the event-driven web server indeed created threads to handle the I/O where is the gain?
A possible explanation for my confusion could be that on both scenarios, traditional and event-driven, three separate threads were indeed created to handle the I/O (not shown in the pictures) but the difference is really on the number of threads on the web server per-se, not on the I/O threads. Is that accurate?
Node may use threads for IO. The JS code runs in a single thread, but all the IO requests are running in parallel threads. If you want some JS code to run in parallel threads, use thread-a-gogo or some other packages out there which mitigate that behaviour.
Same as 1., threads are created by Node for IO operations.
You don't have to handle threading, unless you want to. Easier to develop. At least that's my point of view.
A node application can be coded to run like another web server. Typically, JS code runs in a single thread, but there are ways to make it behave differently.
Personally, I recommend threads-a-gogo (the package name isn't that revealing, but it is easy to use) if you want to experiment with threads. It's faster.
Node also supports multiple processes, you may run a completely separate process if you also want to try that out.
The best way to picture NodeJS is like a furious squirrel (i.e. your thread) running in a wheel with an infinite number of pigeons (your I/O) available to pass messages around.
I/O in node is "free". Your squirrel works to set up the connection and send the pigeon off, then can go on to do other things while the pigeon retrieves the data, only dealing with the data when the pigeon returns.
If you write bad code, you can end up having the squirrel waiting for each pigeon.
So always write non-blocking i/o code.
If you can encourage your Pigeons to promise to come back ;)
Promises and generators are probably the best approach you can take to this.
HOWEVER you can always use Node cluster to establish a master squirrel that will procreate child squirrels based on the number of CPUs the master squirrel can find to dole out the work.
Hope this helps and note the complete lack of a car analogy.
Why does being thread safe matter in a web app? Pylons (Python web framework) uses a global application variable which is not thread safe. Does this matter? Is it only a problem if I intend on using multi-threading? Or, does it mean that one user might not have updated state if another user... I'm just confusing myself. What's so important with this?
Threading errors can lead to serious and subtle problems.
Say your system has 10 members. One more user signs up to your system and the application adds him to the roster and increments the count of members; "simultaneously", another user quits and the application removes him from the roster and decrements the count of members.
If you don't handling threading properly, your member count (which should be 10) could easily be nine, 10, or 11, and you'll never be able to reproduce the bug.
So be careful.
You should care about thread safety. E.g in java you write a servlet that provides some functionality. The container will deploy an instance of your servlet, and as HTTP requests arrive from clients, over different TCP connections, each request is handled by a separate thread which in turn will call your servlet. As a result, you will have your servlet being call from multiple threads. So if it is not thread-safe, then erroneous result will be returned to the user, due to data corruption of access to shared data by threads.
It really depends on the application framework (which I know nothing about in this case) and how the web server handles it. Obviously, any good webserver is going to be responding to multiple requests simultaneously, so it will be operating with multiple threads. That web server may dispatch to a single instance of your application code for all of these requests, or it may spawn multiple instances of your web application and never use a given instance concurrently.
Even if the app server does use separate instances, your application will probably have some shared state--say, a database with a list of users. In that case, you need to make sure that state can be accessed safely from multiple threads/instances of your web app.
Then, of course, there is the case where you use threading explicitly in your application. In that case, the answer is obvious.
Your Web Application is almost always multithreading. Even though you might not use threads explicitly. So, to answer your questions: it's very important.
How can this happen? Usually, Apache (or IIS) will serve several request simultaneously, calling multiple times from multiple threads your python programs. So you need to consider that your programs run in multiple threads concurrently and act accordingly.
(This was too long to add a comment to the other fine answers.)
Concurrency problems (read: multiple access to shared state) is a super-set of threading problems. The (concurrency problems) can easily exist at an "above thread" level such as a process/server level (the global variable in the case you mention above is process-unique value, which in turn can lead to an inconsistent view/state if there are multiple processes).
Care must be taken to analyze the data consistency requirements and then implement the software to fulfill those requirements. I would always err on the side of safe, and only degrade in carefully analyzed areas where it is acceptable.
However, note that CPython runs only one thread context for Python code execution (to get true concurrent threads you need to write/use C extensions), so, while you can get a form of race condition upon expected data, you won't get (all) the same kind of partial-write scenarios and such that may plague C/C++ programs. But, once again. Err on the side of a consistent view.
There are a number of various existing methods of making access to a global atomic -- across threads or processes. Use them.