I'm now a couple of weeks into my node deep dive. I've learned a lot from Anthony's excellent course on udemy and I'm currently going through a book " nodejs the right way". I've also gone through quite a few articles that brought up some very good points about real world scenarios with node and coupling other technologies out there.
HOWEVER, it seems to be accepted as law, that you don't perform computationally heavy tasks with Node as its a single thread architecture. I get the idea of the event loop and asynch callbacks etc. In fact nodes strength stems from tons of concurrent IO connections if I understand correctly. No matter where I'm reading though, the source warns against hanging up that thread executing a task. I can't seem to find any rule of thumb of things to avoid using nodes process for. I've seen a solution saying that node should pass computationally heavy tasks to a message service like RabbitMQ which a dedicated app server can churn through(any suggestions on what pairs well with node for this task? I read something about an N-tier architecture). The reason I'm so confused is because I see node being used for reading and writing files to highlight the usage of streams but in my mind fetching/reading/writing files is an expensive task (I feel mistaken).
Tl;Dr What kind of tasks should node pass off to a work horse server ? What material can I read that explains the paradigm in detail?
Edit: it seems like my lack of understanding stemmed from not knowing what would even halt a thread in the first place outside of an obviously synchronous IO request . So if I understand correctly reading and writing data is IO where mutating said data or doing mathematical computations is computationally expensive (at varying levels depending on the task of course ) . Thanks for all the answers!
If you're using node.js as a server, then running a long running synchronous computational task ties up the one thread and during that computation, your server is non-responsive to other requests. That's generally a bad situation for servers.
So, the general design principles for node.js server design is this:
Use only asynchronous I/O functions. For example, use fs.readFile(), not fs.readyFileSync().
If you have a computationally intense operation, then move it to a child process. If you do a lot of these, then have several child processes that can process these long running operations. This keeps your main thread free so it can be responsive to I/O requests.
If you want to increase the overall scalability and responsiveness of your server, you can implement clustering with a server process per CPU. This isn't really a substitute for #2 above, but can also improve scalability and responsiveness.
The reason I'm so confused is because I see node being used for
reading and writing files to highlight the usage of streams but in my
mind fetching/reading/writing files is an expensive task (I feel
mistaken).
If you use the asynchronous versions of the I/O functions, then read/writing from the disk does not block the main JS thread as they present an asynchronous interface and the main thread can do other things while the system is fetching data from the disk.
What kind of tasks should node pass off to a work horse server ?
It depends a bit on the server load that you are trying to support, what you're asking it to do and your tolerance for responsiveness delays. The higher the load you're aiming for, then the more you need to get any computationally intensive task off the main JS thread and into some other process. At a medium number of long running transactions and a modest server load, you may just be able to use clustering to reach your scalability and responsiveness goal, but at some threshold of either length of the transaction or the load you're trying to support, you have to get the computationally intensive stuff out of the main JS thread.
HOWEVER, it seems to be accepted as law, that you don't perform computationally heavy tasks with Node as its a single thread architecture.
I would reword this:
don't perform computationally heavy tasks unless you need to with Node
Sometimes, you need to crunch through a bunch of data. There are times when it's faster or better to do that in-process than it is to pass it around.
A practical example:
I have a Node.js server that reads in raw log data from a bunch of servers. No standard logging utilities could be used as I have some custom processing being done, as well as custom authentication schemes for getting the log data. The whole thing is HTTP requests, and then parsing and re-writing the data.
As you can imagine, this uses a ton of CPU. Here's the thing though... is that CPU wasted? Am I doing anything in JS that I could do faster had I written it in another language? Often times the CPU is busy for a real reason, and the benefit of switching to something more native might be marginal. And then, you have to factor in the overhead of switching.
Remember that with Node.js, you can compile native extensions, so it's possible to have the best of both worlds in a well established framework.
For me, the human trade-offs came in. I'm a far more efficient Node.js developer than anything that runs natively. Even if my Node.js app were prove to be 5x slower than something native (which I'd imagine would be on the extreme), I could just buy 5 more servers to run, at much less cost than it would take for me to develop and maintain the native solution.
Use what you need. If you need to burn a lot of CPU in Node.js, just make sure you're doing it as efficiently as you can be. If you find that you could optimize something with native code, consider making an extension and besure to measure the performance differences afterwards. If you feel the desire to throw out the whole stack... reconsider your approach, as there might be something you're not considering.
Reading and writing files are I/O operations, so they are NOT CPU intensive. You can do a fair amount of concurrent I/O with Node without tying up any single request (in a Node HTTP server for example).
But people use Node in general for CPU-intensive tasks all the time and its fine. You just have to realize that if it uses all of the CPU for any significant amount of time then you will block all other requests to that server, which generally won't be acceptable if you need the server to stay available. But there are lots of times when your Node process is not trying to be a server firing back responses to many requests, such as when you have a Node program that just processes data and isn't a server at all.
Also, using another process is not the only way to do background tasks in Node. There is also webworker-threads which allows you to use threads if that is more convenient (you do have to copy the data in and out).
I would stop reading and do some targeted experiments so you can see what they are talking about. Try to create and test three different programs: 1) HTTP server handles lots of requests, always returns immediately with file contents 2) HTTP server handles lots of requests, but a certain request causes a large math computation that takes many seconds to return (which will block all the other requests -- big problem) 3) a Node program that is not an HTTP server, which does that large math computation and spits out the result in the terminal (which even though it takes awhile to work, is not handling other requests since its not a server, so its fine for it to block).
Related
I have read about node.js and other servers such as Apache, where the threading is different. I simply do not understand what the threading means.
If I have a webpage that runs SQL to hit a database, say three different databases in the one server side page, what does that mean for threading in node.js? Apache? What does "thread" mean here?
Or as an article I saw, "start a new thread to handle each request."
What does it mean to say Apache spawns a thread per request, but node.js does not?
EDIT: I am hoping for an example that I can grasp. I'm used to having a server side page that hits a database(s). Several connections inside that file.
A thread is a context of program execution. Programs that are single-threaded can only do one thing at once, where multi-threaded programs can do many things at once.
Think of it like a kitchen at a restaurant. A single chef can really only do one task at a time, be that chopping onions or putting something in an oven. If an order comes in that requires lots of work from the chef (such as making salads vs. putting stuff in the oven and waiting) some meals may get delayed because that chef is busy. On the other hand, if that chef just has to bake a bunch of stuff, there isn't much work for him to do and he can make other meals while waiting for the food in the oven to be done.
With multiple chefs, many of these tasks can be done simultaneously. Many meals can be prepared simultaneously.
Apache's threading model is like hiring a fixed number of chefs (regardless of how many customers your restauarant has that night) and each chef can only work on one meal at a time. That means that if a meal order comes in, a dedicated chef is assigned to that meal. There will be times when that chef is busy chopping up ingredients and mixing cake batter, but there will also be times when he's just standing around waiting for the potatoes to boil. At any given time, you could have most of your chefs sitting idle, waiting on potatoes to boil and cake to bake and no more orders will be worked on, since each chef is dedicated to one order at a time.
To make matters worse, your kitchen is only as big as you can afford to make it. Each chef takes up space and resources, and you may have a situation where a bunch of chefs standing around holding the only spoons available are preventing other chefs from getting their food made.
Nginx is another web server (often used as a proxy) that you didn't ask about, but I'm including it to explain another threading model. It also hires a fixed number of chefs, but it hires fewer of them. Each chef can work on multiple meals at a time. So, if they're waiting on potatoes to boil while an order comes in for a chopped salad, they can go work on that salad instead of standing around idle. You can have a smaller kitchen (relative to the size of restaurant/number of customers) and get the same number of meals out, or more. It's a tight crew that is effective at not wasting time and resources.
Node.js is a bit different. It is single-threaded from a JavaScript perspective, but other tasks like disk and network IO are handled on separate threads automatically. It's like having a kitchen with only one chef, but that makes sense in some cases. If your kitchen has a lot of busy work for that chef, perhaps it makes sense to hire more chefs to do work. (To do this in Node.js, you can only spawn more processes, which is effectively like building a bunch of small kitchens right next to each other. You can have one guy standing out front coordinating the orders for all those kitchens.) However, if you're just a bakery (mainly just IO, with little busy-work for the chef), maybe you only need one chef.
To sum all this up, different threading models are used to divide work and process it effectively. Which threading model makes sense depends on your needs, and the other characteristics of the server you are choosing.
Node.js is single threaded in that it can only do one thing at once. You can run multiple instances of the node process on pretty much all cloud service providers, though. The apache process can multi-task on threads.
If the node process hangs for some reason, nothing else can happen. That's why its important to write node in an asyncronous way so that if a database query hangs, node can still take requests.
Without getting too technical, a thread can be thought of as a lane in the highway of the program. Its a specific channel of execution. In the lifetime of a request, a lot of things have to happen. All of those things are in one box.
Node doesn't have threads! You can think of it like a one lane road. But the way node is deployed you get many instances of that one lane road. They don't share anything though. If you a value gets added to an array in one, its not in the other. Anything that needs to be shared has to be shared in a cache or database layer.
What people confuse between is Threads, Process & Async, Non-blocking I/O.
Threads are child level 'runnable' to a process. All the execution environment is set up for a thread. Right from the Stack to Addressable memory locations it's allocated to a thread. If a child-level thread has to communicate back to the the main process thread, it has to use safe-messaging,notification models. There are multiple ways to do this, based on the language.
Node.js is Single Threaded and obviously single Process based. It's not meant for high CPU intensive blocking calls. But if you still want to use, You could consider Node clustering. So instead of creating threads, it creates multiple "process" that works like a thread.
Async - All the code that carries a callback functions are not actually Async.
Okay in other words, Literally, they are Asynchrounous as they don't block the call.
But in Node.js context, When someone says, Node is Async, it's completely linked to the OS interfacing. The capability of Node depends on the Non-blocking I/O capabilities of the underlying OS. So whatever objects the OS supports Non-blocking I/O for example, Sockets, Files, Pipes, Node utilizes them to maximum.
And btw, when you talk about Apache, you should ideally be comparing Nginx. Not Node.js.
Node.js is not meant to serve as a Web Server. It's a basically a Process that puts effective use of Async I/O.
node.js scalable, what is meant by the that? What part of a node.js server is scalable. I read that it is a single-threaded technology that is not suitable for applications that would need a lot of CPU resources. These facts do not fit with scalability, so what is meant by that?
The javascript that node runs is single threaded, but a lot of the things you call in node - such as network or file io - run in background threads. See this post for a basic overview:
Node is not single threaded
If you need the gritty details, you should look into libuv which is the 'magic' piece converting threads into event loops:
http://nikhilm.github.io/uvbook/basics.html#event-loops
Additionally, if you need to do something CPU intensive in node itself, you can easily send this to a child process - see http://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options for details
Is not that is more scalable "per se".
Is more that all you do is (I/O) scalable without having to do anything special.
I/O is safer and easier to do in parallel, as it tends to share no data between execution threads. Node.js lets you do it using event programming which is simple, elegant and easy to use. Is an old and proved programming paradigm, used for years by GUIs and other graphical intensive apps like games for example.
In fact is less scalable than full fledged languages like c++, c, java, etc.. which can scale out much better by the usage of full multithreading. This allows to scale CPU too, but also opens a can of worms.
To share CPU, you have to share data, and that is another story (semaphores, locks, etc...)
You can do the same as node.js with any of the above languages, but is not part of the language itself, so you have to roll your own or use libraries that provide it to you.
That said, is not that hard, but sure harder than in node.js.
Most web services, are IO bound, so Node.js fits well, and is Ok for most cases. But once you start to use CPU intensive work, the events are not serviced and all stops.
In that case better use another language. There's no really a good solution in Node for that. You can spawn multiple processes, but then you will not be able to share data between them. Without data sharing, there's no way to scale CPU efficiently, so better don't try.
Use Node.js for IO, and some better suited language with proper multithreading for CPU intensive work.
it is scalable due to load balancing. Essentially you can have multiple jobs for node to process and it can handle it with no significant burden. This makes it scalable.
All APIs of Node are written is such a way that they supports callbacks.
For example, a function to read a file may start reading file and return the control to execution environment immidiately so that next instruction can be executed. Once file I/O is complete, it will call the callback function while passing the callback function, the content of the file as parameter. So there is no blocking or wait for File I/O. This makes Node.js highly scalable, as it can process high number of request without waiting for any function to return result. -- Tutorial's Point.
I stumbled over node.js sometime ago and like it a lot. But soon I found out that it lacked badly the ability to perform CPU-intensive tasks. So, I started googling and got these answers to solve the problem: Fibers, Webworkers and Threads (thread-a-gogo). Now which one to use is a confusion and one of them definitely needs to be used - afterall what's the purpose of having a server which is just good at IO and nothing else? Suggestions needed!
UPDATE:
I was thinking of a way off-late; just needing suggestions over it. Now, what I thought of was this: Let's have some threads (using thread_a_gogo or maybe webworkers). Now, when we need more of them, we can create more. But there will be some limit over the creation process. (not implied by the system but probably because of overhead). Now, when we exceed the limit, we can fork a new node, and start creating threads over it. This way, it can go on till we reach some limit (after all, processes too have a big overhead). When this limit is reached, we start queuing tasks. Whenever a thread becomes free, it will be assigned a new task. This way, it can go on smoothly.
So, that was what I thought of. Is this idea good? I am a bit new to all this process and threads stuff, so don't have any expertise in it. Please share your opinions.
Thanks. :)
Node has a completely different paradigm and once it is correctly captured, it is easier to see this different way of solving problems. You never need multiple threads in a Node application(1) because you have a different way of doing the same thing. You create multiple processes; but it is very very different than, for example how Apache Web Server's Prefork mpm does.
For now, let's think that we have just one CPU core and we will develop an application (in Node's way) to do some work. Our job is to process a big file running over its contents byte-by-byte. The best way for our software is to start the work from the beginning of the file, follow it byte-by-byte to the end.
-- Hey, Hasan, I suppose you are either a newbie or very old school from my Grandfather's time!!! Why don't you create some threads and make it much faster?
-- Oh, we have only one CPU core.
-- So what? Create some threads man, make it faster!
-- It does not work like that. If I create threads I will be making it slower. Because I will be adding a lot of overhead to the system for switching between threads, trying to give them a just amount of time, and inside my process, trying to communicate between these threads. In addition to all these facts, I will also have to think about how I will divide a single job into multiple pieces that can be done in parallel.
-- Okay okay, I see you are poor. Let's use my computer, it has 32 cores!
-- Wow, you are awesome my dear friend, thank you very much. I appreciate it!
Then we turn back to work. Now we have 32 cpu cores thanks to our rich friend. Rules we have to abide have just changed. Now we want to utilize all this wealth we are given.
To use multiple cores, we need to find a way to divide our work into pieces that we can handle in parallel. If it was not Node, we would use threads for this; 32 threads, one for each cpu core. However, since we have Node, we will create 32 Node processes.
Threads can be a good alternative to Node processes, maybe even a better way; but only in a specific kind of job where the work is already defined and we have complete control over how to handle it. Other than this, for every other kind of problem where the job comes from outside in a way we do not have control over and we want to answer as quickly as possible, Node's way is unarguably superior.
-- Hey, Hasan, are you still working single-threaded? What is wrong with you, man? I have just provided you what you wanted. You have no excuses anymore. Create threads, make it run faster.
-- I have divided the work into pieces and every process will work on one of these pieces in parallel.
-- Why don't you create threads?
-- Sorry, I don't think it is usable. You can take your computer if you want?
-- No okay, I am cool, I just don't understand why you don't use threads?
-- Thank you for the computer. :) I already divided the work into pieces and I create processes to work on these pieces in parallel. All the CPU cores will be fully utilized. I could do this with threads instead of processes; but Node has this way and my boss Parth Thakkar wants me to use Node.
-- Okay, let me know if you need another computer. :p
If I create 33 processes, instead of 32, the operating system's scheduler will be pausing a thread, start the other one, pause it after some cycles, start the other one again... This is unnecessary overhead. I do not want it. In fact, on a system with 32 cores, I wouldn't even want to create exactly 32 processes, 31 can be nicer. Because it is not just my application that will work on this system. Leaving a little room for other things can be good, especially if we have 32 rooms.
I believe we are on the same page now about fully utilizing processors for CPU-intensive tasks.
-- Hmm, Hasan, I am sorry for mocking you a little. I believe I understand you better now. But there is still something I need an explanation for: What is all the buzz about running hundreds of threads? I read everywhere that threads are much faster to create and dumb than forking processes? You fork processes instead of threads and you think it is the highest you would get with Node. Then is Node not appropriate for this kind of work?
-- No worries, I am cool, too. Everybody says these things so I think I am used to hearing them.
-- So? Node is not good for this?
-- Node is perfectly good for this even though threads can be good too. As for thread/process creation overhead; on things that you repeat a lot, every millisecond counts. However, I create only 32 processes and it will take a tiny amount of time. It will happen only once. It will not make any difference.
-- When do I want to create thousands of threads, then?
-- You never want to create thousands of threads. However, on a system that is doing work that comes from outside, like a web server processing HTTP requests; if you are using a thread for each request, you will be creating a lot of threads, many of them.
-- Node is different, though? Right?
-- Yes, exactly. This is where Node really shines. Like a thread is much lighter than a process, a function call is much lighter than a thread. Node calls functions, instead of creating threads. In the example of a web server, every incoming request causes a function call.
-- Hmm, interesting; but you can only run one function at the same time if you are not using multiple threads. How can this work when a lot of requests arrive at the web server at the same time?
-- You are perfectly right about how functions run, one at a time, never two in parallel. I mean in a single process, only one scope of code is running at a time. The OS Scheduler does not come and pause this function and switch to another one, unless it pauses the process to give time to another process, not another thread in our process. (2)
-- Then how can a process handle 2 requests at a time?
-- A process can handle tens of thousands of requests at a time as long as our system has enough resources (RAM, Network, etc.). How those functions run is THE KEY DIFFERENCE.
-- Hmm, should I be excited now?
-- Maybe :) Node runs a loop over a queue. In this queue are our jobs, i.e, the calls we started to process incoming requests. The most important point here is the way we design our functions to run. Instead of starting to process a request and making the caller wait until we finish the job, we quickly end our function after doing an acceptable amount of work. When we come to a point where we need to wait for another component to do some work and return us a value, instead of waiting for that, we simply finish our function adding the rest of work to the queue.
-- It sounds too complex?
-- No no, I might sound complex; but the system itself is very simple and it makes perfect sense.
Now I want to stop citing the dialogue between these two developers and finish my answer after a last quick example of how these functions work.
In this way, we are doing what OS Scheduler would normally do. We pause our work at some point and let other function calls (like other threads in a multi-threaded environment) run until we get our turn again. This is much better than leaving the work to OS Scheduler which tries to give just time to every thread on system. We know what we are doing much better than OS Scheduler does and we are expected to stop when we should stop.
Below is a simple example where we open a file and read it to do some work on the data.
Synchronous Way:
Open File
Repeat This:
Read Some
Do the work
Asynchronous Way:
Open File and Do this when it is ready: // Our function returns
Repeat this:
Read Some and when it is ready: // Returns again
Do some work
As you see, our function asks the system to open a file and does not wait for it to be opened. It finishes itself by providing next steps after file is ready. When we return, Node runs other function calls on the queue. After running over all the functions, the event loop moves to next turn...
In summary, Node has a completely different paradigm than multi-threaded development; but this does not mean that it lacks things. For a synchronous job (where we can decide the order and way of processing), it works as well as multi-threaded parallelism. For a job that comes from outside like requests to a server, it simply is superior.
(1) Unless you are building libraries in other languages like C/C++ in which case you still do not create threads for dividing jobs. For this kind of work you have two threads one of which will continue communication with Node while the other does the real work.
(2) In fact, every Node process has multiple threads for the same reasons I mentioned in the first footnote. However this is no way like 1000 threads doing similar works. Those extra threads are for things like to accept IO events and to handle inter-process messaging.
UPDATE (As reply to a good question in comments)
#Mark, thank you for the constructive criticism. In Node's paradigm, you should never have functions that takes too long to process unless all other calls in the queue are designed to be run one after another. In case of computationally expensive tasks, if we look at the picture in complete, we see that this is not a question of "Should we use threads or processes?" but a question of "How can we divide these tasks in a well balanced manner into sub-tasks that we can run them in parallel employing multiple CPU cores on the system?" Let's say we will process 400 video files on a system with 8 cores. If we want to process one file at a time, then we need a system that will process different parts of the same file in which case, maybe, a multi-threaded single-process system will be easier to build and even more efficient. We can still use Node for this by running multiple processes and passing messages between them when state-sharing/communication is necessary. As I said before, a multi-process approach with Node is as well as a multi-threaded approach in this kind of tasks; but not more than that. Again, as I told before, the situation that Node shines is when we have these tasks coming as input to system from multiple sources since keeping many connections concurrently is much lighter in Node compared to a thread-per-connection or process-per-connection system.
As for setTimeout(...,0) calls; sometimes giving a break during a time consuming task to allow calls in the queue have their share of processing can be required. Dividing tasks in different ways can save you from these; but still, this is not really a hack, it is just the way event queues work. Also, using process.nextTick for this aim is much better since when you use setTimeout, calculation and checks of the time passed will be necessary while process.nextTick is simply what we really want: "Hey task, go back to end of the queue, you have used your share!"
(Update 2016: Web workers are going into io.js - a Node.js fork Node.js v7 - see below.)
(Update 2017: Web workers are not going into Node.js v7 or v8 - see below.)
(Update 2018: Web workers are going into Node.js Node v10.5.0 - see below.)
Some clarification
Having read the answers above I would like to point out that there is nothing in web workers that is against the philosophy of JavaScript in general and Node in particular regarding concurrency. (If there was, it wouldn't be even discussed by the WHATWG, much less implemented in the browsers).
You can think of a web worker as a lightweight microservice that is accessed asynchronously. No state is shared. No locking problems exist. There is no blocking. There is no synchronization needed. Just like when you use a RESTful service from your Node program you don't worry that it is now "multithreaded" because the RESTful service is not in the same thread as your own event loop. It's just a separate service that you access asynchronously and that is what matters.
The same is with web workers. It's just an API to communicate with code that runs in a completely separate context and whether it is in different thread, different process, different cgroup, zone, container or different machine is completely irrelevant, because of a strictly asynchronous, non-blocking API, with all data passed by value.
As a matter of fact web workers are conceptually a perfect fit for Node which - as many people are not aware of - incidentally uses threads quite heavily, and in fact "everything runs in parallel except your code" - see:
Understanding the node.js event loop by Mikito Takada
Understanding node.js by Felix Geisendörfer
Understanding the Node.js Event Loop by Trevor Norris
Node.js itself is blocking, only its I/O is non-blocking by Jeremy Epstein
But the web workers don't even need to be implemented using threads. You could use processes, green threads, or even RESTful services in the cloud - as long as the web worker API is used. The whole beauty of the message passing API with call by value semantics is that the underlying implementation is pretty much irrelevant, as the details of the concurrency model will not get exposed.
A single-threaded event loop is perfect for I/O-bound operations. It doesn't work that well for CPU-bound operations, especially long running ones. For that we need to spawn more processes or use threads. Managing child processes and the inter-process communication in a portable way can be quite difficult and it is often seen as an overkill for simple tasks, while using threads means dealing with locks and synchronization issues that are very difficult to do right.
What is often recommended is to divide long-running CPU-bound operations into smaller tasks (something like the example in the "Original answer" section of my answer to Speed up setInterval) but it is not always practical and it doesn't use more than one CPU core.
I'm writing it to clarify the comments that were basically saying that web workers were created for browsers, not servers (forgetting that it can be said about pretty much everything in JavaScript).
Node modules
There are few modules that are supposed to add Web Workers to Node:
https://github.com/pgriess/node-webworker
https://github.com/audreyt/node-webworker-threads
I haven't used any of them but I have two quick observations that may be relevant: as of March 2015, node-webworker was last updated 4 years ago and node-webworker-threads was last updated a month ago. Also I see in the example of node-webworker-threads usage that you can use a function instead of a file name as an argument to the Worker constructor which seems that may cause subtle problems if it is implemented using threads that share memory (unless the functions is used only for its .toString() method and is otherwise compiled in a different environment, in which case it may be fine - I have to look more deeply into it, just sharing my observations here).
If there is any other relevant project that implements web workers API in Node, please leave a comment.
Update 1
I didn't know it yet at the time of writing but incidentally one day before I wrote this answer Web Workers were added to io.js.
(io.js is a fork of Node.js - see: Why io.js decided to fork Node.js, an InfoWorld interview with Mikeal Rogers, for more info.)
Not only does it prove the point that there is nothing in web workers that is against the philosophy of JavaScript in general and Node in particular regarding concurrency, but it may result in web workers being a first class citizen in server-side JavaScript like io.js (and possibly Node.js in the future) just as it already is in client-side JavaScript in all modern browsers.
Update 2
In Update 1 and my tweet I was referring to io.js pull request #1159
which now redirects to
Node PR #1159
that was closed on Jul 8 and replaced with Node PR #2133 - which is still open.
There is some discussion taking place under those pull requests that may provide some more up to date info on the status of Web workers in io.js/Node.js.
Update 3
Latest info - thanks to NiCk Newman for posting it in
the comments: There is the workers: initial implementation commit by Petka Antonov from Sep 6, 2015
that can be downloaded and tried out in
this tree. See comments by NiCk Newman for details.
Update 4
As of May 2016 the last comments on the still open PR #2133 - workers: initial implementation were 3 months old. On May 30 Matheus Moreira asked me to post an update to this answer in the comments below and he asked for the current status of this feature in the PR comments.
The first answers in the PR discussion were skeptical but later
Ben Noordhuis wrote that "Getting this merged in one shape or another is on my todo list for v7".
All other comments seemed to second that and as of July 2016 it seems that Web Workers should be available in the next version of Node, version 7.0 that is planned to be released on October 2016 (not necessarily in the form of this exact PR).
Thanks to Matheus Moreira for pointing it out in the comments and reviving the discussion on GitHub.
Update 5
As of July 2016 there are few modules on npm that were not available before - for a complete list of relevant modules, search npm for workers, web workers, etc. If anything in particular does or doesn't work for you, please post a comment.
Update 6
As of January 2017 it is unlikely that web workers will get merged into Node.js.
The pull request #2133 workers: initial implementation by Petka Antonov from July 8, 2015 was finally closed by Ben Noordhuis on December 11, 2016 who commented that "multi-threading support adds too many new failure modes for not enough benefit" and "we can also accomplish that using more traditional means like shared memory and more efficient serialization."
For more information see the comments to the PR 2133 on GitHub.
Thanks again to Matheus Moreira for pointing it out in the comments.
Update 6
I'm happy to announce that few days ago, in June 2018 web workers appeared in Node v10.5.0 as an experimental feature activated with the --experimental-worker flag.
For more info, see:
Node v10.5.0 release blog post
Pull Request #20876 - worker: initial implementation by Anna Henningsen
My original tweet of happiness when I learned that this got into v10.5.0:
🎉🎉🎉 Finally! I can make the 7th update to my 3 year old Stack Overflow answer where I argue that threading a la web workers is not against Node philosophy, only this time saying that we finally got it! 😜👍
I come from the old school of thought where we used multi-threading to make software fast. For past 3 years i have been using Node.js and a big supporter of it. As hasanyasin explained in detail how node works and the concept of asyncrous functionality. But let me add few things here.
Back in the old days with single cores and lower clock speeds we tried various ways to make software work fast and parallel. in DOS days we use to run one program at a time. Than in windows we started running multiple applications (processes) together. Concepts like preemptive and non-preemptive (or cooperative) where tested. we know now that preemptive was the answer for better multi-processing task on single core computers. Along came the concepts of processes/tasks and context switching. Than the concept of thread to further reduce the burden of process context switching. Thread where coined as light weight alternative to spawning new processes.
So like it or not signal thread or not multi-core or single core your processes will be preempted and time sliced by the OS.
Nodejs is a single process and provides async mechanism. Here jobs are dispatched to under lying OS to perform tasks while we waiting in an event loop for the task to finish. Once we get a green signal from OS we perform what ever we need to do. Now in a way this is cooperative/non-preemptive multi-tasking, so we should never block the event loop for a very long period of time other wise we will degrade our application very fast.
So if there is ever a task that is blocking in nature or is very time consuming we will have to branch it out to the preemptive world of OS and threads.
there are good examples of this is in the libuv documentation. Also if you read the documentation further you find that FileI/O is handled in threads in node.js.
So Firstly its all in the design of our software. Secondly Context switching is always happening no matter what they tell you. Thread are there and still there for a reason, the reason is they are faster to switch in between then processes.
Under hood in node.js its all c++ and threads. And node provides c++ way to extend its functionality and to further speed out by using threads where they are a must i.e., blocking tasks such as reading from a source writing to a source, large data analysis so on so forth.
I know hasanyasin answer is the accepted one but for me threads will exist no matter what you say or how you hide them behind scripts, secondly no one just breaks things in to threads just for speed it is mostly done for blocking tasks. And threads are in the back bone of Node.js so before completely bashing multi-threading is in correct. Also threads are different from processes and the limitation of having node processes per core don't exactly apply to number of threads, threads are like sub tasks to a process. in fact threads won;t show up in your windows task manager or linux top command. once again they are more little weight then processes
I'm not sure if webworkers are relevant in this case, they are client-side tech (run in the browser), while node.js runs on the server. Fibers, as far as I understand, are also blocking, i.e. they are voluntary multitasking, so you could use them, but should manage context switches yourself via yield. Threads might be actually what you need, but I don't know how mature they are in node.js.
worker_threads has been implemented and shipped behind a flag in node#10.5.0. It's still an initial implementation and more efforts are needed to make it more efficient in future releases. Worth giving it a try in latest node.
In many Node developers' opinions one of the best parts of Node is actually its single-threaded nature. Threads introduce a whole slew of difficulties with shared resources that Node completely avoids by doing nothing but non-blocking IO.
That's not to say that Node is limited to a single thread. It's just that the method for getting threaded concurrency is different from what you're looking for. The standard way to deal with threads is with the cluster module that comes standard with Node itself. It's a simpler approach to threads than manually dealing with them in your code.
For dealing with asynchronous programming in your code (as in, avoiding nested callback pyramids), the [Future] component in the Fibers library is a decent choice. I would also suggest you check out Asyncblock which is based on Fibers. Fibers are nice because they allow you to hide callback by duplicating the stack and then jumping between stacks on a single-thread as they're needed. Saves you the hassle of real threads while giving you the benefits. The downside is that stack traces can get a bit weird when using Fibers, but they aren't too bad.
If you don't need to worry about async stuff and are more just interested in doing a lot of processing without blocking, a simple call to process.nextTick(callback) every once in a while is all you need.
Maybe some more information on what tasks you are performing would help. Why would you need to (as you mentioned in your comment to genericdave's answer) need to create many thousands of them? The usual way of doing this sort of thing in Node is to start up a worker process (using fork or some other method) which always runs and can be communicated to using messages. In other words, don't start up a new worker each time you need to perform whatever task it is you're doing, but simply send a message to the already running worker and get a response when it's done. Honestly, I can't see that starting up many thousands of actual threads would be very efficient either, you are still limited by you CPUs.
Now, after saying all of that, I have been doing a lot of work with Hook.io lately which seems to work very well for this sort of off-loading tasks into other processes, maybe it can accomplish what you need.
Let's suppose we have an instant messaging application, client-server based, not p2p. The actual protocol doesn't matter, what matters is the server architecture. The said server can be coded to operate in single-threaded, non-parallel mode using non-blocking sockets, which by definition allow us to perform operations like read-write effectively immediately (or instantly). This very feature of non-blocking sockets allows us to use some sort of select/poll function at the very core of the server and waste next to no time in the actual socket read/write operations, but rather to spend time processing all this information. Properly coded, this can be very fast, as far as I understand. But there is the second approach, and that is to multithread aggressively, creating a new thread (obviously using some sort of thread pool, because that very operation can be (very) slow on some platforms and under some circumstances), and letting those threads to work in parallel, while the main background thread handles accept() and stuff. I've seen this approach explained in various places over the Net, so it obviously does exist.
Now the question is, if we have non-blocking sockets, and immediate read/write operations, and a simple, easily coded design, why does the second variant even exist? What problems are we trying to overcome with the second design, i.e. threads? AFAIK those are usually used to work around some slow and possibly blocking operations, but no such operations seem to be present there!
I'm assuming you're not talking about having a thread per client as such a design is usually for completely diffreent reasons, but rather a pool of threads each handles several concurrent clients.
The reason for that arcitecture vs a single threaded server is simply to take advantage of multiple processors. You're doing more work than simply I/O. You have to parse the messages, do various work, maybe even run some more heavyweight crypto algorithms. All this takes CPU. If you want to scale, taking advantage of multiple processors will allow you to scale even more, and/or keep the latency even lower per client.
Some of the gain in such a design can be a bit offset by the fact you might need more locking in a multithreaded environment, but done right, and certainly depening on what you're doing, it can be a huge win - at the expense of more complexity.
Also, this might help overcome OS limitations . The I/O paths in the kernel might get more distributed among the procesors. Not all operating systems might fully be able to thread the IO from a single threaded applications. Back in the old days there were'nt all the great alternatives to the old *nix select(), which usually had a filedesciptor limit of 1024, and similar APIs severly started degrading once you told it to monitor too many socket. Spreading all those clients on multiple threads or processes helped overcome that limit.
As for a 1:1 mapping between threads, there's several reasons to implement that architecture:
Easier programming model, which might lead to less hard to find bugs, and faster to implement.
Support blocking APIs. These are all over the place. Having a thread handle many/all of the clients and then go on to do a blocking call to a database is going to stall everyone. Even reading files can block your application, and you usually can't monitor regular file handles/descriptors for IO events - or when you can, the programming model is often exceptionally complicated.
The drawback here is it won't scale, atleast not with the most widely used languages/framework. Having thousands of native threads will hurt performance. Though some languages provides a much more lightweight approach here, such as Erlang and to some extent Go.
How do you make your application multithreaded ?
Do you use asynch functions ?
or do you spawn a new thread ?
I think that asynch functions are already spawning a thread so if your job is doing just some file reading, being lazy and just spawning your job on a thread would just "waste" ressources...
So is there some kind of design when using thread or asynch functions ?
If you are talking about .Net, then don't forget the ThreadPool. The thread pool is also what asynch functions often use. Spawning to much threads can actually hurt your performance. A thread pool is designed to spawn just enough threads to do the work the fastest. So do use a thread pool instead of spwaning your own threads, unless the thread pool doesn't meet your needs.
PS: And keep an eye out on the Parallel Extensions from Microsoft
Spawning threads is only going to waste resources if you start spawning tons of them, one or two extra threads isn't going to effect the platforms proformance, infact System currently has over 70 threads for me, and msn is using 32 (I really have no idea how a messenger can use that many threads, exspecialy when its minimised and not really doing anything...)
Useualy a good time to spawn a thread is when something will take a long time, but you need to keep doing something else.
eg say a calculation will take 30 seconds. The best thing to do is spawn a new thread for the calculation, so that you can continue to update the screen, and handle any user input because users will hate it if your app freezes untill its finished doing the calculation.
On the other hand, creating threads to do something that can be done almost instantly is nearly pointless, since the overhead of creating (or even just passing work to an existing thread using a thread pool) will be higher than just doing the job in the first place.
Sometimes you can break your app into a couple of seprate parts which run in their own threads. For example in games the updates/physics etc may be one thread, while grahpics are another, sound/music is a third, and networking is another. The problem here is you really have to think about how these parts will interact or else you may have worse proformance, bugs that happen seemingly "randomly", or it may even deadlock.
I'll second Fire Lancer's answer - creating your own threads is an excellent way to process big tasks or to handle a task that would otherwise be "blocking" to the rest of synchronous app, but you have to have a clear understanding of the problem that you must solve and develope in a way that clearly defines the task of a thread, and limits the scope of what it does.
For an example I recently worked on - a Java console app runs periodically to capture data by essentially screen-scraping urls, parsing the document with DOM, extracting data and storing it in a database.
As a single threaded application, it, as you would expect, took an age, averaging around 1 url a second for a 50kb page. Not too bad, but when you scale out to needing to processes thousands of urls in a batch, it's no good.
Profiling the app showed that most of the time the active thread was idle - it was waiting for I/O operations - opening of a socket to the remote URL, opening a connection to the database etc. It's this sort of situation that can easily be improved with multithreading. Rewriting to be multi-threaded and with just 5 threads instead of one, even on a single core cpu, gave an increase in throughput of over 20 times.
In this example, each "worker" thread was explicitly limited to what it did - open the remote a remote url, parse the data, store it in the db. All the "high level" processing - generating the list of urls to parse, working out which next, handling errors, all remained with the control of the main thread.
The use of threads makes you think more about the way your application needs threading and can in the long run make it easier to improve / control your performance.
Async methods are faster to use but they are a bit magic - a lot of things happen to make them possible - so it's probable that at some point you will need something that they can't give you. Then you can try and roll some custom threading code.
It all depends on your needs.
The answer is "it depends".
It depends on what you're trying to achieve. I'm going to assume that you're aiming for more performance.
The simplest solution is to find another way to improve your performance. Run a profiler. Look for hot spots. Reduce unnecessary IO.
The next solution is to break your program into multiple processes, each of which can run in their own address space. This is easiest because there is no chance of the individual processes messing each other up.
The next solution is to use threads. At this point you're opening a major can of worms, so start small, and only multi-thread the critical path of the code.
The next solution is to use asynch IO. Generally only recommended for people writing some of very heavily loaded server, and even then I would rather re-use one of the existing frameworks that abstract away the details e.g. the C++ framework ICE, or an EJB server under java.
Note that each of these solutions has multiple sub-solutions - there are different breeds of threads and different kinds of asynch IO, each with slightly different performance characteristics, but again, it's generally best to let the framework handle it for you.