I've been looking into Node.js and it seems like it offers a whole new dimension to web server development. However, I'm confused about the asynchronous processing that is held up as its main point of awesomeness.
Does the runtime take care of all the asynchronous processing automatically or does it simply offer APIs that you can use to build asynchronous apps? Or perhaps both?
automatically. The Event Loop provides async processing of your function calls
Callback functions (or promises) let you break up your logic into separate (non-blocking) chunks that are added to the Event Loop's queue, so your program flow ends up being a bunch of async function calls
Node.js is good for asynchronous model since it uses callback functions will be event-driven.
Node.js is single threaded so it can manage resources more efficiently and in a very simple manner for asynchronous model.
Node.js provides many asynchronous modules/APIs but there are synchronous APIs too.
But you can also write synchronous functions in node.js too.
e.g. fs can be written synchronously or asynchronously.
My answer to you question will be:
It is the way how the code is written.
You can also write async code in other languages like Java but the resource management will be very complex since you need to handle mulit-thread environment.
But David is correct about the even driven loop.
The Node.js API is designed around asynchronous I/O processing. You will still have to develop using async patterns. It is not "automatic" in the sense that the following synchronous code would work in Node.js: results = db.query();
Related
I'm using nodejs for one of my server applications. But I found out there's some latency while running my app and I found that I didn't use await for a async function. Is there any performance issues while not using await for a async function?
The async-await is used to simplify the complexities created by nested callbacks. So, both async-await and callback patterns serialize asynchronous processes.
This means that one process fires after the completion of another and, this could be experienced as a latency depending on the application.
So, NO - async-await does not create any latency in its own right.
I have a problem with the concept of async in NodeJS. I have read a lot about the event poll in NodeJS. They say things like:
The event loop is what allows Node.js to perform non-blocking I/O
operations
or
Node uses the Worker Pool to handle "expensive" tasks. This includes
I/O for which an operating system does not provide a non-blocking
version, as well as particularly CPU-intensive tasks.
or
These are the Node module APIs that make use of this Worker Pool such
as File System(fs)
So, I found that Node manages I/O running using a Thread pool. Now my question is, if Node is managing them, why do we need to utilize async programming at all in NodeJS? And whats the reason behind some modules like BlueBird?
tl;dr: You need async to prevent a blocking of the Event-Loop.
NodeJS uses a certain number of threads to handle clients. There basically are two types of threads:
Event Loop (or your main thread)
Worker Pool (or threadpool)
The Event Loop:
Basically the reason why async programming is needed:
Once all events are registered, NodeJS enters the Event Loop and handles all incoming requests as well as outgoing responses. All of them pass through the Event Loop.
Worker Pool
As you already said, NodeJS uses the Worker Pool to perform I/O and CPU intensive tasks .
Asynchronous Code
In order to prevent blocking the main thread, you want to keep your Event Loop clean, and delegate certain task. This is where async code is needed. That way your code becomes non-blocking. The terminology concerning async and non-blocking is a bit vague though. To clarify:
Async Code: Performs certain tasks in parallel
Non-Blocking: Basically Polling without blocking further code.
In NodeJS however, Async is often used for I/O operations. There it doesn't just mean "perform in parallel", because it mostly means "don't block and get the signal".
So in order to make the Event Loop of NodeJS efficient, we don't want to wait for an operation to finish. Therefore we register an async "listener" instead. This allows NodeJS to efficiently manage its own resources.
BlueBird (or Promises in general):
Libraries like BlueBird which you mentioned, aren't required anymore because NodeJS supports promises out of the box (see note here).
Promises are just another way of writing asynchronous code. So are Async/Await and Generator Functions.
Side note: Functions defined with the async keyword actually yield a promise.
It has been said that Node is an event driven language.EX:
http.get('www.google.com', (response) => {
let data = ''
response.on('data', (chunk) => {
data += chunkenter code here
})
response.on('end', () => {
console.log(data)
})
})
Here, a request has been forwarded to www.google.com and we set up event listeners to listen for the 'data' and 'end' events so as the request data comes in, we concatenate all the data together and log it once the request ends. The flow of the program here is determined by events.
So, My question is that, can we do same/similar things in any other language(framework)? Can this thing be only done in Node?
No.
Any language can do this, but Node is currently considered the best due to its enormous package library.
Other cleaner languages such as Scala and C# also include Promises/Futures by default. Golang has channels, Python has async/await, and Ruby/Lua/Java also offer promises or promise-like functionality in the form of external packages.
My question is that, can we do same/similar things in any other language (framework)?
Any language can implement events and have some operations that use events. You can even do it in C or assembly language if you want. But (see discussion below), node.js is deeply, deeply architected to be event driven (at the lowest level) which is not generally true of other languages. That does make a difference.
Can this thing be only done in Node?
No. Levels of event notification can be done in any language.
But, node.js is fundamentally a single-threaded, event driven architecture and internally runs off an event queue so lots of things only work as event driven such as timers, network I/O, asynchronous file I/O. The internal event queue is fundamental to the architecture of how node.js handles multiple asynchronous operations in flight at the same time.
For example, all network I/O is asynchronous and non-blocking and the ONLY way you can get network data is via callbacks which are internally triggered via the event queue.
One thing to keep in mind is that the single-threaded, event driven architecture of node.js is not exactly the same thing as some code that might decide to use the EventEmitter style event-driven notification system. The internal architecture is pretty fundamental to the node.js design and it defines node.js non-blocking I/O architecture and affects how multiple asynchronous operations in node.js are run, when they run and what types of synchronization exists between multiple asynchronous operations in flight at the same time and how you code that. This is very fundamental to node.js and I am not aware of another mainstream language that has this same internal design. So, some of this is specific to node.js.
Creating some code that uses an EventEmitter style object to notify listeners of various events that are occurring is a capability that pretty much any language can add on top of their existing design (including node.js). For example, you could create an EventEmitter style object in C that encapsulates a system timer and use events to notify listeners of timer events. That's somewhat analogous to the publish/subscribe model which can be implemented on top of any language.
Which process handles async or simultaneous work to happen in node js. Is there any specific api that takes care of all these events to happen in queue?
Is there any specific api that takes care of all these events to happen in queue?
No, not accessible from Javascript. The event queue is completely under the covers. You don't access it directly.
The implementation of asynchronous operations is all handled in native code. When an async operation completes, its native code calls an internal C++ API that inserts the completion event into the node.js event queue. If no Javascript is currently running in node.js at that moment, then inserting the item in the event queue will trigger it to get pulled out of the queue and the callback associated with it will be run. If Javascript is running at the moment, it will stay in the event queue until the current piece of running Javascript finishes at which point the interpreter will check the event queue, see there is an event in there and will pull that event out and run the callback associated with that event.
Which process handles async or simultaneous work to happen in node js.
It is not entirely clear what you mean by this. Each node.js function that is asynchronous has its own implementation. Networking uses OS-level event driven networking (not threads). Async file I/O uses a native thread pool. Timers use OS level timers. Some other asynchronous operation will have its own implementation and do it some other way as it completely depends upon what the async operation is for who it will accomplish its work.
The only three ways (I know of) for you to write your own asynchronous operation are:
Compose your own operation entirely using existing asynchronous operations such as request this data from another server, then write it to this file.
Use native code to write your own node.js add-on that can expose an asynchronous interface and use native code to implement that asynchronous interface in whatever manner is most appropriate for your operation.
Run some other process and communicate back the result from that other process. This can be some other program written in any language or it can be Javascript that you run in another node.js process.
Now, there are a few ways you can influence the event queue timing of some things from Javascript. For example, setTimeout(fn, t), process.nextTick(fn) and setImmediate(fn) all have slightly different ways they insert your callback function into the event queue that determines what (that is already in the event queue) they run before or after. But, these by themselves just schedule a callback sometime in the future - they don't actually implement an asynchronous operation that accomplishes some tasks in a non-blocking way.
You may want to read some of these references:
The Node.js Event Loop, Timers, and process.nextTick()
setImmediate() vs nextTick() vs setTimeout(fn,0) – in depth explanation
Demystifying Asynchronous Programming Part 1: Node.js Event Loop
You might be thinking of child_process.spawn().
From the NodeJS documentation
The child_process.spawn(), child_process.fork(), child_process.exec(),
and child_process.execFile() methods all follow the idiomatic
asynchronous programming pattern typical of other Node.js APIs.
Each of the methods returns a ChildProcess instance. These objects
implement the Node.js EventEmitter API, allowing the parent process to
register listener functions that are called when certain events occur
during the life cycle of the child process.
The child_process.exec() and child_process.execFile() methods
additionally allow for an optional callback function to be specified
that is invoked when the child process terminates.
The reactor pattern which is utilized by libuv for handling IO is synchronous by design but libuv supports async io. How is this possible? Does libuv extend the reactor's design somehow to support async io? Does using multiple threads/event loops aid in achieving this?
The I/O model of Node and libuv is very similar to what nginx does internally.
The libuv uses a single-threaded event loop and non-blocking asynchronous I/O. All functions are synchronous in a way that they run to completion but some clever hackery with promises and generators can be used to appear that they don't (when in fact both the invocation of the generator function is non-blocking and returns the generator object immediately and the generator methods like .next() run to completion), plus the new async/await syntax makes it very convenient.
For operations that cannot be accomplished in a non-blocking way Node uses a thread pool to run the blocking operations in separate threads but this is done transparently and it is never exposed to the application code written in JavaScript (you need to step down to C++ to work with that directly).
See: http://docs.libuv.org/en/v1.x/design.html
Unlike network I/O, there are no platform-specific file I/O primitives libuv could rely on, so the current approach is to run blocking file I/O operations in a thread pool. [...]
libuv currently uses a global thread pool on which all loops can queue work on. 3 types of operations are currently run on this pool:
File system operations
DNS functions (getaddrinfo and getnameinfo)
User specified code via uv_queue_work()
See also those answers for more details:
what is mean by event loop in node.js ? javascript event loop or libuv event loop?
*NodeJS event loop internal working
Prevent NodeJS from exiting event-loop
How node.js server serve next request, if current request have huge computation?
Which would be better for concurrent tasks on node.js? Fibers? Web-workers? or Threads?
Speed up setInterval
Async.js - Is parallel really parallel?
Node.js: Asynchronous Callback Execution. Is this Zalgo?
See the links and illustration in those answers. There are a lot of resources to read bout that topic.