I'm trying to understand the internals of node.js and how it works under the hood
So if I've understood correctly,
The event loop is executing on a single thread, when app generates a new I/O task it makes a non blocking call to the event demultiplexer, the call immediately returns without any data, allowing the event loop to continue with other tasks. When data is ready the demultiplexer pushes an event (or more) in the event queue. Event loop takes out this event (handler with data)...
As I understand it, the demultiplexer is executing in the os(epoll in Linux) not in the application's main thread (in which event loop executes) and by definition demultiplexer is synchronous and blocking that's because the watch() call is blocking and this is exactly where my question stands.
I know that the demultiplexer is watching until one or more events are ready but:
If Watch() is executed in the os and not in the event loop what does blocking mean? It blocks what?
(Sorry for my English, I'm not a native)
Related
We all know that in Node.js, the functions are handled by worker thread for execution and then send to the event queue and then the event loop looks into the call stack.
If the call stack is empty then the event loop takes the function's context environment to call stack, and then call stack process it and give it as a response.
My question is if we have multiple functions with same timeout function and then all the function is given to worker thread then worker thread sends their context environment to the event queue,
and if the timeout of all the functions are same then they all come into the event queue at the same time and then if the call stack is empty then the event loop will send all the functions to call stack, and we all know the property of stack is FILO.
so if this happened resulting the last function should be sent in response first but this is not happening the first function is coming in response first if all the timeouts are the same?
There are lots of things wrong in how you describe things in your question, but I'll speak to the timeout issue that you ask about.
nodejs has its own timer system. It keeps a sorted list of timers and the ONLY Javascript timeout that has a physical system timer is the next one to fire. If multiple Javascript timeouts are set to fire at the same point in time, then they all share that one OS timer.
When that OS timer fires and when the main JS thread is free and able to pull the next event from the event loop, it will see a JS timer is ready to call its callback. If there are more than one ready to go, all for the same time, then the interpreter will call each of their callbacks one after the other, in the order the timers were configured (FIFO).
We all know that in Node.js, the functions are handled by worker thread for execution and then send to the event queue and then the event loop looks into the call stack. If the call stack is empty then the event loop takes the function's context environment to call stack, and then call stack process it and give it as a response.
My question is if we have multiple functions with same timeout function and then all the function is given to worker thread
That part is wrong. They aren't given to a worker thread.
Then worker thread sends their context environment to the event queue, and if the timeout of all the functions are same then they all come into the event queue at the same time
As I described above, timers are a special type of event in the event loop code. They use only one system timer at a time to schedule the next timer to fire. Multiple timers set to fire at the same time are all stored in the same list (a list element for that particular time) and all share the same OS timer when it's their turn to be next. So, they don't all come into the event queue at the same time. Nodejs has set one system timer for the group of timers set to fire at the same time. When that system timer fires and when the interpreter is free to pull the next event from the event loop, then it will call each callback for each timer set for that time one after another in FIFO order.
and then if the call stack is empty then the event loop will send all the functions to call stack, and we all know the property of stack is FILO.
I don't know what "send all the functions to that call stack" means. That's not how things work at all. node.js runs your Javascript in a single thread (except for WorkerThreads which are not in play here) so it calls the callback for one timer, runs that to completion, then calls the callback for the next timer, runs it to completion and so on...
so if this happened resulting the last function should be sent in response first but this is not happening the first function is coming in response first if all the timeouts are the same?
As I've said a couple times above, multiple timers set to fire at the same use one system timer and when that system timer fires, the callbacks for each of those timers are called one after the other in FIFO order.
References
You can learn more about the node.js timer architecture here: How does Node.js manage timers.
And, here's some more info about the node.js timer architecture taken from comments in the source code: How many concurrent setTimeouts before performance issues?.
Looking for a solution between setting lots of timers or using a scheduled task queue
Six Part Series on the Node.js Event Loop and How It Works
My question is if we have multiple functions with same timeout
function
poll phase controll the timer
First read here
I highly recommend this
series
So the main question is how does Node decide what code to run next?
As we know the Event Loop and the Worker Pool maintain queues for pending events and pending tasks, respectively.
don't confuse with Worker thread
Worker threads is different concept Read
here
But In realtiy the Event Loop does not actually maintain a queue. Instead, it has a collection of file descriptors that it asks the operating system to monitor, using a mechanism like epoll (Linux), kqueue (OSX), event ports (Solaris), or IOCP (Windows). These file descriptors correspond to network sockets, any files it is watching, and so on. When the operating system says that one of these file descriptors is ready, the Event Loop translates it to the appropriate event and invokes the callback(s) associated with that event
The Worker Pool uses a real queue whose entries are tasks to be processed. A Worker pops a task from this queue and works on it, and when finished the Worker raises an "At least one task is finished" event for the Event Loop.
timer callback is called depends on the performance of the system (Node has to check the timer for expiration once before executing the callback, which takes some CPU time) as well as currently running processes in the event loop.
Are there explicit considerations about the latency of any single request in the node.js event loop? AFAICT every IO call returns an eventEmitter which emits an event. The processing of all the events is multiplexed through the use of a pipe. So it is possible that the event that needs to be processed for an important request may be placed too far back into the pipe. Is there some sort of priority queue that can be used to schedule the order of execution of eventHandlers ?
Here's why I asked this question in the first place. I decided to give a gist.github link because the reason is long and related to the technical question
It's not clear exactly what you're asking here. Your Javascript does not directly add things to the event queue (that is only done with native code). Instead, you call some async operation and the native code behind that operation adds something to the event queue when the async operation completes.
This article The Node.js Event Loop, Timers, and process.nextTick() gives you a lot of details about how the event queue is serviced and how it handles different types of events (timers, I/O, etc...).
In general things are FIFO (first in, first out) within an event type with some exceptions.
process.nextTick() will run its callback BEFORE waiting I/O events.
setImmediate() will runs its callback AFTER waiting I/O events.
More detail here: setImmediate vs. nextTick and nextTick vs setImmediate, visual explanation and setTimeout vs. setImmediate vs. nextTick
So it is possible that the event that needs to be processed for an important request may be placed too far back into the pipe.
You'd have to show us the specific situation you're concerned about. If you yourself are scheduling a callback with setTimeout(), setImmediate() or process.nextTick(), then you have some control over when it happens by which of these three you pick. If you aren't scheduling it yourself (e.g. it's the completion callback of some async operation), then you don't control it's scheduling in the event loop. It will go into the sub-queue that matches its type and be served FIFO from that phase or the event loop (as described in the above articles).
Is there some sort of priority queue that can be used to schedule the order of execution of eventHandlers ?
There is no exposed priority system. Within an event type, things are FIFO. Again, if you give us an actual coding example so we can see exactly what you're trying to do, we could offer some help on what your choices are. You may be able to use the setTimeout(), setImmediate() and process.nextTick() tools that are already available or you may want to implement your own task queuing and prioritization system that runs off some of the above three methods that allows you to prioritize things that are already queued yourself.
About priorities of execution in the event loop:
setImmediate() runs before setTimeout(fn, 0)
nextTick() triggers the callback on next tick (iteration)
Natively the event loop in node.js does not support priorities. You can always implement your own priority queue or use an existing one like here and assign your functions to the priority queue.
This question already has answers here:
asynchronous and non-blocking calls? also between blocking and synchronous
(15 answers)
Closed 4 years ago.
I am reading 'Operation System Concepts With Java'. I am quite confused by the concept of
blocking and synchronous, what are the differences between them?
Blocking may or may not be the same as synchronous, depending on the context. When we talk about method calls, then a synchronous call can also be said to be blocking (I'll get back to this in a bit), because the thread calling the method cannot proceed forward until the method returns. The antonym in this case would be asynchronous.
In lock terminology, a lock is said to be blocking if the thread waiting to acquire it is put in a suspended mode until the lock becomes available (or until a timeout elapses). The antonym in this case is a non-blocking lock, meaning that the thread returns immediately even if it cannot acquire the lock. This can be used to implement the so called spinning lock, where you keep polling the state of the lock while keeping the thread active.
Having said this, you can extrapolate the difference between the concepts: synchronous generally means an activity that must wait for a reply before the thread can move forward. Blocking refers to the fact that the thread is placed in a wait state (generally meaning it will not be scheduled for execution until some event occurs). From here you can conclude that a synchronous call may involve blocking behavior or may not, depending on the underlying implementation (i.e. it may also be spinning, meaning that you are simulating synchronous behavior with asynchronous calls).
Blocking - operation are said to have blocking behavior if it waits for some event to get complete. For example: if a lock is not available a thread may enter a wait state on event till lock is available. Such an operation is said to be blocking.
Synchronous - Synchronous call can be easily understood with an example of http protocol where client waits for reply from server an then proceeds. Synchronous call can be blocking or non blocking.
Asynchronous - A method can asynchronous call other method. After a call it can continue to execute its next instruction. When called method completes it execution it will send an reply/callback to caller method of it's success or failure.
Non-blocking - Non blocking behavior is like checking the condition at that instance. For example- in case of locks if it is not available it will not wait till it is available like blocking operation. Also we need to repeatedly check the availability of locks as there will be no callback like asynchronous calls.
Summary:
Blocking is always synchronous.
Synchronous call have blocking operations if it waits for some event to get complete, caller method may enter wait state.
Synchronous call is non blocking, if it repeatedly check for some event to occur before proceeding for next instruction. Caller method does not enter wait state on some event to complete.
Asynchronous call cannot be blocking and it involves callback from called method which needs to handle.
I would classify them as follows:
Blocking - Thread will wait on action untill success or failure (highlight on 'will wait', failure is commonly a timeout)
Synchronous - Thread will complete the action, either by success or failure, before reaching any line after it (highlight on action completion)
Non-blocking - Thread will not wait to complete the action, executes action immediately
Asynchronous - Another thread (either logical or physical) will complete the action or inform it is ready using a callback, will not wait before performing following commands.
Note: from here the name asynchronous originates, since you cant be sure in which order the commands will execute
synchronous means that the work is done in the thread that calls the function and the method does not return until it is finished.
asynchronous methods return immediately because another thread does the work and raises a flag or fires an event when the work is done.
blocking means that the thread executing a blocking event will wait until the event has occurred. for example you try to read from a socket and none sends you a message. the blocking call will not return until the message has been revived from the socket.
well and nonblocking means the opposite to blocking with implies that nonblocking calls are asynchronous.
Is "event based" the same as "asynchronous"?
No it doesn't imply that the events are asynchronous.
In a event driven single threaded system, you can fire events, but they are all processed serially. They may yield as part of their processing, but nothing happens concurrently, if they yield, they stop processing and have to wait until they are messaged again to start processing again.
Examples of this are Swing ( Java ), Twisted ( Python ), Node.js ( JavaScript ), EventMachine ( Ruby )
All of these examples are event driven message loops, but they are all single threaded, every event will block all subsequent events on that same thread.
In programming, asynchronous events are those occurring independently of the main program flow. Asynchronous actions are actions executed in a non-blocking scheme, allowing the main program flow to continue processing.
So just because something is event driven doesn't make it asynchronous, and just because something is asynchronous doesn't make it event driven either; much less concurrent.
They are essentially orthogonal concepts.
"event driven" essentially means that the code associated to certain function calls is bind at runtime (and can change through the execution).
Who "fires" the event doesn't know what will handle it, and who handle the event is defined to respond to the event through an association defined while the program executes. Typically though function pointers, reference or pointers to object carrying virtual methods etc.)
"asynchronous" means that a program flow doesn't have to wait for a call to be executed before proceed over (mostly implemented with a call that returns immediately after delegating the execution to another thread or process)
Not all events are asynchronous (think to the windows SendMessage, respect to PostMessage), and not all asynchronous calls are necessary implemented by "events" (although the use of the event mechanism is quite common to implement asynchronous calls)
One meaning of asynchronous is that at a point where you emit an computation, you don't wait for an answer, but you get the answer later. The answer comes in orthogonal to you normal control flow.
One way the answer comes in is using events: They happen spontaneously in this case, without your code triggering them. In a handler you may process the result.
Whereas the computation and answer is connected by the point in control flow for synchronous mode, you have to do the connection yourself in asynchronous mode. For example by use of a sequence number or something.
If I understand node.js correctly, then a single event handler that does not release the cpu (infinite loop for example) can freeze the whole system. Is this correct?