For this snippet:
const foo = [1, 2];
const bar = ['a', 'b'];
foo.forEach( num => {
console.log(`setting setImmmediate ${num}`);
setImmediate(() => {
console.log(`running setImmediate ${num}`);
bar.forEach(char => {
console.log(`setting nextTick ${char}`);
process.nextTick(() => {
console.log(`running nextTick ${char}`);
})
})
});
} )
The output is
$ node scratch.js
setting setImmmediate 1
setting setImmmediate 2
running setImmediate 1
setting nextTick a
setting nextTick b
running setImmediate 2
setting nextTick a
setting nextTick b
running nextTick a
running nextTick b
running nextTick a
running nextTick b
From the docs
the nextTickQueue will be processed after the current operation completes, regardless of the current phase of the event loop.
As I understand, process.nextTick() will add to the current event's nextTickQueue, and executed immediately after the current event, no matter what phase the event loop is in.
Shouldn't the output therefor be the following?
setting setImmmediate 1
setting setImmmediate 2
running setImmediate 1
setting nextTick a
setting nextTick b
running nextTick a
running nextTick b
running setImmediate 2
setting nextTick a
setting nextTick b
running nextTick a
running nextTick b
the nextTickQueue will be processed after the current operation completes, regardless of the current phase of the event loop.
I misunderstood the event loop documentation in thinking that "current operation" means the currently processing event, where in actually it means the currently processing phase.
From Danial Khan's What you should know to really understand the Node.js Event Loop/:
Related
Please see below code:
Promise.resolve().then(() => console.log("promise1 resolved"));
Promise.resolve().then(() => console.log("promise2 resolved"));
Promise.resolve().then(() => {
console.log("promise3 resolved");
process.nextTick(() => {
console.log("Inner Next tick");
});
});
Promise.resolve().then(() => console.log("promise4 resolved"));
setImmediate(() => console.log("set immediate1"));
setImmediate(() => console.log("set immediate2"));
process.nextTick(() => console.log("next tick1"));
process.nextTick(() => console.log("next tick2"));
setTimeout(() => console.log("set timeout"), 0);
Output of this code is:
next tick1
next tick2
promise1 resolved
promise2 resolved
promise3 resolved
promise4 resolved
Inner Next tick
set timeout
set immediate1
set immediate2
What i cannot understand is why is the callback for innerTextTick i.e console.log("Inner Next tick") executed before setTimeout.
As per my understanding, first the nextTick queue will be executed, nextTick 1 and 2, then all the resolved Promises and then to the timers phase, which happens as expected. But in the thid promise, i have registered a new callBack in the nextTick queue which should be called after the event loop reaches the nextTick queue again, i.e after the timers phase.
But why does the event loop go back from the promise microtask queue instead of proceeding to the next phase, i.e the timers queue.
https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/#process-nexttick:
You may have noticed that process.nextTick() was not displayed in
the diagram, even though it's a part of the asynchronous API. This is
because process.nextTick() is not technically part of the event
loop. Instead, the nextTickQueue will be processed after the current
operation is completed, regardless of the current phase of the event
loop. Here, an operation is defined as a transition from the
underlying C/C++ handler, and handling the JavaScript that needs to be
executed.
Looking back at our diagram, any time you call process.nextTick() in
a given phase, all callbacks passed to process.nextTick() will be
resolved before the event loop continues.
Because in the global scope of main executed file, the Event Loop doesn't come into play yet. All the Promise.resolve() in global goes into MicrotaskQueue and all global nextTick goes into nextTick queue. So your inner nextTick goes into the nextTick queue before the Timeouts Phase of the Event Loop run for the first time. That's also why promise4 resolved even before Inner Next Tick too.
If i hit /route2 , i get reponse 15s later,but during that time /route1 gives response immediately.
Shouldn't server wait for 15s and then give the response to /route1.
I read https://medium.com/#cinish/nodejs-multiple-client-requests-694d6353218b but could not the idea.
What i understood is as /route2 is hit console P1 then as there is setTimeout() ,put it in external thread,then console P2 then as setTimeout() ,put it in external thread.Now wait for setTimeouts() to finish.
(At this time the event loop must be busy as it waiting for p1 and p2 fulfillment, so it should accept no new client request).
But it does. Why?
app.get("/route1", (req, res) => {
res.send("Route1");
});
app.get("/route2", async (req, res) => {
const p1 = new Promise((resolve, reject) => {
console.log("P1");
setTimeout(() => {
console.log(5);
resolve();
}, 15000);
})
const p2 = new Promise((resolve, reject) => {
console.log("P2")
setTimeout(() => {
console.log(1);
resolve();
}, 1000);
});
const ans = await Promise.all([p1, p2]);
res.send("Route2");
})
setTimeout() is "non-blocking". That means it sets the timer, returns immediately from the setTimeout() and then continues to execute the rest of the code.
So, in your case /route2 creates the two promises, sets two timers and then waits for Promise.all() to finish. At the point it hits the await, your async route handler returns a promise and returns control back to the event loop.
So, as soon as the /route1 request arrives, it's ready to be processed. /route2 is not active until both timers are done.
Then, when the longer timer is ready to fire, the next time the JS interpreter goes back to the event loop to check for anything else to do, it will see that timer, process the timer callback and that timer callback will resolve the p1 promise and then cause the Promise.all() to be resolved.
If i hit /route2 , i get reponse 15s later,but during that time /route1 gives response immediately. Shouldn't server wait for 15s and then give reponse to /route1.
As explained about setTimeout() is non-blocking so while the /route2 handler is waiting for the two timers to fire, nodejs is free to process other events (like the incoming /route1 event).
What i understood is as /route2 is hit console P1 then as there is setTimeout() ,put it in external thread,then console P2 then as setTimeout() ,put it in external thread.Now wait for setTimeouts() to finish.
Timers are not run in external threads. They are a somewhat unique design in the nodejs event loop. Timers are stored in a sorted linked list in the order of when they are due. Each time the JS interpreter gets back to the event loop and gets to the timer section of the event loop, it just checks the front most timer in the linked list to see if its time has arrived. If so, it triggers that event. If not, it just goes and looks for other types of events.
Eventually, if the event loop has nothing to do and goes to sleep, it will sleep for an appropriate time to wake up in time for the next timer scheduled to run (if nothing else wakes it up before that).
(At this time the event loop must be busy as it waiting for p1 and p2 fullfillment,so it should accept no new client request). But it does.Why?
This is your main wrong assumption. At this time, the event loop is entirely free to process other events. The two timers are set and the event loop will run their callbacks when their time is due, but until then the event loop will process any other incoming events like the /route1 request.
I'm struggled to understand how crypto's result returned before nextTick, Immediate or timeout.
I know this is synchronous crypto so it block the event loop, but how exactly it block when others function got called first and time to finish is faster than cryto?
The order of code line here is not important?
const fs = require('fs');
const crypto = require('crypto');
const start = Date.now();
setTimeout(() => console.log('Timer 1 finished'), 0);
setImmediate(() => console.log('Immediate 1 finished'));
fs.readFile('test-file.txt', () => {
console.log('Reading file.... and finished');
console.log('-----------------');
setTimeout(() => console.log('Timer 2 finished'), 0);
setTimeout(() => console.log('Timer 3 finished'), 2000);
setImmediate(() => console.log('Immediate 2 finished'));
process.nextTick(() => console.log('Process nextTick'));
crypto.pbkdf2Sync('password', 'salt', 100000, 1024, 'sha512');
console.log(Date.now() - start, 'Password encrypted');
crypto.pbkdf2Sync('password', 'salt', 100000, 1024, 'sha512');
console.log(Date.now() - start, 'Password encrypted');
crypto.pbkdf2Sync('password', 'salt', 100000, 1024, 'sha512');
console.log(Date.now() - start, 'Password encrypted');
crypto.pbkdf2Sync('password', 'salt', 100000, 1024, 'sha512');
console.log(Date.now() - start, 'Password encrypted');
});
Result
Timer 1 finished
Immediate 1 finished
Reading file.... and finished
-----------------
2003 'Password encrypted'
3980 'Password encrypted'
5961 'Password encrypted'
7961 'Password encrypted'
Process nextTick
Immediate 2 finished
Timer 2 finished
Timer 3 finished
Why crypto.pbkdf2Sync get executed before setTimeout() in Node JS?
In a nushell, because crypto.pbkdf2Sync() is synchronous and node.js must complete all synchronous code in the current event processing before it can process the completion of any other asycnhronous operations.
setTimeout(), setImmediate() and process.nextTick() all schedule their callback for a FUTURE tick of the event loop. When it's their turn to run, they insert a callback into the event queue. But, NOTHING in the event queue gets to run until the current piece of Javascript is done executing. So, that means that ALL the synchronous code in your fs.readFile() callback runs to completion before anything else can be pulled from the event queue and run.
This sequencing is all because of the "event-driven" nature of node.js. Asynchronous operations always complete on a future tick of the event loop, thus any synchronous code that is already in the process of running must complete before any asynchronous operation can get its completion callback called, even if the asynchronous operation is of the "very-soon" types such as setTimeout(..., 0), process.nextTick() or .setImmediate().
So, the sequence of events goes like this (ignoring the things before fs.readFile() as they aren't relevant since they both occur before the fs.ReadFile() callback completes.
Call fs.readFile(). It starts the file operation and immediately returns.
Node.js goes back to event loop and waits for the next event.
fs.feadFile() completes and inserts a completion callback event in the event queue.
When the interpreter has finished anything else it was running, it grabs the next event from the event queue and calls the fs.feadFile() callback.
setTimeout(..., 0) is called. That schedules a timer event for the next tick of the event loop.
setTimeout(..., 2000) is called. That schedules a timer event for 2 seconds from now.
setImmediate(...) is called. That schedules an event for a future tick of the event loop.
crypto.pbkdf2Sync(...) is called. This is a synchronous operation so it runs immediately and then finishes.
crypto.pbkdf2Sync(...) is called. This is a synchronous operation so it runs immediately and then finishes.
crypto.pbkdf2Sync(...) is called. This is a synchronous operation so it runs immediately and then finishes.
crypto.pbkdf2Sync(...) is called. This is a synchronous operation so it runs immediately and then finishes.
The callback for fs.readFile() finishes. The JS interpreter then pulls the next event from the event queue.
Based on the circumstances and the relevant priority of various types of events, the next item pulled from the event queue is the process.nextTick() callback.
Then, the setImmediate() event.
Then, the setTimeout(..., 0).
Then, after some delay, the setTimeout(..., 2000).
I am new to node.js and little bit confused on understanding the event-loop. As far as i know from https://github.com/nodejs/node/blob/master/doc/topics/event-loop-timers-and-nexttick.md, the event-loop phases only process setTimeout, setInterval, setImmediate, process.nextTick, promises and some I/O callbacks.
My question is, if i have following code:
for (var i = 0; i < 100000000; i++)
;
in which phase the above code will get executed ?
Regular JavaScript code, like the for loop in your example, is executed before the queues are cleared. The first thing node will do is run your code, and will only call callbacks, timeout results, I/O results, and so on after your code finishes.
As an example, you could try this code:
fs.open('filename', 'r', () => {
console.log('File opened.');
});
for (var i = 0; i < 100000000; i++);
console.log('Loop complete.');
No matter how big or small your loop variable, 'Loop complete' will always appear before 'File opened'. This is because with only one thread, node can't run the callback you've supplied to the fs.open function until the loop code has finished.
Remember that there isn't a "main" thread that node keeps going back to. Most long-running node programs will run through the code in main.js pretty quickly, and subsequent code is all going to come from callbacks. The purpose of the initial execution is to define how and when those callbacks happen.
In the node event loop doc (https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick), the following code is given as an example:
const fs = require('fs');
function someAsyncOperation(callback) {
// Assume this takes 95ms to complete
fs.readFile('/path/to/file', callback);
}
const timeoutScheduled = Date.now();
setTimeout(() => {
const delay = Date.now() - timeoutScheduled;
console.log(`${delay}ms have passed since I was scheduled`);
}, 100);
// do someAsyncOperation which takes 95 ms to complete
someAsyncOperation(() => {
const startCallback = Date.now();
// 10ms loop
while (Date.now() - startCallback < 10) {
// do nothing
}
});
The loop keeps scanning according to phases and after fs.readFile() finishes, the poll queue is is empty, so its callback will be added and immediately executed. The callback holds a blocking 10ms loop before the timer is executed. That is why the delay will display:
105ms have passed since I was scheduled instead of the 100ms you might expect.
Most of your code will live in callbacks so will be executed in the poll phase. If not, like in your example, it will be executed before entering any phases as it will block the event loop.
The caveat are callbacks scheduled by setImmediate that will enter the check phase before resuming the poll phase in the next loop.
I have alrady watched this wonderfully video related to the event loop work
https://www.youtube.com/watch?v=8aGhZQkoFbQ
From some reading,I suppose process.nextTick will put the callback at the very beginning of the event queue .Also I see some people use nextTick to execute a function immediately
I try to write some code to confirm it
function printNextTick(s){
process.nextTick(function(){
console.log(s)
})
}
for (var i=0;i<5;i++){
printNextTick(i)
}
what I expect is 4,3,2,1,0,because
nextTick will always put the callback at the beginning of the queue,and the script is "execute to complete", which means no callback will be execute during "put callback to the queue","4" is push last,so after the script complete,"4" should at the first place of the queue,that is why I expect "4" comes out first
But it is not correct
================================ Update ====================================
Here the new code
function printNextTick(s){
process.nextTick(function(){
console.log("nextTick : "+s)
})
}
function printTimeout(s){
setTimeout(function(){
console.log("timeout : "+s)
},0)
}
for (var i=0;i<5;i++){
printTimeout(i)
}
for (var i=0;i<5;i++){
printNextTick(i)
}
//nextTick : 0
//nextTick : 1
//nextTick : 2
//nextTick : 3
//nextTick : 4
//timeout : 0
//timeout : 1
//timeout : 2
//timeout : 3
//timeout : 4
Just wonder why some people treat nextTick as immediate Originally,I suppose nextTick will put callback at the beginning of the event queue rather than ending as normal,for now,I suppose the queue for nextTick has a higher priority than other queues ,at least higher than queue for timeout
================== Update ===============
Not true above !
The result for code below is quite unpredictable
function printNextTick(s){
process.nextTick(function(){
console.log("nextTick : "+s)
})
}
function printTimeout(s){
setTimeout(function(){
console.log("timeout : "+s)
for (var i=0;i<5;i++){
printNextTick(i)
}
},0)
}
for (var i=0;i<5;i++){
printTimeout(i)
}
Any callbacks you pass to process.nextTick() are appended to an internal queue that gets flushed (to a maximum IIRC so as not to completely starve the rest of the event loop), in order, at the "next tick." So in your example you are first appending a callback that prints '0', then a callback that prints '1', etc. These are then executed in order and you see the numbers printed in numerical order.
Just wonder why some people treat nextTick as immediate
From the documentation:
process.nextTick() fires immediately on the same phase
setImmediate() fires on the following iteration or 'tick' of the event loop
In essence, the names should be swapped. process.nextTick() fires more immediately than setImmediate() but this is an artifact of the past which is unlikely to change. Making this switch would break a large percentage of the packages on npm. Every day more new modules are being added, which mean every day we wait, more potential breakages occur.
Originally,I suppose nextTick will put callback at the beginning of the event queue rather than ending as normal,for now,I suppose the queue for nextTick has a higher priority than other queues, at least higher than queue for timeout
From process docs:
process.nextTick() runs before any additional I/O events (including timers)
To understand the difference between nextTick and setTimeout, consider my implementation of setTimeout:
function setTimeout2(callback, ms) {
var t = process.hrtime(),
_next = () =>
process.nextTick(process.hrtime(t)[0] > !!ms?callback:_next);
_next();
}
function printNextTick(s){
process.nextTick(() => console.log('n'+s))
}
function printTimeout(s){
setTimeout2(() => {
console.log('t'+s);
for (var i=0;i<5;i++)
printNextTick(s+':'+i);
});
}
for (var i=0;i<5;i++)
printTimeout(i);
It behaves very similar to setTimeout, except it is less efficient. To answer your question, no, process.nextTick(callback) appends callback to the nextTick queue. setTimeout(callback, ms) also appends its callback to the nextTick queue if diff > ms (not >= ms).