This an Angular app, and this specific code is inside a webworker in Typescript. I'm still new to webworkers, but both the sleep and the loop execute inside the same thread.
The intent is to poll a service and exit the loop when a the process is completed. My problem is the sleep call below is not sleeping. I need it to delay for at least 9 seconds, and ideally I'd like it to be configurable. But it runs as though the sleep didn't run.
I have two questions:
Why is the sleep not working?
This is an Angular app served by a NodeJS container on cirrus. When the Angular app is requested and served by this NodeJS server, I'd like to pass a secret defined at the NodeJS server along with the Angular app. This secret would be the polling delay. Would a cookie value be returned in the NodeJS Angular App response? Not sure what the best response would be.
Code below:
function sleep(ms: number) {
return new Promise((resolve) => {
log('DEBUG','Sleeping for ' + ms + ' ms');
setTimeout(resolve, ms);
});
}
while (jobIsStillRunning(jobExecutionResult)) {
postMessageWithLog(jobExecutionResult);
log('DEBUG','sendFilePolling() jobExecutionResult=' + JSON.stringify(jobExecutionResult));
sleep(30000); // Sleep function runs but doesn't do anything.
jobExecutionResult = await getAsyncFilesResult(jobExecutionResult);
}
You need to await on sleep function
I think you need to know macro and micro tasks.
Microtasks come solely from our code. They are usually created by promises: an execution of .then/catch/finally handler becomes a microtask
If a microtask recursively queues other microtasks, it might take a long time until the next macrotask is processed. This means, you could end up with a blocked UI, or some finished I/O idling in your application.
example:
macrotasks: setTimeout, setInterval, setImmediate, I/O, UI rendering, etc/
microtasks: process.nextTick, Promises, etc.
code:
console.log('1')
setTimeout(()=>{
console.log('2')
},0)
Promise.resolve().then(()=>{
console.log('3')
})
console.log('4')
output:
1
4
3
2
wait what?!
explain:
console.log('1') it's a normal code and immediately run.
setTimeout it's a macrotask then it will add to macro queue
Promise.resolve it's a microtask then it will add to micro queue
console.log('4') it a normal code and immediately run.
now when normal codes are end and now microtaskqueue and macrotaskqueue need to be dequeue.
inside microtaskqueue we have one task and it's Promise.resolve then immediately run .then function
now microtaskqueue is clear too, inside macrotaskqueue we have one task and it's setTimeout then immediately run timeoutCallback function
if you want to use sleep function without await you need to handle this.
but you can add await before your sleep function and wait on it
Related
Have a question for those with more experience with worker threads here..
I have been doing some testing with worker threads and have a question on how they work, or maybe how they should work.
I am using a worker thread pool called piscina, this has been setup and appears to be working. 'Appears' to is the key..
Here is my scenario. I have a 'workers.js' file that has a 'longer' run script ( this is for testing and purposefully a long loop)
When running it, it does what it should, the main EL is still open to process other tasks etc, however what I have noticed is that there only appears to be 1 worker.
What I mean by that, is subsequent request to that route seem to get queued up, so the request don't run in parallel, instead, waits until the first worker finished, then goes on to the next.
What I would like to have happen, is that each request fires off a new 'worker' (to a point, then place them in the que) right now we have the app running in containers, so if the CPU or Mem gets too high, it should push traffic to the other container etc.
Now with that being said, we would still like to have the workers spawn out with each request as to not bottleneck the inbound request to that 'page'.
Again, maybe I'm not understanding this properly but anyone with more experience with workers that would help me out would be greatly appreciated.
*** edit with code ***
Route:
*Route contains imports of piscina library
router.get('/:error?', auth("3","edit"),function(req,res){
console.log('running')
let piscina = new Piscina({
filename: path.resolve(__dirname, 'worker.js'),
minThreads:5
});
const result = piscina.run({accountID:req.session.AccountID,cID:req.session.cID,cCode:req.session.cCode}).then(data =>{
console.log(date)
})
});
Worker file
module.exports = async({accountID,cID,cCode}) => {
const n = 10000000000;
for (let i = 1; i <= n; i++) {
}
return 'finished';
})
After running the loop, it simply return back the 'finished' string, as noted that works, however if I hit the page in multiple tabs they do not all finish at the same time, instead, lets say tab 1 takes 7 seconds to finish, tab 2 and 3 tak 14, then 21 etc.
Note: The variables we are passing to the worker have no use RN, its just a loop that we run for testing to verify the flow
I am new to node.js and working through the API. In the stream module docs I came across this example of the "unpipe event" (actually a fusion of two examples in the docs).
const fs = require("fs);
const writable = fs.createWriteStream("write.txt");
const readable = fs.createReadStream("read.txt");
readable.pipe(writable);
setTimeout(function(){
console.log("Stop writing to file.txt");
readable.unpipe(writable);
console.log("Manually close the file stream");
writable.end();
}, 0);
writable.on("unpipe", function(src){
console.log("Something has stopped piping into the writer");
});
I can't understand the following console.log order:
"Stop writing to file.txt"
"Something has stopped piping into the writer"
"Manually close the file stream"
Given the setTimeout callback is running - which is the first phase of the event loop as I understand - how on earth does the callback for the "unpipe" event start to run before the setTimeout callback has finished.
Originally I had the setTimeout firing after a time above zero seconds, however I was finding that the unpipe call back was always called first. I reasoned that my computer was reading the file always first before the setTimeout was ready. (Although I can't see any mention in the docs about the completion of the write to the file eliciting the "unpipe" event, but this makes sense I suppose). However I can't for the life of me reason how the above program flow is occurring. Thanks in advance for any help.
As specified by the node.js documentation:
The EventEmitter calls all listeners synchronously in the order in which they were registered.
That is, when .emit is called, it synchronously runs through all listeners for the emitted event and calls them.
Note that if necessary you can wrap your callback code in process.nextTick to ensure that it will always run asynchronously, but in your case it's likely that's unnecessary.
Also the source of the call to .emit (the emission of the event) will often be asynchronous.
I'm writing a script to batch process some text documents and insert them into a mysql database. I'm trying to use the async library because using a standard while loop blocks the event queue and prevents the insert queries from getting run until all are generated. Since that may take 10 minutes or more, I get a timeout. So, I am trying to use async to avoid blocking the main thread. However, it's not working as expected. When I run the simplest form of the code below, using node test.js, in the command line, it only executes once, instead of infinitely. It seems like the computer is terminating the node process early since it is non-blocking. This, of course, is not what I want. Why is this, and how can I get it to work correctly?
//this code should run forever, constantly printing "working". However it only runs once.
var async = require('async')
async.whilst(function(){return true},function(){console.log("working")})
The second parameter for whilst() is a function that takes in a callback that needs to be called when the current iteration is "done."
So if you modify the code this way, you'll get what you're expecting:
var async = require('async');
async.whilst(function() {
return true
}, function(cb) {
console.log("working");
cb();
});
Consider:
node -e "setTimeout(function() {console.log('abc'); }, 2000);"
This will actually wait for the timeout to fire before the program exits.
I am basically wondering if this means that node is intended to wait for all timeouts to complete before quitting.
Here is my situation. My client has a node.js server he's gonna run from Windows with a Shortcut icon. If the node app encounters an exceptional condition, it will typically instantly exit, not leaving enough time to see in the console what the error was, and this is bad.
My approach is to wrap the entire program with a try catch, so now it looks like this: try { (function () { ... })(); } catch (e) { console.log("EXCEPTION CAUGHT:", e); }, but of course this will also cause the program to immediately exit.
So at this point I want to leave about 10 seconds for the user to take a peek or screenshot of the exception before it quits.
I figure I should just use blocking sleep() through the npm module, but I discovered in testing that setting a timeout also seems to work. (i.e. why bother with a module if something builtin works?) I guess the significance of this isn't big, but I'm just curious about whether it is specified somewhere that node will actually wait for all timeouts to complete before quitting, so that I can feel safe doing this.
In general, node will wait for all timeouts to fire before quitting normally. Calling process.exit() will exit before the timeouts.
The details are part of libuv, but the documentation makes a vague comment about it:
http://nodejs.org/api/all.html#all_ref
you can call ref() to explicitly request the timer hold the program open
Putting all of the facts together, setTimeout by default is designed to hold the event loop open (so if that's the only thing pending, the program will wait). You can programmatically disable or re-enable the behavior.
Late answer, but a definite yes - Nodejs will wait around for setTimeout to finish - see this documentation. Coincidentally, there is also a way to not wait around for setTimeout, and that is by calling unref on the object returned from setTimeout or setInterval.
To summarize: if you want Nodejs to wait until the timeout has been called, there's nothing you need to do. If you want Nodejs to not wait for a particular timeout, call unref on it.
If node didn't wait for all setTimeout or setInterval calls to complete, you wouldn't be able to use them in simple scripts.
Once you tell node to listen for an event, as with the setTimeout or some async I/O call, the event loop will loop until it is told to exit.
Rather than wrap everything in a try/catch you can bind an event listener to process just as the example in the docs:
process.on('uncaughtException', function(err) {
console.log('Caught exception: ' + err);
});
setTimeout(function() {
console.log('This will still run.');
}, 500);
// Intentionally cause an exception, but don't catch it.
nonexistentFunc();
console.log('This will not run.');
In the uncaughtException event, you can then add a setTimeout to exit after 10 seconds:
process.on('uncaughtException', function(err) {
console.log('Caught exception: ' + err);
setTimeout(function(){ process.exit(1); }, 10000);
});
If this exception is something you can recover from, you may want to look at domains: http://nodejs.org/api/domain.html
edit:
There may actually be another issue at hand: your client application doesn't do enough (or any?) logging. You can use log4js-node to write to a temp file or some application-specific location.
Easy way Solution:
Make a batch (.bat) file that starts nodejs
make a shortcut out of it
Why this is best. This way you client would run nodejs in command line. And even if nodejs program returns nothing would happen to command line.
Making bat file:
Make a text file
put START cmd.exe /k "node abc.js"
Save it
Rename It to abc.bat
make a shortcut or whatever.
Opening it will Open CommandLine and run nodejs file.
using settimeout for this is a bad idea.
The odd ones out are when you call process.exit() or there's an uncaught exception, as pointed out by Jim Schubert. Other than that, node will wait for the timeout to complete.
Node does remember timers, but only if it can keep track of them. At least that is my experience.
If you use setTimeout in an arrow / anonymous function I would recommend to keep track of your timers in an array, like:
=> {
timers.push(setTimeout(doThisLater, 2000));
}
and make sure let timers = []; isn't set in a method that will vanish, so i.e. globally.
Following is my nodejs code
var emitter = require('events'),
eventEmitter = new emitter.EventEmitter();
eventEmitter.on('data', function (result) { console.log('Im From Data'); });
eventEmitter.on('error', function (result) { console.log('Im Error'); });
require('http').createServer(function (req, res) {
res.end('Response');
var start = new Date().getTime();
eventEmitter.emit('data', true);
eventEmitter.emit('error', false);
while(new Date().getTime() - start < 5000) {
//Let me sleep
}
process.nextTick(function () {
console.log('This is event loop');
});
}).listen(8090);
Nodejs is single threaded and it runs in an eventloop and the same thread serves the events.
So, in the above code on a request to my localhost:8090 node thread should be kept busy serving the request [there is a sleep for 5s].
At the same time there are two events being emitted by eventEmitter. So, both these events must be queued in the eventloop for processing once the request is served.
But that is not happening, I can see the events being served synchronously as they are emitted.
Is that expected? I understand that if it works as I expect then there would be no use of extending events module. But how are the events emitted by eventEmitter handled?
Only things that require asynchronous processing are pushed into the event loop. The standard event emitter in node will dispatch an event immediately. Only code using things like process.nextTick, setTimeout, setInterval, or code explicitly adding to it in C++ affect the event loop, like node's libraries.
For example, when you use node's fs library for something like createReadStream, it returns a stream, but opens the file in the background. When it is open, node adds to the event loop and when the function in the loop gets called, it will trigger the 'open' event on the stream object. Then, node will load blocks from the file in the background, and add to the event loop to trigger data events on the stream.
If you wanted those events to be emitted after 5 seconds, you'd want to use setTimeout or put the emit calls after your busy loop.
I'd also like to be clear, you should never have a busy loop like that in Node code. I can't tell if you were just doing it to test the event loop, or if it is part of some real code. If you need more info, please you expand on the functionality you are looking to achieve.