Node.js Control Flow and Callbacks - node.js

I've been confused on this for a month and searched everything but could not find an answer.
I want to get control of what runs first in the node.js. I know the way node deals with the code is non-blocking. I have the following example:
setTimeOut(function(){console.log("one second passed");}, 1000);
console.log("HelloWorld");
Here I want to run first one, output "one second passed", and then run the second one. How can I do that? I know setTimeOut is a way to solve this problem but that's not the answer I am looking for. I've tried using callback but not working. I am not sure about if I got the correct understanding of callbacks. Callbacks just mean function parameters to me and I don't think that will help me to solve this problem.
One possible way to solve this problem is to define a function that contains the "error first callback" like the following example:
function print_helloworld_atend(function helloworld(){
console.log("HelloWorld");
}){
setTimeOut(function(){console.log("one second passed");}, 1000);
helloworld();
}
Can I define a function with a callback who will know when the previous tasks are done. In the above function, how to make the callback function: helloworld to run after the "setTimeOut" expression?
If there is a structure that can solve my problem, that's my first choice. I am tired of using setTimeOuts.
I would really appreciate if anyone can help. Thanks for reading

You should be using promises. Bluebird is a great promise library. Faster than native and comes with great features. With promises you can chain together functions, and know that one will not be called until the previous function resolves. No need to set timeouts or delays. Although you can if you'd like. Below is example of a delay. Function B wont run until 6 seconds after A finishes. If you remove .delay(ms) B will run immediately after A finishes.
var Promise = require("bluebird");
console.time('tracked');
console.time('first');
function a (){
console.log('hello');
console.timeEnd('first');
return Promise.resolve();
}
function b (){
console.log('world');
console.timeEnd('tracked');
}
a().delay(6000)
.then(b)
.catch(Promise.TimeoutError, function(e) {
console.log('Something messed up yo', e);
});
This outputs:
→ node test.js
hello
first: 1.278ms
world
tracked: 6009.422ms
Edit: Promises are, in my opinion, the most fun way of control flow in node/javascript. To my knowledge there is not a .delay() or .timeout() in native javascript promises. However, there are Promises in general. You can find their documentation on mozilla's site. I would recommend that you use Bluebird instead though.
Use bluebird instead of native because:
It's faster. Petka Antonov, the creator of bluebird, has a great understanding of the V8 engines two compile steps and has optimized the library around it's many quirks. Native has little to no optimization and it shows when you compare their performance. More information here and here.
It has more features: Nice things like .reflect(), .spread(), .delay(), .timeout(), the list goes on.
You lose nothing by switching: all features in bluebird which also exist in native function in exactly the same way in implementation. If you find yourself in a situation where only native is available to you, you wont have to relearn what you already know.

Just execute everything that you want to execute after you log "one second passed", after you log "one second passed":
setTimeOut(function(){
console.log("one second passed");
console.log("HelloWorld");
}, 1000);

You can use async module to handle the callbacks.
To understand callbacks I'll give you a high level glance:
function: i want to do some i/o work.
nodejs: ok, but you shouldn't be blocking my process as I'm single threaded.
nodejs: pass a callback function, and I will let you know from it when the i/o work is done.
function: passes the callback function
nodejs: i/o work is done, calls the callback function.
function: thanks for the notification, continue processing other work.

Related

JS: Why Promise then() method executes synchronously?

I need to make part of my method's code asynchronous, so it will execute in a non-blocking manner. For this purpose I've tried to create a "dummy" Promise and put the specified code in then block. I have something like this:
public static foo(arg1, arg2) {
const prom = new Promise((resolve, reject) => {
if (notValid(arg1, arg2)) {
reject();
}
resolve(true);
});
prom.then(() => {
...my code using arg1 and arg2...
});
}
However, then block always executes synchronously and blocks whole app, even though each and every JS documentation tells that then always runs asynchronously. I've also tried to replace the Promise with this:
Promise.resolve().then(() => {
...my code using arg1 and arg2...
});
but got same result.
The only way I've managed to make then block work asynchronously is by using setTimeout:
const pro = new Promise(resolve => {
setTimeout(resolve, 1);
});
pro.then(() => {
...my code using arg1 and arg2...
})
What can be the reason behind then block working synchronously? I don't want to proceed with using setTimeout, because it is kind of a "dirty" solution, and I think that there should be a way to make then run asynchronously.
Application is NodeJS with Express, written using Typescript.
I need to make part of my method's code asynchronous, so it will execute in a non-blocking manner. For this purpose I've tried to create a "dummy" Promise and put the specified code in then block.
Promises don't really make things asynchronous in and of themselves. What they do is wrap around something that's already asynchronous, and give a convenient way to tell when that thing is done. If you wrap a promise around something synchronous, your code is mostly still synchronous, just with a few details about when the .then callback executes.
even though each and every JS documentation tells that then always runs asynchronously.
By that, they mean that it waits for the current call stack to finish, and then runs the code in the .then callback. The technical term for what it's doing is a "microtask". This delay is done so that the order of operations of your code is the same whether the promise is already in a resolved state, or if some time needs to pass before it resolves.
But if your promise is already resolved (eg, because it's wrapped around synchronous code), the only thing your .then callback will be waiting for is the currently executing call stack. Once the current call stack synchronously finishes, your microtask runs synchronously to completion. The event loop will not be able to progress until you're done.
The only way I've managed to make then block work asynchronously is by using setTimeout
setTimeout will make things asynchronous*, yes. The code will be delayed until the timer goes off. If you want to wrap this in a promise you can, but the promise is not the part that makes it asynchronous, the setTimeout is.
I don't want to proceed with using setTimeout, because it is kind of a "dirty" solution
Ok, but it's the right tool for the job.
* when i say setTimeout makes it asynchronous, i just mean it delays the execution. This is good enough for many cases, but when your code eventually executes, it will tie up the thread until it's done. So if it takes a really long time you may need to write the code so it just does a portion of the work, and then sets another timeout to resume later

Sleep main thread but do not block callbacks

This code works because system-sleep blocks execution of the main thread but does not block callbacks. However, I am concerned that system-sleep is not 100% portable because it relies on the deasync npm module which relies on C++.
Are there any alternatives to system-sleep?
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100) // without this line the while loop causes problems because it is a spin wait
console.log('sleeping')
}
console.log('If this is displayed then it works!')
PS Ideally, I want a solution that works on Node 4+ but anything is better than nothing.
PPS I know that sleeping is not best practice but I don't care. I'm tired of arguments against sleeping.
Collecting my comments into an answer per your request:
Well, deasync (which sleep() depends on) uses quite a hack. It is a native code node.js add-on that manually runs the event loop from C++ code in order to do what it is doing. Only someone who really knows the internals of node.js (now and in the future) could imagine what the issues are with doing that. What you are asking for is not possible in regular Javascript code without hacking the node.js native code because it's simply counter to the way Javascript was designed to run in node.js.
Understood and thanks. I am trying to write a more reliable deasync (which fails on some platforms) module that doesn't use a hack. Obviously this approach I've given is not the answer. I want it to support Node 4. I'm thinking of using yield / async combined with babel now but I'm not sure that's what I'm after either. I need something that will wait until the callback is callback is resolved and then return the value from the async callback.
All Babel does with async/await is write regular promise.then() code for you. async/await are syntax conveniences. They don't really do anything that you can't write yourself using promises, .then(), .catch() and in some cases Promise.all(). So, yes, if you want to write async/await style code for node 4, then you can use Babel to transpile your code to something that will run on node 4. You can look at the transpiled Babel code when using async/await and you will just find regular promise.then() code.
There is no deasync solution that isn't a hack of the engine because the engine was not designed to do what deasync does.
Javascript in node.js was designed to run one Javascript event at a time and that code runs until it returns control back to the system where the system will then pull the next event from the event queue and run its callback. Your Javascript is single threaded with no pre-emptive interruptions by design.
Without some sort of hack of the JS engine, you can't suspend or sleep one piece of Javascript and then run other events. It simply wasn't designed to do that.
var one = 0;
function delay(){
return new Promise((resolve, reject) => {
setTimeout(function(){
resolve('resolved')
}, 2000);
})
}
while (one == 0) {
one = 1;
async function f1(){
var x = await delay();
if(x == 'resolved'){
x = '';
one = 0;
console.log('resolved');
//all other handlers go here...
//all of the program that you want to be affected by sleep()
f1();
}
}
f1();
}

Promise is neither resolved nor rejected. What could be the reason for this?

I am running into a strange problem. I am using a module to look up the geo location from the IP address. The lookup method by default is sync.
I converted the method to async using bluebird, but its promise never gets resolved or rejected!
Here is the snippet:
var Promise = require('bluebird');
var geoip = Promise.promisifyAll(require('geoip-lite'));
geoip.lookupAsync('52.39.138.72').then((r) => {
console.log(r);
}).catch((err) => {
console.log(err);
})
console.log(geoip.lookup('52.39.138.72').country + '^^^^');
In the above snippet, the last console.log always gets printed but neither of the statement inside then or catch gets executed. What could be the reason for this?
In the above snippet, the last console.log always gets printed but neither of the statement inside then or catch gets executed. What could be the reason for this?
The function you are trying to promisify does not follow the required asynchronous calling convention so promisifying it this way will not work.
For Bluebird's promisify to work properly, the function you promisify must follow the node.js async calling convention. That means the function must take a callback as its last argument and that callback must be called with two arguments err and result when the operation completes. If the function does not follow this convention, then promisifying it will not work.
And, there is really no reason to take a synchronous operation and promisify it either. Promisfying it won't suddenly make its functionality asynchronous.
So, your promise is never getting resolved or rejected because the underlying function doesn't use a callback that gets called with the right calling convention.
So, if geoip.lookup('52.39.138.72') is completely synchronous (as it appears to be) and gets called this way, then the underlying operation isn't asynchronous so there is no reason to even try to promisify it.
If you explain what problem you're actually trying to solve by promisifying it, we could likely offer another way (perhaps in a new question). One thing to keep in mind about stack overflow. If you describe your actual problem and show us the relevant code rather than asking about issues with one attempted solution, then we are much more likely to be able to help you and to offer you the best solution.

Synchronous NodeJs (or other serverside JS) call

We are using Node for developing and 95% of code is Async, working fine.
For some 5% (one small module), which is sync in nature [and depends on other third party software],
and we are looking for
1. "Code to block until call back is finished"
2. At a time only one instance of function1 + its callback should be executed.
PS 1: I do completely agree, Node is for async work, We should avoid that, but this is separate non-realtime process.
PS 2: If not with Node any other Serverside JS framework? Last option is to use other lang like python, but if anything in JS possible, we are ready to give it a shot!
SEQ should solve your problem.
For an overview about sync modules please look at http://nodejsrocks.blogspot.de/2012/05/how-to-avoid-nodejs-spaghetti-code-with.html
Seq()
.seq(function () {
mysql.query("select * from foo",[], function(err,rows,fields) {
this(null, rows);
});
})
.seq(function(mysqlResult) {
console.log("mysql callback returnes:"+mysqlResult);
})
There are lots and lots of options, look at node-async, kaffeine async support, IcedCoffeescript, etc.
I want to make a plug for IcedCoffeeScript since I'm its maintainer. You can get by with solutions like Seq, but in general you'll wind up encoding control flow with function calls. I find that approach difficult to write and maintain. IcedCoffeeScript makes simple sequential operations a breeze:
console.log "hello, just wait a sec"
await setTimeout defer(), 100
console.log "ok, what did you want"
But more important, it handles any combination of async code and standard control flow:
console.log "Let me check..."
if isRunningLate()
console.log "Can't stop now, sorry!"
else
await setTimeout defer(), 1000
console.log "happy to wait, now what did you want?"
resumeWhatIWasDoingBefore()
Also loops work well, here is serial dispatch:
for i in [0...10]
await launchRpc defer res[i]
done()
And here is parallel dispatch:
await
for i in [0...10]
launchRpc defer res[i]
done()
Not only does ICS make sequential chains of async code smoother, it also encourages you to do as much as possible in parallel. If you need to change your code or your concurrency requirements, the changes are minimal, not a complete rewrite (as it would be in standard JS/CS or with some concurrency libraries).

Nodejs convert expensive synchronous tasks to async series

In nodejs I have an expensive function such as:
function expensiveCode(){
a.doExensiveOperation(1);
b.doAnotherExensiveOperation(2);
c.doADiffererentExensiveOperation(3);
d.doExensiveOperation(4);
}
Such that each sub-function call has different parameters and therefore can't be done in a loop. I'd like to throttle this expensive function call so that each sub-call is done on nextTick such as:
function expensiveCode(){
process.nextTick(function(){
a.doExensiveOperation(1);
process.nextTick(function(){
b.doAnotherExensiveOperation(2);
process.nextTick(function(){
c.doADiffererentExensiveOperation(3);
process.nextTick(function(){
d.doExensiveOperation(4);
});
});
});
});
}
That's obviously ugly, and if there are 20 lines of different operations will too hideous to even consider.
I've reviewed a number of libraries like "async.js" but they all appear to be expecting the called functions to be async - to have a callback function on completion. I need a simple way to do it without converting all my code to the 'callback when done' method which seems like overkill.
Sorry to burst your bubble, but async.waterfall or perhaps async.series combined with async.apply is what you want, and yes you'll need to make those operations async. I find it hard to believe you have found 20 different computationally-intense operations, none of which do any IO whatsoever. Check out the bcrypt library for an example of how to offer both synchronous and asynchronous versions of a CPU-intensive call. Converting your code to callback on completion isn't overkill, it's node. That's the rule in node. Either your function does no IO and completes quickly or you make it async with a callback. End of options.

Resources