I am trying to improve the readability of a biggish nodejs app that uses a lot of mongooose. The problem is that with a lot of dependent queries the callbacks get out of hand.
What are the practices of handling this issue?
There are three common solutions for your problem.
First one is async.js lib.
Second one is using Promises. There are more then one implementation of promises in node.js. I know three implementations:
node promise
vow
Q promise
Third one is to use Fibers. There is fibers promise library that do all the tricky work for you.
There was plenty of similar questions before. For example, check this one.
All this libraries do the same thing - they make node.js asynchronous code pretty and readable. So, just choose one that looks simpler for you.
As for me, I prefer async.js lib.
Update: mongoose.js have its own build-in promise - mpromise. You can access it as mongoose.promise. But whenever you call exec() function on a query in mongoose it returns you a promise. I never actually used mongoose.js promises except for the REPL, but you may give it a try.
Related
The background
I'm writing my next server project in node. The previous version was written in language D, which is majorly an synchronized implementation. We all know, that JS uses a lot of async This is good for performance, but hard to maintain (at least for me). Because node and its node modules might use different async approach (callback, promise and await/async), which makes the code looks not in a consistent style. I'm looking for a balanced approach, that not only can create a high quality and performance node application, but fit the common programming style.
The problem
After first round prototyping, I have my first server. When I start the server, it will first read configuration and then load a set of modules. I expect them to be loaded sequentially. But as I mentioned above, node modules might work in async way. I expect module b to be loaded exactly when module a is loaded, I must use Promise pattern. I have to write code in crazy way, although Promise is already a very programmer friendly approach.
No, I go to the other direction. Currently I choose deasync, which is a gyp module for node. It can really blocks the code execution like this.
const deasync = require('deasync');
let done = false;
loadModule1Async((err, result) => {
done = true;
});
deasync.loopWhile(() => !done);
done = false;
loadModule2Async((err, result) => {
done = true;
});
deasync.loopWhile(() => !done);
OR even more magical like this
const deasync = require('deasync');
const loadModule1Sync = deasync(loadModule1Async);
const loadModule2Sync = deasync(loadModule2Async);
try {
loadModule1Sync();
loadModule2Sync();
} catch(ex) {
console.error(ex);
}
IMO, the deasync way is more straightforward to me, but I have two concerns:
It need to be compiled against the target platform, so that the deployment process will be complicated for production.
I checked its issue list. There are some known serious issue, e.g. hanging in some extream cases.
The question
Can the coming new feature async/await really do the same thing as deasync does? If it is never gonna happen due to the JS nature. I might stop thinking about this.
Please share me some lights on the way to node.
I don't know too much about how deasync works, I took a look at it and looks very convenient, but I don't think it's thought to be used through all your code in a server.
little bit performance downgrade
It depends on your case, but it could be more than a little downgrade. You are wrapping all your asynchronous methods in your code, that does not sound good in general. Doing a benchmark could be a good idea.
Other thing:
Because node and its node modules might use different async approach (callback, promise and await/async)
In general modules follow callback style, Promise style, or both of them. Most of the big ones are compatibles with both of them (in general if you don't pass a callback a Promise is returned).
async/await is different, this works on top of Promises, so a module method that returns a Promise can be used with async/await in your code.
So answering your question, I would stick with Promises in the whole code, it's the tendency now, and in general they are easier to deal with, compared to callbacks.
To me async/await is the easiest way, since it keeps calls asynchronous, but the code is written in a synchronous way. Sadly at the moment async/await is only usable in Node.js through a flag, so it's not recommended to be used in production.
This question already has an answer here:
Are there still reasons to use promise libraries like Q or BlueBird now that we have ES6 promises? [closed]
(1 answer)
Closed 6 years ago.
During studying promise in node.js. i have some doubt.
'promise' is already defined in node.js
but usally it seems to use additional promise framework like Q,bluebird,RSVP etc.
Is there any reason?
Is the reason that core node.js function cannot support promise with promise.denodeify function?
I can't say for Q, but Bluebird is a lot faster than native Promises, and provides a bunch of extra features on top of the native promise.
It's the same reason people use lodash despite having [].map() for years now.
Additionally, Bluebird has better browser support than the native Promise implementation.
You normally only use Bluebird on the server, though. The extra features and speed are not worth the extra size of the library the user has to download.
Well, promise is the native JavaScript object. All those libraries are userland implementations. For example if we look at bluebird it has:
A lot of utility functions and helpers that make your life easier.
It has typed .catch which makes sure you don't catch programmer errors by mistake.
It has .some .any .map .filter and so on for working with collections easily.
It has .reflect and synchronous inspection of promises.
It doesn't swallow errors by default when throwing in a then handler.
Built in coroutine (generator) support for flattening asynchronous flow.
It's faster (usually between 4x and 10x) than native promises in different browsers.
It provides more debugging hooks and better stack traces.
It provides warnings against common promise anti-patterns.
It lets you override the scheduler so you can determine how it schedules tasks.
It supports promise cancellation with sound semantics which is proposed for native promises but not yet adopted.
So in a tl;dr; :
It's faster.
It's more debuggable.
It has a richer API.
Now, whether or not you should use it is up to you - there is always overhead in including libraries - I'm biased as a core contributor.
With the introduction of http://es6-features.org/#PromiseUsage in ES6, is https://github.com/caolan/async still relevant? Do async.waterfall and async.series provide any benefit over promises?
Yes and no. The async functions allow to just have it prepared for you rather then you building it yourself and the async still work everywhere. ES6 promises are pretty great. They allow to do a lot and when paired with the proposed ES7 async/await they might remove need for the package all together depending on how its implemented in the future.
Seems like Bluebird overlaps Co in generator/coroutine related functionality. Bluebird is espoused to have exceptional speed-performance, so for discussion sake, (assuming the aforementioned overlap premise is true) if one wanted to substitute Bluebird for Co in Koa (Node.js context), could it be easily be done without diminishing Koa's functionality, and if so how?
(My guess is it can't practically be done since it seems Koa is built over Co and doesn't explicitly expose it, but facades it. Such a substitution it seems would be tantamount to replacing jQuery with something else in Bootstrap)
First of all, bluebird and co are not comparable like that. You mean Bluebird.coroutine vs co (short for coroutine).
Now, the difference between Bluebird.coroutine and co is that co only allows you to yield a certain set of hard-coded types. While Bluebird.coroutine can be configured to support yielding arbitrary types, the documentation for example contains examples how you can add support for yielding thunks and callbacks.
Async generators are so trivial that the only differences there can be between implementations is what types you can yield and how it performs. Not much room to be better or worse.
However bluebird.coroutine is only a fraction of bluebird features.
Generators only solve the problem of making a sequence of actions less verbose. There is a lot of useful functionality for more advanced needs like resource management, concurrency coordination, error handling, cancellation+timeouts and long stack traces which are impossible or extremely painful if you only have async generators powered by thunks/callbacks/minimal promises.
You can make a drop-in replacement for co by configuring all the yield types that co supports and then just using bluebird.coroutine:
var co = require("bluebird").coroutine;
// Configure all yield types you need using co.addYieldHandler
// See documentation for examples
module.exports = co;
However this doesn't really make any sense since very little code actually should run directly in your request handler - the functions that the request handler calls however do. And those functions are not helped by koa (hmm so what is the point of koa again? :D), so they can be bluebird coroutines directly.
esailija said this about Bluebird,
a feature is being added that allows not only yielding callbacks, thunks etc but any arbitrary thing that comes to your mind. Bluebird is also the fastest. So after this version koa should be just using bluebird indeed. See https://github.com/petkaantonov/bluebird/issues/131#issuecomment-36975495
That said, I don't believe him. And, I don't believe a bluebird wrapper would be faster than Co -- if such a thing were possible. Co.js works, and there is no way possible to get Bluebird.js to pass the tests currently. If you're using ES6, ignore Bluebird entirely and use Co.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I know there are many control flow libraries for node.js. Some of them let one chain async functions with callbacks (like async, asyncblock, etc), the others use promise concept (Q, deferred, futures, etc). Given a long running script making a series of actions one after another that may fail at any time, which control flow would you prefer and why? What are the pros and cons?
Pros for callbacks:
Simple to understand and create.
Somewhat more efficient, because fewer objects are created and garbage collected.
Node opted for (error,result) callbacks throughout. I recommend following their argument order for consistency. (As opposed to say (result1, result2, result3, error).)
Pros for promises:
Provides a fluent interface, which can sometimes help to mitigate nested callback hell, as shown here. Code appears to flow linearly by chaining .then(foo).then(bar) calls.
A good promises library will let you run many asynchronous operations in parallel, and continue only when they are all completed. The Deferred library does this seamlessly through map, Q has allResolved, and ES6 Promises offer Promise.all(). (This is also possible with callbacks, e.g. using async.parallel(), but not built in.)
A good promises library will let you specify one error handling function which will be called if any of the queued functions fail. To do this with callbacks requires a little boilerplate: if (err) return callback(err); at the start of every callback.
It would make sense to use callbacks near the bottom of the stack, for code which will be run many times per second. Higher up the stack, promises may be preferable as they are easier to read and understand, and can handle errors more elegantly.
It is worth noting that promises can be built from callbacks at runtime. So you can implement your core code in the minimalist callback form, and still expose a promises version of the library if you want to. (As in Q.nfbind().)
I would be interested to hear other pros/cons.
Bonus tip: Always handle errors! With both methods, if you do not handle the error then it will simply disappear, leaving you in the dark about why your code did not work as expected.
Callbacks should always handle if (err) ... and Promises should always have a .catch() if they do not return.
Even if you expect errors sometimes, and don't need to handle those, not handling unexpected errors means you won't hear about errors from developer mistakes such as typos, if the code is changed in future.
An alternative to .catch() for Promises is to listen for unhandled rejections. Personally I use this to issue a warning that .catch() was missing!
I don't think there are many objective pros and cons. Async is very popular (based on npm packages that depend on it).
I like the control flow libraries (specifically async), because it is easier for me to understand. Promises confuse me, while async is easily understandable. I suspect its just a learning curve thing though, and promises would be more readable if I spent the effort in learning them. But should I expect that of people trying to read my code too?
There is a 3rd type too - Fibers. Fibers does not work on Windows yet, but (IMO) offers the clearest syntax for things that should execute in series.
I've been experimenting with a declarative approach with this library: http://chainsjs.org still more work to do on it, but it gives you the ability to define an "execution map" where you can pretty much completely control the flow of execution from a simple mapping.