I'm using the async module to help with creating an object that depends on other objects that are created asynchronously. The asynchronously created objects have validations that are ran against them that ultimately return a promise. The problem I'm having is that the "final" async callback seems to never be called and I cannot figure out why.
Example code:
function construct(self, opts) {
async.parallel({
venue: function(callback) {
new Venue(opts.venue).then(function(venue) {
callback(null, venue);
}).catch(function(err) {
callback(err);
});
},
artists: function(callback) {
new Artist(opts.artist).then(function(artist) {
callback(null, artist);
}).catch(function(err) {
callback(err);
});
},
function(err, res) {
console.log(res); // THIS IS NEVER CALLED.
}
}
It looks like the issue is that your callback function is inside the object that you're passing to async.parallel instead of as its own argument.
You've got
async.parallel({
venue: func,
artists: func,
callback
})
Instead of
async.parallel({
venue: func,
artists: func,
}, callback
);
BUT, it's really worth questioning what you gain from mixing promises and async like this; they're essentially designed to accomplish the same task; it'd probably be worth picking one or the other. Something like Promise.all or Promise.join is an alternative to async.parallel.
Pure promise version of this would look like: (this is assuming bluebird promises; but other libraries would be similar, if not identical)
Promise.join(new Venue(opts.venue), new Artists(opts.artists))
.then(function(venues, artists) {
//
});
That's a lot cleaner, I think.
Related
I need the callback function of find from Node.js mongodb 3.1.6 to be triggered before the return statement of an async function, however the return statement is called before the callback function even-though there is a wait in front.
async function(myId) {
const myObject = MyObject()
await collection.find({where: {id: myId}}, async (err, results) => {
if (err) {
logger.error('error');
}
myObject.add(results);
});
return myObject
}
I have seen some examples where instead of find(query, callback) the pattern find(query).toArray() was used. But this doesn't run at all in my case. We use Node.js mongodb 3.1.6 with loopback-connector-mongodb maybe this is related to the problem.
If mongo doesn't provide a promise-answering function, then promisify this one yourself. Neither that promise-creating wrapper nor the anonymous callback it uses should be declared async, but the caller should....
function findById(collection, myId) {
return new Promise((resolve, reject) => {
collection.find({where: {id: myId}}, (err, results) => {
(err)? reject(err): resolve(results);
});
});
}
// now callers can use the async await pattern...
async someFunction() {
try {
let myId = // ...
let collection = // ...
let results = await findById(collection, myId);
// do something with results
} catch (err) {
// error
}
}
The key idea is that collection.find with the callback isn't eligible for await, because it doesn't return a promise. The anonymous callback function you pass to it isn't an async function... it does its work right away, as soon as find calls it back. So we build a promise around mongo, then use the new async/await syntax with that promise.
So I'm new to Node.js and Im just wondering if the way I have my code setup makes sense. Im coming from a Java background so the nested callback structure is new. I have a Node program that runs a bunch of code that I broke down into different methods. The thing is that the methods need to be called in order. My code has this structure right now:
functionOne(data, callback(err) {
functionTwo(data, callback(err) {
functionThree(data, callback(err) {
functionFour(data, callback(err) {
//Code
});
});
});
});
This is very minimalistic, but is this structure ok? With Java, I'd take the return values of all the methods, then just pass them to the next function. From my understanding so far, the Java approach I just mentioned is one of the main things that Node.js was trying to eliminate. But anyway... Does that structure look ok, and is that how its intended to look? Just want to be sure that I'm not making any major errors with Node in general. Thanks!
Your code structure looks fine if you work with callback pattern.
But if you're interested in make your code cleaner and readable you would like to use Promises in your asynchronous function, so instead of pass a callback to your functions you could do something like this :
function asyncFunction (data){
return new Promise(function(resolve, reject){
// Do something with data
// Here you can call reject(error) to throw an error
resolve();
});
}
And instead of nested function callbacks you can call then method of Promise.
asyncFunction(data)
.then(function(){
// Promise resolved
// Something has been done with data
});
With Promises you can also execute async fuctions in parallel :
Promise.all([asyncFunctionA(data), asyncFunctionB(data), asyncFunctionC(data)])
.then(function(){...});
EDIT
If you need to pass values of one function to another, your code should look like this :
asyncFunctionA(data)
.then(function(dataA){
return asyncFunctionB(dataA);
})
.then(function(dataB){
return asyncFunctionC(dataB);
})
.then(function(dataC){
// ...
});
You should try to use promises to avoid your callback hell, so it could be something like these...
const Q = require('q'); // you can do a research for this module.
var myModule = {};
myModule.functionOne = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.functionTwo = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.doAll = (params) => {
myModule.functionOne(params)
.then((outputFunctionOne) => {
// this is called after functionOne ends
return myModule.functionTwo(outputFunctionOne);
})
.then((outputFunctionTwo) => {
// this is called after function 2 ends
if (outputFunctionTwo.success) {
// if everything ok, resolve the promise with the final output
deferred.resolve(outputFunctionTwo);
} else {
// reject the promise with an error message
deferred.reject('error');
}
})
.fail((err) => {
// this is call if the promise is rejected or an exception is thrown
console.log(err); // TODO: Error handling
})
.done();
}
module.exports = myModule;
You can Chain as many promises as you want really easily, that way you get rid of the callback hell. Best part, you can do promises on Javascript or Node.js
Reference Link https://github.com/kriskowal/q
Hope this helps
Most of the other answers give Promise/A as the answer to your callback woes. This is correct, and will work for you. However I'd like to give you another option, if you are willing to drop javascript as your working language.
Introducing Iced Coffee, a branch of the CoffeeScript project.
With Iced Coffee you would write:
await functionOne data, defer err
await functionTwo data, defer err2
await functionThree data, defer err3
//etc
This then compiles to the CoffeeScript:
functionOne data, (err) ->
functionTwo data, (err2) ->
functionThree data, (err3) ->
//etc
Which then compiles to your Javascript.
functionOne(data, callback(err) {
functionTwo(data, callback(err2) {
functionThree(data, callback(err3) {
//etc
});
});
});
So I have sails app with some help service to make it easier to create and get complex models.
one of theses are
getMerits: function(profileId, limit){
return async.waterfall([
function(callback){
Merit.find({employeeProfile: profileId}).then(function(merits){
callback(null, merits);
});
},
function(merits, callback){
async.forEach(merits, function(item, loop_callback){
MeritIndex.findOne({id: item.index}).then(function(meritIndex){
merits[merits.indexOf(item)].index = meritIndex;
loop_callback();
});
}, function(err, results){
callback(null, merits);
});
}
], function(err, results){
return results;
});
}
the trouble is when I try to call this function to get the result(list of merits with their meritindexes inserted.) I cant figure out the correct way to get the results returned from the async waterfall:
async.forEach(profiles, function(item, loop_callback){
MeritService.getMerits(item.id, 5).exec(function(err, merits){
console.log(merits)
profiles[profiles.indexOf(item)].merits = merits;
loop_callback();
});
// MeritService.getMerits(item.id, 5).exec(function(m){
// console.log(m)
// profiles[profiles.indexOf(item)].merits = m;
// loop_callback();
// });
}, function(err){
console.log("PROFILES" + JSON.stringify(profiles))
});
the print of merits here results in undefined. Is there any way to treat async waterfall as a promise and use then instead of exec?
You don't need to use async.waterfall since you already have promises, promises chain already - so adding another library for that logic is redundant. Waterline uses bluebird promises which come with convenience methods already.
Your getMerits can be written as:
getMerits: function(profileId, limit){
var merits = Merit.find({employeeProfile: profileId});
var items = merits.map(function(item) {
return MeritIndex.findOne({id: item.index }).then(function(meritIndex) {
item.index = meritIndex;
});
});
return items.return(merits); // wait for items to be done, and return the merits
}
P.S.
If you use Node 4+ let me know since it gets even simpler.
I'm using the Q library and async library in nodejs.
Here's an example of my code:
async.each(items, cb, function(item) {
saveItem.then(function(doc) {
cb();
});
}, function() {
});
saveItem is a promise. When I run this, I always get cb is undefined, I guess then() doesn't have access. Any ideas how to work around this?
Your issue doesn't lie with promises, it lies with your usage of async.
async.each(items, handler, finalCallback) applies handler to every item of the items array. The handler function is asynchronous, i.e. it is handed a callback, that it must call when it has finished its work. When all handlers are done, the final callback is called.
Here's how you'd fix your current issue:
var handler = function (item, cb) {
saveItem(item)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
);
}
var finalCallback = function (err, results) {
// ...
}
async.each(items, handler, finalCallback);
However, you don't need to use async for this particular piece of code: promises alone fill this job quite nicely, especially with Q.all():
// Create an array of promises
var promises = items.map(saveItem);
// Wait for all promises to be resolved
Q.all(promises)
.then(
function () { // all is well!
cb();
},
function (err) { // something bad happened!
cb(err);
}
)
I'm trying to get my head around when to use process.nextTick. Below I'm using async library to control my code flow and was wondering if I should be calling nextTick in the end callback or not.
async.parallel([
function (callback) {
// do something
callback(data);
},
function (callback) {
// do something
callback(data);
}
],
// callback
function (data) {
process.nextTick(function () {
// do something
});
});
Without knowing exactly what your 'do something' is, I don't see any reason that you'd need to use nextTick in this situation. You use nextTick when you have reason to defer execution of a function to the next iteration of the event loop. You might want to read Understanding process.nextTick for more detail.
Also, note that async's parallel task completion callbacks take arguments err, data, so you should be doing:
async.parallel([
function (callback) {
// do something
callback(null, data);
},
function (callback) {
// do something
callback(null, data);
}
],
// callback
function (err, data) {
if (err) throw err;
// do something
});
The major difference being that the final callback is invoked immediately if one of the parallel tasks returns an error (calls callback with a truthy first argument).
My experience is that for async.parallel and other similar functions to work correctly, that you need to make all tasks definitely asynchronous; I love the async lib and it's my #1 choice over promises etc, but there can be some weird behavior if you don't force tasks to be async.
See here:
https://github.com/caolan/async#common-pitfalls-stackoverflow
so it should be:
async.parallel([
function(cb){
process.nextTick(function(){
doSomePotentiallySyncThing(cb);
});
},
function(cb){
process.nextTick(function(){
doSomePotentiallySyncThing(cb);
});
},
function(cb){
definitelyAsync(cb); // if it's **definitely** async, then we don't need process.nextTick
}
], function complete(err, results){
// we are in the final callback
});
I am 99% sure about this.