Using Q/promises vs callbacks - node.js

I'm using the Q library in nodejs and haven't worked too much with promises in the past, but I have semi complex logic that requires lots of nesting and thought Q would be a good solution, however I'm finding that it seems to be almost the same as just "callback hell".
Basically I have say 5 methods, all which require data from the previous or one of the previous. Here's an example:
We start with some binary data that has a sha1 hash generated based on the binary.
var data = {
hash : "XXX"
, binary: ''
}
First we want to see if we already have this, using this method:
findItemByHash(hash)
If we don't have it, we need to save it, using:
saveItem(hash)
Now we need to associate this to a user, but not only the results of the save. There's now a much larger hierarchy that we associate, so we need to get that first, doing:
getItemHierarchy(item_id), we use the item_id returned from our previous saveItem
Now, we can "copy" these results to a user:
saveUserHierarchy(hierarchy)
Now we're done, however, this assumes the item didn't exist yet. So we need to handle a case where the item did exist. This would be:
We need to check if the user may aleady have this:
getUserItemByItemId(item_id) - item_id was returned from findItemByHash
If it exists, we're done.
If it doesn't:
getItemHierarchy(item_id)
Then
saveUserHierarchy(hierarchy)
Ok, so right now we have callbacks that do these checks, which is fine. But we need to handle errors in each case along the way. That's fine too, just adds to the mess. Really, if any part of the flow throws an error or rejects then it can stop and just handle it in a single place.
Now with Q, we could do something like this:
findItemByHash(hash).then(function(res) {
if (!res) {
return saveItem(hash).then(function(item) {
return getItemHierarchy(item.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
})
} else {
return getUserItemByItemId(res.id).then(function(user_item) {
if (user_item) {
return user_item;
}
return getItemHierarchy(res.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
});
}
})
//I think this will only handle the reject for findItemByHash?
.fail(function(err) {
console.log(err);
})
.done();
So, I guess my question is this. Are there better ways to handle this in Q?
Thanks!

One of the reasons why I love promises is how easy it is to handle errors. In your case, if any one of those promises fail, it will be caught at the fail clause you have defined. You can specify more fail clauses if you want to handle them on the spot, but it isn't required.
As a quick example, sometimes I want to handle errors and return something else instead of passing along the error. I'll do something like this:
function awesomeFunction() {
var fooPromise = getFoo().then(function() {
return 'foo';
}).fail(function(reason) {
// handle the error HERE, return the string 'bar'
return 'bar';
});
return fooPromise;
}
awesomeFunction().then(function(result) {
// `result` will either be "foo" or "bar" depending on if the `getFoo()`
// call was successful or not inside of `awesomeFunction()`
})
.fail(function(reason) {
// This will never be called even if the `getFoo()` function fails
// because we've handled it above.
});
Now as for your question on getting out of "return hell" - as long as the next function doesn't require information about the previous one, you can chain .then clauses instead of nesting them:
doThis().then(function(foo) {
return thenThis(foo.id).then(function(bar) {
// `thenThat()` doesn't need to know anything about the variable
// `foo` - it only cares about `bar` meaning we can unnest it.
return thenThat(bar.id);
});
});
// same as the above
doThis().then(function(foo) {
return thenThis(foo.id);
}).then(function(bar) {
return thenThat(bar.id);
});
To reduce it further, make functions that combine duplicate promise combinations and we're left with:
function getItemHierarchyAndSave(item) {
return getItemHierarchy(item.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
}
findItemByHash(hash).then(function(resItem) {
if (!resItem) {
return saveItem(hash).then(function(savedItem) {
return getItemHierarchyAndSave(savedItem);
});
}
return getUserItemByItemId(resItem.id).then(function(userItem) {
return userItem || getItemHierarchyAndSave(resItem);
});
})
.fail(function(err) { console.log(err); })
.done();
Disclaimer: I don't use Q promises, I perfer when promises primarily for the extra goodies it comes with, but the principles are the same.

Related

Node promises: use first/any result (Q library)

I understand using the Q library it's easy to wait on a number of promises to complete, and then work with the list of values corresponding to those promise results:
Q.all([
promise1,
promise2,
.
.
.
promiseN,
]).then(results) {
// results is a list of all the values from 1 to n
});
What happens, however, if I am only interested in the single, fastest-to-complete result? To give a use case: Say I am interested in examining a big list of files, and I'm content as soon as I find ANY file which contains the word "zimbabwe".
I can do it like this:
Q.all(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
return fileContents.contains('zimbabwe') ? fileContents : null;
}));
})).then(function(results) {
var zimbabweFile = results.filter(function(r) { return r !== null; })[0];
});
But I need to finish processing every file even if I've already found "zimbabwe". If I have a 2kb file containing "zimbabwe", and a 30tb file not containing "zimbabwe" (and suppose I'm reading files asynchronously) - that's dumb!
What I want to be able to do is get a value the moment any promise is satisfied:
Q.any(fileNames.map(function(fileName) {
return readFilePromise(fileName).then(function(fileContents) {
if (fileContents.contains('zimbabwe')) return fileContents;
/*
Indicate failure
-Return "null" or "undefined"?
-Throw error?
*/
}));
})).then(function(result) {
// Only one result!
var zimbabweFile = result;
}).fail(function() { /* no "zimbabwe" found */ });
With this approach I won't be waiting on my 30tb file if "zimbabwe" is discovered in my 2kb file early on.
But there is no such thing as Q.any!
My question: How do I get this behaviour?
Important note: This should return without errors even if an error occurs in one of the inner promises.
Note: I know I could hack Q.all by throwing an error when I find the 1st valid value, but I'd prefer to avoid this.
Note: I know that Q.any-like behavior could be incorrect, or inappropriate in many cases. Please trust that I have a valid use-case!
You are mixing two separate issues: racing, and cancelling.
Racing is easy, either using Promise.race, or the equivalent in your favorite promise library. If you prefer, you could write it yourself in about two lines:
function race(promises) {
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve, reject)));
}
That will reject if any promise rejects. If instead you want to skip rejects, and only reject if all promises reject, then
function race(promises) {
let rejected = 0;
return new Promise((resolve, reject) =>
promises.forEach(promise => promise.then(resolve,
() => { if (++rejected === promises.length) reject(); }
);
}
Or, you could use the promise inversion trick with Promise.all, which I won't go into here.
Your real problem is different--you apparently want to "cancel" the other promises when some other one resolves. For that, you will need additional, specialized machinery. The object that represents each segment of processing will need some way to ask it to terminate. Here's some pseudo-code:
class Processor {
promise() { ... }
terminate() { ... }
}
Now you can write your version of race as
function race(processors) {
let rejected = 0;
return new Promise((resolve, reject) =>
processors.forEach(processor => processor.promise().then(
() => {
resolve();
processors.forEach(processor => processor.terminate());
},
() => { if (++rejected === processors.length) reject(); }
);
);
}
There are various proposals to handle promise cancellation which might make this easier when they are implemented in a few years.

Require.js: is it possible for `define` to use node-style `next()` to delay returning a value?

see codes below:
define(["dep1"], function (dep1, next) {
// necessary works here so that `dep2` is ready
// kind of circular dependency.
someNecessaryWork();
require(["dep2"], function (dep2) {
// return value in node-style, using `next()`
next(someValue);
}
// do not return value as normal, no `return someValue` here
}
Is require.js able to do this? For now I'm using functions to achieve this.
define(["dep1", "dep2Fn"], function (dep1, dep2Fn) {
someNecessaryWork();
dep2Fn();
return someValue;
});
but it feels not intuitive.
define does not allow you to set the return value of a module through a callback. What you show in your second snippet is how you do it. You just have to get used to it.

How do I carry some data through Bluebirds .map?

I am using Blubird and Sequelize (which uses Blubird under the covers).
Suppose I have a code similar to:
Feed.findAll()
.map(function (feed) { // <---- this is what I'm interested in below
// do some stuff here
return some_promise_here;
})
.map(function (whatever) {
// What is the best way to access feed here?
})
....
I have found some replies which hinted at possible solutions, but I can't quite put my finger on it.
I have tried with Promise.all(), .spread(), but I never managed to make it work.
Feed.findAll()
.map(function (feed) { // <---- this is what I'm interested in below
// do some stuff here
return some_promise_here.then(function(result){
return { result: result, feed: feed};// return everything you need for the next promise map below.
});
})
.map(function (whatever) {
// here you are dealing with the mapped results from the previous .map
// whatever -> {result: [Object],feed:[Object]}
})
This looks very similar to How do I access previous promise results in a .then() chain?, however you're dealing with a .map call here and seem to want to access the previous result for the same index of the processed array. In that case, not all solutions do apply, and a closure seems to be the simplest solution:
Feed.findAll().map(function (feed) {
// do some stuff here
return some_promise_here.then(function (whatever) {
// access `feed` here
});
})
You can apply explicit pass-through as well, though, like outlined in #bluetoft's answer.

different redirects during different promise fails causes Error: cannot call http.ServerResponse.end()

I have a chain of promises that could fail at different points. Depending on where it fails, I might want it to do different things. In some places I might want to render a page, other places redirect. The problem I'm finding is when it runs through all the fail functions and then has errors with http.ServerResponse.end() being called numerous times.
Example:
Parse.Promise.as(1).then(function() {
if (apples) {
return apples.fetch().fail(function() { res.redirect('/somewhere') } );
} else {
return {};
}
}).then(function() {
// doing other stuff
},function() {
res.redirect('/elsewhere');
}).fail(function() {
res.render('error.ejs');
});
Should I be doing this a different way?
( I start with as(1) just to get into the promise chain since there are two different starting cases of apples and !apples that both need to continue to the next part of the chain, but only one of them could start the chain since {} can't. Not sure if that's the best way to do that either. )
Just like exceptions, you can throw different errors and get to different places.
Parse.Promise.as(1).then(function() {
if (apples) {
return apples.fetch();
} else {
throw new Error("No Apples");
}
}).then(function() {
// doing other stuff
}).then(null,function(e) {
if(e.message !== "No Apples"){ // can also subclass Error
res.render("error.ejs");
} else {
res.redirect("/elsewhere");
}
});
Note, like I said in your different question - what you really want is .then(null,function(){ rather than .fail usually. It's a poor name choice for .fail on their side imo.
Because I like them, here is the synchronous analogy:
try {
if(apples){
var a = apples.fetch(); // might throw too
} else{
throw new Error("No Apples");
}
}catch(e){
if(e.message !== "No Apples"){ // can also subclass Error
res.render("error.ejs");
} else {
res.redirect("/elsewhere");
}
}

Conditionally call async function in Node

I have the following example code - the first part can result in an async call or not - either way it should continue. I cannot put the rest of the code within the async callback since it needs to run when condition is false. So how to do this?
if(condition) {
someAsyncRequest(function(error, result)) {
//do something then continue
}
}
//do this next whether condition is true or not
I assume that putting the code to come after in a function might be the way to go and call that function within the async call above or on an else call if condition is false - but is there an alternative way that doesn't require me breaking it up in functions?
A library I've used in Node quite often is Async (https://github.com/caolan/async). Last I checked this also has support for the browser so you should be able to npm / concat / minify this in your distribution. If you're using this on server-side only you should consider https://github.com/continuationlabs/insync, which is a slightly improved version of Async, with some of the browser support removed.
One of the common patterns I use when using conditional async calls is populate an array with the functions I want to use in order and pass that to async.waterfall.
I've included an example below.
var tasks = [];
if (conditionOne) {
tasks.push(functionOne);
}
if (conditionTwo) {
tasks.push(functionTwo);
}
if (conditionThree) {
tasks.push(functionThree);
}
async.waterfall(tasks, function (err, result) {
// do something with the result.
// if any functions in the task throws an error, this function is
// immediately called with err == <that error>
});
var functionOne = function(callback) {
// do something
// callback(null, some_result);
};
var functionTwo = function(previousResult, callback) {
// do something with previous result if needed
// callback(null, previousResult, some_result);
};
var functionThree = function(previousResult, callback) {
// do something with previous result if needed
// callback(null, some_result);
};
Of course you could use promises instead. In either case I like to avoid nesting callbacks by using async or promises.
Some of the things you can avoid by NOT using nested callbacks are variable collision, hoisting bugs, "marching" to the right > > > >, hard to read code, etc.
Just declare some other function to be run whenever you need it :
var otherFunc = function() {
//do this next whether condition is true or not
}
if(condition) {
someAsyncRequest(function(error, result)) {
//do something then continue
otherFunc();
}
} else {
otherFunc();
}
Just an another way to do it, this is how I abstracted the pattern. There might be some libraries (promises?) that handle the same thing.
function conditional(condition, conditional_fun, callback) {
if(condition)
return conditional_fun(callback);
return callback();
}
And then at the code you can write
conditional(something === undefined,
function(callback) {
fetch_that_something_async(function() {
callback();
});
},
function() {
/// ... This is where your code would continue
});
I would recommend using clojurescript which has an awesome core-async library which makes life super easy when dealing with async calls.
In your case, you would write something like this:
(go
(when condition
(<! (someAsyncRequest)))
(otherCodeToHappenWhetherConditionIsTrueOrNot))
Note the go macro which will cause the body to run asynchronously, and the <! function which will block until the async function will return. Due to the <! function being inside the when condition, it will only block if the condition is true.

Resources