I have two mongoose models that control mongo actions for each model. from an index route I am calling a function from one of my models I want to call a function from my other model when the first one finishes.
Assume the following:
ThingOneSchema = new Schema({ _id :String, name :String});
ThingOneSchema.statics.saveUser = function(thing, callback){
...do some stuff
ThingTwoSchema = new Schema({_id :String, property:String});
ThingTwoSchema.statics.doSomething = function(prop, callback){
... do some stuff
In my calls from index I would like to call doSomething from thingTwoSchema and then saveUser from ThingOneSchema. I find Q kinda confusing when trying to apply it in my program. Often examples take the form of doSomething(result).then(function(result))
but I faile to see how q is invoked this way.
If you would like to chain the promises as following
doSomething(res).then(function callback(res) {
//some stuff again
})
Make sure function doSomething return a promise.
function doSomething (res) {
//do stuff..
return Q(some_result);
}
Then, your callback which passed to then will be called with arguments some_result later on.
If your async operations are callback based instead of promise based, make sure you have checked out deferred. It allows you to wrap those async calls with Q promise.
Here is another bunch of examples which captures common problems when dealing with promise.
Related
I'm struggling with callbacks in Node.js. I simply want playerNumber to be set to the number of players in my collection of Players. The console.log works, but I can't get the variable out of the function and into the playerNumber variable.
And if there's a simpler way get this value for use in the rest of my backend code, I'm all ears. I'm clearly new at Node.js, but the code always seems more involved than I'm expecting.
Thanks in advance!
var playerNumber = function countPlayers(callback){
Player.count(function(err, numOfDocs) {
console.log('I have '+numOfDocs+' documents in my collection');
callback(err, numOfDocs);
});
}
It's probably async, and it's a typical first-timer experience to want to "get back to normal" on the call chain on the way back from async call. This can't be done, but it's not so bad to live with it. Here's how...
Step 1: Promises are better than callbacks. I'll leave the long story
to others.
Step 2: Callbacks can be made into promises
In the OP case...
// The promise constructor takes a function that has two functions as params
// one to call on success, and one to call on error. Instead of a callback
// call the 'resolve' param with the data and the 'reject' param with any error
// mark the function 'async' so callers know it can be 'await'-ed
const playerNumber = async function countPlayers() {
return new Promise((resolve, reject) => {
Player.count(function(err, numOfDocs) {
err ? reject(err) : resolve(numOfDocs);
});
});
}
Step 3: Yes, the callers must deal with this, and the callers of the callers, and so on. It's not so bad.
In the OP case (in the most modern syntax)...
// this has to be async because it 'awaits' the first function
// think of await as stopping serial execution until the async function finishes
// (it's not that at all, but that's an okay starting simplification)
async function printPlayerCount() {
const count = await playerNumber();
console.log(count);
}
// as long as we're calling something async (something that must be awaited)
// we mark the function as async
async function printPlayerCountAndPrintSomethingElse() {
await printPlayerCount();
console.log('do something else');
}
Step 4: Enjoy it, and do some further study. It's actually great that we can do such a complex thing so simply. Here's good reading to start with: MDN on Promises.
When I hover on the keyword 'function' the description says:
"(local function)(this: any, next: (err?: mongoose.CallbackError | undefined) => void): Promise<void>"
So does It return a Promise<void> or a simple <void>? I can't even understand what does this function returns? And to be honest I don't understand really well the concept of Promise<void>...
userSchema.pre('save', async function (next) {
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
return next();
})
This question is really interesting. Your function returns a Promise<void>, which is compatible with the void return type that pre is expecting, but Mongoose is quietly smart enough to know what to do with your Promise so you don't even have to call next at all.
First some background:
void has a special meaning in TypeScript to mean that the return value could be any value; the value is frequently undefined (because that's what a function returns without a return statement) but it doesn't have to be. As in the TypeScript FAQ, this makes it convenient to accept or pass functions that return a value by indicating the return value is unused. If you need to supply a function with return type void, you could pass back a function that returns a string, Promise<void>, Promise<SomeObject>, null, undefined, or anything else.
All async functions return Promises, and this is no exception. A Promise<number> is a Promise that says that its then function will receive a number; a Promise<void> is a Promise that doesn't tell you anything about what its then function receives. The then function will still be called, unless it has an error to catch; you just don't know much about its argument.
In Mongoose's types, pre takes a PreSaveMiddlewareFunction<T> function, which is the type of the function you wrote. It accepts a function called next and returns void: Mongoose claims not to care what you return. Your middleware function is allowed to be asynchronous; when you're done you're expected to call next (with an error object, if you have one), and that call to next also returns void.
Your function passed to pre returns type Promise<void>: The function is async so it absolutely returns a promise, and your return next(); means that the Promise resolves to whatever next returns, which is defined as void. You don't know what next returns and shouldn't care about it. You don't even need to return next(), you just need to call it: It's just a callback so you can tell Mongoose your middleware is done and report any errors.
So your async function returns Promise<void>, but that works with the definition of pre: pre doesn't care what kind of return value your function has (void) as long as you call next to indicate you're done.
But wait! Reporting that your asynchronous function is done and whether or not there were errors is exactly the problem that Promises were designed to solve, and the next callback pattern is exactly the kind of pattern that Promises were designed to replace. If you're returning a Promise, why would you need to call next at all when Mongoose can just watch the promise you return?
In fact, in Mongoose 5.x or later, that's exactly what happens: If the function you pass into pre returns a Promise, then you can use that instead of calling next. You can still call next manually for compatibility's sake, but in your case you could delete return next() and everything would keep working. See the middleware docs:
In mongoose 5.x, instead of calling next() manually, you can use a function that returns a promise. In particular, you can use async/await.
schema.pre('save', function() {
return doStuff().
then(() => doMoreStuff());
});
// Or, in Node.js >= 7.6.0:
schema.pre('save', async function() {
await doStuff();
await doMoreStuff();
});
The docs further explain why return next() is a pattern at all:
If you use next(), the next() call does not stop the rest of the code in your middleware function from executing. Use the early return pattern to prevent the rest of your middleware function from running when you call next().
const schema = new Schema(..);
schema.pre('save', function(next) {
if (foo()) {
console.log('calling next!');
// `return next();` will make sure the rest of this function doesn't run
/*return*/ next();
}
// Unless you comment out the `return` above, 'after next' will print
console.log('after next');
});
In summary, the expected return type of void is compatible with the fact that you're returning a Promise<void>, but it hides the fact that recent versions of Mongoose are smart enough to check whether you're returning a Promise and do the right thing without needing a call to next. They're two different styles that both work.
Long answer short: It return a Promise<void>
Callbacks
To understand why, here are some details.
First one must understand Callbacks in node.js. Callbacks are one of the basic structure/feature of how node.js works.
You could say that node.js is basically an Event-Driven Programming "framework" (most people will frown to the framework word...). That means that you tell node that in the event of a certain thing happening, it should do a certain action/function (callback).
For node to understand us, we normally give the callback function as a parameter to another function that will do the work of "listening to the event" and executing the callback that we give it. So it is not "us" that execute the callback, it is the event listener.
In your case,
userSchema.pre('save', async function (next) {
pre is the function (a method in Mongoose's userSchema), save is the event that one must react to, async function (next) { is the callback or what must be done after the event.
You will note that your callback is returning next(), but next() returns void, which mean that your callback is returning void.
So why is it returning Promise<void>?
The fact is that in your case, your callback is an async function. And every async functions will return a promise. It is an async function because it is awaiting another promise (two promises even) inside of it. They are hidden because of the await
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
Note: The bcrypt methods are very expensive in terms of CPU and time (also a security feature among other things).
It also means that normally in your code
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
you couldn't have available "right away" the hash value for the user.password and, worse, you couldn't even know when it would come. Will your program stop and wait until bcrypt finish its business?
If you have many async functions, your program will be a great favourite for the slowest champion in the Olympics.
What is going on with those promises and how can we not be labelled as a geriatric program?
Promises
Here is a quick/long comment to try to explain the concept of promises.
In "normal" code, each lines of code is executed and "finished" before the next one. Ex: (with cooking)
Combine the Butter and Sugar,
Add Eggs One at a Time, etc.
Or in your code:
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
A promise is a certain code that is executed but not finished before the next line of code. Ex:
while the cake is in the oven (promise),
you prepare the frosting,
but you can't put it until the cake in baked (the "then" action of promises).
Note: Your code is using await so there is no "explicit" then method.
You will have many example of "promises" things in everyday life. you may have heard of asynchronous code = not one after the other, not in sync, ...
Turning on an alarm to wake you in the morning, then you make the promise that you will not ignore it;
putting a reminder on the calendar then you make the promise that you will go to that job interview; etc.
All the while, you continue with your life after making those promises.
In code, a function that returns a promise will have a then method where you tell the computer what to do when when the "alarms goes off".
It is usually written like this
mypromise().then(doThisThingFunction)
const continueWithMyLife = true
In this way the then method is very similar to the callback of node.js. It is just expressed in a different way in the code and is not specific to node (callbacks are also not specific to node...).
One very important difference between them is that callbacks are something that the listener "do" and promises is something that resolves (hopefully) to a returning value.
Async/Await
Nowadays it is common to use async/await. Fortunately/unfortunately it basically hides the asynchronous behaviour. Better flow of reading the code, but also much worse understanding of promises for new programmers.
After a await, there is no then method (Or you could say that the following line of code is the then action). There is no "continuing with your life". There is only "waiting until the alarms goes off", So the next line after the await is essentially the "get out of the bed action".
That is why, in your code, the hash value is available in the next line. Basically in the "old way" to write promises
user.password = hash;
would be inside the then function.
And that is also why it is returning Promise<void>
But still, all these analogies won't really help. The best is to try it in everyday code. There is nothing like experience to understand anything.
I've got a problem with redis and nodejs. I have to loop through a list of phone numbers, and check if this number is present in my redis database. Here is my code :
function getContactList(contacts, callback) {
var contactList = {};
for(var i = 0; i < contacts.length; i++) {
var phoneNumber = contacts[i];
if(utils.isValidNumber(phoneNumber)) {
db.client().get(phoneNumber).then(function(reply) {
console.log("before");
contactList[phoneNumber] = reply;
});
}
}
console.log("after");
callback(contactList);
};
The "after" console log appears before the "before" console log, and the callback always return an empty contactList. This is because requests to redis are asynchronous if I understood well. But the thing is I don't know how to make it works.
How can I do ?
You have two main issues.
Your phoneNumber variable will not be what you want it to be. That can be fixed by changing to a .forEach() or .map() iteration of your array because that will create a local function scope for the current variable.
You have create a way to know when all the async operations are done. There are lots of duplicate questions/answers that show how to do that. You probably want to use Promise.all().
I'd suggest this solution that leverages the promises you already have:
function getContactList(contacts) {
var contactList = {};
return Promise.all(contacts.filter(utils.isValidNumber).map(function(phoneNumber) {
return db.client().get(phoneNumber).then(function(reply) {
// build custom object
constactList[phoneNumber] = reply;
});
})).then(function() {
// make contactList be the resolve value
return contactList;
});
}
getContactList.then(function(contactList) {
// use the contactList here
}, funtion(err) {
// process errors here
});
Here's how this works:
Call contacts.filter(utils.isValidNumber) to filter the array to only valid numbers.
Call .map() to iterate through that filtered array
return db.client().get(phoneNumber) from the .map() callback to create an array of promises.
After getting the data for the phone number, add that data to your custom contactList object (this is essentially a side effect of the .map() loop.
Use Promise.all() on the returned array of promises to know when they are all done.
Make the contactList object we built up be the resolve value of the returned promise.
Then, to call it just use the returned promise with .then() to get the final result. No need to add a callback argument when you already have a promise that you can just return.
The simplest solution may be to use MGET with a list of phone numbers and put the callback in the 'then' section.
You could also put the promises in an array and use Promise.all().
At some point you might want your function to return a promise rather than with callback, just to stay consistent.
Consider refactoring your NodeJS code to use Promises.
Bluebird is an excellent choice: http://bluebirdjs.com/docs/working-with-callbacks.html
you put async code into a for loop (sync operations). So, each iteration of the for loop is not waiting for the db.client(...) function to end.
Take a look at this stackoverflow answer, it explains how to make async loops :
Here
I found 2 ways to execute queries using mongoose find(), but was wondering if they are different:
When should we use:
Model.find({},cb)
And when should we use:
Model.find({}).exec(cb)
Both execute the query and then run the callback.
The main difference is that the first one, returns a Query object while the second returns a Promise which is useful if you need promises.
const query = Model.find({}, cb);
Then you can work with the query variable.
While the promise...
const promise = Model.find({}).exec();
Then you can work with the promise and do things like:
promise.then(cb);
promise.catch((err) => {
console.error(err);
});
But if you do Model.find({}).exec(cb); the callback is also called without using promises...
I hope it helps
The difference is that the first one executes the query and calls your callback. Whereas in the second one where you omit the callback, the query will not be executed. Instead it will return a Query object which can be used to chain methods, specify search terms, and cursor options, etc...
http://mongoosejs.com/docs/2.7.x/docs/query.html
If you don't need to do any kind of chaining, or anything else with the underlying Cursor then just use the first method.
But the second method can be helpful to do things like:
findCars : function(options, callback) {
var query = Model.find({});
if(options.limit) query.limit(options.limit);
if(options.skip) query.limit(options.skip);
if(options.populate) query.populate(options.populate);
return query.exec(callback);
}
I have task to search meaning of certain words via lookup on a database for which i am making asynchronous calls to the database, every request would look for sat n number of terms.
The issue i have is I want to call another call back, say, grand_callback, the goal of this callback is to aggregate data from all other callbacks and process the next set of codes after aggregating all the data.
Is there a way I can implement the same..
Some details:
terms = [........] // 1000 terms
grand_callback = () ->
#called with aggreagted data.
getbucket_data = (bucket ,callback) ->
#some treatment over terms
callback null , data
some_func = (term) ->
bucket.push term
if bucket.length is 15{
getbucket_data bucket , (err, data)->
#i need to aggregate this data
}
_.map terms , some_func
You can, of course, just keep track of which callbacks are done manually, but this can be a pain and pretty error prone if you have to do it a lot. Perhaps you could benefit from one of these solutions:
1. Use an async library
I personally like using async by caolan. It has a bunch of functions that can be useful for managing asynchronous operations; the one you're looking for is probably parallel (docs):
parallel(tasks, [callback])
Run the tasks array of functions in parallel, without waiting until the previous
function has completed. If any of the functions pass an error to its
callback, the main callback is immediately called with the value of the error.
Once the tasks have completed, the results are passed to the final callback as an
array.
So you would do something like:
async.parallel([
asyncFunctionOne,
asyncFunctionTwo,
asyncFunctionThree
], function(error, results) {
// ...
});
2. Use promises
Promises are a nice abstraction on top of asynchronous operations. A promise represents a value that either exists now or will exist in the future; you don't have to care which. You can wait for multiple promises to complete by creating a new promise that contains them all. One of the most popular promise libraries for Node is Q by kriskowal.
// creating a promise manually
var deferred1 = Q.defer();
var promise1 = deferred1.promise;
asyncFunction1(function(err, value) {
if (err) deferred1.reject(err);
else deferred1.resolve(value);
});
// wrapping a Node-style function to create promises
var promise2 = Q.nfcall(asyncFunction2, arg1, arg2);
// Create a promise that waits for all other promises
var grandPromise = Q.all([promise1, promise2]);
grandPromise.then(Q.spread(function(promise1result, promise2result) {
// ...
}));