I found 2 ways to execute queries using mongoose find(), but was wondering if they are different:
When should we use:
Model.find({},cb)
And when should we use:
Model.find({}).exec(cb)
Both execute the query and then run the callback.
The main difference is that the first one, returns a Query object while the second returns a Promise which is useful if you need promises.
const query = Model.find({}, cb);
Then you can work with the query variable.
While the promise...
const promise = Model.find({}).exec();
Then you can work with the promise and do things like:
promise.then(cb);
promise.catch((err) => {
console.error(err);
});
But if you do Model.find({}).exec(cb); the callback is also called without using promises...
I hope it helps
The difference is that the first one executes the query and calls your callback. Whereas in the second one where you omit the callback, the query will not be executed. Instead it will return a Query object which can be used to chain methods, specify search terms, and cursor options, etc...
http://mongoosejs.com/docs/2.7.x/docs/query.html
If you don't need to do any kind of chaining, or anything else with the underlying Cursor then just use the first method.
But the second method can be helpful to do things like:
findCars : function(options, callback) {
var query = Model.find({});
if(options.limit) query.limit(options.limit);
if(options.skip) query.limit(options.skip);
if(options.populate) query.populate(options.populate);
return query.exec(callback);
}
Related
When I hover on the keyword 'function' the description says:
"(local function)(this: any, next: (err?: mongoose.CallbackError | undefined) => void): Promise<void>"
So does It return a Promise<void> or a simple <void>? I can't even understand what does this function returns? And to be honest I don't understand really well the concept of Promise<void>...
userSchema.pre('save', async function (next) {
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
return next();
})
This question is really interesting. Your function returns a Promise<void>, which is compatible with the void return type that pre is expecting, but Mongoose is quietly smart enough to know what to do with your Promise so you don't even have to call next at all.
First some background:
void has a special meaning in TypeScript to mean that the return value could be any value; the value is frequently undefined (because that's what a function returns without a return statement) but it doesn't have to be. As in the TypeScript FAQ, this makes it convenient to accept or pass functions that return a value by indicating the return value is unused. If you need to supply a function with return type void, you could pass back a function that returns a string, Promise<void>, Promise<SomeObject>, null, undefined, or anything else.
All async functions return Promises, and this is no exception. A Promise<number> is a Promise that says that its then function will receive a number; a Promise<void> is a Promise that doesn't tell you anything about what its then function receives. The then function will still be called, unless it has an error to catch; you just don't know much about its argument.
In Mongoose's types, pre takes a PreSaveMiddlewareFunction<T> function, which is the type of the function you wrote. It accepts a function called next and returns void: Mongoose claims not to care what you return. Your middleware function is allowed to be asynchronous; when you're done you're expected to call next (with an error object, if you have one), and that call to next also returns void.
Your function passed to pre returns type Promise<void>: The function is async so it absolutely returns a promise, and your return next(); means that the Promise resolves to whatever next returns, which is defined as void. You don't know what next returns and shouldn't care about it. You don't even need to return next(), you just need to call it: It's just a callback so you can tell Mongoose your middleware is done and report any errors.
So your async function returns Promise<void>, but that works with the definition of pre: pre doesn't care what kind of return value your function has (void) as long as you call next to indicate you're done.
But wait! Reporting that your asynchronous function is done and whether or not there were errors is exactly the problem that Promises were designed to solve, and the next callback pattern is exactly the kind of pattern that Promises were designed to replace. If you're returning a Promise, why would you need to call next at all when Mongoose can just watch the promise you return?
In fact, in Mongoose 5.x or later, that's exactly what happens: If the function you pass into pre returns a Promise, then you can use that instead of calling next. You can still call next manually for compatibility's sake, but in your case you could delete return next() and everything would keep working. See the middleware docs:
In mongoose 5.x, instead of calling next() manually, you can use a function that returns a promise. In particular, you can use async/await.
schema.pre('save', function() {
return doStuff().
then(() => doMoreStuff());
});
// Or, in Node.js >= 7.6.0:
schema.pre('save', async function() {
await doStuff();
await doMoreStuff();
});
The docs further explain why return next() is a pattern at all:
If you use next(), the next() call does not stop the rest of the code in your middleware function from executing. Use the early return pattern to prevent the rest of your middleware function from running when you call next().
const schema = new Schema(..);
schema.pre('save', function(next) {
if (foo()) {
console.log('calling next!');
// `return next();` will make sure the rest of this function doesn't run
/*return*/ next();
}
// Unless you comment out the `return` above, 'after next' will print
console.log('after next');
});
In summary, the expected return type of void is compatible with the fact that you're returning a Promise<void>, but it hides the fact that recent versions of Mongoose are smart enough to check whether you're returning a Promise and do the right thing without needing a call to next. They're two different styles that both work.
Long answer short: It return a Promise<void>
Callbacks
To understand why, here are some details.
First one must understand Callbacks in node.js. Callbacks are one of the basic structure/feature of how node.js works.
You could say that node.js is basically an Event-Driven Programming "framework" (most people will frown to the framework word...). That means that you tell node that in the event of a certain thing happening, it should do a certain action/function (callback).
For node to understand us, we normally give the callback function as a parameter to another function that will do the work of "listening to the event" and executing the callback that we give it. So it is not "us" that execute the callback, it is the event listener.
In your case,
userSchema.pre('save', async function (next) {
pre is the function (a method in Mongoose's userSchema), save is the event that one must react to, async function (next) { is the callback or what must be done after the event.
You will note that your callback is returning next(), but next() returns void, which mean that your callback is returning void.
So why is it returning Promise<void>?
The fact is that in your case, your callback is an async function. And every async functions will return a promise. It is an async function because it is awaiting another promise (two promises even) inside of it. They are hidden because of the await
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
Note: The bcrypt methods are very expensive in terms of CPU and time (also a security feature among other things).
It also means that normally in your code
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
you couldn't have available "right away" the hash value for the user.password and, worse, you couldn't even know when it would come. Will your program stop and wait until bcrypt finish its business?
If you have many async functions, your program will be a great favourite for the slowest champion in the Olympics.
What is going on with those promises and how can we not be labelled as a geriatric program?
Promises
Here is a quick/long comment to try to explain the concept of promises.
In "normal" code, each lines of code is executed and "finished" before the next one. Ex: (with cooking)
Combine the Butter and Sugar,
Add Eggs One at a Time, etc.
Or in your code:
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
A promise is a certain code that is executed but not finished before the next line of code. Ex:
while the cake is in the oven (promise),
you prepare the frosting,
but you can't put it until the cake in baked (the "then" action of promises).
Note: Your code is using await so there is no "explicit" then method.
You will have many example of "promises" things in everyday life. you may have heard of asynchronous code = not one after the other, not in sync, ...
Turning on an alarm to wake you in the morning, then you make the promise that you will not ignore it;
putting a reminder on the calendar then you make the promise that you will go to that job interview; etc.
All the while, you continue with your life after making those promises.
In code, a function that returns a promise will have a then method where you tell the computer what to do when when the "alarms goes off".
It is usually written like this
mypromise().then(doThisThingFunction)
const continueWithMyLife = true
In this way the then method is very similar to the callback of node.js. It is just expressed in a different way in the code and is not specific to node (callbacks are also not specific to node...).
One very important difference between them is that callbacks are something that the listener "do" and promises is something that resolves (hopefully) to a returning value.
Async/Await
Nowadays it is common to use async/await. Fortunately/unfortunately it basically hides the asynchronous behaviour. Better flow of reading the code, but also much worse understanding of promises for new programmers.
After a await, there is no then method (Or you could say that the following line of code is the then action). There is no "continuing with your life". There is only "waiting until the alarms goes off", So the next line after the await is essentially the "get out of the bed action".
That is why, in your code, the hash value is available in the next line. Basically in the "old way" to write promises
user.password = hash;
would be inside the then function.
And that is also why it is returning Promise<void>
But still, all these analogies won't really help. The best is to try it in everyday code. There is nothing like experience to understand anything.
I am really struggling here. Admittedly I am no guru especially when it comes to node and asynchronous programming, I am an old C# .net developer so I am comfortable with code but struggling here.
Here's the back story, short and sweet. I have a pg database and I am using the sequelize ORM tools to create a relatively simple CRUD app.
Here's what I want to do.
Make a call to the findAll function on one object.
I need a piece of information from that first call so that I can make a subsequent call.
For instance. Lookup the current user to get their details, grab their ID and now lookup their display preferences.
I know I can run two requests that are not linked using Promise.all, here is an example of this already working.
var delConfig = deliverabiltyConfigs.findAll2(req.signedCookies.tsUser);
var delPack = deliverabilityPackages.findAll2();
Promise.all([delConfig, delPack]).then((results) =>{
res.render('index', { title: 'Deliverability Calculator', UserEmail : req.signedCookies.tsUser, UserName : req.signedCookies.tsUserName, data:results[0], packs:results[1]});
});
Where I am stuck is passing data from one promise to then next and needing them to run asynchronously.
Please help!
There are a few way you can do this. Either use promise chaining or with async & await.
Promise chaining might be the simplest way to do this now, but I would suggest using async await as its easier to read. Since you didn't really provide a sample of what you were trying to do I will make something generic that should hopefully help.
So using promise chaining you would do something like:
pgConnection.findAll().then((data) => {
const foo = data.foo;
pgConnection.findSomething(foo).then((data2) => {
console.log(data2);
});
});
What is happening here is once the promise from findAll() is resolved successfully it will call the .then method and will pass the resulting data there for you to use in your next db query and then I am just printing out the result of the final db query.
This is how you could do it using async & await:
async function getFoo() {
const data = await pgConnection.findAll();
const foo = data.foo;
const data2 = await pgConnection.findSomething(foo);
console.log(data2);
}
The await keyword can only be used inside of an async function so it might not be as simple to change as just using a .then promise chain.
I have complex solution and I just need to run knex synchronously, is it possible?
I have scenario when knex query is run inside Promise.mapSeries for array with unknown number of elements. For each element some knex query is called, including insert query.
So, this insert could affect result for the next element of array.
var descriptionSplitByCommas = desc.split(",");
Promise.mapSeries(descriptionSplitByCommas, function (name) {
// knex.select
// knex.insert if select doesn't return results
});
This was not my initial code, so maybe even Promise.mapSeries should be removed. But I need each descriptionSplitByCommas array elements to be processed syncrhonously.
Otherwise often while processing next description in array I get SQL error, because of duplicate elements inserted for column with unique index. This would not happen if query would be synchronous.
I am using native promises, so I do not have experience with mapSeries, therefore I cannot tell you what exactly is going on at current state.
However running several asynchronous commands in series instead of parallel is quite common. There is one important thing, you have to know - once you create Promise, you do not have control about how and when it will be resolved. So if you create 100 Promises, they all start resolving in parallel.
This is the reason, there is no method for native promises like Promise.series - it is not possible.
What are your options? If you need to "create promise at one place, but run it in another", then factory method is your friend:
const runPromiseLater = () => Promise.resolve(25);
// some code
const myRealPromise = runPromiseLater();
myRealPromise.then( //
Of course, you can create array with these methods, then is question - how to run it in series?
If you can use Node with support for async/await, then for cycle is good enough
async function runInSeries(array) {
for (let i=0;i < array.length; i++){
await array[i]();
// or if you have only instructions in array then you get the value and then call some // await myMethod(array[i])
}
}
If you cant use that, then async library is your friend: https://caolan.github.io/async/docs.html#series
If you need to use the value from previous calls, you can use .waterfall
I've got a problem with redis and nodejs. I have to loop through a list of phone numbers, and check if this number is present in my redis database. Here is my code :
function getContactList(contacts, callback) {
var contactList = {};
for(var i = 0; i < contacts.length; i++) {
var phoneNumber = contacts[i];
if(utils.isValidNumber(phoneNumber)) {
db.client().get(phoneNumber).then(function(reply) {
console.log("before");
contactList[phoneNumber] = reply;
});
}
}
console.log("after");
callback(contactList);
};
The "after" console log appears before the "before" console log, and the callback always return an empty contactList. This is because requests to redis are asynchronous if I understood well. But the thing is I don't know how to make it works.
How can I do ?
You have two main issues.
Your phoneNumber variable will not be what you want it to be. That can be fixed by changing to a .forEach() or .map() iteration of your array because that will create a local function scope for the current variable.
You have create a way to know when all the async operations are done. There are lots of duplicate questions/answers that show how to do that. You probably want to use Promise.all().
I'd suggest this solution that leverages the promises you already have:
function getContactList(contacts) {
var contactList = {};
return Promise.all(contacts.filter(utils.isValidNumber).map(function(phoneNumber) {
return db.client().get(phoneNumber).then(function(reply) {
// build custom object
constactList[phoneNumber] = reply;
});
})).then(function() {
// make contactList be the resolve value
return contactList;
});
}
getContactList.then(function(contactList) {
// use the contactList here
}, funtion(err) {
// process errors here
});
Here's how this works:
Call contacts.filter(utils.isValidNumber) to filter the array to only valid numbers.
Call .map() to iterate through that filtered array
return db.client().get(phoneNumber) from the .map() callback to create an array of promises.
After getting the data for the phone number, add that data to your custom contactList object (this is essentially a side effect of the .map() loop.
Use Promise.all() on the returned array of promises to know when they are all done.
Make the contactList object we built up be the resolve value of the returned promise.
Then, to call it just use the returned promise with .then() to get the final result. No need to add a callback argument when you already have a promise that you can just return.
The simplest solution may be to use MGET with a list of phone numbers and put the callback in the 'then' section.
You could also put the promises in an array and use Promise.all().
At some point you might want your function to return a promise rather than with callback, just to stay consistent.
Consider refactoring your NodeJS code to use Promises.
Bluebird is an excellent choice: http://bluebirdjs.com/docs/working-with-callbacks.html
you put async code into a for loop (sync operations). So, each iteration of the for loop is not waiting for the db.client(...) function to end.
Take a look at this stackoverflow answer, it explains how to make async loops :
Here
I have two mongoose models that control mongo actions for each model. from an index route I am calling a function from one of my models I want to call a function from my other model when the first one finishes.
Assume the following:
ThingOneSchema = new Schema({ _id :String, name :String});
ThingOneSchema.statics.saveUser = function(thing, callback){
...do some stuff
ThingTwoSchema = new Schema({_id :String, property:String});
ThingTwoSchema.statics.doSomething = function(prop, callback){
... do some stuff
In my calls from index I would like to call doSomething from thingTwoSchema and then saveUser from ThingOneSchema. I find Q kinda confusing when trying to apply it in my program. Often examples take the form of doSomething(result).then(function(result))
but I faile to see how q is invoked this way.
If you would like to chain the promises as following
doSomething(res).then(function callback(res) {
//some stuff again
})
Make sure function doSomething return a promise.
function doSomething (res) {
//do stuff..
return Q(some_result);
}
Then, your callback which passed to then will be called with arguments some_result later on.
If your async operations are callback based instead of promise based, make sure you have checked out deferred. It allows you to wrap those async calls with Q promise.
Here is another bunch of examples which captures common problems when dealing with promise.