I have a test that tests an external module which updates the database asynchronously and then immediately I use the (updated) values for the next call. This is done by chaining promises like below.
var expectedCateg = 'expected';
ExternalModule.loadUsers(2)
.then(function firstCallback(){
return ExternalModule.updateDBAsync(db, expectedCateg);
})
.then(function secondCallback(updatedRecord){
return ExternalModule.readDBAsync(db, categ);
})
.then(function thirdCallback(resultCateg){
if (resultCateg == expectedCateg) {
console.log("hurray")
}
})
The ExternalModule that updates the DB async has a few calls like this:
exports.updateDBAsync(db, expectedCateg) {
return Db.getMyDb()
.updateAsync(
{ $and: [{"keyA": keyA}, {"keyB": keyB}],
"$isolated": true,
},
{ $set: {"categ": expectedCateg}},
{multi: false})
.then(function success (results) {
return results
})
}
This should log "hurray" but it does not, however, if I execute it and then I look in the database manually, using robomongo, for instance, it is updated. This means that when secondCallback is called, the database update operation has not finished.
How is this possible? Isn't the point of promises to resolve only when the operation is finished? How can we ensure that control is passed on only after the database update is actually finished?
Related
I have a basic CRUD application using html forms, nodejs/express and mongodb. I have been learning about synchronous vs asynchronous code via callbacks, promises, and async/await and to my understanding for a crud application you would want the operations to be asynchronous so multiple users can do the operations at the same time. I am trying to implement aync/await with my express crud operations and am not sure if they are executing synchronously or asynchronously.
Here is my update function, which allows a user to type in the _id of the blog they want to change, then type in a new title and new body for the blog and submit it. In its current state, to my knowledge it is executing synchronously:
app.post('/update', (req, res) => {
const oldValue = { _id: new mongodb.ObjectId(String(req.body.previousValue)) }
const newValues = { $set: { blogTitle: req.body.newValue, blogBody: req.body.newValue2 } }
db.collection("miscData").updateOne(oldValue, newValues, function (err, result) {
if (err) throw err;
console.log("1 document updated");
res.redirect('/')
});
})
The way in which I was going to change this to asynchronous was this way:
app.post('/update', async (req, res) => {
const oldValue = { _id: new mongodb.ObjectId(String(req.body.previousValue)) }
const newValues = { $set: { blogTitle: req.body.newValue, blogBody: req.body.newValue2 } }
await db.collection("miscData").updateOne(oldValue, newValues, function (err, result) {
if (err) throw err;
console.log("1 document updated");
res.redirect('/')
});
})
Both blocks of code work, however I am not sure if the second block of code is doing what I am intending it to do, which is allow a user to update a blog without blocking the call stack, or if the second block of code would only make sense if I was running more functions after the await. Does this achieve the intended purpose, if not how could/should I do that?
db.collection(...).updateOne is always asynchronous, so you need not worry that a long-running database operation might block your application. There are two ways how you can obtain the asynchronous result:
With a callback function
db.collection(...).updateOne(oldValues, newValues, function(err, result) {...});
console.log("This happens synchronously");
The callback function with the two parameters (err, result) will be called asynchronously, after the database operation has completed (and after the console.log). Either err contains a database error message or result contains the database result.
With promises
try {
var result = await db.collection(...).updateOne(oldValues, newValues);
// Do something with result
} catch(err) {
// Do something with err
}
console.log("This happens asynchronously");
The updateOne function without a callback function as third parameter returns a promise that must be awaited. The statements that do something with result will be executed asynchronously, after the database operation has successfully completed. If a database error occurs, the statements in the catch block are executed instead. In either case (success or error), the console.log is only executed afterwards.
(If updateOne does not have a two-parameter version, you can write
var result = await util.promisify(db.collection(...).updateOne)(oldValues, newValues);
using util.promisify.)
Your second code snippet contains a mixture of both ways (third parameter plus await), which does not make sense.
So i am novice at Node.js, and have been checking out its mongoose package. I have a mongodb collection with a document with field name = "Test Entry"
I have tried executing a .findOneAndDelete() method call on a model wrapped within a timeout, wrapped within a function.
One issue that baffles me is why this piece of code
function deleteEntry() {
setTimeout(function () {
Test_Model.findOneAndDelete({
name: "Test Entry",
});
}, 5000);
}
deleteEntry();
does not delete the entry within mongodb
whereas
function deleteEntry() {
setTimeout(async function () {
await Test_Model.findOneAndDelete({
name: "Test Entry",
});
}, 5000);
}
deleteEntry();
this one does.
will the mongoose method not execute unless there is a callback made of some kind. No matter if it's for vain?
There were mentions of something similar in the documentation . Does the actual execution of the command occur only when there is a callback/await/then ?
"Why does not calling this operation NOT trigger the task?" - I think MongoDB driver check if you passed a callback (as argument) or not, If you didn't pass callback, Instead, It return Promise. So no matter what, You operation will get executed.
Also you should handle Error in setTimeout function, Because It don't handle any error or error from any async operation.
Try this example:
let delay = ms => new Promise(ok => setTimeout(ok, ms));
async function deleteEntry() {
await delay(5000);
return Test_Model.findOneAndDelete({ name: "Test Entry" });
}
deleteEntry()
.then(result => console.log(result))
.catch(err => console.log(err));
When I make an API call, I want to update 2 separate collections with MongoDB. The official way of thing this will be using transactions but because this is a prototype for now, I will just use 2 separate function calls to update each collection.
Here is what I am doing:
async.waterfall([
function(callback) {
callback(null);
}, function(callback) {
connectToDatabase(MONGODB_URI)
.then(db => updateDocument1(arguments))
.then(result => {
callback(null);
});
}
, function(callback) {
connectToDatabase(MONGODB_URI)
.then(db => updateDocument2(arguments))
.then(result => {
callback(null);
});
, function(err, result) {
callback(null, null)
}
])
But somehow, it does not execute updateDocument2. Also, updateDocument1 stops in the middle of the process, so none of documents are updated.
Do you know why? And Does anyone know a better way?
async parallel : do function execution parallelly without dependent of
other function
async series : do function execution in series (one by one)
async waterfall : do first function execution then pass output of
first function as input to next function
now in your case : if updating two collection not depending on each other you can use async parallel
I am working on a script where I need to take the data from a mongodb collection, process it with some renames and mapping and put the data to other collection. I am using this script in expressjs with mongodb-native client for nodejs.
Here is my script with all the running functions
const syncCustomerWithCustomerv1 = function(params){
utils.connectDB().then((client) => {
Promise.map(aggregateDocumentsv1(client, params), function(cursor){
Promise.map(getCustomerDatav1(cursor), function(customer){
var hashedMap = makeHashedObjectv1(customer);
makeDatav1(client, customer, hashedMap).then((response)=> {
console.log("success");
}).catch((error) => {
console.log(error);
reject(error);
})
}, {concurrency: 500});
}, {concurrency: 500}).then((reponse) => {
console.log("data inserted");
})
}).catch((error) => {
console.log(error);
});
}
Now in function named syncCustomerWithCustomerv1, I am able to fetch the data from old collection to the new collection, but I do not think any concurrent requests are taken into this. while the above operation is running, I am not able to hit api, so when the operation is running it does not allow other requests to run along.
In the Promise.map(aggregateDocumentsv1), I am taking the list of cursors. each element in listOfCursor array has a cursor which when queried yields 500 records.
I am expecting it gets each cursor and assigns it to the next Promise.map(getCustomerDatav1(cursor)), Now this yields each customer I have in my previous mongo collection, and we can perform the mapping operations on the object we got and then insert the data into the new collection.
If anyone gets the issue and know how I can make it better concurrent so that I can run this script and also my APIs also don't get any downtime.
I don't know about MongoDB, but there are a few problems with your promise code:
For any function (be it Promise.map or then) to be able to wait for the result of an asynchronous callback, that callback must return a promise to be awaited
You are doing 500 concurrent operations, where each of those operations does 500 concurrent operations. That's a total concurrency factor of 250000! You probably want to reduce that a bit.
function syncCustomerWithCustomerv1(params){
utils.connectDB().then(client => {
return Promise.map(aggregateDocumentsv1(client, params), cursor => {
// ^^^^^^
return Promise.map(getCustomerDatav1(cursor), customer => {
// ^^^^^^
var hashedMap = makeHashedObjectv1(customer);
return makeDatav1(client, customer, hashedMap).then(response => {
// ^^^^^^
console.log("success");
}, error => {
console.log(error);
});
}, {concurrency: 500});
}, {concurrency: 500})
}).then(reponse => {
console.log("data inserted");
}, error => {
console.log(error);
});
}
I am new to mongoose.
Here is my scenario:
var childSchema = new Schema({ name: 'string' });
var parentSchema = new Schema({
children: [childSchema]});
var Parent = mongoose.model('Parent', parentSchema);
Say I have created a parent 'p' with children, and I am querying for 'p', using
var query = Parent.find({"_id":"562676a04787a98217d1c81e"});
query.select('children');
query.exec(function(err,person){
if(err){
return console.error(err);
} else {
console.log(person);
}
});
I need to access the person object outside the async function. Any idea on how to do this?
Mongoose's find() method is asynchronous which means you should use a callback that you can wrap the query from the find() method. For example, in your case, you can define a callback as
function getChildrenQuery(parentId, callback){
Parent.find({"_id": parentId}, "children", function(err, docs){
if (err) {
callback(err, null);
} else {
callback(null, docs);
}
});
}
which you can then call like this:
var id = "562676a04787a98217d1c81e";
getChildrenQuery(id, function(err, children) {
if (err) console.log(err);
// do something with children
children.forEach(function(child){
console.log(child.name);
});
});
Another approach you may take is that of promises where the exec() method returns a Promise, so you can do the following:
function getChildrenPromise(parentId){
var promise = Parent.find({_id: parentId}).select("children").exec();
return promise;
}
Then, when you would like to get the data, you should make it async:
var promise = getChildrenPromise("562676a04787a98217d1c81e");
promise.then(function(children){
children.forEach(function(child){
console.log(child.name);
});
}).error(function(error){
console.log(error);
});
you cannot access it outside of the callback (="the async function" you mentioned). That's how node.js works:
your call to the database will take some time - a very long time when you compare it to just execute a simple code statement.
Node.js is non-blocking, so it will not wait for the database to return the result, and it wlll continue immediately by executing the code after your query.exec statement.
so the code you write after the query.exec statement is run BEFORE the database returns the result, it is therefore impossible to use that result there.
BUT... embrace async programming:
just write all the code you need into the "async function"
pass a callback into your function, call it from "the async function" and pass it the query result