I am looking for a different ways to update 2 collections at the same time, in the following case i updating number of cases for a doctor
My Mongo DB document structure:
Doctors Collection:
{
"_id" : ObjectId("5a18a637346826574416a588"),
"doc_fname" : "Bob",
"doc_lname" : "Smith",
"numOfCases" : NumberInt(6),
"__v" : NumberInt(0)
}
CustomerCases Collection:
{
"_id" : ObjectId("5a18c02221344b58585105ea"),
"doctor_id" : ObjectId("5a18a637346826574416a588"),
"cust_fname" : "test",
"cust_lname" : "test",
"case_type" : "Crowns",
"__v" : NumberInt(0)
}
NodeJS Mongoose Code:
var NewCustomerCase = req.body;
CustomerCases.create(NewCustomerCase, function(err,CustomerCase){
if(err){
throw err;
}
else{
var query = {_id:NewCustomerCase.doctor_id};
// if the field doesnt exsit $set will set q new field
var update = {
'$inc': {'numOfCases': 1}
}
// when true return the updated document
var options = {new:true};
Doctors.findOneAndUpdate(query,update,options,function(err,updatedDocter){
if(err){
throw err;
}
});
res.json(CustomerCase);
}
});
This approach works fine, is there a better way to approach this problem, as far as i know its not possible to execute 2 collection at the same time
If the documents you are updating don't depend on each other you can update them both at the same time. For example by making two promises and then running them at the same time with Promise.all():
const customerPromise = CustomerCases.create(NewCustomerCase);
const doctorPromise = Doctors.findOneAndUpdate(query, update, options);
Promise.all([customerPromise, doctorPromise])
.then((result) => {
return res.status(200).json(result);
}).catch((err) => {
return res.status(400).json(err);
});
The promise returned by Promise.all() will be resolved when both the customer and doctor promises are resolved. The documentation is here.
The problem occurs if there is an error. If the customer fails to update the doctor will still increment numOfCases by 1. If the doctor document fails to update the customer case will still be made. So in fact, you do have some dependencies between the documents, and Promise.all() is maybe not such as good solution after all. This is often where transactions come in to play, but transactions in MongoDB is outside the scope of this answer, so I'll just post this link instead.
You already run the DB operations sequentially with callbacks. Here is a more modern way of doing it based on async/await:
try {
const newCustomerCaseFromDb = await CustomerCases.create(NewCustomerCase);
const doctorUpdated = await Doctors.findOneAndUpdate(query, update, options);
return res.status(200).json({ newCustomerCaseFromDb, doctorUpdated });
} catch(err) {
return res.status(400).json(err);
}
A function that uses await needs the async keyword.
The customer case is created first. If it fails it jumps into the catch block. If it succeeds it continues with finding and updating the doctor. Using await makes it look like synchronous code.
These are my thoughts:-
Use promises instead of callback.
Use of promise.all to concatenate your requests and execute them simultaneously.
Related
First of all, I'm sorry that I couldn't come up with a better title for this post. Also, I'm a beginner in nodejs. I have a problem for which I am hoping to find answer. Please help me and thank you so much.
I am trying to create a collection and insert data into it and retrieve the same data that has been inserted in the same code. This piece of code dbo.collection(nameofCollection).insertMany(data) inserts the data into the collection if exists already and if it doesn't exist, it will create the collection and then insert.
This is the code that I've written for this task :
var excel = require('excel4node');
const mongoose = require('mongoose');
const mongodb = require('mongodb')
const mdb = mongodb.MongoClient;
mdb.connect("mongodb://localhost:27017/", async function(err, db) {
if (err) throw err
const nameCollection = "tempCollection"
const dbo = db.db("reports")
const data = [
{
empID : "001ev",
empName : "xyz",
salary : 20000
},
{
empID : "00234",
empName : "abc",
salary : 10000
},
{
empID : "11345",
empName : "pqr",
salary : 15000
}
];
dbo.collection(nameCollection).insertMany(data, function(err, res) {
if (err) throw err;
console.log("Number of documents inserted: " + res.insertedCount);
console.log("first")
})
const records = await dbo.collection(nameCollection).find({}, { projection: { _id: 0} }).toArray()
console.log(records)
console.log("second")
})
Now, the issue that I'm facing with this code is that, when there is no collection in the data base, it is creating one and inserting the data. Up to that it is working fine. But, the thing is, find query is executing first and then insert is executing. And if that collection already exists then it is working fine, it is inserting the data and then find query is executing.
So, to put it short, if there is no collection with the name, nameCollection; find query is executed first, which returns []. Then the collection is created and insertion of data is happening. But, why is this happening, although in my code I wrote to create the collection first, insert data and then retrieve it?
This is because of asynchronous runtime. There is no guarantee that find function will be executed the last.
try adding this:
try {
const res = await dbo.collection(nameCollection).insertMany(data);
console.log("Number of documents inserted: " + res.insertedCount);
console.log("first");
const records = await dbo.collection(nameCollection).find({}, { projection: { _id: 0} }).toArray()
console.log(records)
console.log("second")
} catch (error) {
throw err;
}
In this case you can omit try/catch block since it does not have impact on error handling.
I am using .pull to remove a record from an array in mongo db and it works fine, but a comment I read somewhere on stack overflow (can't find it again to post the link) is bothering me in that it commented that it was bad to use .save instead of using .findByIdAndUpdate or .updateOne
I just wanted to find out if this is accurate or subjective.
This is how I am doing it currently. I check if the product with that id actually exists, and if so I pull that record from the array.
exports.deleteImg = (req, res, next) => {
const productId = req.params.productId;
const imgId = req.params.imgId;
Product.findById(productId)
.then(product => {
if (!product) {
return res.status(500).json({ message: "Product not found" });
} else {
product.images.pull(imgId);
product.save()
.then(response => {
return res.status(200).json( { message: 'Image deleted'} );
})
}
})
.catch(err => {
console.log(err);
});
};
I think what they were saying though was it should rather be done something like this (an example I found after a google)
users.findByIdAndUpdate(userID,
{$pull: {friends: friend}},
{safe: true, upsert: true},
function(err, doc) {
if(err){
console.log(err);
}else{
//do stuff
}
}
);
The main difference is that when you use findById and save, you first get the object from MongoDB and then update whatever you want to and then save. This is ok when you don't need to worry about parallelism or multiple queries to the same object.
findByIdAndUpdate is atomic. When you execute this multiple times, MongoDB will take care of the parallelism for you. Folllowing your example, if two requests are made at the same time on the same object, passing { $pull: { friends: friendId } }, the result will be the expected: only one friend will be pulled from the array.
But let's say you've a counter on the object, like friendsTotal with starting value at 0. And you hit the endpoint that must increase the counter by one twice, for the same object.
If you use findById, then increase and then save, you'd have some problems because you are setting the whole value. So, you first get the object, increase to 1, and update. But the other request did the same. You'll end up with friendsTotal = 1.
With findByIdAndUpdate you could use { $inc: { friendsTotal: 1 } }. So, even if you execute this query twice, on the same time, on the same object, you would end up with friendsTotal = 2, because MongoDB use these update operators to better handle parallelism, data locking and more.
See more about $inc here: https://docs.mongodb.com/manual/reference/operator/update/inc/
I have a mongo 4.2 replica set. I have N processes running concurrently and trying to read a collection. This collection is like a queue. I'd like to read 100 elements and update them in a transaction so other processes won't try to read those.
My code goes:
const collection = client.db("test").collection(TEST_COLLECTION);
const session = client.startSession();
try {
let data = null;
await session.withTransaction(async () => {
console.log("starting transaction")
data = await collection.find({ runId: null }, { _id: 1, limit: 100 }).toArray();
const idList = data.map(item => item._id.toHexString());
await collection.updateMany(
{ runId: { $in: idList } },
{ $set: { runId: runId } },
{ session });
console.log("Successful transaction")
});
data.map(item => {
// process element one by one and update them (no need for transaction here)
})
} catch (e) {
console.error("The transaction was aborted due to an unexpected error: " + e);
} finally {
await session.endSession();
console.log("Closing transaction")
}
this is the code I've got right now. The thing is that find() won't accept options so I can't pass the session. This means it won't be part of the transaction.
the mongodb documentations states that: When using the drivers, each operation in the transaction must be associated with the session (i.e. pass in the session to each operation).
So I'm assuming that this is actually not transactional only the update part which not solves my problem. Is there any way to include both in my transaction? Any ideas on this? Other/better options?
Thanks
EDIT:
So I was staring at my question for 15 minutes when it hit me. If I update first using the transaction. Then querying with the runId even outside of the transaction I can achieve my goal. Am I right? Is it so easy?
EDIT2:
Edit1 was stupid now I can't limit to 100 items. Back to the start.
EDIT3:
I'am using native mongodb nodejs driver.
To use a find in a transaction, pass the session using the session method:
doc = await Customer.findOne({ name: 'Test' }).session(session);
See Transactions in Mongoose
I have a sever connected to a mongodb database. When I add a first level data and then save that, it works.
For example :
// this works fine
router.post('/user/addsomedata', async (req,res)=>{
try {
const user = await User.findOne({email : req.body.email})
user.username = req.body.username
await user.save()
res.send()
} catch(e) {
res.status(404).send(e)
}
})
BUT if I try to save the object with deeper level data, it's not getting saved. I guess the update is not detected and hence the user didn't get replaced.
Example :
router.post('/user/addtask', auth ,async (req,res)=>{
const task = new Task({
name : req.body.name,
timing : new Date(),
state : false,
})
try {
const day = await req.user.days.find((day)=> day.day == req.body.day)
// day is found with no problem
req.user.days[req.user.days.indexOf(day)].tasks.push(task)
// console.log(req.user) returns exactly the expected results
await req.user.save(function(error,res){
console.log(res)
// console.log(res) returns exactly the expected results with the data filled
// and the tasks array is populated
// but on the database there is nothing
})
res.status(201).send(req.user)
} catch(e) {
res.status(400).send(e)
}
})
So I get the tasks array populated on the console even after the save callback but nothing on the db image showing empty tasks array
You're working on the user from the request, while you should first find the user from the DB like in your first example (User.findOne) and then update and save that model.
Use .lean() with your find queries whenever you are about to update the results returned by mongoose. Mongoose by default return instance objects which are immutable by nature. lean() method with find returns normal js objects which can be modified/updated.
eg. of using lean()
const user = await User.findOne({email : req.body.email}).lean();
You can read more about lean here
Hope this helps :)
when I run code
var collection = db.get('categories');
console.log(collection.find().limit(1).sort( { _id : -1 } ));
on nodejs using mongodb I am getting error Object # has no method 'limit' . I am a beginner to node and really stuck on this section of node
here is full code for geting last insert document.
router.post('/addcategory', function(req, res) {
// Set our internal DB variable
var db = req.db;
// Get our form values. These rely on the "name" attributes
var name = req.body.name;
var description = req.body.description;
// Set our collection
var collection = db.get('categories');
// Submit to the DB
collection.insert({
"name" : name,
"description" : description,
}, function (err, doc) {
if (err) {
// If it failed, return error
res.send("There was a problem adding the information to the database.");
}
else {
// And forward to success page
/******************/
console.log(collection.find().limit(1).sort( { _id : -1 } ));
/*************/
}
});
});
The key piece of missing information here was that you are using Monk, not the native MongoDB Node.JS driver. The command you have for find() is how you would use the native driver (with the changes suggested by #BlakesSeven above for asynchronity), but Monk works a little bit differently.
Try this instead:
collection.find({}, { limit : 1, sort : { _id : -1 } }, function (err,res) {
console.log(res);
});
The method is still asynchronous so you still need to invoke itm either as a promise with .then() or a callback. No methods are sychronous and return results in-line.
Also the result returned from the driver is s "Cursor" and not the object(s) you expect. You either iterate the returned cursor or just use .toArray() or similar to convert:
collection.find().limit(1).sort({ "_id": -1 }).toArray().then(function(docs) {
console.log(docs[0]);
});
Or:
collection.find().limit(1).sort({ "_id": -1 }).toArray(function(err,docs) {
console.log(docs[0]);
});
But really the whole premise is not correct. You seem to basically want to return what you just inserted. Event with the correction in your code, the returned document is not necessarily the one you just inserted, but rather the last one inserted into the collection, which could have occurred from another operation or call to this route from another source.
If you want what you inserted back then rather call the .insertOne() method and inspect the result:
collection.insertOne({ "name": name, "description": description },function(err,result) {
if (err) {
res.send("There was an error");
} else {
console.log(result.ops)
}
});
The .insert() method is considered deprecated, but basically returns the same thing. The consideration is that they return a insertWriteOpResult object where the ops property contains the document(s) inserted and their _id value(s),