I am having an issue when performing find queries in nodejs with mongoose. While using mongo shell client, for the same query, i get a response in 2-10ms. But when i run that same query with Mongoose i get a response in 400-550ms. I've tried the mongodb driver also, but got same results.
First of all. Laptop specs are as follows: 2 CPU's and 4GB DDR3 RAM. I'm not sure what i am missing here.
mongoose.connect(config.database, function (err) {
if (err) return console.error(err);
console.log('Connected');
var startTime = Date.now();
UserModel.find().lean().exec()
.then(function (docs) {
console.log('Found ' + docs.length + ' documents');
console.log('Finished after ' + (Date.now() - startTime) + 'ms')
mongoose.connection.close();
})
.catch(function (err) {
console.err(err)
});
});
Result:
Connected
Found 2317 documents
Finished after 456ms
Shell:
"executionStats" : {
"executionSuccess" : true,
"nReturned" : 2317,
"executionTimeMillis" : 9,
"totalKeysExamined" : 0,
"totalDocsExamined" : 2317,
"executionStages" : {
"stage" : "COLLSCAN",
"filter" : {
"$and" : [ ]
},
"nReturned" : 2317,
"executionTimeMillisEstimate" : 10,
Update
When limited to 100 on both, i get the following results:
Shell: 0ms
Nodejs: ~70ms
Update
This is an example of the document structure. I have not added it before since it is not complex.
{
"_id" : ObjectId("56e6cfda713e98cc1191cae0"),
"email" : "doppelganger113#gmail.com",
"password" : "$2a$10$egjew7mBQtKMgVbK6uXV5uD1hO8yQZ1tU5nztmVZQ4nfhLovOKN2q",
"firstName" : "Marko",
"lastName" : "Kovacevicc",
"index" : 20100140,
"gender" : "Male",
"role" : "SuperAdmin",
"isActive" : true,
"__v" : 0,
"avatar" : {
"small" : "http://localhost:3000/uploads/avatar/3a8ed7bb-4f60-4f69-b8fc-372d1db9b044.png",
"medium" : "http://localhost:3000/uploads/avatar/246c6067-403d-4e47-9e65-828cfab66d84.png",
"source" : "http://localhost:3000/uploads/avatar/01dbb51c-c957-4f67-ba36-9e8cc227168c.png"
},
"birthDate" : ISODate("1991-11-28T23:00:00Z")
}
Will test this on a server instead locally just to check if it's to the laptop. But still, those searches appear to take a lot of time, which bugs me. I know they are not the same, but i did not expect this much of a difference.Also i accidentally added mongoose debug, it was tested with and without it.
Update
Shell command: db.users.find({}).explain("executionStats")
Related
I have developed a code where I am able to retrieve all videos from filesystem whose file path is stored in mongo database:
app.get("/myvideos", function(request, result){
database.collection("videos").find({}).toArray(function(error, videos){
result.render("myvideos", {
"isLogin": request.session.user_id ? true : false,
"videos": videos
});
});
});
the above code is retreiving the files properly. However I wish to retrieve files of a particular user. My database schema is:
{
"_id" : ObjectId("60914d02185c2a08add68fa2"),
"user" : {
"_id" : ObjectId("60914c73185c2a08add68fa0"),
"name" : "Saumitra Lele",
"image" : "",
"subscribers" : 0
},
"filePath" : "public/videos/1620135170700-bigbuck.mp4",
"thumbnail" : "public/thumbnails/1620135170700-Photo - Saumitra Lele.jpeg",
"title" : "First video uploaded",
"description" : "First video uploaded by Saumitra Lele",
"tags" : "Saumitra Lele video",
"category" : "Technology",
"createdAt" : 1620135170708,
"minutes" : 0,
"seconds" : 32,
"hours" : 0,
"watch" : 1620135170708,
"views" : 0,
"playlist" : "",
"likers" : [ ],
"dislikers" : [ ],
"comments" : [ ]
}
However when I try to retrieve files of a particular user like so it doesn't work:
app.get("/myvideos", function(request, result){
database.collection("videos").find({user:{_id:{$in:[user._id]}).toArray(function(error, videos){
result.render("myvideos", {
"isLogin": request.session.user_id ? true : false,
"videos": videos
});
});
});
I got the solution. This worked:
When user session got logged in, I created a new variable to store user name like so:
request.session.user_name = user.name
Then used the above for comparison as:
app.get("/myvideos", function(request, result){
//var logInId = request.session.user_id;
database.collection("videos").find({"user.name":request.session.user_name}).sort({
"createdAt": -1
}).toArray(function(error, videos){
result.render("myvideos", {
"isLogin": request.session.user_id ? true : false,
"videos": videos
});
});
});
And the above worked properly!
I am trying to update a subdocument inside of an array.
{
"_id" : ObjectId("5a7a017e0aa5311d9c75c86b"),
"campaignId" : "5a7a017e0aa5311d9c75c86b",
"subCampaigns" : [
{
"_id" : ObjectId("5a7a0eee6849f507d0499a18"),
"name" : "test5",
"campaignId" : ObjectId("5a7a017e0aa5311d9c75c86b"),
"subCampaignId" : "1aKysewspn6CqNGVkv6pyL"
}
{
"name" : "test8",
"amountRaised" : 20.0,
"campaignId" : ObjectId("5a7a017e0aa5311d9c75c86b"),
"subCampaignId" : "5a7a153fe035de396c6f5b3c",
"_id" : ObjectId("5a7a153fe035de396c6f5b3c")
}
],
"name" : "test8",
"__v" : 12,
"totalDonated" : 110.0
}
When I run:
db.getCollection('campaigns').update({"_id": ObjectId("5a7a017e0aa5311d9c75c86b"), "subCampaigns.subCampaignId":"5a7a153fe035de396c6f5b3c"}, {$inc: {"subCampaigns.$.amountRaised": 10}})
In the mongo console it updates the sub document fine. But when I run this in my Mongoose code:
this.campaignModel.update(
{ campaignId: '5a7a017e0aa5311d9c75c86b', 'subCampaigns.subCampaignId': '5a7a153fe035de396c6f5b3c' },
{ $inc: { 'subCampaigns.$.amountRaised': 10}},
(err: any, subResult: any) => {...
The sub document does not have its amountRaised field incremented. From what I can tell they should be identical calls?
If I run an inc at the level higher:
this.campaignModel.findOneAndUpdate(
{campaignId: object.campaignId},
{$inc: {totalDonated: donationAmount}},
() => {...
It works fine, so this must have something to do with the sub document?
I have been playing around with multiple mongoose calls for most of the day but cannot seem to get it to update correctly.
how to find data from mongodb in nodejs i have below query but its return null, i want to find data in child_categories array Key name child_categoryname please help me i am very thanful to you
i am using nodejs i want to find 'truck repair' and in response i want to get child_categoryhrs in response related to truck repair
example = 4
JobCategories.findOne({ child_categories : { child_categoryname : 'truck repair'} })
.exec().then(result => { console.log(result); })
.catch(handleError(res));
//mongodb
{
"_id" : ObjectId("58cea64cfd70bb1aa472ef2c"),
"category_name" : "Repairing",
"date_updated" : ISODate("2017-03-20T12:04:16.323Z"),
"child_categories" : [
{
"child_categoryname" : "truck repair",
"child_categoryhrs" : 4,
"_id" : ObjectId("58cea64cfd70bb1aa472ef2d")
},
{
"child_categoryname" : "car repair",
"child_categoryhrs" : 8,
"_id" : ObjectId("58cfc5405895461b286238fa")
}
],
"__v" : 0
}
I have written an update query in MongoDB/NodeJS that deletes objects from an array of a document, based on the parameters I define. After I pull these objects, I would like to to increment another variable in the document based on how many documents were modified by the update query.
Here is an example of one of my events documents:
{
"_id" : ObjectId("575ed7fca7b89bb4027dded9"),
"dateCreated" : "6/13/2016",
"latitude" : "56.294786195890076",
"longitude" : "-43.59161567687988",
"instructorName" : "Test User",
"instructorEmail" : "test#user.com",
"instructorRating" : 5,
"eventName" : "We gon exercise",
"eventDescription" : "We gon exercise",
"spacesAvailable" : 15,
"streetAddress" : "123 wer",
"city" : "rty",
"state" : "NY",
"zip" : "12332",
"date" : "06/21/2016",
"startTime" : "12:00",
"endTime" : "02:10",
"tags" : [
"Cardio",
"Crossfit"
],
"price" : 5,
"attendies" : [
{
"_id" : ObjectId("5759cfcdb71d80fb2d1203ef"),
"name" : "Buddy Loester",
"email" : "Bud18#gmail.com",
"timeStamp" : 1467048318510,
"payed" : true
},
{
"_id" : ObjectId("574f257b05086e2c7f7940ca"),
"name" : "Trainer Trainer",
"email" : "trainer#user.com",
"timeStamp" : 1467055627894,
"payed" : true
}
],
"unpayed" : 0
}
Here is my code to give a better visualization:
var eventCollection = req.db.get('events');
// get current time since epoch in milliseconds
var milliSinceEpoch = new Date().getTime();
eventCollection.update(
{"attendies.payed" : {$eq : false}},
{
$pull:
{
"attendies" : {"timeStamp": {$lt: milliSinceEpoch /*- 600000*/}}
},
$inc:
{
spacesAvailable: numberAffected
}
},
{
multi: true
}, function(err, numberAffected) {
console.log(numberAffected);
return res.end();
}
);
If I specify 'numberAffected' in the query portion to '1', then it works as expected and increments by 1. However, I would like to increment by the number affected.
I know this code will not work with 'numberAffected' in the query. Using 'numberAffected' in the callback actually does return the number of documents modified by my query.
Does there exist a way in MongoDB to do what I am trying to do?
I have solved my problem by rewriting the query. It is as follows:
var ObjectID = require("mongodb").ObjectID;
var eventCollection = req.db.get('events');
var milliSinceEpoch = new Date().getTime();
// find and return all the documents in the events DB where there is a user who has not payed for an event
// they RSVP'd for
eventCollection.find({"attendies.payed" : {$eq : false}}, function(err, documentsWithUnpayedUsers) {
// if error finding, print it and return
if(err) {
console.log(err);
return res.sendStatus(400, "Error cancelling");
}
// if everyone has payed for all RSVP'd events
if(!documentsWithUnpayedUsers) return res.sendStatus(404, "Everyone has payed!");
// loop through every document which has people who have not yet payed for RSVP'd events
for(var i = 0; i < documentsWithUnpayedUsers.length; i++) {
// for each of these documents:
eventCollection.update(
{_id: ObjectID(documentsWithUnpayedUsers[i]._id)},
{
// remove the user from the attendie list if they have not payed,
// and it has been 10 minutes since they RSVP'd
$pull:
{
"attendies" : {"timeStamp": {$lt: milliSinceEpoch - 600000}, "payed" : {$eq : false}}
},
// then modify the number of spaces available for the event by the number of people who were
// removed from the attendie list
// then modify the amount of people who have not payed for the event yet (will now be 0)
$inc:
{
spacesAvailable: documentsWithUnpayedUsers[i].unpayed,
unpayed: -documentsWithUnpayedUsers[i].unpayed
}
}, function(err) {
// error checking for the update query
if(err){
console.log(err);
return res.sendStatus(400, "There was an error removing an attendie fom the event: "
+ documentsWithUnpayedUsers[i].eventName);
}
}
); // end of update query
} // end of for loop
return res.end();
}
); // end of find()
}); // end of checkPayed
Considering the following
User collection & sample User document
{
"_id" : ObjectId("575c01f7b8e5999addeb598c"),
"username" : "test.1#gmail.com",
"password" : "<password>",
"firstName" : "Test,
"lastName" : "User"
}
I am trying to run an update request to add an entry in userData.eventData which is meant to be an array
In mongo script I can do
> db.Users.update({_id: ObjectId("575c01f7b8e5999addeb598c")}, {"$addToSet":{"userData.eventData":"My event"}} )
And I have the following result : userData is created as an Object and eventData as a nested Array
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.Users.find({_id: ObjectId("575c01f7b8e5999addeb598c")})
{ "_id" : ObjectId("575c01f7b8e5999addeb598c"), "username" : "test.1#gmail.com", "password" : "<password>", "firstName" : "Test", "lastName" : "User", "userData" : { "eventData" : [ "My event" ] } }
While running the same logic in mongo (using driver version 2.1.21)
// with a properly initialized db object
db.collection("Users").update({"_id" : ObjectId("575c01f7b8e5999addeb598c")}, {"$addToSet": { "userData.eventData": "My Event"}}, function(err, result) {
// do something
});
I receive the following response
result:Object
n:0
nModified:0
ok:1
And indeed the database entry is unchanged.
Is that the way it is meant to behave? I can easily fix this by creating the userData.eventData array but I found disturbing the fact that node's Mongo driver and mongo shell didn't behave the same on this
Thanks in advance for your help & advice
Edit 13/6/16
Mistake was on my side, I missed a 'new' before 'ObjectId(...' in node. With it, it behaves exactly the same as in mongo shell (i.e. 'userData' is created as an Object and it includes 'eventData' array)
No issue, then :)
Update, updates an already existing object in your document.
What you want is insert or use upset which creates a new document when no document matches the query criteria
db.collection.update(
{ name: "Andy" },
{
name: "Andy",
rating: 1,
score: 1
},
{ upsert: true }
);
If you wanted to add an object to your array, you would need $push
// Insert a document in the capped collection
function push (db, collection, search, data, callback) {
db.collection(collection).update(
search, {
$push: data
}, function (err, result) {
console.log("New object pushed");
callback(err);
});
}