Big query in MongoDB with Mongoose - node.js

I have an User model defined. This model has two lists, one of the items the user has liked and the other one is of the items the user has disliked.
I need a list of the items that one User hasn't qualified (neither liked nor disliked) and the other users did. I'm using the Mongoose library for NodeJS and also the lodash(_) library,
my code looks like this:
function itemsUserHasntQualified(var user){
items = [];
User.find().exec(function(err, users){
for(var user_it: users){
if(user_it != user){
items.push(_.difference(user_it.tracks.liked, user_it.tracks.disliked, user.tracks.liked, user.tracks.disliked);
}
}
});
}
This is the schema for the User:
var UserSchema = new Schema({
name: String,
username: {type: String, lowercase:true },
email: { type: String, lowercase: true },
role: {
type: String,
default: 'user'
},
hashedPassword: String,
provider: String,
salt: String,
facebook: {},
twitter: {},
google: {},
github: {},
tracks: {
liked: [{type:Schema.ObjectId, ref: "Track"}],
disliked: [{type:Schema.ObjectId, ref: "Track"}],
later: [{type:Schema.ObjectId, ref: "Track"}]
}
});
But actually I'm feeling this is not the correct way of do it.
Is there a simpler or more correct way of query this?

I'm not sure what classifies as correct, but you can at least run the bulk of the query in mongodb without returning the entire collection.
Mongoose Query#distinct which is db.collection.distinct() will return distinct array items and can be supplied a query.
User.distinct('tracks.disliked', { username: { $ne: username } })
User.distinct('tracks.liked', { username: { $ne: username } })
This will give you the arrays for liked and disliked, which you can then difference for a user.
UserSchema.methods.itemsUserHasntQualified = function () {
var user = this
var liked = User.distinct('tracks.liked', { username: { $ne: user.username } })
var disliked = User.distinct('tracks.disliked', { username: { $ne: user.username } })
Promise.all([liked, disliked]).then(function (results) {
var all_ratings = _.union( results[0], results[1] )
var users_ratings = _.union( user.tracks.liked, user.tracks.disliked )
var missing = _.difference( users_ratings, all_ratings )
return missing
})
}
Depending on your access patterns, you might want to run this collection scan somewhere else, less frequently and cache the array results for use in itemsUserHasntQualified.

Related

add a item inside a nested schema mongoose with addToSet

I know populating schemas is not a new question but I am having a little trouble following the logic on this in regards to multiple schemas. I am working with
"mongoose": "^4.8.5",
"express": "^4.15.0",
I have a schema with a collection of caffeine drinks. When a user selects a drink i would like for that drink to be assigned to the user.
** If at any point I am missing something simple in the architecture please let me know. This project has been my intro to mongodb.
I am reading through populating on the mongoose documentation http://mongoosejs.com/docs/populate.html.
Essentially, if I am to assign the drinks to the list it looks like I want to add them as a reference in an array. This was my approach with caffeine_list
const SelectedDrinks = require('./userDrinks');
const UserSchema = mongoose.Schema({
name: {
type: String
},
email: {
type: String,
required: true
},
username: {
type: String,
required: true
},
password: {
type: String,
required: true
},
caffeine_list: caffeine_list: [ // attempting to reference selected drinks
{
type: mongoose.Schema.Types.ObjectId,
ref: 'SelectedDrinks'
}
]
})
SelectedDrinks comes from the schema below. I added a reference to the user as the creator below
const User = require('./user');
let userDrinkSchema = new mongoose.Schema({
creator : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
caffeine: Number,
mgFloz: Number,
name: String,
size: Number,
updated_at: {
type: Date,
default: Date.now()
}
});
This is where I start to get confused. I initially tried populate but could not get it going. If that was correct please let me know.
In regards to my task of adding a selected drink to the user I used addToSet. I was hoping that this would give me the drink info. I did my set up like so....
const User = require('../../models/user');
const UserDrinks = require('../../models/userDrinks');
router.post('/addDrink', (req, res, next) => {
let newDrink = new UserDrinks({
creator: req.body.creator,
caffeine: req.body.caffeine,
mgFloz: req.body.mgFloz,
name: req.body.name,
size: req.body.size,
updated_at: req.body.updated_at
});
newDrink.save( (err) => {
if(err) {
res.send(err);
} else {
User.findOne({ _id: newDrink.creator}, (err, user) => {
user.caffeine_list.addToSet(newDrink)
user.save( function (err) {
if(err) {
console.log(err);
}else {
res.status(201).json(newDrink);
}
})
})
}
})
});
However, after i do a post in postman I check caffeine_list and the result is
"caffeine_list" : [
ObjectId("58d82a5ff2f85e3f21822ab5"),
ObjectId("58d82c15bfdaf03f853f3864")
],
Ideally I would like to have an array of objects being passed with the caffeine info like so
"caffeine_list" : [
{
"creator": "58d6245cc02b0a0e6db8d257",
"caffeine": 412,
"mgFloz": 218.7,
"name": "1.95 Perfect Drink!",
"size": 42.93,
"updated_at": "2017-03-24T18:04:06.357Z"
}
]
Change your else part with below code instead of findOne and save use update
User.update(
{ _id: newDrink.creator},
{ $addToSet:{
caffeine_list: newDrink
}}).exec(function (err, updatedrink){
if(err) {
console.log(err);
}else {
res.status(201).json(updatedrink);
}
})
Although I am not sure this is the best approach I did find this to be give me the result that I was desiring. I had to make two small changes and I was able to get the caffeine_list to give me the desired response
I had to access the schema for selected drinks
const SelectedDrinks = require('./userDrinks').schema; //** need schema
Afterwards I was able to change
caffeine_list: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'UserDrinks' // name of the file
}
]
to
caffeine_list: [SelectedDrinks]
Now that I have the schema I am able to add the drinks directly into the caffeine_list on the UserSchema.

What is the best practice to insert new entry with nodejs and mongoose

I have two models:
This is the first:
var CommentSchema = new Schema({
title: {
type: String
},
owner: {
type: Schema.ObjectId,
ref: 'User'
}
});
This is the second:
var UserSchema = new Schema({
name: {
type: String
},
comments: [{
type: Schema.ObjectId,
ref: 'Comment'
}]
});
There are really a lot of comments. So, i add the id of each comment in the array of comment of owner. The goal is to create a simple function like "list my comments".
The comments are verry actually, i use this code to add new comment:
var comment = ...
comment.title = "title";
comment.owner = req.user
comment.save(function(err) {
if (err)
...
else {
User.findOne({ _id: req.user._id }, function (err, tmpUser){
tmpUser.comments.push(comment);
tmpUser.save(function(err) {
if (err)
...
else {
//END
}
});
});
}
}
How to optimise this code ? Is it possible to optimise directly the model ? Thank you !
Why not simply query the Comments collection? If you're worried about the performance when there are a lot of comments, you can (and should) index the owner field.
db.comments.ensureIndex( { owner: 1 } )
And then using mongoose:
Comment.find({ owner: req.user._id }, function(err, comments) {
// do something with comments
});

Insert into embedded document

I have the following schema:
var UserSchema = new Schema({
username: { type: String, required: true },
password: { type: String, required: true },
userType: { type: String, default: 'user'},
quizzHistory: [{
quizzName: String,
quizzScore: Number
}]
});
my goal is to change document into embedded quizzHistory or insert new one if not exists document in embedded quizzeHistory
I try to set document into embedded quizzHistory :
User.findOneAndUpdate({ _id: req.session.user['_id'], 'quizzHistory.quizzName': testName},
{
'$set': {
'quizzHistory.$.quizzName': testName,
'quizzHistory.$.quizzScore': finalScore
}
}, {upsert: true},
function(err, upd) {
console.log("added");
})
code above works if there is document in quizzHistory with required _id and quizzHistory.quizzName,but don't pushed new one if there isn't any document.
Is there any way in Mongodb to change document into embedded collection or insert new one if not exists ?
the reason is because you are using "find and update" you are not handling the condition when the row hasn't been found and create a new document, being said that you need manage the in a different way like
User.update({ _id: req.session.user['_id'], 'quizzHistory.quizzName': testName},
{
'$push': {
'quizzHistory.quizzName': testName,
'quizzHistory.quizzScore': finalScore
}
}, {upsert: true},
function(err, upd) {
console.log("added");
})
this worked for me
User.update({ _id: req.session.user['_id'],'quizzHistory.quizzName':testName},
{
$set: {
"quizzHistory.$.quizzName":testName,
"quizzHistory.$.quizzScore":finalScore
}
},
function(err, upd) {
if(!upd){
User.update({ _id: req.session.user['_id']},
{ "$addToSet": { quizzHistory: newScoreData }},function(err,data){
});
}
});
If you want to benefit for all possible plugins and methods added to model and don't want to fight with actual Mongo queries you should first retrieve user object, then push new element to quizzHistory array and do save. Something like below (you need to align that code to your needs).
var entry = {
quizzName : 'abc',
quizzScore : 123
};
User.findOne({_id:id, function(err, object) {
if(err) {
return someErrorCallback();
}
var
object.quizzHistory.push(entry);
object.save(function(err) {
if(err) {
return someErrorCallback();
}
someSuccessCallback();
});
});
Updating directly may be efficient, but questions usage of mongoose.
Hope it makes sense.

Mongoose - Better solution with appending additional information

I have two Schemas:
var ProgramSchema = new Schema({
active: Boolean,
name: String,
...
});
var UserSchema = new Schema({
username: String,
email: { type: String, lowercase: true },
...
partnerships: [{
program: { type: Schema.Types.ObjectId, ref: 'Program' },
status: { type: Number, default: 0 },
log: [{
status: { type: Number },
time: { type: Date, default: Date.now() },
comment: { type: String },
user: { type: Schema.Types.ObjectId, ref: 'User' }
}]
}]
});
Now I want to get all Program docs, but also append 'status' to each doc, to return if the program is already in a partnership with the logged in user.
My solution looks like this:
Program.find({active: true}, 'name owner image user.payments', function (err, p) {
if(err) { return handleError(res, err); }
})
.sort({_id: -1})
.exec(function(err, programs){
if(err) { return handleError(res, err); }
programs = _.map(programs, function(program){
var partner = _.find(req.user.partnerships, { program: program._id });
var status = 0;
if(partner){
status = partner.status;
}
program['partnership'] = status;
return program;
});
res.json(200, programs);
});
The req.user object contains all information about the logged in user, including the partnerships array.
To get this solution to work, I have to append
partnership: Schema.Types.Mixed
to the ProgramSchema.
This looks a bit messy and thats why I am asking for help. What do you think?
When you want to freely modify the result of a Mongoose query, add lean() to the query chain so that the docs (programs in this case) are plain JavaScript objects instead of Mongoose doc instances.
Program.find({active: true}, 'name owner image user.payments')
.lean() // <= Here
.sort({_id: -1})
.exec(function(err, programs){ ...
Then you can remove partnership from your schema definition. Your query will also execute faster.

Dealing with $in using mongoose

This is the code:
usermodel.findOne({ user: req.session.user }, function (err, user){
usermodel.find({ _id: {$in: user.follow } }, { user: true }, function (err, users){
usermodel.find({ author: { $in: users.user }}, function (err, images){
console.log(users);
console.log(images);
});
});
And this is the Schema:
var userschema = new mongoose.Schema({
user: String,
follow: [String],
message: [{
title: String,
author: String,
message: String
}],
});
The followarray contains the _ids of the users that the actual user is following. With the first usermodel.find y get the follow array. With the second usermodel.find, I get the users names:
[ { _id: 50fd9c7b8e6a9d087d000006, user: 'Mrmangado' },
{ _id: 50fd9d3ce20da1dd7d000006, user: 'kirbo' } ]
And, with the third and last usermodel.find, I'm trying to get the message of the users. I get the users' names in the previous usermodel.find, and the author of the message has the same value of the user that has created it. The problem is the way I get the users' names, I think I have to have the users' names like this way:
[{ user: 'Mrmangado' },
{ user: 'kirbo' }]
If I receive an array like this, the $in statement will work perfectly. Is there any way to get an output like this...?
Thank's advance!
Are you sure that's what you want? The author field is a user's name, not its _id? If so, you could just map the resulting object to get the name.
var usernames = users.map(function(u) { return u.user; });
//= ["Mrmangado", "kirbo"]

Resources