Mongodb select random query from collection except few ids - node.js

I have a "User" and "Chat" collection
The User scheme is like:
username: {
type: String,
unique: true, required: true
}
And the Chat scheme:
from: {
type: Schema.Types.ObjectId,
required: true
},
to:{
type: Schema.Types.ObjectId,
required: true
},
content:{
type:String,
required:true
}
My purpose is to start a chat between an user and another random selected user.
My code so far to select random user:
User.aggregate([
{ "$match": { "_id": { "$ne": mongoose.Types.ObjectId(from_id) }}},
{$sample: {size: 1}}]
I'm using "$ne": mongoose.Types.ObjectId(from_id) so it won't select the same user.
What I want to achieve is choosing an user except same user (done) and also except someone that the first user has already chat with.
Like if there is already a chat between user1 and user2 then the code must choose another user except user2.

As an alternative to your approach:
const count = model.estimatedDocumentCount()
if (count === 2) {
throe new Error('Not possible')
}
let newUser = null
let i = 0
while(true) {
const random = Math.random() * count
newUser = await model.findOne({_id: {$ne: fromId}}).skip(random).exec()
if (!(await chatModel.findOne({$or: [{from: fromId, to: newUser._id}, {to: fromId, from: newUser._id}]}, {_id: 1}).exec())) {
break
}
if (i === 1000) {
throw new Error('Ah well not today') // can come up with some alternative strategy
}
++i
}
Use a compound index like ({from: 1, to: 1}) on Chat collection.
Should work nicely on large collection (because probability of hitting the same is low), as well as, small because collection should fit into memory and be easy to read from disk.

Related

Concurrency problems updating another's collection stats

I'm trying to make a notation system for movies
A user can note a Movie in their List.
Whenever the user clicks on the frontend, the listId, movieId, note are sent to the server to update the note. The note can be set to null, but it does not remove the entry from the list.
But if the user clicks too much times, the movie's totalNote and nbNotes are completely broken. Feels like there is some sort of concurrency problems ?
Is this the correct approach to this problem or am I updating in a wrong way ?
The mongoose schemas related :
// Movie Schema
const movieSchema = new Schema({
// ...
note: { type: Number, default: 0 },
totalNotes: { type: Number, default: 0 },
nbNotes: { type: Number, default: 0 },
})
movieSchema.statics.updateTotalNote = function (movieId, oldNote, newNote) {
if (!oldNote && !newNote) return
const nbNotes = !newNote ? -1 : (!oldNote ? 1 : 0) // If oldNote is null we +1, if newNote is null we -1
return Movie.findOneAndUpdate({ _id: movieId }, { $inc: { nbNotes: nbNotes, totalNotes: (newNote - oldNote) } }, { new: true }).catch(err => console.error("Couldn't update note from movie", err))
}
// List Schema
const movieEntry = new Schema({
_id: false, // movie makes an already unique attribute, which is populated on GET
movie: { type: Schema.Types.ObjectId, ref: 'Movies', required: true },
note: { type: Number, default: null, max: 21 },
})
const listSchema = new Schema({
user: { type: Schema.Types.ObjectId, ref: 'Users', required: true },
movies: [movieEntry]
})
The server update API (add / Remove movieEntry are similar with $push and $pull instead of $set)
exports.updateEntry = (req, res) => {
const { listId, movieId } = req.params
const movieEntry = { movieId: movieId, note: req.body.note }
List.findOneAndUpdate({ _id: listId, 'movies.movie': movieId }, { $set: { 'movies.$[elem]': movieEntry } }, { arrayFilters: [{ 'elem.movie': movieId }] })
.exec()
.then(list => {
if (!list) return res.sendStatus(404)
const oldNote = list.getMovieEntryById(movieId).note // getMovieEntryById(movieId) = return this.movies.find(movieEntry => movieEntry.movie == movieId)
Movie.updateTotalNote(movieId, oldNote, movieEntry.note)
let newList = list.movies.find(movieEntry => movieEntry.movie == movieId) // Because I needed the oldNote and findOneAndUpdate returns the list prior to modification, I change it to return it
newList.note = movieEntry.note
newList.status = movieEntry.status
newList.completedDate = movieEntry.completedDate
return res.status(200).json(list)
})
.catch(err => {
console.error(err)
return res.sendStatus(400)
})
}
The entries I needed to update were arrays that could grow indefinitely so I had to first change my models and use virtuals and another model for the the list entries.
Doing so made the work easier and I was able to create, update and delete the entries more easily and without any concurrency problems.
This might also not have been a concurrency problem in the first place, but a transaction problem.

Mongoose - query doc if element not in array

I am dealing with an issue while querying a notification schema
receiver: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Profile' }],
readBy: [{
readerId: { type: mongoose.Schema.Types.ObjectId,
ref: 'Profile', default: [] },
readAt: { type: Date }
}]
In order to query latest notifications, this is the query I have written:
GOAL is to check if the "profile.id" DOES NOT exist in the readBy array (which means its unread by that user)
const notifications = await Notification.find({
receiver: profile.id, // this works
readBy: { // but, adding this returns an empty array
$elemMatch: {
readerId: { $ne: profile.id }
}
}
})
Would really appreciate the help, stuck here for days.
I think is easier than we are trying to do.
If you want to know if the element is into the array, then, query looking for it. If the query is empty, it implies that not exists, otherwise yes.
Look this example.
Is a simple query, only check if one document has been received and readed by user profile.id.
db.collection.find([
{
"receiver": profile.id,
"readBy.readerId": profile.id
}
])
Please check if the output is as expected or I misunderstood your petition.
In mongoose you can use this:
var find = await model.find({"receiver":1,"readBy.readerId":1}).countDocuments()
It will return how many documents match the query.
Edit to get documents where readerId is not present in the array:
Here, you only need to add $ne operator
db.collection.find([
{
"receiver": profile.id,
"readBy.readerId": {
"$ne": profile.id
}
}
])
You can check this here

Update Mongoose Array

so basically I have this and I am trying to update the STATUS part of an array.
However, everything I try does nothing. I have tried findOneAndUpdate also. I am trying to identify the specific item in the array by the number then update the status part of that specific array
(Sorry for formatting, I have no idea how to do that on the site yet ...) (Full code can be found here: https://sourceb.in/0811b5f805)
Code
const ticketObj = {
number: placeholderNumber,
userID: message.author.id,
message: m.content,
status: 'unresolved'
}
let tnumber = parseInt(args[1])
let statuss = "In Progress"
await Mail.updateOne({
"number": tnumber
}, { $set: { "status": statuss } })
Schema
const mongoose = require('mongoose')
const mailSchema = new mongoose.Schema({
guildID: { type: String, required: true },
ticketCount: { type: Number, required: true },
tickets: { type: Array, default: [] }
}, { timestamps: true });
module.exports = mongoose.model('Mail', mailSchema)
You need to use something like Mail.updateOne({"guildID": message.guild.id}, {$set: {`tickets.${tnumber}.status`: statuss}})
or for all objects in array:
Mail.updateOne({"guildID": message.guild.id}, {$set: {'tickets.$[].status': statuss}})
Also, you need to create a schema for the tickets, as it is described in docs:
one important reason to use subdocuments is to create a path where there would otherwise not be one to allow for validation over a group of fields

Mongoose — findOneAndUpdate() causing document duplication

I'm using findOneAndUpdate() with upsert: true in order for a document to be updated if it exists and to be created otherwise. The tracks variable contains an array of Track instances. tracks does contain a few duplicates and that's where the problem begins. It causes the piece of code on line 7 (Observation.findOneAndUpdate(...)) to create a (low) number of duplicates, i.e. multiple documents that have the same (user, track) pair. Note that those duplicates are inserted randomly: running twice this piece of code brings different duplicated documents. My guess is that it has something to do with how the locking of data is done in MongoDB and that I'm doing too many operations at the same time. Any idea on how I could overcome this problem?
const promises = [];
​
tracks.forEach((track) => {
const query = { user, track };
const options = { new: true, upsert: true };
const newOb = { user, track, type: 'recent' };
promises.push(Observation.findOneAndUpdate(query, newOb, options));
});
​
return Promise.all(promises);
I'm using mongoose 5.5.8 and node 11.10.0.
Here's the Observation model:
const { Schema } = mongoose;
const ObservationSchema = new Schema({
track: { type: Schema.Types.ObjectId, ref: 'Track' },
user: { type: Schema.Types.ObjectId, ref: 'User' },
type: String
});
ObservationSchema.index({ track: 1, user: 1 }, { unique: true });
const Observation = mongoose.model('Observation', ObservationSchema);
And this is a sample of what the tracks array contains:
[
{ artists: [ 5da304b140185c5cb82d7eee ],
_id: 5da304b240185c5cb82d7f48,
spotifyId: '4QrEErhD78BjNFXpXDaTjH',
__v: 0,
isrc: 'DEF058230916',
name: 'Hungarian Dance No.17 In F Sharp Minor',
popularity: 25 },
{ artists: [ 5da304b140185c5cb82d7eee ],
_id: 5da304b240185c5cb82d7f5d,
spotifyId: '06dn1SnXsax9kJwMEpgBhD',
__v: 0,
isrc: 'DEF058230912',
name: 'Hungarian Dance No.13 In D',
popularity: 25 }
]
Thanks :)
I think this is due to your Promise.all method.
You should await every single query in the loop instead of awaiting everything at the same time at the end. Here an example with find:
async function retrieveApples() {
const apples = [];
arr.forEach(apple => {
const foundApple = await AppleModel.findOne({ apple });
apples.push(foundApple);
});
return apples
}

How to join two collections in mongoose

I have two Schema defined as below:
var WorksnapsTimeEntry = BaseSchema.extend({
student: {
type: Schema.ObjectId,
ref: 'Student'
},
timeEntries: {
type: Object
}
});
var StudentSchema = BaseSchema.extend({
firstName: {
type: String,
trim: true,
default: ''
// validate: [validateLocalStrategyProperty, 'Please fill in your first name']
},
lastName: {
type: String,
trim: true,
default: ''
// validate: [validateLocalStrategyProperty, 'Please fill in your last name']
},
displayName: {
type: String,
trim: true
},
municipality: {
type: String
}
});
And I would like to loop thru each student and show it's time entries. So far I have this code which is obviously not right as I still dont know how do I join WorksnapTimeEntry schema table.
Student.find({ status: 'student' })
.populate('student')
.exec(function (err, students) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
}
_.forEach(students, function (student) {
// show student with his time entries....
});
res.json(students);
});
Any one knows how do I achieve such thing?
As of version 3.2, you can use $lookup in aggregation pipeline to perform left outer join.
Student.aggregate([{
$lookup: {
from: "worksnapsTimeEntries", // collection name in db
localField: "_id",
foreignField: "student",
as: "worksnapsTimeEntries"
}
}]).exec(function(err, students) {
// students contain WorksnapsTimeEntries
});
You don't want .populate() here but instead you want two queries, where the first matches the Student objects to get the _id values, and the second will use $in to match the respective WorksnapsTimeEntry items for those "students".
Using async.waterfall just to avoid some indentation creep:
async.waterfall(
[
function(callback) {
Student.find({ "status": "student" },{ "_id": 1 },callback);
},
function(students,callback) {
WorksnapsTimeEntry.find({
"student": { "$in": students.map(function(el) {
return el._id
})
},callback);
}
],
function(err,results) {
if (err) {
// do something
} else {
// results are the matching entries
}
}
)
If you really must, then you can .populate("student") on the second query to get populated items from the other table.
The reverse case is to query on WorksnapsTimeEntry and return "everything", then filter out any null results from .populate() with a "match" query option:
WorksnapsTimeEntry.find().populate({
"path": "student",
"match": { "status": "student" }
}).exec(function(err,entries) {
// Now client side filter un-matched results
entries = entries.filter(function(entry) {
return entry.student != null;
});
// Anything not populated by the query condition is now removed
});
So that is not a desirable action, since the "database" is not filtering what is likely the bulk of results.
Unless you have a good reason not to do so, then you probably "should" be "embedding" the data instead. That way the properties like "status" are already available on the collection and additional queries are not required.
If you are using a NoSQL solution like MongoDB you should be embracing it's concepts, rather than sticking to relational design principles. If you are consistently modelling relationally, then you might as well use a relational database, since you won't be getting any benefit from the solution that has other ways to handle that.
It is late but will help many developers.
Verified with
"mongodb": "^3.6.2",
"mongoose": "^5.10.8",
Join two collections in mongoose
ProductModel.find({} , (err,records)=>{
if(records)
//reurn records
else
// throw new Error('xyz')
})
.populate('category','name') //select only category name joined collection
//.populate('category') // Select all detail
.skip(0).limit(20)
//.sort(createdAt : '-1')
.exec()
ProductModel Schema
const CustomSchema = new Schema({
category:{
type: Schema.ObjectId,
ref: 'Category'
},
...
}, {timestamps:true}, {collection: 'products'});
module.exports = model('Product',CustomSchema)
Category model schema
const CustomSchema = new Schema({
name: { type: String, required:true },
...
}, {collection: 'categories'});
module.exports = model('Category',CustomSchema)

Resources