Related
I have a schema
const membershipsSchema = new Schema({
spaceId: {
type: Schema.Types.ObjectId,
ref: 'Space',
},
member: {
type: Schema.Types.ObjectId,
ref: 'User',
},
....
);
mongoose.model('Membership', membershipsSchema);
I want to use join statement like
Select * from membershipPlans as plans join User as users on plans.member=users._id
where plans.spaceId=id and users.status <> 'archived'; // id is coming from function arguments
I tried the aggregate pipeline like
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$match: {
'users.status': {$ne: 'archived'}
}
},
]);
But on console.log(memberships); I am getting an empty array. If I try return Memberships.find({ spaceId: id }) it returns populated memberships of that space. But when I try
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
]}
It still returns an empty array. Not sure if I know how to use an aggregate pipeline.
There are two things that you need to do:
Cast id to ObjectId.
Instead of using $match, just filter the contents of the users array using $filter.
Try this:
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: new mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$project: {
users: {$filter: {
input: "$users",
as: "user",
cond: {
$ne: ["$$user.status", "archived"]
}
}}
}
},
]);
I am currently working on a project that has the following schema using mongoose.
User schema
const userSchema = {
name: string
email: string
medicalVisits: [{type: Schema.ObjectId, ref: "records"}]
createdAt: Date
}
Records schema
const recordSchema = {
medication: [String],
rating: Number
user: [{type: Schema.ObjectId, ref: "user"}]
tests: [{type: Schema.ObjectId, ref: "tests"}]
createdAt: Date
}
Tests schema
testScore: Number
answers: Object
user: [{type: Schema.ObjectId, ref: "user"}]
createdAt: Date
From the little schema above, I have a setup where a patient can take tests multiple times and their respective tests are saved in the Tests collection. Also, the date is recorded for all tests they take. A doctor can request to see a patient's record, in this case, the patient has only one record document that has their tests records embedded in them. Currently, I am faced with the problem of getting a patient's newest and oldest test score alongside their initial details.
I can do a mongoose populate to get all information regarding a user, e.g
await User.findById(userId).populate({
path: "medicalVisits"
model: "records"
populate: {
path: "tests"
model: "test"
}
})
And that operation returns the patient's record and all the tests they have taken since they signed up to date. But when I make such a call to the Database, I just want to retrieve the patient's newest and oldest score. In other words, I want to get the patients, Initial test score, and their most recent test score. I am new to Mongoose aggregation, I tried to use the Mongoose aggregate function, but it returns an empty array, I guess I am missing something.
Currently, this is what my aggregate pipeline looks like.
const user = await Doctor.aggregate([
{ $match: { _id: docId } },
{
$lookup: {
from: "users",
localField: "patients",
foreignField: "_id",
as: "patients",
},
},
{ $unwind: "$patients" },
{ $unwind: "$patients.medicalVisits" },
{
$lookup: {
from: "records",
localField: "patients.user",
foreignField: "_id",
as: "patientRecord",
},
},
{ $unwind: "$patientRecord" },
// { $sort: { createdAt: 1 } },
{
$group: {
_id: docId,
user: { $last: "$patients" },
record: { $last: "$patientRecord"}
},
},
]);
return user[0];
From the above snippet, my intention is:
given a doctor Id, they can see a list of their patients and also see their newest and oldest test score.
Expected Output
const output = {
userId: 6e12euido....
name: "John doe"
email: "john#john.com"
rating: 2
initialTestScore: 10
recentTestScore: 30
}
How do I go about this? Or what could be a better alternative? Thank you very much.
tried my best to understand your case, and I think your aggregation pipeline should be like:
const patientsWithNewestRecord = await Doctor.aggregate([
{ $match: { _id: docId } },
{
$lookup: {
from: "users",
localField: "patients",
foreignField: "_id",
as: "patients",
},
},
// one patient, per doc
{ $unwind: "$patients" },
// one patient with all his/her visit records, per doc
{
$lookup: {
from: "records",
localField: "patients.medicalVisits",
foreignField: "_id",
as: "patientRecords",
},
},
// one patient with one visit record, per doc
{ $unwind: "$patientRecords" },
// sort by patient first, createdAt second
{ $sort: { 'patientRecords.user': 1, 'patientRecords.createdAt': 1 } },
{
$group: {
_id: { patient: '$patientRecords.user' },
user: { $last: "$patients" },
record: { $last: "$patientRecords"}
},
},
]);
this pipeline return a list of a doctor's patients and also see their newest test record. Oldest test record should be in similar war.
Based on these collections (as I understand them from your question):
// doctor collection:
{ _id: "doc1", patients: ["user1"] }
// user collection:
{
_id: "user1", name: "John", email: "john#gmail.com",
medicalVisits: ["record1", "record2"]
}
// record collection:
{ _id: "record1", rating: 2, tests: ["test1", "test2"] }
{ _id: "record2", rating: 4, tests: ["test3"] }
// test collection:
{ _id: "test1", testScore: 12, createdAt: ISODate("2021-12-04") }
{ _id: "test2", testScore: 9, createdAt: ISODate("2021-12-05") }
{ _id: "test3", testScore: 15, createdAt: ISODate("2021-12-24") }
we can apply:
db.doctor.aggregate([
{ $match: { _id: "doc1" } }
{ $lookup: {
from: "user",
localField: "patients", foreignField: "_id",
as: "patients"
}},
{ $unwind: "$patients" }, { $unwind: "$patients.medicalVisits" },
{ $lookup: {
from: "record",
localField: "patients.medicalVisits", foreignField: "_id",
as: "records"
}},
{ $unwind: "$records" }, { $unwind: "$records.tests" },
{ $lookup: {
from: "test",
localField: "records.tests", foreignField: "_id",
as: "tests"
}},
{ $unwind: "$tests" },
{ $sort: { "tests.createdAt": 1 } },
{ $group: {
_id: "$patients._id",
name: { $first: "$patients.name" },
email: { $first: "$patients.email" },
rating: { $first: "$records.rating" },
initialTestScore: { $first: "$tests.testScore" },
recentTestScore: { $last: "$tests.testScore" }
}},
{ $set: { "userId": "$_id" } }, { $unset: "_id" }
])
in order to extract:
{
userId: "user1",
name: "John",
email: "john#gmail.com",
rating: 2,
initialTestScore: 12,
recentTestScore: 15
}
Differences compared to your query:
I $lookup the test collection as it seems you information from there to get both test dates and test scores.
I $sort by test date (createdAt) before the $group by user such that we'll be able to define the right order for selecting the $first and $last test scores.
I extract user's information by using a $first on each group on user's field (since all unwind records for a given user have the same user information): for instance email: { $first: "$patients.email" }
I extract the $first and $last test scores for a user as defined by the $sort order: initialTestScore: { $first: "$tests.testScore" } and recentTestScore: { $last: "$tests.testScore" }.
I finally $set/$unset to rename the _id field into userId
I would suggest to do the following once you have the userId / patientId:
Get their tests (all) from the database in a sorted order
Take the first and last element of the array for your initial and final test report based on the sorting order you have applied (ascending or descending)
If you can just retrieve the user details and all the tests without any sorting, then you can proceed the following way:
Run a loop through all the tests and sort the tests according to test date.
Take the first and last element of the array for your initial and final test report based on the sorting order you have applied (ascending or descending)
You will not be performing the operations on DB end, so there might be a minor speed issue, but the difference would still come out to be in milliseconds unless a user takes a billion tests.
Let me know if this helps, let me know if it doesn't
I have the following schema:
const UserQualificationSchema = new Schema(
{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
qualification: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Qualification',
},
expiry_date: {
type: Date
}
}
const QualificationSchema = new Schema(
{
fleet: {
type: [String], // Eg ["FleetA", "FleetB", "FleetC"]
required: true,
}
}
I am searching the UserQualifications with filters in a table, to search them by fleet, qualification or expiry date. I so far have the following aggregate:
db.UserQualifications.aggregate([{
{
$lookup: {
from: 'qualifications',
localField: 'qualification',
foreignField: '_id',
as: 'qualification',
},
},
{
$unwind: '$qualification',
},
{
$match: {
$and: [
'qualification.fleet': {
$in: ["Fleet A", "Fleet C"], // This works
},
expiry_date: {
$lt: req.body.expiry_date, // This works
},
qualification: { // Also tried 'qualification._id'
$in: ["6033e4129070031c07fbbf29"] // Adding this returns blank array
}
]
},
}
}])
Filtering by fleet, and expiry date both work, independently and in combination, however when adding by the qualification ID, it returns blank despite the ID's being sent in being valid.
Am i missing something here?
Looking at your schema I can infer that qualification in ObjectId and in the query you are passing only the string value of ObjectId. You can pass the ObjectId to get your expected output
db.UserQualifications.aggregate([
{
$lookup: {
from: "Qualifications",
localField: "qualification",
foreignField: "_id",
as: "qualification",
},
},
{
$unwind: "$qualification",
},
{
$match: {
"qualification.fleet": {
$in: [
"FleetA",
"FleetC"
],
},
expiry_date: {
$lt: 30 // some dummy value to make it work
},
"qualification._id": {
$in: [
// some dummy value to make it work
ObjectId("5a934e000102030405000000")
]
}
},
}
])
I have created a playground with some dummy data to test the query: Mongo Playground
Also, In $match stage there is no need to combine query explicitly in $and as by default behaviour will be same as $and only so I have remove that part in my query
I have two models, made using Mongoose Schema.
Book {
title: String,
chapters: [{
type: Schema.Types.ObjectId,
ref: 'chapter'
}],
}
Chapter {
title: String,
status: String,
book: {
type: Schema.Types.ObjectId,
ref: 'book'
},
}
I want to find Books that have a chapter with "status":"unfinished". What is the most efficient way to achieve this? Since the Book model stores ObjectIds, how can I make the find query so that the filtered results will be fetched directly from the DB?
I think the most optimal way would be to denormalize your schema, as a book will have a limited amount of chapters and a chapter can belong to at most one book, we can store the schema like this
Book {
title: String,
chapters: [{
title: String,
status: String,
}],
}
with this schema, we can then create an index on 'chapters.status' and simply get the answer in a single query without the need of $lookup.
db.books.find({'chapters.status': 'unfinished'});
But in any case, you still need to go with the above schema, we always have an option for $lookup
db.book.aggregate([
{
$unwind: "$chapters",
},
{
$lookup: {
from: "chapter",
localField: "chapters",
foreignField: "_id",
as: "chapter",
},
},
{
$match: {
"chapter.status": "unfinished",
},
},
{
$group: {
_id: "$_id",
title: { $first: "$title" },
},
},
]);
You can always adjust the above query to your needs.
Example
You can try using aggregate(),
$lookup with pipeline, join Chapter collection
$match 2 conditions first match chapter _id in chaptersIds, second status is equal to unfinished
$match to match chapters not equal to empty array
$project to show or hide required fields
db.Book.aggregate([
{
"$lookup": {
from: "Chapter",
as: "chapters_list",
let: { chapterIds: "$chapters" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $in: ["$_id", "$$chapterIds"] },
{ $eq: ["$status", "unfinished"] }
]
}
}
}
]
}
},
{
$match: { chapters_list: { $ne: [] } }
},
// if you want chapters_list array then remove $project this part
{
$project: { chapters: 1, title: 1 }
}
])
Playground
I appreciate some help. I'm doing an api rest with express and mongodb (v3.4.4), using mongoose (v4.10.5). I need to do an aggregation operation, but I do not deal with it. I show you some code. The models (it has more properties, but I have left it simple):
const CategoryModel = mongoose.model('Category', new Schema({
slug: { type: String, unique: true, lowercase: true, index: true },
description: String
}));
const MyModel = mongoose.model('MyModel', new Schema({
category: { type: Schema.Types.ObjectId, ref: 'Category' },
other: [{ type: Schema.Types.ObjectId, ref: 'Other' }],
times_count: { type: Number, default: 0 }
}));
Important, I'm interested in populate category field of MyModel, not other field.
Suppose Category and MyModel has certain records well formed. The request:
MyModel.aggregate([
{
$group : {
_id : '$_id',
times: { $sum: '$times_count' }
}
},
{
$limit: 5
}
]).limit(5).exec().then((data) => {
console.log(data);
}).catch((err) => {
console.error(err);
});
data is correct, has 5 records, but not include category. Now, I try with:
MyModel.aggregate([
{
$group : {
_id : '$_id',
times: { $sum: '$times_count' }
}
},
{
$limit: 5
},
{
$lookup: {
from: 'Category', // I tried with 'Categories' and 'categories'
localField: 'category',
foreignField: '_id',
as: 'category'
}
},
{
$unwind: '$category'
}
]).limit(5).exec().then((data) => {
console.log(data);
}).catch((err) => {
console.error(err);
});
Now data is empty. I set mongoose.set('debug', true); and the operations they look right, inclusive the last operation aggregate, but data is empty...
I do not know if I explained well. Obviously there is something that I am not fully understanding. Thanks in advance.
I get the desired records in objs, the problem is that I only come with the _id and times properties, and I need to populate the category.
That's about right since you didn't explicitedly add the stage to join the other collection.
I've tried adding $project to the aggregation after the $ group but nothing.
In simple terms, $project is for including and excluding new fields using one collection, not joining.
You are looking for $lookup which is for joining one collection with another. When you join a new collection, each document will have a new array field containing the "joined" documents from the other collection.
In your case, your new array field will have one document from the other collection, so you probably want to $unwind also.
MyModel.aggregate([
{
$group : {
_id : '$_id',
times: { $sum: '$times_count' },
category: { $first: '$category' }
}
},
/*
{
$limit: 5
},
*/
{
$lookup: {
from: 'Categories',
localField: 'category',
foreignField: '_id',
as: 'category'
}
},
{
$unwind: '$category'
}
]).exec(...);
In terms of your initial problem, try uncommenting the 2nd stage above and not using limit(5) in your first example.