Aggregate Function Mongoose - Node - node.js

I have a schema
const membershipsSchema = new Schema({
spaceId: {
type: Schema.Types.ObjectId,
ref: 'Space',
},
member: {
type: Schema.Types.ObjectId,
ref: 'User',
},
....
);
mongoose.model('Membership', membershipsSchema);
I want to use join statement like
Select * from membershipPlans as plans join User as users on plans.member=users._id
where plans.spaceId=id and users.status <> 'archived'; // id is coming from function arguments
I tried the aggregate pipeline like
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$match: {
'users.status': {$ne: 'archived'}
}
},
]);
But on console.log(memberships); I am getting an empty array. If I try return Memberships.find({ spaceId: id }) it returns populated memberships of that space. But when I try
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
]}
It still returns an empty array. Not sure if I know how to use an aggregate pipeline.

There are two things that you need to do:
Cast id to ObjectId.
Instead of using $match, just filter the contents of the users array using $filter.
Try this:
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: new mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$project: {
users: {$filter: {
input: "$users",
as: "user",
cond: {
$ne: ["$$user.status", "archived"]
}
}}
}
},
]);

Related

How to get the first element from a child lookup in aggregation - Mongoose

I'm trying to find all the docs from groupUserRoleSchema with a specific $match condition in the child. I'm getting the expected result, but the child application inside groupSchema is coming as an array.
I just need the first element from the application array as an object. How to convert this application into a single object.
These are my models
const groupUserRoleSchema = new mongoose.Schema({
group: {
type: mongoose.Schema.Types.ObjectId,
ref: 'group'
},
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'user'
}
});
const groupSchema = new mongoose.Schema({
application: {
type: mongoose.Schema.Types.ObjectId,
ref: 'application'
}
});
Here is my aggregate condition.
groupUserRoleModel.aggregate([
{
$lookup: {
from: "groups", //must be PHYSICAL collection name
localField: "group",
foreignField: "_id",
as: "group",
}
},
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user",
}
},
{
$lookup: {
from: "applications",
localField: "group.application",
foreignField: "_id",
as: "group.application",
}
},
{
$addFields: {
group: {
$arrayElemAt: ["$group", 0],
},
user: {
$arrayElemAt: ["$user", 0],
}
},
},
{
$match: {
"user.email_id": requestHeaders.user_email
},
}
]);
Here the $lookup group.application is coming as an array. Instead i need it as an object.
Below added is the current output screen-shot
Here is the expected output screen-shot
Any suggestions ?
A alternative is re-writing the main object return, and merge the fields.
The lookup field, gonna use the $arrayElemAt attribute at position 0, with the combination of the root element '$$ROOT'.
const result = await groupUserRoleModel.aggregate([
{
$match: {
"user.email_id": requestHeaders.user_email //The Match Query
},
},
{
$lookup: {
from: 'groups', // The collection name
localField: 'group',
foreignField: '_id',
as: 'group', // Gonna be a group
},
},
{
// Lookup returns a array, so get the first once it is a _id search
$replaceRoot: {
newRoot: {
$mergeObjects: [ // merge the object
'$$ROOT',// Get base object
{
store: {
$arrayElemAt: ['$group', 0], //Get first elemnt
},
},
],
},
},
},
]);

Combining $lookup aggregation inside updateMany?

I have a collection of users like this
[
{ _id: ObjectId("61a6d586e56ea12d6b63b68e"), fullName: "Mr A" },
{ _id: ObjectId("6231a89b009d3a86c788bf39"), fullName: "Mr B" },
{ _id: ObjectId("6231a89b009d3a86c788bf3a"), fullName: "Mr C" }
]
And a collection of complains like this
[
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("61a6d586e56ea12d6b63b68e" },
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("6231a89b009d3a86c788bf3c" },
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("6231a89b009d3a86c788bf3b" },
]
I want to check if the postedBy fields of complains are not existed in users, then update by using the updateMany query
By the way, I have an optional way to achieve the goal but must use 2 steps:
const complains = await Complain.aggregate()
.lookup({
from: "users",
localField: "postedBy",
foreignField: "_id",
as: "postedBy",
})
.match({
$expr: {
$eq: [{ $size: "$postedBy" }, 0],
},
});
complains.forEach(async (complain) => {
complain.type = "other";
await complain.save();
});
Therefore, can I combine 2 steps into a single updateMany query? Like $match and $lookup inside updateMany query?
With MongoDB v4.2+, you can use $merge to perform update at last stage of aggregation.
db.complains.aggregate([
{
"$lookup": {
from: "users",
localField: "postedBy",
foreignField: "_id",
as: "postedByLookup"
}
},
{
$match: {
postedByLookup: []
}
},
{
"$addFields": {
"type": "other"
}
},
{
"$project": {
postedByLookup: false
}
},
{
"$merge": {
"into": "complains",
"on": "_id",
"whenMatched": "replace"
}
}
])
Here is the Mongo playground for your reference.

Filter results using $match in MongoDB aggregate returning blank array

I have the following schema:
const UserQualificationSchema = new Schema(
{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
qualification: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Qualification',
},
expiry_date: {
type: Date
}
}
const QualificationSchema = new Schema(
{
fleet: {
type: [String], // Eg ["FleetA", "FleetB", "FleetC"]
required: true,
}
}
I am searching the UserQualifications with filters in a table, to search them by fleet, qualification or expiry date. I so far have the following aggregate:
db.UserQualifications.aggregate([{
{
$lookup: {
from: 'qualifications',
localField: 'qualification',
foreignField: '_id',
as: 'qualification',
},
},
{
$unwind: '$qualification',
},
{
$match: {
$and: [
'qualification.fleet': {
$in: ["Fleet A", "Fleet C"], // This works
},
expiry_date: {
$lt: req.body.expiry_date, // This works
},
qualification: { // Also tried 'qualification._id'
$in: ["6033e4129070031c07fbbf29"] // Adding this returns blank array
}
]
},
}
}])
Filtering by fleet, and expiry date both work, independently and in combination, however when adding by the qualification ID, it returns blank despite the ID's being sent in being valid.
Am i missing something here?
Looking at your schema I can infer that qualification in ObjectId and in the query you are passing only the string value of ObjectId. You can pass the ObjectId to get your expected output
db.UserQualifications.aggregate([
{
$lookup: {
from: "Qualifications",
localField: "qualification",
foreignField: "_id",
as: "qualification",
},
},
{
$unwind: "$qualification",
},
{
$match: {
"qualification.fleet": {
$in: [
"FleetA",
"FleetC"
],
},
expiry_date: {
$lt: 30 // some dummy value to make it work
},
"qualification._id": {
$in: [
// some dummy value to make it work
ObjectId("5a934e000102030405000000")
]
}
},
}
])
I have created a playground with some dummy data to test the query: Mongo Playground
Also, In $match stage there is no need to combine query explicitly in $and as by default behaviour will be same as $and only so I have remove that part in my query

how too use $lookup in mongoose to fetch array object

I have the following aggregation and It does not return the user profiles properly
const newConverSation = await Messenger.Messenger.aggregate([
{ $match: {
users: mongoose.Types.ObjectId("6084036ad4d4cd40a47afba4")}},
{ $sort: { updatedAt: -1 } },
{
$group: {
_id: { $setUnion: "$users" },
message: { $first: "$$ROOT" }
}
},
{
$lookup: {
from: 'users',
localField: 'users',
foreignField: '_id',
as: 'users'
}
}
])
In the outcome, there is an users array like this
"users" : [
ObjectId("60841d03f6ccad2b0400f619"),
ObjectId("6084036ad4d4cd40a47afba4")
],
and I just want to fetch user profiles depending on these two Id's but it does not return profiles in the current way.
Try adding the .populate after the .find property where you want to show this data.

How to get array of objects from current collection and data from another collection in same query

I have a users collection. Each user can have multiple friends whose _id values are stored in an array in their user document.
I want to render a list which has a user's list of friend names and under each individual name I want to have a location but the locations are in another document because each user can have multiple locations with no limit on the number, like their visited/favourite places.
These are the 2 queries in isolation but I need to somehow combine them.
This is the query which looks at the user ID values in the user's array of friends and then gets the relevant user info based on the ID.
friends = await User.findOne({ _id: userId })
.populate("friends", "name mobile", null, { sort: { name: 1 } })
.exec();
Then, to get the places for a particular user:
const place = await Place.find({ creator: userId });
So, I basically want to list the friends in a loop, each with their places under their name like:
Joe Soap
- London Bridge
- Eiffel Tower
Bob Builder
- Marienplatz
The data in mongoDb look like this:
Users:
{
"_id": {
"$oid": "5f2d9ec5053d4a754d6790e8"
},
"friends": [{
"$oid": "5f2da51e053e4a754d5790ec"
}, {
"$oid": "5f2da52e053d4a754d6790ed"
}],
"name": "Bob",
"email": "bob#gmail.com",
"created_at": {
"$date": "2020-08-07T18:34:45.781Z"
}
}
Places:
{
"_id": {
"$oid": "5f3695d79864bd6c7c94e38a"
},
"location": {
"latitude": -12.345678,
"longitude": 12.345678
},
"creator": {
"$oid": "5f2da51e053e4a754d5790ec"
},
}
The first join is basically getting data from the array inside the same user collection. The second one is getting data from another collection, places.
UPDATE: almost working
friends = await User.aggregate([
{ $match: { _id: new mongoose.Types.ObjectId(userId) } },
{
$lookup: {
from: "users",
localField: "friends",
foreignField: "_id",
as: "friends_names",
},
},
{ $unwind: "$friends_names" },
{
$lookup: {
from: "places",
localField: "friends",
foreignField: "creator",
as: "friends_places",
},
},
{ $unwind: "$friends_places" },
{
$project: {
"friends_names.name": 1,
"friends_places.saved_at": 1,
},
},
]);
Data returned:
[
{
_id: 5f2d9ec5053d4a754d6790e8,
friends_names: { name: 'Bob' },
friends_places: { saved_at: 2020-08-17T13:40:28.334Z }
},
{
_id: 5f2d9ec5053d4a754d6790e8,
friends_names: { name: 'Natalie' },
friends_places: { saved_at: 2020-08-17T13:40:28.334Z }
}
]
Edit Okay, I believe I have reverse engineered your minimal schema correctly:
const mongoose = require("mongoose");
mongoose.connect("mongodb://localhost/test", { useNewUrlParser: true });
mongoose.set("debug", true);
const db = mongoose.connection;
db.on("error", console.error.bind(console, "connection error:"));
db.once("open", async function () {
await mongoose.connection.db.dropDatabase();
// we're connected!
console.log("Connected");
const userSchema = new mongoose.Schema({
friends: [{ type: mongoose.Schema.Types.ObjectId, ref: "User" }],
name: String,
});
const placesSchema = new mongoose.Schema({
latitude: String,
creator: { type: mongoose.Schema.Types.ObjectId, ref: "User" },
});
const User = mongoose.model("User", userSchema);
const Place = mongoose.model("Place", placesSchema);
const bob = new User({ name: "Bob", friends: [] });
await bob.save();
const natalie = new User({ name: "Natalie", friends: [bob] });
await natalie.save();
//const chris = new User({ name: "Chris", friends: [] });
//await chris.save();
const john = new User({ name: "John", friends: [natalie, bob] });
await john.save();
const place1 = new Place({ latitude: "Place1", creator: bob });
const place3 = new Place({ latitude: "Place3", creator: bob });
const place2 = new Place({ latitude: "Place2", creator: natalie });
await place1.save();
await place2.save();
await place3.save();
await User.find(function (err, users) {
if (err) return console.error(err);
console.log(users);
});
await Place.find(function (err, places) {
if (err) return console.error(err);
//console.log(places);
});
var cc = await mongoose
.model("User")
.aggregate([
{ $match: { _id: john._id } },
{ $unwind: "$friends" },
{
$lookup: {
from: "places",
localField: "friends",
foreignField: "creator",
as: "friends_places",
}
}, { $lookup: {from: 'users', localField: 'friends', foreignField: '_id', as: 'friend_name'} },//, { $include: 'friends' }
{ $unwind: "$friends_places" }, { $unwind: "$friend_name" }//, { $skip: 1}, {$limit: 1}
])
.exec();
console.log(cc);
});
This is the most relevant part:
var cc = await mongoose
.model("User")
.aggregate([
{ $match: { _id: john._id } },
{ $unwind: "$friends" },
{
$lookup: {
from: "places",
localField: "friends",
foreignField: "creator",
as: "friends_places",
}
}, { $lookup: {from: 'users', localField: 'friends', foreignField: '_id', as: 'friend_name'} },//, { $include: 'friends' }
{ $unwind: "$friends_places" }, { $unwind: "$friend_name" }//, { $skip: 1}, {$limit: 1}
])
.exec();
The first unwind: friends 'flattens' the collections initially. So, basically we've got 'userId | friendId' for each user and friend. Then, for each row, we simply look up places created by him ($lookup). Finally, we unwind friends_places because we don't want them to be rendered as [object] in console output. Additionally, there this $match, because we only want to check one user's friends' places. Considering we want to know friend's name as well, we have to do one more join - this is why there's this second $lookup. After that a simple $unwind to get friend's detail and that's it.
Code yields the following:
[ { _id: 5f3aa3a9c6140e3344c78a45,
friends: 5f3aa3a9c6140e3344c78a44,
name: 'John',
__v: 0,
friends_places:
{ _id: 5f3aa3a9c6140e3344c78a48,
latitude: 'Place2',
creator: 5f3aa3a9c6140e3344c78a44,
__v: 0 },
friend_name:
{ _id: 5f3aa3a9c6140e3344c78a44,
friends: [Array],
name: 'Natalie',
__v: 0 } },
{ _id: 5f3aa3a9c6140e3344c78a45,
friends: 5f3aa3a9c6140e3344c78a43,
name: 'John',
__v: 0,
friends_places:
{ _id: 5f3aa3a9c6140e3344c78a46,
latitude: 'Place1',
creator: 5f3aa3a9c6140e3344c78a43,
__v: 0 },
friend_name:
{ _id: 5f3aa3a9c6140e3344c78a43,
friends: [],
name: 'Bob',
__v: 0 } },
{ _id: 5f3aa3a9c6140e3344c78a45,
friends: 5f3aa3a9c6140e3344c78a43,
name: 'John',
__v: 0,
friends_places:
{ _id: 5f3aa3a9c6140e3344c78a47,
latitude: 'Place3',
creator: 5f3aa3a9c6140e3344c78a43,
__v: 0 },
friend_name:
{ _id: 5f3aa3a9c6140e3344c78a43,
friends: [],
name: 'Bob',
__v: 0 } } ]
So, we've got a flat list of John's friends' places.
Important bit: from: places in $lookup is critical, as we have to use MongoDB's name of the collection, not model name 1.

Resources