How to do nested pagination on mongoDB aggregation - node.js

I have 3 main collections
Blogs{
id,
title
}
Comments{
id,
message,
blog_id
}
Likes{
id,
user_id,
blog_id
}
I want to fetch blogs, their comments, and likes altogether. Comments should come in pagination 5 at a time and blog posts are also paginated so also keep in mind that User might just want more comments to show and not Blog posts. So how to paginate comments without affecting blog's array?
Expected output:
{
_id: "POST_ID"
title: "STRING",
comments: [
{
commentMessage: "",
userName: ""
},
{
commentMessage: "",
userName: ""
}
]
}
Sample Document:
{
"_id": "5f987b80811aa021ecce5566",
"title": "nodejs2",
"comments": [
{
"commentMessage": "this is comment message 2",
"userName": "user2",
"createdTime": 123456789
},
{
"commentMessage": "this is comment message 3",
"userName": "user2",
"createdTime": 124456789
},
{
"commentMessage": "this is comment message 6",
"userName": "user2",
"createdTime": 123456789
}
]
},
I have got the expected output but unable to figure out where would I put $skip and $limit for comment's array.
Here is my sample code, the question is where should I put $skip and $limit to paginate comments
BlogPost.aggregate([
{ $match: { $or: [{ title: req.body.title }, { tags: req.body.tag }] } },
{
$lookup: {
from: PostComment.collection.name,
localField: "_id",
foreignField: "postId",
as: "comments"
},
},
{
$lookup: {
from: PostLike.collection.name,
localField: "_id",
foreignField: "postId",
as: "blog_like"
}
},
{
$lookup: {
from: User.collection.name,
localField: "comments.userId",
foreignField: "_id",
as: "UserDetail",
},
},
{
$unwind: "$UserDetail"
},
{
$project:
// working code, not an issue
},
{ $skip: 5}, // for blogs <=========== if this is wrong, let me know
{ $limit: 5} // for blogs <=========== if this is wrong, let me know
]).exec((err, blogs) => {
if (err) throw err
console.log("blogs: ", blogs)
return res.send(blogs)
})

Related

Mongoose - How to get unique data based on some fields using aggregation

I have these fields in the document,
doc: {
"id": "632ac8cba7723378033fef10",
"question": 1,
"text": "aasdfghjk,mnbvcxswertyuikmnbvcxsrtyuiknbvcdrtyujnbvcddtyjnbvfty",
"slug": "xcvbnrddfghjktdxjjydcvbyrsxcvbhytrsxggvbjkytrdgc",
"subject": 25866,
"tutorInfo": {
"tutorId": "632ac8cba7723378033fa0fe",
"tutorIncrementalId": 95947
}
}
the same tutorInfo can Occur in multiple documents.
const allQuestionBySubject = await QuestionParts.aggregate([
{
$match: {
$and: [
{
subject: subjectIncrementalId
},
{tutorInfo: {$exists: true}}
]
}
},
{ "$skip": page * limit },
{ "$limit": limit },
{
$lookup: {
from: "profiles",
localField: "tutorInfo.tutorIncrementalId",
foreignField: "incrementalId",
as: "tutorDetails"
}
}
])
Code to get a list of questions as per subject.
I am filtering documents based on subject and as I mentioned the same tutorInfo can be present in multiple documents so in the result same tutor can be present in multiple documents, How can I get a unique list of documents in which tutorInfo shouldn't be repeated.
Since the same tutorInfo is present in multiple records, You can use $group to group the document on the tutorInfo.tutorId field.
const allQuestionBySubject = await QuestionParts.aggregate(
[
{
$match: {
$and: [
{
subject: subjectIncrementalId
},
{ tutorInfo: { $exists: true } }
]
}
},
{ "$skip": page * limit },
{ "$limit": limit },
{
"$group": {
_id: "$tutorInfo.tutorId",
question: { $first: "$question" },
text: { $first: "$text" },
slug: { $first: "$slug" },
subject: { $first: "$orderId" },
tutorInfo: { $first: "$tutorInfo" },
}
},
{
$lookup: {
from: "profiles",
localField: "tutorInfo.tutorIncrementalId",
foreignField: "incrementalId",
as: "tutorDetails"
}
}
]
)

How can I make lookup on in with productId and categoryId

How can make I lookup on in with productId and categoryId.
Below is the sample of JSON
"data": {
"status": 1,
"offerId": "634017ad34c7b3545bd47681",
"offerType": "buyOneGetOne",
"categoryId": [
"62de9b22dcafd44290ab03c1",
"62dfe695ec4d69cfeab0265d"
],
"productId": [
"633170230e88e2d859d15ee6",
"62de9d102b2b72f80eda7585",
"62de9d1e2b2b72f80eda7588"
],
"title": "Buy One Get One Deal ",
"description": "buy one get more",
"amountType": "amount",
"amount": 12341,
"startDate": "2022-10-04T06:43:39.000Z",
"endDate": "2022-10-08T06:43:39.000Z"
}
I'm going to assume that you are using ObjectId as _id in your categories and products collection. If I understood your question your document as the following minimal structure:
_id: '1',
data: {
...
}
So, you are looking for something like:
db.nameOfYourCollectionWithData.aggregate([{
$addFields: {
'data.categoryId': {
$map: {
input: '$data.categoryId',
'in': {
$toObjectId: '$$this'
}
}
},
'data.productId': {
$map: {
input: '$data.productId',
'in': {
$toObjectId: '$$this'
}
}
}
}
}, {
$lookup: {
from: 'categories',
localField: 'data.categoryId',
foreignField: '_id',
as: 'data.categoryId'
}
}, {
$lookup: {
from: 'products',
localField: 'data.productId',
foreignField: '_id',
as: 'data.productId'
}
}])
First, you have to convert your array ids into ObjectId. After do this, you can perform the lookup stages directly. If your document is 'data', just remove any reference to data in the aggregate pipeline.
If your _id in categories and products are string instead of ObjectId, just remove the $addFields stage, because is no longer neccessary.

mongodb (aggregation) - $lookup with the lookup result

i am new in aggregation framework and have the following problem/question.
My collections looks like this:
users
_id: ObjectId('604caef28aa89e769585f0c4')
baseData: {
firstName: "max"
username: "max_muster"
...
...
}
activitiesFeed
_id: ObjectId('604cb97c99bbd77b54047370')
userID: "604caef28aa89e769585f0c4" // the user id
activity: "604caf718aa89e769585f0c8" // the activity id
viewed: false
activities
_id: ObjectId('604caf718aa89e769585f0c8')
"baseData": {
"userID":"604caef28aa89e769585f0c4",
"type":0,
"title":"Alessandro send you a friend request.",
"description":"",
"actionID":"604caef28aa89e769585f0c4"
},
aggregate function
const response = ActivityFeed.aggregate([
{
$match: { userID: ObjectId(args.user) },
},
{
$lookup: {
from: 'activities',
localField: 'activity',
foreignField: '_id',
as: 'lookup_activities',
},
},
{ $unwind: '$activities' },
{
$lookup: {
from: 'users',
localField: 'lookup_activities.baseData.actionID',
foreignField: '_id',
as: 'lookup_users',
},
},
]);
How i can made a lookup with the first lookup result?
I made a lookup to the activities collection. This collection holds the id for the second lookup baseData.userID. But with this approach i have no result with the second lookup.
The reason why i made a join to the activities collection is that the actionID can hold a userID or a document id from another collection. It depend on the type in the activities collection.
Is it in the aggregation framework possible to made a lookup which depends on the type and of the previous lookup?
Thank you in advance.
All I did is fixing your aggregation. You unwind the wrong field and please make sure all your ID fields are ObjectId. I highly recommend using MongoDB Compass to assist your aggregation if you are new to the aggregation function. They have stage by stage assistance GUI which will really make your life easier.
mongoplayground
db.activitiesFeed.aggregate([
{
$match: {
userID: ObjectId("604caef28aa89e769585f0c4")
},
},
{
$lookup: {
from: "activities",
localField: "activity",
foreignField: "_id",
as: "lookup_activities",
},
},
{
$unwind: "$lookup_activities"
},
{
$lookup: {
from: "users",
localField: "lookup_activities.baseData.actionID",
foreignField: "_id",
as: "lookup_users",
},
},
])
Sample Used
db={
"users": [
{
_id: ObjectId("604caef28aa89e769585f0c4"),
baseData: {
firstName: "max",
username: "max_muster"
}
}
],
"activitiesFeed": [
{
_id: ObjectId("604cb97c99bbd77b54047370"),
userID: ObjectId("604caef28aa89e769585f0c4"),
activity: ObjectId("604caf718aa89e769585f0c8"),
viewed: false
}
],
"activities": [
{
_id: ObjectId("604caf718aa89e769585f0c8"),
"baseData": {
"userID": ObjectId("604caef28aa89e769585f0c4"),
"type": 0,
"title": "Alessandro send you a friend request.",
"description": "",
"actionID": ObjectId("604caef28aa89e769585f0c4")
},
}
]
}

How to lookup inside lookup in MongoDB Aggregate?

I have a simple 3 collections. This bellow is their pseudocode. I want to get all shipments and for each shipment, I want to have all bids for that shipment and in each bid, I need userDetails object.
User: {
name: string,
}
Shipment: {
from: string,
to: string
}
Bid: {
amount: number,
shipmentId: Ref_to_Shipment
userId: Ref_to_User
}
This is what I have tried:
const shipments = await ShipmentModel.aggregate([
{
$lookup: {
from: "bids",
localField: "_id",
foreignField: "shipmentId",
as: "bids"
}
},
{
$lookup: {
from: "users",
localField: "bids.userId",
foreignField: "_id",
as: "bids.user"
}
}
])
And I got the following result:
[
{
"_id": "5fad2fc04458ac156531d1b1",
"from": "Belgrade",
"to": "London",
"__v": 0,
"bids": {
"user": [
{
"_id": "5fad2cdb4d19c80d1b6abcb7",
"name": "Amel",
"email": "Muminovic",
"password": "d2d2d2",
"__v": 0
}
]
}
}
]
I am trying to get all Shipments with their bids and users within bids. Data should look like:
[
{
"_id": "5fad2fc04458ac156531d1b1",
"from": "Belgrade",
"to": "London",
"__v": 0,
"bids": [
{
"_id": "5fad341887c2ae1feff73402",
"amount": 400,
"userId": "5fad2cdb4d19c80d1b6abcb7",
"shipmentId": "5fad2fc04458ac156531d1b1",
"user": {
"name": "Amel",
}
"__v": 0
}
]
}
]
Try with the following code:
const shipments = await ShipmentModel.aggregate([
{
$lookup: {
from: "bids",
localField: "_id",
foreignField: "shipmentId",
as: "bids"
}
},
{
$unwind: {
path: "$bids",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "users",
localField: "bids.userId",
foreignField: "_id",
as: "bids.user"
}
}
])
If you want to prevent null and empty arrays then set
preserveNullAndEmptyArrays: false
Try this query and chek if works and the behaviour is as you expected:
db.Shipment.aggregate([
{
$lookup: {
from: "Bid",
localField: "id",
foreignField: "shipmentId",
as: "bids"
}
},
{
$lookup: {
from: "user",
localField: "id",
foreignField: "id",
as: "newBids"
}
},
{
$project: {
"newBids.id": 0,
"newBids._id": 0,
}
},
{
$match: {
"bids.userId": 1
}
},
{
$addFields: {
"newBids": {
"$arrayElemAt": [
"$newBids",
0
]
}
}
},
{
$set: {
"bids.user": "$newBids"
}
},
{
$project: {
"newBids": 0
}
}
])
This query do your double $lookup and then a $project to delete the fields you don't want, and look for the userId to add the field user. As $lookup generate an array, is necessary use arrayElemAt to get the first position.
Then $set this value generated into the object as bids.user and remove the old value.
Note that I have used a new field id instead of _id to read easier the data.
Try this
I figured out it based on MongoDB $lookup on array of objects with reference objectId and in the answer from J.F. (data organization). Note that he used id instead of _id
The code is
db.Shipment.aggregate([
{
$lookup: {
from: "Bid",
localField: "id",
foreignField: "shipmentId",
as: "bids"
}
},
{
$lookup: {
from: "user",
localField: "bids.userId",
foreignField: "id",
as: "allUsers"
}
},
{
$set: {
"bids": {
$map: {
input: "$bids",
in: {
$mergeObjects: [
"$$this",
{
user: {
$arrayElemAt: [
"$allUsers",
{
$indexOfArray: [
"$allUsers.id",
"$$this.userId"
]
}
]
}
}
]
}
}
}
}
},
{
$unset: [
"allUsers"
]
},
// to get just one
//{
// $match: {
// "id": 1
// }
// },
])

Best a very overloaded query mongodb - Nodejs; aggregate, group, porject and lookup

its my first question in this forum. (I am a Spanish speaker, sorry for my basic English)
I have 3 collections: images, artists and exhibitions.
Images has: filename (unique), path and _id and etc.
Artists has: name, url (unique) and description and etc
Exhibitions has: name, url (unique), year, abstract, aristUrl (artist.url) imageCover (image.fieldname) and etc.
Whit my query this is my summary result:
[
_id: null,
documents: [{...}, {...}],
totalExhibitions: 2
]
Full result
[
{
"_id": null,
"documents": [
{
"_id": "5e84d6599891212db0a6dc7e",
"url": "chile",
"imageCover": [
{
"path": "http://localhost:2616/uploads/images/1585763637647.jpg"
}
],
"name": "Almacén Verdad y Justicia",
"year": 2010,
"releaseDate": "2010-08-30T00:00:00.000Z",
"artist": [
{
"name": "Bernardo de Castro Saavedra",
"url": "bernardo-castro-saavedra"
}
]
},
{
"_id": "5e84e0575a3f201aac2df1c2",
"url": "sin-cera",
"imageCover": [
{
"path": "http://localhost:2616/uploads/images/1585766437587.jpg"
}
],
"name": "Sin cera",
"year": 2020,
"releaseDate": "2020-01-31T00:00:00.000Z",
"artist": [
{
"name": "Gonzalo Tapia",
"url": "gonzalo-tapia"
}
]
}
],
"totalExhibitions": 2
}
]
This is my code
getByLastYear(){
const documents = this.mongoDB.aggregate([
{
$addFields: {
"artistObjectId": { $toObjectId: "$artistId" }
}
},{
$lookup: {
from: 'artists',
localField: 'artistObjectId',
foreignField: "_id",
as: "artist"
},
},{
$lookup: {
from: 'images',
localField: 'imageCover',
foreignField: "filename",
as: "imageCover"
},
},{
$project: {
name: 1,
year: 1,
url: 1,
releaseDate: 1,
artist: {
name: 1,
url: 1
},
imageCover: {
path: 1,
alt: 1,
}
}
}, {
$group: {
_id: null,
documents: {
$push: "$$ROOT"
},
totalExhibitions: {
$sum: 1
}
}
}
]);
return documents || [];
};
that is the best form to get my result?
is there any better?
Thank you for your comments and opinions.<3
Assuming that:
you're starting from the exhibitions collection;
the "_id" fields have an ObjectId format;
there's only 1 artist and 1 cover image for each exhibition...
I'd skip the first step and add in $unwind stages to have the artist and the cover image as a subdocument instead of an array.
The reason for that is that it will make it easier to reference the result.
This should be able to run in your Mongo shell:
db.exhibitions.aggregate([
{ $lookup: {
from: 'artists',
localField: "artistId",
foreignField: "_id",
as: "artist"
}},
{ $unwind: {
path: "$artist",
preserveNullAndEmptyArrays: true
}},
{ $lookup: {
from: 'images',
localField: 'imageCover',
foreignField: "filename",
as: "imageCover"
}},
{ $unwind: {
path: "$imageCover",
preserveNullAndEmptyArrays: true
}},
{ $project: {
name: 1,
year: 1,
url: 1,
releaseDate: 1,
artist: {
name: "$artist.name",
url: "$artist.url"
},
imageCover: {
path: "$imageCover.path",
alt: "$imageCover.alt",
}
}},
{ $group: {
_id: null,
documents: {
$push: "$$ROOT"
},
totalExhibitions: {
$sum: 1
}
}}
]);
In the "$project" stage, for artist and imageCover, you would need to specify the full path to build the reduced subdocument.
Hope that answered your question...

Resources