I have 3 collections: Categories, Exams and Results.
For each Category I have some exams. If one exam is done, I add its result on "Result" collection.
I have take all exams for each category.
The problem is that I can't count how many of them are done.
This is what I get right now:
{ category: "Test", total_exams: 10 }
This is what I want:
{ category: "Test", total_exams: 10, done_exam: 4 }
Below you can see what I have done until now:
https://mongoplayground.net/p/upSAPkYe4DR
Quick fixes:
second $lookup stage, let to pass examsCount._id,
inside pipeline $match stage match expression, $exam_id with $$examId
put user_id match condition outside expression
$count total documents in count
$addFieldsto getdoneExams.countthis will return count in zero index, get it using$firstor$arrayElemAt`
db.categories.aggregate([
{
$lookup: {
from: "exams",
localField: "_id",
foreignField: "categoryId",
as: "examsCount"
}
},
{
$lookup: {
from: "results",
let: { examId: "$examsCount._id" },
pipeline: [
{
$match: {
$expr: {
$and: [{ $in: ["$exam_id", "$$examId"] }]
},
user_id: 22222222
}
},
{ $count: "count" }
],
as: "doneExams"
}
},
{
$addFields: {
examsCount: { $size: "$examsCount" },
doneExams: { $first: "$doneExams.count" }
}
}
])
Playground
There are many approaches to get this result, i have just resolved your issue in your approach.
Related
I'm trying to adapt my code so I could after use the pipeline to $project only the fields I need.
This first code is achieving the result expected;
Order.aggregate([
{ $unwind: "$candidate" },
{ $match: { "candidate.groupId": "A" } },
{
$lookup: {
from: "users",
localField: "candidate.autonomousId",
foreignField: "_id",
as: "candidateData",
},
},
]).toArray();
The second one doesn't return the same output. It returns an empty candidateData.
Order.aggregate([
{ $unwind: "$candidate" },
{ $match: { "candidate.groupId": "A" } },
{
$lookup: {
from: "users",
let: { id: "$candidate.autonomousId" },
pipeline: [{ $match: { $expr: { $eq: ["$_id", "$$id"] } } }],
as: "candidateData",
},
},
]).toArray();
What am I doing wrong when I try to write in the second way? I couldn't find my syntax mistake.
Since candidate.autonomousId is an array (contrary to its name) and you are using the $eq operator to compare $$id (array) with _id (ObjectId), the pipeline returns no User documents in candidateData. You should use the $in aggregation operator instead.
db.orders.aggregate([
{
$unwind: "$candidate"
},
{
$match: {
"candidate.groupId": "A"
}
},
{
$lookup: {
from: "users",
let: {
id: "$candidate.autonomousId"
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$_id",
"$$id"
]
}
}
}
],
as: "candidateData",
},
},
])
MongoPlayground
I would recommend renaming the let variable ids for clarity.
I have the following documents
loanRequest (Writing just the keys that I want to project)
{
"_id": "5f2bf26783f65d33026ea592",
"lendingpartner": {
/* some keys here */
},
"loans": [
{
"loanid": 43809,
"loanamount": 761256,
"jewels": [
"5f2bf26783f65d33026ea593",
"5f2bf26783f65d33026ea594"
"5f2bf26783f65d33026ea595"
],
}
]
}
pledgedJewel
{
"_id": "5f2bf26783f65d33026ea593",
"netweight": 8.52,
"purity": 19,
}
What I want to achieve is
{
"_id": "5f2bf2b583f65d33026ea603",
"lendingpartner": {
/* some keys here */
},
"loans": [
{
"loanid": 40010,
"loanamount": 100000,
"jewels": [
{
"_id": "5f2bf26783f65d33026ea593",
"netweight": 8.52,
"purity": 19,
},
{
"_id": "5f2bf26783f65d33026ea594",
"netweight": 5.2,
"purity": 40,
},
{
"_id": "5f2bf26783f65d33026ea595",
"netweight": 4.52,
"purity": 39,
}
]
}
]
}
Since I want the jewel details to be populated inside the jewels array of each loan, $unwind would not help me. (I tried experimenting with it)
I thought I could run a $map on loans array and then run $lookup for each jewel of the loan(double map?), but could not come up with a workable solution.
That didn't seem to be the right approach anyway.
This is the best I could come up with (Far from my desired result). I'm using map to selectively pick keys from loans object.
const loanrequests = await db.collection('loanrequest').aggregate([
{ $match: { requester: ObjectID(user.id) } },
{
$project: {
lendingpartner: {
name: 1,
branchname: '$branch.branchname',
},
loans: {
$map: {
input: '$loans',
as: 'loan',
in: {
loanid: '$$loan.loanid',
loanamount: '$$loan.amount',
jewels: '$$loan.jewels',
},
},
},
},
},
/*
* I experimented with unwind here. Tried adding
* { $unwind: '$loans' },
* { $unwind: '$loans.jewels' }
* but it does not give me the result I need (as already said before)
*/
]).toArray();
I figure, I need to do the $lookup before the projection, but I'm it hard to write a workable solution due to 2 level nested structure of the document (First, the loans array and then loans.jewels)
I started working with mongodb aggregators today and while looking for answers, I stumbled upon a similar Question but it seemed more complex and hence harder for me to understand.
Thanks!
If there are not other things you are trying to achieve with aggregate you can use .populate in mongoose.
LoanReqests
.find(
{requester: user.id},
{name: 1, branch: 1, loans: 1} // Projection
)
.populate('loans.jewels');
If you have to use aggregate to do something not in your example, then $unwind is really your best bet, but then $grouping after the $lookup to get the output you desire. If this doesn't work for you, can you expand on what the issue with $unwind is? I am guessing it is to do with fields not listed in your question.
https://mongoplayground.net/p/O5pxWNy99J4
db.loanRequests.aggregate([
{
$project: {
name: 1,
loans: 1,
branch: "$branch.name"
}
},
{
$unwind: "$loans"
},
{
$lookup: {
localField: "loans.jewels",
foreignField: "_id",
from: "jewels",
as: "loans.jewels"
}
},
{
$group: {
_id: "$_id",
name: {
$first: "$name"
},
branch: {
$first: "$branch"
},
loans: {
$push: "$loans"
}
}
}
])
As mentioned by #GitGitBoom in the previous answer, $unwind followed by $group should have been the approach.
Ofcourse, prior to grouping (I think of it as "unspreading" the outcome of running unwind), I needed to run $lookup in order to populate loans.jewels
Here is the entire solution build on top of the previous answer.
const loanRequests = await db.collection('loanRequest').aggregate([
{ $match: { requester: ObjectID(user.id) } },
{
$project: {
lender: '$lendingpartner.name',
branch: '$lendingpartner.branch.branchname',
loans: 1,
},
},
{ $unwind: '$loans' },
{
$lookup: {
localField: 'loans.jewels',
from: 'pledgedJewel',
foreignField: '_id',
as: 'loans.jewels',
},
},
{
$group: {
_id: '$_id',
branch: { $first: '$branch' },
lender: { $first: '$lender' },
loans: { $push: '$loans' },
},
},
{
$project: {
_id: 1,
branch: 1,
lender: 1,
loans: 1,
},
},
]).toArray();
Issue with mismatch of types
Another issue was, my $lookup was not working due to mismatch of types. In loanRequest collection, on which I'm running the aggregate, the ids inside loans.jewels are of type string whereas the foreign field _id in pledgedJewel is an ObjectId
This can be solved by using $toObjectId or $toString (only supported in mongodb version >= 4.0)
{ $project: { jewelObjId: { $toObjectId: '$loans.jewels' } } }, // for mongodb >= 4.0
{
$lookup: {
localField: 'jewelObjId', // for mongodb >= 4.0
from: 'pledgedjewel',
foreignField: '_id',
as: 'loans.jewels',
},
},
But, I was running on a lower version of mongodb, hence these aggregations were not working for me. Only solution to this was to change the type of loans.jewels to ObjectId instead of keeping it as string which I did.
More on type mismatch
Need a workaround for lookup of a string to objectID foreignField
Mongodb Join on _id field from String to ObjectId
I'm trying to use $lookup aggregation in order to know how many documents in the other collection, so no need to to grab the whole documents, which's, in this case, will give me an exception because the size of the total document exceeded the 16MB limit.
the exception I got
MongoError: Total size of documents in visits matching pipeline's $lookup stage exceeds 104857600 bytes
the query
Link.aggregate(
[
{$match: {...query}},
{
$lookup: {
from:"visits",
localField: "_id",
foreignField: "linkId",
as: "visits"
}
},
{
$addFields: { totalVisits: { $size: "$visits" } },
},
{
$sort: {totalVisits: -1}
}
]
)
You can return $count from custom $lookup pipeline:
{
$lookup: {
from: "visits"
let: { id: "$_id" },
pipeline: [
{ $match: { $expr: { $eq: [ "$$id", "$linkId" ] } } },
{ $count: "total" }
],
as: "totalVisits"
}
}
{
$lookup: {
from: "Comment",
let: {
p_id: "$_id",
d_id: "$data_id",
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$_id",
"$$p_id"
]
},
{
$eq: [
"$data_id",
"$$d_id"
]
}
]
}
}
}
],
as: "subComment"
}
}
https://mongoplayground.net/p/GbEgnVn3JSv
I am good at mongoplayground but tried to put there my thought
I want to fetch the comment of posts based on doc_id and post_id for mainComment query looks good to me but subcommand is not good. Please guide on this
Its simple as a post can have multiple comment need comment count base on Post.data._id which is equal to Comment.doc_id and Post._id is in Comment.post_id
Not sure what "mainComment" and "subComment" are, I believe you missed the dollar sign before them
{
$project: {
_id: 1,
main_comments_count: {
$size: "$mainComment"
},
sub_comments_count: {
$size: "$subComment"
},
}
}
Update
What you did wrong in the playground is that you used $data in the lookup.let stage. $data is a document and the field you actually want to lookup is $data._id.
sidenote: if you are looking up using just one field, you can simply use the localField and foreign in the lookup stage. Using let and pipeline is not necessary there.
db.setting.aggregate([
{
$lookup: {
from: "site",
"let": {
"pid": "$data._id" //here
},
"pipeline": [
{
"$match": {
"$expr": {
"$in": [
"$doc_id",
"$$pid"
]
}
}
}
],
"as": "subComment"
}
},
{
$addFields: {
countRecord: "$subComment"
}
}
])
i.e. this gives the same output
db.setting.aggregate([
{
$lookup: {
from: "site",
localField: "data._id",
foreignField: "doc_id",
as: "subComment"
}
},
{
$addFields: {
countRecord: "$subComment"
}
}
])
I am aggregating a large data where i need to group the data according to their types and also i need to lookup the data from another collections.inside $group i want my lookup's data.
my code for aggregation goes like :
NotificationSchema.aggregate([{
$match: condition
}, {
$group: {
_id: "$type",
details: {
$push: "$$ROOT"
},
count: {
$sum: 1
}
}
}, {
$sort: {
_id: -1
}
}, {
$lookup: {
from: "vehicles",
localField: "details.device_id",
foreignField: "device_id",
as: "vehicle"
}
}], function(err, result) {
if (err) {
res.status(500);
return res.json({
result: err
});
}
console.log('res', result[0].details[0]);
res.json({
result: result
});
});
if i remove or comment the $group code i get the data with Vehicle array but using $group i get vehicle array empty, as i have only two types in records in the database, i get two empty array of vehicles. but i have 102 records so i need 102 arrays of vehicles how can i get such result.
what i am getting in console right now is
res [ { _id: 'Vehicle Delay Alert!',
details:
[ [Object],
....57 object...
[Object] ],
count: 57,
vehicle: [] },
and inside every object i dont find vehicle array so i wish to remove vehicle array from here and get a vehicle array that is generated from $lookup inside every object.
Any suggestions are highly appreciated.
You $lookup from details.device_id which comes from an array. To $lookup from a regular field, you can place $lookup after the $match :
NotificationSchema.aggregate([{
$match: condition
}, {
$lookup: {
from: "vehicles",
localField: "device_id",
foreignField: "device_id",
as: "vehicle"
}
}, {
$group: {
_id: "$type",
details: {
$push: "$$ROOT"
},
count: {
$sum: 1
}
}
}, {
$sort: {
_id: -1
}
}])