I'm trying to use $lookup aggregation in order to know how many documents in the other collection, so no need to to grab the whole documents, which's, in this case, will give me an exception because the size of the total document exceeded the 16MB limit.
the exception I got
MongoError: Total size of documents in visits matching pipeline's $lookup stage exceeds 104857600 bytes
the query
Link.aggregate(
[
{$match: {...query}},
{
$lookup: {
from:"visits",
localField: "_id",
foreignField: "linkId",
as: "visits"
}
},
{
$addFields: { totalVisits: { $size: "$visits" } },
},
{
$sort: {totalVisits: -1}
}
]
)
You can return $count from custom $lookup pipeline:
{
$lookup: {
from: "visits"
let: { id: "$_id" },
pipeline: [
{ $match: { $expr: { $eq: [ "$$id", "$linkId" ] } } },
{ $count: "total" }
],
as: "totalVisits"
}
}
Related
I am trying to get the data from another collection via lookup
collection "users"
{
"_id":{
"$oid":"60bf4bb31f45d98903d1851f"
},
"name":"Dave",
"center":"THGJ556",
}
collection "addresses"
{
"_id":{
"$oid":"60bf4bb31f45d98903d1851f"
},
"userId":"60bf4bb31f45d98903d1851f",
}
collection "applications"
{
"_id":{
"$oid":"60bf4bb31f45d98903d1851f"
},
"userId":"60bf4bb31f45d98903d1851f",
"centerId":"THGJ556",
},
{
"_id":{
"$oid":"60bf4bb31f45d98903d3647j"
},
"userId":"60bf4bb31f45d98903d1851f",
"centerId":"JHGJ5476",
}
Now I need data from all the tables.
here is my code:
users.aggregate([
{
$lookup: {
from: "addresses",
localField: "_id",
foreignField: "userId",
as: "addressData"
}
},
{
$lookup: {
from: "applications",
pipeline: [
{ $match:
{ userId:"$_id", centerId: "JHGJ5476"}
},
],
as: "applicationData"
}
},
] ,function(err, result) {
if (err) {
console.log(err)
} else {
console.log(result)
}
});
I am doing something wrong while using aggregate and match in pipeline.
I am getting addressData correctly, but I get nothing [] in applicationData because I suspect something is wrong with userId:"$_id"
As docs explain:
A $match stage requires the use of an $expr operator to access the variables. $expr allows the use of aggregation expressions inside of the $match syntax.
So you have to use $expr into $match stage and also a let stage.
let stage is to define variable to use into $expr: id: $_id.
$expr used with $and and $eq is to get both conditions.
db.users.aggregate([
{
$lookup: {
from: "addresses",
localField: "_id",
foreignField: "userId",
as: "addressData"
}
},
{
$lookup: {
from: "applications",
let: {"id": "$_id"},
pipeline: [
{
$match: {
"$expr": {
"$and": [
{"$eq": ["$userId","$$id"]},
{"$eq": ["$centerId","JHGJ5476"]}
]
}
}
}
],
as: "applicationData"
}
}
])
Check this example.
I'm trying to adapt my code so I could after use the pipeline to $project only the fields I need.
This first code is achieving the result expected;
Order.aggregate([
{ $unwind: "$candidate" },
{ $match: { "candidate.groupId": "A" } },
{
$lookup: {
from: "users",
localField: "candidate.autonomousId",
foreignField: "_id",
as: "candidateData",
},
},
]).toArray();
The second one doesn't return the same output. It returns an empty candidateData.
Order.aggregate([
{ $unwind: "$candidate" },
{ $match: { "candidate.groupId": "A" } },
{
$lookup: {
from: "users",
let: { id: "$candidate.autonomousId" },
pipeline: [{ $match: { $expr: { $eq: ["$_id", "$$id"] } } }],
as: "candidateData",
},
},
]).toArray();
What am I doing wrong when I try to write in the second way? I couldn't find my syntax mistake.
Since candidate.autonomousId is an array (contrary to its name) and you are using the $eq operator to compare $$id (array) with _id (ObjectId), the pipeline returns no User documents in candidateData. You should use the $in aggregation operator instead.
db.orders.aggregate([
{
$unwind: "$candidate"
},
{
$match: {
"candidate.groupId": "A"
}
},
{
$lookup: {
from: "users",
let: {
id: "$candidate.autonomousId"
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$_id",
"$$id"
]
}
}
}
],
as: "candidateData",
},
},
])
MongoPlayground
I would recommend renaming the let variable ids for clarity.
I have 3 collections: Categories, Exams and Results.
For each Category I have some exams. If one exam is done, I add its result on "Result" collection.
I have take all exams for each category.
The problem is that I can't count how many of them are done.
This is what I get right now:
{ category: "Test", total_exams: 10 }
This is what I want:
{ category: "Test", total_exams: 10, done_exam: 4 }
Below you can see what I have done until now:
https://mongoplayground.net/p/upSAPkYe4DR
Quick fixes:
second $lookup stage, let to pass examsCount._id,
inside pipeline $match stage match expression, $exam_id with $$examId
put user_id match condition outside expression
$count total documents in count
$addFieldsto getdoneExams.countthis will return count in zero index, get it using$firstor$arrayElemAt`
db.categories.aggregate([
{
$lookup: {
from: "exams",
localField: "_id",
foreignField: "categoryId",
as: "examsCount"
}
},
{
$lookup: {
from: "results",
let: { examId: "$examsCount._id" },
pipeline: [
{
$match: {
$expr: {
$and: [{ $in: ["$exam_id", "$$examId"] }]
},
user_id: 22222222
}
},
{ $count: "count" }
],
as: "doneExams"
}
},
{
$addFields: {
examsCount: { $size: "$examsCount" },
doneExams: { $first: "$doneExams.count" }
}
}
])
Playground
There are many approaches to get this result, i have just resolved your issue in your approach.
{
$lookup: {
from: "Comment",
let: {
p_id: "$_id",
d_id: "$data_id",
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$_id",
"$$p_id"
]
},
{
$eq: [
"$data_id",
"$$d_id"
]
}
]
}
}
}
],
as: "subComment"
}
}
https://mongoplayground.net/p/GbEgnVn3JSv
I am good at mongoplayground but tried to put there my thought
I want to fetch the comment of posts based on doc_id and post_id for mainComment query looks good to me but subcommand is not good. Please guide on this
Its simple as a post can have multiple comment need comment count base on Post.data._id which is equal to Comment.doc_id and Post._id is in Comment.post_id
Not sure what "mainComment" and "subComment" are, I believe you missed the dollar sign before them
{
$project: {
_id: 1,
main_comments_count: {
$size: "$mainComment"
},
sub_comments_count: {
$size: "$subComment"
},
}
}
Update
What you did wrong in the playground is that you used $data in the lookup.let stage. $data is a document and the field you actually want to lookup is $data._id.
sidenote: if you are looking up using just one field, you can simply use the localField and foreign in the lookup stage. Using let and pipeline is not necessary there.
db.setting.aggregate([
{
$lookup: {
from: "site",
"let": {
"pid": "$data._id" //here
},
"pipeline": [
{
"$match": {
"$expr": {
"$in": [
"$doc_id",
"$$pid"
]
}
}
}
],
"as": "subComment"
}
},
{
$addFields: {
countRecord: "$subComment"
}
}
])
i.e. this gives the same output
db.setting.aggregate([
{
$lookup: {
from: "site",
localField: "data._id",
foreignField: "doc_id",
as: "subComment"
}
},
{
$addFields: {
countRecord: "$subComment"
}
}
])
I am having issues with $aggregate with mongoose. I am using $lookup to get the document from the raps table, but as the size of the raps documents with matching userId is huge it fails every time.
errmsg: 'Total size of documents in raps matching { $match: { $and: [ { owner._id: { $eq: "ID" } }, {} ] } }
exceeds maximum document size',
code: 4568
I have tried with allowDiskUse and it didn't work also tried $out and adding it into temp table, didn't work either.
I have also tried using $unwind after the $lookup as mentioned in one of the answer, but it doesn't seem to work.
I have the following snippet.
userAccountModel.aggregate([
{
$match:
{
_id: userId
}
},
{
$lookup: {
from: "raps",
localField: "_id",
foreignField: "owner._id",
as: "rapsDocs"
}
},
{
$project : {
"likes": { $sum: "$rapsDocs.likes" }
}
}
]).allowDiskUse(true).exec(function(err, result){
})