So what I currently have is working. But what I want in addition is, that I get in the same query the matched data from Collection C with the currently logged in user.
So the point is the following: A user can have multiple CollectionA's. And users can join the CollectionA, which is specified in CollectionB.
In the aggregation I get every joined user. But what I want in addition is, that I can see the currently logged in users data from Collection B.
CollectionA.aggregate([
{
$lookup: {
from: "collectionB",
localField: "_id",
foreignField: "col_a_id",
as: "colB",
},
},
{
$match: {
"colB.user_id": mongoose.Types.ObjectId(
request.user.id
),
},
},
{
$lookup: {
from: "collectionB",
let: { id: "$_id" },
pipeline: [
{
$match: {
$expr: { $eq: ["$$id", "$col_a_id"] },
},
},
{
$lookup: {
from: "user",
localField: "user_id",
foreignField: "_id",
as: "user",
},
},
{
$addFields: {
x: { $arrayElemAt: ["$x", 0] },
},
},
{
$project: {
param1: "$x.a",
param2: "$x.b",
},
},
],
as: "completeList",
},
},
{
$project: {
_id: 1,
param: 1,
completeList: "$completeList",
},
},
]);
Collection A:
_id: 11
param: "fff"
Collection B:
_id: 543
col_a_id: 11,
user_id: 789,
param_I_want: "only from the user who is logged in"
User:
_id: 789
param1: "trfewe",
param2: "fewfew"
So my current output is like this:
[
{
"_id": "11",
"param": "fff",
"completeList": [
{
"_id": "222",
"user_id": 789,
"col_a_id": 11,
"param1": "trfewe",
"param2": "fewfew",
},
{
"_id": "333",
"user_id": 899,
"col_a_id": 11,
"param1": "fwfer",
"param2": "gerwa",
},
]
}
...
]
And what I want is this:
[
{
"_id": "11",
"param": "fff",
"completeList": [
{
"_id": "222",
"user_id": 789,
"col_a_id": 11,
"param1": "trfewe",
"param2": "fewfew",
},
{
"_id": "333",
"user_id": 899,
"col_a_id": 11,
"param1": "fwfer",
"param2": "gerwa",
},
],
"param_I_want": "only from the user who is logged in"
}
...
]
And since I'm asking a question I add another one into it: In the aggregation I use the same lookup. The second one is in a pipeline.
Is there a way to do it in a better way? I mean if I imagine it being used by thousands of people, could there be performance issues?
If yes why and how can I improve it?
Related
I have a Ranks collection with documents which looks like this:
[
{
"_id": "1",
"url": "ex1.com",
"keyword": "k1",
"rank": 19,
"createdAt": "2021-06-02",
"user": "616c542660d23fc17469b47e"
},
{
"_id": "2",
"url": "ex1.com",
"keyword": "k1",
"rank": 14,
"createdAt": "2021-06-01",
"user": "616c542660d23fc17469b47e"
},
{
"_id": "3",
"url": "ex1.com",
"keyword": "k2",
"rank": 8,
"createdAt": "2021-05-01",
"user": "616c542660d23fc17469b47e"
},
{
"_id": "4",
"url": "ex2.com",
"keyword": "k3",
"rank": 4,
"createdAt": "2021-05-01",
"user": "616c542660d23fc17469b47e"
}
]
users collection with documents which looks like this:
[
{
_id: "616c542660d23fc17469b47e",
email: "some#email.com"
}
]
I'm trying to run an aggregation which will return each user object + user's data array that grouped by url, each url object has keywords array that includes unique and last (by date) rank keyword
This is what I tried but the query returns all url's keywords, how can i make it return unique and last (by createdAt date) keywords
Rank.aggregate([
{
$match: {}
},
{
$lookup: {
from: 'users',
localField: 'user',
foreignField: '_id',
as: 'user'
}
},
{
$project: {
user: {
$arrayElemAt: ['$user', 0]
},
url: '$url',
keyword: '$keyword',
rank: '$rank',
createdAt: '$createdAt',
}
},
{
$sort: {
createdAt: -1
}
},
{
$group: {
_id: '$user._id',
user: {
$first: '$user'
},
data: {
$push: {
id: '$_id',
url: '$url',
keyword: '$keyword',
rank: '$rank',
createdAt: '$createdAt',
}
}
}
}
])
Expected output:
[{
user: {
_id: "616c542660d23fc17469b47e",
email: "some#email.com"
},
data: [
{
url: "ex1.com",
keywords: [
{
keyword: "k1",
rank: 19,
createdAt: "2021-06-02",
},
{
keyword: "k2",
rank: 8,
createdAt: "2021-05-01"
},
]
},
{
url: "ex2.com",
keywords: [
{
keyword: "k3",
rank: 4,
createdAt: "2021-05-01"
},
]
}
]
}]
Here it is the solution that I came out with. Playground
Full explanation:
We group by "$url","$user" and "$keyword" to get the unique combinations of this fields. AT this point waht we want is only the unique keywords, but we have to use the user and url fields, becouse we would groupBy those later too.Because we order them by createdAt, if we get the first document it will be the last one created.
{
"$sort": {
"createdAt": 1
}
},
{
"$group": {
"_id": [
"$url",
"$user",
"$keyword"
],
"keywords": {
$first: "$$ROOT"
}
}
},
Then we will format this keyword information a bit to group it by url. This step will give us the keywords per URL.
{
"$project": {
"url": "$keywords.url",
"user": "$keywords.user",
"keywords": "$keywords",
"_id": 0
}
},
{
"$group": {
"_id": [
"$user",
"$url"
],
"data": {
$push: "$$ROOT"
}
}
},
Finally we will group the URLs by user. Notice that we have grouped by URL and by user in each groupBy in order to not lose those fields.
{
"$project": {
"url": {
$first: "$data.keywords.url"
},
"user": {
$first: "$data.keywords.user"
},
"keywords": "$data.keywords",
"_id": 0
}
},
{
"$group": {
"_id": "$user",
"data": {
$push: "$$ROOT"
}
}
},
At this step we have almost all the information we needed grouped together. We would perform a lookUp to get the email from the Users collection and do the final mapping to remove some redundant data.
{
$lookup: {
from: "users",
localField: "_id",
foreignField: "_id",
as: "user"
}
},
{
"$unwind": "$user"
},
{
"$project": {
"_id": 0,
"data.user": 0,
"data.keywords._id": 0,
"data.keywords.url": 0,
"data.keywords.user": 0
}
},
I currently have 3 different collections.
ColA
{
id: 1,
some_other_data: "fff"
}
ColB
{
id: 1,
colA_id: 1,
spec_id: 5,
data: "test"
}
and
ColC
{
id: 5,
colA_id: 1,
name: "xxx"
}
My current code:
const list = await ColA.aggregate([
{
$lookup: {
from: "ColB",
localField: "_id",
foreignField: "ColA_id",
as: "col_b_list",
},
},
{
$lookup: {
from: "ColC",
localField: "col_b_list.spec_id",
foreignField: "_id",
as: "col_c_list",
},
},
{
$project: {
_id: 1,
list: "$col_b_list",
},
},
]);
return list;
I have following output.
[
{
"_id": "6123858b5b8dcc0e749c9e39",
"list": [
{
"_id": "6123858b5b8dcc0e749c9e43",
"spec_id": "60d33125f81840c010052e03",
"createdAt": "2021-08-23T11:24:59.292Z",
"updatedAt": "2021-08-23T11:24:59.292Z",
"__v": 0
},
{
"_id": "612386317dd1cb0ebcef1862",
"spec_id": "60d33125f81840c010052e03",
"createdAt": "2021-08-23T11:27:45.515Z",
"updatedAt": "2021-08-23T11:27:45.515Z",
"__v": 0
}
]
}
]
What I'm trying to achieve is to get spec_id in ColB, and the id from ColC together in one object.
But what I want is, that in the list object the looked up Collection ColC is included, like:
[
{
"_id": "6123858b5b8dcc0e749c9e39",
"some_other_data": "fff",
"list": [
{
"_id": "6123858b5b8dcc0e749c9e43",
"spec_id": "60d33125f81840c010052e03",
"name": "xxx",
"createdAt": "2021-08-23T11:24:59.292Z",
"updatedAt": "2021-08-23T11:24:59.292Z",
"__v": 0
},
{
"_id": "612386317dd1cb0ebcef1862",
"spec_id": "60d33125f81840c010052e03",
"name": "yyy",
"createdAt": "2021-08-23T11:27:45.515Z",
"updatedAt": "2021-08-23T11:27:45.515Z",
"__v": 0
}
]
}
]
I tried to map it in the project stage, but somehow there where multiple same objects, so I could not get it working properly.
Thanks for the help in advance!
You can use nested $lookup, using lookup with aggregation pipeline,
const list = await ColA.aggregate([
{
$lookup: {
from: "ColB",
let: { id: "$_id" },
pipeline: [
{
$match: {
$expr: { $eq: ["$$id", "$ColA_id"] }
}
},
{
$lookup: {
from: "ColC",
localField: "spec_id",
foreignField: "id",
as: "ColC"
}
},
{
$addFields: {
ColC: { $arrayElemAt: ["$ColC", 0] }
}
},
// { $project: {} }
],
as: "list"
}
}
])
I would like to share my problem with you.
So I have a 3 entities that needs to be accessed on my query:
Evaluation:
[
{
_id: 1,
questionary: 1,
subject: 1
},
{
_id: 1,
questionary: 1,
subject: 2
},
]
User
[
{
_id: 1
name: "John Doe",
photo: "photo1.jpg"
},
{
_id: 2
name: "Paul Smith",
photo: "photo2.jpg"
},
]
questionary
[
{
_id: 1,
title: "questionary 1",
"date": "2020-02-08T00:00:00.000Z"
},
{
_id: 2,
title: "questionary 2",
"date": "2020-02-09T00:00:00.000Z"
}
]
So my target is getting a data like this: A list of questionaries inside that, a list of evaluations related to a questionary, and inside a evaluation I need a user object. Like this:
[
{
"_id": "1",
"title": "questionary 1",
"evaluations": [
{
"_id": "1",
"date": "2020-04-05T18:53:46.948Z"
"user": {
_id: 1,
"name": "John Doe",
"photo": "photo1.jpg"
}
},
{
"_id": "2",
"date": "2020-04-06T18:53:46.948Z",
"user": {
_id: 1,
"name": "John Doe",
"photo": "photo1.jpg"
}
}
]
}
]
My query is:
return await Questionary.aggregate([{
$lookup: {
from: "evaluation",
localField: "_id",
foreignField: "questionary",
as: "evaluations",
}
},
{
$lookup: {
from: "user",
localField: "evaluations.user",
foreignField: "_id",
as: "user",
}
},
{
$project: {
_id: 1,
title: 1,
status: 1,
evaluations: {
_id: 1,
date: 1,
user: "$user"
}
},
},
]);
And my result is:
[
{
"_id": "1",
"title": "questionary 1",
"evaluations": [
{
"_id": "1",
"date": "2020-04-05T18:53:46.948Z"
"user": [
{
_id: 1,
"name": "John Doe",
"photo": "photo1.jpg"
},
{
_id: 2,
"name": "Paul Smith",
"photo": "photo2.jpg"
}
]
},
{
"_id": "2",
"date": "2020-04-06T18:53:46.948Z",
"user": [
{
_id: 1,
"name": "John Doe",
"photo": "photo1.jpg"
},
{
_id: 2,
"name": "Paul Smith",
"photo": "photo2.jpg"
}
]
}
]
}
]
The users of my evaluations are merging, but this is not that I want, I just want the internal information of the user inside the evaluation.
Any suggestion?
You should be able to use a nested $lookup by using uncorrelated sub-queries as described at https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/#join-conditions-and-uncorrelated-sub-queries
In your case that would be:
return await Questionary.aggregate([{
$lookup: {
from: "evaluation",
let: {questId: "$_id"},
pipeline: [{
$match: {
$expr: {
$eq: ["$$questId", "$questionary"]
}
},
}, {
$lookup: {
from: "user",
localField: "subject",
foreignField: "_id",
as: "user",
}
}],
as: "evaluations"
}
}
]);
This question already has an answer here:
Conditional $lookup in MongoDB?
(1 answer)
Closed 3 years ago.
I have two collections in MongoDB and want to join the two collections based on some condition.
I want to join 'order' and 'order-status' table to get all orders assigned to '123' with status 'ready'
orders
{
"_id":"1",
"name": "Fridge",
"assignee": "123"
},
{
"_id":"2",
"name": "TV",
"assignee": "567"
},
{
"_id":"3",
"name": "Music system",
"assignee": "123"
}
order-status
{
"_id":"1",
"status": "ready",
"orderId": "1"
},
{
"_id":"2",
"status": "cancelled",
"orderId": "2"
},
{
"_id":"3",
"status": "cancelled",
"orderId": "3"
}
assignee
{
"_id":"123",
"name": "Jak"
}
{
"_id":"567",
"name": "Mac"
}
I want to join 'order' and 'order-status' table to get all orders assigned to '123' with status 'ready'
Expecting a final result as
[
{
"_id":"1",
"name": "Fridge",
"assignee": "123",
"status": {
"_id":"1",
"status": "ready",
"orderId": "1"
}
}
]
Tried following but how to check order status in another table with lookup
const resultObject = orders.aggregate([
{ $match : {assignee: Objectid('123')} },
{
$lookup: {
from: 'user-status',
localField: 'assignee',
foreignField : '_id',
as : 'assignee'
}
},
{
$unwind: '$assignee'
}
]);
First you need to use match to filter by "assignee": "123", then you need to lookup order-status, match "orderStatus.status": "ready".
const resultObject = orders.aggregate([
{
$match: {
assignee: "123"
}
},
{
$lookup: {
from: "order-status",
localField: "_id",
foreignField: "orderId",
as: "statuses"
}
},
{
$match: {
"statuses.status": "ready"
}
},
{
$project: {
id: "_id",
name: "$name",
assignee: "$assignee",
status: {
$arrayElemAt: ["$statuses", 0]
}
}
}
]);
This will give result like this:
[
{
"_id": "1",
"assignee": "123",
"name": "Fridge",
"status": {
"_id": "1",
"orderId": "1",
"status": "ready"
}
}
]
Playground
I would use the following pipeline:
const resultObject = orders.aggregate([
{
$match: {
assignee: Objectid('123')
}
},
{
$lookup:
{
from: "order-status",
let: {order_id: "$_id"},
pipeline: [
{
$match:
{
$expr:
{
$and:
[
{$eq: ["$orderId", "$$order_id"]},
{$eq: ["$status", "ready"]}
]
}
}
}
],
as: "stock"
}
},
{
$unwind: "$stock"
},
// now we get the assignee info.
{
$lookup: {
from: 'user-status',
localField: 'assignee',
foreignField: '_id',
as: 'assignee'
}
},
{
$unwind: '$assignee'
},
//finaly create the required structure.
{
$project: {
name: "$assignee.name",
assignee: "$assignee._id",
status: "$stock.0"
}
}
]);
I have separate collections for 'comments','products' and 'users'. The 'comments' collection contains text, product_id and user_id. When a product is fetched I want the details of product along with details of the user in the result.
I have created schema using mongoose odm. I am using aggregate function to populate the product with comments using $lookup.
Product.aggregate([
{
$match:{
_id:mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "product",
as: "comments"
}
},
{
$match:{
"comments.product":mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: "users",
localField: "comments.user._id",
foreignField: "user",
as: "comments.user"
}
}
])
expected result is
[
{
"_id": "5cc9441feed4c258881c99cd",
"title": "Batman",
"imageUrl": "images\\1556694047310_Batman.jpg",
"price": 555,
"description": "The dark knight",
"user": "5cbca36d4acc5d538c209014",
"__v": 2,
"comments": [
{
"_id": "5cc947125c69600d58c1be05",
"date": "2019-05-01T07:12:42.229Z",
"text": "This product is very nice",
"user":{
"_id": "5cbca36d4acc5d538c209014",
"name": "Clark Kent"
}
},
{
"_id": "5cc96eb4b2834d43f8a24470",
"date": "2019-05-01T09:46:34.774Z",
"text": "Anyone can be Batman",
"user":{
"_id": "5cbca5504acc5d538c209015",
"name": "Bruce Wayne"
},
}
}
]
actual result is
[
{
"_id": "5cc9441feed4c258881c99cd",
"title": "Batman",
"imageUrl": "images\\1556694047310_Batman.jpg",
"price": 555,
"description": "The dark knight",
"user": "5cbca36d4acc5d538c209014",
"__v": 2,
"comments": {
"user": [
{
"_id": "5cbca5504acc5d538c209015",
"name": "Bruce Wayne",
"email": "batman#gotham.com",
"password": "$2a$12$L.t/nBXq/xlic25Y0a884uGxjlimuNH/tcmWLg.sNkcjJ/C40Q14m",
"contactNumber": 9999999999,
"address": "Somewhere in Gotham",
"__v": 0
},
{
"_id": "5cbca7334acc5d538c209016",
"name": "Superman",
"email": "superman#metro.com",
"password": "$2a$12$mrogzC1Am86b0DnvTzosm.qfu38Ue7RqSNcnVSoCR55PtmLddeZv.",
"contactNumber": 9999999999,
"address": "Somewhere in metropolis",
"__v": 0
},
{
"_id": "5cbca7e54acc5d538c209017",
"name": "Wonder Woman",
"email": "ww#amazon.com",
"password": "$2a$12$Vt9XZUyOTULvel5zNAsMLeoMi3HlaGJJZN7OH2XkWuoAiZtDIGaMq",
"contactNumber": 9999999999,
"address": "Somewhere in Amazon",
"__v": 0
},
{
"_id": "5cbe192934ae2944c8704a5a",
"name": "Barry Allen",
"email": "barry#flash.com",
"password": "$2a$12$k73Wp1HTMv/MhUV3BOok3OSh.nnLq3vWG1Qz9ZTO7iB7saFlxhLjW",
"contactNumber": 9999999999,
"address": "Somewhere in Central City",
"__v": 0
}
]
}
}
]
Your $lookup query of users is overwriting the comments array. Its not working as you think it'll.
You need to unwind the comments array and then run that $lookup of users and then group by the products.
Edit: I have updated the query with $group by code too. Also you can playa around with the query here:
https://mongoplayground.net/p/2EA-Glz8Hrm
Product.aggregate([
{
$match: {
_id: "5cc9441feed4c258881c99cd"
}
},
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "product",
as: "comments"
}
},
{
$unwind: "$comments"
},
{
$lookup: {
from: "users",
localField: "comments.user",
foreignField: "_id",
as: "comments.user"
}
},
{
$unwind: "$comments.user"
},
{
$group: {
_id: "$_id",
// add other fields you want to include
comments: {
$addToSet: "$comments"
}
}
},
])
As suggested by Hamza, I made following changes to my query
Product.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: "comments",
localField: "_id", //field from input document
foreignField: "product", // field from documents of the 'from' collection
as: "comments"
}
},
{
$unwind: "$comments"
},
{
$lookup: {
from: "users",
localField: "comments.user", //field from input document
foreignField: "_id", // field from documents of the 'from' collection
as: "comments.user"
}
},
{
$unwind: "$comments.user"
},
{
$group: {
_id: "$_id",
title: { $first: "$title" }, // $first returns the first expression of the document it encounters, ex. first title
price: { $first: "$price" },
imageUrl: { $first: "$imageUrl" },
description: { $first: "$description" },
rating: { $first: "$rating" },
comments: {
$addToSet: "$comments" // group comments and create an array
}
}
},
{
$project: {
_id: 1,
title: 1,
price: 1,
imageUrl: 1,
description: 1,
rating: 1,
comments: {
_id: 1,
text: 1,
date: 1,
user: {
_id: 1,
name: 1
}
}
}
}
])
With this I got the desired result.