I have following users collection
[{
"_id" : ObjectId("5afadfdf08a7aa6f1a27d986"),
"firstName" : "bruce",
"friends" : [ ObjectId("5afd1c42af18d985a06ac306"),ObjectId("5afd257daf18d985a06ac6ac") ]
},
{
"_id" : ObjectId("5afbfe21daf4b13ddde07dbe"),
"firstName" : "clerk",
"friends" : [],
}]
and have friends collection
[{
"_id" : ObjectId("5afd1c42af18d985a06ac306"),
"recipient" : ObjectId("5afaab572c4ec049aeb0bcba"),
"requester" : ObjectId("5afadfdf08a7aa6f1a27d986"),
"status" : 2,
},
{
"_id" : ObjectId("5afd257daf18d985a06ac6ac"),
"recipient" : ObjectId("5afadfdf08a7aa6f1a27d986"),
"requester" : ObjectId("5afbfe21daf4b13ddde07dbe"),
"status" : 1,
}]
suppose I have an user logged in with _id: "5afaab572c4ec049aeb0bcba" and this _id matches the recipient of the friends
Now I have to add a field friendsStatus which contains the status from friends collection... And if does not matches the any recipient from the array then its status should be 0
So when I get all users then my output should be
[{
"_id" : ObjectId("5afadfdf08a7aa6f1a27d986"),
"firstName" : "bruce",
"friends" : [ ObjectId("5afd1c42af18d985a06ac306") ],
"friendStatus": 2
},
{
"_id" : ObjectId("5afbfe21daf4b13ddde07dbe"),
"firstName" : "clerk",
"friends" : [],
"friendStatus": 0
}]
Thanks in advance!!!
If you have MongoDB 3.6 then you can use $lookup with a "sub-pipeline"
User.aggregate([
{ "$lookup": {
"from": Friend.collection.name,
"let": { "friends": "$friends" },
"pipeline": [
{ "$match": {
"recipient": ObjectId("5afaab572c4ec049aeb0bcba"),
"$expr": { "$in": [ "$_id", "$$friends" ] }
}},
{ "$project": { "status": 1 } }
],
"as": "friends"
}},
{ "$addFields": {
"friends": {
"$map": {
"input": "$friends",
"in": "$$this._id"
}
},
"friendsStatus": {
"$ifNull": [ { "$min": "$friends.status" }, 0 ]
}
}}
])
For earlier versions, it's ideal to actually use $unwind in order to ensure you don't breach the BSON Limit:
User.aggregate([
{ "$lookup": {
"from": Friend.collection.name,
"localField": "friends",
"foreignField": "_id",
"as": "friends"
}},
{ "$unwind": { "path": "$friends", "preserveNullAndEmptyArrays": true } },
{ "$match": {
"$or": [
{ "friends.recipient": ObjectId("5afaab572c4ec049aeb0bcba") },
{ "friends": null }
]
}},
{ "$group": {
"_id": "$_id",
"firstName": { "$first": "$firstName" },
"friends": { "$push": "$friends._id" },
"friendsStatus": {
"$min": {
"$ifNull": ["$friends.status",0]
}
}
}}
])
There is "one difference" from the most optimal form here in that the pipeline optimization does not actually "roll-up" the $match condition into the $lookup itself:
{
"$lookup" : {
"from" : "friends",
"as" : "friends",
"localField" : "friends",
"foreignField" : "_id",
"unwinding" : {
"preserveNullAndEmptyArrays" : true
}
}
},
{
"$match" : { // <-- outside will preserved array
Because of the preserveNullAndEmptyArrays option being true then the "fully optimized" action where the condition would actually be applied to the foreign collection "before" results are returned does not happen.
So the only purpose of unwinding here is purely to avoid what would normally be a target "array" from the $lookup result causing the parent document to grow beyond the BSON Limit. Additional conditions of the $match are then applied "after" this stage. The default $unwind without the option presumes false for the preservation and a matching condition is added instead to do this. This of course would result in the documents with no foreign matches being excluded.
And not really advisable because of that BSON Limit, but there is also applying $filter to the resulting array of $lookup:
User.aggregate([
{ "$lookup": {
"from": Friend.collection.name,
"localField": "friends",
"foreignField": "_id",
"as": "friends"
}},
{ "$addFields": {
"friends": {
"$map": {
"input": {
"$filter": {
"input": "$friends",
"cond": {
"$eq": [
"$$this.recipient",
ObjectId("5afaab572c4ec049aeb0bcba")
]
}
}
},
"in": "$$this._id"
}
},
"friendsStatus": {
"$ifNull": [
{ "$min": {
"$map": {
"input": {
"$filter": {
"input": "$friends",
"cond": {
"$eq": [
"$$this.recipient",
ObjectId("5afaab572c4ec049aeb0bcba")
]
}
}
},
"in": "$$this.status"
}
}},
0
]
}
}}
])
In either case we're basically adding the "additional condition" to the join being not just on the directly related field but also with the additional constraint of the queried ObjectId value for "recipient".
Not really sure what you are expecting for "friendsStatus" since the result is an array and there can possibly be more than one ( as far as I know ) and therefore just applying $min here to extract one value from the array in either case.
The governing condition in each case is $ifNull which is applied where there isn't anything in the "friends" output array to extract from and then you simply return the result of 0 where that is the case.
All output the same thing:
{
"_id" : ObjectId("5afadfdf08a7aa6f1a27d986"),
"firstName" : "bruce",
"friends" : [
ObjectId("5afd1c42af18d985a06ac306")
],
"friendsStatus" : 2
}
{
"_id" : ObjectId("5afbfe21daf4b13ddde07dbe"),
"firstName" : "clerk",
"friends" : [ ],
"friendsStatus" : 0
}
Related
The main collection is User, and we have a User profile collection which having experience details and other stuff. Also, we have a Skill collection.
USER
[{
"_id": "5f1eef8ec68d306fbbf13b0f",
"name": "John Davis",
"email": "John#gmail.com",
"__v": 0
},
{
"_id": "9q1eef8ec68d306fbbf13bh6",
"name": "Mik Luca",
"email": "Mik#gmail.com",
"__v": 0
}]
User profile
[{
"_id": "5f1eef8ec68d306fbbf13b10",
"other_skills": [
null
],
"user_id": "5f1eef8ec68d306fbbf13b0f",
"phone_number": "1234569870",
"location": "5f16b72617fee02922688751",
"primary_skills": [
{
"_id": "5f32635cf764cc40447503a6",
"years": 1,
"skill_id": "5f0da75907a96c3040b3667d"
}
]
},
{
"_id": "5f1eef8ec68d306fbbf13b10",
"other_skills": [
null
],
"user_id": "9q1eef8ec68d306fbbf13bh6",
"phone_number": "1234569870",
"location": "5f16b72617fee02922688751",
"primary_skills": [
{
"_id": "6s0da75907a96c3040b36690",
"years": 1,
"skill_id": "5f0da75907a96c3040b3667d"
}
]
}]
Skill
[{
"_id": "5f0da75907a96c3040b3667d",
"skill": "Mongo"
},
{
"_id": "6s0da75907a96c3040b36690",
"skill": "Node"
}]
I need to list the users with their user profile info and need to filter with skills as well
I have tried
db.getCollection("users").aggregate(
[
{
"$project" : {
"_id" : NumberInt(0),
"users" : "$$ROOT"
}
},
{
"$lookup" : {
"localField" : "users._id",
"from" : "userprofiles",
"foreignField" : "user_id",
"as" : "userprofiles"
}
},
{
"$unwind" : {
"path" : "$userprofiles",
"preserveNullAndEmptyArrays" : true
}
},
{
"$lookup" : {
"localField" : "userprofiles.primary_skills.skill_id",
"from" : "skills",
"foreignField" : "_id",
"as" : "skills"
}
},
{
"$unwind" : {
"path" : "$skills",
"preserveNullAndEmptyArrays" : true
}
},
{
"$match" : {
"skills._id" : ObjectId("5f0dce8d07a96c3040b36687")
}
}
],
{
"allowDiskUse" : true
}
);
But not getting the proper results.
How can I populate the user profile and skill information with the User list and filter the user list with Skill ids?
Greetings and thanks.
You can match filters inside lookup using lookup with pipeline,
$lookup with userProfile collection
pipelines $match to match profile id
other filters for profile like skill_id match here
$unwind deconstruct primary_skills array because we are going to lookup with skill_id
$lookup will skills collection
$unwind deconstruct primary_skills.skill_id array because we need it as object
$grpup reconstruct primary_skills array
$match if userProfiles not equal to empty []
db.users.aggregate([
{
$lookup: {
from: "usersProfile",
let: { id: "$_id" },
as: "userProfiles",
pipeline: [
{
$match: {
$expr: { $eq: ["$$id", "$user_id"] },
// match here user profile filters
"primary_skills.skill_id": "5f0da75907a96c3040b3667d"
}
},
{ $unwind: "$primary_skills" },
{
$lookup: {
from: "skills",
localField: "primary_skills.skill_id",
foreignField: "_id",
as: "primary_skills.skill_id"
}
},
{ $unwind: "$primary_skills.skill_id" },
{
$group: {
_id: "$_id",
other_skills: { $first: "$other_skills" },
phone_number: { $first: "$phone_number" },
location: { $first: "$location" },
primary_skills: {
$push: {
_id: "$primary_skills._id",
skill: "$primary_skills.skill_id.skill",
years: "$primary_skills.years"
}
}
}
}
]
}
},
{ $match: { userProfiles: { $ne: [] } } }
])
Playground
I have Users Collection. devices are all in array of Objects.
[{
"_id" : ObjectId("5c66a979e109fe0f537c7e37"),
"devices": [{
"dev_token" : "XXXX",
"_id" : ObjectId("5ccc0fa5f7778412173d22bf")
}]
},{
"_id" : ObjectId("5c66b6382b18fc4ff0276dcc"),
"devices": [{
"dev_token" : "XXXX",
"_id" : ObjectId("5c93316cc33c622bdcfaa4be")
}]
}]
I need to query the documents with adding the new field date in devices like
"devices": [{
"dev_token" : "XXXX",
"_id" : ObjectId("5c93316cc33c622bdcfaa4be"),
"date": ISODate("2012-10-15T21:26:17Z")
}]
date key from devices._id.getTimestamp()
I tried using aggregate this one, donno how to use getTimestamp()
db.getCollection('users').aggregate([ {
"$unwind": "$devices"
}, {
"$group": {
"_id": "$_id",
"devices": {
"$push": "$devices._id.getTimestamp()"
}
}
}])
I use $devices._id.getTimestamp(), this could be error.. Here how I handle this one.. Thanks for advance
You can use $toDate to get Timestamp from the _id field.
Add date field to each devices element after unwind stage, using $addFields
Try this :
db.getCollection('users').aggregate([ {
"$unwind": "$devices"
},{
$addFields : {
"devices.date": { $toDate: "$_id" }
}
}, {
"$group": {
"_id": "$_id",
"devices": {
"$push": "$devices"
}
}
}])
You can check the result at Mongo Playground (just press "run")
Using MongoDb 3.6
The $dateFromParts operator comes in handy here where you can use it in conjunction with the other date operators. You won't need
to $unwind the array as you can use $map to map over the devices array documents and add the extra date field with the above expression.
This can be followed with an example pipeline below :
db.getCollection('users').aggregate([
{ "$addFields": {
"devices": {
"$map": {
"input": "$devices",
"in": {
"dev_token": "$$this.dev_token",
"_id": "$$this._id",
"date": {
"$dateFromParts": {
'year': { "$year": "$$this._id"},
'month': { "$month": "$$this._id"},
'day':{ "$dayOfMonth": "$$this._id"},
'hour': { "$hour": "$$this._id"},
'minute': { "$minute": "$$this._id"},
'second': { "$second": "$$this._id"},
'millisecond': { "$millisecond": "$$this._id"}
}
}
}
}
}
} }
])
Output
/* 1 */
{
"_id" : ObjectId("5c66a979e109fe0f537c7e37"),
"devices" : [
{
"dev_token" : "XXXX",
"_id" : ObjectId("5ccc0fa5f7778412173d22bf"),
"date" : ISODate("2019-05-03T09:53:41.000Z")
}
]
}
/* 2 */
{
"_id" : ObjectId("5c66b6382b18fc4ff0276dcc"),
"devices" : [
{
"dev_token" : "XXXX",
"_id" : ObjectId("5c93316cc33c622bdcfaa4be"),
"date" : ISODate("2019-03-21T06:38:36.000Z")
}
]
}
Using MongoDb 4.0 and newer:
The pipeline can be tweaked slightly to use the new $toDate or $convert operators. Their respective uses follow:
$toDate
db.getCollection('users').aggregate([
{ "$addFields": {
"devices": {
"$map": {
"input": "$devices",
"in": {
"dev_token": "$$this.dev_token",
"_id": "$$this._id",
"date": { "$toDate": "$$this._id" }
}
}
}
} }
])
$convert
db.getCollection('users').aggregate([
{ "$addFields": {
"devices": {
"$map": {
"input": "$devices",
"in": {
"dev_token": "$$this.dev_token",
"_id": "$$this._id",
"date": {
"$convert": { "input": "$$this._id", "to": "date" }
}
}
}
}
} }
])
How can i perform $lookup in aggregate (mongodb) for an array
{
messages: [{
"_id" : ObjectId("5bfc43f2bbc4176ecc1c5f83"),
"text" : "text1",
"sender" : {
"id" : "36046fc2e70dd508a0bf1f36fd2daa20"
}
}, {
"_id" : ObjectId("5bfc43f2bbc4176ecc1c5f83"),
"text" : "text2",
"sender" : {
"id" : "36046fc2e70dd508a0bf1f36fd2daa22"
}
}],
"filed1": { ... },
"filed2": { ... }
}
how can i do a $lookup for sender id from accounts collection?
tried:
...
{
$lookup: {
from: "accounts",
localField: "messages.sender.id",
foreignField: "id",
as: "messages.sender.user"
}
}
...
You can use below aggregation
db.collection.aggregate([
{ "$unwind": "$messages" },
{ "$lookup": {
"from": "accounts",
"localField": "messages.sender.id",
"foreignField": "id",
"as": "messages.sender.user"
}},
{ "$unwind": "$messages.sender.user" }
{ "$group": {
"_id": "$_id",
"messages": { "$push": "$messages" },
"filed1": { "$first": "$filed1" },
"filed2": { "$first": "$filed2" }
}}
])
I have a mongoDB collection called "conference" with an array of participants as below :
[
{
"_id" : 5b894357a0c84d5a5d221f25,
"conferenceName" : "myFirstConference",
"startDate" : 1535722327,
"endDate" : 1535722420,
"participants" : [
{
"name" : "user1",
"origin" : "internal",
"ip" : "192.168.0.2"
},
{
"name" : "user2",
"origin" : "external",
"ip" : "172.20.0.3"
},
]
},
...
]
I would like to get the following result :
[
{
"conferenceName" : "myFirstConference",
"startDate" : 1535722327,
"endDate" : 1535722420,
"internalUsersCount" : 1
"externalUsersCount" : 1,
},
...
]
I tried the request below but it's not working :
db.getCollection("conference").aggregate([
{
$addFields: {
internalUsersCount : {
$size : { "$participants" : {$elemMatch : { origin : "internal" }}}
},
externalUsersCount : {
$size : { "$participants" : {$elemMatch : { origin : "external" }}}
}
}
}
])
How is it possible to count "participant" array elements that match {"origin" : "internal"} and {"origin" : "external"} ?
You need to use $filter aggregation to filter out the external origin and internal origin along with the $size aggregation to calculate the length of the arrays.
Something like this
db.collection.aggregate([
{ "$addFields": {
"internalUsersCount": {
"$size": {
"$filter": {
"input": "$participants",
"as": "part",
"cond": { "$eq": ["$$part.origin", "internal"]}
}
}
},
"externalUsersCount": {
"$size": {
"$filter": {
"input": "$participants",
"as": "part",
"cond": { "$eq": ["$$part.origin", "external"] }
}
}
}
}}
])
Output
[
{
"conferenceName": "myFirstConference",
"endDate": 1535722420,
"externalUsersCount": 1,
"internalUsersCount": 1,
"startDate": 1535722327
}
]
I have the following document
{
"userid": "5a88389c9108bf1c48a1a6a7",
"email": "abc#gmail.com",
"lastName": "abc",
"firstName": "xyz",
"__v": 0,
"friends": [{
"userid": "5a88398b9108bf1c48a1a6a9",
"ftype": "SR",
"status": "ACCEPT",
"_id": ObjectId("5a9585b401ef0033cc8850c7")
},
{
"userid": "5a88398b9108bf1c48a1a6a91111",
"ftype": "SR",
"status": "ACCEPT",
"_id": ObjectId("5a9585b401ef0033cc8850c71111")
},
{
"userid": "5a8ae0a20df6c13dd81256e0",
"ftype": "SR",
"status": "pending",
"_id": ObjectId("5a9641fbbc9ef809b0f7cb4e")
}]
},
{
"userid": "5a88398b9108bf1c48a1a6a9",
"friends": [{ }],
"lastName": "123",
"firstName": "xyz",
.......
},
{
"userid": "5a88398b9108bf1c48a1a6a91111",
"friends": [{ }],
"lastName": "456",
"firstName": "xyz",
...
}
First Query
Here I want to get userId from friends array ,which having status equals to "ACCEPT".
ie
[5a88398b9108bf1c48a1a6a9,5a88398b9108bf1c48a1a6a91111]
Second Query
After that, I have to make another query on the same collection to get details of each userid returned in the first query.
final Query will return details of [5a88398b9108bf1c48a1a6a9,5a88398b9108bf1c48a1a6a91111]
both userid ie
[
{
userid" : "5a88398b9108bf1c48a1a6a9",
"lastName" : "123",
"firstName" : "xyz"
},
{
"userid" : "5a88398b9108bf1c48a1a6a91111",
"lastName" : "456",
"firstName" : "xyz"
}
]
I have tried so far with
Users.find ({'_id':5a88389c9108bf1c48a1a6a7,"friends.status":'ACCEPT'}, (error, users) => {})
or
Users.find ({'_id':5a88389c9108bf1c48a1a6a7, friends: { $elemMatch: { status: 'ACCEPT' } } }, (error, users) => {})
Use the aggregation framework's $map and $filter operators to handle the task. $filter will filter the friends array based on the specified condition that the status should equal "ACCESS" and $map will transform the results from the filtered array to the desired format.
For the second query, append a $lookup pipeline step which does a self-join on the users collection to retrieve the documents which match the ids from the previous pipeline.
Running the following aggregate operation will produce the desired array:
User.aggregate([
{ "$match": { "friends.status": "ACCEPT" } },
{ "$project": {
"users": {
"$map": {
"input": {
"$filter": {
"input": "$friends",
"as": "el",
"cond": { "$eq": ["$$el.status", "ACCEPT"] }
}
},
"as": "item",
"in": "$$item.userid"
}
}
} },
{ "$lookup": {
"from": "users",
"as": "users",
"localField": "users",
"foreignField": "userid"
} },
]).exec((err, results) => {
if (err) throw err;
console.log(results[0].users);
});
I did not test it. just for an idea, give it a try and let me know.
db.Users.aggregate(
[
{
$unwind: "$friends"
},
{
$match:{ "$friends.status": "ACCEPT"}
},
{
$project:{ "FriendUserID":"$friends.userid"}
},
{
$lookup:{
from:"Users",
as: "FriendsUsers",
localField: "FriendUserID",
foreignField: "userid"
}
},
{
$project: { FriendsUsers.lastName:1,FriendsUsers.firstName:1 }
}
]
)
filtering nested elements
const products = await Product.aggregate<ProductDoc>([
{
$match: {
userId: data.id,
},
},
{
$project: {
promotions: {
$filter: {
input: '$promotions',
as: 'p',
cond: {
$eq: ['$$p.status', PromotionStatus.Started],
},
},
},
userId: 1,
name: 1,
thumbnail: 1,
},
},
]);
for multiple condition
cond: {
$and: [
{
$eq: [
"$$c.product",
"37sd87hjsdj3"
]
},
{
$eq: [
"$$c.date",
"date-jan-4-2022"
],
}
]
},