How to to get distinct values in mongodb. I am using mongodb aggregation. How to use distinct in aggregation query. I am using it in node.js
below is my query in model
const criteria = [
{ $match: { $and: [{ $text: { $search: search } }, { $or: [publicData, { $and: [publisherId, { $or: [unListedData, privateData] }] }] }] } },
{ $lookup: { from: 'user', localField: 'userId', foreignField: 'id', as: 'publisherDetails' } },
{ $addToSet: '$_id' },
{ $sort: { created_at: -1 } }
];
Here I have used $addToset, but it did not work.
I have followed the doc's
distinct,
addToset,
but still coud not get proper solution
Related
I have a schema
const membershipsSchema = new Schema({
spaceId: {
type: Schema.Types.ObjectId,
ref: 'Space',
},
member: {
type: Schema.Types.ObjectId,
ref: 'User',
},
....
);
mongoose.model('Membership', membershipsSchema);
I want to use join statement like
Select * from membershipPlans as plans join User as users on plans.member=users._id
where plans.spaceId=id and users.status <> 'archived'; // id is coming from function arguments
I tried the aggregate pipeline like
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$match: {
'users.status': {$ne: 'archived'}
}
},
]);
But on console.log(memberships); I am getting an empty array. If I try return Memberships.find({ spaceId: id }) it returns populated memberships of that space. But when I try
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: id
}
},
]}
It still returns an empty array. Not sure if I know how to use an aggregate pipeline.
There are two things that you need to do:
Cast id to ObjectId.
Instead of using $match, just filter the contents of the users array using $filter.
Try this:
const memberships = await Memberships.aggregate([
{
$match: {
spaceId: new mongoose.Types.ObjectId(id)
}
},
{
$lookup: {
from: 'User',
localField: 'member',
foreignField: '_id',
as: 'users',
},
},
{
$project: {
users: {$filter: {
input: "$users",
as: "user",
cond: {
$ne: ["$$user.status", "archived"]
}
}}
}
},
]);
I have a collection of users like this
[
{ _id: ObjectId("61a6d586e56ea12d6b63b68e"), fullName: "Mr A" },
{ _id: ObjectId("6231a89b009d3a86c788bf39"), fullName: "Mr B" },
{ _id: ObjectId("6231a89b009d3a86c788bf3a"), fullName: "Mr C" }
]
And a collection of complains like this
[
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("61a6d586e56ea12d6b63b68e" },
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("6231a89b009d3a86c788bf3c" },
{ _id: ObjectId("6231aaba2a038b39d992099b"), type: "fee", postedBy: ObjectId("6231a89b009d3a86c788bf3b" },
]
I want to check if the postedBy fields of complains are not existed in users, then update by using the updateMany query
By the way, I have an optional way to achieve the goal but must use 2 steps:
const complains = await Complain.aggregate()
.lookup({
from: "users",
localField: "postedBy",
foreignField: "_id",
as: "postedBy",
})
.match({
$expr: {
$eq: [{ $size: "$postedBy" }, 0],
},
});
complains.forEach(async (complain) => {
complain.type = "other";
await complain.save();
});
Therefore, can I combine 2 steps into a single updateMany query? Like $match and $lookup inside updateMany query?
With MongoDB v4.2+, you can use $merge to perform update at last stage of aggregation.
db.complains.aggregate([
{
"$lookup": {
from: "users",
localField: "postedBy",
foreignField: "_id",
as: "postedByLookup"
}
},
{
$match: {
postedByLookup: []
}
},
{
"$addFields": {
"type": "other"
}
},
{
"$project": {
postedByLookup: false
}
},
{
"$merge": {
"into": "complains",
"on": "_id",
"whenMatched": "replace"
}
}
])
Here is the Mongo playground for your reference.
I have the following schema:
const UserQualificationSchema = new Schema(
{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
qualification: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Qualification',
},
expiry_date: {
type: Date
}
}
const QualificationSchema = new Schema(
{
fleet: {
type: [String], // Eg ["FleetA", "FleetB", "FleetC"]
required: true,
}
}
I am searching the UserQualifications with filters in a table, to search them by fleet, qualification or expiry date. I so far have the following aggregate:
db.UserQualifications.aggregate([{
{
$lookup: {
from: 'qualifications',
localField: 'qualification',
foreignField: '_id',
as: 'qualification',
},
},
{
$unwind: '$qualification',
},
{
$match: {
$and: [
'qualification.fleet': {
$in: ["Fleet A", "Fleet C"], // This works
},
expiry_date: {
$lt: req.body.expiry_date, // This works
},
qualification: { // Also tried 'qualification._id'
$in: ["6033e4129070031c07fbbf29"] // Adding this returns blank array
}
]
},
}
}])
Filtering by fleet, and expiry date both work, independently and in combination, however when adding by the qualification ID, it returns blank despite the ID's being sent in being valid.
Am i missing something here?
Looking at your schema I can infer that qualification in ObjectId and in the query you are passing only the string value of ObjectId. You can pass the ObjectId to get your expected output
db.UserQualifications.aggregate([
{
$lookup: {
from: "Qualifications",
localField: "qualification",
foreignField: "_id",
as: "qualification",
},
},
{
$unwind: "$qualification",
},
{
$match: {
"qualification.fleet": {
$in: [
"FleetA",
"FleetC"
],
},
expiry_date: {
$lt: 30 // some dummy value to make it work
},
"qualification._id": {
$in: [
// some dummy value to make it work
ObjectId("5a934e000102030405000000")
]
}
},
}
])
I have created a playground with some dummy data to test the query: Mongo Playground
Also, In $match stage there is no need to combine query explicitly in $and as by default behaviour will be same as $and only so I have remove that part in my query
I have the following aggregation and It does not return the user profiles properly
const newConverSation = await Messenger.Messenger.aggregate([
{ $match: {
users: mongoose.Types.ObjectId("6084036ad4d4cd40a47afba4")}},
{ $sort: { updatedAt: -1 } },
{
$group: {
_id: { $setUnion: "$users" },
message: { $first: "$$ROOT" }
}
},
{
$lookup: {
from: 'users',
localField: 'users',
foreignField: '_id',
as: 'users'
}
}
])
In the outcome, there is an users array like this
"users" : [
ObjectId("60841d03f6ccad2b0400f619"),
ObjectId("6084036ad4d4cd40a47afba4")
],
and I just want to fetch user profiles depending on these two Id's but it does not return profiles in the current way.
Try adding the .populate after the .find property where you want to show this data.
I have two models, made using Mongoose Schema.
Book {
title: String,
chapters: [{
type: Schema.Types.ObjectId,
ref: 'chapter'
}],
}
Chapter {
title: String,
status: String,
book: {
type: Schema.Types.ObjectId,
ref: 'book'
},
}
I want to find Books that have a chapter with "status":"unfinished". What is the most efficient way to achieve this? Since the Book model stores ObjectIds, how can I make the find query so that the filtered results will be fetched directly from the DB?
I think the most optimal way would be to denormalize your schema, as a book will have a limited amount of chapters and a chapter can belong to at most one book, we can store the schema like this
Book {
title: String,
chapters: [{
title: String,
status: String,
}],
}
with this schema, we can then create an index on 'chapters.status' and simply get the answer in a single query without the need of $lookup.
db.books.find({'chapters.status': 'unfinished'});
But in any case, you still need to go with the above schema, we always have an option for $lookup
db.book.aggregate([
{
$unwind: "$chapters",
},
{
$lookup: {
from: "chapter",
localField: "chapters",
foreignField: "_id",
as: "chapter",
},
},
{
$match: {
"chapter.status": "unfinished",
},
},
{
$group: {
_id: "$_id",
title: { $first: "$title" },
},
},
]);
You can always adjust the above query to your needs.
Example
You can try using aggregate(),
$lookup with pipeline, join Chapter collection
$match 2 conditions first match chapter _id in chaptersIds, second status is equal to unfinished
$match to match chapters not equal to empty array
$project to show or hide required fields
db.Book.aggregate([
{
"$lookup": {
from: "Chapter",
as: "chapters_list",
let: { chapterIds: "$chapters" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $in: ["$_id", "$$chapterIds"] },
{ $eq: ["$status", "unfinished"] }
]
}
}
}
]
}
},
{
$match: { chapters_list: { $ne: [] } }
},
// if you want chapters_list array then remove $project this part
{
$project: { chapters: 1, title: 1 }
}
])
Playground