I have a used mongoose aggregate pipeline with many stages in one of my api endpoints. In that pipeline there is a $unwind stage and multiple $lookup stages that must only run if images array in document in not empty. How can I setup the pipeline such that a select few stages may or may not run based on a condition derived from a field within the documents?
Thank you for reading.
For reference, here is a sample document.
{
_id: "6151387333f8e9001ff92429",
title: "Boost Your Car With These Tips",
description: "Follows these steps to get the most out of your car",
images: [
{
image: "5ebac534954b54139806c112",
caption: "Riding Ferreri",
credit: "5ebac534954b54139806c112"
},
{
image: "4ebac534954b54139806c142",
caption: "Motor Garage",
credit: "6ebac534954b54139806c11e"
},
]
}
Related
I have a transaction operation and I want to make a merge request into a table (that doesn't have a schema)
This is my implementation but it's not working in transactions, I get: $merge cannot be used in a transaction
await User.aggregate([
{
$match: {
_id: new mongoose.mongo.ObjectID(id),
},
},
{
$merge: {
into: 'deleted-users',
},
},
]).option({ session });
is there an alternative to do this scenario which is to add a record in a Newley created collection inside a transaction ?
Seeing that merge is
Excluding the following stages As we can read in official docs:
The following read/write operations are allowed in transactions: (among others) ... aggregate command.
Excluding the following stages: (among others) merge.
And here You have reference to merge and more...
I have a MongoDB collection called users with documents that look like:
{
_id: ObjectId('123'),
username: "abc",
avatar: "avatar/long-unique-random-string.jpg",
connections: [ObjectId('abc'), ObjectId('xyz'), ObjectId('lmn'), ObjectId('efg')]
}
This document belongs to the users collection.
What I want to do:
First, find one document from the users' collection that matches _id -> '123'.
Project the connections field received from step 1, which is an array of ObjectIds of other users within the same collection.
Find all documents of users from the array field projected in step 2.
Project and return an array of only the username and avatar of all those users from step 3.
While I know that I can do this in two separate queries. First using findOne which returns the friends array. Then, using find with the results of findOne to get all the corresponding usernames and avatars.
But, I would like to do this in one single query, using the aggregation pipeline.
What I want to know, is it even possible to do this in one query using aggregation?
If so, what would the query look like?
What, I currently have:
await usersCollection
.aggregate([
{ $match: { _id: new ObjectId(userId) } },
{ $project: { ids: "$connections" } },
{ $match: { _id: { $in: "ids" } } },
{
$project: {
username: "$username",
avatar: { $ifNull: ["$avatar", "$$REMOVE"] },
},
},
])
.toArray()
I know this is wrong because each aggregation stage receives the results from the previous stage. So, the second match cannot query on the entire users' collection, as far as I know.
I'm using MongoDB drivers for nodeJS. And I would like to avoid $lookup for possible solutions.
I have a tricky query that hits my MongoDB know-how. Here the simplified szenario.
We have a collection Restaurant and a collection Subsidary.
They look roughly like this (simplified - using mongoose):
const restaurantSchema = new Schema(
{
name: { type: String, required: true },
categories: { type: [String], required: true },
...
})
const subsidarySchema = new Schema(
{
restaurant: { type: Schema.Types.ObjectId, ref: 'Restaurant' },
location: {
type: { type: String, enum: ['Point'], required: true },
coordinates: { type: [Number], required: true },
},
...
})
What is required:
Always: Find restaurants that have a subsidary within 3.5 KM radius and sort by distance.
Sometimes filter those restaurants also by a string that should fuzy-match the Restaurant name.
Apply further filters and pagination (e.g. filter by categories, ...)
I'm trying to tackle this with a mongodb aggregation. The problem:
The aggregation pipeline stages geoNear and text require each to be first in the pipeline - which means they exclude each other.
Here my thought so far:
Start aggregation with subsidary, $geoNear stage first. This cuts away already all restaurants outside the 3.5 KM.
$group the subsidaries by restaurant and keep the minimal distance value per cluster.
$lookup to get the matchin restaurant for each cluster. Maybe $unwind here.
??? Here the text/search match should be, fuzy-matching the restaurants' name. ???
$match for other values (category, openingHours, ...)
$sort and $limit and $skip for sorting andd pagination.
Here the same as illustration.
Question
Does this approach make sense? What would be a possible way to implement stage 4?
I was searching a lot but there seems no way to use something like { $match: { $text: { $search: req.query.name } } } as a 4th stage.
An alternative would be to run a second query before that just handles the text search and then build an intersection. This could lead to a massive amount of restaurant IDs being passed in that stage. Is that something mongodb could handle?
I'm very thankful for your comments!
Some ways around the requirement that both text search and geo query must be the first stage:
Use text search as the first stage, then manually calculate the distance using $set/$expr in a subsequent stage.
Use geo query as the first stage, then perform text filtering in your application (allowing you also to use any text matching/similarity algorithm you like).
I am using mongoDB and nodejs
I have 4 collections
collection 1 is teacher with fields teacher_id, teacher_name
collection 2 is subject with fieldssubject _id, subject_name
collection 3 is book with fields book_id, book_name
collection is student which have fields -- _id, student_name, teacher_id, subject_id, book_id
how can I fetch ids from 1, 2, 3 collections simultaneously and insert to corresponding id in collection
I have tried some which always ask for a matching field... is ther any function which returns data from collection even though no match field?
can someone please help
Well, in that case, you need to fetch all the documents from those collections. That will be a bit costly aggregation but I'm adding here in code:
Firstly, I'm grouping on null, to avoid to attach lookup value to every single document in teacher collection.
db.teacher.aggregate([
{
$group:{
"_id":null,
"root":{
$push:"$$ROOT"
}
}
},
{
$lookup:
{
from:"subject",
pipeline: [],
as: "subjectLookup"
}
},
{
$lookup:
{
from:"book",
pipeline: [],
as: "bookLookup"
}
},
{
$lookup:
{
from:"student",
pipeline: [],
as: "studentLookup"
}
}
]).pretty()
These lookups will give the array which contains all the documents from respective collections, you can limit the documents by adding $match stage in the pipeline of lookup stage.
Hope this will solve your problem :)
I'm using nodejs + mongoosejs with mongodb 2.6. I have a static function on a model that sums the value of all items in the collection. Each item is assigned to a project using a projectNo property. I need the static function to be able to give me the total for the collection, and if a projectNo argument is passed, add a $match pipeline operator to the aggregation. This will save me from having to make 2 static functions that essentially does the same thing.
To spice things up a bit I use bluebird promisifyAll method to make the aggregation framework return a promise.
my static function that sums the entire collection:
db.collection.aggregateAsync([
{$group:{_id: null, amount: { $sum: "$amount" }}}
])
my static function that sums only the records with a matching projectNo:
db.collection.aggregateAsync([
{$match: { projectNo: projectNo }},
{$group:{_id: null, amount: { $sum: "$amount" }}}
])
I really want to use the Aggregate.append method to append the $match pipeline only if a req.params.projectNo is included.
When I try to add it to the async aggregation it gets an error, which makes sense because its just a promise. If I try this:
db.collection.aggregateAsync([
{$group:{_id: null, amount: { $sum: "$amount" }}}
]).then(function(aggregate){
aggregate.append({$match: { projectNo: projectNo }})
})
I get an error, (append is undefined). How should I go about doing this? Or just live with the fact that I have two functions that do the same thing?
I read the source code in mongodb to see exactly how to use the aggregate.append method. If you're building the aggregation using the chained methods, you can use append to add any pipeline operations.
So what I did instead is put the array of aggregation pipelines into an array. If there is a projectNo then I add the $match pipeline to the array using unshift(). I used unshift because you usually want the $match pipeline to first limit the number of records, then do the rest of the pipeline operations.
var pipeline = [{$group:{_id: null, amount: { $sum: "$amount" }}}];
if(req.params.projectNo){
pipeline.unshift({$match: { projectNo: req.params.projectNo }});
}
db.collection.aggregateAsync(pipeline);
I usually make things way more complicated than I need to...