I have this aggregation pipeline i wrote in NodeJS in my Atlas Trigger :
const pipeline = [
{$match: {"score": {$gt: 0}, "update": true}},
{$setWindowFields: {sortBy: {"score": -1}, output: {"rank": {$denseRank: {}}}}},
{$merge: {into: "ranking"}}
];
await ranking_col.aggregate(pipeline);
I have written this pipeline in python first for testing and it's working just fine :
self.db.ranking.aggregate([
{
"$match": {
"score": {"$gt": 0},
"update": True
}
},
{
'$setWindowFields': {
'sortBy': {'score': -1},
'output': {
'rank': {
'$denseRank': {
}
}
}
}
},
{
"$merge": {
"into": "ranking"
}
}
])
I have no errors from the Trigger logs but it seems that the pipeline is simply not executed as it should modify the ranking as it's done in python.
Can you please tell me what am i doing wrong here ?
EDIT : The database scheme (as simple as the query is)
See below one document of ranking_col :
{
"_id": "7dqe1kcA7R1YGjdwHsAkV83",
"score": 294,
"update": false,
"rank": 0,
}
The aggregation is simply here to calculate the rank attribute according to the score.
Ok so the issue relies in the Mongo driver the function uses on atlas and your understanding of it.
aggregate returns an AggregateCursor, which means until you trigger it no command is actually getting executed, this means your trigger is actually running fine, but because no one is using the cursor it just exits the function without doing anyways.
A super simple solution would be to just add .toArray(), this will convert the cursor to an array of documents. essentially triggering the functionality:
await ranking_col.aggregate(pipeline).toArray();
Related
this is my sample data in this I have a userId and a array "watchHistory", "watchHistory" array contains the list of videos that is watched by the user :
{
"_id": "62821344445c30b35b441f11",
"userId": 579,
"__v": 0,
"watchHistory": [
{
"seenTime": "2022-05-23T08:29:19.781Z",
"videoId": 789456,
"uploadTime": "2022-03-29T12:33:35.312Z",
"description": "Biography of Indira Gandhi",
"speaker": "andrews",
"title": "Indira Gandhi",
"_id": "628b45df775e3973f3a670ec"
},
{
"seenTime": "2022-05-23T08:29:39.867Z",
"videoId": 789455,
"uploadTime": "2022-03-31T07:37:39.712Z",
"description": "What are some healthy food habits to stay healthy",
"speaker": "morris",
"title": "Healthy Food Habits",
"_id": "628b45f3775e3973f3a670"
},
]
}
I need to match the userId and after that i need to sort it with "watchHistory.seenTime", seenTime field indicates when the user saw the video. so i need to sort like the last watched video should come first in the list.
I don't have permission to use unwind so can any one help me from this. Thank you.
If you are using MongoDB version 5.2 and above, you can use $sortArray operator in an aggregation pipeline. Your pipeline should look something like this:
db.collection.aggregate(
[
{"$match":
{ _id: '62821344445c30b35b441f11' }
},
{
"$project": {
_id: 1,
"userId": 1,
"__v": 1,
"watchHistory": {
"$sortArray": { input: "$watchHistory", sortBy: { seenTime: -1 }}
}
}
}
]
);
Please modify the filter for "$match" stage, according to the key and value you need to filter on. Here's the link to the documentation.
Without using unwind, it's not possible to do it via an aggregation pipeline, but you can use update method and $push operator, as a workaround like this:
db.collection.update({
_id: "62821344445c30b35b441f11"
},
{
$push: {
watchHistory: {
"$each": [],
"$sort": {
seenTime: -1
},
}
}
})
Please see the working example here
I have a group stage in my pipeline like:
$group: {
_id: null,
count: {$sum: 1},
results: {$push: '$$ROOT'}
}
Sometimes $$ROOT is empty and there are no results. In theses cases, I get an empty result after the group stage which is not my desired behaviour. I want to get the below object, instead with a zero count and empty results.
[
{
count: 0,
results: []
}
]
How can I achieve this? I tried to replace the root (by replaceRoot) but it seems when there is nothing there is no root either.
Query
the normal way to do it i think its on the driver with simple code
but you can also do it on the database, with more complicated code
"empty_result" is a collection containing this
[{"count": 0,"results": []}]
PlayMongo
docs.aggregate(
[{"$group":
{"_id": null, "count": {"$sum": 1},
"results": {"$push": "$$ROOT"}}},
{"$unionWith": {"coll": "empty_result"}},
{"$sort": {"count": -1}},
{"$limit": 1}])
Imaging I have an array of objects, available before the aggregate query:
const groupBy = [
{
realm: 1,
latest_timestamp: 1318874398, //Date.now() values, usually different to each other
item_id: 1234, //always the same
},
{
realm: 2,
latest_timestamp: 1312467986, //actually it's $max timestamp field from the collection
item_id: 1234,
},
{
realm: ..., //there are many of them
latest_timestamp: ...,
item_id: 1234,
},
{
realm: 10,
latest_timestamp: 1318874398, //but sometimes then can be the same
item_id: 1234,
},
]
And collection (example set available on MongoPlayground) with the following schema:
{
realm: Number,
timestamp: Number,
item_id: Number,
field: Number, //any other useless fields in this case
}
My problem is, how to $group the values from the collection via the aggregation framework by using the already available set of data (from groupBy) ?
What have been tried already.
Okay, let skip crap ideas, like:
for (const element of groupBy) {
//array of `find` queries
}
My current working aggregation query is something like that:
//first stage
{
$match: {
"item": 1234
"realm" [1,2,3,4...,10]
}
},
{
$group: {
_id: {
realm: '$realm',
},
latest_timestamp: {
$max: '$timestamp',
},
data: {
$push: '$$ROOT',
},
},
},
{
$unwind: '$data',
},
{
$addFields: {
'data.latest_timestamp': {
$cond: {
if: {
$eq: ['$data.timestamp', '$latest_timestamp'],
},
then: '$latest_timestamp',
else: '$$REMOVE',
},
},
},
},
{
$replaceRoot: {
newRoot: '$data',
},
},
//At last, after this stages I can do useful job
but I found it a bit obsolete, and I already heard that using [.mapReduce][1] could solve my problem a bit faster, than this query. (But official docs doesn't sound promising about it) Does it true?
As for now, I am using 4 or 5 stages, before start working with useful (for me) documents.
Recent update:
I have checked the $facet stage and I found it curious for this certain case. Probably it will help me out.
For what it's worth:
After receiving documents after the necessary stages I am building a representative cluster chart, that you may also know as a heatmap
After that I was iterating each document (or array of objects) one-by-one to find their correct x and y coordinated in place which should be:
[
{
x: x (number, actual $price),
y: y (number, actual $realm),
value: price * quantity,
quantity: sum_of_quantity_on_price_level
}
]
As for now, it's old awful code with for...loop inside each other, but in the future, I will be using $facet => $bucket operators for that kind of job.
So, I have found an answer to my question in another, but relevant way.
I was thinking about using $facet operator and to be honest, it's still an option, but using it, as below is a bad practice.
//building $facet query before aggregation
const ObjectQuery = {}
for (const realm of realms) {
Object.assign(ObjectQuery, { `${realm.name}` : [ ... ] }
}
//mongoose query here
aggregation([{
$facet: ObjectQuery
},
...
])
So, I have chosen a $project stage and $switch operator to filter results, such as $groups do.
Also, using MapReduce could also solve this problem, but for some reason, the official Mongo docs recommends to avoid using it, and choose aggregation: $group and $merge operators instead.
I have some dynamic query for a collection
var condition = dynmic_query
db.collection.find(condition)
is giving me an output
now what I need whatever I am getting from previous query need to lookup with an another collection.
I am scared if this is not possible yet in mongodb
[https://jira.mongodb.org/browse/SERVER-22497]
I did some google but not getting idea how to achieve this one
I have to execute this query in node.js
Please help
Thanks
It sounds like you need the Aggregation Framework with the $lookup pipeline stage with a $match pipeline. It's a bit like doing a JOIN from the SQL world. Here's an example:
db.orders.aggregate([
{
$lookup:
{
from: "warehouses",
let: { order_item: "$item", order_qty: "$ordered" },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$stock_item", "$$order_item" ] },
{ $gte: [ "$instock", "$$order_qty" ] }
]
}
}
},
{ $project: { stock_item: 0, _id: 0 } }
],
as: "stockdata"
}
}
])
You can not use one query output in second query. But you need to handle this at your code level.
hit one query and kept result in one variable then use that result to search in another query.
Mongodb is not meant for complex fetch. That's the whole reason it gives you high performance.
Here is my sample :
Two simple Mongoose models:
a Note model, with among other fields an id field that is a ref for the Notebook model.
a Notebook model, with the id I mentioned above.
My goal is to output something like that:
[
{
"notes_count": 7,
"title": "first notebook",
"id": "5585a9ffc9506e64192858c1"
},
{
"notes_count": 3,
"title": "second notebook",
"id": "558ab637cab9a2b01dae9a97"
}
]
Using aggregation and population on the Note model like this :
Note.aggregate(
[{
"$group": {
"_id": "$notebook",
"notes_count": {
"$sum": 1
}
}
}, {
"$project": {
"notebook": "$_id",
"notes_count": "$notes_count",
}
}]
gives me this kind of result :
{
"_id": "5585a9ffc9506e64192858c1",
"notes_count": 7,
"notebook": {
"_id": "5585a9ffc9506e64192858c1",
"title": "un carnet court",
"__v": 0
}
}
Forget about __v and _id fields, would be easy to handle with a modified toJSON function.
But in this function neither doc nor ret params gives me access to the computed notes_count value.
Obviously, I could manage this in the route handler (parse result and recreate the datas that will be returned) but, is there a proper way to do that with mongoose ?
You can't use the aggregate method to update. As you have noted, you'll need to use output from the aggregate constructor to update the relevant documents.
As the Mongoose aggregate method will return a collection of plain objects, you can iterate through this and utilise the _id field (or similar) to update the documents.