Aggregate recursive tree structure (parent reference) - node.js

We are trying to build a simple CMS that can manage pages and its subPages. And every subPage can also have subPages etc.
The Problem we are currently facing is that we receive a flat tree after the second level and we would have to reorganise the tree with JS.
Is it possible to receive a real tree structure (see "desired output") without the use of JS.
The following tree structure is currently used:
[{
_id: 1,
name: "top-level",
parent: null
},{
_id: 2,
name: "second-level-a",
parent: 1 // top-level
},{
_id: 3,
name: "second-level-b",
parent: 1 // top-level
},{
_id: 4,
name: "third-level",
parent: 2 // second-level-a
}]
We are currently trying to receive the complete tree via this query:
Page.aggregate([
{
"$graphLookup": {
"from": "Page",
"startWith": "$_id",
"connectFromField": "_id",
"connectToField": "parent",
"as": "subPages",
"maxDepth": 1,
"depthField": "depth"
}
}, {
"$match": {
"options.parent": null
}
}
]).exec((err, pages) => {
if (err) {
res.send(err);
return;
}
res.json(pages);
});
What we are receiving is something like this:
[
{
_id: 1,
name: "first-level",
parent: null,
subPages: [
{
_id: 2,
name: "second-level-a",
depth: 0,
[...]
},{
_id: 3,
name: "second-level-b",
depth: 0,
[...]
},{
_id: 4,
name: "third-level",
depth: 1,
[...]
}
]
}
]
With this output it's not directly visible where third-level is placed inside the tree. The desired output should look like this:
[
{
_id: 1,
name: "first-level",
parent: null,
subPages: [
{
_id: 2,
name: "second-level-a",
depth: 0,
[...]
subPages: [
{
_id: 4,
name: "third-level",
depth: 1,
[...]
}
]
},{
_id: 3,
name: "second-level-b",
depth: 0,
[...]
}
]
}
]
The question is, is this even possible to achieve using MongoDB in combination with mongoose or do we need to sort the result with JS.

Related

Find rank of a player based on 2 fields in a collection in mongoose

I have more than 100,000 documents in a collection and I want to get rank of player based on his totalCoins field first and in case 2 players have same coins, then based on their level.
Below is Player collection example:
const players = [
{
_id: 1,
name: "abc",
totalCoins: 100,
level: 1
},
{
_id: 2,
name: "bcd",
totalCoins: 200,
level: 2
},
{
_id: 3,
name: "cde",
totalCoins: 300,
level: 3
},
{
_id: 4,
name: "def",
totalCoins: 100,
level: 4
},
{
_id: 5,
name: "efg",
totalCoins: 100,
level: 4
}
]
Let's say I am finding rank of player with _id = 4.
How can i achieve this in mongoose.
Thanks in advance.
If you are using MongoDB 5, then $rank method should work (not 100% sure because I am using version 4.4 ).
If you are not using 5.x then this should work
Step 1 - sorted the data based on level first and then totalCoins
Step 2 - pushed the sorted data in an array using group and push
Step 3 - unwinded the data and added the index of document based on sorted array with the help of includeArrayIndex and named it globalRank (name it whatever you want :) )
Step 4 - finally projected the data again and added 1 in globalRank so that the rank starts at 1 instead of 0 (this return the rank like 1.0 or 2.0... if you don't want this then you can avoid $add and handle this in frontend)
Step 5 - $match to find a specific index
db.collection.aggregate([
{
$project: {
_id: 1,
name: "$name",
totalCoins: "$totalCoins",
level: "$level",
},
},
{
$sort: {
level: -1,
totalCoins: -1,
},
},
{
$group: {
_id: null,
arr: {
$push: {
_id: "$_id",
name: "$name",
totalCoins: "$totalCoins",
level: "$level",
},
},
},
},
{
$unwind: {
path: "$arr",
includeArrayIndex: "globalRank",
},
},
{
$project: {
_id: "$arr._id",
name: "$arr.name",
level: "$arr.level",
totalCoins: "$arr.totalCoins",
globalRank: { $add: ["$globalRank", 1] },
},
},
{ $match: { _id: 4 } },
]);
collection.count({
"$expr": {
"$gt": [
{ "$add": [ "$totalCoins", {"$divide":["$level",10000] ] },
100.0004
]
}
})
The idea is to count the number of records with higher coins/level than the user. The idea I put here is to divide the level by maximum level + 1 (here the level is max 9999) so you will always end up with lower than 1 decimal. When you add the coins to that you will get 100.0004, coins.level . Then it is just to compare two numbers accordingly.

Filter on tag-field on children

First of all, I'm not sure I've set this up as it should be, like by the book. I'm from the SQL world and jumped into the NOSQL land.
Ok, so. I have this collection with Projects, and inside the projects I have files as a child-ref. I can populate and all that stuff, works really well. But I want to filter with tags. I have a tags field inside the File collection, an array with strings, pretty straight forward.
What I would like to do is; send a projectId and a string with a spec filter and get the files, belonging to the project and also containing the tag. Oh, and also, populated.
Is this even the right approach with NOSQL/MONGO? I know how I would do it in SQL, with parent_id's and with some joins etc. I've looked into some aggregate but I'm too novice to work it out it seems.
edit, just to show how my collections are built:
Project Collection
[{
id: 1,
name: 'Project01',
files: [
id: 1,
id: 2,
id: 3,
id: 4,
id: 5,
...
]
},
...
]
Files Collection
[{
id: 1,
name: 'filename'
tags: ['a','b']
},{
id: 2,
name: 'filename2'
tags: ['b', 'c']
},{
id: 3,
name: 'filename3'
tags: ['a', 'd', 'e', 'f']
},
...]
The result I'm going for (get all files in project 1 where tags includes 'b'.
{
id: 1,
name: 'Project01',
files: [
{
id: 1,
name: 'filename'
tags: ['a','b']
},{
id: 2,
name: 'filename2'
tags: ['b', 'c']
}
]
}
try this $unwind operation in mongodb
Collections as per your requirement
[
{
_id: 1,
name: "Project01",
files: [
{
id: 1,
name: "filename11",
tags: [
"a",
"b"
]
},
{
id: 2,
name: "filename12",
tags: [
"b",
"c"
]
},
{
id: 3,
name: "filename13",
tags: [
"a",
"c"
]
}
]
},
{
_id: 2,
name: "Project02",
files: [
{
id: 1,
name: "filename21",
tags: [
"a",
"b"
]
},
{
id: 2,
name: "filename22",
tags: [
"a",
"c"
]
},
{
id: 3,
name: "filename23",
tags: [
"b",
"c"
]
}
]
}
]
Method 1: for your project collection
db.collection.aggregate([
{
$match: {
_id: 1
}
},
{
$unwind: "$files"
},
{
$match: {
_id: 1,
"files.tags": {
$in: [
"b"
]
}
}
}
])
Method 2 for files collection
db.collection.aggregate([
{
$unwind: "$tags"
},
{
$match: {
tags: "xyz"
}
}
])
Try it here Mongoplayground

Mondogb $sum not working with SchemaTypes.Double

i'm fairly new to MongoDB and Mongoose and i'm working on a bug here. Apparently $sum is not working with a field whose type is SchemaTypes.Double. This double type is available thanks to a package called mongoose-double.
I don't know if MongoDB doesn't support Double so that's why we have this package, or is this because of MongoDB Version (it's on 3.6 AFAIK). But anyways, here's the code:
Schedule.aggregate([{
$match: findTerm
},
{
$facet: {
totalizer: [{
$group: {
_id: '$store',
totalServices: {
$sum: 1
},
totalValue: {
$sum: '$value'
},
totalComission: {
$sum: '$comissionValue'
}
}
}
],
data: [{
$project: {
'employee.name': 1,
'customer.name': 1,
'service.name': 1,
'info.channel': 1,
value: 1,
scheduleDate: 1,
scheduleStart: 1,
scheduleEnd: 1,
comissionValue: 1,
status: 1,
paymentMethod: 1
}
},
{
$sort: sort
},
{
$skip: req.body.limit * req.body.page
},
{
$limit: req.body.limit
}
]
}
}
]).exec((e, response) => {
if (e) {
// handle error
}
res.status(200).send(response[0]);
});
This findTerm is sent by the frontend app and has this format:
{ store: '5b16cceb56a44e2f6cd0324b',
status: { '$in': [ 0, 1, 2, 3 ] },
paymentMethod: { '$in': [ 0, 1, 2, 3, 4, 5 ] },
'info.channel': { '$in': [ 'app', 'admin' ] },
scheduleStart: { '$gte': '2019-11-01 00:00' },
scheduleEnd: { '$lte': '2020-03-31 23:59' }
}
My comissionValue field is in the root of my Schedule Schema:
comissionValue: {
type: SchemaTypes.Double,
default: 0
},
But my result is the following, as shown in my console.log in the frontend
As you can see my totalComission inside my totalizer is null, but my first object inside data has a comissionValue of 0.6.
How can i kno what's wrong here? I've tried different combinations of $facet, filtering only Schedules that has a comissionValue not equal 0 and null, but i only got a result of 0 for totalComission.
EDIT
Here's some sample data:
A Schedule object:
customer: {
id: "5e41a7ba11340930742aa689",
name: "renan lima",
phone: "+5511999999999",
email: "sampleemail#gmail.com",
doc: "00000000000",
avatar: null
},
employee: {
id: "5b16cebd29bcf613f02b6fb4",
name: "Anderson Zanardi",
avatar: "anderson.jpg"
},
service: {
noValue: false,
filters: [],
id: "5b637acd634e14086c9a3aea",
name: "Barba Masculina"
},
info: {
channel: "app",
id: "5e41a7ba11340930742aa689",
name: "renan lima"
},
comissionType: null,
comissionValue: 0,
paymentMethod: 0,
_id: "5e41a7c011340930742aa68a",
store: "5b16cceb56a44e2f6cd0324b",
scheduleDate: "2020-03-16T15:00:00.000Z",
scheduleStart: "2020-03-16 09:00",
scheduleEnd: "2020-03-16 09:30",
status: 2,
value: 30,
color: "blue",
logStatus: [],
__v: 0,
created: "2020-02-10T18:58:08.845Z",
updated: "2020-02-10T18:58:08.845Z"
My response received for the Schedule.aggregate:
{
"totalizer": [{
"_id": null,
"storesCount": [{
"store": "5b16cceb56a44e2f6cd0324b",
"count": 12
}],
"totalValue": 410.5,
"totalServices": 12,
"totalComission": 75
}],
"data": [{
"_id": "5e5d04dcb4a2f42204598ebf",
"service": {
"name": "Outros"
},
"info": {
"channel": "admin"
},
"comissionValue": 0,
"paymentMethod": 0,
"customer": {
"name": "teste"
},
"employee": {
"name": "Gabriel Barreto"
},
"scheduleDate": "2020-03-02T13:06:00.000Z",
"scheduleStart": "2020-03-02 10:06",
"scheduleEnd": "2020-03-02 10:36",
"status": 0,
"value": null
}]
}
Here the comission is 75, i don't know if it's because of the scheduleStart and scheduleDate in the findTerm on my $match that this time is starting at 2020-03-01 and ends at 2020-03-31 and in that range there's 3 schedules with 25 of comission.
Maybe my pagination is making it return null? Since i need it to sum all my comission for a given start/end range, even if in a certain page it doesn't have a comission.
EDIT 2
I added a sample data in Mongo Playground, the schedule array in the configuration column matchs the query used in the $match property on the query column.
Here's the link: https://mongoplayground.net/p/nmyAsY4g7LS

How to find Mongoose data recursively?

I am newbie in MEANJS and i have a problem i.e, there are collection called employee and have multiple documents with their boss field. Now i want get all employees with their lower level.
For example:-
1) {_id:ObjectId('587dcd3edca5f235f862fdfd'), name:'John'} //he doesn't have boss
2) {_id:ObjectId('587dcd3edca5f235f86dddew'), name: 'Jimmy', 'boss': ObjectId('587dcd3edca5f235f862fdfd')} //john is boss
3) {_id:ObjectId('587dcd3edca5f235f863ew'), name: 'David', 'boss': ObjectId('587dcd3edca5f235f86dddew')} //john,Jimmy are bosses
4) {_id:ObjectId('587dcd3edca5f235f86qwa'), name: 'Dyan', 'boss': ObjectId('587dcd3edca5f235f86dddew')} //john,Jimmy,David are bosses
5) {_id:ObjectId('587dcd3edca5f235f8ew32'), name:'Jack', 'boss': ObjectId('587dcd3edca5f235f862fdfd')} //john is boss
6) {_id:ObjectId('587dcd3edca5f2wsw23rlot'), name: 'Loren', 'boss':ObjectId('587dcd3edca5f235f8ew32')} //john,Jack is boss
If we take
Jonh then output will ['Jimmy','Jack','David','Dyan','Loren']
Jack then output will ['Loren']
Here is my try code:-
getBosses(user._id)
function getBosses(id){
User.find({boss:id})
.exec(function(err,users){
if(err)
return console.log(err);
//How handle here 'users' array
//for something getBosses call recursively
})
}
As far as I understood you need to find all subordinates of that people. I think the best way to do it is using $graphLookup.
db.bosses.insertMany([
{ _id: "587dcd3edca5f235f862fdfd", name: "John" },
{
_id: "587dcd3edca5f235f86dddew",
name: "Jimmy",
boss: "587dcd3edca5f235f862fdfd",
},
{
_id: "587dcd3edca5f235f863ew",
name: "David",
boss: "587dcd3edca5f235f86dddew",
},
{
_id: "587dcd3edca5f235f86qwa",
name: "Dyan",
boss: "587dcd3edca5f235f86dddew",
},
{
_id: "587dcd3edca5f235f8ew32",
name: "Jack",
boss: "587dcd3edca5f235f862fdfd",
},
{
_id: "587dcd3edca5f2wsw23rlot",
name: "Loren",
boss: "587dcd3edca5f235f8ew32",
},
]);
db.bosses.aggregate([
{
$graphLookup: {
from: "bosses",
startWith: "$_id",
connectFromField: "_id",
connectToField: "boss",
as: "subordinates",
},
},
{
$project: {
_id: false,
name: true,
subordinates: {
$reduce: {
input: "$subordinates",
initialValue: "",
in: { $concat: ["$$value", ", ", "$$this.name"] },
},
},
},
},
{
$project: {
name: true,
subordinates: { $substrBytes: ["$subordinates", 2, -1] },
},
},
]);
The result of the last one is:
[
{ name: 'John', subordinates: 'David, Dyan, Loren, Jack, Jimmy' },
{ name: 'Jimmy', subordinates: 'David, Dyan' },
{ name: 'David', subordinates: '' },
{ name: 'Dyan', subordinates: '' },
{ name: 'Jack', subordinates: 'Loren' },
{ name: 'Loren', subordinates: '' }
]
The most important thing is $graphLookup stage of the aggregate pipeline. Last two $project stages is just response formatting - return only name and subordinates as string field with comma separated names.
To get data for a specific person you can use $match stage before $graphLookup like that:
db.bosses.aggregate([
{ $match: { name: "John" } },
{
$graphLookup: ...

MongoDB find projection, matching elements with $ne

I have a query that returns objects containing an array of objects. Within that array of objects, there are some that should not be processed. Here is an object simmilar to what I have:
{
_id: 12345,
data: [
{
state: 1,
info: "abc"
},{
state: 2,
info: "cde"
},{
state: 2,
info: "efg"
}
]
}
I want to show only the objects where state does not equal to 1. So I want to get back something like this:
{
_id: 12345,
data: [
{
state: 2,
info: "cde"
},{
state: 2,
info: "efg"
}
]
}
There can be hundreds of "main" objects with tens of "sub" objects. I tried using the query:
col.find({'data.info': {$in: [] }, {_id: 1, data: { $elemMatch: { state: {$ne: 1 } } } }, {}, callback);
But that only gives me this:
{
_id: 12345,
data: [
{
state: 2,
info: "cde"
}
]
}
In other words, $elemMatch does what it is supposed to do, but I need to get a different result. So is there a way to do that in one query or without pre-processing results (removing entries before any further code reads the data)?
The $elemMatch projection operator only returns the first matching element in an array.
To filter the whole array, the best approach in MongoDB 2.2+ would be using the Aggregation Framework. Alternative, you could also do this using Map/Reduce or in your application code.
Example aggregation:
db.data.aggregate(
// Initial match to limit number of documents
{ $match : {
data: { $elemMatch: { state: {$ne: 1 } } }
}},
// Convert the data array into a stream of documents
{ $unwind: "$data" },
// Limit to matching elements of the data array
{ $match : {
"data.state": {$ne: 1 }
}},
// Re-group by original document _id
{ $group: {
_id: "$_id",
data: { $push: "$data" }
}}
)
Sample output:
{
"_id" : 12345,
"data" : [
{
"state" : 2,
"info" : "cde"
},
{
"state" : 2,
"info" : "efg"
}
]
}

Resources