How to use $subtract with array of document in mongoDB - node.js

We have two collections in which have users and organizations.
db.users.aggregate([
{$match: {"organization.metaInfo":{"$ne": "disabled"}}},
{"$unwind":"$organization"},
{"$group":{"_id":"$organization.organizationId", "count":{"$sum":1}}},
{$lookup: {from: "organisations",
localField: "_id",
foreignField: "_id", as: "orgDta"}},
{"$project":{"_id":1, "count":1,
"orgDta.organizationLicence.userCount":1
}}
])
When this query is performed return a result like which is good to me.
{
"_id" : "768d3090-d4f5-11e7-a503-9b68b90cdb4e",
"count" : 5.0,
"orgDta" : [{
"organizationLicence" : {
"userCount" : 50
}
}]
},
{
"_id" : "d9933740-c29c-11e7-9481-b52c5f3e2e70",
"count" : 1.0,
"orgDta" : [{
"organizationLicence" : {
"userCount" : 1
}
}]
},
{
"_id" : "5386ebc0-c29b-11e7-9481-b52c5f3e2e70",
"count" : 1.0,
"orgDta" : [{
"organizationLicence" : {
"userCount" : 1
}
}]
}
Now, I want to perform a subtract operation in between count and userCount.But I don't know that how to use here.
I was trying together with $project
{"$project":{"_id":1, "count":1, "orgDta.dObjects":1, "orgDta.organizationLicence.userCount":1,
"remainingUser": { $subtract: [ "$orgDta.organizationLicence.userCount", "$count"]}
But Mongo returns error
{
"message" : "cant $subtract adouble from a array",
"stack" : "MongoError: cant $subtract adouble from a array" }

Use $group instead wih $unwind (before) like this,
Aggregate pipeline
db.users.aggregate([
{
$unwind: '$orgDta'
}, {
$group: {
_id: '$_id',
remainingUser: {
$push: {
$subtract: ['$orgDta.organizationLicence.userCount', '$count']
}
}
}
}
])
What we are doing here is unwind the target array, subtract all the elements (in your case, the element child's element value) and then group the array back with result (of substraction) value.
Add other items you might want in your final result document, above is just a sample MongoDB aggregate query.

Related

Query a MongoDB collection by the property of an embedded relation

I am trying to find documents in a collection, but filtered based on the value of an embedded ObjectID relation.
Mongoose schema is as follows:
const UserQualificationSchema = new Schema(
{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
}
const UserSchema = new Schema(
{
fleet: {
type: [String], // Eg ["FleetA", "FleetB", "FleetC"]
required: true,
}
}
I need to find all UserQualifications where an item in the user's fleet equals a value in a filter array.
For example: Find all User Qualifications where user.fleet: {$in: ["FleetA", "FleetC"]}
I've looked at aggregations and querying inside .populate() but can't seem to get it to work.
Any ideas much appreciated.
Use aggregation query for your problem, I have created a query for you.
users.collection.json
/* 1 */
{
"_id" : ObjectId("61056c4a8cca27df3db2e4c8"),
"firstName" : "Rahul",
"lastName" : "soni",
"fleet" : [
"FleetA",
"FleetB",
"FleetC"
],
"createdAt" : ISODate("2021-07-31T15:29:14.918Z")
}
userqualifications.collection.json
/* 1 */
{
"_id" : ObjectId("61056c908cca27df3db2e4c9"),
"user" : ObjectId("61056c4a8cca27df3db2e4c8"),
"createdAt" : ISODate("2021-07-31T15:30:24.510Z")
}
aggregation query:
it will get the result only if a user has FleetA and FleetC.
if anyone is not matched then it will return 0 records
db.userqualifications.aggregate([{
"$lookup": {
"from": "users",
"localField": "user",
"foreignField": "_id",
"as": "user"
}
}, {
"$unwind": "$user"
}, {
"$match": {
"user.fleet": {
"$elemMatch": {
"$eq": "FleetA",
"$eq": "FleetC"
}
}
}
}])
Result:
/* 1 */
{
"_id" : ObjectId("61056c908cca27df3db2e4c9"),
"user" : {
"_id" : ObjectId("61056c4a8cca27df3db2e4c8"),
"firstName" : "Rahul",
"lastName" : "soni",
"fleet" : [
"FleetA",
"FleetB",
"FleetC"
],
"createdAt" : ISODate("2021-07-31T15:29:14.918Z")
},
"createdAt" : ISODate("2021-07-31T15:30:24.510Z")
}
if the goal is to get only UserQualifications then the following should be an efficient way as it can use an index on the fleet field of the User collection.
db.User.aggregate([
{
$match: {
fleet: { $in: ["FleetA", "FleetB"] }
}
},
{
$lookup: {
from: "UserQualification",
localField: "_id",
foreignField: "user",
as: "qualifications"
}
},
{
$unwind: "$qualifications"
},
{
$replaceWith: "$qualifications"
}
])
on the other hand if you start from the UserQualifications collection, you can't efficiently narrow down the records as you're filtering on something that it doesn't have the data for.
Thank you for the answer - it did achieve the results I was looking for - however I am now struggling to add a $match with $and to the aggregate to only return the qualifications where the user ID equals one inside a submitted array AND a given fleet.
I have the following aggregate:
db.UserQualifications.aggregate([{
{
$lookup: {
from: 'users',
localField: 'user',
foreignField: '_id',
as: 'user',
},
},
{
$unwind: '$user',
},
{
$match: {
$and: [
'user.fleet': {
$in: ["Fleet A", "Fleet C"], // This works on it's own
},
user: { // Also tried 'user._id'
$in: ["6033e4129070031c07fbbf29"] // Adding this returns blank array
}
]
},
}
}])
I have double checked that I am passing in the correct User ID's inside the arrays, but when I add this to the $and inside match, it only returns a blank array.
Is there another way to do this?
// Updated users collection
/* 1 */
{
"_id" : ObjectId("61056c4a8cca27df3db2e4c8"),
"firstName" : "Rahul",
"lastName" : "soni",
"fleet" : [
"Fleet A",
"Fleet B",
"Fleet C"
],
"createdAt" : ISODate("2021-07-31T15:29:14.918Z")
}
Query:
// userqualifications => this is my collection name, you can add your collection name here db.<YOUR>
db.userqualifications.aggregate([{
$lookup: {
from: 'users',
localField: 'user',
foreignField: '_id',
as: 'user',
},
},
{
$unwind: '$user',
},
{
$match: {
// $and operatory syntax [{}, {}]
$and: [{
'user.fleet': {
// "Fleet A", "Fleet C" (FleetA, FleetC) this is the my first options,
// I have changes according to your problem
$in: ["Fleet A", "Fleet C"], // This works on it's own
}
}, {
// Convert user id to ObjectId type (_bsonType)
"user._id": ObjectId("61056c4a8cca27df3db2e4c8")
}]
}
}
])
Result:
/* 1 */
{
"_id" : ObjectId("61056c908cca27df3db2e4c9"),
"user" : {
"_id" : ObjectId("61056c4a8cca27df3db2e4c8"),
"firstName" : "Rahul",
"lastName" : "soni",
"fleet" : [
"Fleet A",
"Fleet B",
"Fleet C"
],
"createdAt" : ISODate("2021-07-31T15:29:14.918Z")
},
"createdAt" : ISODate("2021-07-31T15:30:24.510Z")
}
Difference Image:

MongoDB v4.2, Apply changes in aggregate pipline

I have a large documents like that and want to trim blabla: from all documents.
[{
"_id" : 1,
"videoHistory" : [
"blabla:FSFS",
"blabla:CZXC",
"ADSK",
"DAOW"
]
},
{
"_id" : 2,
"videoHistory" : [
"blabla:POQW",
"blabla:QWEE",
"VCXV",
"FSGG"
]
},
{
"_id" : 3,
"videoHistory" : [
"blabla:FSSS",
"AVCK",
"DAOC"
]
}
]
What I have did?
db.collection.aggregate([
{$match: {
$and: [
{'videoHistory.1': {$exists: true}},
{videoHistory: { '$elemMatch': {'$regex': 'blabla:'} }},
]}},
{ "$set": {
"videoHistory": {
"$map": {
"input": "$videoHistory",
"as": "vid",
"in": { "$ltrim": { input: "$$vid", chars: "blabla:" } }
}
}
}},
{ $project: {"videoHistory": 1}},
])
When I run the code, the result as expected, but it doesn't apply changes to documents, So my question how can i apply this to documents?
I'm using MongoDB V4.2
this aggregation just provides the projected result to somewhere for example to the client side or to the shell, but doesn't update the original documents. Try $merge. Based on the doc, you should use the MongoDB 4.4 to output to the same collection that is being aggregated.

Mongodb aggregation distinct, average and sum with group by between two collection

I have user collection having data like this
{
"_id" : ObjectId("5da594c15324fec81d000027"),
"password" : "******",
"activation" : "Active",
"userType" : "Author",
"email" : "something#gmail.com",
"name" : "Something",
"profilePicture" : "profile_pictures/5da594c15324fec81d0000271607094354423image.png",
"__v" : 0
}
and On the other hand userlogs has data like this
{
"_id" : ObjectId("5fcb7bb4485c34a41900002b"),
"duration" : 2.54,
"page" : 1,
"activityDetails" : "Viewed Page for seconds",
"contentType" : "article",
"activityType" : "articlePageStayTime",
"label" : 3,
"bookId" : ObjectId("5f93e2cc74153f8c1800003f"),
"ipAddress" : "::1",
"creator" : ObjectId("5da594c15324fec81d000027"),
"created" : ISODate("2020-12-05T12:23:16.867Z"),
"__v" : 0
}
What I am trying to do is equivalent to this sql query
SELECT name,count(page),sum(duration),avg(DISTINCT(label)),COUNT(DISTINCT(bookId)) FROM users JOIN userlogs ON users._id=userlogs.creator where userlogs.activityType<>"articleListeningTime" group by users._id.
I can do normal group by and sum together.But How to do avg distinct and count distinct with this? I am using mongodb version 3.2
I don't think this require $group stage, you can use $setUnion and $size, $avg operators,
$lookup with userlogs collection
$project to show required fields, and filter userlogs as per your condition
$project to get statistics from userlogs
get total logs count using $size
get total duration sum using $sum
get average of unique label using $setUnion and $avg
get count of unique bookId using $serUnion and $size
db.users.aggregate([
{
$lookup: {
from: "userlogs",
localField: "_id",
foreignField: "creator",
as: "userlogs"
}
},
{
$project: {
name: 1,
userlogs: {
$filter: {
input: "$userlogs",
as: "u",
cond: { $ne: ["$$u.activityType", "articleListeningTime"] }
}
}
}
},
{
$project: {
name: 1,
totalCount: { $size: "$userlogs" },
durationSum: { $sum: "$userlogs.duration" },
labelAvg: { $avg: { $setUnion: "$userlogs.label" } },
bookIdCount: { $size: { $setUnion: "$userlogs.bookId" } }
}
}
])
Playground

Using mongose aggregation to populate from collection in mongo

Sample document
{
_id:"123",
"completed" : [
{
"Id" : ObjectId("57caae00b2c40dd21ba089be")
"subName" : "oiyt",
"Name" : "Endo",
},
{
"Id" : ObjectId("57caae00b2c40dd21ba089be"),
"subName" : "oiyt",
"Name" : "Endo",
}
]
}
How do I access the name and subname from complete where _id matches?
You can use $filter or $unwind (or both).
This example shows how to use $filter to get the document only with one matched element in the array, and then $unwind to get easier access to the matched element.
but there are many more options to get the desired result.
db.collection.aggregate([
{
$project: {
filtered_completed: {
$filter:{
input: "$completed",
as: "complete",
cond: {
$eq: [input_id, "$$complete.Id"]
}
}
}
}
},
{
$unwind: "$filtered_completed"
// because we already filtered the 'completed' array, we will get only one document.
// but you can use it as the first aggreagation pipeline stage and match the _id
},
{
$project: {
"filtered_completed.Name": 1,
"filtered_completed.subName": 1
}
}
])
read more about $filter and $unwind

MongoDb - $match filter not working in subdocument

This is Collection Structure
[{
"_id" : "....",
"name" : "aaaa",
"level_max_leaves" : [
{
level : "ObjectIdString 1",
max_leaves : 4,
}
]
},
{
"_id" : "....",
"name" : "bbbb",
"level_max_leaves" : [
{
level : "ObjectIdString 2",
max_leaves : 2,
}
]
}]
I need to find the subdocument value of level_max_leaves.level filter when its matching with given input value.
And this how I tried,
For example,
var empLevelId = 'ObjectIdString 1' ;
MyModel.aggregate(
{$unwind: "$level_max_leaves"},
{$match: {"$level_max_leaves.level": empLevelId } },
{$group: { "_id": "$level_max_leaves.level",
"total": { "$sum": "$level_max_leaves.max_leaves" }}},
function (err, res) {
console.log(res);
});
But here the $match filter is not working. I can't find out exact results of ObjectIdString 1
If I filter with name field, its working fine. like this,
{$match: {"$name": "aaaa" } },
But in subdocument level its returns 0.
{$match: {"$level_max_leaves.level": "ObjectIdString 1"} },
My expected result was,
{
"_id" : "ObjectIdString 1",
"total" : 4,
}
You have typed the $match incorrectly. Fields with $ prefixes are either for the implemented operators or for "variable" references to field content. So you just type the field name:
MyModel.aggregate(
[
{ "$match": { "level_max_leaves.level": "ObjectIdString 1" } },
{ "$unwind": "$level_max_leaves" },
{ "$match": { "level_max_leaves.level": "ObjectIdString 1" } },
{ "$group": {
"_id": "$level_max_leaves.level",
"total": { "$sum": "$level_max_leaves.max_leaves" }
}}
],
function (err, res) {
console.log(res);
}
);
Which on the sample you provide produces:
{ "_id" : "ObjectIdString 1", "total" : 4 }
It is also good practice to $match first in your pipeline. That is in fact the only time an index can be used. But not only for that, as without the initial $match statement, your aggregation pipeline would perform an $unwind operation on every document in the collection, whether it met the conditions or not.
So generally what you want to do here is
Match the documents that contain the required elements in the array
Unwind the array of the matching documents
Match the required array content excluding all others

Resources