I have a collection of songs and its metadata with the following structure:
[{
title:"title",
artist:"artist",
album:"album,
...
},...
Now I want to get a list of every artist with the number of songs and the number of albums it has using Node.js. So far, using the aggregation framework, I've been able to get an array of objects with each artist, its number of songs and an array with the album titles (instead of just the count), using the following pipeline:
collection.aggregate([
{ $project:{
artist:1,
album:1
}},
{ $group: {
_id: "$artist",
songs:{$sum: 1},
albums:{$addToSet:"$album"}
}},
{ $sort: { artist: 1 } }
]
If I replace $addToSet with $sum, I get albums:0 in every artist, because it expects numbers and not strings to sum.
I just can't get around it!
You need to add a couple of steps to your pipeline - the array of albums needs to be unwound and then counted. Here is what it would look like:
collection.aggregate([
{ $project:{
artist:1,
album:1
}},
{ $group: {
_id: "$artist",
songs:{$sum: 1},
albums:{$addToSet:"$album"}
}},
{ $unwind: "$albums"},
{ $group: {
_id: "$_id",
songs:{$first: 1},
albums:{$sum: 1}
}},
{ $sort: { artist: 1 } }
]
)
Related
I have a sample mongoose object that looks like this:
{
_id: 5fa849ad4f6be0382363809c,
ratings: {
ratedPersonId: 7,
rating: 7,
timeSpent: 30,
timestamp: 78,
userThreshold: 5
}
},
it contains an _id and a list of ratings which is a subdocument with the following features.
I have created an aggregation pipeline like this:
const ratedUser = await this.ratingModel
.aggregate([
{ $project: { ratings: 1 } },
{ $unwind: '$ratings' },
{
$match: {
$and: [{ 'ratings.ratedPersonId': userId }, { 'ratings.rating': { $gte: 5 } }],
},
},
])
.exec()
This works for the first condition ratings.ratedPersonId: userId
My problem is that my second condition is the rating should be greater than or equal to the userThreshold field in the same object.
whenever I type that in the query it returns nothing
$and: [{ 'ratings.ratedPersonId': userId }, { 'ratings.rating': { $gte: 'ratings.threshold'} }],
Demo - https://mongoplayground.net/p/AQMsJGkoFcu
Use $expr to compare the fields
Read aggregation-expressions
$expr can build query expressions that compare fields from the same document in a $match stage.
If the $match stage is part of a $lookup stage, $expr can compare fields using let variables. See Specify Multiple Join Conditions with $lookup for an example.
$expr only uses indexes on the from the collection for equality matches in a $match stage.
$expr does not support multikey indexes.
db.collection.aggregate([
{
$project: {
ratings: 1
}
},
{
$unwind: "$ratings"
},
{
$match: {
$and: [
{
"ratings.ratedPersonId": 7
},
{
$expr: {
$gte: [
"$ratings.rating",
"$ratings.userThreshold"
]
}
}
],
},
},
])
I need to get docs from MongoDB collection where ID's are in array:
[
'5f80a44d0179262f7c2e6a42',
'5f8c00762fae890e9c4d029c',
'5f802cf8abac1116a46bf9d4'
]
The issue is, docs are not coming in sequence of my array ID's. They are coming (1, 0, 2) for above array ID's.
How can I make them in sequence of my ID's array? I am using, NodeJs + Mongoose.
My code:
var ids = ['5f80a44d0179262f7c2e6a42','5f8c00762fae890e9c4d029c','5f802cf8abac1116a46bf9d4']
Product.find({
_id: {
$in: ids
}
})
I don't think its possible with find(), or any functionality available in MongoDB related to this,
It is possible with aggregate() but this will just fulfil your expected result,
I am not recommending to use this because this will affect performance of query response, this could be a heavy transaction.
$match your conditions
$group by null and make array of all matching objects in root
$addFields to add ids array that we search for
$unwind deconstruct ids array, this will deconstruct in sequence as per we provided in ids array
$project, $reduce input as root array and check condition if id match then return object
$replaceWith to replace root object to root
var ids = [
ObjectId("5f802cf8abac1116a46bf9d4"),
ObjectId("5f8c00762fae890e9c4d029c"),
ObjectId("5f80a44d0179262f7c2e6a42")
];
Product.aggregate([
{ $match: { _id: { $in: ids } } },
{
$group: {
_id: null,
root: { $push: "$$ROOT" }
}
},
{ $addFields: { ids: ids } },
{ $unwind: "$ids" },
{
$project: {
root: {
$reduce: {
input: "$root",
initialValue: {},
in: { $cond: [{ $eq: ["$$this._id", "$ids"] }, "$$this", "$$value"] }
}
}
}
},
{ $replaceWith: "$root" }
])
Playground
I am trying to do something very simple but an new to MongoDB! I have a document called Device and a sub-document called Movement. I want to get the last two movement sub-documents out of Device ordered by last_seen (a date). Here is what I have along with the error I am getting:
Device.findOne({device_id: "1234"}, {movements: { $sort: {last_seen: -1}, $slice: 2 }}, function(err, device){
...
});
The Error:
MongoError: >1 field in obj: { $sort: { last_seen: -1 }, $slice: 2 }
You can use aggregate:
Device.aggregate(
{ $match: { device_id: "1234"}}, // query documents (can return more than one element)
{ $unwind: '$movements'}, //deconstruct the documents
{ $sort: { '$movements.last_seen': -1}},
{ $limit: 2 },
{ $group: { _id: '$device_id', movements: { $push: '$movements }}} //reconstruct the documents
function(err, devices){
//returns an array, probably with one elements depending on `$match` query
});
I have a data set which is the following:
{
item: '123',
array: [{
array2:[{
array3: [{
property1: 1234
}]
}],
anotherArray: [{
property2: 1234
}]
}]
}
Im trying to aggregate sum of property2 and property one in the same request.
here is my current aggregate function:
Item.aggregate([
{$match: {itemId: 1234}},
{$unwind: "$array"},
{$unwind: "$array.array2"},
{$unwind: "$array.array2.array3"},
{$unwind: "$array.anotherArray"},
{$group: {
_id: 0,
property1: {$sum: '$array.array2.array3.property1'},
property2: {$sum: '$array.anotherArray.property2'}
}},
{$project: {
_id: 0,
property1: "$property1",
property2: "$property2",
}},
], function (err, aggregate) {
callback(null, aggregate);
});
The problem is that the aggregates results of property one and two are always double the value they should be.
I guess the problem is with the $unwind of "anotherArray", because when i remove it I get the correct aggregation value.
Is it possible to make aggregation on multiple arrays with one aggregation function?
Currently im just making 2 different requests to the db with async parallel, but I want to make more complex aggregation in the future without making extra db calls.
As noted the structure is not a good one and should probably be reviewed as to it's intent. It's really not clear why it is so structured or if anything else in the array's in either case could mess up the results here.
But there is a general approach when you have multiple arrays in a document, that is basically to treat each array separately and get your "totals" per document first. Then sum the totals from all documents afterwards:
Item.aggregate([
// Unwind only 1 inner array first
{ "$unwind": "$array" },
{ "$unwind": "$array.array2" },
{ "$unwind": "$array.array2.array3" },
// Group back the sum of the element and the first of the other array
// and only per document
{ "$group": {
"_id": "$_id",
"property1": { "$sum": "$array.array2.array3.property1" },
"anotherArray": { "$first": "$array.anotherArray" }
}},
// Unwind the other array
{ "$unwind": "$anotherArray" },
// Group back the total and the first summed per document
{ "$group": {
"_id": "$_id",
"property1": { "$first": "$property1" },
"property2": { "$sum": "$anotherArray.property2" }
}},
// Total all documents and output
{ "$group": {
"_id": null,
"property1": { "$sum": "$property1" },
"property2": { "$sum": "$property2" },
}},
{ "$project": {
"_id": 0,
"property1": 1,
"property2": 1
}}
],callback);
So by containing to only one array at a time and getting the totals only within the original document first you avoid the duplication problems of creating multiple copies for each unwound item of the other array. With discrete document totals it is then simple to get the overall totals from your required selection.
Finally I've found a solution for my use case with MongoDB $setUnion.
Here is the code i used for my question:
Item.aggregate([
{$match: { itemID: '1234'}},
{$unwind: "$array1"},
{$unwind: "$array1.array2"},
{$project: {
_id: 0,
combined: {$setUnion: ['$array1.anotherArray', '$array1.array2.array3']},
}},
{$unwind: "$combined"},
{$group: {
_id: 0,
property1: {$sum: '$combined.property1'},
property2: {$sum: '$combined.property2'}
}},
], function (err, aggregate) {
cb(aggregate);
});
I have the following MongoDB documents:
{
_id: ObjectId('09de14821345dda65c471c99'),
items: [
_id: ObjectId('34de64871345dfa655471c99'),
_id: ObjectId('34de64871345dfa655471c91'),
_id: ObjectId('34de64871345dfa655471c99'),
]
},
{
_id: ObjectId('09de14821345dda65c471c98'),
items: [
_id: ObjectId('24de64871345dfa61271c10'),
_id: ObjectId('24de64871345dfa61271c11'),
_id: ObjectId('24de64871345dfa61271c11'),
]
},
{
_id: ObjectId('09de14821345dda65c471c07'),
items: [
_id: ObjectId('24de64871345dfa61271c05'),
_id: ObjectId('24de64871345dfa61271c06'),
_id: ObjectId('24de64871345dfa61271c07'),
]
}
I need to find all documents with repeated items array elements. So from the documents above I want to get the following result:
db.collection.documents.find({/** need query*/}).toArray(function (err, documents) {
console.dir(documents); // documents with id's 09de14821345dda65c471c99 and 09de14821345dda65c471c98
});
How could I do that?
In order to group and match results you will need to use the Aggregation Framework or Map/Reduce rather than a simple find() query.
Example data
Your example document include some errors: a few of the ObjectIDs are too short and the array elements should either be embedded documents ({_id: ObjectId(...)}) or simple values.
For test data I've used:
db.mydocs.insert([
{
_id: ObjectId('09de14821345dda65c471c99'),
items: [
ObjectId('34de64871345dfa655471c99'),
ObjectId('34de64871345dfa655471c91'),
ObjectId('34de64871345dfa655471c99')
]
},
{
_id: ObjectId('09de14821345dda65c471c98'),
items: [
ObjectId('24de64871345ddfa61271c10'),
ObjectId('24de64871345ddfa61271c11'),
ObjectId('24de64871345ddfa61271c11')
]
},
{
_id: ObjectId('09de14821345dda65c471c07'),
items: [
ObjectId('24de64871345ddfa61271c05'),
ObjectId('24de64871345ddfa61271c06'),
ObjectId('24de64871345ddfa61271c07')
]
}])
Aggregation query
Here is an aggregation query using the mongo shell:
db.mydocs.aggregate(
// Unpack items array into stream of documents
{ $unwind: "$items" },
// Group by original document _id and item
{ $group: {
_id: { _id: "$_id", item: "$items" },
count: { $sum: 1 }
}},
// Limit to duplicated array items (1 or more count per document _id)
{ $match: {
count: { $gt: 1 }
}},
// (Optional) clean up the result formatting
{ $project: {
_id: "$_id._id",
item: "$_id.item",
count: "$count"
}}
)
Sample results
{
"_id" : ObjectId("09de14821345dda65c471c98"),
"count" : 2,
"item" : ObjectId("24de64871345ddfa61271c11")
}
{
"_id" : ObjectId("09de14821345dda65c471c99"),
"count" : 2,
"item" : ObjectId("34de64871345dfa655471c99")
}