Is there an easy way to "$push" all fields of a document?
For example:
Say I have a Mongo collection of books:
{author: "tolstoy", title:"war & peace", price:100, pages:800}
{author: "tolstoy", title:"Ivan Ilyich", price:50, pages:100}
I'd like to group them by author - for each author, list his entire book objects:
{ author: "tolstoy",
books: [
{author: "tolstoy", title:"war & peace", price:100, pages:800}
{author: "tolstoy", title:"Ivan Ilyich", price:50, pages:100}
]
}
I can achieve this by explicitly pushing all fields:
{$group: {
_id: "$author",
books:{$push: {author:"$author", title:"$title", price:"$price", pages:"$pages"}},
}}
But is there any shortcut, something in the lines of:
// Fictional syntax...
{$group: {
_id: "$author",
books:{$push: "$.*"},
}}
You can use $$ROOT
{ $group : {
_id : "$author",
books: { $push : "$$ROOT" }
}}
Found here: how to use mongodb aggregate and retrieve entire documents
Actually you cant achieve what you are saying at all, you need $unwind
db.collection.aggregate([
{$unwind: "$books"},
{$group: {
_id: "$author",
books:{$push: {
author:"$books.author",
title:"$books.title",
price:"$books.price",
pages:"$books.pages"
}},
}}
])
That is how you deal with arrays in aggregation.
And what you are looking for to shortcut typing all of the fields does not exist, yet.
But specifically because of what you have to do then you could not do that anyway as you are in a way, reshaping the document.
If problem is that you don't want to explicitly write all fields (if your document have many fields and you need all of them in result), you could also try to do it with Map-Reduce:
db.books.mapReduce(
function () { emit(this.author, this); },
function (key, values) { return { books: values }; },
{
out: { inline: 1 },
finalize: function (key, reducedVal) { return reducedVal.books; }
}
)
Related
unfortunately I can only find very specific questions on stackoverflow and no exact details in the documentation, so here is a more general example that may help others too.
I just want to add a value to the array (arr) in the object with the "title: 'title3".
{
_id: <id>,
prop1: val1,
prop2: val2,
prop3: [
{
title: 'title1',
arr: ['val1', 'val2', 'val3'],
},
{
title: 'title2',
arr: ['val1', 'val2', 'val3'],
},
{
title: 'title3',
arr: ['val1', 'val2', 'val3'], //only update this array
}
]
}
My current approach looks something like this:
SomeModel.findOneAndUpdate(
{ _id: id, "prop3.title": "title3" },
{$push: { "prop3.$[].arr": someDoc._id.toString() }},
(err, doc) => {
if (err) {
console.log('Error updating doc: ', err);
resolve(false);
} else {
resolve(doc);
}
}
);
However, the problem here is that a value is added not only in the array of the object with the title "title3", but everywhere.
How can I add a value exclusively to the array in the object with the title value "title3"?
I would also be very grateful for links to documentation explaining this.
You can use positional $ operator in this way to do it in a single operation:
Using $ you tell mongo "update the object found in the find stage". In this case "update the object where prop3.title is title3".
Note that you are using arrayFilters instead of positional operator.
db.collection.update({
"_id": 1,
"prop3.title": "title3"
},
{
"$push": {
"prop3.$.arr": "val4"
}
})
Example here
This is one solution which may or may not make sense in your full context.
You would have much more luck using the mongoose Document.save() method. Once you have a local copy of the document, you can simply push to the array:
const doc = await SomeModel.findOne({
_id: id,
"prop3.title": "title3"
});
doc.prop3[3].arr.push(item);
await doc.save();
$project: {
_id: 1,
edited: 1,
game: {
gta: {
totalUserNumber: {
$reduce: {
input: "$gta.users",
initialValue: 0,
in: { $add: [{ $size: "$$this" }, "$$value"] },
},
},
userList: "$gta.users", <----- paginating this
},
DOTA2: {
totalUserNumber: {
$reduce: {
input: "$dota2.users",
initialValue: 0,
in: { $add: [{ $size: "$$this" }, "$$value"] },
},
},
userList: "$dota2.users", <------ paginating this
},
},
.... More Games
},
I have this $project. I have paginated the list of games by using $facet,$sort, $skip and $limit after $project.
I am trying also trying to paginate each game's userList. I have done to get the total value in order to calculate the page number and more.
But, I am struggling to apply $sort and $limit inside the $project. So far, I have just returned the document and then paginated with the return value. However, I don't think this is very efficient and wondering if there is any way that I can paginate the field inside the $project.
Is there any way that I can apply $sort and $limit inside the $project, in order to apply pagination to the fields and return?
------ Edit ------
this is for paginating the field. Because, I am already paginating the document (game list), I could not find any way that I can paginate the field, because I could not find any way that I can apply $facet to the field.
e.g. document
[
gta: {
userID: ['aa', 'bb', 'cc' ......],
},
dota: {
userID: ['aa', 'bb', 'cc' ......],
}
....
]
I am using $facet to paginate the list of games (dota, gta, lol and more). However, I did not want to return all the userID. I had to return the entire document and then paginate the userID to replace the json doc.
Now, I can paginate the field inside the aggregate pipeline by using $function.
thanks to Mongodb sort inner array !
const _function = function (e) {
e // <---- will return userID array. You can do what you want to do.
return {
};
};
game
.collection("game")
.aggregate([
{},
{
$set: {
"game": {
$function: {
body: _function,
args: ["$userID"],
lang: "js",
},
},
},
},
])
.toArray();
By using $function multiple time, you will be able to paginate the field. I don' really know if this is faster or not tho. Plus, make sure you can use $function. I read that you can't use this if you are on the free tier at Atlas.
What you are looking for is the $slice Operator.
It requires three parameters.
"$slice": [<Array>, <start-N>, <No-Of.elements to fetch>]
userList: {"$slice": ["$dota2.users", 20, 10]} // <-- Will ignore first 20 elements in array and gets the next 10
Actually, in the database I got a job that I request with a GET route:
So when I populate candidates I got this response format :
My problem here is I don't need that "id" object, I just need a "selected_candidates" array with users inside as objects. Actually it's an object, in another object that is in an Array.
Here the code from my controller (the populate is in the jobsService):
If I change the data format of the job like that way:
...It is working great (with a path: "candidates_selected") like expected BUT I don't have that "status" string (Normal because I don't have it anymore in the DataBase. Because of ObjectId):
I would like a solution to have them both, but maybe it's the limit of noSQL?
A solution without populate but with a Loop (I don't think it's a good idea):
I think there is no convenience way to achieve it. However you may try the aggregate framework from the native MongoDB driver.
Let your Mongoose schemas be ASchema and BSchema
const result = await ASchema.aggregate([
{$addFields: {org_doc: '$$ROOT'}}, // save original document to retrieve later
{$unwind: '$candidates_selected'},
{
$lookup: {
from: BSchema.collection.name,
let: {
selected_id: '$candidates_selected.id',
status: '$candidates_selected.status',
},
pipeline: [
{
$match: {$expr: {$eq: ['$$selected_id', '$_id']}}, // find candidate by id
},
{
$addFields: {status: '$$status'} // attach status
}
],
as: 'found_candidate'
}
},
{
$group: { // regroup the very first $unwind stage
_id: '$_id',
org_doc: {$first: '$org_doc'},
found_candidates: {
$push: {$arrayElemAt: ['$found_candidate', 0]} // result of $lookup is an array, concat them to reform the original array
}
}
},
{
$addFields: {'org_doc.candidates_selected': '$found_candidates'} // attach found_candidates to the saved original document
},
{
$replaceRoot: {newRoot: '$org_doc'} // recover the original document
}
])
I'd like to do a search in MongoDB using either Mongo or Mongoose based on the value of a field in a document.
Let's say I had three MongoDB documents that looked like this:
{
name: "Michael",
mentored: ["Dwight", "Ryan", "Jim"]
},
{
name: "Jim",
mentored: ["Toby", "Roy", "Darryl"]
},
{
name: "Stanley",
mentored: ["Pam", "Meredith", "Angela"]
}
Let's further say I want to do a search for anyone who Michael has not mentored, which in this case would be Stanley (let's assume that the people in the arrays don't necessarily have their own records). I know I can do a search like this in Mongoose to get the result I want:
User.findOne({ name: "Michael" })
.then((person) => {
const mentored = person.mentored
return User.find({ name: { $nin: mentored } })
)
.then((person2) => {
console.log(person2); // Stanley
})
However, is there any way to do this without first returning the array from the database and then doing a second search? Something like this:
User.findOne({ name: { $nin: { "Michael's mentored people array" } } })
Ultimately I'm looking to see if there's any way to make more efficient such a situation in which arrays can get 10s of thousands of values long. Many thanks.
I think what you are doing is already efficient for large arrays.
But you could try the following, as suggested by #D.SM. but using aggregation the intermediate results will have to be loaded in memory, which does not seem efficient to me.
User.aggregate([
{
$match: {
name: "Michael"
}
},
{
$lookup: {
from: "collection",
as: "notMentored",
let: {
mentored: "$mentored",
/*
You probably want to remove "Michael" from the result, one way is to add him to the mentored array
mentored: { $concatArrays: ["$mentored", ["$name"]]}
*/
},
pipeline: [{
$match: {
$expr: { $not: { $in: ["$name", "$$mentored"] } }
}
}]
}
},
{
$unwind: "$notMentored"
},
{
$replaceRoot: {
newRoot: "$notMentored"
}
}
])
As the question title says, I'm trying to use an array field returned from a $match stage to query another collection in the next stage using $lookup and the $in operator to retrieve all documents that have at least one category inside this array. (I'm using Mongoose in Node, by the way)
I want to match a "configurations" collection by '_id' that have this simplified schema:
{
title: {type: String, required: true},
categories: {
allow: {type: Boolean, required: true},
list: [
{
name: {type: String, required: true},// DENORMALIZED CATEGORY NAME
_id: {type: mongoose.Schema.Types.ObjectId}
}
]
}
}
And in the next stage I want to aggregate all "partners" that belongs to at least one of those categories array. "partners" have the following schema:
{
company: {type: String, required: true},
categories: [
{type: mongoose.Schema.Types.ObjectId}
]
}
This is what I'm doing right now:
configuration.aggregate([
{$match: {_id: ObjectID(configurationId)}},
{
$lookup: {
from: "partners",
pipeline: [
{
$match: {
active: true,// MATCH ALL ACTIVE PARTNERS
categories: {
$in: {// HERE IS THE PROBLEM: I CAN'T RETRIEVE AN ARRAY FROM $map OPERATOR
$map: {// MAP CONFIGURATION CATEGORY LIST TO OUTPUT AN ARRAY ONLY WITH ID OBJECTS
input: '$categories.list',
as: 'category',
in: '$$category._id'
}
}
}
}
},
{ $project: { _id: 1, company: 1 } }
],
as: "partners"
}
},
])
The $map operator works as expected in a $project stage, but in this case I just can't use it's result as an array to be used with $in operator.
Is there any way to do this?
Thanks!
UPDATE
Doing like #Veeram suggested eliminates the need of $map operator in the $lookup stage:
{
"$lookup":{
"from":"partners",
"let":{"categories_id":"$categories.list._id"},
"pipeline":[
{"$match":{"active":true,"$expr":{"$in":["$categories","$$categories_id"]}}},
{"$project":{"_id":1,"company":1}}
],
"as":"partners"
}
}
But the problem persists with the $in operator. Like I've commented, this $in use case is the same as the 4th example in the official documentation (docs.mongodb.com/manual/reference/operator/aggregation/in), and it results in a false statement, because we are trying to check if an array ("$categories") is an element of another array ("$$categories_id"), which will fail, because the elements of "$$categories_id" are id objects and not arrays.
Does anyone know if there is any workaround for this?
Thanks!
You don't need to use $map. You can use dot notation to access the ids.
$let is required to access the values from local collection and $expr to compare the document fields.
Something like
{
"$lookup":{
"from":"partners",
"let":{"categories_id":"$categories.list._id"},
"pipeline":[
{"$match":{
"active":true,
"$expr":{
"$gt":[
{"$size":{"$setIntersection":["$categories","$$categories_id"]}},
0
]
}
}},
{"$project":{"_id":1,"company":1}}
],
"as":"partners"
}
}