MongoDB find projection, matching elements with $ne - node.js

I have a query that returns objects containing an array of objects. Within that array of objects, there are some that should not be processed. Here is an object simmilar to what I have:
{
_id: 12345,
data: [
{
state: 1,
info: "abc"
},{
state: 2,
info: "cde"
},{
state: 2,
info: "efg"
}
]
}
I want to show only the objects where state does not equal to 1. So I want to get back something like this:
{
_id: 12345,
data: [
{
state: 2,
info: "cde"
},{
state: 2,
info: "efg"
}
]
}
There can be hundreds of "main" objects with tens of "sub" objects. I tried using the query:
col.find({'data.info': {$in: [] }, {_id: 1, data: { $elemMatch: { state: {$ne: 1 } } } }, {}, callback);
But that only gives me this:
{
_id: 12345,
data: [
{
state: 2,
info: "cde"
}
]
}
In other words, $elemMatch does what it is supposed to do, but I need to get a different result. So is there a way to do that in one query or without pre-processing results (removing entries before any further code reads the data)?

The $elemMatch projection operator only returns the first matching element in an array.
To filter the whole array, the best approach in MongoDB 2.2+ would be using the Aggregation Framework. Alternative, you could also do this using Map/Reduce or in your application code.
Example aggregation:
db.data.aggregate(
// Initial match to limit number of documents
{ $match : {
data: { $elemMatch: { state: {$ne: 1 } } }
}},
// Convert the data array into a stream of documents
{ $unwind: "$data" },
// Limit to matching elements of the data array
{ $match : {
"data.state": {$ne: 1 }
}},
// Re-group by original document _id
{ $group: {
_id: "$_id",
data: { $push: "$data" }
}}
)
Sample output:
{
"_id" : 12345,
"data" : [
{
"state" : 2,
"info" : "cde"
},
{
"state" : 2,
"info" : "efg"
}
]
}

Related

MongoDB - Mongoose find command to get only values from an embedded object

My Schema looks something like this.
{
_id: '1',
items: {
'id1': 'item1',
'id2': 'item2',
'id3': 'item3'
}
}
Following is the query
ItemModel.find({}, {
items: 1,
_id: 0
});
And the result of the find query is:
{ "items" : { "21" : "item21", "22" : "item22", "23" : "item23" } }
{ "items" : { "31" : "item31", "32" : "item32", "33" : "item33" } }
{ "items" : { "11" : "item11", "12" : "item32", "13" : "item13" } }
What I want is:
["item21", "item22", "item23",
"item31", "item32", "item33",
"item11", "item12", "item13"]
Currently, I am doing the processing on the node.js end for getting the above. I want to reduce the output payload size coming from MongoDB. The "items" key is redundant and the IDs mentioned are not required as well when I fetch it. Here, the IDs are quite small like 21, 22, 13, etc. but those are acutally 50 characters in length.
If not the above, any other efficient alternatives are also welcome.
One example of how to achieve that is the following aggregation:
[
{
$project: {
items: {
$objectToArray: '$items',
},
},
},
{ $unwind: '$items' },
{
$project: {
items: '$items.v',
},
},
{
$group: {
_id: null,
items: {
$push: '$items',
},
},
}
];
What this does is first we convert with $project & $objectToArray field to an array so that we could use $unwind. This way we'll have documents with different items. Now we convert with another $project to make it a string instead of an object (which would be { v: <value>, k: <value> }. And, finally, we $group them together.
Final result:
To get exactly that list, you'll need in your code to access items field, like result[0].items ([0] because aggregation will return an array).

mongoose filter array in query , i can find this in mongodb but not in mongoose [duplicate]

This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Closed 5 years ago.
The community reviewed whether to reopen this question 5 months ago and left it closed:
Original close reason(s) were not resolved
I have array in subdocument like this
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter subdocument for a > 3
My expect result below
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 4
},
{
"a" : 5
}
]
}
I try to use $elemMatch but returns the first matching element in the array
My query:
db.test.find( { _id" : ObjectId("512e28984815cbfcb21646a7") }, {
list: {
$elemMatch:
{ a: { $gt:3 }
}
}
} )
The result return one element in array
{ "_id" : ObjectId("512e28984815cbfcb21646a7"), "list" : [ { "a" : 4 } ] }
and I try to use aggregate with $match but not work
db.test.aggregate({$match:{_id:ObjectId("512e28984815cbfcb21646a7"), 'list.a':{$gte:5} }})
It's return all element in array
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter element in array to get result as expect result?
Using aggregate is the right approach, but you need to $unwind the list array before applying the $match so that you can filter individual elements and then use $group to put it back together:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $unwind: '$list'},
{ $match: {'list.a': {$gt: 3}}},
{ $group: {_id: '$_id', list: {$push: '$list.a'}}}
])
outputs:
{
"result": [
{
"_id": ObjectId("512e28984815cbfcb21646a7"),
"list": [
4,
5
]
}
],
"ok": 1
}
MongoDB 3.2 Update
Starting with the 3.2 release, you can use the new $filter aggregation operator to do this more efficiently by only including the list elements you want during a $project:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $project: {
list: {$filter: {
input: '$list',
as: 'item',
cond: {$gt: ['$$item.a', 3]}
}}
}}
])
$and:
get data between 0-5:
cond: {
$and: [
{ $gt: [ "$$item.a", 0 ] },
{ $lt: [ "$$item.a", 5) ] }
]}
Above solution works best if multiple matching sub documents are required.
$elemMatch also comes in very use if single matching sub document is required as output
db.test.find({list: {$elemMatch: {a: 1}}}, {'list.$': 1})
Result:
{
"_id": ObjectId("..."),
"list": [{a: 1}]
}
Use $filter aggregation
Selects a subset of the array to return based on the specified
condition. Returns an array with only those elements that match the
condition. The returned elements are in the original order.
db.test.aggregate([
{$match: {"list.a": {$gt:3}}}, // <-- match only the document which have a matching element
{$project: {
list: {$filter: {
input: "$list",
as: "list",
cond: {$gt: ["$$list.a", 3]} //<-- filter sub-array based on condition
}}
}}
]);

mongodb filter arrays inside each property [duplicate]

This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Closed 5 years ago.
The community reviewed whether to reopen this question 5 months ago and left it closed:
Original close reason(s) were not resolved
I have array in subdocument like this
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter subdocument for a > 3
My expect result below
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 4
},
{
"a" : 5
}
]
}
I try to use $elemMatch but returns the first matching element in the array
My query:
db.test.find( { _id" : ObjectId("512e28984815cbfcb21646a7") }, {
list: {
$elemMatch:
{ a: { $gt:3 }
}
}
} )
The result return one element in array
{ "_id" : ObjectId("512e28984815cbfcb21646a7"), "list" : [ { "a" : 4 } ] }
and I try to use aggregate with $match but not work
db.test.aggregate({$match:{_id:ObjectId("512e28984815cbfcb21646a7"), 'list.a':{$gte:5} }})
It's return all element in array
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter element in array to get result as expect result?
Using aggregate is the right approach, but you need to $unwind the list array before applying the $match so that you can filter individual elements and then use $group to put it back together:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $unwind: '$list'},
{ $match: {'list.a': {$gt: 3}}},
{ $group: {_id: '$_id', list: {$push: '$list.a'}}}
])
outputs:
{
"result": [
{
"_id": ObjectId("512e28984815cbfcb21646a7"),
"list": [
4,
5
]
}
],
"ok": 1
}
MongoDB 3.2 Update
Starting with the 3.2 release, you can use the new $filter aggregation operator to do this more efficiently by only including the list elements you want during a $project:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $project: {
list: {$filter: {
input: '$list',
as: 'item',
cond: {$gt: ['$$item.a', 3]}
}}
}}
])
$and:
get data between 0-5:
cond: {
$and: [
{ $gt: [ "$$item.a", 0 ] },
{ $lt: [ "$$item.a", 5) ] }
]}
Above solution works best if multiple matching sub documents are required.
$elemMatch also comes in very use if single matching sub document is required as output
db.test.find({list: {$elemMatch: {a: 1}}}, {'list.$': 1})
Result:
{
"_id": ObjectId("..."),
"list": [{a: 1}]
}
Use $filter aggregation
Selects a subset of the array to return based on the specified
condition. Returns an array with only those elements that match the
condition. The returned elements are in the original order.
db.test.aggregate([
{$match: {"list.a": {$gt:3}}}, // <-- match only the document which have a matching element
{$project: {
list: {$filter: {
input: "$list",
as: "list",
cond: {$gt: ["$$list.a", 3]} //<-- filter sub-array based on condition
}}
}}
]);

Group items by timeframe

I have a collection db.activities, each item of which has a dueDate. I need to present data in a following format, which basically a list of activities which are due today and this week:
{
"today": [
{ _id: 1, name: "activity #1" ... },
{ _id: 2, name: "activity #2" ... }
],
"thisWeek": [
{ _id: 3, name: "activity #3" ... }
]
}
I managed to accomplish this by simply querying for the last week's activities as a flat list and then grouping them with javascript on the client, but I suspect this is a very dirty solution and would like to do this on server.
look up mongo aggregation pipeline.
your aggregation has a match by date, group by date and a maybe a sort/order stage also by date.
lacking the data scheme it will be along the lines of
db.collection.aggregate([{ $match: {"duedate": { "$gte" : start_dt, "$lte" : end_dt} } ,
{ $group: {_id: "$duedate", recordid : "$_id" , name: "$name" },
{"$sort" : {"_id" : 1} } ] );
if you want 'all' records remove the $match or use { $match: {} } as one does with find.
in my opinion, you cannot aggregate both by day and week within one command. the weekly one may be achieved by projecting duedate using mongos $dayOfWeek. along the lines of
db.collection.aggregate([
{ $match: {"duedate": { "$gte" : start_dt, "$lte" : end_dt} } ,
{ $project : { dayOfWeek: { $dayOfWeek: "$duedate" } },
{ $group: {_id: "$dayOfWeek", recordid : "$_id" , name: "$name" },
{"$sort" : {"_id" : 1} } ] );
check out http://docs.mongodb.org/manual/reference/operator/aggregation/dayOfWeek/

How to get documents with non unique array elements?

I have the following MongoDB documents:
{
_id: ObjectId('09de14821345dda65c471c99'),
items: [
_id: ObjectId('34de64871345dfa655471c99'),
_id: ObjectId('34de64871345dfa655471c91'),
_id: ObjectId('34de64871345dfa655471c99'),
]
},
{
_id: ObjectId('09de14821345dda65c471c98'),
items: [
_id: ObjectId('24de64871345dfa61271c10'),
_id: ObjectId('24de64871345dfa61271c11'),
_id: ObjectId('24de64871345dfa61271c11'),
]
},
{
_id: ObjectId('09de14821345dda65c471c07'),
items: [
_id: ObjectId('24de64871345dfa61271c05'),
_id: ObjectId('24de64871345dfa61271c06'),
_id: ObjectId('24de64871345dfa61271c07'),
]
}
I need to find all documents with repeated items array elements. So from the documents above I want to get the following result:
db.collection.documents.find({/** need query*/}).toArray(function (err, documents) {
console.dir(documents); // documents with id's 09de14821345dda65c471c99 and 09de14821345dda65c471c98
});
How could I do that?
In order to group and match results you will need to use the Aggregation Framework or Map/Reduce rather than a simple find() query.
Example data
Your example document include some errors: a few of the ObjectIDs are too short and the array elements should either be embedded documents ({_id: ObjectId(...)}) or simple values.
For test data I've used:
db.mydocs.insert([
{
_id: ObjectId('09de14821345dda65c471c99'),
items: [
ObjectId('34de64871345dfa655471c99'),
ObjectId('34de64871345dfa655471c91'),
ObjectId('34de64871345dfa655471c99')
]
},
{
_id: ObjectId('09de14821345dda65c471c98'),
items: [
ObjectId('24de64871345ddfa61271c10'),
ObjectId('24de64871345ddfa61271c11'),
ObjectId('24de64871345ddfa61271c11')
]
},
{
_id: ObjectId('09de14821345dda65c471c07'),
items: [
ObjectId('24de64871345ddfa61271c05'),
ObjectId('24de64871345ddfa61271c06'),
ObjectId('24de64871345ddfa61271c07')
]
}])
Aggregation query
Here is an aggregation query using the mongo shell:
db.mydocs.aggregate(
// Unpack items array into stream of documents
{ $unwind: "$items" },
// Group by original document _id and item
{ $group: {
_id: { _id: "$_id", item: "$items" },
count: { $sum: 1 }
}},
// Limit to duplicated array items (1 or more count per document _id)
{ $match: {
count: { $gt: 1 }
}},
// (Optional) clean up the result formatting
{ $project: {
_id: "$_id._id",
item: "$_id.item",
count: "$count"
}}
)
Sample results
{
"_id" : ObjectId("09de14821345dda65c471c98"),
"count" : 2,
"item" : ObjectId("24de64871345ddfa61271c11")
}
{
"_id" : ObjectId("09de14821345dda65c471c99"),
"count" : 2,
"item" : ObjectId("34de64871345dfa655471c99")
}

Resources