here is the document
formId: 123,
title:"XYZ"
eventDate:"2022-04-15T05:40:57.182Z"
responses:[
{
orderId:98422,
name:"XYZ1",
email:"a#gmal.com",
paymentStatus:"pending",
amount:250,
phone:123456789
},
{
orderId:98422,
name:"XYZ1",
email:"a#gmal.com",
paymentStatus:"success",
amount:250,
phone:123456791
}
]
I used $elemMatch to filter the array such that I get only the matched object.
const response = await Form.findOne({ formId:123 }, {
_id:0,
title: 1,
eventDate: 1,
responses: {
$elemMatch: { orderId: 98422 },
},
})
But this returns all the fields inside the object present in the array "responses".
title:"XYZ"
eventDate:"2022-04-15T05:40:57.182Z"
responses:[
{
orderId:98422,
name:"XYZ1",
email:"a#gmal.com",
paymentStatus:"pending",
amount:250,
phone:123456789
}
]
But I want only specific fields to be returned inside the object like this
title:"XYZ"
eventDate:"2022-04-15T05:40:57.182Z"
responses:[
{
name:"XYZ1",
email:"a#gmal.com",
paymentStatus:"pending",
}
]
How can i do that ?
Query
aggregation way to keep some members and edit them also
map on responses if orderId matches keep the fields you want, the others are replaced with null
filter to remove those nulls (members that didnt match)
here 2 matches if you want to keep only one member of the array you can use
[($first ($filter ...)]
*$elemMatch that you used can be combined with the $ project operator to avoid the aggregation, but with $ operator we get all the matching member (here you want only some fields so i think aggregation is the way)
Playmongo
aggregate(
[{"$match": {"formId": {"$eq": 123}}},
{"$project":
{"_id": 0,
"title": 1,
"eventDate": 1,
"responses":
{"$map":
{"input": "$responses",
"in":
{"$cond":
[{"$eq": ["$$this.orderId", 98422]},
{"name": "$$this.name",
"email": "$$this.email",
"paymentStatus": "$$this.paymentStatus"},
null]}}}}},
{"$set":
{"responses":
{"$filter":
{"input": "$responses", "cond": {"$ne": ["$$this", null]}}}}}])
Trying to create a MongoDB data source with icCube. The idea is to return the size of an array as a new field. Something like :
$project:
{
"people": 1,
"Count myFieldArray" : {$size : "$myFieldArray" }
}
But I'm getting for some records the following error :
The argument to $size must be an Array, but was of type: EOO
Is there a way that size is 0 if the field is empty or not an array (getting rid of the error) ?
You can use the $ifNull operator here. It seems the field is either not an array or not present by the given error:
{ "$project": {
"people": 1,
"Count": {
"$size": { "$ifNull": [ "$myFieldArray", [] ] }
}
}}
Also you might want to check for the $type in your $match in case these do exist but are not an array.
From MongoDB 3.2 and newer, you can use $isArray to check if your field is an array along with the $cond operator to return the field on evaluating with $isArray:
{ "$project": {
"people": 1,
"myFieldArrayCount": {
"$size": {
"$cond": [
{ "$isArray": "$myFieldArray" },
"$myFieldArray",
[]
]
}
}
}}
Alternative solution would be to eliminate the documents with nulls using
$match: {myFieldArray: { $elemMatch: { $exists: true } }}
Also, document fields which are used as arguments to $size by '$' reference (here: "$myFieldArray") must also be the part of projections.
$project:
{
"people": 1,
"myFieldArray":1,
"Count myFieldArray" : {$size : "$myFieldArray" }
}
Given that I have a document of the following structure:
{
selectedId: ObjectId("57b5fb2d7b41dde99009bc75"),
children: [
{_id: ObjectId("57b5fb2d7b41dde99009bc75"), val: 10},
{_id: ObjectId("57b5fb2d7b41dde99009bc75"), val: 20},
]
}
where the parent value "selectedId" always references one of the children Id, how do I just get the child subdocument where _id = selectedId?
I attempted:
parentModel.findOne({'selectedId': 'this.children._id'})
however, as I now know, the second string is taken as a literal. So how do I reference the parent's field in the query?
Edit: obviously, this could be done with two queries, getting the parent's "selectedId" value and then querying again. However, I want to do this in a single query.
You could use the aggregation framework, in particular leverage the $arrayElemAt and $filter operators to return the child subdocument. The following example shows this:
parentModel.aggregate([
{
"$project": {
"children": {
"$arrayElemAt": [
{
"$filter": {
"input": "$children",
"as": "item",
"cond": {
"$eq": ["$$item._id", "$selectedId"]
}
}
}, 0
]
}
}
}
]).exec(callback);
I'm fairly good with sql queries, but I can't seem to get my head around grouping and getting sum of mongo db documents,
With this in mind, I have a job model with schema like below :
{
name: {
type: String,
required: true
},
info: String,
active: {
type: Boolean,
default: true
},
all_service: [
price: {
type: Number,
min: 0,
required: true
},
all_sub_item: [{
name: String,
price:{ // << -- this is the price I want to calculate
type: Number,
min: 0
},
owner: {
user_id: { // <<-- here is the filter I want to put
type: Schema.Types.ObjectId,
required: true
},
name: String,
...
}
}]
],
date_create: {
type: Date,
default : Date.now
},
date_update: {
type: Date,
default : Date.now
}
}
I would like to have a sum of price column, where owner is present, I tried below but no luck
Job.aggregate(
[
{
$group: {
_id: {}, // not sure what to put here
amount: { $sum: '$all_service.all_sub_item.price' }
},
$match: {'not sure how to limit the user': given_user_id}
}
],
//{ $project: { _id: 1, expense: 1 }}, // you can only project fields from 'group'
function(err, summary) {
console.log(err);
console.log(summary);
}
);
Could someone guide me in the right direction. thank you in advance
Primer
As is correctly noted earlier, it does help to think of an aggregation "pipeline" just as the "pipe" | operator from Unix and other system shells. One "stage" feeds input to the "next" stage and so on.
The thing you need to be careful with here is that you have "nested" arrays, one array within another, and this can make drastic differences to your expected results if you are not careful.
Your documents consist of an "all_service" array at the top level. Presumably there are often "multiple" entries here, all containing your "price" property as well as "all_sub_item". Then of course "all_sub_item" is an array in itself, also containg many items of it's own.
You can think of these arrays as the "relations" between your tables in SQL, in each case a "one-to-many". But the data is in a "pre-joined" form, where you can fetch all data at once without performing joins. That much you should already be familiar with.
However, when you want to "aggregate" accross documents, you need to "de-normalize" this in much the same way as in SQL by "defining" the "joins". This is to "transform" the data into a de-normalized state that is suitable for aggregation.
So the same visualization applies. A master document's entries are replicated by the number of child documents, and a "join" to an "inner-child" will replicate both the master and initial "child" accordingly. In a "nutshell", this:
{
"a": 1,
"b": [
{
"c": 1,
"d": [
{ "e": 1 }, { "e": 2 }
]
},
{
"c": 2,
"d": [
{ "e": 1 }, { "e": 2 }
]
}
]
}
Becomes this:
{ "a" : 1, "b" : { "c" : 1, "d" : { "e" : 1 } } }
{ "a" : 1, "b" : { "c" : 1, "d" : { "e" : 2 } } }
{ "a" : 1, "b" : { "c" : 2, "d" : { "e" : 1 } } }
{ "a" : 1, "b" : { "c" : 2, "d" : { "e" : 2 } } }
And the operation to do this is $unwind, and since there are multiple arrays then you need to $unwind both of them before continuing any processing:
db.collection.aggregate([
{ "$unwind": "$b" },
{ "$unwind": "$b.d" }
])
So there the "pipe" first array from "$b" like so:
{ "a" : 1, "b" : { "c" : 1, "d" : [ { "e" : 1 }, { "e" : 2 } ] } }
{ "a" : 1, "b" : { "c" : 2, "d" : [ { "e" : 1 }, { "e" : 2 } ] } }
Which leaves a second array referenced by "$b.d" to further be de-normalized into the the final de-normalized result "without any arrays". This allows other operations to process.
Solving
With just about "every" aggregation pipeline, the "first" thing you want to do is "filter" the documents to only those that contain your results. This is a good idea, as especially when doing operations such as $unwind, then you don't want to be doing that on documents that do not even match your target data.
So you need to match your "user_id" at the array depth. But this is only part of getting the result, since you should be aware of what happens when you query a document for a matching value in an array.
Of course, the "whole" document is still returned, because this is what you really asked for. The data is already "joined" and we haven't asked to "un-join" it in any way.You look at this just as a "first" document selection does, but then when "de-normalized", every array element now actualy represents a "document" in itself.
So not "only" do you $match at the beginning of the "pipeline", you also $match after you have processed "all" $unwind statements, down to the level of the element you wish to match.
Job.aggregate(
[
// Match to filter possible "documents"
{ "$match": {
"all_service.all_sub_item.owner": given_user_id
}},
// De-normalize arrays
{ "$unwind": "$all_service" },
{ "$unwind": "$all_service.all_subitem" },
// Match again to filter the array elements
{ "$match": {
"all_service.all_sub_item.owner": given_user_id
}},
// Group on the "_id" for the "key" you want, or "null" for all
{ "$group": {
"_id": null,
"total": { "$sum": "$all_service.all_sub_item.price" }
}}
],
function(err,results) {
}
)
Alternately, modern MongoDB releases since 2.6 also support the $redact operator. This could be used in this case to "pre-filter" the array content before processing with $unwind:
Job.aggregate(
[
// Match to filter possible "documents"
{ "$match": {
"all_service.all_sub_item.owner": given_user_id
}},
// Filter arrays for matches in document
{ "$redact": {
"$cond": {
"if": {
"$eq": [
{ "$ifNull": [ "$owner", given_user_id ] },
given_user_id
]
},
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}},
// De-normalize arrays
{ "$unwind": "$all_service" },
{ "$unwind": "$all_service.all_subitem" },
// Group on the "_id" for the "key" you want, or "null" for all
{ "$group": {
"_id": null,
"total": { "$sum": "$all_service.all_sub_item.price" }
}}
],
function(err,results) {
}
)
That can "recursively" traverse the document and test for the condition, effectively removing any "un-matched" array elements before you even $unwind. This can speed things up a bit since items that do not match would not need to be "un-wound". However there is a "catch" in that if for some reason the "owner" did not exist on an array element at all, then the logic required here would count that as another "match". You can always $match again to be sure, but there is still a more efficient way to do this:
Job.aggregate(
[
// Match to filter possible "documents"
{ "$match": {
"all_service.all_sub_item.owner": given_user_id
}},
// Filter arrays for matches in document
{ "$project": {
"all_items": {
"$setDifference": [
{ "$map": {
"input": "$all_service",
"as": "A",
"in": {
"$setDifference": [
{ "$map": {
"input": "$$A.all_sub_item",
"as": "B",
"in": {
"$cond": {
"if": { "$eq": [ "$$B.owner", given_user_id ] },
"then": "$$B",
"else": false
}
}
}},
false
]
}
}},
[[]]
]
}
}},
// De-normalize the "two" level array. "Double" $unwind
{ "$unwind": "$all_items" },
{ "$unwind": "$all_items" },
// Group on the "_id" for the "key" you want, or "null" for all
{ "$group": {
"_id": null,
"total": { "$sum": "$all_items.price" }
}}
],
function(err,results) {
}
)
That process cuts down the size of the items in both arrays "drastically" compared to $redact. The $map operator processes each elment of an array to the given statement within "in". In this case, each "outer" array elment is sent to another $map to process the "inner" elements.
A logical test is performed here with $cond whereby if the "condiition" is met then the "inner" array elment is returned, otherwise the false value is returned.
The $setDifference is used to filter down any false values that are returned. Or as in the "outer" case, any "blank" arrays resulting from all false values being filtered from the "inner" where there is no match there. This leaves just the matching items, encased in a "double" array, e.g:
[[{ "_id": 1, "price": 1, "owner": "b" },{..}],[{..},{..}]]
As "all" array elements have an _id by default with mongoose (and this is a good reason why you keep that) then every item is "distinct" and not affected by the "set" operator, apart from removing the un-matched values.
Process $unwind "twice" to convert these into plain objects in their own documents, suitable for aggregation.
So those are the things you need to know. As I stated earlier, be "aware" of how the data "de-normalizes" and what that implies towards your end totals.
It sounds like you want to, in SQL equivalent, do "sum (prices) WHERE owner IS NOT NULL".
On that assumption, you'll want to do your $match first, to reduce the input set to your sum. So your first stage should be something like
$match: { all_service.all_sub_items.owner : { $exists: true } }
Think of this as then passing all matching documents to your second stage.
Now, because you are summing an array, you have to do another step. Aggregation operators work on documents - there isn't really a way to sum an array. So we want to expand your array so that each element in the array gets pulled out to represent the array field as a value, in its own document. Think of this as a cross join. This will be $unwind.
$unwind: { "$all_service.all_sub_items" }
Now you've just made a much larger number of documents, but in a form where we can sum them. Now we can perform the $group. In your $group, you specify a transformation. The line:
_id: {}, // not sure what to put here
is creating a field in the output document, which is not the same documents as the input documents. So you can make the _id here anything you'd like, but think of this as the equivalent to your "GROUP BY" in sql. The $sum operator will essentially be creating a sum for each group of documents you create here that match that _id - so essentially we'll be "re-collapsing" what you just did with $unwind, by using the $group. But this will allow $sum to work.
I think you're looking for grouping on just your main document id, so I think your $sum statement in your question is correct.
$group : { _id : $_id, totalAmount : { $sum : '$all_service.all_sub_item.price' } }
This will output documents with an _id field equivalent to your original document ID, and your sum.
I'll let you put it together, I'm not super familiar with node. You were close but I think moving your $match to the front and using an $unwind stage will get you where you need to be. Good luck!
i have a Mongodb collection named "EVENTS" and in the collection i have an object of array which looks like this:
{
"Events":[
{
"_id":"53ae59883d0e63aa77f7b5b2",
"Title":"Title Blank",
"Desc":"Description Blank",
"Date":"2014-06-04 00:30",
"Link":"http://googleparty.com",
"Event":"Victoria Centre",
"dateCreated":"28/6/2014 06:58"
},
{
"_id":"53ae59883d0e63aa77f7b5b3",
"Title":"Hello World",
"Desc":"hello",
"Date":"2014-06-04 00:30",
"Link":"http://linkedinparty.com",
"Event":"social",
"dateCreated":"30/2/2014 11:10"
}
]
}
how would i delete an object by id in node.js so " delete(53ae59883d0e63aa77f7b5b2)" will yield this:
{
"Events":[
{
"_id":"53ae59883d0e63aa77f7b5b3",
"Title":"Hello World",
"Desc":"hello",
"Date":"2014-06-04 00:30",
"Link":"http://linkedinparty.com",
"Event":"social",
"dateCreated":"30/2/2014 11:10"
}
]
}
Regards
If all you really want to do is "empty" the array then you just need to use the $set operator with an .update() and "set" the array as an empty one:
db.collection.update({},{ "$set": { "Events": [] } },{ "mutli": true})
So the .update() operation takes a "query" to select the documents in your collection, a blank query as shown selects everything. The "update" section contains the $set operation that just replaces the current "Events" field with an empty array.
The "multi" option there makes sure this is applied to every document that matches. The default is false and will only update the first document that matches.
For more specific operations removing selected array elements, look at the $pull operator. Your edit shows now that this is what you want to do:
db.collection.update(
{ "Events._id": ObjectId("53ae59883d0e63aa77f7b5b2") },
{ "$pull": { "Events": { "_id": ObjectId("53ae59883d0e63aa77f7b5b2") } } }
)
But your inclusion of arrays with _id fields seems to indicate that you are using mongoose, so the ObjectId values are cast automatically:
Model.update(
{ "Events._id": "53ae59883d0e63aa77f7b5b2" },
{ "$pull": { "Events": { "_id": "53ae59883d0e63aa77f7b5b2" } } },
function(err,numAffected) {
}
);