this is my first time asking in StackOverflow and I hope I can explain what I'm aiming for.
I've got documents that look like this:
"_id" : ObjectId("5fd76b67a7e0fa652a297a9f"),
"type" : "play",
"session" : "5b0b5d57-c3ca-415f-8ef6-49bbd5805a23",
"episode" : 1,
"show" : 1,
"user" : 1,
"platform" : "spotify",
"currentTime" : 0,
"date" : ISODate("2020-12-14T13:40:51.906Z"),
"__v" : 0
}
I'd like to fetch for a show and group them by episode. I've got this far with my aggregattion:
const filter = { user, show, type: { $regex: /^(play|stop|close)$/ } }
const requiredFields = { "episode": 1, "session": 1, "date": 1, "currentTime": 1 }
// Get sessions grouped by episode
const it0 = {
_id: '$episode',
session:
{$addToSet:
{_id: "$session",
date:{$dateToString: { format: "%Y-%m-%d", date: "$date" }},
averageOfSession: {$cond: [ { $gte: [ "$currentTime", 0.1 ] }, "$currentTime", null ] }
},
},
count: { $sum: 1 }
}
// Filter unique sessions by session id and add them to a sessions field
const reduceSessions = {$addFields:
{sessions: {$reduce: {input: "$session",initialValue: [],in:
{$concatArrays: ["$$value",{$cond: [{$in: ["$$this._id","$$value._id"]},[],["$$this"]]}]}
}}}}
const projection = { $project: { _id: 0, episode: "$_id", plays: {$size: '$sessions'}, dropoff: {$avg: "$sessions.averageOfSession"}, sessions: '$session.date', events: "$count" } }
const arr = await Play.aggregate([
{ $match: filter }, {$project: requiredFields}, {$group: it0}, reduceSessions,
projection,{ $sort : { _id : 1 } }
])
and this is what my result looks like so far:
{
"episode": 5,
"plays": 4,
"dropoff": 3737.25,
"sessions": [
"2020-11-15",
"2020-11-15",
"2020-11-16",
"2020-11-15"
],
"events": 4
}...
What I'd like is for the 'sessions' array to be an object with one key for each distinct date which would contain the count, so something like this:
{
"episode": 5,
"plays": 4,
"dropoff": 3737.25,
"sessions": {
"2020-11-15": 3,
"2020-11-16": 1
},
"events": 4
}...
Hope that makes sense, thank you!!
You can first map sessions into key-value pairs. Then $group them to add up the sum. Then use $arrayToObject to convert to the format you want.
This Mongo playground is referencing this example.
Related
So I have a code for the aggregation
const documents = await deviceCollection
.aggregate([
{
$match: {
'readings.t': sensorType,
},
},
{ $unwind: '$readings' },
{
$project: {
_id: 0,
data: ['$readings.r', '$created_at'],
},
},
])
.toArray();
return documents.map(({ data }) => data);
and I have a document structure like this one
{
"readings" : [
{
"t" : "temperature",
"r" : 6
},
{
"t" : "humidity",
"r" : 66
}
],
"created_at" : ISODate("2021-02-24T09:45:09.858Z"),
"updated_at" : ISODate("2021-02-24T09:45:09.858Z")
}
I need to aggregate r value and created_at as UTC number for a particular reading type in a date range.
For example, the expected output for temperature reading is:
[
[6, 1616061903204],
[5.6, 1616061903204]
]
But the code returns this
[
[
6,
"2021-02-24T09:45:09.858Z"
],
[
66,
"2021-02-24T09:45:09.858Z"
],
[
5.6,
"2021-02-24T09:50:09.820Z"
],
[
68,
"2021-02-24T09:50:09.820Z"
],
]
And it means that I get the humidity type value as well.
$match your condition
$unwind deconstruct readings array
$match again to filter readings object
$toLong to convert ISO date format to timestamp
$group by null and construct readings array in a single array
const documents = await deviceCollection.aggregate([
{ $match: { "readings.t": sensorType } },
{ $unwind: "$readings" },
{ $match: { "readings.t": sensorType } },
{
$project: {
readings: [
"$readings.r",
{ $toLong: "$created_at" }
]
}
},
{
$group: {
_id: null,
readings: { $push: "$readings" }
}
}
]).toArray();
return documents.length ? documents[0].readings : [];
Playground
My Schema looks something like this.
{
_id: '1',
items: {
'id1': 'item1',
'id2': 'item2',
'id3': 'item3'
}
}
Following is the query
ItemModel.find({}, {
items: 1,
_id: 0
});
And the result of the find query is:
{ "items" : { "21" : "item21", "22" : "item22", "23" : "item23" } }
{ "items" : { "31" : "item31", "32" : "item32", "33" : "item33" } }
{ "items" : { "11" : "item11", "12" : "item32", "13" : "item13" } }
What I want is:
["item21", "item22", "item23",
"item31", "item32", "item33",
"item11", "item12", "item13"]
Currently, I am doing the processing on the node.js end for getting the above. I want to reduce the output payload size coming from MongoDB. The "items" key is redundant and the IDs mentioned are not required as well when I fetch it. Here, the IDs are quite small like 21, 22, 13, etc. but those are acutally 50 characters in length.
If not the above, any other efficient alternatives are also welcome.
One example of how to achieve that is the following aggregation:
[
{
$project: {
items: {
$objectToArray: '$items',
},
},
},
{ $unwind: '$items' },
{
$project: {
items: '$items.v',
},
},
{
$group: {
_id: null,
items: {
$push: '$items',
},
},
}
];
What this does is first we convert with $project & $objectToArray field to an array so that we could use $unwind. This way we'll have documents with different items. Now we convert with another $project to make it a string instead of an object (which would be { v: <value>, k: <value> }. And, finally, we $group them together.
Final result:
To get exactly that list, you'll need in your code to access items field, like result[0].items ([0] because aggregation will return an array).
In the application im building I have two updates that I want to do in the same query. I want to find the subdocument with the matching task_id and update its priority. In the same call I want to increment all the subdocuments with a priority higher than 3. Is it possible to combine these two in the same query?
const project = await Project.updateOne(
// first search
{ _id: req.params.project_id },
{ $set: {'tasks.$[element].priority': req.body.priority, 'tasks.$[element].state': req.body.state }},
{ arrayFilters: [{ 'element._id': req.params.task_id }] }
// second search
{ _id: req.params.project_id },
{ $inc: {'tasks.$[element].priority': 1 }},
{ arrayFilters: [{ 'element.priority': { $gt: 3 } }] }
);
you must be used different identifiers for your arrayFilters. your identifier is element. your code must be like that:
const project = await Project.updateOne(
// first search
{_id: req.params.project_id},
{
$set: {'tasks.$[elementA].priority': req.body.priority, 'tasks.$[elementA].state': req.body.state},
$inc: {'tasks.$[elementB].priority': 1}
},
{
arrayFilters: [
{'elementA._id': req.params.task_id},
{'elementB.priority': {$gt: 3}}
]
},
)
NOTE: The identifier must begin with a lowercase letter and contain only alphanumeric characters (from MongoDB official website, link)
You can do it simultaneously by using two arrayFilters. Consider the below:
Current collection:
{
"_id" : 1,
"array1" : [
{
"k1" : 1,
"v1" : 100
},
{
"k1" : 2,
"v1" : 15
},
{
"k1" : 1,
"v1" : 100
}
],
"array2" : [
{
"k2" : 1,
"v2" : 10
},
{
"k2" : 2,
"v2" : 1000
},
{
"k2" : 1,
"v2" : 20
}
]
}
Query:
db.collection.update(
{ _id: 1 },
{ $set:
{
'array1.$[elem1].v1': 100,
'array2.$[elem2].v2': 1000
}
},
{ arrayFilters:
[
{'elem1.k1':1},
{'elem2.k2': 2}
],
multi: true
}
)
As you can see that, I have created two filtered positional operator (elem1 and elem2), with the help of arrayFilters option. I can used this to perform my updates.
Result:
{
"_id" : 1,
"array1" : [
{
"k1" : 1,
"v1" : 100
},
{
"k1" : 2,
"v1" : 15
},
{
"k1" : 1,
"v1" : 100
}
],
"array2" : [
{
"k2" : 1,
"v2" : 10
},
{
"k2" : 2,
"v2" : 1000
},
{
"k2" : 1,
"v2" : 20
}
]
}
You can see in the above updated collection that the k1 field in array1 with value 1, it's v1 field have been updated to 100 and the k2 field in array2 with value 2, it's v2 field have been updated to 100.
So in your case you need to do something like below:
updateOne(
{ _id: req.params.project_id},
{
$set: {
'tasks.$[elem1].priority': req.body.priority,
'tasks.$[elem1].state': req.body.state
},
$inc: {
'tasks.$[elem2].priority': 1
}
},
{
arrayFilters: [
{ 'elem1._id': req.params.task_id },
{ 'elem2.priority':
{ $gt: 3 }
}
]
}
)
I hope it's helpful.
I would like to group records by day for certain period. I have tried so far using this code added into the aggregate function:
{
$group : {
_id : { day: { $dayOfMonth: "$timestamp" }},
count: { $sum: 1 }
}
}
And this is how a document looks like:
{
"_id" : ObjectId("ec9cddd50e08a84cd3f4cccb"),
"orgid" : "5c48500d84430a3a4b828e85",
"timestamp" : ISODate("2019-03-28T14:00:00.000Z"),
"apiid" : {
"zxczxczxczxczxc" : {
"errortotal" : 6,
"hits" : 6,
"humanidentifier" : "Feedback",
"identifier" : "663cfc345e42401c6443cfd635301f8f",
"lasttime" : ISODate("2019-03-28T14:58:07.355Z"),
"success" : 0,
"totalrequesttime" : 0.0,
"requesttime" : 0.0
}
},
"apikeys" : {
"00000000" : {
"errortotal" : 3,
"hits" : 3,
"humanidentifier" : "",
"identifier" : "00000000",
"lasttime" : ISODate("2019-03-28T14:55:10.438Z"),
"success" : 0,
"totalrequesttime" : 0.0,
"requesttime" : 0.0
},
"cae81afc" : {
"errortotal" : 3,
"hits" : 3,
"humanidentifier" : "EE5RqcXMTqcWEx8nZv3vRATLspK2",
"identifier" : "cbe81afc",
"lasttime" : ISODate("2019-03-28T14:58:07.355Z"),
"success" : 0,
"totalrequesttime" : 0.0,
"requesttime" : 0.0
}
}
Any idea how can I achieve this?
Result I get is: [ { _id: { day: null }, count: 3 } ], it seems wrong for me since I have two documents with the same date and another document with different timestamp
UPDATE:
I also have this inside aggregate fuction:
// Project things as a key/value array, along with the original doc
{
$project: {
array: {$objectToArray: '$apikeys'},
doc: '$$ROOT',
}
},
// Match the docs with a field value of 'x'
{$match: {'array.v.humanidentifier': {$in: trialCustomerUsers}}},
If I comment this part it will work fine the grouping, but the thing is I would also do some where statement in cases where I also dont know what woudl be the key, that's why I had to add this piece of code
Just accumulate the records in a new field with the $push operator
{
$group : {
_id : { day: { $dayOfMonth: "$timestamp" }},
records: { $push: "$$ROOT" }
}
}
You have $projected your all the root document in the doc field using $$ROOT. Now your aggregation should be as followed
db.collection.aggregate([
{ "$project": {
"array": { "$objectToArray": "$apikeys" },
"doc": "$$ROOT"
}},
{ "$match": { "array.v.humanidentifier": { "$in": trialCustomerUsers }}},
{ "$group" : {
"_id" : { "day": { "$dayOfMonth": "$doc.timestamp" }},
"count": { "$sum": 1 }
}}
])
Change this line
_id : { day: { $dayOfMonth: "$timestamp" }}
to
_id : { day: { $day: "$timestamp" } }
or you can do something like this
$group : {
_id : null,
day: '$timestamp',
count: { $sum: 1 }
}
I want to build online test application using mongoDB and nodeJS. And there is a feature which admin can view user test history (with date filter option).
How to do the query, if I want to display only user which the test results array contains date specified by admin.
The date filter will be based on day, month, year from scheduledAt.startTime, and I think I must use aggregate framework to achieve this.
Let's say I have users document like below:
{
"_id" : ObjectId("582a7b315c57b9164cac3295"),
"username" : "lalalala#gmail.com",
"displayName" : "lalala",
"testResults" : [
{
"applyAs" : [
"finance"
],
"scheduledAt" : {
"endTime" : ISODate("2016-11-15T16:00:00.000Z"),
"startTime" : ISODate("2016-11-15T01:00:00.000Z")
},
"results" : [
ObjectId("582a7b3e5c57b9164cac3299"),
ObjectId("582a7cc25c57b9164cac329d")
],
"_id" : ObjectId("582a7b3e5c57b9164cac3296")
},
{
.....
}
],
"password" : "andi",
}
testResults document:
{
"_id" : ObjectId("582a7cc25c57b9164cac329d"),
"testCategory" : "english",
"testVersion" : "EAX",
"testTakenTime" : ISODate("2016-11-15T03:10:58.623Z"),
"score" : 2,
"userAnswer" : [
{
"answer" : 1,
"problemId" : ObjectId("581ff74002bb1218f87f3fab")
},
{
"answer" : 0,
"problemId" : ObjectId("581ff78202bb1218f87f3fac")
},
{
"answer" : 0,
"problemId" : ObjectId("581ff7ca02bb1218f87f3fad")
}
],
"__v" : 0
}
What I have tried until now is like below. If I want to count total document, which part of my aggregation framework should I change. Because in query below, totalData is being summed per group not per whole returned document.
User
.aggregate([
{
$unwind: '$testResults'
},
{
$project: {
'_id': 1,
'displayName': 1,
'testResults': 1,
'dayOfTest': { $dayOfMonth: '$testResults.scheduledAt.startTime' },
'monthOfTest': { $month: '$testResults.scheduledAt.startTime' },
'yearOfTest': { $year: '$testResults.scheduledAt.startTime' }
}
},
{
$match: {
dayOfTest: date.getDate(),
monthOfTest: date.getMonth() + 1,
yearOfTest: date.getFullYear()
}
},
{
$group: {
_id: {id: '$_id', displayName: '$displayName'},
testResults: {
$push: '$testResults'
},
totalData: {
$sum: 1
}
}
},
])
.then(function(result) {
res.send(result);
})
.catch(function(err) {
console.error(err);
next(err);
});
You can try something like this. Added the project stage to keep the test results if any of result element matches on the date passed. Add this as the first step in the pipeline and you can add the grouping stage the way you want.
$map applies an equals comparison between the date passed and start date in each test result element and generates an array with true and false values. $anyElementTrue inspects this array and returns true only if there is atleast one true value in the array. Match stage to include only elements with matched value of true.
aggregate([{
"$project": {
"_id": 1,
"displayName":1,
"testResults": 1,
"matched": {
"$anyElementTrue": {
"$map": {
"input": "$testResults",
"as": "result",
"in": {
"$eq": [{ $dateToString: { format: "%Y-%m-%d", date: '$$result.scheduledAt.startTime' } }, '2016-11-15']
}
}
}
}
}
}, {
"$match": {
"matched": true
}
}])
Alternative Version:
Similar to the above version but this one combines both the project and match stage into one. The $cond with $redact accounts for match and when match is found it keeps the complete tree or else discards it.
aggregate([{
"$redact": {
"$cond": [{
"$anyElementTrue": {
"$map": {
"input": "$testResults",
"as": "result",
"in": {
"$eq": [{
$dateToString: {
format: "%Y-%m-%d",
date: '$$result.scheduledAt.startTime'
}
}, '2016-11-15']
}
}
}
},
"$$KEEP",
"$$PRUNE"
]
}
}])