I Have a bunch of sensordata stored in mongoDB. They are stored like this:
{
"data": [
{
"date": ISODate("2020-02-08T18:06:25.507+00:00"),
"temperature": 20.3,
"humidity": 53.7
},
{
"date": ISODate("2020-02-08T18:07:25.507+00:00"),
"temperature": 21,
"humidity": 54
}
]
}
The day-field is generated by new Date() with JavaScript.
Now i just want the get all the data and convert the "date"-field to a time-field. The result should look like this:
{
"data": [
{
"date": "18:06:25",
"temperature": 20.3,
"humidity": 53.7
},
{
"date": "18:07:25",
"temperature": 21,
"humidity": 54
}
]
}
So is there a way to convert every "date"-field in the array to a "time"-field by using db.collection.aggregate?
I tried using this:
db.collection.aggregate([
{},
{
"$project": {
"data.date": { $dateToString: { format: "%H:%M:%S",date: "$date" } },
"daydata.temperature": 1,
"daydata.humidity": 1
}
}
])
I know it doesnt work, because i dont have any "date"-field outside of "data". But i dont know how to reach the date-field of every data-object and convert it.
You can use $unwind then $project then $group in aggregate.
db.data.aggregate([
{ $unwind: "$data" },
{
$project: {
"data.date": {
$dateToString: { format: "%H:%M:%S", date: "$data.date" },
},
"data.temperature": 1,
"data.humidity": 1,
},
},
{
$group: {
_id: "$_id",
data: { $push: "$data" },
},
},
]);
Also, date should be ISODate
{
"data": [
{
"date": ISODate("2020-02-08T18:06:25.507+00:00"),
"temperature": 20.3,
"humidity": 53.7
},
{
"date": ISODate("2020-02-08T18:07:25.507+00:00"),
"temperature": 21,
"humidity": 54
}
]
}
Since the data.date field is a string, use the sub-string operator to extract the time part of the date field.
db.test.aggregate( [
{
$unwind: "$data"
},
{
$addFields: {
"data.time": { $substrCP: [ "$data.date", 11, 8 ] }
}
},
{
$project: { "data.date": 0 }
},
{
$group: {
_id: "$_id",
data: { $push: "$data" },
// other_fld: { $first: "$other_fld" }
}
}
] ).pretty()
Related
I have the following MongoDB schema:
const userSchema = new mongoose.Schema({
email: {
type: String,
required: [true, 'Email is required.']
},
transactions: [
{
categoryName: {
type: String,
required: [true, 'Category name in transaction is required.']
},
categoryType: {
type: String,
required: [true, 'Category type in transaction is required.']
},
amount: {
type: Number,
required: [true, 'Transaction amount is required.']
}
}
]})
transactions.categoryType can only be Income or Expense. Now per queried _id, I want to return the ratio/percentage of transactions.CategoryName per Income and Expense. Meaning if I have the following data:
{
"_id": 000001,
"email": "asdasd#email.com"
"transactions": [
{
"categoryName": "Food",
"categoryType": "Expense",
"amount": 200
},
{
"categoryName": "Rent",
"categoryType": "Expense",
"amount": 1000
},
{
"categoryName": "Salary",
"categoryType": "Income",
"amount": 15000
}
]
}
the result that I would want is:
{ "email": "asdasd#email.com",
"Income": [["Salary", 100]],
"Expense": [["Food", 16.67],["Rent",83.33]],
}
Now, I have the following query:
return User.aggregate([
{ $match: { _id : ObjectId(request.params.id) } },
{ $unwind : "$transactions"},
{ $group : { _id : { type: "$transactions.categoryType" },
total: {$sum : "$transactions.amount"},
transactionsArray: { $push: "$transactions"}
}
},
{ $project: {
_id: 0,
transactionsArray:1,
type: "$_id.type",
total:1
}
}
])
which returns a data like this:
[
{
"total": 1200,
"transactions": [
{
"categoryName": "Food",
"categoryType": "Expense",
"amount": 200,
},
{
"categoryName": "Rent",
"categoryType": "Expense",
"amount": 1000,
}
],
"type": "Expense"
},
{
"total": 15000,
"transactions": [
{
"categoryName": "Salary",
"categoryType": "Income",
"amount": 15000,
}
],
"type": "Income"
}
]
Now, I do not know how am I going to further process the result set to divide the transactions.amount by the total to get the result that I want.
You may go with multiple steps in aggregations
$unwind to deconstruct the array
$group- first group to group by _id and $categoryType. So we can get the total amount and an amount for particular transaction. This helps to calculate the ratio.
$map helps to loop over the array and calculate the ratio
$reduce- You need comma separated string array of objects. So loop it and get the structure.
$group to group by _id only so we can get the key value pair of category type and Income/Expense when we push
$replaceRoot to make the $grp object as root which should be merged with already existing fields ($mergeObjects)
$project for remove unwanted fields
Here is the code
db.collection.aggregate([
{ "$unwind": "$transactions" },
{
"$group": {
"_id": { id: "$_id", catType: "$transactions.categoryType" },
"email": { "$first": "$email" },
"amount": { "$sum": "$transactions.amount" },
"category": {
$push: { k: "$transactions.categoryName", v: "$transactions.amount" }
}
}
},
{
$addFields: {
category: {
$map: {
input: "$category",
in: {
k: "$$this.k",
v: {
"$multiply": [
{ "$divide": [ "$$this.v","$amount" ]},
100
]
}
}
}
}
}
},
{
"$addFields": {
category: {
"$reduce": {
"input": "$category",
"initialValue": [],
"in": {
"$concatArrays": [
[
[ "$$this.k", { $toString: "$$this.v" } ]
],
"$$value"
]
}
}
}
}
},
{
"$group": {
"_id": "$_id.id",
"email": { "$first": "$email" },
"grp": { "$push": { k: "$_id.catType", v: "$category" } }
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [ { "$arrayToObject": "$grp" }, "$$ROOT" ]
}
}
},
{ "$project": { grp: 0 } }
])
Working Mongo playground
I saved my data in MongoDb as
{
"_id": "ObjectId(\"5fec31b6b9022035abbbf7cc\")",
"message": {
"date": "2020-12-30 13:20:26",
"time": "2020-12-30T07:50:26.000Z",
"ID": "005",
"P": 1.36
}
},
{
"_id": "ObjectId(\"5fec31b5b9022035abbbf7c2\")",
"message": {
"date": "2020-12-30 13:20:24",
"time": "2020-12-30T07:50:24.000Z",
"ID": "005",
"P": 1.5
}
},
{
"_id": "ObjectId(\"5fec31b0b9022035abbbf7b3\")",
"message": {
"date": "2020-12-30 13:20:19",
"time": "2020-12-30T07:50:19.000Z",
"ID": "005",
"P": 1.63
}
}
I want to find the average of P value per min.
I've tried Group result by 15 minutes time interval in MongoDb
but I got an error in the time field.
db.pressure.aggregate([
{ "$group": {
"_id": {
"year": { "$year": "message.time" },
"dayOfYear": { "$dayOfYear": "message.time" },
"hour": { "$hour": "message.time" },
"interval": {
"$subtract": [
{ "$minute": "message.time" },
{ "$mod": [{ "$minute": "message.time"}, 1] }
]
}
},
"count": { "$sum": 1 }
}
}
])
{
"message" : "can't convert from BSON type string to Date",
"ok" : 0,
"code" : 16006,
"codeName" : "Location16006",
"name" : "MongoError"
}
Few Fixes:
You need to convert your string date to date type using $toDate (From MongoDB 4.0), $addFields will update message.time field to date type
$year, $hour etc operators requires reference of field using $ sign you missed it in message.time
use $avg to get average of message.P
db.collection.aggregate([
{ $addFields: { "message.time": { $toDate: "$message.time" } } },
{
"$group": {
"_id": {
"year": { "$year": "$message.time" },
"dayOfYear": { "$dayOfYear": "$message.time" },
"hour": { "$hour": "$message.time" },
"interval": {
"$subtract": [
{ "$minute": "$message.time" },
{ "$mod": [{ "$minute": "$message.time" }, 15] }
]
}
},
"count": { "$count": 1 },
"average": { "$avg": "$message.P" }
}
}
])
Playground
MongoDB 3.6: you can use $dateFromString
{
$addFields: {
"message.time": {
$dateFromString: { dateString: "$message.time" }
}
}
}
Playground
I am trying to generate data for a heat map on my dashboard. I need to group the data by days i.e Monday, Tuesday e.t.c and then group the data by time intervals. I am able to group the data by days. But the time interval part doesnt work. I am using MongoDB.
Sample of my Schema is shown below
{
"createdAt": "2020-06-30T22:47:32+00:00",
"day": "Wednesday",
"user": {
"$oid": "5ec51d59ddfb380017649591"
},
"country": "NG",
"city": "",
"userType": "superadmin",
"fullName": "Jane Doe",
"__v": 0
}
This is what I could come up with so far
let heatMap = await await Session.aggregate([
{
$group: {
_id: "$day",
count: { $sum: 1 },
},
},
]);
This groups by day for me already.
I need to further group by intervals of hours
Sample Expected Data
[
{
_id: "Sunday",
intervals: [
{
interval1: "12am-6am",
count: 34,
},
{
interval2: "6am-12noon",
count: 44,
},
],
},
{
_id: "Monday",
intervals: [
{
interval1: "12am-6am",
count: 34,
},
{
interval2: "6am-12noon",
count: 44,
},
],
},
];
You can use this pipeline that utilizes $mod with some date operators to achieve the result you want.
The strategy is first to group by the intervals and the finally by the day of the week as this simplifies the process.
db.collection.aggregate([
{
"$group": {
"_id": {
"$toDate": {
"$subtract": [
{
"$toLong": {
$toDate: "$createdAt"
}
},
{
"$mod": [
{
"$toLong": {
$toDate: "$createdAt"
}
},
21600000
]
}
]
}
},
count: {
$sum: 1
}
}
},
{
$group: {
_id: {
day: {
$dayOfWeek: "$_id"
},
hour: {
$hour: "$_id"
}
},
count: {
$sum: "$count"
},
interval: {
$first: "$_id"
}
}
},
{
$group: {
_id: "$_id.day",
intervals: {
$push: {
count: "$count",
interval: "$interval"
}
}
}
}
])
Notice that the result is not exactly how you want, i.e _id: 3 is equal to _id: "Tuesday" As Mongo does not provide any out of the box conversions you'll have to do it yourself either in code or by using something like $switch
Mongo Playground
here is the query
[
{
"$project": {
"formattedDate": {
"$dateToString": { "format": "%Y-%m-%d", "date": "$ceatedAt" }
},
"createdAtMonth": { "$month": "$ceatedAt" },
"rating": 1
}
},
{
"$group": {
"_id": "$formattedDate",
"average": { "$avg": "$rating" },
"month": { "$first": "$createdAtMonth" },
}
}
]
I need the date in timestamp. How to do that?
Mongodb 4.0 has introduced $toLong aggregation which convert date to timestamp
db.collection.aggregate([
{ "$project": {
"createdAt": {
"$toLong": "$createdAt"
}
}}
])
You can try it here
Use $subtract arithmetic aggregation operator with your Date as minuend and new Date("1970-01-01") as subtrahend.
db.collection.aggregate(
{
$project: { "timestamp": { $subtract: [ "$createdAt", new Date("1970-01-01") ] } }
}
);
For document
{ "_id": 1, "createdAt": ISODate("2016-09-01T14:35:14.952Z") }
the result is
{ "_id": 1, "timestamp": NumberLong("1472740514952") }
If you want to group both by timestamp and (year, month, date) you can divide timestamp by the amount of milliseconds in a day, so that it will be unique for each day (and not for each millisecond)
db.collection.aggregate(
{
$project:
{
"timestampByDay":
{
$floor:
{
$divide:
[
{ $subtract: [ "$createdAt", new Date("1970-01-01") ] },
24 * 60 * 60 * 1000
]
}
},
"date": "$createdAt"
}
},
{
$group:
{
"_id": "$timestampByDay",
"date": { $first: "$date" }
}
}
);
If you want to update timestamp. with the current date and time use the below query.
db.getCollection('movie').update(
{"applicationId":"2b5958d9629026491c30b42f2d5256fa8","type":"shortcut"},
{$set : {userName:"vipin+testkm23052020#applozic.com",created_at: NumberLong(new Date()),"updated_at":NumberLong(new Date()),"docIndex":UUID()}}, {multi:true, upsert: false}
)
I have the following pipeline in my aggregation:
$group: {
_id: {
$dateToString: {
format: '%Y-%m-%d',
date: '$created_at'
}
},
num: {
$sum: 1
}
}
This returns me the sum of documents grouped by data, as such:
[
{
"_id": "2015-04-21",
"num": 1871
}
]
Now I would like to change the output to something like this:
[
["2015-04-21", 1871]
]
Is this doable within the aggregation pipeline? Or do I have to write my own transformation method?
You can use the $addToSet and $setUnion operators in your pipeline as follows:
db.collection.aggregate([
{
"$group": {
"_id": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created_at"
}
},
"num": {
"$sum": 1
}
}
},
{
"$group": {
"_id": "$_id",
"A": {
"$addToSet": "$_id"
},
"B": {
"$addToSet": "$num"
}
}
},
{
"$project": {
"_id": 0,
"finalArray": {
"$setUnion": [ "$A", "$B" ]
}
}
}
]);
Output:
/* 0 */
{
"result" : [
{
"finalArray" : ["2015-04-21", 1871]
}
],
"ok" : 1
}