retrieve all whose `date` is within last 6 minutes - node.js

I have a Product collection stored in MongoDB. It has a attribute date :
const productSchema = new mongoose.Schema<ProductAttrs>({
date: {
type: Date,
required: true,
}
...
})
I would like to retrieve all products whose date is within last 6 minutes comparing with current time. What is the efficient way to retrieve that in mongoose ?
I am using express.js + typescript + mongoose v5.11 in my project.

You must use Mongoose to search every record where the time is greater than now - 6 minutes.
I have no way to test it, but the query should look something like this:
const sixMinutes = (6*60*1000); // 6 minutes
let sixMinutesAgo = new Date();
sixMinutesAgo.setTime(sixMinutesAgo.getTime() - sixMinutes);
const products = Product.find({
date: {
$gte: sixMinutesAgo
}
})

If your MongoDB server version is 5.0 you could use the new operator $dateDiff. In this case it is very handy and you do not even need an aggregate:
db.collection.find({
"$expr": {
"$lte": [
{
"$dateDiff": {
startDate: "$date",
endDate: new Date(),
unit: "minute"
}
},
6
]
}
})
It allows you to specify the unit field, which makes the query very straightforward in my opinion.
$expr evaluates an expression and returns true or false, just that. If the expression result is true then the row is returned, otherwise it is not.
$lte means less than or equal (the <= operator) and it returns true only when the first argument is less than or equal the second operator. Basically, this is what it does in pseudocode:
foreach doc in collection
if (minute)(currentDate - document.date) <= 6 then
return document

Related

mongodb query referenced objects inside array

I'm using mongoose in nodejs. I'm trying to query some fields that contains a list made of referenced objects. I created two schemas
PILOT
var PilotSchema = new mongoose.Schema({
races: [{type: mongoose.Schema.Types.ObjectId, ref: 'Race'}],
});
RACE
var RaceSchema = new mongoose.Schema({
start_ms: Number,
});
Here I wrote only the relevant fields.
In the race schema I have a key start_ms that indicates the date of the start of the race in millisecons, and in the PilotSchema races field I save the references to many races.
What I want to do is:
Take a date in the future ( my_date ) in milliseconds
Retrive all the pilots that have not a race in the interval (my_date - 2_hours, my_date + 2_hours)
I try to do an example for explain better.
My date: Feb 23 2020 - 12:00:00 ( in milliseconds ),
In the pilot field races I have many refences to the RaceSchema and in each refenced object ther is the time of the start of a race.
Now querying the Pilots collection I want to retrieve all the pilots that in they races field have not a race that start during (Feb 23 2020 - 10:00:00 - Feb 23 2020 - 14:00:00)
I tried using this query but didn't work
races: {$not:
{$elemMatch:
{start_ms:
{ "$gt" : min_time, "$lt" : max_time }
}
}
}
Where
min_time, max_time = my_date - 2_hours, my_date + 2_hours
Actually I don't know how to do it, I saw that for retrive a number in an intervall I can use $gt and $lt but I don't understand how to query an array of referenced objects that respect this specific conditions.
You'd have to write an aggregate query for this, finding your reference of races from pilot & filter them based on your condition
Pilot.aggregate([
{
$lookup:{
from:'races',
localField:'races',
foreignField:'_id',
as:'races'
}
},{
$unwind:'$races'
},
{
$match:{
'races.start_ms':{
"$gt" : min_time, "$lt" : max_time
}
}
},{
$group:{
_id:'$_id',
name:{$first:'$name'}
}
}
]);
Assuming that you have name in your pilot schema, you can get list of all such pilots by writing this query

How to get date difference in mongoDB ? Date should be come in difference of days?

I wants to filter the collection on the basis of subtract expiry date object with current date and which is less than equal to 10 days.
I am using below code but I am getting date difference in millisecond. I want in exact day difference.
db.metaobject.aggregate(
{ $unwind :'$certifications.expiry_date'},
{$project:{
_id:1,name:1,date-Difference: { $divide:[ {$subtract: [ "$certifications.expiry_date",new Date ]},86400000] }
}
},
{$match:{
dateDifference:{$lte:10}
}
}
)
If its to be used in node, you dont need to compute the difference :
Compute directly the date + 10 in node js then just to the $lte :
var date = new Date();
var date10 = new Date(date.getTime());
date10.setDate(date10.getDate() + 10);
db.metaobject.aggregate(
{ $unwind :'$certifications.expiry_date'},
{$match:{
dateDifference:{$lte: new Date(date10).toJSON()}
}
}
)

Mongoose find all documents where array.length is greater than 0 & sort the data

I am using mongoose to perform CRUD operation on MongoDB. This is how my schema looks.
var EmployeeSchema = new Schema({
name: String,
description: {
type: String,
default: 'No description'
},
departments: []
});
Each employee can belong to multiple department. Departments array will look like [1,2,3]. In this case departments.length = 3. If the employee does not belong to any department, the departments.length will be equal to 0.
I need to find all employee where EmployeeSchema.departments.length > 0 & if query return more than 10 records, I need to get only employees having maximum no of departments.
Is it possible to use Mongoose.find() to get the desired result?
Presuming your model is called Employee:
Employee.find({ "departments.0": { "$exists": true } },function(err,docs) {
})
As $exists asks for the 0 index of an array which means it has something in it.
The same applies to a maximum number:
Employee.find({ "departments.9": { "$exists": true } },function(err,docs) {
})
So that needs to have at least 10 entries in the array to match.
Really though you should record the length of the array and update with $inc every time something is added. Then you can do:
Employee.find({ "departmentsLength": { "$gt": 0 } },function(err,docs) {
})
On the "departmentsLength" property you store. That property can be indexed, which makes it much more efficient.
By some reason, selected answer doesn't work as for now. There is the $size operator.
Usage:
collection.find({ field: { $size: 1 } });
Will look for arrays with length 1.
use can using $where like this:
await EmployeeSchema.find( {$where:'this.departments.length>0'} )
If anyone is looking for array length is greater than 1, you can do like below,
db.collection.find({ "arrayField.1" : { $exists: true }})
The above query will check if the array field has value at the first index, it means it has more than 1 items in the array. Note: Array index start from 0.

In Mongoose, how to filter an array of objects

I have the following schema:
var sampleSchema = new Schema({
name: String,
dates: [{
date: Date,
duration: Number
}]
});
I'd need to filters the records according to the following rule: if one of dates is later than a given date date_begin, keep the record, otherwise, don't.
I have the impression that $gte or $lte are the function I need, but I can't find a way to use them correctly. I tried
sampleSchema.find({date_begin: {$gte: 'date'}});
or some variants of that, but I can't seem to be able to make it work. Anyone has an idea of how I am supposed to do this?
To do querying on elements inside arrays, $elemMatch is used :
SampleModel.find( { dates : { $elemMatch: { date : { $gte: 'DATE_VALUE' } } } } )
If you're using a single query condition, you can directly filter:
SampleModel.find( { 'dates.date': { $gte: 'DATE_VALUE' } } )

MongoDB/Mongoose querying at a specific date?

Is it possible to query for a specific date ?
I found in the mongo Cookbook that we can do it for a range Querying for a Date Range
Like that :
db.posts.find({"created_on": {"$gte": start, "$lt": end}})
But is it possible for a specific date ?
This doesn't work :
db.posts.find({"created_on": new Date(2012, 7, 14) })
That should work if the dates you saved in the DB are without time (just year, month, day).
Chances are that the dates you saved were new Date(), which includes the time components. To query those times you need to create a date range that includes all moments in a day.
db.posts.find({ //query today up to tonight
created_on: {
$gte: new Date(2012, 7, 14),
$lt: new Date(2012, 7, 15)
}
})
...5+ years later, I strongly suggest using date-fns instead
import endOfDayfrom 'date-fns/endOfDay'
import startOfDay from 'date-fns/startOfDay'
MyModel.find({
createdAt: {
$gte: startOfDay(new Date()),
$lte: endOfDay(new Date())
}
})
For those of us using Moment.js
const moment = require('moment')
const today = moment().startOf('day')
MyModel.find({
createdAt: {
$gte: today.toDate(),
$lte: moment(today).endOf('day').toDate()
}
})
Important: all moments are mutable!
tomorrow = today.add(1, 'days') does not work since it also mutates today. Calling moment(today) solves that problem by implicitly cloning today.
Yeah, Date object complects date and time, so comparing it with just date value does not work.
You can simply use the $where operator to express more complex condition with Javascript boolean expression :)
db.posts.find({ '$where': 'this.created_on.toJSON().slice(0, 10) == "2012-07-14"' })
created_on is the datetime field and 2012-07-14 is the specified date.
Date should be exactly in YYYY-MM-DD format.
Note: Use $where sparingly, it has performance implications.
Have you tried:
db.posts.find({"created_on": {"$gte": new Date(2012, 7, 14), "$lt": new Date(2012, 7, 15)}})
The problem you're going to run into is that dates are stored as timestamps in Mongo. So, to match a date you're asking it to match a timestamp. In your case I think you're trying to match a day (ie. from 00:00 to 23:59 on a specific date). If your dates are stored without times then you should be okay. Otherwise, try specifying your date as a range of time on the same day (ie. start=00:00, end=23:59) if gte doesn't work.
similar question
You can use following approach for API method to get results from specific day:
# [HTTP GET]
getMeals: (req, res) ->
options = {}
# eg. api/v1/meals?date=Tue+Jan+13+2015+00%3A00%3A00+GMT%2B0100+(CET)
if req.query.date?
date = new Date req.query.date
date.setHours 0, 0, 0, 0
endDate = new Date date
endDate.setHours 23, 59, 59, 59
options.date =
$lt: endDate
$gte: date
Meal.find options, (err, meals) ->
if err or not meals
handleError err, meals, res
else
res.json createJSON meals, null, 'meals'
i do it in this method and works fine
public async getDatabaseorderbyDate(req: Request, res: Response) {
const { dateQuery }: any = req.query
const date = new Date(dateQuery)
console.log(date)
const today = date.toLocaleDateString(`fr-CA`).split('/').join('-')
console.log(today)
const creationDate = {
"creationDate": {
'$gte': `${today}T00:00:00.000Z`,
'$lt': `${today}T23:59:59.999Z`
}
};
`
``
Problem I came into was filtering date in backend, when setting date to 0 hour, 0 minute, 0 second, 0 milisecond in node server it does in ISO time so current date 0 hour, 0 minute, 0 second, 0 milisecond of client may vary i.e. as a result which may gives a day after or before due to conversion of ISO time to local timezone
I fixed those by sending local time from client to server
// If client is from Asia/Kathmandu timezone it will zero time in that zone.
// Note ISODate time with zero time is not equal to above mention
const timeFromClient = new Date(new Date().setHours(0,0,0,0)).getTime()
And used this time to filter the documents by using this query
const getDateQuery = (filterBy, time) => {
const today = new Date(time);
const tomorrow = new Date(today.getDate() + 1);
switch(filterBy) {
case 'past':
return {
$exists: true,
$lt: today,
};
case 'present':
return {
$exists: true,
$gte: today,
$lt: tomorrow
};
case 'future':
return {
$exists: true,
$gte: tomorrow
};
default:
return {
$exists: true
};
};
};
const users = await UserModel.find({
expiryDate: getDateQuery('past', timeFromClient)
})
This can be done in another approach using aggregate if we have timezoneId like Asia/Kathmandu
const getDateQuery = (filterBy) => {
const today = new Date();
const tomorrow = new Date(today.getDate() + 1);
switch(filterBy) {
case 'past':
return {
$exists: true,
$lt: today,
};
case 'present':
return {
$exists: true,
$gte: today,
$lt: tomorrow
};
case 'future':
return {
$exists: true,
$gte: tomorrow
};
default:
return {
$exists: true
};
};
};
await UserModel.aggregate([
{
$addFields: {
expiryDateClientDate: {
$dateToParts: {
date: '$expiryDate',
timezone: 'Asia/Kathmandu'
}
}
},
},
{
$addFields: {
expiryDateClientDate: {
$dateFromParts: {
year: '$expiryDateClientDate.year',
month: '$expiryDateClientDate.month',
day: '$expiryDateClientDate.day'
}
}
},
},
{
$match: {
expiryDateClientDate: getDateQuery('past')
}
}
])
We had an issue relating to duplicated data in our database, with a date field having multiple values where we were meant to have 1. I thought I'd add the way we resolved the issue for reference.
We have a collection called "data" with a numeric "value" field and a date "date" field. We had a process which we thought was idempotent, but ended up adding 2 x values per day on second run:
{ "_id" : "1", "type":"x", "value":1.23, date : ISODate("2013-05-21T08:00:00Z")}
{ "_id" : "2", "type":"x", "value":1.23, date : ISODate("2013-05-21T17:00:00Z")}
We only need 1 of the 2 records, so had to resort the javascript to clean up the db. Our initial approach was going to be to iterate through the results and remove any field with a time of between 6am and 11am (all duplicates were in the morning), but during implementation, made a change. Here's the script used to fix it:
var data = db.data.find({"type" : "x"})
var found = [];
while (data.hasNext()){
var datum = data.next();
var rdate = datum.date;
// instead of the next set of conditions, we could have just used rdate.getHour() and checked if it was in the morning, but this approach was slightly better...
if (typeof found[rdate.getDate()+"-"+rdate.getMonth() + "-" + rdate.getFullYear()] !== "undefined") {
if (datum.value != found[rdate.getDate()+"-"+rdate.getMonth() + "-" + rdate.getFullYear()]) {
print("DISCREPENCY!!!: " + datum._id + " for date " + datum.date);
}
else {
print("Removing " + datum._id);
db.data.remove({ "_id": datum._id});
}
}
else {
found[rdate.getDate()+"-"+rdate.getMonth() + "-" + rdate.getFullYear()] = datum.value;
}
}
and then ran it with mongo thedatabase fixer_script.js
Well a very simple solution to this is given below
const start = new Date(2020-04-01);
start.setHours(0, 0, 0, 0);
const end = new Date(2021-04-01);
end.setHours(23, 59, 59, 999);
queryFilter.created_at={
$gte:start,
$lte:end
}
YourModel.find(queryFilter)
So, the above code simply finds the records from the given start date to the given end date.
Seemed like none of the answers worked for me. Although someone mentioned a little hint, I managed to make it work with this code below.
let endDate = startingDate
endDate = endDate + 'T23:59:59';
Model.find({dateCreated: {$gte: startingDate, $lte: endDate}})
startingDate will be the specific date you want to query with.
I preferred this solution to avoid installing moment and just to pass the startingDate like "2021-04-01" in postman.

Resources