NodeJS MongoDB: Querying ISO Date - node.js

I have a collection of ISO dates stored in MongoDB as strings, like so:
{ "date": "2014-12-12T03:33:33.333Z" },
{ "date": "2014-12-13T03:33:33.333Z" }
In the console, I can query these perfectly using
{ "date": ISODate("2014-12-12T03:44:00.000Z") }
However, I'm using the NodeJS driver, and I cannot use ISODate. I have found several questions pertaining to this problem on here, but none of the proposed solutions seem to work. For instance:
// These does not find any matches
db.find({ "date": new Date("2014-12-12T03:44:00.000Z") })
db.find({ "date": { '$eq': '2014-12-12T03:44:00.000Z' } })
db.find({ "date": { '$eq': new Date('2014-12-12T03:44:00.000Z') } })
//This throws an error stating $date is not an operator
db.find({ "date": { '$date': '2014-12-12T03:44:00.000Z' } })
Why are these queries failing?
Edit: Here's another sample, straight from the database:
{
"_id": "5a7e88f34b5916723589183f",
"date": "2014-12-12T03:42:00.000Z",
"granularity": 180
}
EDIT 2: This query produces the following error MongoError: $dateFromString requires that 'dateString' be a string, found: date with value 2014-12-12T03:44:00.000Z
async loadCandle(date, granularity) {
date = date + ''; //Aded to ensure date is a string, but still get the error.
var candle = await this.data.collection('dates').findOne(
{ $expr :
{$eq :
[
{$dateFromString : {dateString : "$date"}},
new Date("2014-12-12T03:33:33.333Z") //Normally would pass in the `date` variable here
]
} });

because $date is not an operator
you need to use $dateFromString to convert string date to ISODate for comparison
db.datez.find(
{$expr :
{$eq :
[
{$dateFromString : {dateString : "$date"}},
new Date("2014-12-12T03:33:33.333Z")
]
}
}
)
using aggregation
db.datez.aggregate([
{$match :
{$expr :
{$eq :
[
{$dateFromString : {dateString : "$date"}},
new Date("2014-12-12T03:33:33.333Z")
]
}
}
}
])
collection
> db.datez.find()
{ "_id" : ObjectId("5a7e795e80aae386f73cf0fe"), "date" : "2014-12-12T03:33:33.333Z" }
{ "_id" : ObjectId("5a7e795e80aae386f73cf0ff"), "date" : "2014-12-13T03:33:33.333Z" }
>
result
> db.datez.find({$expr : {$eq : [{$dateFromString : {dateString : "$date"}}, new Date("2014-12-12T03:33:33.333Z")]}})
{ "_id" : ObjectId("5a7e795e80aae386f73cf0fe"), "date" : "2014-12-12T03:33:33.333Z" }

You can use $dateToString operator which generate the string date of any specified format, which can be compared later.
For string comparison of date, input format should be YYYY-MM-DD, any other format would get fail for ranges date queries
Let me explain through example:
Here is my collection in mongoDb :
{
"_id" : ObjectId("5f2d0a0c632ec022e08c3191"),
"date" : ISODate("2020-07-12T00:00:00Z")
}
{
"_id" : ObjectId("5f2d0a12632ec022e08c3192"),
"date" : ISODate("2020-07-13T00:00:00Z")
}
Now the query to be fired from Node for comparison of such stored ISODates is as follow
db.collection.aggregate(
[
{
$addFields: {
formattedDate: { // An extra field "formattedDate" is added in each document which can be compared later through pipeline using $match
$dateToString: {
format: "%Y-%m-%d",
date: "$date" // in "$date" date is variable from db
}
}
}
},
{
$match: {
formattedDate: {
$eq: "2020-07-12" // here you can provide your input date yyyy-mm-dd
}
}
}
]
)
So for above query you will get output as
{
"_id" : ObjectId("5f2d0a0c632ec022e08c3191"),
"date" : ISODate("2020-07-12T00:00:00Z"),
"formattedDate" : "2020-07-12"
}
Hope this will help you or somebody else!

Related

Mongoose: can't query array of dates

I have a test document like that:
{
"_id" : ObjectId("5fb6b0ed9cad6e97cfc24c2d"),
"dates" : [
{
"date" : ISODate("2020-01-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-02-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-03-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-04-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-05-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-06-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-07-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-08-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-09-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-10-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-11-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-12-01T00:00:00.000Z")
}
]
}
Now I want to retrieve only dates $gt: 2020-14-01T00:00:00.000Z. I tried a lot of combinations but none of them worked for me.
This is one of the queries I tried (taken from Mongodb docs):
db.getCollection('things').find({_id: ObjectId("5fb6b0ed9cad6e97cfc24c2d"), "dates.date": { $gt: new Date("2020-04-01T00:00:00.000Z")} } )
But it return all the document, not the gt... i tried using new Date and new ISODate too, but same effect.
Thank you.
First of all, according to mongo documentation dates are in the format "<YYYY-mm-dd>".
Also more formats are allowed, but If you try to use 2020-14-01 as date it will fail (unless you convert string to date with an specific format) because month is 14.
But, answering the question, you need a query like this:
EDITED
db.collection.aggregate({
"$match": {
"_id": ObjectId("5fb6b0ed9cad6e97cfc24c2d"),
}
},
{
"$project": {
"dates": {
"$filter": {
"input": "$dates",
"as": "item",
"cond": {
"$gt": [
"$$item.date",
ISODate("2020-01-14T00:00:00.000Z")
]
}
}
}
}
})
First $match by _id to get only the document you want. And then using $project to create the fields you want to get. You can filter in the array those values whose field date is greater than your date using $filter and $gt
Note that I've used 2020-01-14 to avoid errors.
Example here.
Also another example using $dateFromString in this query.
Edit: You can use $dateFromString and specify a format. Check this example

The api is returning the date that matches and also dates that does not matches the specified date

I'am developing an api in nodejs.
I have the document in follwing stucture:
{
"_id" : ObjectId("5ecd26504df3372a38afffd9"),
"balance" : 104000,
"bankID" : "Bank-1",
"userEmail" : "kumarshreyas073#gmail.com",
"bankName" : "Corporation Bank",
"accountNumber" : "03214569874563",
"ifsCode" : "CORP0001236",
"branch" : "Udupi",
"address" : "Udupi",
"city" : "Udupi",
"state" : "Karnataka",
"openingBalance" : 100000,
"transactions" : [
{
"credit" : 2000,
"debit" : 0,
"_id" : ObjectId("5ecd26614df3372a38afffea"),
"transactionID" : "CashTransaction-5ecd26614df3372a38afffe8",
"date" : "30-05-2026",
"particulars" : "By Cash-1",
"voucherType" : "Cash"
},
{
"credit" : 0,
"debit" : 2000,
"_id" : ObjectId("5ecd272d4df3372a38b00012"),
"transactionID" : "Receipt-5ecd272d4df3372a38b00009",
"date" : "29-07-2020",
"particulars" : "To Suresh kumar",
"voucherType" : "Receipt"
},
{
"credit" : 0,
"debit" : 2000,
"_id" : ObjectId("5ecd272d4df3372a38b00014"),
"transactionID" : "Receipt-5ecd272d4df3372a38b00003",
"date" : "30-05-2024",
"particulars" : "To Karthik",
"voucherType" : "Receipt"
}
],
"idCounter" : 1,
"__v" : 0
}
I need to extract only those transactions between from date = "20/07/2020" and to date = "31/07/2020".
The code I written is as follows:
exports.trail_balance = async (req, res, next) => {
var trailBalance = {
userEmail: req.body.userEmail,
fromDate: req.body.fromDate,
toDate: req.body.toDate,
};
var bankAccount = await Bank.aggregate([
{ $match: { userEmail: req.body.userEmail } },
{
$addFields: {
transactions: {
$filter: {
input: "$transactions",
as: "transactions",
cond: {
$and: [
{
$gte: ["$$transactions.date", trailBalance.fromDate],
},
{
$lte: ["$$transactions.date", trailBalance.toDate],
},
],
},
},
},
},
},
]);
res.status(200).json({
result: 1,
bankAccount: bankAccount.length > 0 ? bankAccount : [],
});
};
Actual result I expect is:
{
"result": 1,
"bankAccount": [
{
"_id": "5ecd26504df3372a38afffd9",
"balance": 104000,
"bankID": "Bank-1",
"userEmail": "kumarshreyas073#gmail.com",
"bankName": "Corporation Bank",
"accountNumber": "03214569874563",
"ifsCode": "CORP0001236",
"branch": "Udupi",
"address": "Udupi",
"city": "Udupi",
"state": "Karnataka",
"openingBalance": 100000,
"transactions": [
{
"credit" : 0,
"debit" : 2000,
"_id" : ObjectId("5ecd272d4df3372a38b00012"),
"transactionID" : "Receipt-5ecd272d4df3372a38b00009",
"date" : "29-07-2020",
"particulars" : "To Suresh kumar",
"voucherType" : "Receipt"
}
],
"idCounter": 1,
"__v": 0
}
But, I'am getting all transactions.
I even tried passing date in, from date = "20-07-2020" and to date = "31-07-2020". This too returns all transaction.
All the date stored in DB are of type String.
The problem is your date format. As you have started your date with day in your saved data in database and type of date is string, so in comparison to your query, it always start with day and it's incorrect. because in date comparison, at first years must be compared, then months and lastly, day. But you are doing it in wrong way.
In this scenario, mongodb is doing write! because in your from date, 2 is less or equal to 2 and 3 and in your to date, 3 is greater or equal to 2 and 3. So its doing well.
I changed your saved data date format to yyyy-mm-dd and your query was correct.
If changing data is not possible for you, you can also change data in a pipeline stage of your aggregate query. Use the link below:
https://docs.mongodb.com/manual/reference/operator/aggregation/dateFromString/
You are almost there. The problem is with your data. It has invalid date format.
mongo playground
I changed to proper date format for one of the transactions as below
"date": ISODate("2026-05-30"),
mongo date format
So if you have proper format, then the query will work.
new Date("<YYYY-mm-dd>") returns the ISODate with the specified date.
new Date("<YYYY-mm-ddTHH:MM:ss>") specifies the datetime in the client’s local timezone and returns the ISODate with the specified datetime in UTC.
new Date("<YYYY-mm-ddTHH:MM:ssZ>") specifies the datetime in UTC and returns the ISODate with the specified datetime in UTC.
new Date(<integer>) specifies the datetime as milliseconds since the Unix epoch (Jan 1, 1970), and returns the resulting ISODate instance.
Suggesting few fixes,
convert your string date to ISO date using new Date("2020-07-31"),
var trailBalance = {
userEmail: req.body.userEmail,
fromDate: new Date(req.body.fromDate),
toDate: new Date(req.body.toDate),
};
convert collection's field transactions.date string to ISO date using $dateFromString
format: %d-%m-%Y should match exact format of date in transactions.date
{
$dateFromString: {
dateString: "$$transactions.date",
format: "%d-%m-%Y"
}
}
Look at the Working Playground: https://mongoplayground.net/p/-KWgRCSwD8h
Your final query,
var bankAccount = await Bank.aggregate([
{
$match: {
userEmail: trailBalance.userEmail
}
},
{
$addFields: {
transactions: {
$filter: {
input: "$transactions",
as: "transactions",
cond: {
$and: [
{
$gte: [
{
$dateFromString: {
dateString: "$$transactions.date",
format: "%d-%m-%Y"
}
},
trailBalance.fromDate
]
},
{
$lte: [
{
$dateFromString: {
dateString: "$$transactions.date",
format: "%d-%m-%Y"
}
},
trailBalance.toDate
]
}
]
}
}
}
}
}
]);

Native NodeJS MongoDB driver returns different result than direct query

I have a Node/Express backend using the native mongodb driver to handle data. I have the following function that extracts data from my mongodb database:
else if (year && grouping == 2){
collection.aggregate([
{ $match: {postdate: { $gte : new Date(yearstart), $lte : new Date(yearend)},factor:1 }},
{ $group: { _id: {"category":"$category","month":{ "$month": "$postdate" }} , "total": { $sum: "$debit" } } },
{ $project: { "_id":0, "category":"$_id.category", "month":"$_id.month", "total":1 } }
]
).toArray(function(err, items) {
console.log("get transactions by year and month category grouping result: ", items);
res.json(GroupByMonthCat(items));
}
The value of yearstart is: '2018-01-01' and the value of yearend is '2018-12-01'. So in my query I am grouping the matched records on the month part of postdate.
However, the result I get is the following for (category: Bills and month:1):
{
"subtotal" : 475,
"category" : "Bills",
"month" : 1
}
When I run the same query outside my app directly against my mongodb:
db.transtest.aggregate(
[
{ $match: {category: "Bills" ,postdate: { $gte : new Date("2018-01-01"), $lte : new Date("2018-01-31")},factor:1 }},
{ $group: { _id: {"category":"$category","month":{ "$month": "$postdate" }} , "subtotal": { $sum: "$debit" } } },
{ $project: { "_id":0, "category":"$_id.category", "month":"$_id.month", "subtotal":1} },
{ $sort : { category : -1, month: 1 } }
]
);
I get the following result:
{
"subtotal" : 183.9,
"category" : "Bills",
"month" : 1
}
Why would this be happening? There are only two results in my database for the month of January 2018, hence my expected subtotal of 183.9 as shown when I run the same exact query directly against my database:
{
"_id" : ObjectId("5a7641b0d4b6828ad37c99a1"),
"transid" : 5500,
"postdate" : ISODate("2018-01-16T05:00:00.000Z"),
"category" : "Bills",
"debit" : 53.9
},
{
"_id" : ObjectId("5a7641b0d4b6828ad37c99a5"),
"transid" : 5504,
"postdate" : ISODate("2018-01-25T05:00:00.000Z"),
"category" : "Bills",
"debit" : 130
}

Mongo query using filter not fetching exact result?

My mongo table contains contains collection 'Shops' and data like the below:
{
"ShopId" : 9999,
"products" : [
{
"productId" : "1234",
"productName" : "abcd",
},
{
"productId" : "5678",
"productName" : "abc",
},
{
"productId" : "2345",
"productName" : "def",
}
],
}
There will be several shops in the table having a list of products.
Requirement:
I want to fetch the records having shopId=9999 and products matches the string abc
My query
model.Shops.aggregate([{$match:{"ShopId":9999}},{$project:{products:{$filter:{input:'$products',cond: {'productName':/abc/ }}}}}])
Problem:
It is fetching the productname:defwith the other data that matches productname:abc.
You can't use regex search with the $filter operator. The only way to achieve this is to unwind products, filter document and then re-group them in an array
model.Shops.aggregate([
{
$match:{
"ShopId":9999
}
},
{
$unwind:"$products"
},
{
$match:{
"products.productName":/abc/
}
},
{
$group:{
_id:null,
products:{
$push:{
"productName":"$products.productName",
"productId":"$products.productId"
}
}
}
}
])
output:
{
"_id":null,
"products":[
{
"productName":"abcd",
"productId":"1234"
},
{
"productName":"abc",
"productId":"5678"
}
]
}
to use it with a variable, declare your regex like this:
var regex: /abc/;
and then use it directly in the query
$match:{
"products.productName": regex
}
Below code worked on mongoshell for me, and your code was giving error to me, that Missing 'as' parameter to $filter.
db.Shops.aggregate([
{$match:{"ShopId":9999}},
{$project:{
products:{$filter:{input:'$products',as:"product",cond: { $eq: [ "$$product.productName", "abc" ] }}}
}}
])

How to get count and max date with single query in MongoDB?

I have Post collection as like as following:
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-01T08:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-01T09:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-15T09:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-04-04T11:21:39.736Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-04-04T21:23:13.331Z") }
I need to get total count and max date of post. So desired result for coeumtns above is the following:
{count: 5, date: ISODate("2014-04-04T21:23:13.331Z")}
How to get desired result with single query to MongoDB without handling and counting in application code?
EDIT: #chridam thanks for the response. I've accepted your answer as best one! Could help me with one more thing?
Let's say that posts are not exists yet, so I need to fetch result with zero count and current date as timestamp like the following:
{count: 0, [Date.now()]}
Is it possible with MongoDB ?
Use the $max and $sum operators as
Model.aggregate([
{
"$group": {
"_id": null,
"count": { "$sum": 1 },
"date": { "$max": "$date" }
}
}
]).exec(function (err, result) {
console.log(result);
})
EDIT: Addressing your further question with regards to an empty collection, the aggregate function will return an empty cursor since there wont be any documents to aggregate in the collection. So you would need to address this logic on the client i.e. check the results from the above aggregation, if the result is an empty array then create the placeholder doc as required:
Model.aggregate([
{
"$group": {
"_id": null,
"count": { "$sum": 1 },
"date": { "$max": "$date" }
}
}
]).exec(function (err, result) {
console.log(result);
if (!result.length) {
result = [{ count:0, date: new Date() }];
}
});

Resources