Mongo: strings converted to date but not 'querable' - string

Question

I have a collection with date stored as strings YYYY-mm-DD_HH:MM:SS.UUUZ like 2020-10-20_12:15:22.123+0100
My goal is to query on strings treating those as dates.
What am I doing:
I'm unwinding some header data on multiple documents:
{
"$unwind": {
"path": "$events",
"preserveNullAndEmptyArrays": true
}
}
and also
{
"$unwind": {
"path": "$events.hi2",
"preserveNullAndEmptyArrays": true
}
}
I'm adding a new field made with the string parsed as Date
{
"$addFields": {
"events.hi2.ConnectTimets": {
"$dateFromString": {
"dateString": "$events.hi2.ConnectTime",
"format": "%Y-%m-%d_%H:%M:%S.%L%Z"
}
}
}}
then on a $match stage I try to filter all records with date newer than 1 June 2020:
{
"$match":{
"events.hi2.ConnectTimets": {
"$gt": {"$dateFromString": {
"dateString": "2020-06-01",
"format": "%Y-%m-%d"
}
}
}
}
}
my result is Fetched 0 record(s) in 0ms
even though exists (at least a single document) in the database with a date matching the filter:
{
"_id" : ObjectId("5f438dfbf1feb13c4352e9f4"),
"timestamp" : NumberLong(1598262779045),
"attribute1" : [
"common"
],
"events" : [
{
"eventType" : "ty1",
"timestamp" : NumberLong(1598262779018),
"docId" : NumberLong(282578800148736),
"hi2" : {
"Priority" : 3,
"ClientId" : "client1",
"ConnectTime" : "2020-08-24_09:52:58.993+0000",
"Direction" : 1
}
},
{
"eventType" : "ty2",
"timestamp" : NumberLong(1598262781071),
"docId" : NumberLong(282578800148736),
"hi2" : {
"ref" : "bbbb"
}
}
]
}
When I espected something like
{
"_id" : ObjectId("5f438dfbf1feb13c4352e9f4"),
"timestamp" : NumberLong(1598262779045),
"attribute1" : [
"common"
],
"events" : [
{
"eventType" : "ty1",
"timestamp" : NumberLong(1598262779018),
"docId" : NumberLong(282578800148736),
"hi2" : {
"Priority" : 3,
"ClientId" : "client1",
"ConnectTime" : "2020-08-24_09:52:58.993+0000",
"Direction" : 1
}
}
}
Note_: the add field is ok because if i fire it without the match stage outputs a field with the string parsed as date

Answer 1

You should never store date/time values as string, use always proper Date objects.
Then the query is much simpler:
db.logging.aggregate([
{
$addFields: {
"events.hi2.ConnectTimets": {
$dateFromString: {
dateString: "$events.hi2.ConnectTime",
format: "%Y-%m-%d_%H:%M:%S.%L%Z"
}
}
}
},
{ $match: { "events.hi2.ConnectTimets": { $gte: ISODate("2020-06-01") } } }
])
When you have to work with date/time values then I recommend moment.js. Then you query could look like these:
{ $match: { "events.hi2.ConnectTimets": { $gte: moment("2020-06-01").toDate() } } }
{ $match: { "events.hi2.ConnectTimets": { $gte: moment("2020-06-01").tz('Europe/Zurich').toDate() } } }
{ $match: { "events.hi2.ConnectTimets": { $lte: moment.tz('Europe/Zurich').endOf('day').toDate() } } }