I am having hard time to understand the concept of querying date in mongodb/node.js, even after going through many SO and other articles in google.
While storing the date from a react datepicker in mongodb it gets stored a day less as per my time zone but when it is fetched it is shown the same,I can understand that it is again being converted into locale
but how do query responds?
When I query mongodb from node.js/mongoose - I don't find the correct resultSo my question is while querying the mongodb - how dates are passed to mongodb ?
for example :
post.find({publish_date:{ $gte:new Date()}}
What is the new Date value - localtime zone of the server? or browser?
is the new date value converted to utc while comparing the database or compared as passed from the server ?
I have a post model where I want the post to be published on the published_date as per my timezone.
for example - if I put 2021-01-21 mongo stores 2021-01-20T18:30:00.000+00:00 - it is understood that it stored as UTC
Now when I am querying the post to be published when the server date/time is 2021-01-21 - it is not fetch any document!
When I change the publish_date as 2021-01-22 mongo stores 2021-01-21T18:30:00.000+00:00 then query produces on document.
I am not sure how to solve this problem
Any help in this regard would be helpful
Thanks.
In MongoDB all Date values are stored as UTC-Times only.
Your client application is responsible to display these UTC times as local times, if required.
MongoDB does not preserve the input time zone, if you need this for any reason then you have to store it in a separate field.
In Mongo you can display locals date/times values with $dateToString:
{ $dateToString: {
date: <dateExpression>,
format: <formatString>,
timezone: <tzExpression>,
onNull: <expression>
} }
Whenever one has to work with date/time values then I recommend the moment.js library.
The query could be this for example:
{ $gte: moment().startOf('day').toDate() }
I am in Switzerland where we have UTC+01:00, thus moment().startOf('day').toDate() returns ISODate("2021-01-20T23:00:00Z")
Related
This is about a different date-time recorded in mongoDb via Node.js.
e.g. the date time recorded is 2022-02-01T13:32:01.967Z,
the actual time is 19:02 for the same date.
Looks like it has recorded GMT. This code will be eventually run on a VPS based in my country. However, at the moment I am running the code on localhost.
below is the code in Node.js
/// part of a controller method
const xDate = new Date();
console.log('new Date()', xDate);
// output: 2022-02-01T13:32:01.967Z, actual time 19:02 on same date
candidate.history.push({
actionDate: xDate,
userId: user.acctId,
userName: user.userName,
remarks: 'bla bla bla'
})
await candidate.save();
Any idea how to correct it? In Angular we can set the input field to datetime-local type to save the date/time without timezone. Can we do something similar in Node.js /mongodb? How do I get the correct time stored in mongo?
I have read that MongoDB always stores the timestamp in UTC. Is there any way this can be handled application-wide to handle all date values recd from Mongo in both the below cases:
at Node.js levelfor performing some date calculations based on date value recd from Angular?
At Angular level for date values recd from Node(as it is from Mongo)
UPDATE 2
Using the suggestions from #Bergi, I added a db execution helper to set the timezone before executing the query. This allows me to control the timezone based on the logged in user's preferences.
async function execWithTZ(sql, params, timezone) {
const client = await this.pool.connect();
await client.query<any>(`SET TIMEZONE TO '${timezone}'`, []);
return client.query<any>(sql, params);
}
const result = await execWithTZ('SELECT ...', [], 'Australia/Sydney');
QUESTION
I am trying to get Postgresql to return the day of the week of a specific date in a view. I later use this to display values by day.
The data is in a table with a column named updated which is a timestamptz field.
When running the following query using DBeaver SQL client against a PostgreSQL 12 database I get the values as I expect, with records updated on Thursday local time (just after midnight) showing up as Thursday
SELECT
count(v.id) AS count,
btrim(to_char(v.updated, 'DAY'::text)) AS day,
date_trunc('DAY'::text, v.updated) AS "updatedDay"
FROM table v
GROUP BY (to_char(v.updated, 'DAY'::text)), (date_trunc('DAY'::text, v.updated))
I put this in a view and when I query this view using node-postgres the counts are for the previous day (Wednesday), which I presume is because the database thinks it should interpret the dates in the UTC timezone.
To further prove the point, when I change the above query to use DAYTZ instead of just DAY in DBeaver I get THURSDAYAEST but in nodejs as result I get WEDNESDAYUTC
Changing all my dates to be without timezone and forcing everything to UTC on the way in is not an option for me.
How can I make node-postgres tell the database what timezone I want these dates interpreted as so that I can get the correct day?
UPDATE 1
I managed to get PostgreSQL to return the correct values to postgres node by setting the database user's timezone.
ALTER ROLE visuo_ingest SET TIMEZONE TO 'Australia/Sydney';
Now the counts for things that happened on Thursday Sydney time is counted for Thursday and not Wednesday.
Still interested in a way to do this on the connection rather than the database user level.
Still interested in a way to do this on the connection rather than the database user level.
You already found the right setting, there's no reason to alter it on a role only. You can also change the setting in a client session by running the SET command
SET TIMEZONE TO 'Australia/Sydney';
Just put that in a pgClient.query("…"); right after connecting the client.
The first answer of this question suggests that mongoose would adapt the date according to server timezone when retrieving data.
However, I don't have this comportement.
I set the (node) server timezone with :
process.env.TZ='Europe/Paris'
For exemple if I create a simple model like :
const mongoose = require("mongoose");
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
exports.Comment = mongoose.default.model('TestSchema', testSchema);
But if I create a date with 2020-01-01 20:20:20, when doing TestSchema.find() the date will be: 2020-01-01T19:20:20.000Z so there are two things that I don't understand :
Europe/Paris is actually UTC +2, so I would expect the date to be either 2020-01-01T18:20:20.000Z in UTC or 2020-01-01T20:20:20.000Z with the server timezone
How to have mongoose automatically set the date to the correct timezone?
I know that myDate is a Date object, so I can convert it manually but I'd rather not have to do it myself for simple reasons like forgetting to convert one of the dates in the application or not having to do it every time a Date field is added
An easy solution that I can think of would be to register a global plugin for mongoose which would use schema.set('toJSON', ... and schema.set('toObject', ...) with the transform method so I can loop through schema fields and if the field is a Date, update it to my timezone.
But I see two problems with this approch :
It doesn't sound very good performance-wise if I am querying a lot of documents each with a lot of fields
As you can see here I am currently not able to register global plugins...
What would be the best method to get the date in the server timezone format? I would rather still store them in UTC but set the hour according to the server timezone.
EDIT :
I just saw that while console.log(myDate) outputs 2018-01-01T19:20:20.000Z console.log(myDate.toString() outputs Mon Jan 01 2018 20:20:20 GMT+0100 (Central European Standard Time) so it seems likes this could be used, even tho I'd rather still have a Date object and converting it to string just before sending it to the client (would need some formatting tho since this format is not very user friendly). But then again, how would I do this globally and not for every date
A few things:
Europe/Paris at 2020-01-01T20:20:20 is UTC+1. It doesn't switch to UTC+2 until Summer Time kicks in on March 29th. Reference here. Thus the conversion to 2020-01-01T19:20:20Z is correct.
The output of console.log when passed a Date object is implementation specific. Some implementations will emit the output of .toString() (which is in local time in RFC 2822 format), and some will emit the output of .toISOString() (which is in UTC in ISO 8601 extended format). That is why you see the difference.
In general, it is not good to send a local time without also sending a time zone offset. ISO 8601 format is ideal, but you should send either 2020-01-01T19:20:20Z, or 2020-01-01T20:20:20+01:00. Don't just send the date and time without an offset to the client. Otherwise, if your client could be in a different time zone then they would interpret the value incorrectly.
Keep in mind that Date objects are not time zone aware. They contain only a Unix timestamp internally, and they convert only to the system's local time zone for the functions that work in local time. They cannot work in any other time zone.
Relying on the system local time zone is bad for portability. One doesn't always have the ability to change it, and it doesn't do well when you have to work in multiple time zones. It would be better to not rely on setting a local time zone from Node's TZ variable. Instead, consider writing your code to be independent of any local time zone setting.
A time zone aware date library can help with most of your concerns. I can recommend Luxon, js-Joda, Moment + Moment-Timezone, or date-fns + date-fns-timezone.
"how would I do this globally" is something I'm not following in your question. Try the approach I described, and if you still have issues then open a new question. Try to be specific and ask a single question. You're likely to get better results that way. Please read How do I ask a good question? and How to create a Minimal, Complete, and Verifiable example. Thanks.
To solve the issue:
npm i mongoose-timezone
In your schema file:
import timeZone from "mongoose-timezone";
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
// mongoose will save the dates based on user's timezone
testSchema.plugin(timeZone)
mongoose-timezone basically adds the current timezone offset to the
date before store and removes the offset when retrieving data. This
way dates are kept proportional in the database and in the app.
I do new Date() then save it into mongodb, the date became UTC format?
The datetime I inserted into mongodb when pull back is not the same, what should I do to solve this? Like I create a document on 1/12/2016, and I select range from 1/12/2016, the record is not there. Strange.
As the documentation says :
The mongo shell wraps the Date object with the ISODate helper. The ISODate is in UTC.
So my best guess is that your new Date() is not in UTC.
Math.floor((new Date()).getTime() / 1000)
as seen here: How do I get a UTC Timestamp in JavaScript?
However, Mongo is smart enough to sort this all out for you, so new Date() should be enough.
To query a Date in mongo, you can also use pure Date Objects, without the need to convert them to ISOString.
Keep everything as Date Objects, and it'll work as you intend it to!
The default timezone for node.js new Date() object is UTC (aka GMT).
You should leave it this way and just convert it on your front end. Javascript in the browser can convert the UTC timestamp to the browsers local timezone automatically.
new Date(Date.UTC(saved-timezone))
How do you create a JavaScript Date object with a set timezone without using a string representation
I'm wondering if this is normal, or if I'm missing something in the schema setup or query process:
My app, and mongoose, is correctly storing a date as UTC in mongodb. This is confirmed by viewing the documents via the mongo shell. When I retrieve the documents from mongodb via mongoose the date is now local time.
Is there a way to have mongoose keep the date as UTC when queried?
Mongoose and node.js aren't doing anything to your dates, it's simply that the JavaScript Date type produces a local time string when you call toString() on it even though it actually contains the time in UTC.
Explicitly call toUTCString() on your Date object if you want a UTC time string.
The timestamps are stored timezone agnostically, as a unix timestamp. This timestamp will work across timezones, and node interprets it using your current timezone. You can retrieve the UTC value from the date object using the getUTC* methods such as get getUTCHours()