postgresql to_char(date, 'day') get different result when using nodejs - node.js

UPDATE 2
Using the suggestions from #Bergi, I added a db execution helper to set the timezone before executing the query. This allows me to control the timezone based on the logged in user's preferences.
async function execWithTZ(sql, params, timezone) {
const client = await this.pool.connect();
await client.query<any>(`SET TIMEZONE TO '${timezone}'`, []);
return client.query<any>(sql, params);
}
const result = await execWithTZ('SELECT ...', [], 'Australia/Sydney');
QUESTION
I am trying to get Postgresql to return the day of the week of a specific date in a view. I later use this to display values by day.
The data is in a table with a column named updated which is a timestamptz field.
When running the following query using DBeaver SQL client against a PostgreSQL 12 database I get the values as I expect, with records updated on Thursday local time (just after midnight) showing up as Thursday
SELECT
count(v.id) AS count,
btrim(to_char(v.updated, 'DAY'::text)) AS day,
date_trunc('DAY'::text, v.updated) AS "updatedDay"
FROM table v
GROUP BY (to_char(v.updated, 'DAY'::text)), (date_trunc('DAY'::text, v.updated))
I put this in a view and when I query this view using node-postgres the counts are for the previous day (Wednesday), which I presume is because the database thinks it should interpret the dates in the UTC timezone.
To further prove the point, when I change the above query to use DAYTZ instead of just DAY in DBeaver I get THURSDAYAEST but in nodejs as result I get WEDNESDAYUTC
Changing all my dates to be without timezone and forcing everything to UTC on the way in is not an option for me.
How can I make node-postgres tell the database what timezone I want these dates interpreted as so that I can get the correct day?
UPDATE 1
I managed to get PostgreSQL to return the correct values to postgres node by setting the database user's timezone.
ALTER ROLE visuo_ingest SET TIMEZONE TO 'Australia/Sydney';
Now the counts for things that happened on Thursday Sydney time is counted for Thursday and not Wednesday.
Still interested in a way to do this on the connection rather than the database user level.

Still interested in a way to do this on the connection rather than the database user level.
You already found the right setting, there's no reason to alter it on a role only. You can also change the setting in a client session by running the SET command
SET TIMEZONE TO 'Australia/Sydney';
Just put that in a pgClient.query("…"); right after connecting the client.

Related

Date-time recorded by mongo-db is different from clientPC (server location is in same region)

This is about a different date-time recorded in mongoDb via Node.js.
e.g. the date time recorded is 2022-02-01T13:32:01.967Z,
the actual time is 19:02 for the same date.
Looks like it has recorded GMT. This code will be eventually run on a VPS based in my country. However, at the moment I am running the code on localhost.
below is the code in Node.js
/// part of a controller method
const xDate = new Date();
console.log('new Date()', xDate);
// output: 2022-02-01T13:32:01.967Z, actual time 19:02 on same date
candidate.history.push({
actionDate: xDate,
userId: user.acctId,
userName: user.userName,
remarks: 'bla bla bla'
})
await candidate.save();
Any idea how to correct it? In Angular we can set the input field to datetime-local type to save the date/time without timezone. Can we do something similar in Node.js /mongodb? How do I get the correct time stored in mongo?
I have read that MongoDB always stores the timestamp in UTC. Is there any way this can be handled application-wide to handle all date values recd from Mongo in both the below cases:
at Node.js levelfor performing some date calculations based on date value recd from Angular?
At Angular level for date values recd from Node(as it is from Mongo)

Mongodb and Node.js datetime timezone issue

I am having hard time to understand the concept of querying date in mongodb/node.js, even after going through many SO and other articles in google.
While storing the date from a react datepicker  in mongodb it gets stored a day less as per my time zone but when it is fetched it is shown the same,I can understand that it is again being converted into locale
but how do query responds?
When I query  mongodb from node.js/mongoose - I don't find the correct resultSo my question is while querying the mongodb - how dates are passed to mongodb ?
for example :
post.find({publish_date:{ $gte:new Date()}}
What is the new Date value - localtime zone of the server? or  browser?
is the new date value converted to utc while comparing the database or compared as passed from the server ?
I have a post model where I want the post to be published on the published_date as per my timezone.
for example - if I put 2021-01-21 mongo stores 2021-01-20T18:30:00.000+00:00 - it is understood that it stored as UTC
Now when I am querying the post to be published when the server date/time is 2021-01-21 - it is not fetch any document!
When I change the publish_date as 2021-01-22 mongo stores 2021-01-21T18:30:00.000+00:00 then query produces on document.
I am not sure how to solve this problem
Any help in this regard would be helpful
Thanks.
In MongoDB all Date values are stored as UTC-Times only.
Your client application is responsible to display these UTC times as local times, if required.
MongoDB does not preserve the input time zone, if you need this for any reason then you have to store it in a separate field.
In Mongo you can display locals date/times values with $dateToString:
{ $dateToString: {
date: <dateExpression>,
format: <formatString>,
timezone: <tzExpression>,
onNull: <expression>
} }
Whenever one has to work with date/time values then I recommend the moment.js library.
The query could be this for example:
{ $gte: moment().startOf('day').toDate() }
I am in Switzerland where we have UTC+01:00, thus moment().startOf('day').toDate() returns ISODate("2021-01-20T23:00:00Z")

Remove timezone completely from SQL Server

EDIT: Some users noted that this is a duplicate of ADO.NET question at
How can I get DateTime data from SQL Server ignoring time zone issues?
Please DO READ my tags, and my codes. This is nodejs question,
that has nothing to do with ADO library whatsoever. It doesn't
even run on Windows. The option noted at the link provided doesn't even exists at mssql-npm
I am creating a database for company attendance, which spans for 3 timezones.
The problem with SQL Server is that I read the working hour differently from each timezone. It registered like 1969-12-31T23:00:00.000Z, and registers as 06:00 for one timeline, and 08:00 for another. I need it to be exactly at 08:00 regardless of timezone. It means, 08:00 at western part of my country, and also 08:00 at eastern part of my country.
When I try to disable UseUTC, nodejs translated my timezone twice. For example, this is when I sent masuk field as 08:00
Sent to server:
Query sent to SQL Server:
UPDATE shift SET name = 'Shift 1', masuk = '2019-10-28T01:00:00.000Z', keluar = '2019-10-28T10:00:00.000Z', tolerance = 15 WHERE id = 1
Received the result back
And shown to the user as:
My country sits at GMT+7 to GMT+9. I don't mind if I have to flatten the timezone. But how to do that? Is there a way to read the time completely ignoring the time zone? It makes my work unnecesarily complicated.
I am using:
SQL Server 2012
Material UI
date-fns
Nodejs 12
mssql module
and this is my code to get the time from user input
<MuiPickersUtilsProvider utils={DateFnsUtils}>
<KeyboardTimePicker
key={component.field.toString()}
variant="inline"
value={value}
label={component.title}
onChange={this.eventHandler(component.field).bind(this)}
format="HH:mm"
/>
</MuiPickersUtilsProvider>
Thank you for help.
THe database uses time(7) to store the time

Mongoose datetime and timezone, how to get datetime based on server timezone

The first answer of this question suggests that mongoose would adapt the date according to server timezone when retrieving data.
However, I don't have this comportement.
I set the (node) server timezone with :
process.env.TZ='Europe/Paris'
For exemple if I create a simple model like :
const mongoose = require("mongoose");
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
exports.Comment = mongoose.default.model('TestSchema', testSchema);
But if I create a date with 2020-01-01 20:20:20, when doing TestSchema.find() the date will be: 2020-01-01T19:20:20.000Z so there are two things that I don't understand :
Europe/Paris is actually UTC +2, so I would expect the date to be either 2020-01-01T18:20:20.000Z in UTC or 2020-01-01T20:20:20.000Z with the server timezone
How to have mongoose automatically set the date to the correct timezone?
I know that myDate is a Date object, so I can convert it manually but I'd rather not have to do it myself for simple reasons like forgetting to convert one of the dates in the application or not having to do it every time a Date field is added
An easy solution that I can think of would be to register a global plugin for mongoose which would use schema.set('toJSON', ... and schema.set('toObject', ...) with the transform method so I can loop through schema fields and if the field is a Date, update it to my timezone.
But I see two problems with this approch :
It doesn't sound very good performance-wise if I am querying a lot of documents each with a lot of fields
As you can see here I am currently not able to register global plugins...
What would be the best method to get the date in the server timezone format? I would rather still store them in UTC but set the hour according to the server timezone.
EDIT :
I just saw that while console.log(myDate) outputs 2018-01-01T19:20:20.000Z console.log(myDate.toString() outputs Mon Jan 01 2018 20:20:20 GMT+0100 (Central European Standard Time) so it seems likes this could be used, even tho I'd rather still have a Date object and converting it to string just before sending it to the client (would need some formatting tho since this format is not very user friendly). But then again, how would I do this globally and not for every date
A few things:
Europe/Paris at 2020-01-01T20:20:20 is UTC+1. It doesn't switch to UTC+2 until Summer Time kicks in on March 29th. Reference here. Thus the conversion to 2020-01-01T19:20:20Z is correct.
The output of console.log when passed a Date object is implementation specific. Some implementations will emit the output of .toString() (which is in local time in RFC 2822 format), and some will emit the output of .toISOString() (which is in UTC in ISO 8601 extended format). That is why you see the difference.
In general, it is not good to send a local time without also sending a time zone offset. ISO 8601 format is ideal, but you should send either 2020-01-01T19:20:20Z, or 2020-01-01T20:20:20+01:00. Don't just send the date and time without an offset to the client. Otherwise, if your client could be in a different time zone then they would interpret the value incorrectly.
Keep in mind that Date objects are not time zone aware. They contain only a Unix timestamp internally, and they convert only to the system's local time zone for the functions that work in local time. They cannot work in any other time zone.
Relying on the system local time zone is bad for portability. One doesn't always have the ability to change it, and it doesn't do well when you have to work in multiple time zones. It would be better to not rely on setting a local time zone from Node's TZ variable. Instead, consider writing your code to be independent of any local time zone setting.
A time zone aware date library can help with most of your concerns. I can recommend Luxon, js-Joda, Moment + Moment-Timezone, or date-fns + date-fns-timezone.
"how would I do this globally" is something I'm not following in your question. Try the approach I described, and if you still have issues then open a new question. Try to be specific and ask a single question. You're likely to get better results that way. Please read How do I ask a good question? and How to create a Minimal, Complete, and Verifiable example. Thanks.
To solve the issue:
npm i mongoose-timezone
In your schema file:
import timeZone from "mongoose-timezone";
const testSchema = new mongoose.Schema({
myDate: { type: Date, required: true },
}, { timestamps: true });
// mongoose will save the dates based on user's timezone
testSchema.plugin(timeZone)
mongoose-timezone basically adds the current timezone offset to the
date before store and removes the offset when retrieving data. This
way dates are kept proportional in the database and in the app.

Sequelize with Postgresql truncating result unexpectedly

I'm using a raw sequelize query in node to execute a postgresql function which returns a query. The problem I'm having is that the result set obtained from running the query in pgAdmin does not match the result set returned in node when using the .query() method, even though the query in pgAdmin and the query in node are both exactly the same.
here's the query I'm performing:
SELECT * FROM foo('2015-07-01','2015-07-31');
This query returns 91875 rows in pgAdmin, however, in node using sequelize, running the following only returns 87500 rows:
models.sequelize.query("SELECT * FROM foo('2015-07-01','2015-07-31');")
.spread(function (results, metadata) {
console.log(results.length);
});
output: 87500
The rows which are excluded correspond to the last date 2015-07-31.
I found that I can obtain all the rows in node by increasing the 2nd date parameter to go beyond the end date. For example, if my goal is to get results which include 2015-07-31, setting the end date to be 2015-08-03 works. Although, this is masking the problem and not ideal.
I thought there must be something wrong with the WHERE clause in my stored procedure function. However, if that was the case, why do I get back the desired number of rows when I run the query though pgAdmin?
The reason for the difference in results had to do with timezone. I had sequelize using UTC but my DB was using "US/Eastern".
After I set the DB to use UTC using the command, "SET time zone utc" I was able to replicate the results that sequelize was seeing.

Resources