Node's Postgres module pg returns wrong date - node.js

I have declared a date column in Postgres as date.
When I write the value with node's pg module, the Postgres Tool pgAdmin displays it correctly.
When I read the value back using pg, Instead of plain date, a date-time string comes with wrong day.
e.g.:
Date inserted: 1975-05-11
Date displayed by pgAdmin: 1975-05-11
Date returned by node's pg: 1975-05-10T23:00:00.000Z
Can I prevent node's pg to appy time-zone to date-only data? It is intended for day of birth and ihmo time-zone has no relevance here.

EDIT Issue response from Developer on github
The node-postgres team decided long ago to convert dates and datetimes
without timezones to local time when pulling them out. This is consistent
with some documentation we've dug up in the past. If you root around
through old issues here you'll find the discussions.
The good news is its trivially easy to over-ride this behavior and return
dates however you see fit.
There's documentation on how to do this here:
https://github.com/brianc/node-pg-types
There's probably even a module somewhere that will convert dates from
postgres into whatever timezone you want (utc I'm guessing). And if
there's not...that's a good opportunity to write one & share with everyone!
Original message
Looks like this is an issue in pg-module.
I'm a beginner in JS and node, so this is only my interpretation.
When dates (without time-part) are parsed, local time is assumed.
pg\node_modules\pg-types\lib\textParsers.js
if(!match) {
dateMatcher = /^(\d{1,})-(\d{2})-(\d{2})$/;
match = dateMatcher.test(isoDate);
if(!match) {
return null;
} else {
//it is a date in YYYY-MM-DD format
//add time portion to force js to parse as local time
return new Date(isoDate + ' 00:00:00');
But when the JS date object is converted back to a string getTimezoneOffset is applied.
pg\lib\utils.js s. function dateToString(date)

Another option is change the data type of the column:
You can do this by running the command:
ALTER TABLE table_name
ALTER COLUMN column_name_1 [SET DATA] TYPE new_data_type,
ALTER COLUMN column_name_2 [SET DATA] TYPE new_data_type,
...;
as descripted here.
I'd the same issue, I changed to text.

Just override node-postgres parser for the type date (1082) and return the value without parsing it:
import pg from pg
pg.types.setTypeParser(1082, value => value)

Related

Issue of Reading and Converting Date of Google Calendar Events when language is set to Arabic in C#

I am Using Google APis .net client library to read calendar events.
I have following line of code
newRow["Start"] = pEventItem.Start.DateTime.HasValue ?
Convert.ToDateTime(pEventItem.Start.DateTime) : Convert.ToDateTime(pEventItem.Start.Date);
Where PEventItem is of type Google.Apis.Calendar.v3.Data.Event and NewRow[...] is of type DataRow. The Value of pEventItem.Start.Date is "2019-06-24" (as seen in debug window)
The above line of code works perfect, But fails when UI language / Culture is set to Arabic (SaudiArabia) The same Convert.ToDateTime throws error "String was not recognized as a valid DateTime".
btw, How i am changing the UI language is as below for your information.
Thread.CurrentThread.CurrentCulture = System.Globalization.CultureInfo.GetCultureInfo(ChangeLanguageTo);
Thread.CurrentThread.CurrentUICulture = System.Globalization.CultureInfo.GetCultureInfo(ChangeLanguageTo);
I tried to set 2nd parameter of the Convert.ToDateTime function in an hope that it will convert date correctly...
CultureInfo enUsCulture = new CultureInfo("en-us");
newRow["Start"] = pEventItem.Start.DateTime.HasValue ? Convert.ToDateTime(pEventItem.Start.DateTime, enUsCulture) : Convert.ToDateTime(pEventItem.Start.Date, enUsCulture);
Well Now it does not throw exception, but the returned date is incorrect. value retuned is {21/10/40 12:00:00 ص}
while The actual date pEventItem.Start.Date is "2019-06-24"
I also tried invariant culture also, but result is same, converted date is wrong. What could be the issue?
Regards
There are a few things going on here.
Firstly, if you use EventDateTime.DateTime (e.g. via pEventItem.Start.DateTime) you don't need to call Convert.ToDateTime, because that's already a DateTime?... you can just take the Value property to get a DateTime from a DateTime?. However, I should warn that that can perform time zone conversions that you may not want. (We can't fix the library to avoid those, as it would be a breaking change.) Instead, you may want to parse EventDateTime.DateTimeRaw, which is the original string value returned by the API.
When parsing, I'd suggest using the invariant culture using CultureInfo.InvariantCulture (instead of creating an en-US culture), and parse using DateTime.ParseExact, specifying the format you expect based on whether you're parsing a date or a full date/time.
In terms of "the returned date is incorrect" - I believe that's really just the formatted value that's using the default culture, including the calendar system. We can see that at play in the code below, which constructs the DateTime directly (so can't be affected by any text parsing etc). When formatted using the invariant culture, it shows as 2019-06-24, but when formatted with ar-SA, it shows as "1440-10-21" due to the default calendar system for that culture being System.Globalization.UmAlQuraCalendar:
// Note: when a calendar system is not specified,
// it's implicitly Gregorian.
DateTime date = new DateTime(2019, 6, 24);
Console.WriteLine(date.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture));
Console.WriteLine(date.ToString("yyyy-MM-dd", new CultureInfo("ar-SA")));
So what you're seeing in the debugger is the correct date - but formatted in a way you weren't expecting.

Parsing postgres date in logstash using datefilter plugin

I am using logstashto read the postgres data using jdbc input plugin and push to elastic
The data is coming properly and things seems to be working fine just for a small problem i.e.
My logs table has a field requesttimestamp with datatype of timestamp. Since there is a historical data and also to ensure that the timelines are based on the data and not the time of run, I am trying to set the value of #timestamp with requettimestamp.
the filter configuration is as follows:
{
match => ["requesttimestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
}
but while running it is tagging it as _dateparsefailure and using the system time as #timestamp
I also tried using the following format : ISO8601 but nothing seems to be working.
I am sure it is something simple but not able to put the finger to it.

Node.js - Oracle DB and fetchAsString format

I am stuck on a problem and I am not sure what is the best way to solve it. I have a date column that I want to select and I want to fetch it as a string. Which is great, node-oracledb module has this option with fetchAsString mehotd. But it fetches the date like this for example 10-JAN-16 and I want to fetch it like this 10-01-2016. Is there a way to do that from the node-oracledb module, or I should modify the date after I get the result from the query?
UPDATE: I mean solution without to_char in the query and without query modifications
Check out this section of my series on Working with Dates in JavaScript, JSON, and Oracle Database:
https://dzone.com/articles/working-with-dates-using-the-nodejs-driver
The logon trigger shows an example of using alter session to set the default date format. Keep in mind that there is NLS_DATE_FORMAT, NLS_TIMESTAMP_FORMAT, NLS_TIMESTAMP_TZ_FORMAT.
I only show NLS_TIMESTAMP_TZ_FORMAT because I convert to that type in the examples that follow as I need to do some time zone conversion for the date format I'm using.
Another way to set the NLS parameters is to use environment variables of the same name. Note that this method will not work unless you set the NLS_LANG environment variable as well.

Search formula not working on Linux machine

I have a strange behavior: an agent called via an AJAX request should search documents to display in a calendar. For that reason I compute a search formula and then run the search method of my database in Lotusscript. This is the formula:
form="mholiday" | form="mserviceevent" | (form="mereignis" & co_status!="9") & #texttotime(#text(startdatetime)) >= [29.09.2014] & #texttotime(#text(enddatetime)) =< [10.11.2014]
Everything's fine on Domino on Windows but fails with "formula error" on a Linux machine. Am I missing something?
If I omit the term with the dates everything is fine, so this is the part that causes the error.
Try it with #ToTime() and #Date() instead. That might help to get away from local settings' influence on server:
... & #ToTime(startdatetime) >= #Date(2014; 9; 29) & ...
#ToTime() doesn't convert the field if it's a date time value already.
#Date doesn't depend on local settings whereas [29.09.2014] probably does.
I don't think it's a Linux problem, I think it's a data problem. It sounds like either a date format problem or a problem with the UNK table, used by full text search.
If the first document created on that server that had a field called "startdatetime" had a text value, then any search expects "startdatetime" to be a text value, even if there is another field in the database called startdatetime that is a date or the startdatetime field is subsequently changed to be a date. To confirm this, you can use the search bar and select the field. The operators it offers will confirm if it's expecting a date or a text value. See this answer for details on how to resolve "Query is not understandable" - Full text searching where field types have changed.
Alternatively, it may be a problem with the date format, as Knut says. In which case a test for 9/9/2014 would work but 29/9/2014 wouldn't.

BigQuery / Node.js timestamp way off

Doing a streaming insert into Google BigQuery, from a light Node.js app, using this package: https://www.npmjs.org/package/bigquery
I generated a timestamp on my server via this simple line of code:
jsonData['createdAt'] = new Date().getTime();
I then insert that into BigQuery, into a field with type 'timestamp'. There is no intermediate step (besides the Node package).
But many, although not all, of dates look waaaaaay off. For example:
46343-08-28 05:58:59 UTC
When that should say something like 11:45pm on 05-16-2014. However, some of my createdAt dates are correct, and I can't find a reason for the difference.
Any suggestions?
Without actually debugging the JS code, this seems to be an "off by a thousand" problem.
Check this out:
SELECT USEC_TO_TIMESTAMP(1400341611711851)
2014-05-17 15:46:51 UTC
SELECT USEC_TO_TIMESTAMP(1400341611711851*1000)
46345-01-22 13:01:51 UTC
SELECT MSEC_TO_TIMESTAMP(1400341611711851)
46345-01-22 13:01:51 UTC
SELECT MSEC_TO_TIMESTAMP(1400341611711851/1000)
2014-05-17 15:46:51 UTC
So to get a UNIX timestamp in seconds, divide the new Date().getTime() number by 1000.
I had a requirement where I am supposed to send Timestamp type to BigQuery table from NodeJS.
I did it as follows:
bigqueryClient.timestamp(new Date(Date.now()))
I hope this may help someone.
Use:
bigquery.datetime(new Date().toISOString())

Resources