I just discovered, that storing dates in utc is not ideally correct if we are also dealing with dates in the future. It seems to be the case because, timezones seem to change more often than we think they do. Fortunately, we seem to have the IANA tzdb that seems to get updated periodically, but, confusingly, postgres seems to use a specific version of the db which it seems to have at build time..
So, my question is, if the timezones are changing, with daylight saving going on, with political, geographical adjustments happening, and our database is not with the latest of the tzdb, how would we be able to keep track of the accuracy of the dates in the system? Additionally, would libraries like date-fns-tz basically not be accurate to account for new timezone changes?
Ideally I would think a library would make a network call to a central server that would maintain the latest changes, but, it doesn't seem to be the case. How are the latest date/timezone changes usually dealt with?
The IANA time zone database collects the global knowledge about what time zone was in effect at what time in every part of the world. That information is naturally incomplete, specifically when it comes to the future. A (IANA) time zone is not an offset from UTC, but a rule that says when which offset from UTC is active. EST is not a time zone in that sense, it is an abbreviation for a certain UTC offset. If you live in New York, you will sometimes have EST, sometimes EDT, depending on the rules for the time zone America/New_York. Of course you should update the time zone database, but not because the timestamps change (they are immutable), but because the way that the timestamps are displayed in a certain time zone can change.
What is stored in the database is always an UTC timestamp, so the timestamp itself is immutable. What changes is the representation. So if you predict that the world will end next July 15 at noon Austrian time, and the Austrian government abolishes daylight savings time, your prediction will be an hour off (unless you expect the cataclysm to follow Austrian legislation). If you are worried about that, make your predictions in UTC or at least add the UTC offset to the timestamp.
If you store the timestamp with time zone in the database, and you query it today with timezone set to Europe/Vienna, you will get a certain result. If you update the time zone database, and the new legislation is reflected in the update, then the same query will return a different result tomorrow. However, it will still be the same timestamp, only the UTC offset in use will be different:
SELECT TIMESTAMP WITH TIME ZONE '2023-07-15 12:00:00+02'
= TIMESTAMP WITH TIME ZONE '2023-07-15 11:00:00+01';
?column?
══════════
t
(1 row)
To clarify #Laurenz's statement in the comments further with an example, lets take an extreme case of samoa , where they switched from GMT-11 timezone, to GMT+13 skipping an entire day.
While ignoring what a timezone actually is (different similar opinions in the comments), for the purpose of the calculations below, lets just consider it a value offset from the standard UTC. Also, do note, I use my own symbolic ways to calculate, but, it is very understandable, hopefully ;-)
so, samoa on Dec 29, 2011 skipped a day, how? Based on what I found, when the clock struck midnight they effectively skipped Friday. But, the unix timestamp
remains equivalent/unchanged:
GMT-11
(-)GMT+13
__________
= 24hrs
Let, WST=GMT-11
2011-12-29 T 24:00:00 - 11 (clock strikes midnight)
= 2011-12-30 T 00:00:00 - 11 (WST)
= 2011-12-30 T 11:00:00 (UTC)
now the switch occurs, WST=GMT+13
2011-12-31 T 00:00:00 + 13 (WST)
= 2011-12-31 T-13:00:00 (UTC)
= 2011-12-30 T 11:00:00 (UTC)
So, as far as I can see, storing future dates does not really affect the value of the date itself. But, what it does affect is the way the dates are displayed, e.g. if the timezone info was not updated, people would still see the day after the 29th at samoa as Friday, 30th. But, in that case, it would be Fri, 30th GMT-11, whereas if the information was updated, it would be Sat, 31, GMT+13. So, all is well.
more details in the comment section of #Laurenz's answer
Also, as #Adrian mentions above, softwares that deal with timezones, come packaged with a version of tzdb if they support the conversion at all. It seems to be the case in postgres as well though it seem you can configure it to use the system's version. For such cases, you gotta update the software or the system's db itself.
I understand that you want to store a future point in time, like "10:00am on July 5th 2078 in the time zone of Australia/Sydney", regardless of what offset that time zone has compared to UTC when you retrieve the point in time again. And when the time comes, the point in time might not even exist, because it is being skipped for the introduction of daylight saving time (or it might exist more than once).
Speaking XML Schema, the information you want to store consists of
a dateTime without timezoneOffset, in the given example 2078-07-05T10:00:00 (no trailing Z)
plus a time zone, given as a string from the IANA database, in the given example Australia/Sydney.
I don't know how this is best stored in a PostgreSQL database, whether as two separate strings, or in a special data type. The PostgreSQL documentation says:
All timezone-aware dates and times are stored internally in UTC. They are converted to local time in the zone specified by the TimeZone configuration parameter before being displayed to the client.
That sounds to me as if the UTC value was fixed, and the local time value in a given time zone might change if daylight saving time is introduced or abolished in that time zone. (Am I correct here?) You want it the other way round: The local time remains the same and the UTC value might change after DST introduction/abolition.
For example, assume that polling stations for the next general election open at 2025-09-21T08:00:00+02:00 in my time zone. But if my country abolishes DST before then, they will open instead on 2025-09-21T08:00:00+01:00 without an explicit rescheduling. In other words: The UTC time changes, but the local time does not.
Or consider a flight whose local departure time and time zone are stored, which has a duration of 10 hours and arrives in another time zone. Its local arrival time then changes when the offset of the departure time zone changes, for example, because daylight saving time is introduced or abolished in that country on day X, but the offset of the arrival time zone does not change. An app that computes the local arrival time must then show a changed arrival time when it is executed on day X or later, although the stored data (the local departure time, departure time zone, arrival time zone and flight duration) have not changed. The required change can happen automatically if the app uses a library that is based on the IANA time zone database and receives an upgrade that includes the DST introduction/abolition before day X arrives.
For an example of such a library, see https://day.js.org/docs/en/timezone/parsing-in-zone.
From the Google Cloud Firestore documentation:
https://cloud.google.com/nodejs/docs/reference/firestore/0.15.x/Timestamp#toDate
Timestamp
CLASS
A Timestamp represents a point in time independent of
any time zone or calendar, represented as seconds and fractions of
seconds at nanosecond resolution in UTC Epoch time. It is encoded
using the Proleptic Gregorian Calendar which extends the Gregorian
calendar backwards to year one. It is encoded assuming all minutes are
60 seconds long, i.e. leap seconds are "smeared" so that no leap
second table is needed for interpretation. Range is from
0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z.
Bold text is my emphasis
What exactly does it mean by leap seconds are "smeared"?
In practice, day to day, let's say storing a created Timestamp in Firestore, and using it to order records whilst querying,
let querySnap = await colRef.orderBy('created', 'asc').limit(10).get();
do I need to consider it?
Read Google's documentation about time smearing:
Since 2008, instead of applying leap seconds to our servers using
clock steps, we have "smeared" the extra second across the hours
before and after each leap. The leap smear applies to all Google
services, including all our APIs.
You and your users are highly unlikely to notice this effect, and it removes the need to write special code to handle sudden shifts in time that would normally be required to account for a leap second.
I need to convert the Node.js datetime '2016-07-13T07:38:15.500Z' to oracle format, where as the .500z represents the timezone.
Im using oracle 11g
The .500z doesn't represent the time zone. The .500 is fractional seconds. The z is short for Zulu, which means the time zone has been normalized to GMT/UTC.
In Oracle Database, the DATE data type doesn't support fractional seconds or time zones. For fractional seconds you'd need to use any of the 3 TIMESTAMP data types. If you need to store the actual time zone, use TIMESTAMP WITH TIME ZONE. If you don't need the actual time zone (most people don't) and want to make converting time zones really easy, then use TIMESTAMP WITH LOCAL TIME ZONE.
You haven't provided enough information to offer much of an answer, but here's a simple example that may sufficient:
select to_date('2016-07-13T07:38:15', 'YYYY-MM-DD"T"HH24:MI:SS'),
to_timestamp('2016-07-13T07:38:15.500Z', 'YYYY-MM-DD"T"HH24:MI:SS.FF"Z"')
from dual;
I have a strange thing occurring and I hope someone can point out what i am missing.
In MongoDB I have a field DT that is of Type Date
An example of what the date looks like in MongoDB is 2014-10-01 10:28:04.329-04:00
When I query MongoDB from Node.js using MongoClient, Node.js is returning this:
2014-10-01T14:28:04.329Z
As i understand it the driver is suppose to convert UTC to local time. In my case it should be Eastern Time (EDT). Why would Node be adding 4 hours instead?
I am loading the date into MongoDB from Java using the Java driver. The variable is set using
new Date();
Node isn't adding 4 hours. Both show exactly the same instant.
2014-10-01 10:28:04.329-04:00
is exactly the same as
2014-10-01T14:28:04.329Z
only one is in a EDT timezone which has -04:00 offset to UTC (so it's four hours earlier there), and the other is in UTC.
Probably you have your server configured in EDT and your client is set to UTC or the other way around.
Unless you need the exact same strings, I wouldn't worry about it.
Or, even better, set both the client and server machine to the same timezone, preferably UTC.
I am trying to store dates as latest modification timestamp in a ZIP -file. It seems that ZIP format support only dates after 1980-01-01 as a last modification time (at least via Java API java.util.zip.ZipEntry )
Is this correct? Is the earliest supported modification timestamp really 1980-01-01 00:00:00? I tried to find some references to verify this but I couldn't find any.
Zip entry timestamps are recorded only
to two 2 second precision. This
reflects the accuracy of DOS
timestamps in use when PKZIP was
created. That number recorded in the
Zip will be the timestamp truncated,
not the nearest 2 seconds.
When you archive and restore a file,
it will no longer have a timestamp
precisely matching the original. This
is above and beyond he similar problem
with Java using 1 millisecond
precision and Microsoft Windows using
100 nanosecond increments. PKZIP
format derives from MS DOS days and
hence uses only 16 bits for time and
16 bits for date. There is defined an
extended time stamp in the revised
PKZIP format, but Java does not use
it.
Inside zip files, dates and times are
stored in local time in 16 bits, not
UTC as is conventional, using an
ancient MS DOS format. Bit 0 is the
least signifiant bit. The format is
little-endian. There was not room in
16 bit to accurately represent time
even to the second, so the seconds
field contains the seconds divided by
two, giving accuracy only to the even
second.
This means the apparent time of files
inside a zip will suddenly differ by
an hour compared with their
uncompressed counterparts every time
you have a daylight saving change. It
also means that the a zip utility will
extract a different UTC time from a
Zip member date depending on which
timezone the calculation was done.
This is ridiculous. PKZIP format needs
a modern UTC-based timestamp to avoid
these anomalies.
To make matters worse, Standard tools
like WinZip or PKZIP will always round
the time up to the next even second
when they restore, thereby possibly
making the file one second to two
seconds younger. The JDK (i.e.
javaToDosTime in ZipEntry rounds the
time down, thereby making the file one
to two seconds older.
The format does not support dates
prior to 1980-01-01 0:00 UTC. Avoid
file dates 1980-01-01 or earlier
(local or UTC time).
Wait! It gets even worse. Phil Katz,
when he documented the Zip format, did
not bother to specify whether the
local time used in the archive should
be daylight or standard time.
And to cap it off… Info-ZIP, JSE and
TrueZIP apply the DST schedule (days
where DST began and ended in any given
year) for any date when converting
times between system time and DOS
date/time. This is as it should be.
Vista’s Explorer, 7-Zip and WinZip
apply only the DST savings, but do not
apply the schedule. So they use the
current DST savings for any date when
converting times between system time
and DOS date/time. This is just
sloppy.
http://mindprod.com/jgloss/zip.html
tar files are so much better.