I'm working with a data set in Athena where all the timestamps are UTC, but I need to adjust to British Summer Time, i.e. add an hour for all timestamps between 1:00am on the last Sunday in March and 1:00am on the last Sunday in October. Any help would be much appreciated. Thanks!
List of supported time zones does not contain BST but has Europe/London which AFAIK should match it. To convert time zone use AT TIME ZONE:
-- sample data
WITH dataset (time) AS (
VALUES (timestamp '2022-02-01 10:00:00'),
(now())
)
-- query
select time, time AT TIME ZONE 'Europe/London'
from dataset
Output:
time
_col1
2022-02-01 10:00:00.000 UTC
2022-02-01 10:00:00.000 Europe/London
2022-04-13 12:13:06.610 UTC
2022-04-13 13:13:06.610 Europe/London
I try to store availability hours for different services. (Every work days (8, 9 hours), 24/7 etc.)
A work day can have different office times (08:00-16:00 or 09:00-17:00, 08:00-18:00 etc.)
There are also national holidays where a normal work day acts like a weekend, and any given weekend day can be also work day.
What is the best way to store this in a database (PostgreSQL)?
Background
The main goal is to calculate if how many minutes of a given time range is in service time.
A) A service time is only work days (08:00-16:00) and the date range is
[2019-10-21 15:00 (Monday) - 2019-10-22 09:00 (Tuesday)]. The whole range is 18 hours long, but only 2 hours were in service time.
B) I also want to calculate if I know the start time (2019-10-21 15:00) and there is 3 hours to finish the work (in work time) then when will be the end date? (2019-10-22 10:00 - 1 hour on Monday, 2 hours on Tuesday)
I would store the data exactly as how you described it.
You can have a table for work windows, i.e. Mon-Fri 08:00 to 16:00
And each service will have it's date ranges, start and stop. I'm guess a service will have multiple date ranges so you might need a date range table that has dateRangeId, start & stop columns.
When you want to calculate something, you pull data for the service and pull data for the work days and calculate what is the actual service time and any other questions you need answered :)
I am trying to schedule a query to run intraday in Bigquery UI. According to Google's documentation this option uses cron syntax. I have used crontab guru to verify the syntax is correct, although it doesn't matter what syntax you put the scheduler doesn't seem to accept any. Is this a known bug? Below is the cron syntax I'm using to run every 6 hours.
0 */6 * * *
This post attempts to give a more general answer for those that may follow with similar questions.
A full description of the allowed syntax can be found directly here as well as related materials here under "schedule" field information..
For full disclosure, I'm going to lift out some parts directly from that documentation here so this answer can "stand alone".
Generally, the schedule must be of the form:
[TYPE] [INTERVAL_VALUE] [INTERVAL_SCOPE]
and you must decide which of the three different kinds of intervals you will use:
End-time intervals
Start-time intervals
Custom intervals
The kind of interval is chosen implicitly by the schedule you provide.
End-Time Intervals
These are intervals implemented from when a job finishes.
TYPE
Daily intervals must start with the "every" prefix
INTERVAL_VALUE
Valid units of time are as follows:
minutes or mins
hours
INTERVAL_SCOPE
Not applicable for end-time intervals.
Example
If "every 5 mins", and the job finishes at 0201, then then next job waits 5 minutes AFTER completion to begin again, and starts at 0206.
Start-Time Intervals
A strict schedule for the queries to follow.
TYPE
Daily intervals must start with the "every" prefix
INTERVAL_VALUE
Some integer amount of the following units of time:
minutes or mins
hours
The units remain the same even using 1 as the amount.
INTERVAL_SCOPE
Must be of the form:
from [HH:MM] to [HH:MM] with HH=00,01,...,23 and MM=00,01,..., 59.
OR
synchronized
synchronized repeats a time interval and spreads it evenly across the 24 hour period (e.g. like end-time scheduling but fixing it to a start-time).
The [INTERVAL_VALUE] given in conjunction with this option must be a factor of 24 (1, 2, 3, 4, 6, 8, 12, or 24), presumably so that each day has the same schedule (otherwise, you would get a "spill" over into the next day).
Examples
Example 1: every 5 minutes from 10:00 to 14:00
If the job starts at 1000, and takes 6 minutes, then it will run 1000, 1010, 1020, ..., because the 1005,1015, ..., jobs were skipped because they were still running.
Direct quote:
Because the start time of a job is strict, if an instance of a job
runs longer than the defined time interval, then the Cron service can
skip a job. An individual start time in the interval can be skipped if
the prior job has not completed or times out.
Example 2: every 2 hours synchronized
Runs 0000,0200,0400, ..., 2200.
Custom Intervals
These specify intervals on the day or month level, and cannot specify sub-daily intervals.
TYPE
Using every specifies a repeating interval:
every day 06:00
every monday
every tuesday
...
every sunday
Specific days can be specified with ordinal numbers (1st, 2nd, 3rd, OR, first, second, third, ..., up to 31st OR thirtyfirst)
1st,3rd tuesday
2nd, third wednesday of month 09:00
Note that the ordinal number and words can be mixed.
INTERVAL_VALUE
Valid days are any mix of the following:
monday or mon
tuesday or tue
wednesday or wed
thursday or thu
friday or fri
saturday or sat
sunday or sun
day for all days of the week
INTERVAL_SCOPE
Can include
of month [HH:MM]
of jan,feb,sep,nov [HH:MM] i.e. a comma-separated list of months
Note, a time must be given with any given month, with HH and MM given as above (00-23 and 00-59, respectively). If "of" is excluded, the job runs every month.
Allowed values:
january or jan
february or feb
march or mar
april or apr
may
june or jun
july or jul
august or aug
september or sep
october or oct
november or nov
december or dec
month for all months in the year
Examples
2nd monday,thu
1,8,15,22 of month 09:00
1st mon,wednesday,thu of sep,oct,nov 17:00
Note, there is no documentation that could be found describing the time a job runs when the time is not explicitly specified (e.g. 2nd monday,thu).
General Examples
second monday,thu -> "Custom Interval"
third, twentysecond, 30th mon -> "Custom Interval"
1 of jan,april,july,oct 00:00 -> "Custom Interval"
1st monday of sep,oct,nov 09:00 -> "Custom Interval"
1st,third monday of month 04:00 -> "Custom Interval"
1,8,15,22 of month 09:00 -> "Custom Interval"
every monday 09:00 -> "Custom Interval"
every 5 minutes from 10:00 to 14:00 -> "Start-time Interval"
every 1 hours from 08:00 to 16:00 -> "Start-time Interval"
every 2 hours synchronized -> "Start-time Interval"
every 5 minutes -> "End-time Interval"
every 1 hours -> "End-time Interval"
IMPORTANT:
Interval types are chosen implicitly when you enter the schedule
You can't mix and match the options for the different interval types.
All specified times are UTC
As said in the intro above, this information is essentially ripped from the existing documentation, but I felt that was buried away and that this question deserved a "stand alone" reference text.
Form the official documentation:
When selecting Custom, a Cron-like time specification is expected, for
example every 3 hours. The shortest allowed period is fifteen minutes.
See the schedule field under TransferConfig for additional valid API
values.
The expected format is "Cron-like" but not pure Cron. Replace your Cron syntax with every 6 hours and this will work (note that this is UTC time), see example below:
I'm creating using node.js and mongodb an application that stores all temperature values by hour in a day.
Temperature values should be shown in a dashboard according the timezone where the sensor is located.
I created a data model following the recommendation for time series data but I don't know how to deal with timezone because mongodb stores dates in UTC so in my data model the object "hours" has static fields for every hour of the day.
// Temperatures by hour in a day.
{
dateStart: ISODate("2016-08-06T00:00:00.000Z"), // This is the start of the day
timeZone: "Europe/Madrid", // We could store the time zone of the sensor.
hours: { // I'm not sure how to deal with these values. Should be UTC hours too?
0: 20
1: 21,
2: 24,
.
.
.
23: 16
}
}
The question is: how can I deal with timezone?
In my local time the start of a day (2016-08-06T00:00:00) is converted to UTC to 2016-08-05T22:00:00.000+02:00.
A first approach could be:
Sensor time zone is 'Europe/Madrid' (CEST, 02:00 diff from UTC)
Get the start of day using 'Europe/Madrid time zone, convert it to UTC and store it in database. Field startDate will be "2016-08-05T22:00:00.000Z"
In order to store a 25 C temperature for an hour: Get local time ('Europe/Madrid'), for example 18h, then convert to UTC. Result is 16h. So hours.16=25.
In this case we have UTC times in star date but the object of hours is not UTC. It is 'Europe/Madrid and I'm not very convinced about this decision.
Any ideas for improving this design?
Your design should work, however your example in 3. should be
{
...
16: 25
...
}
The hours is the offset respect to dateStart. So when displaying you should add 16 hours to the start date which you can display in the timezone that you want.
I have table with all countries GMT/UTC timezones, I need to see what time is in the rest of the countries when in USA is 11am-3pm
Not on particular date just know the difference in time.
I did my calculation like that I -5 GMT in USA and time is 11am then in Russia for example is +4 GMT.
5+4+11=20pm in Russia when USA is 11am, this works with countries that have + GMT zone but ones that have minus it shows wrong time.
I am working in Excel; please help me with advice on how to do it.
I did it already for the +gmt timezones and yes I have times for cities in big countries too; it was not my question.
How can I find out what time zone is in country with -11gmt when in country with +8gmt is 11am?
Someone know?
E.g. I work with dates like this in Excel. I set type of cell data to date and put
1/1/11 4:30 (+4:30 gmt)
1/1/11 1:00 (+1:00 gmt)
Now I have a date e.g. 1/20/11 11:00 (11 am on imaginary date); all I need to do is
"1/20/11 11:00 AM" - "1/1/11 4:30 AM" = "1/19/00 10:00 AM" at (0 gmt)
10am I don't really care about date in this case just time. I cannot think right now how I gound precise time but it seems somehow work without even putting +8 gmt in there...
Anyway solution should look something like that.
What about countries that have more than one time zone?
11am in the USA....where? West coast (PST) or east coast (EST)?
How do you take into consideration daylight savings time?
There are a lot of things to consider to do time conversions correctly.
I personally wouldn't keep a table with country and hour conversions, but two tables. One with timezones and hours from GMT time. And then another one with city names and timezone mappings. This way instead of converting from USA to Russia, you would be converting from New York to Moscow.
I did do a quick search on timezones in excel, and I found this article. I hope that it helps.