I have rankings by country and month, I need to reset the ranking at the beginning of the month 1st day at 00:00.
I have a cron task every 15 minutes checking if with the timezone of the country is day 1 of the month at 0:00 (because some timezones have deviations +x.45 or +x.30 minutes). But, what is the criteria when the country has different timezones?
By example, when is the start time of BlackFriday in one country like USA with different timezones?.
Welcome to the wonderful, horrible world of time! ;-)
In the USA we don't have a unique start time of "Black Friday". We don't even all celebrate New Years at the same moment. Instead we follow our local time zone rules. Not only are there multiple time zone rules covering the single country, even some individual states observe more than one time zone (e.g. Tennessee).
Here is a global map of time zones.
By example, when is the start time of BlackFriday in one country like USA with different timezones?.
Assuming you mean the shopping holiday that follows Thanksgiving Day, which falls this year on Friday, November 23rd 2018:
Time Zone | Local Start Time | UTC Equivalent
----------------------------+------------------------------|------------------------
Guam & Northern Mariana Is. | 2018-11-23T00:00:00+10:00 | 2018-11-22T14:00:00Z
Atlantic (PR, VI) | 2018-11-23T00:00:00-04:00 | 2018-11-23T04:00:00Z
Eastern | 2018-11-23T00:00:00-05:00 | 2018-11-23T05:00:00Z
Central | 2018-11-23T00:00:00-06:00 | 2018-11-23T06:00:00Z
Mountain | 2018-11-23T00:00:00-07:00 | 2018-11-23T07:00:00Z
Pacific | 2018-11-23T00:00:00-08:00 | 2018-11-23T08:00:00Z
Alaska | 2018-11-23T00:00:00-09:00 | 2018-11-23T09:00:00Z
Hawaii | 2018-11-23T00:00:00-10:00 | 2018-11-23T10:00:00Z
American Samoa | 2018-11-23T00:00:00-11:00 | 2018-11-23T11:00:00Z
I have no idea if Black Friday is actually observed in all US territories or not.
As you can see, a single calendar date cannot be mapped to a single range of absolute time, especially when applied to a country with multiple time zones.
BTW, things are even more complicated if you were talking about a date that fell during daylight saving time. For example, much of Arizona doesn't observe DST while the rest of Mountain Time does.
Related
I'm trying to understand the best practices around storing aggregated time series based data.
For instance if I am building a weather service application that's ingesting lots of weather metrics from sensors around the world and storing that weather data in the form of the weather for today, the week, for the month, what's a good way to model that?
Would the day level, week level, and month level each have their own column family?
Then there's the factor of location. Each location would have it's own weather data, so would partitioning by say some zipcode or geohash for a specific area make sense?
The access patterns would be querying for the daily or weekly or monthly weather in a city.
let's say ever 5 minutes. Would that have an impact on the design?
Yes. So sensor updates every 5 minutes happen at 12x per hour or 288x per day.
The access patterns would be querying for the daily or weekly or monthly weather in a city.
That also makes for 2016x per week and 8640x per month (30 days). The reason this is important, is because Cassandra has hard limits of storing 2GB and 2 billion cells per partition. This means that storing time series data by city only, would eventually hit this limit (although things would likely grind to a halt long before that).
But the general idea is that you want to model your tables around:
How you're going to query your data.
Avoiding unlimited partition growth.
So if we're just talking about temperatures and maybe a few other data points (precipitation, etc), partitioning by month and city should work just fine.
CREATE TABLE weather_sensor_data (
city TEXT,
month INT,
temp FLOAT,
recorded_time TIMESTAMP,
PRIMARY KEY ((city,month),recorded_time))
WITH CLUSTERING ORDER BY (recorded_time DESC);
Now, I could query for weather sensor data since 8AM, like this:
> SELECT * FROM weather_sensor_data
WHERE city='Minneapolis, MN'
AND month=202111
AND recorded_time > '2021-11-01 08:00';
city | month | recorded_time | temp
-----------------+--------+---------------------------------+------
Minneapolis, MN | 202111 | 2021-11-01 08:35:00.000000+0000 | 3
Minneapolis, MN | 202111 | 2021-11-01 08:30:00.000000+0000 | 3
Minneapolis, MN | 202111 | 2021-11-01 08:25:00.000000+0000 | 2
Minneapolis, MN | 202111 | 2021-11-01 08:20:00.000000+0000 | 2
Minneapolis, MN | 202111 | 2021-11-01 08:15:00.000000+0000 | 2
(5 rows)
This should help you get started.
#dipen, you could also refer to this documentation where it walks developers through various data models by their use case. #AlexOtt has great questions to begin with the data models for your use case and #aaron has a great example demonstration.
Here is an example. You could very much customize it for your weather use case. For a given access pattern requirement like in the below example,
we would go ahead and design a Cassandra table as follows to answer them,
I'm trying to calculate the time between two dates BUT also whilst taking into account a working hour constraint. I am trying to do this using Excel formulas.
Example follows below, where the working hour constraint would be between 08:00 -> 18:00. I.e only time which follows during these hours should be taken into account.
+---------------+-------------+----------------------+----------------------+
| Start | End | Total Difference | Within Working Hours |
+---------------+-------------+----------------------+----------------------+
| 1/1/19 20:20 | 1/2/19 9:30 | 13 hours, 10 minutes | 1 hour, 30 minutes |
+---------------+-------------+----------------------+----------------------+
The question is how do I calculate the Within Working Hours column?
Thanks!
Use this that counts the days and multiplies that by the 10 hour working days and then adds for hours worked in the window on the work days:
=IF(DATEDIF(A2,B2,"d")>1,NETWORKDAYS.INTL(A2+1,B2-1,1)*10,0)/24+IF(AND(MOD(A2,1)<TIME(18,0,0),WORKDAY.INTL(A2-1,1,1)=INT(A2)),(TIME(18,0,0)-MOD(A2,1)),0)+IF(AND(MOD(B2,1)>TIME(8,0,0),WORKDAY.INTL(B2-1,1,1)=INT(B2)),(MOD(B2,1)-TIME(8,0,0)),0)
Right now we are only excluding weekends, but with NETWORKDAYS.INTL and WORKDAY.INTL one can include a range that will exclude holidays listed.
Then format the cell with a custom number format:
[h] "hour, " mm "minutes"
I have to calculate the time until a machine is available again.
The work time is daily from 06:00am until 22:00pm.
If I simply add the time with the hours, the result looks like this.
| Job Duration (hours) | Start | End |
| 18,75 | 21.09.2017 06:00 | 22.09.2017 06:15 |
| 20,14 | 21.09.2017 11:30 | 22.09.2017 07:38 |
This is wrong for me, because the work time is only from 06:00am to 22:00pm.
I would like to achieve the following result:
| Job Duration (hours) | Start | End |
| 18,75 | 21.09.2017 06:00 | 22.09.2017 07:45 |
| 20,14 | 21.09.2017 11:30 | 22.09.2017 15:38 |
Thanks in advance.
Have a look at this example
To get your End time I calculate the actual time the job takes i.e non-working and working hours.
There's 16 hours in the working day (between 6 and 22) and there's 8 non-working hours (between 22 and 6).
=8*(INT(A2/16)+IF(60*((A2-INT(A2/16)*16)+HOUR(B2))+MINUTE(B2)>60*22,1,0))+A2
To break this formula down this:
Calculates the number of non-working periods by doing INT(A2/16)*8 This divides the number of working hours into the job duration
I Then test if the job duration is going to take longer then a day using IF(60*((A6-INT(A6/16)*16)+HOUR(B6))+MINUTE(B6)>60*22 This calculates whether the job would go past 10PM. I compare the two in minutes instead of hours for accuracy. If the job would go past 10PM I add on another non-working period
I then multiple the total number of non-working periods by the number of non-working hours (8)
And finally add the total of non-working hours onto the original job duration
All in all this calculates the amount of non-working hours + working hours to give me the actual time
I then add the actual time to the start time using:
=B2+C2/24
to get my end time.
Then finally, I've combined the formulas to give you your answer in one column
=B2+(8*(INT(A2/16)+IF(60*((A2-INT(A2/16)*16)+HOUR(B2))+MINUTE(B2)>60*22,1,0))+A2)/24
Giving your results:
I am fairly new to DAX / PowerPivot but have built a nifty Sales Dashboard which is working well. Once combined with the PowerUpdate utility and Office365 PowerBI we will have a awesome solution. I'm excited!
One final thing that I am struggling to get my head around, is how we handle changing targets/budgets when a Sales Rep is promoted, as we assign budget by Job Level.
Let's say we have Sales Rep A - Junior Sales Rep with a target/budget of 30K per Quarter and they they get promoted in Q2 to Mid-Level with a target/budget of 45K per Quarter.
In my current dashboard, as soon as the Job Title is changed, this changes historical budget to the new target which warps any historic performance.
We have too many Sales Reps to create a quarterly budget table rows entry for each Sales Rep and each Quarter which seems to be how all the examples out there handle it - i.e:
Sales Rep A | 2015 | Q1 | 30000
Sales Rep A | 2015 | Q2 | 45000
Sales Rep A | 2015 | Q3 | 45000
My initial thoughts were to add a new user to the User Table with the same name and a "Budget Valid To" date and calculating target based off those dates.
Sales Rep A | Junior Sales Rep | 30000 | 01/01/2015
Sales Rep A | Mid - Sales Rep | 45000 | 01/04/2015
I just can't get my head around how I would handle this in a DAX calculation and I can't help thinking that there must be a more elegant way of handling this anyway! I'm so close to delivering this it tingles.
Let me know if you need an example or more detail.
Cheers,
Ben
Not sure the granularity of your data (daily | weekly | monthly). But one idea that could work well is to divide the target by the granularity so you're effectively pro rating the target. Then you can use basic SUM functions at each quarter boundary to show accurate targets since the sum of the 'daily' targets would equal your hard coded targets. You'd need to be careful if your periods vary in number of days... dividing into months would make it uniform; but mid month promotions would still be hard. HTH.
I have table with all countries GMT/UTC timezones, I need to see what time is in the rest of the countries when in USA is 11am-3pm
Not on particular date just know the difference in time.
I did my calculation like that I -5 GMT in USA and time is 11am then in Russia for example is +4 GMT.
5+4+11=20pm in Russia when USA is 11am, this works with countries that have + GMT zone but ones that have minus it shows wrong time.
I am working in Excel; please help me with advice on how to do it.
I did it already for the +gmt timezones and yes I have times for cities in big countries too; it was not my question.
How can I find out what time zone is in country with -11gmt when in country with +8gmt is 11am?
Someone know?
E.g. I work with dates like this in Excel. I set type of cell data to date and put
1/1/11 4:30 (+4:30 gmt)
1/1/11 1:00 (+1:00 gmt)
Now I have a date e.g. 1/20/11 11:00 (11 am on imaginary date); all I need to do is
"1/20/11 11:00 AM" - "1/1/11 4:30 AM" = "1/19/00 10:00 AM" at (0 gmt)
10am I don't really care about date in this case just time. I cannot think right now how I gound precise time but it seems somehow work without even putting +8 gmt in there...
Anyway solution should look something like that.
What about countries that have more than one time zone?
11am in the USA....where? West coast (PST) or east coast (EST)?
How do you take into consideration daylight savings time?
There are a lot of things to consider to do time conversions correctly.
I personally wouldn't keep a table with country and hour conversions, but two tables. One with timezones and hours from GMT time. And then another one with city names and timezone mappings. This way instead of converting from USA to Russia, you would be converting from New York to Moscow.
I did do a quick search on timezones in excel, and I found this article. I hope that it helps.