Summing time fields over 24 hours in Power Query - excel

I have a Power Query in excel linked to another file. This file has a time column. I understand that M language will not sum above 24 hours automatically without some work as it uses a datetime reference hence if I import a time of 25 hours it reverts back 2 hours to 1 hour...
In the 3rd column along in my image below using the second row as a reference, this is actually supposed to read 47:47:38. How can I get the instances where the value is above 24 hours to show the true hours?
I have tried using duration.hours(#hours()) this also does not work for some reason.
The same data from the source excel file is below also

Power Query doesn't have custom formats for how it displays data. If you have it read your data as a Duration instead of a DateTime it will display as [d].hh.mm.ss format, but still not with the total hours. Ultimately though this doesn't really matter because even when your data is formatted to display total hours in Excel, it's really being stored internally as days+hours+minutes+seconds. So how it displays in Power Query doesn't matter, as you can just use the hour formatting wherever you output the data to.
Now if you need to use the hours for a calculation between something that isn't another Duration, you can extract the hours by doing
Duration.Days([Your Hours]) * 24 + Duration.Hours([Your Hours])
Or now that I look at it, there is also a TotalHours function that gives you the hours plus mm:ss as a fractional amount of that
Duration.TotalHours([Your Hours])

Power BI doesn't handle this case very gracefully. A solution could be to convert the duration to a number to make it additive (so you can perform calculations and aggregations) and when you need to visualize it, to convert it to the desired format (HH:MM:SS).
Duration and Time are often confused. When such Excel files are read, the type of the column usually is DateTime, and date 1899-12-31 is added to the "time" part. You can change the data type of the column to be Decimal Number, but the "zero point" in Excel unfortunately is one day off (1899-12-30), so you need to subtract 1 from the result to get the actual "number of days" of the duration (i.e. 0.25 means 06:00:00).
So you must perform some conversion of the data. I would make a new column in the model to get the duration in the lowest granularity that I need (seconds in your example). In Power Query Editor add a custom column to calculate the duration in seconds (where Column1 is the name of the original duration column):
Duration in seconds = Duration.TotalSeconds([Column1] - #datetime(1899, 12, 31, 0, 0, 0))
Make sure the data type of this column is Whole Number (change it if necessary). Here 9144 seconds are calculated as 2 * 3600 + 32 * 60 + 24, or 02:32:24. Now you can calculate a sum on this column to get total duration in seconds for example. But when you visualize this column, don't do it directly, but make a measure to convert the data to the desired format. It could me made like this:
Measure Duration =
VAR duration_in_seconds = SUM(Sheet1[Duration in seconds])
VAR hours = ROUNDDOWN ( duration_in_seconds / 3600; 0 )
VAR minutes = ROUNDDOWN ( MOD ( duration_in_seconds; 3600 ) / 60; 0 )
VAR seconds = INT ( MOD ( duration_in_seconds; 60 ) )
RETURN hours & ":" & FORMAT(minutes; "00") & ":" & FORMAT(seconds; "00")
duration_in_seconds variable hold the total duration in seconds of the data in the context. From it we are calculating hours, minutes and seconds and constructing a string to represent the duration in the desired format. FORMAT is used to make sure there is a leading zero in case minutes or seconds are less than 10.
Here is how all three columns looks like when visualized:
Hope this helps!

Related

Spotfire AVG value per hour

I have users that have random values per hour across 24 hours. I want to get their average value per hour as it increases. Such as: a value of 3 at 3pm, then 4 at 4pm, 5 at 5pm, find the average per hour and give the total average once there are no more timestamps.
I've tried this:
case
when (DatePart("hour",[AUDIT_TSP])>0) and (DatePart("hour",[AUDIT_TSP])<1)
then Date([AUDIT_TSP]) & " " & ":00" & ":00" & Second([AUDIT_TSP])
when (DatePart("hour",[AUDIT_TSP])=1) and (DatePart("minute",
[AUDIT_TSP])>0) then Date([AUDIT_TSP]) & " " & ":01" & ":00" &
Second([AUDIT_TSP])
else null end
This was based off of sporfire: calculate the avg per 15 minutes and I tweaked it for my use but couldn't get the code to show the avg hour and not 15min avg. So I figured to ask here.
My AUDIT_TSP is formatted with DateTime and example values look like:
4/15/2019 6:16:59 AM
4/15/2019 6:20:05 AM
The values are just shipments so, 1 shipment an hour, 2 shipments an hour, etc. Just trying to get the average per hour.
I don't expect the average per hour to show up on the timeline, the values it's showing here is the amount of shipments for each hour. If the average can be shown on the timeline, then great, if not, then I can audible and show it in a textbox if that's possible as well.
You could produce an intermediate column to mark each hour with
[hourBin] calculated as: Integer(ToEpochSeconds([time-stamp]) / 3600)
Here [time-stamp] is your timestamp column. ToEpochSeconds(..) counts the number of seconds from a reference date (usually 1st Jan 1970). When you divide it by 3600 and take the integer part, you get an hour counter.
Then average your values for each hour
Avg([value]) OVER ([hourBin])
And/or visualise the average and the spread around it in a box-plot of [value] with [hourBin] as the category.
If you want the intermediate column to look more meaningful you can rescale it by its first value so it starts from 0 (or add any number you wish):
Integer(ToEpochSeconds([time-stamp]) / 3600) - Min(Integer(ToEpochSeconds([time-stamp]) / 3600))

In DAX, can I create a function that calculates the working days between two date/time fields?

There are already plenty of options for calculating the number of workdays between two dates if there are no times involved, but if you leave it as date/time, is there any way to get a number of working days (with decimal remainders) between two points in time in DAX (e.g. Power Query/Power BI)?
Assuming that your start and end times occur on working days, then you should be able to take the time difference between two dates and subtract out the number of non-working days during that period.
You'll want a calendar table to help out. Say,
Dates = CALENDARAUTO()
Then your working days measure might look like this:
WorkingDays =
StartFinish[Finish Date] - StartFinish[Start Date] -
SUMX(
FILTER(Dates,
Dates[Date] > StartFinish[Start Date] &&
Dates[Date] < StartFinish[Finish Date]),
IF(WEEKDAY(Dates[Date]) = 1 || WEEKDAY(Dates[Date]) = 7, 1, 0)
)
If you have an IsWorkDay column in your calendar table (which might include holidays as well as weekends), you can just reference that for the last line instead:
IF(Dates[IsWorkDay], 0, 1)
Note that this approach assumes a working day is 24 hours rather than a more standard 8 hours. You'll have to make some adjustments if you don't want the fractional part to indicate the portion of 24 hours. To switch to a portion of 8 hour work days, just multiply the fractional part by 24/3 = 3.

Powerpivot Dax- Calculate number of Hours after a fixed time

I have a list of times in a column [Logtime]:
11:45:44PM
07:05:05PM
I'd like to create a measure that returns the total number of hours after 6:30PM. So given the above times:
5.5
0.58
HoursAfter:=[logtime] -6:30PM doesn't work.
Hour[logtime] - hour(18.5) also doesn't work
EDIT:
timevalue([logtime]) - timevalue("05:00:00") works but it returns a datetime ala
12/30/1899 5:17:16PM
I need to convert the time 5:17:16 into decimal hours i.e. 5.26, how can I do this?
There may be a slightly more elegant way, but I was able to create a calculated column that does what you want as follows,
HoursAfter = DATEDIFF(TIMEVALUE("6:30 PM"), Times[LogTime], SECOND) / 3600
This takes the time difference between 6:30 PM and your LogTime in units of seconds and then converts it to hours by dividing by 60*60 = 3600.
Edit: A simpler formula can be written as follows,
HoursAfter = 24 * (Times[LogTime] - TIMEVALUE("6:30 PM"))
(Multiply by 24 since the datetime values are stored in units of days.)
timevalue([logtime]) - timevalue("05:00:00")

How to count hours in excel

I have xls file in following format
Name 1 2 3 4
John 09:00-21:00 09:00-21:00
Amy 21:00-09:00 09:00-21:00
Where 1,2,3,4 and so on represent days of current month,
09:00-21:00 - working hours.
I want to calculate salary based on the following conditions:
09:00-21:00 - 10$/hour
21:00-00:00 - 15$/hour
00:00-03:00 - 20$/hour
etc.
and so on (every hour can have it's own cost, for example 03:00-04:00 - 20$/hour, 04:00-05:00 - 19$/hour, etc.)
How can i accomplish this using only Excel (functions or VBA)?
P.S. Easy way: export to csv and process in python/php/etc.
Here is a non-VBA solution. It's a pretty nasty formula, but it works. I am sure it could be made even easier to use and understand with some more ingenuity:
Assuming the spreadsheet is set up like this:
Enter this formula in cell G1 and drag down for your data set:
=IF(ISBLANK(B2),"",IF(LEFT(B2,2)<MID(B2,FIND("-",B2)+1,2),SUMIFS($P$2:$P$24,$Q$2:$Q$24,">="&LEFT(B2,2),$Q$2:$Q$24,"<="&MID(B2,FIND("-",B2)+1,2)),SUMIF($Q$2:$Q$24,"<="&MID(B2,FIND("-",B2)+1,2),$P$2:$P$24)+SUMIF($Q$2:$Q$24,">="&LEFT(B2,2),$P$2:$P$24)))
To explain the formula in detail:
IF(ISBLANK(B2),"" will return a empty string if there is no time for a given person / day combination.
LEFT(B2,2) extracts the start-time into an hour.
Mid(B2,Find("-",B2)+1,2) extracts the end-time into an hour.
IF(LEFT(B2,2)<MID(B2,FIND("-",B2)+1,2) will check if the start-time is less than the end-time (meaning no over-night work). If the start-time is less than the end-time, it will use this formula to calculate the total cost per hour: SUMIFS($P$2:$P$24,$Q$2:$Q$24,">="&LEFT(B3,2),$Q$2:$Q$24,"<="&MID(B3,FIND("-",B3)+1,2))
If the start-time is higher than the end-time (meaning overnight work), it will use this formula to calculate: SUMIF($Q$2:$Q$24,"<="&MID(B3,FIND("-",B3)+1,2),$P$2:$P$24)+SUMIF($Q$2:$Q$24,">="&LEFT(B3,2),$P$2:$P$24)
The use of the Find("-",[cell]) splits the start-and- end times into values excel can use to do math against the Time / Cost table.
The formula in column Q of the Time / Cost table is =VALUE(MID(O2,FIND("-",O2)+1,2)) and turns the ending hour to consider the cost into a value Excel can use to add, instead of having the text from your original source format.
Do this in VBA! It is native to excel and is easy to learn. Functionally, I would loop through the table, write a function to calculate the dollars earned based on the info given. If you want your results to be live updating (like a formula in excel) you can write a user defined function. A helpful function might be an HoursIntersect function, as below:
Public Function HoursIntersect(Period1Start As Date, Period1End As Date, _
Period2Start As Date, Period2End As Date) _
As Double
Dim result As Double
' Check if the ends are greater than the starts. If they are, assume we are rolling over to
' a new day
If Period1End < Period1Start Then Period1End = Period1End + 1
If Period2End < Period2Start Then Period2End = Period2End + 1
With WorksheetFunction
result = .Min(Period1End, Period2End) - .Max(Period1Start, Period2Start)
HoursIntersect = .Max(result, 0) * 24
End With
End Function
Then you can determine the start and end time by splitting the value on the "-" character. Then multiply each payment schedule by the hours worked within that time:
DollarsEarned = DollarsEarned + 20 * HoursIntersect(StartTime, EndTime, #00:00:00#, #03:00:00#)
DollarsEarned = DollarsEarned + 10 * HoursIntersect(StartTime, EndTime, #09:00:00#, #21:00:00#)
DollarsEarned = DollarsEarned + 15 * HoursIntersect(StartTime, EndTime, #21:00:00#, #00:00:00#)
I have a method that uses nothing but formulas. First create a lookup table which contains every hour and rate in say columns K & L, something like this:
K L
08:00 15
09:00 10
10:00 10
11:00 10
12:00 10
13:00 10
14:00 10
15:00 10
16:00 10
17:00 10
18:00 10
19:00 10
20:00 10
21:00 15
22:00 15
23:00 15
Make sure you enter the hours as text by entering a single quote before the digits.
Then if your hours were in cell B2 you could then use this formula to calculate the total:
=SUM(INDIRECT("L"&MATCH(LEFT(B2,5),K2:K40,0)&":L"&MATCH(RIGHT(B2,5),K2:K40,0)))
All the formula is doing is getting the left and right text of your work time, using MATCH to find their positions in the lookup table which is used to create a range address which is then passed to SUM via the INDIRECT function.
If you need to worry about minutes all you need to do is create a bigger lookup table which holds every minute of the day. You may need to add some extra logic if your work days span midnight.

Convert 'x hrs y min z sec' to seconds

a) So I have a huge folder of .csv data with a column about time duration where the cells are 'x min y sec' (e.g. 15 min 29 sec) or 'x hrs y min z sec' (e.g. 1 hrs 48 min 28 sec). The cells are formatted by text.
I want to batch change them to the number of seconds, but I have no idea where to start. I can't get the data in another format.
I thought about somehow using 'hrs', 'min' or 'sec' as delimiters, but I don't know how to move from there. I also thought about using ' ' as delimiters, but then the first column is filled with either hours or minutes depending on the time duration.
I also thought about using PostgreSQL's SELECT EXTRACT(EPOCH FROM INTERVAL '5 days 3 hours'), but I haven't been able to work out how to use this on a column from a table.
b) Is there a better way to change this time format 'Fri Mar 14 11:29:27 EST 2014' to epoch time? Right now I'm thinking of using macros in Excel to get rid of 'Fri' and 'EST', then put the columns back together, then use the to_timestamp function in PostgreSQL.
In Excel if you have data in only those 2 formats and starting from A2 you can use this formula in B2 copied down to get the number of seconds:
=IFERROR(LEFT(A2,FIND("hrs",A2)-1)*3600,0)+SUM(MID(0&A2,FIND({"min","sec"},0&A2)-3,2)*{60,1})
It finds the relevant text then gets the number in front for each and multiplies by the relevant number to get seconds
You can do:
SELECT EXTRACT(EPOCH FROM column_name::interval)
FROM my_table;
The interval can use the regular time units (like hour), abbreviations thereof (hr) and plurals (hours). I am not sure about a combination of plural and abbreviation (hrs) though. If that does not work, UPDATE the column and replace() the sub-string "hrs" to "hours".
If you want to save the number of seconds in your table, then you convert the above statement into an UPDATE statement:
UPDATE my_table SET seconds_column = extract(epoch FROM column_name::interval);
I would split with space as the delimiter, then examine the second column. If it contains the string "hrs", then your seconds answer is:
3600 * column 1 + 60 * column 3 + column 5
Otherwise it is:
60 * column 1 + column 3

Resources