Spotfire AVG value per hour - spotfire

I have users that have random values per hour across 24 hours. I want to get their average value per hour as it increases. Such as: a value of 3 at 3pm, then 4 at 4pm, 5 at 5pm, find the average per hour and give the total average once there are no more timestamps.
I've tried this:
case
when (DatePart("hour",[AUDIT_TSP])>0) and (DatePart("hour",[AUDIT_TSP])<1)
then Date([AUDIT_TSP]) & " " & ":00" & ":00" & Second([AUDIT_TSP])
when (DatePart("hour",[AUDIT_TSP])=1) and (DatePart("minute",
[AUDIT_TSP])>0) then Date([AUDIT_TSP]) & " " & ":01" & ":00" &
Second([AUDIT_TSP])
else null end
This was based off of sporfire: calculate the avg per 15 minutes and I tweaked it for my use but couldn't get the code to show the avg hour and not 15min avg. So I figured to ask here.
My AUDIT_TSP is formatted with DateTime and example values look like:
4/15/2019 6:16:59 AM
4/15/2019 6:20:05 AM
The values are just shipments so, 1 shipment an hour, 2 shipments an hour, etc. Just trying to get the average per hour.
I don't expect the average per hour to show up on the timeline, the values it's showing here is the amount of shipments for each hour. If the average can be shown on the timeline, then great, if not, then I can audible and show it in a textbox if that's possible as well.

You could produce an intermediate column to mark each hour with
[hourBin] calculated as: Integer(ToEpochSeconds([time-stamp]) / 3600)
Here [time-stamp] is your timestamp column. ToEpochSeconds(..) counts the number of seconds from a reference date (usually 1st Jan 1970). When you divide it by 3600 and take the integer part, you get an hour counter.
Then average your values for each hour
Avg([value]) OVER ([hourBin])
And/or visualise the average and the spread around it in a box-plot of [value] with [hourBin] as the category.
If you want the intermediate column to look more meaningful you can rescale it by its first value so it starts from 0 (or add any number you wish):
Integer(ToEpochSeconds([time-stamp]) / 3600) - Min(Integer(ToEpochSeconds([time-stamp]) / 3600))

Related

Is there any function in excel to find day time between two date and time in Excel?

I need a formula to calculate between two date and time excluding lunch time, holidays, weekend and between 09:00 and 18:00 work hours.
For example, 25/07/2022 12:00 and 29/07/2022 10:00 and answer has to be 1 day, 06:00
Thanks in advance.
I had a formula but it didn't work when hours bigger than 24 hours.
I don't know how you got to 1 day and 6 hours, but here is a customizable way to filter your time difference calculation:
=LET(
start,E3,
end,E4,
holidays,$B$3:$B$5,
array,SEQUENCE(INT(end)-INT(start)+1,24,INT(start),TIME(1,0,0)),
crit_1,array>=start,
crit_2,array<=end,
crit_3,WEEKDAY(array,2)<6,
crit_4,HOUR(array)>=9,
crit_5,HOUR(array)<=18,
crit_6,HOUR(array)<>13,
crit_7,ISERROR(MATCH(DATE(YEAR(array),MONTH(array),DAY(array)),holidays,0)),
result,SUM(crit_1*crit_2*crit_3*crit_4*crit_5*crit_6*crit_7),
result
)
Limitation
This solution only works on an hourly level, i.e. the start and end dates and times will only be considered on a full hour basis. When providing times like 12:45 as input, the 15 minute increment won't be accounted for.
Explanation
The 4th item in the LET() function SEQUENCE(INT(end)-INT(start)+1,24,INT(start),TIME(1,0,0)) creates an array that contains all hours within the start and end date of the range:
(transposed for illustrative purposes)
then, based on that array, the different 'crit_n' statements are the individual criteria you mentioned. For example, crit_1,array>=start means that only the dates and times after the start date and time will be counted, or crit_6,HOUR(array)<>13 is the lunch break (assuming the 13th hour is lunch time), ...
All of the individual crit_n's are then arrays of the same size containing TRUE and FALSE elements.
At the end of the LET() function, by multiplying all the individual crit_n arrays, the product returns a single array that will then only contain those hours where all individual criteria statements are TRUE:
So then the SUM() function is simply returning the total number of hours that fit all criteria.
Example
I assumed lunch hour to be hour 13, and I assumed the 28th to be a holiday within the given range. With those assumptions and the other criteria you already specified above, I'm getting the following result:
Which looks like this when going into the formula bar:
In cell G2, you can put the following formula:
=LET(from,A2:A4,to,B2:B4,holidays,C2:C2,startHr,E1,endHr,E2, lunchS, E3, lunchE, E4,
CALC, LAMBDA(date,isFrom, LET(noWkDay, NETWORKDAYS(date,date,holidays)=0,
IF(noWkDay, 0, LET(d, INT(date), start, d + startHr, end, d + endHr,
noOverlap, IF(isFrom, date > end, date < start), lunchDur, lunchE-lunchS,
ls, d + lunchS, le, d + lunchE,
isInner, IF(isFrom, date > start, date < end),
diff, IF(isFrom, end-date-1 - IF(date < ls, lunchDur, 0),
date-start-1 - IF(date > le, lunchDur, 0)),
IF(noOverlap, -1, IF(isInner, diff, 0)))))),
MAP(from,to,LAMBDA(ff,tt, LET(wkdays, NETWORKDAYS(ff,tt,holidays),
duration, wkdays + CALC(ff, TRUE) + CALC(tt, FALSE),
days, INT(duration), time, duration - TRUNC(duration),
TEXT(days, "d") &" days "& TEXT(time, "hh:mm") &" hrs "
)))
)
and here is the output:
Explanation
Used LET function for easy reading and composition. The main idea is first to calculate the number of working days excluding holidays from column value to to column value. We use for that NETWORKDAYS function. Once we have this value for each row, we need to adjust it considering the first day and last day of the interval, in case we cannot count as a full day and instead considering hours. For inner days (not start/end of the interval) it is counted as an entire day.
We use MAP function to do the calculation over all values of from and to names. For each corresponding value (ff, tt) we calculate the working days (wkdays). Once we have this value, we use the user LAMBDA function CALC to adjust it. The function has a second input argument isFrom to consider both scenarios, i.e. adjustment at the beginning of the interval (isFrom = TRUE) or to the end of the interval (isFrom=FALSE). The first input argument is the given date.
In case the input date of CALC is a non working day, we don't need to make any adjustment. We check it with the name noWkDay. If that is not the case, then we need we need to determine if there is no overlap (noOverlap):
IF(isFrom, date > end, date < start)
where start, end names correspond to the same date as date, but with different hours corresponding to start Hr and end Hr (E1:E2). For example for the first row, there is no overlap, because the end date doesn't have hour information, i.e. (12:00 AM), in such case the corresponding date should not be taken into account and CALC returns -1, i.e. one day needs to be subtracted.
In case we have overlap, then we need to consider the case the working hours are lower than the maximum working hours (from 9:00 to 18:00). It is identified with the name isInner. If that is the case, we calculate the actual hours. We need to subtract 1 because it is going to be one less full working day and instead to consider the corresponding hours (that should be less than 9hrs, which is the maximum workday duration). The calculation is carried under the name diff:
IF(isFrom, end-date-1 - IF(date < ls, lunchDur, 0),
date-start-1 - IF(date > le, lunchDur, 0))
If the actual start is before the start of the lunch time (ls), then we need to subtract lunch duration (lunchDur). Similarly if the actual end is is after lunch time, we need to discount it too.
Finally, we use CALC to calculate the interval duration:
wkdays + CALC(ff, TRUE) + CALC(tt, FALSE)
Once we have this information, it is just to put in the specified format indicating days and hours.
Now let's review some of the sample input data and results:
The interval starts on Monday 7/25 and ends on Friday 7/29, therefore we have 5 working days, but 7/26 is a holiday, so the maximum number of working days will be 4 days.
For the interval [7/25, 7/29] starts and ends on midnight (12:00 AM), therefore the last day of the interval should not be considered, so actual working days will be 3.
Interval [7/25 10:00, 7/29 17:00]. For the start of the interval we cannot count one day, instead 8hrs and for the end of the interval, the same situation 8hrs, so instead of 4days we are goin to have 2days plus 16hrs, but we need to subtract in both cases the lunch duration (1hr) so the final result will be 2 days 14hrs.

Summing time fields over 24 hours in Power Query

I have a Power Query in excel linked to another file. This file has a time column. I understand that M language will not sum above 24 hours automatically without some work as it uses a datetime reference hence if I import a time of 25 hours it reverts back 2 hours to 1 hour...
In the 3rd column along in my image below using the second row as a reference, this is actually supposed to read 47:47:38. How can I get the instances where the value is above 24 hours to show the true hours?
I have tried using duration.hours(#hours()) this also does not work for some reason.
The same data from the source excel file is below also
Power Query doesn't have custom formats for how it displays data. If you have it read your data as a Duration instead of a DateTime it will display as [d].hh.mm.ss format, but still not with the total hours. Ultimately though this doesn't really matter because even when your data is formatted to display total hours in Excel, it's really being stored internally as days+hours+minutes+seconds. So how it displays in Power Query doesn't matter, as you can just use the hour formatting wherever you output the data to.
Now if you need to use the hours for a calculation between something that isn't another Duration, you can extract the hours by doing
Duration.Days([Your Hours]) * 24 + Duration.Hours([Your Hours])
Or now that I look at it, there is also a TotalHours function that gives you the hours plus mm:ss as a fractional amount of that
Duration.TotalHours([Your Hours])
Power BI doesn't handle this case very gracefully. A solution could be to convert the duration to a number to make it additive (so you can perform calculations and aggregations) and when you need to visualize it, to convert it to the desired format (HH:MM:SS).
Duration and Time are often confused. When such Excel files are read, the type of the column usually is DateTime, and date 1899-12-31 is added to the "time" part. You can change the data type of the column to be Decimal Number, but the "zero point" in Excel unfortunately is one day off (1899-12-30), so you need to subtract 1 from the result to get the actual "number of days" of the duration (i.e. 0.25 means 06:00:00).
So you must perform some conversion of the data. I would make a new column in the model to get the duration in the lowest granularity that I need (seconds in your example). In Power Query Editor add a custom column to calculate the duration in seconds (where Column1 is the name of the original duration column):
Duration in seconds = Duration.TotalSeconds([Column1] - #datetime(1899, 12, 31, 0, 0, 0))
Make sure the data type of this column is Whole Number (change it if necessary). Here 9144 seconds are calculated as 2 * 3600 + 32 * 60 + 24, or 02:32:24. Now you can calculate a sum on this column to get total duration in seconds for example. But when you visualize this column, don't do it directly, but make a measure to convert the data to the desired format. It could me made like this:
Measure Duration =
VAR duration_in_seconds = SUM(Sheet1[Duration in seconds])
VAR hours = ROUNDDOWN ( duration_in_seconds / 3600; 0 )
VAR minutes = ROUNDDOWN ( MOD ( duration_in_seconds; 3600 ) / 60; 0 )
VAR seconds = INT ( MOD ( duration_in_seconds; 60 ) )
RETURN hours & ":" & FORMAT(minutes; "00") & ":" & FORMAT(seconds; "00")
duration_in_seconds variable hold the total duration in seconds of the data in the context. From it we are calculating hours, minutes and seconds and constructing a string to represent the duration in the desired format. FORMAT is used to make sure there is a leading zero in case minutes or seconds are less than 10.
Here is how all three columns looks like when visualized:
Hope this helps!

formating numbers into Time format

I want to convert numbers (that represent seconds) into HH:MM:SS format (like shown in pic..) but don't want to split them.. I want to format it using some formula or
If you just want to display the 715 seconds as hours:minutes:seconds, you will need a helper column that you use just for display.
You can create it either as in your example screenshot, or using a formula:
=TEXT(B2/86400,"[hh]:mm:ss")
Note the brackets around hh. That keeps the hours from rolling over every 24 and displays total hours correctly for instances where seconds > 86,400.
There's no way I know of to have 715 displayed as the associated hours:minutes:seconds in the same cell.
You can use a formula like this to convert 715 to HH:MM:SS:
=TEXT(FLOOR((A1-FLOOR(A1/86400,1)*86400)/3600, 1), "00:") & text(floor((A1 - floor(A1/3600,1)*3600)/60, 1),"00:") & text(A1 - floor(A1/3600,1)*3600 - floor((A1 - floor(A1/3600,1)*3600)/60, 1)*60,"00")
Let's break it down.
Get hours
A1 is seconds
There are 86400 seconds in a day
FLOOR(A1/86400, 1) gives us the floor of days
Multiply that by 86400 will give us seconds in those days FLOOR(A1/86400,1)*86400
Remove those many seconds from A1 and you get seconds remaining in the fraction of the last day A1- FLOOR(A1/86400,1)*86400
Divide those remaining seconds with 3600 to get hours FLOOR((A1- FLOOR(A1/86400,1)*86400 )/3600, 1)
Use TEXT function to format it to 2 digits and a colon at the end TEXT(FLOOR((A1- FLOOR(A1/86400,1)*86400 )/3600, 1), "00:")
Then, find remaining minutes and format it.
Then, take remaining seconds and format it.
Alternate method
Assume that your seconds are in column A6.
In B6, we will put Whole Days =FLOOR(A6/86400, 1)
In C6, we will put remaining seconds =A6-B6*86400
In D6, we will put Whole Hours =FLOOR(C6/3600,1)
In E6, we will put remaining seconds =C6-D6*3600
In F6, we will put Whole Minutes =FLOOR(E6/60, 1)
In G6, we will put remaining seconds =E6-F6*60
In H6, we'll format Hours, Minutes and Seconds =TEXT(D6,"00:") & TEXT(F6,"00:") & TEXT(G6,"00")

How to count hours in excel

I have xls file in following format
Name 1 2 3 4
John 09:00-21:00 09:00-21:00
Amy 21:00-09:00 09:00-21:00
Where 1,2,3,4 and so on represent days of current month,
09:00-21:00 - working hours.
I want to calculate salary based on the following conditions:
09:00-21:00 - 10$/hour
21:00-00:00 - 15$/hour
00:00-03:00 - 20$/hour
etc.
and so on (every hour can have it's own cost, for example 03:00-04:00 - 20$/hour, 04:00-05:00 - 19$/hour, etc.)
How can i accomplish this using only Excel (functions or VBA)?
P.S. Easy way: export to csv and process in python/php/etc.
Here is a non-VBA solution. It's a pretty nasty formula, but it works. I am sure it could be made even easier to use and understand with some more ingenuity:
Assuming the spreadsheet is set up like this:
Enter this formula in cell G1 and drag down for your data set:
=IF(ISBLANK(B2),"",IF(LEFT(B2,2)<MID(B2,FIND("-",B2)+1,2),SUMIFS($P$2:$P$24,$Q$2:$Q$24,">="&LEFT(B2,2),$Q$2:$Q$24,"<="&MID(B2,FIND("-",B2)+1,2)),SUMIF($Q$2:$Q$24,"<="&MID(B2,FIND("-",B2)+1,2),$P$2:$P$24)+SUMIF($Q$2:$Q$24,">="&LEFT(B2,2),$P$2:$P$24)))
To explain the formula in detail:
IF(ISBLANK(B2),"" will return a empty string if there is no time for a given person / day combination.
LEFT(B2,2) extracts the start-time into an hour.
Mid(B2,Find("-",B2)+1,2) extracts the end-time into an hour.
IF(LEFT(B2,2)<MID(B2,FIND("-",B2)+1,2) will check if the start-time is less than the end-time (meaning no over-night work). If the start-time is less than the end-time, it will use this formula to calculate the total cost per hour: SUMIFS($P$2:$P$24,$Q$2:$Q$24,">="&LEFT(B3,2),$Q$2:$Q$24,"<="&MID(B3,FIND("-",B3)+1,2))
If the start-time is higher than the end-time (meaning overnight work), it will use this formula to calculate: SUMIF($Q$2:$Q$24,"<="&MID(B3,FIND("-",B3)+1,2),$P$2:$P$24)+SUMIF($Q$2:$Q$24,">="&LEFT(B3,2),$P$2:$P$24)
The use of the Find("-",[cell]) splits the start-and- end times into values excel can use to do math against the Time / Cost table.
The formula in column Q of the Time / Cost table is =VALUE(MID(O2,FIND("-",O2)+1,2)) and turns the ending hour to consider the cost into a value Excel can use to add, instead of having the text from your original source format.
Do this in VBA! It is native to excel and is easy to learn. Functionally, I would loop through the table, write a function to calculate the dollars earned based on the info given. If you want your results to be live updating (like a formula in excel) you can write a user defined function. A helpful function might be an HoursIntersect function, as below:
Public Function HoursIntersect(Period1Start As Date, Period1End As Date, _
Period2Start As Date, Period2End As Date) _
As Double
Dim result As Double
' Check if the ends are greater than the starts. If they are, assume we are rolling over to
' a new day
If Period1End < Period1Start Then Period1End = Period1End + 1
If Period2End < Period2Start Then Period2End = Period2End + 1
With WorksheetFunction
result = .Min(Period1End, Period2End) - .Max(Period1Start, Period2Start)
HoursIntersect = .Max(result, 0) * 24
End With
End Function
Then you can determine the start and end time by splitting the value on the "-" character. Then multiply each payment schedule by the hours worked within that time:
DollarsEarned = DollarsEarned + 20 * HoursIntersect(StartTime, EndTime, #00:00:00#, #03:00:00#)
DollarsEarned = DollarsEarned + 10 * HoursIntersect(StartTime, EndTime, #09:00:00#, #21:00:00#)
DollarsEarned = DollarsEarned + 15 * HoursIntersect(StartTime, EndTime, #21:00:00#, #00:00:00#)
I have a method that uses nothing but formulas. First create a lookup table which contains every hour and rate in say columns K & L, something like this:
K L
08:00 15
09:00 10
10:00 10
11:00 10
12:00 10
13:00 10
14:00 10
15:00 10
16:00 10
17:00 10
18:00 10
19:00 10
20:00 10
21:00 15
22:00 15
23:00 15
Make sure you enter the hours as text by entering a single quote before the digits.
Then if your hours were in cell B2 you could then use this formula to calculate the total:
=SUM(INDIRECT("L"&MATCH(LEFT(B2,5),K2:K40,0)&":L"&MATCH(RIGHT(B2,5),K2:K40,0)))
All the formula is doing is getting the left and right text of your work time, using MATCH to find their positions in the lookup table which is used to create a range address which is then passed to SUM via the INDIRECT function.
If you need to worry about minutes all you need to do is create a bigger lookup table which holds every minute of the day. You may need to add some extra logic if your work days span midnight.

Excel - Find max value in a 2D array with logic

I'm trying to find the maximum value from the 15 minute interval data that has dates associated with each row seen below:
DATE UOM 00:01-00:15 kW 00:16-00:30 kW 00:31-00:45 kW 00:46-01:00 kW
7/1/2010 KW 907.2 892.8 883.2 883.2
7/2/2010 KW 907.2 849.6 859.2 825.6
7/3/2010 KW 811.2 806.4 806.4 801.6
7/4/2010 KW 763.2 768 758.4 772.8
This data is electrical demands for my school's campus, and I'm trying to find peak, partial peak, and off peak maximum demands. There are approximately 4 years of data with each row consisting of a single data.
Peak hours occur during 12:00 - 18:00 hours
Partial Peak occurs during 08:31 - 11:59 & 18:00 21:30
Off Peak occurs during 21:30 - 08:30
I'd like to be able to get those values for each month of each year. But so far the logic isn't coming to me, and everything I'm looking up just shows me index-match tutorials. Any help would be greatly appreciated.
Simply use MAX or a combination of two MAX functions in order to determine maximums for any given timespan.
In my screenshot, you can see how the ranges are defined by the columns. Therefore you may have to adjust the ranges to correspond to your actual spreadsheet.
For example, for cell CW1 it uses the formula =MAX($AY2:$BV2). This determines the value of the maximum value for all 15-minute time spans within that range. Because 12:01 occurs in column AY, and 18:00 ends in column BV, it's possible to find the maximum between 12:01 - 18:00 by using the MAX function.
For time spans that are not continuous, we can split them into multiple ranges. For CX and CY we do this by using two MAX functions. So a maximum value is retrieved for each continuous time span, and then the outer MAX determines the maximum of the two local maximums.
Therefore, for CX:
=MAX(MAX($AK2:$AX2),MAX($BW2:$CJ2))
For CY:
=MAX(MAX($C2:$AJ2),MAX($CK2:$CT2))
Note that I don't have your full data set, so these values are garbage.

Resources