While using Azure Stream Analysis, I can create a kind of moving average by using AVG and group them by the HoppingWindow like the code shown below.
However, this will create a moving average of the points in the last 5 seconds. Is there a way to create a moving average of the last n datapoint? I understand that I can adjust the windows size to make n points coming into the window, but is there a way to measure the exact last n points like the one in MySQL and PostgresSQL?
SELECT System.TimeStamp AS OutTime, AVG (value)
INTO
[output]
FROM [input]
GROUP BY HoppingWindow(second,5,1)
Today ASA windows are only based on time. However you can use the LAG function to get previous events.
For your query, if you want to make the average in the 3 latest events, it will be something like this:
SELECT System.TimeStamp AS OutTime,
AvgValue= ( LAG(value,1) OVER (LIMIT DURATION(minute, 5))
+ LAG(value,2) OVER (LIMIT DURATION(minute, 5))
+ LAG(value,3) OVER (LIMIT DURATION(minute, 5))
)/3
FROM input
Sorry for the inconvenience.
Thanks,
JS
Related
In PowerBI, I need to create a Performance Indicator (KPI) measure which evaluates dataset values in a scale from 0 to 1, with target (1) being the MAX value in a 20 years history. It's a national airport trip records open database. The formula is basically [value]/[max value].
My dataset has a lot of fields and I wish I could filter it by any of these fields, with a line chart showing the 0-1 indicator for each month based on the filters.
This is my workaround test solution:
Table 1 - Original dataset: if I filter something here, below tables also update (there are more fields to the left, including YEAR and MONTH
Table 2 - Reference to original dataset, aggregating YEAR-MONTH by the sum of "take-offs" (decolagens)
Table 3 - Reference to above (sum) table, aggregating MONTH by the max of "take-offs" (decolagens)
Table 4 - 'Sum table' merged to 'Max table' by MONTH as new table: then do [Value]/[Max] and we've got the indicator
So if i filter the original dataset by any fields, all other tables update accordingly and the indicators always stays between 0-1, works like a charm.
TL;DR
The problem is: I need to create a dashboard of this on Power Bi. So I need this calculation to be in a measure or another workaround.
My possible solution: by pure DAX code in the measure field, to produce Tables 2 and 3 so I'll divide the month sum values by their month max value (which will both be produced according to PowerBi dashboard slicers) and get the indicator dinamically produced.
I'm stuck at: I don't understand how can I reference a sum/max aggregate table in dax code. Something like = SUM (dataset[take-offs]) / MAX (SUM (dataset[take-offs])). Of course these functions do not work like that, but I hope I made my point clear: how can I produce this four table effect with a single measure?
Other solutions are welcome.
Link to the original dataset: https://www.anac.gov.br/assuntos/dados-e-estatisticas/dados-estatisticos/arquivos/DadosEstatsticos.csv
It's an open dataset, so I guess there's no problem sharing it. Please help! :)
EDIT: please download the dataset and try to solve this. Personally I think it's a quality statistics doubt that will eventually help others. The calculation works, it only needs a Power Bi Measure port.
Add the ALL formula:
Measure = SUMX(ALL('Table'),[Valor])/SUM('Table'[Max])
Example
I am trying to do some reporting stuff (engineering field), and I am stuck on some "simple" thing, how to insert/add custom Min/Max values so that I can have on the certain visual (like on the picture) border values, min and max that I have defined!
I am receiving set of data, where Min/Max aren't included, and I have to calculate them (those border values are defined by us). I tried to do add column but when I go to "analytics" and try to put there Min/Max, I am getting doubled lines and its quite messy (on pic below)... Is there any easier and better way to calculate / add that? From additional table or else??
Min would be 60 °C
Max would be 100 °C , it is just temperature range and its limitation.
So on the pic below, I tried Add Column into the Data and also Measure directly (60 and 100 degrees) and got this below....the problem is, you cant edit these lines... the thing is I wanted them from Analytics part but there I cant see them because they re not "existing".
So, after little bit of playing around and some suggests of PBI pros, I will just write two ways of getting the same results, with minor change.
1) You can insert measure/column into your data set (in my case that was Min = 60 and Max = 100), simple as a day. Then on Format tab > Customize series where you can adjsut your Line: width, type, style...
2) Without any calculation: On Analytics tab > add 2x Constant Lines, put values that you want and thats it.
The only difference here is Data labeling, on 2nd option you can see your limiter value on one of the sides...and using 1st option, your limiter value is through whole line, and you cant remove it (for a whole line). But depends what and how you wanna see your data.
Maybe there are some other ways but this one was fine for me, didnt explore fully PBI yet.
I'm currently working on a requirement for a real-time dashboard using steam analytics with a power bi output. The user would like to see a cumulative sum of an amount for the current hour. This would revert back to 0 at the beginning of the next hour.
From my understanding a Tumbling window should do this but the value doesn't change in the Power BI dashboard.
Any help would be appreciated!
Example:
SELECT System.Timestamp AS Timestamp,
SUM(Revenue) AS Revenue
INTO [TotalRevenuePerHour]
FROM [Input]
GROUP BY TUMBLINGWINDOW(hour,1)
Per my understanding of your requirements,you want to sum some value every one hour.And the value need to be refreshed in real-time during this one hour. If my words is right,i think your direction is right. However, you could know about Hopping Windows, not Tumbling Windows.
You could get know about differences between them with below two pictures:
Hopping Windows
Tumbling Windows
Actually,tumbling window is simply a hopping window whose ‘hop’ is equal to its ‘size’.
Please see the example of Hopping Windows from above link:
SELECT System.Timestamp() AS WindowEnd, TollId, COUNT(*)
FROM Input TIMESTAMP BY EntryTime
GROUP BY TollId, HoppingWindow(Duration(hour, 1), Hop(minute, 5), Offset(millisecond, -1))
I have a dataset:
I want to apply the clustering technique to create clusters of every 5 minutes' data and want to calculate the average of the last column i.e. percentage congestion.
How to create such clusters of every 5 minutes? I want to use this analysis further for decision making. The decision will be made on the basis of average percentage calculated.
That is a simple aggregation, and not clustering.
Use a loop, read one record at a time, and every 5 minutes output the average and reinitialize the accumulators.
Or round ever time to the 5 minute granularity. Then take the average of now identical keys. That would be a SQL GROUP_BY.
Is it possible to calculate a percentage change in a Cognos Report?
Currently, I have a crosstab that has years as the columns and widgets in the rows with a calculation of total revenue. I am looking to calculate the annual % increase. New to Cognos, but I could accomplish this using other tools.
Any help will be very much appreciated.
Sorry for the late answer to your question but perhaps it will help others who see it.
It's kind of annoying, but it can be done. If you have one Query which uses an input parameter to select the year, perhaps it is a filter that says [Year] = ?YearPrompt?, then you can make a second Query which uses the filter [Year] = ?YearPrompt? - 1. Then, you can join these two queries, and the third query (made by joining the first two) will have both this year's revenue, and last year's revenue, available to it for a calculation, such as what the percent change is.
You need to create a query calculation item as one of the columns and in it you take Year2 - Year1(You can choose the exact columns you already have in the report using the Data Items tab when you are making the query calc) Also make sure you make this query calc a % in its format.