Related
After a week of research and many attempts, I kindly ask for help, please.
I have got the code works so close yet it's causing missed calculations.
I am trying to perform grouped running total with an if statement that evaluates if the next sum will result in a value greater than the value calculated with the previous total, and if does, add those calculations and then continue evaluating the next steps...
Here is my Code
let
Source = Excel.CurrentWorkbook(){[Name="SourceTable"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1,
Int64.Type),
CorrectTypes = Table.TransformColumnTypes(#"Added Index",{{"Index",
Int64.Type}, {"Filter", type text}, {"Volume", type number}}),
each List.Sum(Table.SelectRows(CorrectTypes, (Q) => Q[Filter] =
[Filter] and Q[Index] <= [Index])[Volume]), type number),
Runing = Table.Group(CorrectTypes,"Filter",{"A", each let
A = Table.AddIndexColumn(_,"i")
in Table.AddColumn(A,"R", each
List.Accumulate(Table.SelectRows(A, (a)=> a[i]<=[i])[Volume],
[Running=0, Verifier = 1],
// Here the challenge begins
(s,l)=> [Running = if s[Running]+l >
(Number.RoundUp(s[Running]+[Volume]/[Cube])*[Cube]) then
((Number.RoundUp(s[Running]/[Cube])*[Cube])-s[Running])+s[Running]+
[Volume] else s[Running]+l , Verifier =Number.From(s[Running]+l
<=Number.RoundUp(s[Running]/[Cube])*[Cube] )] ))
}),
ExpandedR = Table.ExpandRecordColumn(
Table.ExpandTableColumn(Runing, "A", {"Volume","Cube","i", "R"}),
"R", {"Running", "Verifier"})
in
ExpandedR
To Explain the question best here is the Excel File demonstrating desired calculation.
Query Results And Desire Outcome through Excel Formulas
What is a super easy step in excel formulas, So far very challenging in the M language?
I have used many great examples from the web yet none answer the complexity of my query.
The data self contains only 3 columns:
Filter -to do the grouping,
Volume - to sum up,
cube - to use for exception calculations,
Here are the links to resources I have used through my tryouts
Conditional running total in Power Query
[POWER QUERY] Grouped Running Totals with a maximum condition and a verifier
Running Total Power Query with Treshold
Power Query All Over Running Totals
Power Query Running Total with Grouping
Link to Solussion
Here are Two Codes which resolve my problem thanks to chaps on PowerQueryForum... link below
The Function
fxCalc
(A)=>
let
RunningTotal = Table.AddColumn(
A,
"Running Total",
each
if [Volume] <
(Number.RoundUp(List.Sum(List.InsertRange(List.FirstN(A[Volume],
[Index]),0,{0})) / [Cube], 0) * [Cube])
-
List.Sum(List.InsertRange(List.FirstN(A[Volume],[Index]),0,{0}))
then #RunningTotal[Running Total]
{[Index]-1} + [Volume]
else List.Sum(List.FirstN(A[Volume],
[Index]+1))
+
((Number.RoundUp(List.Sum(List.InsertRange(List.FirstN(A[Volume],
[Index]),0,{0})) / [Cube], 0) * [Cube])
-
List.Sum(List.InsertRange(List.FirstN(A[Volume],[Index]),0,
{0})))
),
CubeFill = Table.AddColumn(
RunningTotal,
"Cube Fill",
each [Running Total]/[Cube]
),
PositionCount = Table.AddColumn(
CubeFill,
"Position Count",
each Number.RoundUp([Cube Fill],0)
),
RemainingSpace = Table.AddColumn(
PositionCount,
"Remaining Space",
each [Position Count] * [Cube] - [Running
Total]
)
in
RemainingSpace
Main Query
let
Source =
Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText
("i45Wc
lTSUTLQMzACU4amSrE6cDFTLGLmRKqzRBZzwmKHExa9TljscMYpZmKKRR3C3lgA",
BinaryEncoding.Base64)
, Compression.Deflate)), let _t = ((type
nullable text) meta [Serialized.Text = true]) in type table
[Filter = _t, Volume = _t, Cube = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Filter",
type text}, {"Volume", type number}, {"Cube", type number}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Filter"},
{{"Group", each fxCalc(Table.AddIndexColumn(_,"Index",0,1))}}),
Combine = Table.Combine(#"Grouped Rows"[Group])
in
Combine
Option 2
let
Source =
Table.FromRows(Json.Document(Binary.Decompress
(Binary.FromText("i45WclTSUTLQMzACU4amQMpQKVYHLm6KEDdCFjdHiBvjUG+
CLG6JEDcFizth2GuGLI5kjjmyOJK9FmBxZwxxSyRxEyRzDA2QNSA5yBDo41gA",
BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type
nullable text) meta [Serialized.Text = true]) in type table [Filter
= _t, Volume = _t, Cube = _t, Index = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Filter",
type text}, {"Volume", type number}, {"Cube", type number},
{"Index", type text}}),
Aggregate = List.Accumulate(Table.ToRecords(#"Changed Type"), {},
(a,n)=> a & {
Record.AddField(Record.AddField(n, "CubeFill", n[Volume] + (if
List.IsEmpty(a) or List.Last(a)[CubeFill] + n[Volume] > n[Cube] or
List.Last(a)[Filter] <> n[Filter] then 0 else List.Last(a)
[CubeFill])),
"PositionCount", if List.IsEmpty(a) or List.Last(a)[Filter] <>
n[Filter] then 1 else if List.Last(a)[CubeFill] + n[Volume] >
n[Cube] then List.Last(a)
[PositionCount]+Number.RoundUp(n[Volume]/n[Cube], 0) else
List.Last(a)[PositionCount])}),
Output = Table.FromRecords(Aggregate)
in
Output
If it possible to somehow refer to the previous value in the same column? I know there is option to use it in next column.
But it not really fit for me because I need some logic like this. If (calculation value>check value; yes value; previous value). I got error " A cyclic reference was encountered during evaluation " When I am trying refer back.
IF I understand what you want to do correctly, you can accomplish that with the List.Generate function. You generate a list according to your rules; then combine it with the original table.
M Code
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUTJVitWJBpI6SsZgljmQZQJmGQFZlmCWMZBlBGaZAVlmSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}, {"Column2", Int64.Type}}),
//create conditional adding column
c1 = #"Changed Type"[Column1],
c2 = #"Changed Type"[Column2],
conditionalAdd = List.Generate(
()=>[res= if c1{0} + c2{0} > 10 then c1{0}+c2{0} else 0, idx=0],
each [idx] < List.Count(c1),
each [res=if c1{[idx]+1} + c2{[idx]+1} > 10 then c1{[idx]+1} + c2{[idx]+1} else [res],idx=[idx]+1],
each [res]),
//combine with original table
newTable =
Table.FromColumns(
Table.ToColumns(#"Changed Type") & {conditionalAdd},
Table.ColumnNames(#"Changed Type") & {"Conditional Add"}
)
in
newTable
Source
newTable
bear with me, this is my first attempt using the Power Query Formula Language. I need some advice on how to solve a particular problem sorting and filtering source data.
I now got this current source data, structured like this:
Using this power query:
let
Source = Excel.CurrentWorkbook(){[Name="EmployeeOrganization"]}[Content],
ListEmployees = Table.Group(Source, {"Organization"}, {{"Employee", each Text.Combine([Employee],","), type text}}),
CountEmployees = Table.AddColumn(ListEmployees, "Count", each List.Count(Text.Split([Employee],","))),
SplitEmployees = Table.SplitColumn(ListEmployees, "Employee", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv),List.Max(CountEmployees[Count])),
Transpose = Table.Transpose(SplitEmployees),
PromoteHeaders = Table.PromoteHeaders(Transpose, [PromoteAllScalars=true])
in
PromoteHeaders
I am able to produce the following result:
To avoid having to add the organization name to every single employee in the source, I would like the organization name to act as an parent-group, with the employees as children. I would also like the result to only fetch the organizations (+ employees) that has status Active = Yes.
The desired source should look similar to this:
So that the desired result should look similar to this: (Apple is gone due to Active = NO)
I am stuck at this point and need some advice on how can I modify my Power Query Formula to:
Only fetch Organizations that are Active (Does not matter if they have employees or not)
Somehow link the children Employees to the correct Organizations. (Without having to write the org name in every adjacent employee column)
(Excel file can be found her)
In PQ, you'll need to fill in the blank rows, then Pivot with no aggregation.
See the comments in the code, and follow the Applied Steps to understand the algorithm
Source
Custom Function
Rename: fnPivotAll
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/
(Source as table,
ColToPivot as text,
ColForValues as text)=>
let
PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))),
#"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _),
TableFromRecordOfLists = (rec as record, fieldnames as list) =>
let
PartialRecord = Record.SelectFields(rec,fieldnames),
RecordToList = Record.ToList(PartialRecord),
Table = Table.FromColumns(RecordToList,fieldnames)
in
Table,
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)),
#"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames),
#"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames)
in
#"Expanded Values"
Basic Query
let
//Read in data and set data types
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8k12yc9LzEkpVtJRAqLI1GKlWJ1oEDMgtSS1CCQK5XvlpyLzEvPgXMeCgpxUiH6/fJgC38SiSiT1jjmZyXAN7vn56TAdyDYmluYgaXHKTwLzYgE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Organization = _t, Employee = _t, Active = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Organization", type text}, {"Employee", type text}, {"Active", type text}}),
//replace blanks with null if not already there
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Organization", "Employee", "Active"}),
//fill down the Company and active columns
#"Filled Down" = Table.FillDown(#"Replaced Value",{"Organization", "Active"}),
//Filter to show only Active="Yes and Employee not null
#"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Employee] <> null) and ([Active] = "Yes")),
//Pivot with no aggregation
//could do this with grouping, but easier (and maybe faster, with a custom function
pivotAll = fnPivotAll(#"Filtered Rows","Organization","Employee"),
//remove unneeded Active column and set data types
#"Removed Columns" = Table.RemoveColumns(pivotAll,{"Active"}),
typed = Table.TransformColumnTypes(#"Removed Columns",
List.Transform(Table.ColumnNames(#"Removed Columns"),each {_, Text.Type}))
in
typed
typed Results
I have a column in my table that has some text values (input) which I would like to convert to numbers (output) for each unique text value, so that I can do some regression analysis:
Input
Output
AOP
1
AOS
2
AOS
2
AOS
2
AOP
1
null
0 or null
AOP
1
I initially tried to do this do this with several Transform: Replace Values steps, however I don't know how to:
make this flexible to different numbers of unique values (not hardcode 3 replacements but handle n where n is the number of unique values in input)
repeat this for many columns of my table
avoid looping as far as possible
What's a better approach?
One way is add custom column with below formula, and do that for each column you care to apply it to, using the value of each text character to generate a unique number
= try
List.Accumulate(Text.ToList([Input]), "", (state, current)=>
state&Number.ToText(Character.ToNumber(current), "0000")) otherwise null
this would transform all column's text into unique numbers, replacing the original data:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Function = (x) => try List.Accumulate(Text.ToList(x), "", (state, current)=> state&Number.ToText(Character.ToNumber(current), "0000")) otherwise null,
TransformList = List.Transform(Table.ColumnNames(Source), each {_ , Function}),
Output = Table.TransformColumns(Source, TransformList)
in Output
this would transform all column's text into unique numbers, appending the new columns to existing columns:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Function = (x) => try List.Accumulate(Text.ToList(x), "", (state, current)=> state&Number.ToText(Character.ToNumber(current), "0000")) otherwise null,
TransformList = List.Transform(Table.ColumnNames(Source), each {_ , Function}),
Output = Table.TransformColumns(Source, TransformList),
Numericals=Table.RenameColumns( Output, List.Zip( { Table.ColumnNames( Output), List.Transform(Table.ColumnNames(Output), each _ &"number") } ) ),
#"Merged Queries" = Table.NestedJoin(Table.AddIndexColumn(Source, "Index", 0, 1),{"Index"},Table.AddIndexColumn(Numericals, "Index2", 0, 1),{"Index2"},"Tabl2",JoinKind.LeftOuter),
#"Expanded Tabl2" = Table.ExpandTableColumn(#"Merged Queries", "Tabl2", Table.ColumnNames( Numericals),Table.ColumnNames( Numericals)),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Tabl2",{"Index"})
in #"Removed Columns"
I need to fit all the values of a column in Power Query into a 1-cell string separated by commas, as the example below:
To do this, I have the following piece of code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Transposed Table" = Table.Transpose(Source),
#"Merged Columns" = Table.CombineColumns(#"Transposed Table",{"Column1", "Column2", "Column3"},Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Merged"),
#"KeepString" = #"Merged Columns"[Merged]{0}
in
#"KeepString"
The problem with this code is that it assumes there will always be 3 columns, which is not always the case. How can I merge all columns (regardless of how many there are) into one?
You can do this with List.Accumulate:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
KeepString = List.Accumulate(Source[User], "", (state, current) => if state = "" then current else state & "," & current)
in
KeepString
You can also use Table.ColumnNames to get the list of all the column names. You can pass this into Table.CombineColumns, so your modified solution would be:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Transposed Table" = Table.Transpose(Source),
#"Merged Columns" = Table.CombineColumns(#"Transposed Table", Table.ColumnNames(#"Transposed Table"),Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Merged"),
#"KeepString" = #"Merged Columns"[Merged]{0}
in
#"KeepString"
You can also use a shorter code, like this:
let
Source=Excel.CurrentWorkbook( {[Name="Table1"]}[Content],
Result = Text.Combine(Source[User], ",")
in
Result