I am trying to write a query that takes a table and multiplies every number in the table by 100. I've gotten close, but I am having trouble applying it correctly to every column. Below is the code I have so far. The line starting with ReplaceTable is the line I have working for one column, and the line below was my attempt at getting it to work for other columns. I am dealing with a small subset currently, but the real data will potentially have ~100 columns, so I do not want to do this by hand. If there's a better way to do this task, please let me know. I am new to Power Query, so if able please explain my error/the solution so I can learn. Thanks!
let
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
//Organization will always be of type text. The others will be should be numbers, unless user error
#"Changed Type" = Table.TransformColumnTypes(Source, {{"Organization", type text}, {"A", Int64.Type}, {"B", Int64.Type}, {"C", Int64.Type}}),
//function to replace all values in all columns with multiplied values
MultiplyReplace = (DataTable as table, DataTableColumns as list) =>
let
Counter = Table.ColumnCount(DataTable),
ReplaceCol = (DataTableTemp, i) =>
let
colName = {DataTableColumns{i}},
col = Table.Column(DataTableTemp, colName),
//LINE THAT WORKS- want this functionality for ALL columns
ReplaceTable = Table.ReplaceValue(DataTableTemp, each[A], each if [A] is number then [A]*100 else [A], Replacer.ReplaceValue, colName)
//ReplaceTable = Table.ReplaceValue(DataTableTemp, each col, each if col is number then col*100 else col, Replace.ReplaceValue, colName)
in
if i = Counter-1 then ReplaceTable else #ReplaceCol(ReplaceTable, i+1)
in
ReplaceCol(DataTable, 0),
allColumns = Table.ColumnNames(#"Changed Type"),
#"Multiplied Numerics" = MultiplyReplace(#"Changed Type", allColumns)
//#"Restored Type" = Value.ReplaceTypes(#"Multiplied Numerics", #"Changed Type")
in
#"Multiplied Numerics"
The issue involves the scope of the functions and the variables.
With a hard-coded column name (such as [A]), the code is understanding the shorthand to actually mean _[A]. Within a Table.ReplaceValue function, that _ is referencing the current Record or row. However, the col variable is referencing the entire table column. When used in the replacer function, it causes an error. (Unfortunately(?), errors in replacer functions are just ignored with no error message, so issues can be hard to trace.)
In the corrected code, I got rid of the col variable, since it's being determined at the wrong scope level. I changed colName to being text instead of a list, and then used the Record.Field function with _ (the current record within the Table.ReplaceValue function) and the text value colName to extract the desired record for the calculations with the Table.ReplaceValue function itself.
Corrected Code
let
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
//Organization will always be of type text. The others will be should be numbers, unless user error
#"Changed Type" = Table.TransformColumnTypes(Source, {{"Organization", type text}, {"A", Int64.Type}, {"B", Int64.Type}, {"C", Int64.Type}}),
//function to replace all values in all columns with multiplied values
MultiplyReplace = (DataTable as table, DataTableColumns as list) =>
let
Counter = Table.ColumnCount(DataTable),
ReplaceCol = (DataTableTemp, i) =>
let
colName = DataTableColumns{i},
//LINE THAT WORKS- want this functionality for ALL columns
ReplaceTable = Table.ReplaceValue(DataTableTemp,each Record.Field(_, colName), each if Record.Field(_, colName) is number then Record.Field(_, colName)*100 else Record.Field(_, colName),Replacer.ReplaceValue,{colName})
//ReplaceTable = Table.ReplaceValue(DataTableTemp, each col, each if col is number then col*100 else col, Replace.ReplaceValue, colName)
in
if i = Counter-1 then ReplaceTable else #ReplaceCol(ReplaceTable, i+1)
in
ReplaceCol(DataTable, 0),
allColumns = Table.ColumnNames(#"Changed Type"),
#"Multiplied Numerics" = MultiplyReplace(#"Changed Type", allColumns)
//#"Restored Type" = Value.ReplaceTypes(#"Multiplied Numerics", #"Changed Type")
in
#"Multiplied Numerics"
Related
I need to find and replace the headers of my Source Table in Power query
I am able to do this with BulkReplace
But this searches the entire table, is there a way to restrict BulkReplace to only the headers, or if not then I can demote the headers and run BulkReplace on just Row 1 of the Source Table
Thank you
sumAppHeads (Find Replace Table)
In my Power Query, I have
BulkReplaceStepHeaders = fBulkReplaceStep(#"Demoted Headers", sumAppHeaders, Table.ColumnNames(#"Demoted Headers")),
let BulkReplace = (DataTable as table, FindReplaceTable as table, DataTableColumn as list) =>
let
//Convert the FindReplaceTable to a list using the Table.ToRows function
//so we can reference the list with an index number
FindReplaceList = Table.ToRows(FindReplaceTable),
//Count number of rows in the FindReplaceTable to determine
//how many iterations are needed
Counter = Table.RowCount(FindReplaceTable),
//Define a function to iterate over our list
//with the Table.ReplaceValue function
BulkReplaceValues = (DataTableTemp, n) =>
let
//Replace values using nth item in FindReplaceList
ReplaceTable = Table.ReplaceValue(
DataTableTemp,
//replace null with empty string in nth item
if FindReplaceList{n}{0} = null then "" else FindReplaceList{n}{0},
if FindReplaceList{n}{1} = null then "" else FindReplaceList{n}{1},
Replacer.ReplaceValue,
DataTableColumn
)
in
//if we are not at the end of the FindReplaceList
//then iterate through Table.ReplaceValue again
if n = Counter - 1
then ReplaceTable
else #BulkReplaceValues(ReplaceTable, n + 1),
//Evaluate the sub-function at the first row
Output = BulkReplaceValues(DataTable, 0)
in
Output
in
BulkReplace
Demote the headers
Transpose the table
Replace the old column names that are now all in Column1
Transpose the table back
Promote the headers
Try this
BulkReplaceStepHeaders = fBulkReplaceStep(Table.FirstN(#"Demoted Headers", 1),sumAppHeaders,Table.ColumnNames(#"Demoted Headers")) & Table.Skip(#"Demoted Headers", 1),
This grabs the column names, merges against the replace table to find new names, then does a rename to use the new names
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Merged Queries" = Table.NestedJoin(Table.FromList(Table.ColumnNames(Source)), {"Column1"}, ReplaceTable, {"Find"}, "Table2", JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"Replace"}, {"Replace"}),
#"NewNames" = Table.AddColumn(#"Expanded Table2", "Custom", each if [Replace]=null then [Column1] else [Replace])[Custom],
#"Rename"=Table.RenameColumns( Source, List.Zip( { Table.ColumnNames( Source ), #"NewNames" } ) )
in #"Rename"
After a week of research and many attempts, I kindly ask for help, please.
I have got the code works so close yet it's causing missed calculations.
I am trying to perform grouped running total with an if statement that evaluates if the next sum will result in a value greater than the value calculated with the previous total, and if does, add those calculations and then continue evaluating the next steps...
Here is my Code
let
Source = Excel.CurrentWorkbook(){[Name="SourceTable"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1,
Int64.Type),
CorrectTypes = Table.TransformColumnTypes(#"Added Index",{{"Index",
Int64.Type}, {"Filter", type text}, {"Volume", type number}}),
each List.Sum(Table.SelectRows(CorrectTypes, (Q) => Q[Filter] =
[Filter] and Q[Index] <= [Index])[Volume]), type number),
Runing = Table.Group(CorrectTypes,"Filter",{"A", each let
A = Table.AddIndexColumn(_,"i")
in Table.AddColumn(A,"R", each
List.Accumulate(Table.SelectRows(A, (a)=> a[i]<=[i])[Volume],
[Running=0, Verifier = 1],
// Here the challenge begins
(s,l)=> [Running = if s[Running]+l >
(Number.RoundUp(s[Running]+[Volume]/[Cube])*[Cube]) then
((Number.RoundUp(s[Running]/[Cube])*[Cube])-s[Running])+s[Running]+
[Volume] else s[Running]+l , Verifier =Number.From(s[Running]+l
<=Number.RoundUp(s[Running]/[Cube])*[Cube] )] ))
}),
ExpandedR = Table.ExpandRecordColumn(
Table.ExpandTableColumn(Runing, "A", {"Volume","Cube","i", "R"}),
"R", {"Running", "Verifier"})
in
ExpandedR
To Explain the question best here is the Excel File demonstrating desired calculation.
Query Results And Desire Outcome through Excel Formulas
What is a super easy step in excel formulas, So far very challenging in the M language?
I have used many great examples from the web yet none answer the complexity of my query.
The data self contains only 3 columns:
Filter -to do the grouping,
Volume - to sum up,
cube - to use for exception calculations,
Here are the links to resources I have used through my tryouts
Conditional running total in Power Query
[POWER QUERY] Grouped Running Totals with a maximum condition and a verifier
Running Total Power Query with Treshold
Power Query All Over Running Totals
Power Query Running Total with Grouping
Link to Solussion
Here are Two Codes which resolve my problem thanks to chaps on PowerQueryForum... link below
The Function
fxCalc
(A)=>
let
RunningTotal = Table.AddColumn(
A,
"Running Total",
each
if [Volume] <
(Number.RoundUp(List.Sum(List.InsertRange(List.FirstN(A[Volume],
[Index]),0,{0})) / [Cube], 0) * [Cube])
-
List.Sum(List.InsertRange(List.FirstN(A[Volume],[Index]),0,{0}))
then #RunningTotal[Running Total]
{[Index]-1} + [Volume]
else List.Sum(List.FirstN(A[Volume],
[Index]+1))
+
((Number.RoundUp(List.Sum(List.InsertRange(List.FirstN(A[Volume],
[Index]),0,{0})) / [Cube], 0) * [Cube])
-
List.Sum(List.InsertRange(List.FirstN(A[Volume],[Index]),0,
{0})))
),
CubeFill = Table.AddColumn(
RunningTotal,
"Cube Fill",
each [Running Total]/[Cube]
),
PositionCount = Table.AddColumn(
CubeFill,
"Position Count",
each Number.RoundUp([Cube Fill],0)
),
RemainingSpace = Table.AddColumn(
PositionCount,
"Remaining Space",
each [Position Count] * [Cube] - [Running
Total]
)
in
RemainingSpace
Main Query
let
Source =
Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText
("i45Wc
lTSUTLQMzACU4amSrE6cDFTLGLmRKqzRBZzwmKHExa9TljscMYpZmKKRR3C3lgA",
BinaryEncoding.Base64)
, Compression.Deflate)), let _t = ((type
nullable text) meta [Serialized.Text = true]) in type table
[Filter = _t, Volume = _t, Cube = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Filter",
type text}, {"Volume", type number}, {"Cube", type number}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Filter"},
{{"Group", each fxCalc(Table.AddIndexColumn(_,"Index",0,1))}}),
Combine = Table.Combine(#"Grouped Rows"[Group])
in
Combine
Option 2
let
Source =
Table.FromRows(Json.Document(Binary.Decompress
(Binary.FromText("i45WclTSUTLQMzACU4amQMpQKVYHLm6KEDdCFjdHiBvjUG+
CLG6JEDcFizth2GuGLI5kjjmyOJK9FmBxZwxxSyRxEyRzDA2QNSA5yBDo41gA",
BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type
nullable text) meta [Serialized.Text = true]) in type table [Filter
= _t, Volume = _t, Cube = _t, Index = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Filter",
type text}, {"Volume", type number}, {"Cube", type number},
{"Index", type text}}),
Aggregate = List.Accumulate(Table.ToRecords(#"Changed Type"), {},
(a,n)=> a & {
Record.AddField(Record.AddField(n, "CubeFill", n[Volume] + (if
List.IsEmpty(a) or List.Last(a)[CubeFill] + n[Volume] > n[Cube] or
List.Last(a)[Filter] <> n[Filter] then 0 else List.Last(a)
[CubeFill])),
"PositionCount", if List.IsEmpty(a) or List.Last(a)[Filter] <>
n[Filter] then 1 else if List.Last(a)[CubeFill] + n[Volume] >
n[Cube] then List.Last(a)
[PositionCount]+Number.RoundUp(n[Volume]/n[Cube], 0) else
List.Last(a)[PositionCount])}),
Output = Table.FromRecords(Aggregate)
in
Output
I have a Power query that finds and replaces values listed in a table that I work through from here Bulk Find And Replace In Power Query
But I need to apply it to All columns.
How to do this without listing all the columns as they are dynamic and keep changing
Thanks
What I have so far
let
Source = Excel.CurrentWorkbook(){[Name="MyData"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Job Title", type text}}),
BulkReplaceStep = fBulkReplace(#"Changed Type", MyFindReplace, {"Job Title","Job Title2"})
in
BulkReplaceStep
The find/replace data table
let
Source = Excel.CurrentWorkbook(){[Name="MyFindReplace"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Find", type text}, {"Replace", type text}})
in
#"Changed Type
Bulkreplace
let BulkReplace = (DataTable as table, FindReplaceTable as table, DataTableColumn as list) =>
let
//Convert the FindReplaceTable to a list using the Table.ToRows function
//so we can reference the list with an index number
FindReplaceList = Table.ToRows(FindReplaceTable),
//Count number of rows in the FindReplaceTable to determine
//how many iterations are needed
Counter = Table.RowCount(FindReplaceTable),
//Define a function to iterate over our list
//with the Table.ReplaceValue function
BulkReplaceValues = (DataTableTemp, n) =>
let
//Replace values using nth item in FindReplaceList
ReplaceTable = Table.ReplaceValue(
DataTableTemp,
//replace null with empty string
if FindReplaceList{n}{0} = null then "" else FindReplaceList{n}{0},
if FindReplaceList{n}{1} = null then "" else FindReplaceList{n}{1},
Replacer.ReplaceText,
DataTableColumn
)
in
//if we are not at the end of the FindReplaceList
//then iterate through Table.ReplaceValue again
if n = Counter - 1
then ReplaceTable
else #BulkReplaceValues(ReplaceTable, n + 1),
//Evaluate the sub-function at the first row
Output = BulkReplaceValues(DataTable, 0)
in
Output
in
BulkReplace
This works
Change this:
BulkReplaceStep = fBulkReplace(#"Changed Type", MyFindReplace, {"Job Title","Job Title2"})
To This:
BulkReplaceStep = fBulkReplace(#"Changed Type", MyFindReplace, Table.ColumnNames(#"Changed Type"))
If it possible to somehow refer to the previous value in the same column? I know there is option to use it in next column.
But it not really fit for me because I need some logic like this. If (calculation value>check value; yes value; previous value). I got error " A cyclic reference was encountered during evaluation " When I am trying refer back.
IF I understand what you want to do correctly, you can accomplish that with the List.Generate function. You generate a list according to your rules; then combine it with the original table.
M Code
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUTJVitWJBpI6SsZgljmQZQJmGQFZlmCWMZBlBGaZAVlmSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}, {"Column2", Int64.Type}}),
//create conditional adding column
c1 = #"Changed Type"[Column1],
c2 = #"Changed Type"[Column2],
conditionalAdd = List.Generate(
()=>[res= if c1{0} + c2{0} > 10 then c1{0}+c2{0} else 0, idx=0],
each [idx] < List.Count(c1),
each [res=if c1{[idx]+1} + c2{[idx]+1} > 10 then c1{[idx]+1} + c2{[idx]+1} else [res],idx=[idx]+1],
each [res]),
//combine with original table
newTable =
Table.FromColumns(
Table.ToColumns(#"Changed Type") & {conditionalAdd},
Table.ColumnNames(#"Changed Type") & {"Conditional Add"}
)
in
newTable
Source
newTable
I need to fit all the values of a column in Power Query into a 1-cell string separated by commas, as the example below:
To do this, I have the following piece of code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Transposed Table" = Table.Transpose(Source),
#"Merged Columns" = Table.CombineColumns(#"Transposed Table",{"Column1", "Column2", "Column3"},Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Merged"),
#"KeepString" = #"Merged Columns"[Merged]{0}
in
#"KeepString"
The problem with this code is that it assumes there will always be 3 columns, which is not always the case. How can I merge all columns (regardless of how many there are) into one?
You can do this with List.Accumulate:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
KeepString = List.Accumulate(Source[User], "", (state, current) => if state = "" then current else state & "," & current)
in
KeepString
You can also use Table.ColumnNames to get the list of all the column names. You can pass this into Table.CombineColumns, so your modified solution would be:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Transposed Table" = Table.Transpose(Source),
#"Merged Columns" = Table.CombineColumns(#"Transposed Table", Table.ColumnNames(#"Transposed Table"),Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Merged"),
#"KeepString" = #"Merged Columns"[Merged]{0}
in
#"KeepString"
You can also use a shorter code, like this:
let
Source=Excel.CurrentWorkbook( {[Name="Table1"]}[Content],
Result = Text.Combine(Source[User], ",")
in
Result