I have a table as indicated below where the Category column could have multiple categories each with its own scale which is shown in the Value column. I would like to create a new column that multiplies all the values where the item column does not equal 'scale' by the corresponding scale value that matches its category.
In the example below: if Category = A then multiply the value by 10
I've been stuck with this for a while and would truly appreciate any help.
Note - there are hundreds of categories and each one could have a different value. In the actual data, the category is a date and for each of those dates, there is a corresponding scale value that the remainder of the data that matches that date will need to be multiplied by.
Original Table
Item
Category
Value
Scale
A
10
Scale
B
5
Scale
C
2
Apples
A
100
Fruit
B
10
Car
C
15
Pear
A
20
Lemon
B
5
New Column added
Item
Category
Value
Calc_Val
Scale
A
10
null
Scale
B
5
null
Scale
C
2
null
Apples
A
100
1000
Fruit
B
10
50
Car
C
15
30
Pear
A
20
200
Lemon
B
5
25
Assuming your first table is named Table1, create a new table, Table2, via
let
Source = Table1,
#"Filtered Rows" = Table.SelectRows(Source, each ([Item] = "Scale"))
in
#"Filtered Rows"
Then create a third table, via
let
Source = Table.NestedJoin(
Table1,
{"Category"},
Table2,
{"Category"},
"Table2",
JoinKind.LeftOuter
),
#"Expanded Table2" = Table.ExpandTableColumn(Source, "Table2", {"Value"}, {"Table2.Value"}),
#"Added Custom" = Table.AddColumn(
#"Expanded Table2",
"Custom",
each if [Item] = "Scale" then null else [Value] * [Table2.Value]
)
in
#"Added Custom"
Have a lookup table with each category and the matching multiplication factor, then use the following formula:
=C2*INDEX(H$2:H$5,MATCH(A2,G$2:G$5,0))
Related
I'm new to this forum and I need your help in PowerQuery.
What I would like to do:
I have a list with expected stock changes for article. On the one hand it could be a inrease of the stock in case of incoming goods from an order which I made to my suppliers. On the other hand it could be a decrease in case of outgoing goods from a sales order. I would like to have a list which is writing the article code, the actual stock, the aamount of the decrease/increase of the change and the stock of the article after the change.
For example:
Row Column1 (A) Column2(B) Column3 (C) Column4 (D)
1 Article actual stock amount of change stock after change
2 A 5 -1 4
3 A 5 -2 2
4 A 5 -1 1
5 B 4 -1 3
The stock is always the same because the changes are expected and so for the future.
In Excel this would be a "easy"solution for me to calculate column4.
D2 = IF(A1=A2;D1+C2;B2+C2)
So I'm referencing to the value above the actual row in column4 if the arcicle is still the same. But how I have to do this in PowerQuery.
What I tried in PowerQuery? I added two index columns to the table:
Column1 (A) Column2(B) Column3 (C) Column4 (D) Index(colum5) Index.1(column6)
Article actual stock amount of change stock after change
A 5 -1 4 0 1
A 5 -2 2 1 2
A 5 -1 1 2 3
B 4 -1 3 3 4
I combined this two table by the index and added the artile again in this table.
I added a new conditional column called "Häufigkeit" where a "Doulbe" should be insert if article in column1 is equal to article1 in the new added column7, otherwise "single". To get the value from the row above I for my calculation tried this code:
"= Table.AddColumn(#"addcolum", ""ConditionalColumn", each if [Häufigkeit]="Double" then ([colum4]{-1}+[colum3]) else [colum2+colum3])"
This doesn't work.
So how I can fix this problem. Thanks in advance
See if this helps. It pulls the value from the row below if article matches
If you need the row above, change each instance of [Index]+1 to [Index]-1
Otherwise please make your question less confusing
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each try if [article]= #"Added Index"{[Index]+1}[article] then #"Added Index"{[Index]+1}[value] else null otherwise null),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"})
in #"Removed Columns"
I have two tables and am trying to figure out how to create custom code to add dynamic columns with custom names that are based off of row values in another table. I then need to use the values of rows in Table 2 to not only create the column names but also fill the new dynamic Columns with a value from another column in Table 2. Hopefully my pictures below help
Table 1 has varying amount of rows depending on what the user input.
Table 2 has varying amount of rows depending on how many values the user inputs.
Table 1 Before
Col1
Col2
Col 3
stuff 1
stuff 2
stuff 3
stuff 4
stuff 5
stuff 6
.
.
.
.
.
.
Table 2
Name
Values
Name1
100
Name2
500
.
.
NameX
Y
Table 1 After
Col1
Col2
Col 3
"Column" & Name1
"Column"& Name2
...
"Column"& NameX
stuff 1
stuff 2
stuff 3
100
500
...
Y
stuff 4
stuff 5
stuff 6
100
500
...
Y
.
.
.
100
500
...
Y
.
.
.
100
500
...
Y
The "Column" & Name1 meaning I want to concatenate Column with the values in the Name column in Table 2.
You can make this dynamic by not referring to the absolute column names, but rather using the Table.ColumnNames function to return those names.
I did assume that the column names in Table 2 are fixed. If not, that code can be changed.
Read the code comments and examine the Applied Steps window to better understand the methods used. There are examples of setting the data type, and also re-naming columns without referring to a hard-coded column name.
M Code
let
//read in the two tables and set the data types
Source1 = Excel.CurrentWorkbook(){[Name="Table_2"]}[Content],
Table2 =Table.TransformColumnTypes(Source1,
{{"Name", type text},{"Values", type any}}),
Source = Excel.CurrentWorkbook(){[Name="Table_1_Before"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,
List.Transform(Table.ColumnNames(Source), each {_, type text})),
//create the extra columns by
//Transpose Table2
// Use first row as headers
xpose = Table.Transpose(Table2),
#"Promoted Headers" = Table.PromoteHeaders(xpose, [PromoteAllScalars=true]),
#"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",
List.Transform(Table.ColumnNames(#"Promoted Headers"), each {_, type any})),
//rename the columns
renameNameCols = Table.RenameColumns(#"Changed Type1",
List.Zip(
{Table.ColumnNames(#"Changed Type1"),
List.Transform(Table.ColumnNames(#"Changed Type1"), each "Column " & _)})),
//Combine the tables
combine = Table.Combine({#"Changed Type",renameNameCols}),
//fill up the original table 2 columns and remove the blank Table 1 rows
#"Filled Up" = Table.FillUp(combine,Table.ColumnNames(renameNameCols)),
#"Filtered Rows" = Table.SelectRows(#"Filled Up", each ([Col1] <> null))
in
#"Filtered Rows"
Original Tables
Results
Note that I did NOT add logic to avoid prepending the ... with the word column, as shown in your desired output, but that is easily added if really needed
My version
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
custom = Table.FromColumns(Table.ToColumns(Source) &Table.ToColumns(Table2), List.Combine({Table.ColumnNames(Source),Table.ColumnNames(Table2)}) ),
#"Filled Down" = Table.FillDown(custom,Table.ColumnNames(Table2))
in #"Filled Down"
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I keep trying ways to fix a dataset, but keep running into problems because of how inconsistent it is.
Here's what the data looks like:
Entry1
Age
45
Occupation
Scientist
Phone Number
408-283-3721
User I.D.
390842
Housing Type
Condo
Square Footage
1073.29
Floors
2
Bathrooms
2.5
Budget Max
$289,287
Household Size
3
Pets?
Yes
Entry2
Floors
2
Square Footage
1974.19
User I.D.
379733
Phone Number
312-246-9121
Pets?
No
Budget Max
$481,621
Household Size
4
Bathrooms
3
Housing Type
Apartment
Occupation
Pilot
Age
32
Entry3
User I.D.
379621
Floors
1
Square Footage
1223.12
Pets?
No
Occupation
Managing Director
Budget Max
$402,342
Phone Number
714-343-1358
Household Size
2
Age
31
Bathrooms
2
Housing Type
House
I want to create a new, cleaned dataset with headers along the top (e.g. "Age", "Occupation", etc) and the values associated (to the right of each variable name cell) as the row, underneath each column.
The variable names are all mixed up, not always on the same column or relative row, so it's not only transposing into a clean new dataset but finding the appropriate values depending on where the variable is (so, I'm thinking something like .Cells.Find(What:="the variable name") for each one and somehow returning the value next to it in a loop). Then, there's the issue where some entries have 3 rows and 8 columns and others 4 rows and 6 columns (not all rows being full too). I also struggle with placing the values under the appropriate column header and not replace the former value. (i.e not just changing one cell but adding to the one below and so on)
There are over 400 records like this, so doing it manually would be super tedious. I'm fairly certain these are all the variations though.
Loop through the data row by row.
If only the first column has data it is the header of an entry. Write that to a new workbook column A.
Enrty Name
Entry1
Then go to the next row. If more than 2 columns have data it is a data row to the previous entry. Data rows contain data in blocks of 2 cells, where the first block is the data description and the second cell the data value.
So you need to loop through the columns of the data rows in blocks of 2:
Take the first block which is Age | 45
Check if the column Age exists. Here it does not so we name the next free column Age and fill in the data to the last enty
Enrty Name
Age
Entry1
45
Then we move on to the next block Occupation | Scientist and do the same. Check if a column Occupation exists? No, ok insert next free column:
Enrty Name
Age
Occupation
Entry1
45
Scientist
We do this until the entire row is done, then we move over to the next one and if this is a data row too, we keep going until we find a new entry header.
So after the first entry your data would look like this:
Enrty Name
Age
Occupation
Phone Number
User I.D.
Housing Type
Square Footage
Floors
Bathrooms
Budget Max
Entry1
45
Scientist
408-283-3721
390842
Condo
1073.29
2
2.5
$289,287
Yes
Then you move over to the next entry
Enrty Name
Age
Occupation
Phone Number
User I.D.
Housing Type
Square Footage
Floors
Bathrooms
Budget Max
Entry1
45
Scientist
408-283-3721
390842
Condo
1073.29
2
2.5
$289,287
Yes
Entry2
The first data set here is Floors | 2, so you search in the first row for Floors it is found in column 8. So we write 2 into column 8.
Enrty Name
Age
Occupation
Phone Number
User I.D.
Housing Type
Square Footage
Floors
Bathrooms
Budget Max
Entry1
45
Scientist
408-283-3721
390842
Condo
1073.29
2
2.5
$289,287
Yes
Entry2
2
If you keep that going you have cleaned up data in the end.
If your real data corresponds to your example, where all the parameters are spelled identically, you can do this using Power Query.
If there are variations in your data that this table doesn't show, examples of these variations would be needed to craft a better solution.
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
Select My data does NOT have headers
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
M Code (Modified to deal with missing Parameter Values)
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
//Add grouping column Entries and Unpaivot
#"Added Custom" = Table.AddColumn(Source, "Entry", each
if Text.StartsWith([Column1],"entry",Comparer.OrdinalIgnoreCase) then [Column1] else null),
#"Filled Down" = Table.FillDown(#"Added Custom",{"Entry"}),
//Remove extra entry rows
remRows = Table.SelectRows(#"Filled Down", each [Entry] <> [Column1]),
//Table.ReplaceValue(#"Removed Columns1"," ",null,Replacer.ReplaceValue,{"Value"})
//Replace nulls with space so we don't lose one item of a "pair"
#"Replaced Value" = Table.ReplaceValue(remRows,null," ",Replacer.ReplaceValue,Table.ColumnNames(remRows)),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Replaced Value", {"Entry"}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Attribute"}),
#"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1, Int64.Type),
#"Inserted Integer-Division" = Table.AddColumn(#"Added Index", "Integer-Division", each Number.IntegerDivide([Index], 2), Int64.Type),
#"Removed Columns1" = Table.RemoveColumns(#"Inserted Integer-Division",{"Index"}),
//Group in pairs
//Mark blank subTables
//Extract Entry, Parameter and Value
#"Grouped Rows" = Table.Group(#"Removed Columns1", {"Integer-Division"}, {
{"Empties", each List.NonNullCount(List.ReplaceValue(_[Value]," ", null,Replacer.ReplaceValue))},
{"Entry", each _[Entry]{0}},
{"Parameter", each _[Value]{0}},
{"Value", each _[Value]{1}}
}),
#"Filtered Rows" = Table.SelectRows(#"Grouped Rows", each ([Empties] <> 0)),
#"Removed Columns2" = Table.RemoveColumns(#"Filtered Rows",{"Integer-Division", "Empties"}),
//Group by Entry, Pivot and expand
#"Grouped Rows1" = Table.Group(#"Removed Columns2", {"Entry"}, {
{"Pivot", each Table.Pivot(_, _[Parameter], "Parameter","Value")}
}),
//Column name list for the Pivot table.
//Could sort or order them any way you want
colNames = List.Distinct(#"Removed Columns2"[Parameter]),
#"Expanded Pivot" = Table.ExpandTableColumn(#"Grouped Rows1", "Pivot", colNames,colNames)
in
#"Expanded Pivot"
Original Data
Transformed
I have produced a table like the one below by using 'group' function in excel power query
score 1 score 2 score 3
A 6 25 50
B 8 30 20
C 15 15 30
D 20 0 10
I want to add a totals row (equivalent to "show totals for column" in a normal pivot table), so result would be like this
score 1 score 2 score 3
A 6 25 50
B 8 30 20
C 15 15 30
D 20 0 10
Total 49 70 110
Anyone knows if there is a simple way to do this? Thank you, RY
Another way:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
group = Table.Group(Source, {}, {{"letter", each "Total"},
{"score 1", each List.Sum([score 1])},
{"score 2", each List.Sum([score 2])},
{"score 3", each List.Sum([score 3])}}),
append = Table.Combine({Source, group})
in
append
Or:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
cols = Table.ColumnNames(Source),
group = Table.Group(Source, {}, List.Zip({cols, {each "Total"}&
List.Transform(List.Skip(cols),
(x)=>each List.Sum(Table.Column(_,x)))})),
append = Table.Combine({Source, group})
in
append
Or:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
group = Table.Group(Source, {}, List.TransformMany(Table.ColumnNames(Source),
(x)=>{each if x = "letter" then "Total"
else List.Sum(Table.Column(_,x))}, (x,y)=>{x,y})),
append = Table.Combine({Source, group})
in
append
I don't know why you would do this in PowerQuery rather than a pivottable but the only way I can think of is to duplicate the table, unpivot the columns and then repivot it using sum as an aggregation. Then you could append the table to your orginal query.
Your code would look something like for the table you want to create the totals in.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {}, "Attribute", "Value"),
#"Pivoted Column" = Table.Pivot(#"Unpivoted Columns", List.Distinct(#"Unpivoted Columns"[Attribute]), "Attribute", "Value", List.Sum)
in
#"Pivoted Column"
Note you may have to choose to aggregate by count but you can change the code to sum (On the #"Pivot Column" line change List.Count to List.Sum.
You will get an error for the A,B,C,D col - you can replace this with Total if you would like by using the Replace Errors function.
If you are wanting to add a total row to a Power Query that was loaded to a table. Just skip a row and add a total row. This seems to work for me. Just make sure you lock the first cell in your sum formulas. ex =SUM($C$2:C25)
Example 1
Example 2
I have an excel pivot table (and a table dataset behind) that has the structure like the one below. How can I filter/show only companies (Col A) with Products (Col B) 1 AND 2 AND 3? Sounds like something easy but can't find a way to do that. No problem by achieving this using Power Query (available in Power BI or Excel).
A1: Company 1 | B1: Product 1
A2: Company 1 | B2: Product 2
A3: Company 1 | B3: Product 3
A4: Company 1 | B4: Product 4
A5: Company 2 | B5: Product 1
A6: Company 3 | B6: Product 1
A7: Company 4 | B7: Product 1
A8: Company 4 | B8: Product 2
A9: Company 4 | B9: Product 3
A10: Company 4 | B9: Product 4
A11: Company 4 | B9: Product 5
Here's an approach using Power Query.
Starting with this brought into Power Query from the table in Excel:
I then group on Company (Transform > Group By):
Then I add a new custom column (Add Column > Custom Column) to flag whether each company has the 3 products included in its associated grouped table's Product column:
Then I filter out the FALSE entries from the new custom column (use button at top right of Custom column):
Then I expand the Products column from the embedded table in the AllData column (use button at top right of AllData column).
Then I remove the Custom column:
Here's the M code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Company", type text}, {"Product", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Company"}, {{"AllData", each _, type table}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.ContainsAll([AllData][Product], {"Product 1","Product 2","Product 3"})),
#"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([Custom] = true)),
#"Expanded AllData" = Table.ExpandTableColumn(#"Filtered Rows", "AllData", {"Product"}, {"Product"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded AllData",{"Custom"})
in
#"Removed Columns"
Basically, you'll need to do a couple of things to do this entirely in Excel:
Add a new table that lists the products, with a column indicating whether that product is included/flagged:
Update your company/product table to have 2 helper columns: One to VLOOKUP whether the product is flagged, and one to indicate whether a company has all 3 flagged products:
The first helper column would use a formula like =VLOOKUP([#Product],tProducts,2,FALSE).
The second helper column would use a formula like =COUNTIFS([Company],[#Company],[Product Flagged],TRUE)>=3.
Rows with a TRUE in Column D have 1 each of Products 1, 2, and 3 (unless you have rows with duplicate company/product combinations, where it gets a bit trickier):
In your pivot table, you can filter by this helper column: