Merging tables in excel using power query - excel

im using a certain config of tables to generate a cascading dropdown list that looks as such:
I wanna merge these tables to look as such using excels powerquery:

Since each dropdown list is a different table, you have to create a list of all the tables.
Creating a list of tables for PQ to operate on can be tricky.
In my example, they were named Table19 - Table26 so I used the List.Generate function to do that. But there are other ways to generate such a list, including just hard-coding it.
Then it's just a matter of reading in all the tables, unpivoting and expanding.
See the comments in the code for better understanding.
M Code
let
//create a list of the relevant table names
// and create a column of the names
//many ways to do this
//Since mine are table19-26, I do it like this:
tableNames = List.Generate(
()=>[Table Name = "Table" & "19", IDX=20],
each [IDX] < 28,
each [Table Name = "Table" & Text.From([IDX]), IDX = [IDX]+1],
each [Table Name]
),
#"Converted to Table" = Table.FromList(tableNames, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
//Add custom column containing the actual table referred to in column1
//then delete the table names column
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Tables", each Excel.CurrentWorkbook(){[Name=[Column1]]}[Content]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Column1"}),
//add custom column to unpivot each of the tables
//then remove the original tables column and expand the unpivoted column
#"Added Custom1" = Table.AddColumn(#"Removed Columns", "unPivot", each
Table.UnpivotOtherColumns([Tables],{},"Type","Subtype")),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom1",{"Tables"}),
#"Expanded unPivot" = Table.ExpandTableColumn(#"Removed Columns1", "unPivot", {"Type", "Subtype"}, {"Type", "Subtype"})
in
#"Expanded unPivot"

Related

Importing data to Power Query and filter if group of records has at least one conditional value

I hope I'm going to explain myself well enough. I'm importing spreadsheets from a folder into Power Query, but I want to filter out (drop) any groups of data (by a name column) where that group has at least one instance of a given value.
Example, I have a theoretical table with two columns. One has repeating names and the other has scattered values of A or B or C. I want the query to look for a group of names, then see if any of the records in that group has either and A, B or C in the second column. If found, it drops that entire group and if none, then it allows that entire group through.
I'm not sure if this is too complex for Power Query or whether I need to do this outside Excel. If the latter, what would the equivalent SQL statement be (for MS Access, as I'm trying to keep this as basic as possible)?
Theres about 500 ways to do this. One easy way is to filter Column2 for ABC, remove that column, then remove duplicates from Column1. That give you a unique list of groups to remove, and just merge that against the original data and remove those rows
sample:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
// all unique groups from Col1 that have a/b/c in Col2
#"Filtered Rows" = Table.SelectRows(Source, each ([Column2] = "a" or [Column2] = "b" or [Column2] = "c")),
#"Removed Columns"= Table.SelectColumns(#"Filtered Rows",{"Column1"})
#"Removed Duplicates" = Table.Distinct(#"Removed Columns"),
// merge that into original table and filter
#"Merged Queries" = Table.NestedJoin(Source,{"Column1"},#"Removed Duplicates",{"Column1"},"Table2",JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"Column1"}, {"Column1.1"}),
#"Filtered Rows1" = Table.SelectRows(#"Expanded Table2", each ([Column1.1] <> null)),
#"Removed Columns1" = Table.RemoveColumns(#"Filtered Rows1",{"Column1.1"})
in #"Removed Columns1"

Excel: Combine data from 2 tables

I am trying to achieve something I believe will be quite simple but just can't figure out on my own.
What I'm trying to achieve is a full list of product IDs + Filenames. All filenames are the same for each product ID.
Table 1: Product ID's
Table 2: File Names
Goal: Create a record for each product ID + Filename combination
Thanks in advance.
Assuming the file name table is loaded into PowerQuery and the table is named File_names_table, then
In the in the Product ID table, add column .. custom column ... and use formula:
= File_names_table
Then click the arrows atop the Custom column and expand to new rows
let Source = Excel.CurrentWorkbook(){[Name="Product_ID_Table"]}[Content],
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Added Custom" = Table.AddColumn(#"Promoted Headers", "Custom", each File_names_table),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"File names"}, {"File names"})
in #"Expanded Custom"

Excel Power Query inserting column between other columns

I'm importing a bunch of columns to do some analysis on in Excel power query. Some of the analysis columns need to be inserted after a certain column, but every option for adding a column only lets me append the column to the very end. I want to insert the new columns after the one named "Total" for readability.
Bellow a function than outputs the list of re-arranged column names.
ReorderList:
(tableName as table, toBeMovedColumnName as any, optional afterColumnName as text) as list=>
//tableName - the name of the table we want to reorder.
//toBeMovedColumnName - the name of the column you want to change the position. Can be a list of column names.
//columnName - the name of the column you want the toBeMovedColumnName to be positioned after. If omited toBeMovedColumnName will be placed as the first column.
let
columnNames = Table.ColumnNames(tableName),
positionOf = if afterColumnName is null or afterColumnName = "" then 0 else List.PositionOf(columnNames, afterColumnName) + 1,
toBeMovedList = if Value.Is(toBeMovedColumnName, type list) = true then toBeMovedColumnName else {toBeMovedColumnName},
intermediaryList = List.Combine({List.FirstN(columnNames,positionOf),toBeMovedList}),
intermediaryList2 = List.RemoveItems(columnNames,intermediaryList),
reorderList = List.Combine({intermediaryList,intermediaryList2})
in
reorderList
Usage like this:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom1", each 4),
#"Reordered Columns" = Table.ReorderColumns(#"Added Custom", ReorderList(#"Added Custom","Custom1","Total"))
in
#"Reordered Columns"
Sample below.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
// get baseline column names. Use this before inserting new analysis columns
Names = Table.ColumnNames(Source),
TotalSpot = List.PositionOf(Names,"Total"),
// add any code or steps here ; this is random sample. don't use
#"Added Custom" = Table.AddColumn(Source, "Custom1", each 4),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom2", each 5),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "Custom3", each 6),
// insert this after all your new columns are added
// it moves all new columns to the right of the Total column
// replace #"Added Custom2" in step below with previous step name
#"Reordered Columns" = Table.ReorderColumns(#"Added Custom2",List.Combine ({List.FirstN(Names,TotalSpot+1),List.RemoveItems(Table.ColumnNames(#"Added Custom2"),Names),List.RemoveFirstN (Names,TotalSpot+1)}))
in #"Reordered Columns"

How to Merge Query for multiple column

I have a table of information like this:
And a lookup table for user names to IDs:
How do I do a Merge on each column to lookup the values from the other table so I get this result:
I do not want to manually apply an action to each role column, because the list of roles may grow or shrink. So the solution needs to all columns (except the first) in the table.
Can this be done?
Basically this calls for unpivot on the Project data, merge to the other table, then re-pivot to get back in proper order
Steps:
Load in the ID data; here I am assuming it is loaded in query ID_Table
Load in Project data; here I am assuming it is loaded in range Projects
In the project query, right-click the first (project) column, unpivot other columns
Home ... Merge queries...
Merge the two tables using the Value column in the project query and the Person column in the ID_Table query, and use Left Outer merge
Expand results using double arrows atop column and uncheck all except ID
Right-click the value column and remove
Click attribute column ... transform .. pivot column...
Use ID as value column ... advanced options ... dont aggregate
sample code
let Source = Excel.CurrentWorkbook(){[Name="Projects"]}[Content],
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Promoted Headers", {"Project"}, "Attribute", "Value"),
#"Merged Queries" = Table.NestedJoin(#"Unpivoted Other Columns",{"Value"},ID_Table,{"Person"},"ID_Table",JoinKind.LeftOuter),
#"Expanded ID_Table" = Table.ExpandTableColumn(#"Merged Queries", "ID_Table", {"ID"}, {"ID"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded ID_Table",{"Value"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[Attribute]), "Attribute", "ID")
in #"Pivoted Column"

PowerQuery : add premade column without applying function to each cell

I have a query function :
let ID = () =>
let
Source = Json.Document(Web.Contents("www.example.com")),
#"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
#"Converted to Table"
in ID
that outputs a table.
What I'm trying to do is add a column from that table I'm outputting to my main query table.
My problem :
I don't know how to add a column without having to define a function that will apply to each row (as in "Add a Custom Column), and insted just add a premade column.
The solution I came up with :
Add an index column to my main query table and then using the "Costum Column" functionnality to call the Nth row of my ID column as such :
let
Source = ID(),
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each ID()[Column1]{[Index]})
in
#"Added Custom"
My question :
Isn't there just an existing function that will allow me to do this :
Table.AddPremadeColumn(#"Added Index", "Custom", ID()[Column1])
You can convert the table to columns (i.e. a list of lists), concatenate {ID[Column1]} and convert the result back to a table:
= Table.FromColumns(Table.ToColumns(#"Added Index")&{ID[Column1]})

Resources