Transpose data based on the proper pattern - excel

This is what I want the date to look like when everything is all done and I transpose the data.
Data
2 Witches Winery and Brewing Company
209 Trade Street
Danville, VA 24541-3545
Phone: (434) 549-2739
Type: Taproom
www.2witcheswinebrew.com
View Map
36 Fifty Brewing
120 N Chestnut St
Marion, VA 24354
Type: Taproom
View Map
6 Bears & A Goat Brewing Company, LLC
1140 International Pkwy
Fredericksburg, VA 22406-1126
Phone: 540-356-9056 Ext. 2
Type: Brewpub
www.6bgbrewingco.com
View Map
Each block of cells represents ONE brewery. I am trying to transpose and put this value into rows. Here is the problem…. Not all the values are in the correct place. The first 3 rows are always same for every single brewery. When it gets to the 4th row of each brewery, that is where it gets tricky. Not all the breweries have a phone, so transposing the data makes all the data not in the right spot. The type should typically be in the “5” row, but since there is no number, it is in the 4th row. About 20% of the data is like this. Anyone have any recommendations.
Apologies, edit forgot to add what I have tried, but it doesn't work as expected.
// Table2
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Helper_Column", each if Text.Contains([Column1],"Phone:") then "1 #1" else null),
#"Removed Errors" = Table.RemoveRowsWithErrors(#"Added Custom", {"Helper_Column"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Removed Errors", {{"Helper_Column", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Helper_Column"),
#"Added Custom1" = Table.AddColumn(#"Split Column by Delimiter", "Helper Column 1", each if [Helper_Column] = "#1" then null else [Column1]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Helper Column 1"}),
#"Added Custom2" = Table.AddColumn(#"Removed Other Columns", "Helper Column", each if Text.Contains([Helper Column 1],"View Map") then "1 #1" else null),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Added Custom2", {{"Helper Column", null}}),
#"Split Column by Delimiter1" = Table.ExpandListColumn(Table.TransformColumns(#"Replaced Errors", {{"Helper Column", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Helper Column"),
#"Added Custom3" = Table.AddColumn(#"Split Column by Delimiter1", "Helper", each if [Helper Column] = "#1" then null else [Helper Column 1]),
#"Removed Other Columns1" = Table.SelectColumns(#"Added Custom3",{"Helper"}),
#"Added Index" = Table.AddIndexColumn(#"Removed Other Columns1", "Index", 0, 1, Int64.Type),
#"Inserted Modulo" = Table.AddColumn(#"Added Index", "Modulo", each Number.Mod([Index], 8), type number),
#"Integer-Divided Column" = Table.TransformColumns(#"Inserted Modulo", {{"Index", each Number.IntegerDivide(_, 8), Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Integer-Divided Column", {{"Modulo", type text}}, "en-IN"), List.Distinct(Table.TransformColumnTypes(#"Integer-Divided Column", {{"Modulo", type text}}, "en-IN")[Modulo]), "Modulo", "Helper")
in
#"Pivoted Column"

It depends on how realistic your example is. But the code below may help. It works on your posted data.
But you need to have unambiguous rules.
I derived some from your data and what you wrote, and noted them in the code comments. Of course, if your actual data doesn't follow these rules, the algorithm will not work. And if that is the case, you will have to modify the rules.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//assuming each group is contiguous lines
// with a blank line inbetween each group
// the below few lines will create a column on which to group
// then remove the "blank line between"
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "group", each if [Column1] = null then [Index] else null, Int64.Type),
#"Filled Up" = Table.FillUp(#"Added Custom",{"group"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Up",{"Index"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([Column1] <> null)),
//Group, then extract the data
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"group"}, {
//Line one is always the brewery name
{"Brewery Name", each [Column1]{0}, type text},
//Lines 2 and 3 are always the address
{"Address Part 1", each [Column1]{1}, type text},
{"Address Part 2", each [Column1]{2}, type text},
//Phone number starts with "Phone:"
{"Phone", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"Phone:") then state & current else state), type text},
//Type starts with "Type:"
{"Type", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"Type:") then state & current else state), type text},
//Other 1 starts with "www."
{"Other 1", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"www.") then state & current else state), type text},
//Other 2 is the last line
{"Other 2", each List.Last([Column1]), type text}
}),
//Remove the grouper column
#"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"group"})
in
#"Removed Columns1"
Data
Results

Related

Excel Unpivot Multiple Columns to a Single Column

What I want to achieve:
As the title says, is there any way to convert my table structure? I've tried using Power Query but it didn't work. Any kind of help is greatly appreciated. Thanks!
So, I was trying to pivot type and type value, but it seems impossible if I maintain the current table structure since it would cause duplicates when I wanted to aggregate on type.
Should I remake the table structure or there is any way to get around this problem?
Thanks in advance!
In Power Query, the following is adaptable to any number of type/value column pairs.
Unpivot all except the ID column
Add a custom column to define if the unpivoted value is a Type or a Type Value
Add an Index column and then do an integer/divide by 2 so things will sort in the desired order
Pivot with no aggregation, using a custom function as the "built-in" function will error with multiple items.
Custom function to Pivot with No aggregation
Rename as noted in comments
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/
//Rename: fnPivotAll
(Source as table,
ColToPivot as text,
ColForValues as text)=>
let
PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))),
#"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _),
TableFromRecordOfLists = (rec as record, fieldnames as list) =>
let
PartialRecord = Record.SelectFields(rec,fieldnames),
RecordToList = Record.ToList(PartialRecord),
Table = Table.FromColumns(RecordToList,fieldnames)
in
Table,
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)),
#"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames),
#"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames)
in
#"Expanded Values"
Regular Query
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"ID", type text}, {"Type 1", type text}, {"Type 1 Value", Int64.Type}, {"Type 2", type text}, {"Type 2 Value", Int64.Type}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"),
#"Added Custom" = Table.AddColumn(#"Unpivoted Other Columns", "Custom",
each if Text.EndsWith([Attribute],"Value") then "Value" else "Type"),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Attribute"}),
#"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1, Int64.Type),
#"Inserted Integer-Division" = Table.AddColumn(#"Added Index", "Integer-Division", each Number.IntegerDivide([Index], 2), Int64.Type),
#"Removed Columns1" = Table.RemoveColumns(#"Inserted Integer-Division",{"Index"}),
Pivot = fnPivotAll(#"Removed Columns1","Custom","Value"),
#"Removed Columns2" = Table.RemoveColumns(Pivot,{"Integer-Division"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Removed Columns2",{{"Type", type text}, {"Value", Int64.Type}})
in
#"Changed Type1"
More generically to stack vertically in powerquery while keeping certain columns
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
base_columns=1, groupsof=2, //stack them
Combo = List.Transform(List.Split(List.Skip(Table.ColumnNames(Source),base_columns),groupsof), each List.FirstN(Table.ColumnNames(Source),base_columns) & _),
#"Added Custom" =List.Accumulate(Combo, #table({"Column1"}, {}),(state,current)=> state & Table.Skip(Table.DemoteHeaders(Table.SelectColumns(Source, current)),1)),
#"Rename"=Table.RenameColumns(#"Added Custom",List.Zip({Table.ColumnNames(#"Added Custom"),List.FirstN(Table.ColumnNames(Source),base_columns+groupsof)}))
in #"Rename"
What seems to be fastest method of those I've tested
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
leading=1, groupsof=2,
#"Added Custom" = Table.AddColumn(Source, "Custom", each List.Split( List.RemoveFirstN(Record.ToList( _),leading), groupsof) ),
#"Added Custom0" = Table.AddColumn(#"Added Custom", "Custom0", each Text.Combine(List.FirstN(Record.ToList(_),leading),"|")),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom0",{"Custom0", "Custom"}),
#"Expanded Custom" = Table.ExpandListColumn( #"Removed Other Columns", "Custom"),
#"Extracted Values" = Table.TransformColumns(#"Expanded Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), "|"), type text}),
#"Merged Columns" = Table.CombineColumns(#"Extracted Values",{"Custom0", "Custom"},Combiner.CombineTextByDelimiter("|", QuoteStyle.None),"Custom"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Merged Columns", "Custom", Splitter.SplitTextByDelimiter("|", QuoteStyle.Csv), List.FirstN(Table.ColumnNames(Source),leading+groupsof))
in #"Split Column by Delimiter"
There is probably a better way, but if you first concat the 4 columns with specific unique delimiter to split on later in a custom column, you have a work-around in PQ:
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"Type1", type text}, {"Type1 Val", Int64.Type}, {"Type2", type text}, {"Type2 Val", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom1", each [Type1]&"|"&Number.ToText([Type1 Val])&"$"&[Type2]&"|"&Number.ToText([Type2 Val])),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Type1", "Type1 Val", "Type2", "Type2 Val"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Removed Columns", {{"Custom1", Splitter.SplitTextByDelimiter("$", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Custom1"),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Custom1", type text}}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Changed Type1", "Custom1", Splitter.SplitTextByEachDelimiter({"|"}, QuoteStyle.Csv, false), {"Custom1.1", "Custom1.2"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"Custom1.1", type text}, {"Custom1.2", Int64.Type}})
in
#"Changed Type2"
Just in case you tagged 'Excel-Formula' and you have access to ms365:
Formula in H1:
=REDUCE({"ID","Type","Val"},ROW(A2:A5),LAMBDA(X,Y,VSTACK(X,INDEX(A:E,Y,{1,2,3}),INDEX(A:E,Y,{1,4,5}))))
Or formula:
=SORT(VSTACK(A2:C5,HSTACK(A2:A5,D2:E5)))

Column rows to column headers in Power Query

I have a data set where I would like to have a table with Unique IDs in one Column A and from Column B the rows from a "Input" table above with a different rows as a column headers. In Column A are IDs (unique) and Column B has different rows that have to be in columns but matching values on the rest of the columns.. see on screenshot.
Third Column C is a just observational column that gives info what kind of data type should be there (it can be avoided in this case).
I though I was going to solve it "easily" with Pivot/Unpivot+Transponse method in Power Query but no way....I can get it in one row like in "Output" table..
The dummy data is in link below.
https://docs.google.com/spreadsheets/d/1qKeVj9nJF1usBk-OUZPJfpRqSQnOTCvr/edit?usp=sharing&ouid=101738555398870704584&rtpof=true&sd=true
Merge the value columns into a single column before the pivot, eg
let
Source = Excel.Workbook(File.Contents("C:\Users\david\Downloads\Test1.xlsx"), null, false),
Sheet1_sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
FilterNullAndWhitespace = each List.Select(_, each _ <> null and (not (_ is text) or Text.Trim(_) <> "")),
#"Added Custom" = Table.AddColumn(Sheet1_sheet, "IsEmptyRow", each try List.IsEmpty(FilterNullAndWhitespace(Record.FieldValues(_))) otherwise false),
#"Added Index" = Table.AddIndexColumn(#"Added Custom", "Index", -1),
#"Added Custom1" = Table.AddColumn(#"Added Index", "Section", each if [IsEmptyRow] then -1 else if try #"Added Index"[IsEmptyRow]{[Index]} otherwise true then [Index] else null),
#"Removed Blank Rows" = Table.SelectRows(#"Added Custom1", each not [IsEmptyRow]),
#"Filled Down" = Table.FillDown(#"Removed Blank Rows", {"Section"}),
#"Grouped Rows" = Table.Group(#"Filled Down", {"Section"}, {{"Rows", each _}}, GroupKind.Local),
#"Selected Group" = #"Grouped Rows"[Rows]{1},
#"Removed Columns" = Table.RemoveColumns(#"Selected Group", {"IsEmptyRow", "Index", "Section"}),
#"Promoted Headers" = Table.PromoteHeaders(#"Removed Columns", [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ObjectID", Int64.Type}, {"Feld", type text}, {"Datentyp", type text}, {"boolValue", type text}, {"dateValue", type date}, {"intValue", Int64.Type}, {"stringValue", type text}, {"longStringValue", type text}, {"referencedObjectId", Int64.Type}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type", "Merged", each Text.Combine({[boolValue], Text.From([dateValue], "en-US"), Text.From([intValue], "en-US"), [stringValue], [longStringValue], Text.From([referencedObjectId], "en-US")}, ""), type text),
#"Removed Columns1" = Table.RemoveColumns(#"Inserted Merged Column",{"Datentyp", "boolValue", "dateValue", "intValue", "stringValue", "longStringValue", "referencedObjectId"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[Feld]), "Feld", "Merged")
in
#"Pivoted Column"

Split text column in PowerBI

Hi all, I have a column of text like the one in the picture, how can I split this type of column in multiple column for each occurrence of " |-Starting " substring?
create a "grouper" column to group the different sets of rows
then Group
Split each subgroup into columns
Transpose the results
eg
let
Source = Excel.CurrentWorkbook(){[Name="Table16"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//create a "grouper" column
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "grouper", each if [Column1] = "|-Starting" then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Custom",{"grouper"}),
//group the rows creating a delimiter separated string
// and a counter to obtain the number of columns for the "Split"
#"Grouped Rows" = Table.Group(#"Filled Down", {"grouper"}, {
{"group", each Text.Combine([Column1],";"),type text},
{"numInGroup", each Table.RowCount(_)}
}),
//maximum number of columns in the result
numCols=List.Max(#"Grouped Rows"[numInGroup]),
#"Removed Columns" = Table.RemoveColumns(#"Grouped Rows",{"grouper","numInGroup"}),
//split; then transpose
#"Split Column by Delimiter" = Table.SplitColumn(#"Removed Columns", "group",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv),numCols),
#"Transposed Table" = Table.Transpose(#"Split Column by Delimiter")
in
#"Transposed Table"

Replicate Excel formula logic in power query

If this does not belong here please let me know! or if any more information is needed
providing link to sample data file here - https://www.dropbox.com/s/mz3f24ciby7qthv/example%20PBI%20Community.xlsx?dl=0
Context: rows 18 to 25 contain FORECAST data at the country-customer-grouping grain... and we need to allocate to the finer grain of Label1 and Label2
Rows 2 to 17 contain 2020 data that has the grain we need, so we use the 2020 data to create an apportionment base for the 2021 data which is done by the formulas in columns I and J
Formula/Logic Explanation:
This is in a "database" format. it needs to stay this way even with the addition of new columns
FormulaPart1 is and INDEX of "Base Figure" data based on The Index Key in column N
FormulaPart2 is a SUMIF of "Base Figure" data based on The Index Key in column O
FormulaPart3 is and INDEX of "To be Allocated" data based on The Index Key in column P
Hopefully this is all clear...i would like to move this logic to PQ for efficiency and error minimisation purposes
so any guidance in the right direction would be super useful :)
the real dataset i work with is much much larger and having all these formulas and index key columns in excel sheet is problematic :)
Thank You for your guidance! :)
Assuming range A1:H25 is named Table1, this seems to do what you want
//assumes range A1:H25 from sample data is loaded as named range Table1
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Year", Int64.Type}, {"Label1", Int64.Type}, {"Label2", type text}, {"Type", type text}, {"Country", type text}, {"Customer", type text}, {"Grouping", type text}, {"Original Value", Int64.Type}}),
// add index for re-sorting later
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
// pull Base Figure rows This starts us with Part 1 of formula
#"Filtered Rows" = Table.SelectRows(#"Added Index", each ([Type] = "Base Figure")),
// formula part 2
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"Country", "Customer", "Grouping"}, {{"Original Value 2", each List.Sum([Original Value]), type number}}),
#"Merged Queries" = Table.NestedJoin(#"Filtered Rows",{"Country", "Customer", "Grouping"},#"Grouped Rows" ,{"Country", "Customer", "Grouping"},"Table2",JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"Original Value 2"}, {"Original Value 2"}),
// formula part 3
#"Filtered Rows2" = Table.SelectRows(#"Added Index", each ([Type] = "To be Allocated")),
#"Grouped Rows2" = Table.Group(#"Filtered Rows2", {"Country", "Customer", "Grouping"}, {{"Original Value", each List.Sum([Original Value]), type number}}),
#"Merged Queries1" = Table.NestedJoin(#"Expanded Table2",{"Country", "Customer", "Grouping"},#"Grouped Rows2",{"Country", "Customer", "Grouping"},"Table2",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries1", "Table2", {"Original Value"}, {"Original Value 3"}),
// Add math for new column based on Formula / Formula2 * Formula3
#"Added Custom" = Table.AddColumn(#"Expanded Table1", "Custom", each [Original Value]/[Original Value 2]*[Original Value 3]),
// Replace year with following year, replace Base Figure with Allocated Figure, re-sort on original sort
#"Replaced Value" = Table.ReplaceValue(#"Added Custom", #"Filtered Rows"{0}[Year], #"Filtered Rows2"{0}[Year],Replacer.ReplaceValue,{"Year"}),
#"BlankOriginalValue" = Table.TransformColumns(#"Replaced Value",{{"Original Value", each null}}),
#"Replaced Value1" = Table.ReplaceValue(#"BlankOriginalValue","Base Figure","Allocated Figure",Replacer.ReplaceText,{"Type"}),
#"Removed Columns" = Table.RemoveColumns(#"Replaced Value1",{"Original Value 2", "Original Value 3"}),
#"Sorted Rows" = Table.Sort(#"Removed Columns",{{"Index", Order.Ascending}}),
//combine new Allocated Figure data with original range and remove extra columns
#"Combine" = Table.Combine({#"Added Index" , #"Sorted Rows" }),
#"Added Custom1" = Table.AddColumn(Combine, "Forumla", each if [Custom]=null then [Original Value] else [Custom]),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom1",{"Custom", "Index"})
in #"Removed Columns1"
Second version that should work for multiple To Be Allocated years off a base year. Comments omitted. Refer to first version for explanations
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Year", Int64.Type}, {"Label1", Int64.Type}, {"Label2", type text}, {"Type", type text}, {"Country", type text}, {"Customer", type text}, {"Grouping", type text}, {"Original Value", Int64.Type}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Filtered Rows" = Table.SelectRows(#"Added Index", each ([Type] = "Base Figure")),
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"Country", "Customer", "Grouping"}, {{"Original Value 2", each List.Sum([Original Value]), type number}}),
#"Filtered Rows2" = Table.SelectRows(#"Added Index", each ([Type] = "To be Allocated")),
#"Merged Queries3" = Table.NestedJoin(#"Filtered Rows2",{"Country", "Customer", "Grouping"},#"Filtered Rows" ,{"Country", "Customer", "Grouping"},"Table3",JoinKind.LeftOuter),
#"Removed Columns1" = Table.RemoveColumns(#"Merged Queries3",{"Label1", "Label2"}),
#"Expanded Table3" = Table.ExpandTableColumn(#"Removed Columns1", "Table3", {"Label1", "Label2", "Original Value"}, {"Label1", "Label2", "Formula1"}),
#"Merged Queries" = Table.NestedJoin(#"Expanded Table3" ,{"Country", "Customer", "Grouping"},#"Grouped Rows" ,{"Country", "Customer", "Grouping"},"Table3",JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table3", {"Original Value 2"}, {"Formula2"}),
#"Grouped Rows1" = Table.Group(#"Filtered Rows2" , {"Year", "Country", "Customer", "Grouping"}, {{"Part3", each List.Sum([Original Value]), type number}}),
#"Merged Queries1" = Table.NestedJoin(#"Expanded Table2",{"Year", "Country", "Customer", "Grouping"},#"Grouped Rows1",{"Year", "Country", "Customer", "Grouping"},"Table3",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries1", "Table3", {"Part3"}, {"Formula3"}),
#"Added Custom" = Table.AddColumn(#"Expanded Table1", "New", each [Formula1]/[Formula2]*[Formula3]),
#"Sorted Rows" = Table.Sort(#"Added Custom",{{"Index", Order.Ascending}, {"Label1", Order.Ascending}}),
#"Replaced Value" = Table.ReplaceValue(#"Sorted Rows","To be Allocated","Allocated Figure",Replacer.ReplaceText,{"Type"}),
#"BlankOriginalValue" = Table.TransformColumns(#"Replaced Value",{{"Original Value", each null}}),
#"Removed Columns" = Table.RemoveColumns(BlankOriginalValue,{"Index", "Formula2", "Formula3", "Formula1"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"Year", "Label1", "Label2", "Type", "Country", "Customer", "Grouping", "Original Value", "New"}),
#"Combine" = Table.Combine({#"Changed Type",#"Reordered Columns" })
in #"Combine"

Transpose table partially

I have a CSV-file containing information about some production batches. When loaded into Excels PowerQuery editor, the table looks like this:
Batch Date RawMaterial1 RawMaterial2 RawMaterial3 Amount1 Amount2 Amount3
123 01.01.2020 Fe Cr Ni 70 19 11
234 01.02.2020 Fe Cr Ni 72 17 9
To make this table more readable, I'm looking for a way to transpose it just partially to transform it into a format like this:
Batch Date RawMaterials Amounts
123 01.01.2020 Fe 70
Cr 19
Ni 11
234 01.02.2020 Fe 72
Cr 17
Ni 11
Is there a way to realize this with PowerQueryM alone?
This can be done quite a bit more simply:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],,
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Batch", Int64.Type}, {"Date", type date}, {"RawMaterial1", type text}, {"RawMaterial2", type text}, {"RawMaterial3", type text}, {"Amount1", Int64.Type}, {"Amount2", Int64.Type}, {"Amount3", Int64.Type}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"Batch", "Date"}, "Attribute", "Value"),
#"Split Column by Character Transition" = Table.SplitColumn(#"Unpivoted Other Columns", "Attribute", Splitter.SplitTextByCharacterTransition((c) => not List.Contains({"0".."9"}, c), {"0".."9"}), {"Type", "Index"}),
#"Pivoted Column" = Table.Pivot(#"Split Column by Character Transition", List.Distinct(#"Split Column by Character Transition"[Type]), "Type", "Value")
in
#"Pivoted Column"
Unpivot all but the first two columns.
Split the Attribute column into the text part and index part (in the GUI: Transform > Split Column > By Non-Digit to Digit).
Pivot back on the text part column (choose Don't Aggregate in the Pivot Column Advanced options).
This seems to work
Unpivots all columns but first two
Duplicate the data column
Change the column type to number to force an error on the non numerical columns
Change all errors into something recognizable, like 999999999999
Filter based on that into two tables, and add an index to each table
Merge the two tables together
Add new column, using index to see if Batch is same as prior row to eliminate duplicates
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"Date", "Batch"}, "Attribute", "Value"),
#"Duplicated Column" = Table.DuplicateColumn(#"Unpivoted Other Columns", "Value", "Value - Copy"),
#"Changed Type" = Table.TransformColumnTypes(#"Duplicated Column",{{"Value - Copy", type number}}),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Changed Type", {{"Value - Copy", 999999999999999}}),
#"Filtered Rows" = Table.SelectRows(#"Replaced Errors", each ([#"Value - Copy"] = 999999999999999)),
#"Filtered Rows2" = Table.SelectRows(#"Replaced Errors", each ([#"Value - Copy"] <> 999999999999999)),
Index1 = Table.AddIndexColumn(#"Filtered Rows", "Index", 0, 1),
Index2 = Table.AddIndexColumn(#"Filtered Rows2", "Index", 0, 1),
#"Merged Queries" = Table.NestedJoin(Index2,{"Index"},Index1,{"Index"},"Index3",JoinKind.LeftOuter),
#"Expanded Index3" = Table.ExpandTableColumn(#"Merged Queries", "Index3", {"Value"}, {"Value.1"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Index3",{"Attribute", "Value - Copy"}),
#"Added Custom" = Table.AddColumn(#"Removed Columns", "Batch.1", each if [Index] = 0 then [Batch] else if #"Removed Columns"{[Index]-1}[Batch] = [Batch] then null else [Batch]),
#"Added Custom2" = Table.AddColumn(#"Added Custom", "Date.1", each if [Index] = 0 then [Date] else if #"Removed Columns"{[Index]-1}[Batch] = [Batch] then null else [Date]),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom2",{"Batch", "Date", "Index"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns1",{"Batch.1", "Date.1", "Value.1", "Value"})
in #"Reordered Columns"
First of all a big thank you to #horseyride. I learned a lot from your suggested code. Sadly when I tried to add the date-column to the unpivot area as well I found a little flaw in the code. But thanks to the lessons I learned from it I was able to produce a slightly more generic version which basically follows the same algorithm.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}}),
#"Unpivot Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"Batch"}, "Attribut", "Wert"),
Filter1 = Table.SelectRows(#"Unpivot Columns", each ([Attribut] <> "Amount1" and [Attribut] <> "Amount2" and [Attribut] <> "Amount3")),
ModFilter1 = Table.AddColumn(Filter1, "Benutzerdefiniert", each if [Attribut] = "Date" then [Attribut] else [Wert], type text),
Filter2 = Table.SelectRows(#"Unpivot Columns", each ([Attribut] <> "RawMaterial1" and [Attribut] <> "RawMaterial2" and [Attribut] <> "RawMaterial3")),
#"IndexFilter1" = Table.AddIndexColumn(ModFilter1, "Index", 0, 1),
#"IndexFilter2" = Table.AddIndexColumn(Filter2, "Index", 0, 1),
#"Join Filtered Indexes" = Table.NestedJoin(IndexFilter1,{"Index"},IndexFilter2,{"Index"},"IndexFilter2",JoinKind.LeftOuter),
#"Expand Joined Column" = Table.ExpandTableColumn(#"Join Filtered Indexes", "IndexFilter2", {"Wert"}, {"IndexFilter2.Wert"}),
#"Remove Columns" = Table.RemoveColumns(#"Expand Joined Column",{"Index", "Attribut", "Wert"}),
#"Rename Columns" = Table.RenameColumns(#"Remove Columns",{{"Benutzerdefiniert", "Attribut"}, {"IndexFilter2.Wert", "Wert"}})
in
#"Rename Columns"
I keep horseyrides answer checked as the right answer as he solves my initial question as it was.

Resources