Related
I receive data that looks like this:
Name
01/01/2023
01/02/2023
Revenue
Revenue
Chris
1
3
£100
£300
Colin
5
8
£500
£800
Pete
2
5
£200
£500
Where name is self-explanatory, the next two columns are dates (in UK format) with the number of days worked in the period shown below, and the final two columns are revenue.
I want to modify this data in Power Query so it looks like this:
Name
Date
Work Days
Revenue
Chris
01/01/2023
1
£100
Chris
01/02/2023
3
£300
Colin
01/01/2023
5
£500
Colin
01/02/2023
8
£800
Pete
01/01/2023
2
£200
Pete
01/02/2023
5
£500
I thought this would be some kind of a pivot operation but I can't figure it out.
Any assistance will be gratefully received.
Thanks,
Chris
One simple way
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Set0=List.FirstN(Table.ColumnNames(Source),1),
Set1= List.Combine({Set0,List.Alternate(Table.ColumnNames(Source),1,1)}),
Set2=List.Combine({Set0,List.Alternate(List.RemoveFirstN(Table.ColumnNames(Source),1),1,1)}),
Part1 = Table.SelectColumns(Source,Set1),
Part2 = Table.SelectColumns(Source,Set2),
Date1 = Table.AddColumn(Part1,"Date" , each Table.ColumnNames(Part1){1}),
Date2 = Table.AddColumn(Part2,"Date" , each Table.ColumnNames(Part2){1}),
Rename1 = Table.RenameColumns(Date1,{{Table.ColumnNames(Part1){2}, "Revenue"}, {Table.ColumnNames(Part1){1}, "Work Days"}}),
Rename2 = Table.RenameColumns(Date2,{{Table.ColumnNames(Part2){2}, "Revenue"}, {Table.ColumnNames(Part2){1}, "Work Days"}}),
combined = Rename1 & Rename2
in combined
or
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"Index", "Name"}, "Attribute", "Value"),
#"Added Custom" = Table.AddColumn(#"Unpivoted Other Columns", "Date", each if Text.Start([Attribute],3)="Rev" then null else [Attribute]),
#"Added Custom2" = Table.AddColumn(#"Added Custom", "count", each if Text.Start([Attribute],3)="Rev" then null else [Value]),
#"Added Index1" = Table.AddIndexColumn(#"Added Custom2", "Index.1", 0, 1, Int64.Type),
#"Inserted Modulo" = Table.AddColumn(#"Added Index1", "Modulo", each Number.Mod([Index.1], 2), type number),
#"Sorted Rows" = Table.Sort(#"Inserted Modulo",{{"Index", Order.Ascending}, {"Modulo", Order.Ascending}, {"Attribute", Order.Ascending}}),
#"Filled Down" = Table.FillDown(#"Sorted Rows",{"Date", "count"}),
x=Table.AlternateRows(#"Filled Down",0,1,1),
#"Removed Other Columns" = Table.SelectColumns(x,{"Name", "Value", "Date", "count"})
in #"Removed Other Columns"
Here's another way:
Using List.Generate, create a List of Tables using each Date/Revenue Pair.
For each of the tables, ensure the Revenue Column is named Revenue (and not Revenue2, Revenu3, etc) and then Unpivot the table.
Then expand the column that has the list of tables
The rest is "housekeeping"
Code Edited to provide for varying numbers of "First Columns" to be retained before the sets of Date and Revenue columns
*Change #"Retained Column Count" to reflect that number of Columns
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
colNames = List.Buffer(Table.ColumnNames(Source)),
//How many columns at the beginning are non data pairs
#"Retained Column Count" = 4,
#"First Columns" = List.FirstN(colNames,#"Retained Column Count"),
#"Date Columns" = List.Range(colNames,#"Retained Column Count",(List.Count(colNames)-#"Retained Column Count")/2),
#"Revenue Columns" = List.LastN(colNames,List.Count(#"Date Columns")),
//set data types
types = List.Transform(#"First Columns", each {_, type text}) &
List.Transform(#"Date Columns", each {_, Int64.Type}) &
List.Transform(#"Revenue Columns", each {_, Currency.Type}),
#"Changed Type" = Table.TransformColumnTypes(Source, types, "en-GB"),
//create a list of tables consisting of each date/revenue pair
// then unpivot each table
// ensure Revenue column has the same name throughout
#"Data Pairs" = List.Generate(
()=>[t=Table.SelectColumns(#"Changed Type",#"First Columns" & {#"Date Columns"{0}} & {#"Revenue Columns"{0}}), idx=0],
each [idx] < List.Count(#"Date Columns"),
each [t=Table.SelectColumns(#"Changed Type",#"First Columns" & {#"Date Columns"{[idx]+1}} & {#"Revenue Columns"{[idx]+1}}), idx=[idx]+1],
each Table.Unpivot(
Table.RenameColumns([t], {Table.ColumnNames([t]){#"Retained Column Count"+1},"Revenue"}),
{#"Date Columns"{[idx]}},"Date","Work Days")),
//to a table
// then combine the tables with column names in desired order
#"Converted to Table" = Table.FromList(#"Data Pairs", Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Expanded Column1" = Table.ExpandTableColumn(#"Converted to Table", "Column1", #"First Columns" & {"Date","Work Days","Revenue"}),
#"Changed Type with Locale" = Table.TransformColumnTypes(#"Expanded Column1",
List.Transform(#"First Columns", each {_, type text}) &
{{"Date", type date},
{"Work Days", Int64.Type},
{"Revenue", Currency.Type}}, "en-GB"),
#"Sorted Rows" = Table.Sort(#"Changed Type with Locale",{{"Name", Order.Ascending}, {"Date", Order.Ascending}})
in
#"Sorted Rows"
I have a data set where I would like to have a table with Unique IDs in one Column A and from Column B the rows from a "Input" table above with a different rows as a column headers. In Column A are IDs (unique) and Column B has different rows that have to be in columns but matching values on the rest of the columns.. see on screenshot.
Third Column C is a just observational column that gives info what kind of data type should be there (it can be avoided in this case).
I though I was going to solve it "easily" with Pivot/Unpivot+Transponse method in Power Query but no way....I can get it in one row like in "Output" table..
The dummy data is in link below.
https://docs.google.com/spreadsheets/d/1qKeVj9nJF1usBk-OUZPJfpRqSQnOTCvr/edit?usp=sharing&ouid=101738555398870704584&rtpof=true&sd=true
Merge the value columns into a single column before the pivot, eg
let
Source = Excel.Workbook(File.Contents("C:\Users\david\Downloads\Test1.xlsx"), null, false),
Sheet1_sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
FilterNullAndWhitespace = each List.Select(_, each _ <> null and (not (_ is text) or Text.Trim(_) <> "")),
#"Added Custom" = Table.AddColumn(Sheet1_sheet, "IsEmptyRow", each try List.IsEmpty(FilterNullAndWhitespace(Record.FieldValues(_))) otherwise false),
#"Added Index" = Table.AddIndexColumn(#"Added Custom", "Index", -1),
#"Added Custom1" = Table.AddColumn(#"Added Index", "Section", each if [IsEmptyRow] then -1 else if try #"Added Index"[IsEmptyRow]{[Index]} otherwise true then [Index] else null),
#"Removed Blank Rows" = Table.SelectRows(#"Added Custom1", each not [IsEmptyRow]),
#"Filled Down" = Table.FillDown(#"Removed Blank Rows", {"Section"}),
#"Grouped Rows" = Table.Group(#"Filled Down", {"Section"}, {{"Rows", each _}}, GroupKind.Local),
#"Selected Group" = #"Grouped Rows"[Rows]{1},
#"Removed Columns" = Table.RemoveColumns(#"Selected Group", {"IsEmptyRow", "Index", "Section"}),
#"Promoted Headers" = Table.PromoteHeaders(#"Removed Columns", [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ObjectID", Int64.Type}, {"Feld", type text}, {"Datentyp", type text}, {"boolValue", type text}, {"dateValue", type date}, {"intValue", Int64.Type}, {"stringValue", type text}, {"longStringValue", type text}, {"referencedObjectId", Int64.Type}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type", "Merged", each Text.Combine({[boolValue], Text.From([dateValue], "en-US"), Text.From([intValue], "en-US"), [stringValue], [longStringValue], Text.From([referencedObjectId], "en-US")}, ""), type text),
#"Removed Columns1" = Table.RemoveColumns(#"Inserted Merged Column",{"Datentyp", "boolValue", "dateValue", "intValue", "stringValue", "longStringValue", "referencedObjectId"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[Feld]), "Feld", "Merged")
in
#"Pivoted Column"
This is what I want the date to look like when everything is all done and I transpose the data.
Data
2 Witches Winery and Brewing Company
209 Trade Street
Danville, VA 24541-3545
Phone: (434) 549-2739
Type: Taproom
www.2witcheswinebrew.com
View Map
36 Fifty Brewing
120 N Chestnut St
Marion, VA 24354
Type: Taproom
View Map
6 Bears & A Goat Brewing Company, LLC
1140 International Pkwy
Fredericksburg, VA 22406-1126
Phone: 540-356-9056 Ext. 2
Type: Brewpub
www.6bgbrewingco.com
View Map
Each block of cells represents ONE brewery. I am trying to transpose and put this value into rows. Here is the problem…. Not all the values are in the correct place. The first 3 rows are always same for every single brewery. When it gets to the 4th row of each brewery, that is where it gets tricky. Not all the breweries have a phone, so transposing the data makes all the data not in the right spot. The type should typically be in the “5” row, but since there is no number, it is in the 4th row. About 20% of the data is like this. Anyone have any recommendations.
Apologies, edit forgot to add what I have tried, but it doesn't work as expected.
// Table2
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Helper_Column", each if Text.Contains([Column1],"Phone:") then "1 #1" else null),
#"Removed Errors" = Table.RemoveRowsWithErrors(#"Added Custom", {"Helper_Column"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Removed Errors", {{"Helper_Column", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Helper_Column"),
#"Added Custom1" = Table.AddColumn(#"Split Column by Delimiter", "Helper Column 1", each if [Helper_Column] = "#1" then null else [Column1]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Helper Column 1"}),
#"Added Custom2" = Table.AddColumn(#"Removed Other Columns", "Helper Column", each if Text.Contains([Helper Column 1],"View Map") then "1 #1" else null),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Added Custom2", {{"Helper Column", null}}),
#"Split Column by Delimiter1" = Table.ExpandListColumn(Table.TransformColumns(#"Replaced Errors", {{"Helper Column", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Helper Column"),
#"Added Custom3" = Table.AddColumn(#"Split Column by Delimiter1", "Helper", each if [Helper Column] = "#1" then null else [Helper Column 1]),
#"Removed Other Columns1" = Table.SelectColumns(#"Added Custom3",{"Helper"}),
#"Added Index" = Table.AddIndexColumn(#"Removed Other Columns1", "Index", 0, 1, Int64.Type),
#"Inserted Modulo" = Table.AddColumn(#"Added Index", "Modulo", each Number.Mod([Index], 8), type number),
#"Integer-Divided Column" = Table.TransformColumns(#"Inserted Modulo", {{"Index", each Number.IntegerDivide(_, 8), Int64.Type}}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Integer-Divided Column", {{"Modulo", type text}}, "en-IN"), List.Distinct(Table.TransformColumnTypes(#"Integer-Divided Column", {{"Modulo", type text}}, "en-IN")[Modulo]), "Modulo", "Helper")
in
#"Pivoted Column"
It depends on how realistic your example is. But the code below may help. It works on your posted data.
But you need to have unambiguous rules.
I derived some from your data and what you wrote, and noted them in the code comments. Of course, if your actual data doesn't follow these rules, the algorithm will not work. And if that is the case, you will have to modify the rules.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//assuming each group is contiguous lines
// with a blank line inbetween each group
// the below few lines will create a column on which to group
// then remove the "blank line between"
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "group", each if [Column1] = null then [Index] else null, Int64.Type),
#"Filled Up" = Table.FillUp(#"Added Custom",{"group"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Up",{"Index"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([Column1] <> null)),
//Group, then extract the data
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"group"}, {
//Line one is always the brewery name
{"Brewery Name", each [Column1]{0}, type text},
//Lines 2 and 3 are always the address
{"Address Part 1", each [Column1]{1}, type text},
{"Address Part 2", each [Column1]{2}, type text},
//Phone number starts with "Phone:"
{"Phone", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"Phone:") then state & current else state), type text},
//Type starts with "Type:"
{"Type", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"Type:") then state & current else state), type text},
//Other 1 starts with "www."
{"Other 1", each List.Accumulate([Column1], "", (state, current)=>
if Text.StartsWith(current,"www.") then state & current else state), type text},
//Other 2 is the last line
{"Other 2", each List.Last([Column1]), type text}
}),
//Remove the grouper column
#"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"group"})
in
#"Removed Columns1"
Data
Results
Let's say that I have a column that has a comma seperated list of values in each row
Row ID | Symptoms
1 | Vomiting,Diarrhoea
2 | Diarrhoea
3 | Cough
Would it be possible to transform this into the following in Power Query?:
Row ID | Symptoms - Cough | Symptoms - Diarrhoea | Symptom - Vomiting
1 | No | Yes | Yes
2 | No | Yes | No
3 | Yes| No | No
Unfortunately could not a find a specific question and answers that tackles my challenge on Stack Overflow and tried looking up multiple videos on YouTube with limited success.
I suggest:
Split the symptom column by comma
Select the Row ID column and unpivot other columns
Add a custom column consisting of the word "Yes"
Pivot on the Value column (containing the symptoms) and Don't Aggregate
Replace the null with "No"
The code as written should adjust as you add/remove rows or symptoms from your orginal data table.
If you need to prefix all of the symptom columns with Symptoms - , that can be done with a few more lines of code.
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Row ID", Int64.Type}, {"Symptoms", type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type", "Symptoms", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv)),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Split Column by Delimiter", {"Row ID"}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Attribute"}),
#"Trimmed Text" = Table.TransformColumns(#"Removed Columns",{{"Value", Text.Trim, type text}}),
#"Added Custom" = Table.AddColumn(#"Trimmed Text", "Custom", each "Yes"),
#"Pivoted Column" = Table.Pivot(#"Added Custom", List.Distinct(#"Added Custom"[Value]), "Value", "Custom"),
#"Replaced Value" = Table.ReplaceValue(#"Pivoted Column",null,"No",Replacer.ReplaceValue,Table.ColumnNames(#"Pivoted Column"))
in
#"Replaced Value"
EDIT: Provoked by #AlexisOlson comment, I added a few lines of code to both alphabetize the symptom list, and also to prefix each symptom name with the string Symptoms - .
I also added numerous comments to the code, which you will also be able to see in the Applied Steps window.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Row ID", Int64.Type}, {"Symptoms", type text}}),
//Split the different symptoms into separate columns
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type", "Symptoms", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv)),
//Unpivot other than "Row ID" so as to have all the symptoms in a single column
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Split Column by Delimiter", {"Row ID"}, "Attribute", "Value"),
//Remove the Attribute column
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Attribute"}),
//Trim the symptoms column to get rid of any leading spaces that might be there
#"Trimmed Text" = Table.TransformColumns(#"Removed Columns",{{"Value", Text.Trim, type text}}),
//Sort symptoms alphabetically
#"Sorted Rows" = Table.Sort(#"Trimmed Text",{{"Value", Order.Ascending}}),
//Create a new column with the "Yes"'s
#"Added Custom" = Table.AddColumn(#"Sorted Rows", "Custom", each "Yes"),
//Pivot on the symptoms column to create new table with each symptom as a column header
#"Pivoted Column" = Table.Pivot(#"Added Custom", List.Distinct(#"Added Custom"[Value]), "Value", "Custom"),
#"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",{{"Cough", type text}, {"Diarrhoea", type text}, {"Headache", type text}, {"Myalgia", type text}, {"Vomiting", type text}}),
//Replace the resultant nulls with "No"'s
#"Replaced Value" = Table.ReplaceValue(#"Changed Type1",null,"No",Replacer.ReplaceValue,Table.ColumnNames(#"Changed Type1")),
firstColName = Table.ColumnNames(#"Replaced Value"){0},
//Prefix each column header with "Symptoms - " except first
rename = Table.TransformColumnNames(#"Replaced Value", each if _ <> firstColName then "Symptoms - " & _ else _)
in
rename
I have a column in excel containing a long list similar to the following:
alfa.zulu#test.com
9v46by8
9016767312
TX961779
1DM90F4
bravo.zulu#test.com
B935536
24086942
9486388284
UAUG350583
0P47MB2
asd65f4
813asdg
357yvjy
jxvn97
iopu634
charlie.zulu#test.com
1DM90F4
0P47MB2
delta.zulu#test.com
9016767312
asd65f4
357yvjy
iopu634
echo.zulu#test.com
9v46by8
TX961779
B935536
I need to transpose the list, BUT every time I have an email address, I need to jump on down to the next row and start all over, such as the following:
alfa.zulu#test.com 9v46by8 9016767312 TX961779 1DM90F4
bravo.zulu#test.com B935536 24086942 9486388284 UAUG350583 0P47MB2 asd65f4 813asdg 357yvjy
charlie.zulu#test.com 1DM90F4 0P47MB2
delta.zulu#test.com 9016767312 asd65f4 357yvjy iopu634
echo.zulu#test.com 9v46by8 TX961779 B935536
Is there any way to achieve this without using vba?
Thanks in advance!
This can be done by combining the INDEX, AGGREGATE and SEARCH functions.
But there are some prerequisites:
The SEARCH function will search for cells with the # symbol - so it should be only in email addresses
At the end of the list, the # symbol must be entered in the first blank cell
Formula:
=IFERROR(INDEX(INDEX($A$1:$A$30,AGGREGATE(15,6,(1/ISNUMBER(SEARCH("#",$A$1:$A$30)))*ROW($A$1:$A$30),ROW())):INDEX($A$1:$A$30,AGGREGATE(15,6,(1/ISNUMBER(SEARCH("#",$A$1:$A$30)))*(ROW($A$1:$A$30)-1),ROW()+1)),COLUMN()-2),"")
If the list is very long, it may be better to follow Ron's advice.
With Power Query:
Make the column data type = text
Test if an entry is email -- using the # but could be more sophisticated
Add an Index column
Add another column which contains a unique number each time there is an email in column 1
Fill down with the unique numbers so each "group" will have the same number
Group the rows on the unique numbers column
Extract the data from each row into a delimited list
Add some logic to enable variations in the numbers of potential columns, else power query will not adapt.
Split the list of data into new columns based on the delimiter
Along the way, we delete extraneous columns
Paste the code below into the Power Query Editor
Change the Table in Line 2 to reflect the real table name in your worksheet.
Double click on the statements in the Applied Steps window to explore what is being done at each step
A refresh is all that should be required if your data table changes.
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "isEmail", each Text.Contains([Column1],"#")),
#"Added Index" = Table.AddIndexColumn(#"Added Custom", "Index", 0, 1, Int64.Type),
#"Added Custom1" = Table.AddColumn(#"Added Index", "Grouper", each if [isEmail] then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"Grouper"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"isEmail", "Index"}),
#"Grouped Rows" = Table.Group(#"Removed Columns", {"Grouper"}, {{"Grouped", each _, type table [Column1=nullable text, Grouper=number]}}),
#"Added Custom2" = Table.AddColumn(#"Grouped Rows", "Value", each Table.Column([Grouped],"Column1")),
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom2",{"Grouper", "Grouped"}),
#"Added Custom3" = Table.AddColumn(#"Removed Columns2", "numSplits", each List.Count([Value])),
//Make column splitting dynamic for each refresh, in case maximum number of columns changes
splits = List.Max(Table.Column(#"Added Custom3","numSplits")),
newColList = List.Zip({List.Repeat({"Value"},splits),List.Generate(() => 1, each _ <= splits, each _ +1)}),
#"Converted to Table" = Table.FromList(newColList, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
newColNamesTbl = Table.TransformColumns(#"Converted to Table", {"Column1", each Text.Combine(List.Transform(_, Text.From)), type text}),
newColNamesList = Table.Column(newColNamesTbl,"Column1"),
#"Extracted Values" = Table.TransformColumns(#"Added Custom3", {"Value", each Text.Combine(List.Transform(_, Text.From), ";"), type text}),
#"Removed Columns1" = Table.RemoveColumns(#"Extracted Values",{"numSplits"}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Removed Columns1", "Value", Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), newColNamesList)
in
#"Split Column by Delimiter"
Source Data
Results