Excel merge two tables, repeating 2nd table lookup row - excel

I have two Excel Tables with a 'lookup' column to merge against. I want to merge to a new table with all the lookup values expanded. If I were doing this in python or some such, the pseudo-code would be something like:
for unique day in Tbl1
row1 = day
row2 = ""
for event in Tbl1 day
v = event's lookup value in Tbl2
row1 += event + (len(v) - 1) blank columns
row2 += v
print(row1)
print(row2)
I'd like to avoid VBA, but would like to use new dynamic array functions (preferred) or power query (if necessary), but I can't figure out how to get the repeat to happen. The power query merges I've tried aren't complete.
The original data (where I've used abbreviations for my real data), has a number of events per day. The 'lookup' column shows the different levels of that event for that day.
Tbl1
day
event
lookup
1
Re
eoni2
1
Gr
eoni1
1
We
eoni1
2
Tn
eoneonii2
2
Ga
eon1
2
Gr
eoni1
Tbl2
lookup
c1
c2
c3
c4
c5
c6
c7
c8
eeononii
E
E
O
N
O
N
I
I
eon1
E
O
N
eoneonii2
E
O
N
E
O
N
I
I
eoni1
E
O
N
I
eoni2
E
E
O
O
N
N
I
I
Tbl1
Data will change: number of events per day, event value, what lookup value might be for an event.
The 'event' may or may not repeat from one day to the next, but will be unique within a day.
Order (top to bottom) should be maintained in resulting merge (left to right).
Max number of days = 3.
Tbl2
generally static and top to bottom order can be changed if needed.
may contain entries that are not used by Tbl1.
min of 3 and max of 8 values per row.
Tbl3 output
ideally, the 'event' name would not repeat, as shown below, but can if it keeps formula cleaner.
the number of columns for each day in output Tbl3 may not be the same, as shown, e.g. day 1 rows have 16 and day 2 rows have 15 here.
The output I want:
Tbl3
day
e1
e2
e3
e4
e5
e6
e7
e8
e9
e10
e11
e12
e13
e14
e15
e16
1
Re
Gr
We
E
E
O
O
N
N
I
I
E
O
N
I
E
O
N
I
2
Tn
Ga
Gr
E
O
N
E
O
N
I
I
E
O
N
E
O
N
I
Thanks much.

This can be accomplished using Power Query, available in Windows Excel 2010+ and Excel 365 (Windows or Mac)
To use Power Query
Select some cell in your Data Table
Data => Get&Transform => from Table/Range or from within sheet
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
M Code
let
//Read in both tables
//Edit Source and Source1 lines to reflect your actual table names
Source = Excel.CurrentWorkbook(){[Name="Tbl_1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"day", Int64.Type}, {"event", type text}, {"lookup", type text}}),
Source1 = Excel.CurrentWorkbook(){[Name="Tbl_2"]}[Content],
#"Changed Type1" = Table.TransformColumnTypes(Source1,
List.Transform(Table.ColumnNames(Source1), each {_, type text})),
//Join the two tables based on the lookup column
//then remove that column
#"Join Tables" = Table.NestedJoin(#"Changed Type","lookup", #"Changed Type1","lookup", "joined"),
#"Removed Columns" = Table.RemoveColumns(#"Join Tables",{"lookup"}),
//Add index column to maintain original Event order
#"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1, Int64.Type),
//Expand the joined table and remove the Index column
#"Expanded joined" = Table.ExpandTableColumn(#"Added Index", "joined", {"c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8"}, {"c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8"}),
#"Removed Columns2" = Table.RemoveColumns(#"Expanded joined",{"Index"}),
//Unpivot all the "value" columns
//Then remove the "Attribute" column (the previous column Headers)
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Removed Columns2", {"day", "event"}, "Attribute", "Value"),
#"Removed Columns1" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Attribute"}),
//Group by "day"
#"Grouped Rows" = Table.Group(#"Removed Columns1", {"day"}, {
{"event & value", (t)=> let
remDay = Table.RemoveColumns(t,"day"),
//replace all except first event of a type with null
nullEvents = List.Accumulate(t[event],{}, (state,current)=>
if state = {} then {current}
else if List.Contains(state,current) then state & {null}
else state & {current}),
//then create new table and Transpose to get final format
newTable = Table.Transpose(Table.FromColumns(
{nullEvents, t[Value]}
))
in
newTable}
}),
//Calculate number of columns for creating column names
numCols = List.Max(List.Transform(#"Grouped Rows"[#"event & value"], each Table.ColumnCount(_))),
//expand the grouped columns and set the appropriate names
#"Expanded event & value" = Table.ExpandTableColumn(#"Grouped Rows", "event & value",
List.Transform(List.Numbers(1,numCols), each "Column" & Text.From(_)),
List.Transform(List.Numbers(1,numCols), each "e" & Text.From(_))),
//Replace alternate "day" with null
replaceWithNulls = Table.FromColumns(
{List.Accumulate(#"Expanded event & value"[day], {}, (state,current)=>
if Number.IsOdd(List.Count(state))
then state & {null} else state & {current})} &
Table.ToColumns(Table.RemoveColumns(#"Expanded event & value","day")),
Table.ColumnNames(#"Expanded event & value")
),
//set the data types
typeit = Table.TransformColumnTypes(replaceWithNulls,
{{"day", Int64.Type}} & List.Transform(List.RemoveFirstN(Table.ColumnNames(replaceWithNulls),1), each {_, type text}))
in
typeit

Related

Add Custom and Dynamic columns

I have two tables and am trying to figure out how to create custom code to add dynamic columns with custom names that are based off of row values in another table. I then need to use the values of rows in Table 2 to not only create the column names but also fill the new dynamic Columns with a value from another column in Table 2. Hopefully my pictures below help
Table 1 has varying amount of rows depending on what the user input.
Table 2 has varying amount of rows depending on how many values the user inputs.
Table 1 Before
Col1
Col2
Col 3
stuff 1
stuff 2
stuff 3
stuff 4
stuff 5
stuff 6
.
.
.
.
.
.
Table 2
Name
Values
Name1
100
Name2
500
.
.
NameX
Y
Table 1 After
Col1
Col2
Col 3
"Column" & Name1
"Column"& Name2
...
"Column"& NameX
stuff 1
stuff 2
stuff 3
100
500
...
Y
stuff 4
stuff 5
stuff 6
100
500
...
Y
.
.
.
100
500
...
Y
.
.
.
100
500
...
Y
The "Column" & Name1 meaning I want to concatenate Column with the values in the Name column in Table 2.
You can make this dynamic by not referring to the absolute column names, but rather using the Table.ColumnNames function to return those names.
I did assume that the column names in Table 2 are fixed. If not, that code can be changed.
Read the code comments and examine the Applied Steps window to better understand the methods used. There are examples of setting the data type, and also re-naming columns without referring to a hard-coded column name.
M Code
let
//read in the two tables and set the data types
Source1 = Excel.CurrentWorkbook(){[Name="Table_2"]}[Content],
Table2 =Table.TransformColumnTypes(Source1,
{{"Name", type text},{"Values", type any}}),
Source = Excel.CurrentWorkbook(){[Name="Table_1_Before"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,
List.Transform(Table.ColumnNames(Source), each {_, type text})),
//create the extra columns by
//Transpose Table2
// Use first row as headers
xpose = Table.Transpose(Table2),
#"Promoted Headers" = Table.PromoteHeaders(xpose, [PromoteAllScalars=true]),
#"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",
List.Transform(Table.ColumnNames(#"Promoted Headers"), each {_, type any})),
//rename the columns
renameNameCols = Table.RenameColumns(#"Changed Type1",
List.Zip(
{Table.ColumnNames(#"Changed Type1"),
List.Transform(Table.ColumnNames(#"Changed Type1"), each "Column " & _)})),
//Combine the tables
combine = Table.Combine({#"Changed Type",renameNameCols}),
//fill up the original table 2 columns and remove the blank Table 1 rows
#"Filled Up" = Table.FillUp(combine,Table.ColumnNames(renameNameCols)),
#"Filtered Rows" = Table.SelectRows(#"Filled Up", each ([Col1] <> null))
in
#"Filtered Rows"
Original Tables
Results
Note that I did NOT add logic to avoid prepending the ... with the word column, as shown in your desired output, but that is easily added if really needed
My version
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
custom = Table.FromColumns(Table.ToColumns(Source) &Table.ToColumns(Table2), List.Combine({Table.ColumnNames(Source),Table.ColumnNames(Table2)}) ),
#"Filled Down" = Table.FillDown(custom,Table.ColumnNames(Table2))
in #"Filled Down"

Stacking multiple columns into one using an if statement - vba (excel)

I need to to stack multiple columns into one using an if statement as the example below
Original table looks like the following:
Type
ID
Name
State
X
Y
Pay
01
Joe
NY
-5
0
Pay
02
Ann
FL
-2
-4
Receive
03
Lee
TX
1
0
Pay
04
Ken
CA
0
-1
Receive
05
John
NY
3
2
I would like to have the columns Type, ID, X and Y to be copied from sheet1 to sheet2 using the following conditions:
if Type = "Pay" and X <> 0 then copy columns "Type", "ID" and X * (-1)
if Type = "Pay" and Y <> 0 then copy columns "Type", "ID" and Y * (-1)
if Type = "Receive" and X <> 0 then copy columns "Type", "ID" and X
if Type = "Receive" and Y <> 0 then copy columns "Type", "ID" and Y
I would Like the final result to look like the following:
Type
ID
#
Pay
01
5
X
Pay
02
2
X
Receive
03
1
X
Receive
05
3
X
Pay
02
4
Y
Pay
04
1
Y
Receive
05
2
Y
Please help me
Thanks Phil
As #ScottCraner implied, you can obtain your desired output using Power Query, available in Windows Excel 2010+ and Office 365 Excel
Select some cell in your original table
Data => Get&Transform => From Table/Range
When the PQ UI opens, navigate to Home => Advanced Editor
Make note of the Table Name in Line 2 of the code.
Replace the existing code with the M-Code below
Change the table name in line 2 of the pasted code to your "real" table name
Examine any comments, and also the Applied Steps window, to better understand the algorithm and steps
Note that your "conditions", when applied to the unPivot table, are the same as
Filter out the zero values
Multiple the "Pay" values by -1
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table33"]}[Content],
//delete unneeded columns
#"Removed Columns" = Table.RemoveColumns(Source,{"Name", "State"}),
//set data type
typeIt = Table.TransformColumnTypes(#"Removed Columns",{
{"Type", Text.Type},
{"ID", Text.Type},
{"X", Int64.Type},
{"Y", Int64.Type}
}),
//unPivot, then remove the rows with zeros's
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(typeIt, {"Type", "ID"}, "Attribute", "Value"),
#"Filtered Rows" = Table.SelectRows(#"Unpivoted Other Columns", each ([Value] <> 0)),
//add column where Pay amount are multiplied by -1
//remove unneeded Value column
//Sort and reorder the columns
#"Added Custom" = Table.AddColumn(#"Filtered Rows", "#", each if [Type]="Pay" then [Value] * -1 else [Value]),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Value"}),
#"Sorted Rows" = Table.Sort(#"Removed Columns1",{{"Attribute", Order.Ascending}, {"ID", Order.Ascending}}),
#"Reordered Columns" = Table.ReorderColumns(#"Sorted Rows",{"Type", "ID", "#", "Attribute"})
in
#"Reordered Columns"

Increment difference between cells

I'm trying to duplicate data in a sheet with increments of 12 between each cell from a sheet with 1 cell per row. Between the 12-incremented rows there's other data. This means I can't drag to extend the formula. Like this for customer numbers:
'SheetA'E3 = 'SheetB'Y2
'SheetA'E15 = 'SheetB'Y3
'SheetA'E27 = 'SheetB'Y4
..and so on. I've tried extending 12/24 cells at a time and copying but I can't make it work. Extending doesn't add +1 to one sheet, just +12/+24 to both. Doing this manually will take months. Can this be done without a VBA solution?
Any suggestions? I'm sorry if my terminology isn't on point here.
SheetA:
Try this (run as VBA code):
Sub test1()
For i01 = 0 To 100
Worksheets("SheetA").Cells(3 + 12 * i01, 5) = Worksheets("SheetB").Cells(2 + i01, 25)
Next i01
End Sub
Power Query, available in Windows Excel 2010+ and Office 365, can produce your SheetA given SheetB. Not sure about the effect of the variability you mention.
The query assumes that the correct parameters are listed as column headers in Sheet B. The column headers will get copied over as parameters to sheet A.
To use Power Query:
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
M Code
let
//Read in the data
//Change table name in next line to be the "real" table name
Source = Excel.CurrentWorkbook(){[Name="Table12"]}[Content],
//set data types based on first entry in the column
//will be independent of the column names
typeIt = Table.TransformColumnTypes(Source,
List.Transform(
Table.ColumnNames(Source), each
{_,Value.Type(Table.Column(Source,_){0})})
),
//UNpivot except for the c.number and c.name columns to create the Parameter and Level columns
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(typeIt, {"C. number", "C. name"}, "Parameter", "Level"),
//Group By C.Number
//Add the appropriate rows for each customer
//And a blank row to separate the customers
#"Grouped Rows" = Table.Group(#"Unpivoted Other Columns", {"C. number"}, {
{"All", each _, type table [C. number=nullable number, C. name=nullable text, Parameter=text, Level=any]},
{"custLabel", (t)=> Table.InsertRows(t,0,{
[C. number = null, C. name=null,Parameter = null, Level = null],
[C. number = null, C. name=null, Parameter = "Customer Number", Level="Customer Name"],
[C. number = null, C. name=null,Parameter = t[C. number]{0}, Level = t[C. name]{0}],
[C. number = null, C. name=null,Parameter = "Parameter", Level = "Level"]
})}
}),
//Remove the unneeded columns and expand the remaining table
#"Removed Columns" = Table.RemoveColumns(#"Grouped Rows",{"C. number", "All"}),
#"Expanded custLabel" = Table.ExpandTableColumn(#"Removed Columns", "custLabel", {"Parameter", "Level"}, {"Parameter", "Level"}),
//Remove the top blank row
//promote the new blank row to the Header location
#"Removed Top Rows" = Table.Skip(#"Expanded custLabel",1),
#"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars=true]),
//data type set to text since it will look better on the report
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Customer Number", type text}, {"Customer Name", type text}})
in
#"Changed Type"```
Data
Results
[ Indirect with row() ]
Assuming 'SheetA'E3 column is the target and 'SheetB'Y2 is the source data.
In SheetA!E3 cell put:
=INDIRECT("SheetB!Y"&( ( (row()-3) / 12) + 2)
Press Enter
Then select SheetA!E3 cell, copy. Then paste in SheetA!E24. The formula will update itself.
Idea :
Find the relation between the target cell row number and the source cell row number. [ b > a : 3 > 2 , 15 > 3, 27 > 4 ] leads to a = (b-3)/12 + 2 . (The math is sort of like figuring out a straight line equation from 3 coordinate.) Then use INDIRECT() to combine the calculated row number with the column address.

Move Data in Vertical Cells To Horizontal Cells in Excel 2007

I am using excel 2007
I have a excel sheet with around 1200 records with following structure...
WHAT CAN BE EASIEST WAY TO DO THIS ?
For easy understanding, Adding image :
As per your comment request, here is a Power Query solution.
To enter the code:
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Algorithm
Fill in (fill down) the blank rows for the District and Branch columns
Group by District and Branch
For each Group, extract as a delimited string the entries for President, Secretary and Treasurer.
Create the appropriate column names and split the delimited strings into separate columns.
If you have more officers, or more items per officer/position, or more columns before you get to the officer columns, it should be relatively simple to modify the code to take that into account.
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table16"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"District", Text.Type}, {"Branch", type text},
{"President", type text}, {"Secretary", type text}, {"Treasurer", type text}}),
#"Filled Down" = Table.FillDown(#"Changed Type",{"District", "Branch"}),
#"Grouped Rows" = Table.Group(#"Filled Down", {"District", "Branch"},{
{"President", each Text.Combine([President],";")},
{"Secretary", each Text.Combine([Secretary],";")},
{"Treasurer", each Text.Combine([Treasurer],";")}
}),
colHeaderSuffix = {"","Addr","Mobile"},
PresidentCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"President " & current}})),
#"Split Column by Delimiter" = Table.SplitColumn(#"Grouped Rows", "President",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), PresidentCols),
SecretaryCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"Secretary " & current}})),
#"Split Column by Delimiter2" = Table.SplitColumn(#"Split Column by Delimiter", "Secretary",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), SecretaryCols),
TreasurerCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"Treasurer " & current}})),
#"Split Column by Delimiter3" = Table.SplitColumn(#"Split Column by Delimiter2", "Treasurer",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), TreasurerCols)
in
#"Split Column by Delimiter3"
Original Data
Results
Formula Used as follows (Giving solution here so in future members can use it)
The formula is : In empty cell after Treasurer Column,
=If($a2="","",a2) // copy over next 4 columns to give the District, Branch, Pre name, address, mobile elements as =If($a2="","",a2), =If($b2="","",b2), =If($c2="","",c2), =If($c2="","",c3), =If($c2="","",c3), =If($c2="","",c4)
=if($d2="","",d2) // copy over next 2 columns for Secretary details like =if($d2="","",d2), =if($d2="","",d3), =if($d2="","",d4)
=if($g2="","",g2) // copy over next 2 columns for Treasurer details like =if($g2="","",g2), =if($g2="","",g3), =if($g2="","",g4)
Now Select All New Formula Cells in a row after Treasurer column >> Drag Down Till All Records....
Then Copy all these down to the bottom of your data
Either copy / paste special >> values to somewhere else and
Then sort by District / Branch / Pres to drop the blank rows
I don't know if your excel pc will be able to handle it but you can use the [Paste Transpose][1].
You copy everything, (my advice, go to a new spreadsheet, but you can use the same one),
and then you past it use past transform
*edit
after you edited your question with the example you might want to use the past transform and then use pivot table

Concatenate power query columns that are offset from each other

The problem
I have a data set with two header rows. I've transposed the rows into columns to work with the headers before combining, but I need help with concatenation of column1 into column2, since past row 7 the columns are offset from one another by one row (see example image).
The goal
I've tried to use replace and concatenate myself with an index, but have been unable to achieve the desired end result where column2 row 8 is concatenated with column1 row 7, so that when I combine these columns and transpose again the headers will be correctly labeled (see example image).
Thank you for any suggestions and your time.
Example image:
Here's one way.
I start with your Problem table as a table named Table1:
Then I add an index. (Add Column > Index Column):
Then I add a custom column. (Add Column > Custom Column) With this setup:
(#"Added Index"{[Index]-1}[Column1] references the entry in Column1 at the position record row that is equal to the value in the Index column, minus 1.)
...to get this:
Then I replaced Errors in the new Custom column. (Right-click Custom column title > click Replace Errors > type null > click OK)
Then I select Column1 and Custom column and remove other columns. (Select Column 1 column title > hold Ctrl and click Custom column title > keep holding Ctrl and right click Custom column title > click Remove Other Columns)
Here's my M code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each #"Added Index"{[Index]-1}[Column1]&"-"&[Column2]),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Added Custom", {{"Custom", null}}),
#"Removed Other Columns" = Table.SelectColumns(#"Replaced Errors",{"Column1", "Custom"})
in
#"Removed Other Columns"
Another way.
Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
IndexedTable = Table.AddIndexColumn(Source, "Index", 0, 1),
Transform = Table.TransformRows(IndexedTable, (row)=>[Column1= row[Column1], Column2 = if row[Column1]=null then Text.Combine({IndexedTable{row[Index]-1}[Column1], "-",row[Column2]}) else row[Column2]]),
ToTable = Table.FromRecords(Transform)
in
ToTable
Brief explanation:
Source
Add index to address previous record
Use Table.TransformRows to analyze and transform each row to a record in this manner: Column1 taken from each row's column1 (row[Column1]), Column2 is generated from previous row using Text.Concatenate, IndexedTable{row[Index]-1}[Column1]. This yields value from previous row's Column1. Table.TransformRows returns list of records.
Transform list of records into the table.
This code will fail if 1st row contains null in [Column1]. If this is unacceptable, add another if-then-else.
Another way:
let
Source = Excel.CurrentWorkbook(){[Name="Table"]}[Content],
fillDown = Table.FillDown(Table.DuplicateColumn(Source, "Column1", "copy"),{"copy"}),
replace = Table.ReplaceValue(fillDown, each [Column2], each if [Column2] = null then null
else [copy]&"-"&[Column2], Replacer.ReplaceValue, {"Column2"})[[Column1],[Column2]]
in
replace

Resources