Automatically Expand all "Table Columns" with specific name - excel

I'm a beginner in coding and am not familiar with "M" modeling language. I have an XML file that I want to use to load the data in Query Editor. In the Query, I need to expand only the table columns with a specific name below:
view
viewfolder
Attribute:name
I have found the following post where they give a function by Chris Webb for expanding all the lists(below code).
= (TableToExpand as table, optional ColumnNumber as number) =>
let
ActualColumnNumber = if (ColumnNumber=null) then 0 else ColumnNumber,
ColumnName = Table.ColumnNames(TableToExpand){ActualColumnNumber},
ColumnContents = Table.Column(TableToExpand, ColumnName),
ColumnsToExpand = List.Distinct(List.Combine(List.Transform(ColumnContents, each if _ is table then Table.ColumnNames(_) else {}))),
NewColumnNames = List.Transform(ColumnsToExpand, each ColumnName & "." & _),
CanExpandCurrentColumn = List.Count(ColumnsToExpand)>0,
ExpandedTable = if CanExpandCurrentColumn then Table.ExpandTableColumn(TableToExpand, ColumnName, ColumnsToExpand, NewColumnNames) else TableToExpand,
NextColumnNumber = if CanExpandCurrentColumn then ActualColumnNumber else ActualColumnNumber+1,
OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1) then ExpandedTable else ExpandAll(ExpandedTable, NextColumnNumber)
in
OutputTable
But how to expand only desired lists/tables?

You should be able to simply put a filter on the old ColumnsToExpand line.
That is,
TableColumns = //This is the old ColumnsToExpand definition renamed.
List.Distinct(List.Combine(List.Transform(ColumnContents, each if _ is table then Table.ColumnNames(_) else {}))),
ColumnsToExpand =
List.Select(TableColumns, each (_ = "view" or _ = "viewfolder" or _ = "Attribute:name")),
Or in one line like this,
ColumsToExpand = List.Select(List.Distinct(List.Combine(List.Transform(ColumnContents, each if _ is table then Table.ColumnNames(_) else {}))), each (_ = "view" or _ = "viewfolder" or _ = "Attribute:name")),

Related

How can i get all the sub-children?

source link
Hello guys, so i have a function ("flecheD"),
(ColChild,ColParent,ParentActuel,source)=>
let
mylist=Table.Column(Table.SelectRows(source,each Record.Field(_,ColParent)=ParentActuel),ColChild),
resultat=Text.Combine(mylist)
in
Text.Trim(
if resultat ="" then "" else # resultat &"|" & # flecheD(ColChild,ColParent,resultat,source),"|")
which loops through 2 columns (Parent,Child) to get all children of the main parent (output->Children column). The problem is that when the function is confronted with several children, the resultat variable no longer has a single letter/child but several, which blocks the function from looking for the other sub-children.
In order to solve this, I tried to create a custom function "SubChilldren" with List.Generate()
(children as text, ColChild,ColParent,source)=>
let
i = 1,
length = Text.Length(children),
subchildren = List.Generate( ()=>#flecheD(ColChild,ColParent,Text.At(children,i-1),source), i<=length, i+1 )
in
Text.Combine(subchildren)
which when coupled with my initial function
(ColChild,ColParent,ParentActuel,source)=>
let
mylist=Table.Column(Table.SelectRows(source,each Record.Field(_,ColParent)=ParentActuel),ColChild),
resultat=Text.Combine(mylist)
in
Text.Trim(
if resultat ="" then "" else if Text.Length(resultat) = 1 then # resultat &"|" & # flecheD(ColChild,ColParent,resultat,source)
else #resultat &"|"& SubChildren(resultat,ColChild,ColParent,source),"|")
should normally get the sub-children of each children. However, it doesnt work . Could you please help me . Thx
I thought this was a fun way, but you could write a recursive function as well. I have it hard coded to 4 levels of children deep
(not sure how in your source data D child can have two parents, c and J, but whatever)
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Grouped Rows" = Table.Group(Source, {"Parent"}, {{"data", each List.RemoveNulls(_[Child]), type list}}),
Parent_List = List.Buffer(#"Grouped Rows"[Parent] ),
Child_List = List.Buffer(#"Grouped Rows"[data] ),
Process = (n as list) as list =>
let children = List.Transform(List.Transform(n, each Text.ToList(_)), each Text.Combine( List.Distinct(List.Combine(List.Transform(_, each try Child_List{List.PositionOf( Parent_List, _ )} otherwise null))))) in children,
Level1=Process(Source[Parent]),
Level2=Process(Level1),
Level3=Process(Level2),
Level4=Process(Level3),
Final=List.Transform(List.Positions(Level1),each Level1{_}&"|"&Level2{_}&"|"&Level3{_}&"|"&Level4{_}&"|"),
#"Replaced Value" = Table.ReplaceValue(Table.FromList(Final),"||","",Replacer.ReplaceText,{"Column1"}),
custom1 = Table.ToColumns(Source) & Table.ToColumns(#"Replaced Value"),
custom2 = Table.FromColumns(custom1,Table.ColumnNames(Source) & {"Children"})
in custom2
edited to be generic so it can take text as well as numerical inputs
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Parent", type text}, {"Child", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Parent"}, {{"data", each List.Transform(List.RemoveNulls(_[Child]), each Text.From(_)), type list}}),
Parent_List = List.Buffer(List.Transform(#"Grouped Rows"[Parent], each Text.From(_))),
Child_List = List.Buffer(#"Grouped Rows"[data]),
Process = (n as list) as list =>let children = List.Transform(List.Transform(n, each Text.Split(_,",") ) , each try Text.Combine(List.Distinct(List.Combine(List.Transform(_, each try Child_List{List.PositionOf( Parent_List, _ )} otherwise ""))),"," ) otherwise "") in children,
Level1=Process(#"Changed Type"[Parent]),
Level2=Process(Level1),
Level3=Process(Level2),
Level4=Process(Level3),
Final=List.Transform(List.Positions(Level1),each Level1{_}&"|"&Level2{_}&"|"&Level3{_}&"|"&Level4{_}&"|"),
#"Replaced Value" = Table.ReplaceValue(Table.FromColumns({Final}),"||","",Replacer.ReplaceText,{"Column1"}),
custom1 = Table.ToColumns(#"Changed Type") & Table.ToColumns(#"Replaced Value"),
custom2 = Table.FromColumns(custom1,Table.ColumnNames(#"Changed Type") & {"Children"})
in custom2

Power Query - Remove text between delimiters

I want to remove any text between "( )" including the "( )".
There are many difference instances where I can't simply find and replace.
Small example: ABC (1)
EFG (2)
XYZ (1, 2)
I wish to display
ABC
EFG
XYZ
Found this post, but the code for the function is no longer visible(at least on all the browsers I've tried). https://www.thebiccountant.com/2019/07/15/text-removebetweendelimiters-function-for-power-bi-and-power-query/
I copied the code from one of the comments and it seems to work fine, however when I invoke the function on the column I get all errors with the following: "Expression.Error: The specified index parameter is invalid.
Details:
List"
Does anyone have the code from the author? Or know what I'm doing wrong?
Here is the code from the new custom column after I run the function:
Table.AddColumn(#"Changed Type1", "N", each Query1([#"NEC(s)"], "(", ")", 1, null))
Thanks
Here's a different solution that uses recursion.
(txt as text) =>
[
fnRemoveFirstTag = (DELIM as text)=>
let
OpeningTag = Text.PositionOf(DELIM,"("),
ClosingTag = Text.PositionOf(DELIM,")"),
Output =
if OpeningTag = -1
then DELIM
else Text.RemoveRange(DELIM,OpeningTag,ClosingTag-OpeningTag+1)
in
Output,
fnRemoveDELIM = (y as text)=>
if fnRemoveFirstTag(y) = y
then y
else #fnRemoveDELIM(fnRemoveFirstTag(y)),
Output = #fnRemoveDELIM(txt)
][Output]
It works on your sample data, and should also work if there is more than one set of parentheses delimited substrings in your string.
Copied shamelessly and modified minimally from Power Query: remove all text between delimiters
Is there text to the right of the )?
If not, just split column on custom delimiter ( leftmost, then remove the 2nd column
= Table.SplitColumn(Source, "Column1", Splitter.SplitTextByEachDelimiter({"("}, QuoteStyle.Csv, false), {"Column1.1", "Column1.2"})
OR transform the column to remove anything after the initial (
= Table.TransformColumns(Source,{{"Column1", each Text.Start(_,Text.PositionOf(_,"(")), type text}})
If text to the right of the ), try
= Table.TransformColumns(Source,{{"Column1", each Text.Start(,Text.PositionOf(,"("))&Text.End(,Text.Length()-Text.PositionOf(_,")")-1), type text}})
There is an even simpler solution.
You can create a new function called fun_ReplaceTextBetweenDelimiters, and in it add this code 👇
let
fun_ReplaceTextBetweenDelimiters = (Text as text, StartDelimiter as text, EndDelimiter as text, optional ReplaceDelimiters as nullable logical, optional NewText as nullable text, optional TrimResult as nullable logical, optional FixDoubleSpaces as nullable logical) as text =>
let
// Add Default Parameters
Default_ReplaceDelimiters = if ReplaceDelimiters is null then true else ReplaceDelimiters,
Default_NewText = if NewText is null then "" else NewText,
Default_TrimResult = if TrimResult is null then true else TrimResult,
Default_FixDoubleSpaces = if FixDoubleSpaces is null then true else FixDoubleSpaces,
//Do work
TextBetweenDelimiters = Text.BetweenDelimiters(Text, StartDelimiter, EndDelimiter),
TextToReplace = if Default_ReplaceDelimiters then Text.Combine({StartDelimiter,TextBetweenDelimiters,EndDelimiter}) else TextBetweenDelimiters,
ReplacedText = Text.Replace(Text, TextToReplace, Default_NewText),
//Clean Result
TrimmedText = if Default_TrimResult then Text.Trim(ReplacedText) else ReplacedText,
FixedSpaces = if Default_FixDoubleSpaces then Text.Replace(TrimmedText, " ", " ") else TrimmedText
in
FixedSpaces
in
fun_ReplaceTextBetweenDelimiters
Then, we can test it like this:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WcnRyVtAw1FTSAbGUYnWilVzd3BU0jEAiQBZYJCIyCqhGRwEsCOQoxcYCAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [TestData = _t, TargetData = _t]),
ChangeType = Table.TransformColumnTypes(Source,{{"TestData", type text}, {"TargetData", type text}}),
RunFunction = Table.AddColumn(ChangeType, "NewText", each fun_ReplaceTextBetweenDelimiters([TestData], "(", ")", true), type text),
TestResult = Table.AddColumn(RunFunction, "Test", each [TargetData]=[NewText], type logical)
in
TestResult
Input:
TestData
TargetData
ABC (1)
ABC
EFG (2)
EFG
XYZ (1, 2)
XYZ
Output:
TestData
TargetData
NewText
Test
ABC (1)
ABC
ABC
TRUE
EFG (2)
EFG
EFG
TRUE
XYZ (1, 2)
XYZ
XYZ
TRUE

Power query recursive function: add more steps after the recursive step

Background:
I have posted a question regarding a custom function in Power Query that I found in a blog by Chris Webb which I have already got an answer to.But now I have another question related to the same custom function.
One of the amazing steps in that custom function is the recursive step at the end named "OutputTable" which calls itself using a if statement, basically making it a loop. Below is the step:
OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1) then ExpandedTable else ExpandAll(ExpandedTable, NextColumnNumber)
Question:
Now what I would like to do after this step is to be able to add more transformation on the OutputTable.
For Example, I would like to add a column with just "A" in all the rows. The syntax to do that would be AddNewColumn = Table.AddColumn(OutputTable, "Test", each "A"). But when I do this this gives me an error saying that the column "Test" already exists. But i'm sure that there is no other column with name "Test". Even if I try changing the name of the column to anything else, I get the same error.
Note: Although the actual step I want to add is not AddColumn, I think I can figure out that part If I get a solution for this.
SourceCode:
let
Source = (TableToExpand as table, optional ColumnNumber as number) =>
let
ActualColumnNumber = if (ColumnNumber=null) then 0 else ColumnNumber,
ColumnName = Table.ColumnNames(TableToExpand){ActualColumnNumber},
ColumnContents = Table.Column(TableToExpand, ColumnName),
ColumnsToExpand = List.Select(List.Distinct(List.Combine(List.Transform(ColumnContents, each if _ is table then Table.ColumnNames(_) else {}))), each (_ = "view" or _ = "viewfolder" or _ = "Attribute:name")),
NewColumnNames = List.Transform(ColumnsToExpand, each ColumnName & "." & _),
CanExpandCurrentColumn = List.Count(ColumnsToExpand)>0,
ExpandedTable = if CanExpandCurrentColumn then Table.ExpandTableColumn(TableToExpand, ColumnName, ColumnsToExpand, NewColumnNames) else TableToExpand,
NextColumnNumber = if CanExpandCurrentColumn then ActualColumnNumber else ActualColumnNumber+1,
OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1) then ExpandedTable else ExpandAll(ExpandedTable, NextColumnNumber)
in
OutputTable
in
Source
I'm guessing it's throwing the error due to the recursive nature of the function calling itself and trying to apply the new column twice, once in the innermost loop and once in the outermost loop.
Let's say we have a table with two columns Col1 and Col2 that need to be expanded. If you add the new column after the OutputTable step, you'll get:
Start: Col0, Col1, Col2
OutputTable(1): Col0, Col1.a, Col1.b, Col2
OutputTable(2): Col0, Col1.a, Col1.b, Col2.x, Col2.y, Col2.z, Test
AddNewColumn: Col0, Col1.a, Col1.b, Col2.x, Col2.y, Col2.z, Test, Test
Here are a couple of approaches to try:
1. Only try to add the column when recursion is finished.
I think you can do this by changing your OutputTable line as follows:
OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1)
then Table.AddColumn(ExpandedTable, "Test", each "A")
else ExpandAll(ExpandedTable, NextColumnNumber)
2. Check if the column exists before trying to add it.
AddNewColumn = if Table.HasColumns(OutputTable, "Test")
then OutputTable
else Table.AddColumn(OutputTable, "Test", each "A")

Dynamically changing vlookup table

I recorded this:
Range("B1").FormulaR1C1 = _
"=IF(VLOOKUP(RC1,sorted!R3C1:R35C33,8,FALSE)="""","""",VLOOKUP(RC1,sorted!R3C1:R35C33,8,FALSE))"
I need to have it update dynamically.
I found this (modified for my needs), but I'm not sure what to put in the "?" fields. Hopefully I'm on the right track:
With Worksheets("Move to New")
.Range("?").Offset(x, 0) = Application.WorksheetFunction.VLookup( _
.Range("?").Offset(x, 0), _
Worksheets("sorted").Range("?", .Range("?").End(xlDown)), 8, False)
I got it resolved.
Range("B1").FormulaR1C1 = "=IF(VLOOKUP(RC1,sorted!C1:C33,8,FALSE)="""","""",VLOOKUP(RC1,sorted!C1:C33,8,FALSE))"

Dynamic excel chart doesn't display all of the data

I have this code that allows users to enter chart parameters into some cells and dynamically create a chart. Many series (up to four) are allowed on two vertical (y) axis and one shared horizontal (x) axis. The chart is a mixture of columns and lines normally, and the data ranges are of varying length. I have this code that adds the series like so (I'll try to stick to what I believe is the relevant code)
seriesCount = 1
If hasSeries1 = True Then
ActiveChart.SeriesCollection.NewSeries
ActiveChart.SeriesCollection(seriesCount).Name = .Cells(2, 6) & " " & axisside1
ActiveChart.SeriesCollection(seriesCount).ChartType = chartType1
ActiveChart.SeriesCollection(seriesCount).AxisGroup = axisgroup1
ActiveChart.SeriesCollection(seriesCount).Border.LineStyle = borderStyle1
ActiveChart.SeriesCollection(seriesCount).Border.Color = lineColor1
ActiveChart.SeriesCollection(seriesCount).Format.Line.Weight = lineWidth1
ActiveChart.SeriesCollection(seriesCount).Format.Fill.ForeColor.RGB = seriesColor1
ActiveChart.SeriesCollection(seriesCount).Format.Line.Visible = hasLine1
ActiveChart.SeriesCollection(seriesCount).XValues = dates1
ActiveChart.SeriesCollection(seriesCount).Values = dataset1
seriesCount = seriesCount + 1
End If
If hasSeries2 = True Then
ActiveChart.SeriesCollection.NewSeries
ActiveChart.SeriesCollection(seriesCount).Name = .Cells(3, 6) & " " & axisside2
ActiveChart.SeriesCollection(seriesCount).ChartType = chartType2
ActiveChart.SeriesCollection(seriesCount).AxisGroup = axisgroup2
ActiveChart.SeriesCollection(seriesCount).Border.LineStyle = borderStyle2
ActiveChart.SeriesCollection(seriesCount).Border.Color = lineColor2
ActiveChart.SeriesCollection(seriesCount).Format.Line.Weight = lineWidth2
ActiveChart.SeriesCollection(seriesCount).Format.Fill.ForeColor.RGB = seriesColor2
ActiveChart.SeriesCollection(seriesCount).Format.Line.Visible = hasLine2
ActiveChart.SeriesCollection(seriesCount).XValues = dates2
ActiveChart.SeriesCollection(seriesCount).Values = dataset2
seriesCount = seriesCount + 1
End If
If hasSeries3 = True Then
ActiveChart.SeriesCollection.NewSeries
ActiveChart.SeriesCollection(seriesCount).Name = .Cells(4, 6) & " " & axisside3
ActiveChart.SeriesCollection(seriesCount).ChartType = chartType3
ActiveChart.SeriesCollection(seriesCount).AxisGroup = axisgroup3
ActiveChart.SeriesCollection(seriesCount).Border.LineStyle = borderStyle3
ActiveChart.SeriesCollection(seriesCount).Border.Color = lineColor3
ActiveChart.SeriesCollection(seriesCount).Format.Line.Weight = lineWidth3
ActiveChart.SeriesCollection(seriesCount).Format.Fill.ForeColor.RGB = seriesColor3
ActiveChart.SeriesCollection(seriesCount).Format.Line.Visible = hasLine3
ActiveChart.SeriesCollection(seriesCount).XValues = dates3
ActiveChart.SeriesCollection(seriesCount).Values = dataset3
seriesCount = seriesCount + 1
End If
If hasSeries4 = True Then
ActiveChart.SeriesCollection.NewSeries
ActiveChart.SeriesCollection(seriesCount).Name = .Cells(5, 6) & " " & axisside4
ActiveChart.SeriesCollection(seriesCount).ChartType = chartType4
ActiveChart.SeriesCollection(seriesCount).AxisGroup = axisgroup4
ActiveChart.SeriesCollection(seriesCount).Border.LineStyle = borderStyle4
ActiveChart.SeriesCollection(seriesCount).Border.Color = lineColor4
ActiveChart.SeriesCollection(seriesCount).Format.Line.Weight = lineWidth4
ActiveChart.SeriesCollection(seriesCount).Format.Fill.ForeColor.RGB = seriesColor4
ActiveChart.SeriesCollection(seriesCount).Format.Line.Visible = hasLine4
ActiveChart.SeriesCollection(seriesCount).XValues = dates4
ActiveChart.SeriesCollection(seriesCount).Values = dataset4
End If
Here is the problem: the chart only displays part of the data it is supposed to. When I right-click on the data series, hit Select Data and choose Edit, the correct series (both x and y) become highlighted, but what is being shown is a truncated subset of what should be there.
Here is a sample of what I'm seeing
Here is some of the data for the light blue column
12/30/2005 307%
1/31/2006 302%
2/28/2006 248%
3/31/2006 262%
4/28/2006 285%
5/31/2006 256%
... ...
... ...
... ...
6/30/2014 147%
Notice how this data should be showing on the chart beginning at 12/30/2005, but it's starting at 11/30/2013 instead (though the values appear to be correct, 307%, 302%, etc.). It is almost as though excel is forcing the 2nd and 3rd data series to be the same length as the first one. The first one is charting correctly.
I think you're not using a XY chart, thus you must have the same labels (Xvalues = dates for you) for every series. That means that you need to create a unique dates-set containing all the dates and assign it (as Xvalues) to the first serie.

Resources