Power Query Formula Language - Get children based on parent adjacent column value - excel

bear with me, this is my first attempt using the Power Query Formula Language. I need some advice on how to solve a particular problem sorting and filtering source data.
I now got this current source data, structured like this:
Using this power query:
let
Source = Excel.CurrentWorkbook(){[Name="EmployeeOrganization"]}[Content],
ListEmployees = Table.Group(Source, {"Organization"}, {{"Employee", each Text.Combine([Employee],","), type text}}),
CountEmployees = Table.AddColumn(ListEmployees, "Count", each List.Count(Text.Split([Employee],","))),
SplitEmployees = Table.SplitColumn(ListEmployees, "Employee", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv),List.Max(CountEmployees[Count])),
Transpose = Table.Transpose(SplitEmployees),
PromoteHeaders = Table.PromoteHeaders(Transpose, [PromoteAllScalars=true])
in
PromoteHeaders
I am able to produce the following result:
To avoid having to add the organization name to every single employee in the source, I would like the organization name to act as an parent-group, with the employees as children. I would also like the result to only fetch the organizations (+ employees) that has status Active = Yes.
The desired source should look similar to this:
So that the desired result should look similar to this: (Apple is gone due to Active = NO)
I am stuck at this point and need some advice on how can I modify my Power Query Formula to:
Only fetch Organizations that are Active (Does not matter if they have employees or not)
Somehow link the children Employees to the correct Organizations. (Without having to write the org name in every adjacent employee column)
(Excel file can be found her)

In PQ, you'll need to fill in the blank rows, then Pivot with no aggregation.
See the comments in the code, and follow the Applied Steps to understand the algorithm
Source
Custom Function
Rename: fnPivotAll
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/
(Source as table,
ColToPivot as text,
ColForValues as text)=>
let
PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))),
#"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _),
TableFromRecordOfLists = (rec as record, fieldnames as list) =>
let
PartialRecord = Record.SelectFields(rec,fieldnames),
RecordToList = Record.ToList(PartialRecord),
Table = Table.FromColumns(RecordToList,fieldnames)
in
Table,
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)),
#"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames),
#"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames)
in
#"Expanded Values"
Basic Query
let
//Read in data and set data types
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8k12yc9LzEkpVtJRAqLI1GKlWJ1oEDMgtSS1CCQK5XvlpyLzEvPgXMeCgpxUiH6/fJgC38SiSiT1jjmZyXAN7vn56TAdyDYmluYgaXHKTwLzYgE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Organization = _t, Employee = _t, Active = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Organization", type text}, {"Employee", type text}, {"Active", type text}}),
//replace blanks with null if not already there
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Organization", "Employee", "Active"}),
//fill down the Company and active columns
#"Filled Down" = Table.FillDown(#"Replaced Value",{"Organization", "Active"}),
//Filter to show only Active="Yes and Employee not null
#"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Employee] <> null) and ([Active] = "Yes")),
//Pivot with no aggregation
//could do this with grouping, but easier (and maybe faster, with a custom function
pivotAll = fnPivotAll(#"Filtered Rows","Organization","Employee"),
//remove unneeded Active column and set data types
#"Removed Columns" = Table.RemoveColumns(pivotAll,{"Active"}),
typed = Table.TransformColumnTypes(#"Removed Columns",
List.Transform(Table.ColumnNames(#"Removed Columns"),each {_, Text.Type}))
in
typed
typed Results

Related

Power Query split a column of text according to a separate column containing a list of integers

I'm trying to use Power Query (m) to split a column of text strings called textStringColumn.
The points where I would like to split the column are in another column called indexColumn.
They are both in a table called myTable.
My approach was as here:
#"output" = Table.SplitColumn(myTable, ColumnToSplit,
Splitter.SplitTextByPositions(indexColumn), textStringColumn, "default" )
This resulted in an error:
Expression.Error: There is an unknown identifier.
Did you use the [field] shorthand for a _[field] outside of an 'each' expression?
Subsequent attempts using
#"output" = Table.SplitColumn(myTable, ColumnToSplit,
each Splitter.SplitTextByPositions(indexColumn), textStringColumn, "default" )
produced the same error.
Could you advise how I might correct my query?
Thanks and regards.
EDIT: test data
('pretending' the input data was comma delimited since I can't cut and paste the list column)
textStringColumn,indexColumn
abc 1234 and ghyyyu 432,"[3,19]"
453,"[0]"
hky 7332 4343,"[3,8]"
According to your data, your output will look like this:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSkxKVjA0MjZRSMxLUUjPqKysLFUwMTZS0lGKNtYxtIxVitWJVjIxNQYJGEB4GdmVCubGxkZAdSbGEIUWQJlYAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [textStringColumn = _t, indexColumn = _t]),
#"Replaced Value" = Table.ReplaceValue(Source,"[","",Replacer.ReplaceText,{"indexColumn"}),
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value","]","",Replacer.ReplaceText,{"indexColumn"}),
#"Parsed List" = Table.TransformColumns(#"Replaced Value1",{{"indexColumn", each List.Transform( Text.Split(_,","), each Number.FromText (_) ) }}),
#"Added Custom" = Table.AddColumn(#"Parsed List", "Custom", each Splitter.SplitTextByPositions( [indexColumn] )([textStringColumn])),
#"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), "|"), type text}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Extracted Values", "Custom", Splitter.SplitTextByDelimiter("|", QuoteStyle.Csv))
in
#"Split Column by Delimiter"
There is probably a more efficient way to do this so keen to see if horseyride or Ron have alternatives.

How to pass user parameter into string within function power query

I have the following function, which splits text into sentences using regex. Upon testing, however, these are instances where the regex doesn't quite work, and the text is wrongly split. For example, if the text contains St. bernard, I do not want this sentence to be split on the . of St.
As a nice workaround, I have modified the regex to allow for exceptions to be ignored. Please see here if you are interested in this.
I now wish to incorporate this into excel such that any user can apply their own exceptions however I am having trouble getting the user parameter to be passed into the string (regex) of the function.
Here is what I am trying to achieve (stating |Flam|Liq|St explicitly in the regex):
regex:
\s*((?:\b(?:[djms]rs?|flam|liq|St)\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$)) (https://regex101.com/r/nXf0TM/6)
However, what I would like to achieve is something like:
\s*((?:\b(?:[djms]rs?|"&Exceptions&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))
Where Exceptions is the User parameters entered in a table in excel: flam, liq, St in each row.
i.e.:
M code attempting to achieve this results in an error:
Sentences From text:
let
Exceptions = Exceptions,
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Text", type text}}),
#"Replaced Value1" = Table.ReplaceValue(#"Changed Type","#(lf)"," ",Replacer.ReplaceText,{"Text"}),
#"Replaced Value" = Table.ReplaceValue(#"Replaced Value1","'","&apos",Replacer.ReplaceText,{"Text"}),
#"Invoked Custom Function" = Table.AddColumn(#"Replaced Value", "fnRegexReplace", each fnRegexReplace([Text], "\s*((?:\b(?:[djms]rs?"&Exceptions&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))", "$1|")),
#"Removed Other Columns" = Table.SelectColumns(#"Invoked Custom Function",{"fnRegexReplace"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Removed Other Columns", {{"fnRegexReplace", Splitter.SplitTextByDelimiter("|", QuoteStyle.None), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "fnRegexReplace"),
#"Filtered Rows" = Table.SelectRows(#"Split Column by Delimiter", each ([fnRegexReplace] <> ""))
in
#"Filtered Rows"
Exceptions:
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each "Exceptions"),
#"Grouped Rows" = Table.Group(#"Added Custom", {"Custom"}, {{"Exceptions", each Text.Combine([#"Do not split if:"],"|"), type text}})
in
#"Grouped Rows"
fnRegexReplace
(x,y,z)=>
let
y = Text.Replace(y,"\","\\"),
Source = Web.Page(
"<script>var x="&"'"&x&"'"&";var z="&"'"&z&
"'"&";var y=new RegExp('"&y&"','gmi');
var b=x.replace(y,z);document.write(b);</script>")
[Data]{0}[Children]{0}[Children]{1}[Text]{0}
in
Source
Error:
Raw text Data:
Highly Flammable Liquid Flam. H223 Liq. H334.
St. Bernard Dog was present.
The MW of gold is 100.1. Solubility is 40mg/L.
Im sure this is an easy fix, but whatever I try, i.e. Record.FromTable{0} etc I get various errors.
If anyone could help me out, that would be great.
Thank you.
This is the problem line fixed.
= Table.AddColumn(#"Replaced Value", "fnRegexReplace", each fnRegexReplace([Text], "\s*((?:\b(?:[djms]rs?"&Exceptions[Exceptions]{0}&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))", "$1|"))

Replace second or more instances of duplicates with null

I have the following data with duplicates which I wish to identify. I do not wish to remove these so unique value only won't work. I want to be able to identify them but just saying null.
I have attempted to self-reference the code but end up just duplicating the original result.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Removed Duplicates" = Table.Distinct(#"Changed Type"),
#"Merged Queries" = Table.NestedJoin(Source, {"Column1"}, #"Removed Duplicates", {"Column1"}, "Removed Duplicates", JoinKind.LeftOuter)
in
#"Merged Queries"
You can use List.Generate to generate a list with your requirements. And then you can either replace the first column or add the list as a second column.
This needs to be done in the Advanced Editor.
Please note there is a difference between the text string "null" and the power query null value. Based on your example screenshot, I assumed you wanted the "null" text string. If you prefer the null value, remove the surrounding quotes in the code
M Code
let
//Change next line to reflect your actual data source
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//change 2nd and later duplicates to null
dupsNull = List.Generate(
()=>[v=#"Changed Type"[Column1]{0}, idx=0],
each [idx]<Table.RowCount(#"Changed Type"),
each [v=if List.PositionOf(#"Changed Type"[Column1],#"Changed Type"[Column1]{[idx]+1},Occurrence.First) = [idx]+1
then #"Changed Type"[Column1]{[idx]+1} else "null", idx=[idx]+1],
each [v]),
//either add as a column or replace the first column
#"add de-duped" = Table.FromColumns(
Table.ToColumns(#"Changed Type") & {dupsNull},
type table[Column1=text, Column2=text])
in
#"add de-duped"
Here's another way. First, add an index column. Then add another column using List.PositionOf to get the row of the first occurrence of each value in the column. Then add one last column to compare the index and List.PositionOf, to determine which row entries should be null.
Let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each List.PositionOf(#"Added Index"[Column1],[Column1])),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Index] = [Custom] then [Column1] else null)
in
#"Added Custom1"
Here a solution that doesn't require to add a new column. It returns the same column just with duplicated values replaced with "null":
let
Source = Excel.CurrentWorkbook(){[Name="TB_INPUT"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
removeDups = (lst as list) =>
List.Accumulate(lst, {}, (x, y) => x & {if List.Contains(x, y) then "null" else y}),
replacedValues = removeDups(Table.Column(#"Changed Type", "Column1")),
#"replaced Values" = Table.FromList(replacedValues, null, type table[Column1 = Text.Type ])
in
#"replaced Values"
it uses a List.Accumulate function to simplify the process to generate the corresponding list with the specified requirements.
The output will be the following in Power Query:
and in Excel:
If you want an empty cell instead of "null" token, then in the function removeDups replace "null" with null.
If you want to consider a more general case, lets say you have more than one column in the input Excel Table and you want to replace duplicated values in more than one column at the same time.
Let's say we have the following input in Excel:
The following code can be used to replace duplicates in Column1 and Column2:
let
Source = Excel.CurrentWorkbook(){[Name="TB_GralCase"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", Int64.Type}}),
listOfColumns = {"Column1", "Column2"},
remainingColumns = List.Difference(Table.ColumnNames(#"Changed Type"), listOfColumns),
removeDups = (lst as list) =>
List.Accumulate(lst, {}, (x, y) => x & {if List.Contains(x, y) then "null" else y}),
replacedValues = List.Transform(listOfColumns, each removeDups(Table.Column( #"Changed Type", _))),
#"replaced values" = Table.FromColumns(
replacedValues & Table.ToColumns(Table.SelectColumns( #"Changed Type", remainingColumns)),
listOfColumns & remainingColumns
)
in
#"replaced values"
In listOfColumns variable, you define the list of columns you want to replace duplicates.
The the output in Power Query will be:

Split decimal from text as batch

Attempting to split decimal numbers in batch using a prevo=ious formula provided on here however the result is an error stating that null or "" or "x" (where is a number) cant be converted to the type list.
The formula:
=try Text.Remove([Column1],Text.ToList(Text.Remove([Column1],{"0".."9","."}))) otherwise null works when applied to a single column however when trying to create a create a table from these columns I get the followings errors:
Desired Output:
M Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table19"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each Table.FromColumns({
(try Text.Remove([Column1],Text.ToList(Text.Remove([Column1],{"0".."9","."}))) otherwise null),
(try Text.Remove([Column2],Text.ToList(Text.Remove([Column2],{"0".."9","."}))) otherwise null)
}))
in
#"Added Custom"
I would like to be able to generate a Table.FromColumns, for n columns which I can then expand. This is just an example and in reality, the number of columns can vary quite a lot.
Update
To better visualise what I am trying to do in power query I wish to create this scenario:
Such that this table can be expanded to:
Probably something obvious but any help appreciated.
I would just
split the columns based on character transition, including the decimal in the character list.
Then Trim the resultant columns to remove any leading/following spaces
Note: Code edited to allow for any number of columns to be split in two. Column names can be dynamic also
let
Source = Excel.CurrentWorkbook(){[Name="Table21"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}}),
//Generate new table from all the columns
//create List of columns
colList = Table.ToColumns(#"Changed Type"),
colNames = Table.ColumnNames(#"Changed Type"),
//convert each column
splitCols = List.Generate(
()=>[colPair=
List.Transform(colList{0},(li)=>
Splitter.SplitTextByCharacterTransition(
{"0".."9","."}, (c) => not List.Contains({"0".."9","."}, c))
(li)),
cn = colNames{0},
idx=0],
each [idx] < List.Count(colList),
each [colPair=
List.Transform(colList{[idx]+1},(li)=>
Splitter.SplitTextByCharacterTransition(
{"0".."9","."}, (c) => not List.Contains({"0".."9","."}, c))
(li)),
cn=colNames{[idx]+1},
idx=[idx]+1],
each List.Zip([colPair]) & {List.Transform({1..2}, (n)=> [cn] & "." & Text.From(n))}),
newCols = List.Combine(List.Transform(splitCols, each List.RemoveLastN(_,1))),
newColNames = List.Combine(List.Transform(splitCols, each List.Last(_))),
newTable = Table.FromColumns(newCols,newColNames),
//trim the excess spaces
trimOps = List.Transform(Table.ColumnNames(newTable), each {_, Text.Trim}),
trimAll = Table.TransformColumns(newTable, trimOps)
in
trimAll
Example with three columns
Again, if you want to retain the original columns in your result table, you need to change three lines in the code:
...
newCols = Table.ToColumns(#"Changed Type") & List.Combine(List.Transform(splitCols, each List.RemoveLastN(_,1))),
newColNames = Table.ColumnNames(#"Changed Type") & List.Combine(List.Transform(splitCols, each List.Last(_))),
newTable = Table.FromColumns(newCols,newColNames),
...
Edited to be usable for multiple columns
let Source =Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"Index"}, "Attribute", "Value"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Other Columns", "Value", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, false), {"Value.1", "Value.2"}),
#"Removed Columns" = Table.RemoveColumns(#"Split Column by Delimiter",{"Value.2"}),
#"rename1" = Table.TransformColumns(#"Removed Columns",{{"Attribute", each _&"a", type text}}),
#"Pivoted Column" = Table.RemoveColumns(Table.Pivot(#"rename1", List.Distinct(#"Lowercased Text"[Attribute]), "Attribute", "Value.1"),{"Index"}),
#"Removed Columns2" = Table.RemoveColumns(#"Split Column by Delimiter",{"Value.1"}),
rename = Table.TransformColumns(#"Removed Columns2",{{"Attribute", each _ & "b", type text}}),
#"Pivoted Column1" = Table.RemoveColumns(Table.Pivot(rename, List.Distinct(rename[Attribute]), "Attribute", "Value.2"),{"Index"}),
TFC = Table.FromColumns(Table.ToColumns(Source)&Table.ToColumns(#"Pivoted Column")&Table.ToColumns(#"Pivoted Column1"),Table.ColumnNames(Source)&Table.ColumnNames(#"Pivoted Column")&Table.ColumnNames(#"Pivoted Column1"))
in TFC
I would just duplicate the two original columns (Add Column > Duplicate column) and then split the resulting columns on the left most " " delimiter. No M code needed.

Feed cell value into excel query web browser URL

My problem:
Through New Query -> From Other Sources -> From Web, I entered a static URL that allowed me to load approximately 60k "IDs" from a webpage in JSON format.
I believe each of these IDs corresponds to an item.
So they're all loaded and organised in a column, with one ID per line, inside a Query tab.
For the moment, no problem.
Now I need to import information from a dynamic URL that depends on the ID.
So I need to import from URL in this form:
http://www.example.com/xxx/xxxx/ID
This imports the following for each ID:
name of correspond item,
average price,
supply,
demand,
etc.
After research I came to the conclusion that I had to use the "Advanced Editor" inside the query editor to reference the ID query tab.
However I have no idea how to put together the static part with the ID, and how to repeat that over the 60k lines.
I tried this:
let
Source = Json.Document(Web.Contents("https://example.com/xx/xxxx/" & ID)),
name1 = Source[name]
in
name1
This returns an error.
I think it's because I can't add a string and a column.
Question: How do I reference the value of the cell I'm interested in and add it to my string ?
Question: Is what I'm doing viable?
Question: How is Excel going to handle loading 60k queries?
Each query is only a few words to import.
Question: Is it possible to load information from 60k different URLs with one query?
EDIT : thank you very much for answer Alexis, was very helpful. So to avoid copying what you posted I did it without the function (tell me what you think of it) :
let
Source = Json.Document(Web.Contents("https://example.com/all-ID.json")),
items1 = Source[items],
#"Converted to Table" = Table.FromList(items1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "ID"}}),
#"Inserted Merged Column" = Table.AddColumn(#"Renamed Columns", "URL", each Text.Combine({"http://example.com/api/item/", Text.From([ID], "fr-FR")}), type text),
#"Added Custom" = Table.AddColumn(#"Inserted Merged Column", "Item", each Json.Document(Web.Contents([URL]))),
#"Expanded Item" = Table.ExpandRecordColumn(#"Added Custom", "Item", {"name"}, {"Item.name"})
in
#"Expanded Item"
Now the problem I have is that it takes ages to load up all the information I need from all the URLs.
As it turns out it's possible to extract from multiple IDs at once using this format : http://example.com/api/item/ID1,ID2,ID3,ID4,...,IDN
I presume that trying to load from an URL containing all of the IDs at once would not work out because the URL would contain way too many characters to handle.
So to speed things up, what I'm trying to do now is concatenate every Nth row into one cell, for example with N=3 :
205
651
320165
63156
4645
31
6351
561
561
31
35
would become :
205, 651, 320165
63156, 4645, 31
6351, 561, 561
31, 35
The "Group by" functionnality doesn't seem to be what I'm looking for, and I'm not sure how to automatise that throught Power Query
EDIT 2
So after a lot of testing I found a solution, even though it might not be the most elegant and optimal :
I created an index with a 1 step
I created another costum column, I associated every N rows with an N increasing number
I used "Group By" -> "All Rows" to create a "Count" column
Created a costum column "[Count][ID]
Finally I excracted values from that column and put a "," separator
Here's the code for N = 10 000 :
let
Source = Json.Document(Web.Contents("https://example.com/items.json")),
items1 = Source[items],
#"Converted to Table" = Table.FromList(items1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "ID"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"ID", Int64.Type}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Added Conditional Column" = Table.AddColumn(#"Added Index", "Custom", each if Number.RoundDown([Index]/10000) = [Index]/10000 then [Index] else Number.IntegerDivide([Index],10000)*10000),
#"Reordered Columns" = Table.ReorderColumns(#"Added Conditional Column",{"Index", "ID", "Custom"}),
#"Grouped Rows" = Table.Group(#"Reordered Columns", {"Custom"}, {{"Count", each _, type table}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom.1", each [Count][ID]),
#"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom.1", each Text.Combine(List.Transform(_, Text.From), ","), type text})
in
#"Extracted Values"
I think what you want to do here is create a custom function that you invoke with each of your ID values.
Let me give a similar example that should point you in the right direction.
Let's say I have a table named ListIDs which looks like this:
ID
----
1
2
3
4
5
6
7
8
9
10
and for each ID I want to pull some information from Wikipedia (e.g. for ID = 6 I want to lookup https://en.wikipedia.org/wiki/6 and return the Cardinal, Ordinal, Factorization, and Divisors of 6).
To get this for just one ID value my query would look like this (using 6 again):
let
Source = Web.Page(Web.Contents("https://en.wikipedia.org/wiki/6")),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Column2] = "Cardinal" or [Column2] = "Divisors" or [Column2] = "Factorization" or [Column2] = "Ordinal")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Column1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column2", "Property"}, {"Column3", "Value"}}),
#"Pivoted Column" = Table.Pivot(#"Renamed Columns", List.Distinct(#"Renamed Columns"[Property]), "Property", "Value")
in
#"Pivoted Column"
Now we want to convert this into a function so that we can use it as many times as we want without creating a bunch of queries. (Note: I've named this query/function WikiLookUp as well.) To do this, change it to the following:
let
WikiLookUp = (ID as text) =>
let
Source = Web.Page(Web.Contents("https://en.wikipedia.org/wiki/" & ID)),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Column2] = "Cardinal" or [Column2] = "Divisors" or [Column2] = "Factorization" or [Column2] = "Ordinal")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Column1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column2", "Property"}, {"Column3", "Value"}}),
#"Pivoted Column" = Table.Pivot(#"Renamed Columns", List.Distinct(#"Renamed Columns"[Property]), "Property", "Value")
in
#"Pivoted Column"
in
WikiLookUp
Notice that all we did is wrap it in another set of let...in and defined the parameter ID = text which gets substituted into the Source line near the end. The function should appear like this:
Now we can go back to our table which we've imported into the query editor and invoke our newly created function in a custom column. (Note: Make sure you convert your ID values to text type first since they're being appended to a URL.)
Add a custom column with the following definition (or use the Invoke Custom Function button)
= WikiLookUp([ID])
Expand that column to bring in all the columns you want and you're done!
Here's what that query's M code looks like:
let
Source = Excel.CurrentWorkbook(){[Name="ListIDs"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each WikiLookUp([ID])),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Cardinal", "Ordinal", "Factorization", "Divisors"}, {"Cardinal", "Ordinal", "Factorization", "Divisors"})
in
#"Expanded Custom"
The query should look like this:

Resources