I have the following data with duplicates which I wish to identify. I do not wish to remove these so unique value only won't work. I want to be able to identify them but just saying null.
I have attempted to self-reference the code but end up just duplicating the original result.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Removed Duplicates" = Table.Distinct(#"Changed Type"),
#"Merged Queries" = Table.NestedJoin(Source, {"Column1"}, #"Removed Duplicates", {"Column1"}, "Removed Duplicates", JoinKind.LeftOuter)
in
#"Merged Queries"
You can use List.Generate to generate a list with your requirements. And then you can either replace the first column or add the list as a second column.
This needs to be done in the Advanced Editor.
Please note there is a difference between the text string "null" and the power query null value. Based on your example screenshot, I assumed you wanted the "null" text string. If you prefer the null value, remove the surrounding quotes in the code
M Code
let
//Change next line to reflect your actual data source
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//change 2nd and later duplicates to null
dupsNull = List.Generate(
()=>[v=#"Changed Type"[Column1]{0}, idx=0],
each [idx]<Table.RowCount(#"Changed Type"),
each [v=if List.PositionOf(#"Changed Type"[Column1],#"Changed Type"[Column1]{[idx]+1},Occurrence.First) = [idx]+1
then #"Changed Type"[Column1]{[idx]+1} else "null", idx=[idx]+1],
each [v]),
//either add as a column or replace the first column
#"add de-duped" = Table.FromColumns(
Table.ToColumns(#"Changed Type") & {dupsNull},
type table[Column1=text, Column2=text])
in
#"add de-duped"
Here's another way. First, add an index column. Then add another column using List.PositionOf to get the row of the first occurrence of each value in the column. Then add one last column to compare the index and List.PositionOf, to determine which row entries should be null.
Let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each List.PositionOf(#"Added Index"[Column1],[Column1])),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Index] = [Custom] then [Column1] else null)
in
#"Added Custom1"
Here a solution that doesn't require to add a new column. It returns the same column just with duplicated values replaced with "null":
let
Source = Excel.CurrentWorkbook(){[Name="TB_INPUT"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
removeDups = (lst as list) =>
List.Accumulate(lst, {}, (x, y) => x & {if List.Contains(x, y) then "null" else y}),
replacedValues = removeDups(Table.Column(#"Changed Type", "Column1")),
#"replaced Values" = Table.FromList(replacedValues, null, type table[Column1 = Text.Type ])
in
#"replaced Values"
it uses a List.Accumulate function to simplify the process to generate the corresponding list with the specified requirements.
The output will be the following in Power Query:
and in Excel:
If you want an empty cell instead of "null" token, then in the function removeDups replace "null" with null.
If you want to consider a more general case, lets say you have more than one column in the input Excel Table and you want to replace duplicated values in more than one column at the same time.
Let's say we have the following input in Excel:
The following code can be used to replace duplicates in Column1 and Column2:
let
Source = Excel.CurrentWorkbook(){[Name="TB_GralCase"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", Int64.Type}}),
listOfColumns = {"Column1", "Column2"},
remainingColumns = List.Difference(Table.ColumnNames(#"Changed Type"), listOfColumns),
removeDups = (lst as list) =>
List.Accumulate(lst, {}, (x, y) => x & {if List.Contains(x, y) then "null" else y}),
replacedValues = List.Transform(listOfColumns, each removeDups(Table.Column( #"Changed Type", _))),
#"replaced values" = Table.FromColumns(
replacedValues & Table.ToColumns(Table.SelectColumns( #"Changed Type", remainingColumns)),
listOfColumns & remainingColumns
)
in
#"replaced values"
In listOfColumns variable, you define the list of columns you want to replace duplicates.
The the output in Power Query will be:
Related
I am attempting to extract section heading numbers from a column in excel using power query.
I have already achieved this by matching with an existing list. However, I wonder if there is a better way to achieve this in fewer steps.
M Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table7"]}[Content],
#"Trimmed Text1" = Table.TransformColumns(Source,{{"Column1", PowerTrim, type text}}),
SectionNumbers = Table.AddColumn(#"Trimmed Text1", "SectionNumber", (x) => Text.Combine(Table.SelectRows(SectionNumbers, each Text.Contains(x[Column1],[SectionNumbers], Comparer.OrdinalIgnoreCase))[SectionNumbers],", ")),
#"Split Column by Delimiter2" = Table.SplitColumn(SectionNumbers, "SectionNumber", Splitter.SplitTextByEachDelimiter({","}, QuoteStyle.None, true), {"SectionNumber.1", "SectionNumber.2"}),
#"Added Custom1" = Table.AddColumn(#"Split Column by Delimiter2", "Custom", each if [SectionNumber.2] = null then [SectionNumber.1] else [SectionNumber.2]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Column1", "Custom"})
in
#"Removed Other Columns"
The Section numbers being matched to can be generated using:
SectionNumbers
let
Source = {1..16},
#"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Custom", each {1..9}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Added Custom1" = Table.AddColumn(#"Expanded Custom", "Custom.1", each "."),
#"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Added Custom1", {{"Custom", type text}}, "en-GB"),{"Custom", "Custom.1"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
#"Merged Columns1" = Table.CombineColumns(Table.TransformColumnTypes(#"Merged Columns", {{"Column1", type text}}, "en-GB"),{"Column1", "Merged"},Combiner.CombineTextByDelimiter(".", QuoteStyle.None),"SectionNumbers")
in
#"Merged Columns1"
Essentially I would like a way of extracting any decimal at the start of a cell, either 15.0. or 15.0, or even 15.0.1 etc.
I have considered using regex i.e. \d+\.\d+[.] which should work however I have many rows and find that regex sometimes is computationally intensive in this case, so it takes much longer to load than the Above M Code.
Another power query method
Since you know your section numbers you could:
Generate a (buffered) list of all the section numbers
see if the first space-separated part of the string in column 1 exists in the Section Number list.
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//create list of all serial numbers
SerialNumbers = List.Buffer(
let
Part1 = List.Transform({1..16}, each Text.From(_) & "."),
Part2 =List.Transform({1..10}, each Text.From(_) & "."),
sn = List.Accumulate(Part1,{}, (state, current)=> state &
List.Generate(
()=>[s=current & Part2{0}, idx=0],
each [idx] < List.Count(Part2),
each [s=current & Part2{[idx]+1}, idx=[idx]+1],
each [s]))
in
sn),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each
let
x = Text.Split([Column1]," "){0}
in
if List.Contains(SerialNumbers,x) then x else null, type text)
in
#"Added Custom"
How about
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if Text.PositionOfAny([Column1], {"0".."9"})>=0 then Text.BeforeDelimiter(Text.From([Column1])," ") else null)
in #"Added Custom"
I have the following function, which splits text into sentences using regex. Upon testing, however, these are instances where the regex doesn't quite work, and the text is wrongly split. For example, if the text contains St. bernard, I do not want this sentence to be split on the . of St.
As a nice workaround, I have modified the regex to allow for exceptions to be ignored. Please see here if you are interested in this.
I now wish to incorporate this into excel such that any user can apply their own exceptions however I am having trouble getting the user parameter to be passed into the string (regex) of the function.
Here is what I am trying to achieve (stating |Flam|Liq|St explicitly in the regex):
regex:
\s*((?:\b(?:[djms]rs?|flam|liq|St)\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$)) (https://regex101.com/r/nXf0TM/6)
However, what I would like to achieve is something like:
\s*((?:\b(?:[djms]rs?|"&Exceptions&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))
Where Exceptions is the User parameters entered in a table in excel: flam, liq, St in each row.
i.e.:
M code attempting to achieve this results in an error:
Sentences From text:
let
Exceptions = Exceptions,
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Text", type text}}),
#"Replaced Value1" = Table.ReplaceValue(#"Changed Type","#(lf)"," ",Replacer.ReplaceText,{"Text"}),
#"Replaced Value" = Table.ReplaceValue(#"Replaced Value1","'","&apos",Replacer.ReplaceText,{"Text"}),
#"Invoked Custom Function" = Table.AddColumn(#"Replaced Value", "fnRegexReplace", each fnRegexReplace([Text], "\s*((?:\b(?:[djms]rs?"&Exceptions&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))", "$1|")),
#"Removed Other Columns" = Table.SelectColumns(#"Invoked Custom Function",{"fnRegexReplace"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Removed Other Columns", {{"fnRegexReplace", Splitter.SplitTextByDelimiter("|", QuoteStyle.None), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "fnRegexReplace"),
#"Filtered Rows" = Table.SelectRows(#"Split Column by Delimiter", each ([fnRegexReplace] <> ""))
in
#"Filtered Rows"
Exceptions:
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each "Exceptions"),
#"Grouped Rows" = Table.Group(#"Added Custom", {"Custom"}, {{"Exceptions", each Text.Combine([#"Do not split if:"],"|"), type text}})
in
#"Grouped Rows"
fnRegexReplace
(x,y,z)=>
let
y = Text.Replace(y,"\","\\"),
Source = Web.Page(
"<script>var x="&"'"&x&"'"&";var z="&"'"&z&
"'"&";var y=new RegExp('"&y&"','gmi');
var b=x.replace(y,z);document.write(b);</script>")
[Data]{0}[Children]{0}[Children]{1}[Text]{0}
in
Source
Error:
Raw text Data:
Highly Flammable Liquid Flam. H223 Liq. H334.
St. Bernard Dog was present.
The MW of gold is 100.1. Solubility is 40mg/L.
Im sure this is an easy fix, but whatever I try, i.e. Record.FromTable{0} etc I get various errors.
If anyone could help me out, that would be great.
Thank you.
This is the problem line fixed.
= Table.AddColumn(#"Replaced Value", "fnRegexReplace", each fnRegexReplace([Text], "\s*((?:\b(?:[djms]rs?"&Exceptions[Exceptions]{0}&")\.|\b(?:[a-z]\.){2,}|\.\d[\d.]*|\.(?:com|net|org)\b|[^.?!])+(?:[.?!]+|$))", "$1|"))
Attempting to split decimal numbers in batch using a prevo=ious formula provided on here however the result is an error stating that null or "" or "x" (where is a number) cant be converted to the type list.
The formula:
=try Text.Remove([Column1],Text.ToList(Text.Remove([Column1],{"0".."9","."}))) otherwise null works when applied to a single column however when trying to create a create a table from these columns I get the followings errors:
Desired Output:
M Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table19"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each Table.FromColumns({
(try Text.Remove([Column1],Text.ToList(Text.Remove([Column1],{"0".."9","."}))) otherwise null),
(try Text.Remove([Column2],Text.ToList(Text.Remove([Column2],{"0".."9","."}))) otherwise null)
}))
in
#"Added Custom"
I would like to be able to generate a Table.FromColumns, for n columns which I can then expand. This is just an example and in reality, the number of columns can vary quite a lot.
Update
To better visualise what I am trying to do in power query I wish to create this scenario:
Such that this table can be expanded to:
Probably something obvious but any help appreciated.
I would just
split the columns based on character transition, including the decimal in the character list.
Then Trim the resultant columns to remove any leading/following spaces
Note: Code edited to allow for any number of columns to be split in two. Column names can be dynamic also
let
Source = Excel.CurrentWorkbook(){[Name="Table21"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}}),
//Generate new table from all the columns
//create List of columns
colList = Table.ToColumns(#"Changed Type"),
colNames = Table.ColumnNames(#"Changed Type"),
//convert each column
splitCols = List.Generate(
()=>[colPair=
List.Transform(colList{0},(li)=>
Splitter.SplitTextByCharacterTransition(
{"0".."9","."}, (c) => not List.Contains({"0".."9","."}, c))
(li)),
cn = colNames{0},
idx=0],
each [idx] < List.Count(colList),
each [colPair=
List.Transform(colList{[idx]+1},(li)=>
Splitter.SplitTextByCharacterTransition(
{"0".."9","."}, (c) => not List.Contains({"0".."9","."}, c))
(li)),
cn=colNames{[idx]+1},
idx=[idx]+1],
each List.Zip([colPair]) & {List.Transform({1..2}, (n)=> [cn] & "." & Text.From(n))}),
newCols = List.Combine(List.Transform(splitCols, each List.RemoveLastN(_,1))),
newColNames = List.Combine(List.Transform(splitCols, each List.Last(_))),
newTable = Table.FromColumns(newCols,newColNames),
//trim the excess spaces
trimOps = List.Transform(Table.ColumnNames(newTable), each {_, Text.Trim}),
trimAll = Table.TransformColumns(newTable, trimOps)
in
trimAll
Example with three columns
Again, if you want to retain the original columns in your result table, you need to change three lines in the code:
...
newCols = Table.ToColumns(#"Changed Type") & List.Combine(List.Transform(splitCols, each List.RemoveLastN(_,1))),
newColNames = Table.ColumnNames(#"Changed Type") & List.Combine(List.Transform(splitCols, each List.Last(_))),
newTable = Table.FromColumns(newCols,newColNames),
...
Edited to be usable for multiple columns
let Source =Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"Index"}, "Attribute", "Value"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Other Columns", "Value", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, false), {"Value.1", "Value.2"}),
#"Removed Columns" = Table.RemoveColumns(#"Split Column by Delimiter",{"Value.2"}),
#"rename1" = Table.TransformColumns(#"Removed Columns",{{"Attribute", each _&"a", type text}}),
#"Pivoted Column" = Table.RemoveColumns(Table.Pivot(#"rename1", List.Distinct(#"Lowercased Text"[Attribute]), "Attribute", "Value.1"),{"Index"}),
#"Removed Columns2" = Table.RemoveColumns(#"Split Column by Delimiter",{"Value.1"}),
rename = Table.TransformColumns(#"Removed Columns2",{{"Attribute", each _ & "b", type text}}),
#"Pivoted Column1" = Table.RemoveColumns(Table.Pivot(rename, List.Distinct(rename[Attribute]), "Attribute", "Value.2"),{"Index"}),
TFC = Table.FromColumns(Table.ToColumns(Source)&Table.ToColumns(#"Pivoted Column")&Table.ToColumns(#"Pivoted Column1"),Table.ColumnNames(Source)&Table.ColumnNames(#"Pivoted Column")&Table.ColumnNames(#"Pivoted Column1"))
in TFC
I would just duplicate the two original columns (Add Column > Duplicate column) and then split the resulting columns on the left most " " delimiter. No M code needed.
enter image description here
I have first column on the left and I want to transform to column on the right using PowerQuery.
I don't want to use VBA. It must be in PowerQuery.
Do You know the way to transform?
Try this,
Name of the Source-Table = "tbl_Shop_SKU"
Quelle = Excel.CurrentWorkbook(){[Name="tbl_Shop_SKU"]}[Content],
Change_Type = Table.TransformColumnTypes(Quelle,{{"ShopNo", Int64.Type}, {"SKU", Int64.Type}}),
GroupRows = Table.Group(Change_Type, {"SKU"}, {{"TempTable", each _, type table [ShopNo=nullable number, SKU=nullable number]}}),
Custom_ShopNo = Table.AddColumn(GroupRows, "List_ShopNo", each [TempTable][ShopNo]),
Extract_Value = Table.TransformColumns(Custom_ShopNo, {"List_ShopNo", each Text.Combine(List.Transform(_, Text.From), ","), type text}),
Remove_other_Columns = Table.SelectColumns(Extract_Value,{"SKU", "List_ShopNo"})
in
Remove_other_Columns```
Best regards
Chris
Assuming table is Table1 you could do this, pasting the text from the steps into Home ... Advanced Editor ....
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ShopNo", type text}, {"SKU", Int64.Type}}),
New= Table.Group( #"Changed Type", {"SKU"}, {{"ShopNo2", each Text.Combine([ShopNo], ";"), type text}})
in New
or to save a step, this also converts the numbers to text without the second step
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
New= Table.Group( Source, {"SKU"}, {{"ShopNo2", each Text.Combine(List.Transform([ShopNo], each Text.From(_)), ";"), type text}})
in New
My problem:
Through New Query -> From Other Sources -> From Web, I entered a static URL that allowed me to load approximately 60k "IDs" from a webpage in JSON format.
I believe each of these IDs corresponds to an item.
So they're all loaded and organised in a column, with one ID per line, inside a Query tab.
For the moment, no problem.
Now I need to import information from a dynamic URL that depends on the ID.
So I need to import from URL in this form:
http://www.example.com/xxx/xxxx/ID
This imports the following for each ID:
name of correspond item,
average price,
supply,
demand,
etc.
After research I came to the conclusion that I had to use the "Advanced Editor" inside the query editor to reference the ID query tab.
However I have no idea how to put together the static part with the ID, and how to repeat that over the 60k lines.
I tried this:
let
Source = Json.Document(Web.Contents("https://example.com/xx/xxxx/" & ID)),
name1 = Source[name]
in
name1
This returns an error.
I think it's because I can't add a string and a column.
Question: How do I reference the value of the cell I'm interested in and add it to my string ?
Question: Is what I'm doing viable?
Question: How is Excel going to handle loading 60k queries?
Each query is only a few words to import.
Question: Is it possible to load information from 60k different URLs with one query?
EDIT : thank you very much for answer Alexis, was very helpful. So to avoid copying what you posted I did it without the function (tell me what you think of it) :
let
Source = Json.Document(Web.Contents("https://example.com/all-ID.json")),
items1 = Source[items],
#"Converted to Table" = Table.FromList(items1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "ID"}}),
#"Inserted Merged Column" = Table.AddColumn(#"Renamed Columns", "URL", each Text.Combine({"http://example.com/api/item/", Text.From([ID], "fr-FR")}), type text),
#"Added Custom" = Table.AddColumn(#"Inserted Merged Column", "Item", each Json.Document(Web.Contents([URL]))),
#"Expanded Item" = Table.ExpandRecordColumn(#"Added Custom", "Item", {"name"}, {"Item.name"})
in
#"Expanded Item"
Now the problem I have is that it takes ages to load up all the information I need from all the URLs.
As it turns out it's possible to extract from multiple IDs at once using this format : http://example.com/api/item/ID1,ID2,ID3,ID4,...,IDN
I presume that trying to load from an URL containing all of the IDs at once would not work out because the URL would contain way too many characters to handle.
So to speed things up, what I'm trying to do now is concatenate every Nth row into one cell, for example with N=3 :
205
651
320165
63156
4645
31
6351
561
561
31
35
would become :
205, 651, 320165
63156, 4645, 31
6351, 561, 561
31, 35
The "Group by" functionnality doesn't seem to be what I'm looking for, and I'm not sure how to automatise that throught Power Query
EDIT 2
So after a lot of testing I found a solution, even though it might not be the most elegant and optimal :
I created an index with a 1 step
I created another costum column, I associated every N rows with an N increasing number
I used "Group By" -> "All Rows" to create a "Count" column
Created a costum column "[Count][ID]
Finally I excracted values from that column and put a "," separator
Here's the code for N = 10 000 :
let
Source = Json.Document(Web.Contents("https://example.com/items.json")),
items1 = Source[items],
#"Converted to Table" = Table.FromList(items1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "ID"}}),
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"ID", Int64.Type}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 0, 1),
#"Added Conditional Column" = Table.AddColumn(#"Added Index", "Custom", each if Number.RoundDown([Index]/10000) = [Index]/10000 then [Index] else Number.IntegerDivide([Index],10000)*10000),
#"Reordered Columns" = Table.ReorderColumns(#"Added Conditional Column",{"Index", "ID", "Custom"}),
#"Grouped Rows" = Table.Group(#"Reordered Columns", {"Custom"}, {{"Count", each _, type table}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom.1", each [Count][ID]),
#"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom.1", each Text.Combine(List.Transform(_, Text.From), ","), type text})
in
#"Extracted Values"
I think what you want to do here is create a custom function that you invoke with each of your ID values.
Let me give a similar example that should point you in the right direction.
Let's say I have a table named ListIDs which looks like this:
ID
----
1
2
3
4
5
6
7
8
9
10
and for each ID I want to pull some information from Wikipedia (e.g. for ID = 6 I want to lookup https://en.wikipedia.org/wiki/6 and return the Cardinal, Ordinal, Factorization, and Divisors of 6).
To get this for just one ID value my query would look like this (using 6 again):
let
Source = Web.Page(Web.Contents("https://en.wikipedia.org/wiki/6")),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Column2] = "Cardinal" or [Column2] = "Divisors" or [Column2] = "Factorization" or [Column2] = "Ordinal")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Column1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column2", "Property"}, {"Column3", "Value"}}),
#"Pivoted Column" = Table.Pivot(#"Renamed Columns", List.Distinct(#"Renamed Columns"[Property]), "Property", "Value")
in
#"Pivoted Column"
Now we want to convert this into a function so that we can use it as many times as we want without creating a bunch of queries. (Note: I've named this query/function WikiLookUp as well.) To do this, change it to the following:
let
WikiLookUp = (ID as text) =>
let
Source = Web.Page(Web.Contents("https://en.wikipedia.org/wiki/" & ID)),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Column2] = "Cardinal" or [Column2] = "Divisors" or [Column2] = "Factorization" or [Column2] = "Ordinal")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Column1"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column2", "Property"}, {"Column3", "Value"}}),
#"Pivoted Column" = Table.Pivot(#"Renamed Columns", List.Distinct(#"Renamed Columns"[Property]), "Property", "Value")
in
#"Pivoted Column"
in
WikiLookUp
Notice that all we did is wrap it in another set of let...in and defined the parameter ID = text which gets substituted into the Source line near the end. The function should appear like this:
Now we can go back to our table which we've imported into the query editor and invoke our newly created function in a custom column. (Note: Make sure you convert your ID values to text type first since they're being appended to a URL.)
Add a custom column with the following definition (or use the Invoke Custom Function button)
= WikiLookUp([ID])
Expand that column to bring in all the columns you want and you're done!
Here's what that query's M code looks like:
let
Source = Excel.CurrentWorkbook(){[Name="ListIDs"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each WikiLookUp([ID])),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Cardinal", "Ordinal", "Factorization", "Divisors"}, {"Cardinal", "Ordinal", "Factorization", "Divisors"})
in
#"Expanded Custom"
The query should look like this: