How to use variable column name in a function for other functions? - excel

I have written two functions which take date as input and I'm gonna use them on multiple queries. Instead of doing manual work every time (call both functions, filter rows where first function returns True, expand record of the second function to columns, delete first function column) I thought I'd write another function that takes names of the table and the column with dates as parameters to automatize that process. My current table-based function works if I include specific column's date in the code, but those names will be different between different queries(tables).
Here's the table function's code:
(t as table) =>
let
FunctionFilter = Table.AddColumn(t, "DateFilter", each DateFilter([myDate2])),
FunctionPeriods = Table.AddColumn(#"FunctionFilter", "TimePeriods", each TimePeriods([myDate2])),
ExpandPeriods= Table.ExpandRecordColumn(FunctionPeriods, "TimePeriods", {"Year", "Quarter", "Month", "WeekMon", "WeekTue", "Day"},
{"Year", "Quarter", "Month", "WeekMon", "WeekTue", "Day"}),
TrueDate = Table.SelectRows(ExpandPeriods, each ([DateFilter] = true)),
DeleteDateFilter = Table.RemoveColumns(TrueDate,{"DateFilter"})
in
DeleteDateFilter
My only problem is inserting a variable column name in place of [myDate2] here:
FunctionFilter = Table.AddColumn(t, "DateFilter", each DateFilter([myDate2])),
FunctionPeriods = Table.AddColumn(#"FunctionFilter", "TimePeriods", each TimePeriods([myDate2])),
Using Table.Column(t,[column name]) returns a list instead of a date, which causes called date functions to throw a type mismatch error.

You may use such technique:
// Table
let
Source = #table(3,List.Zip({{"a".."d"},{1..4},List.Numbers(10,4,10)})),
fn = fn(Source, "Column3")
in
fn
// fn
(tbl as table, col as text) =>
let
i = Table.AddIndexColumn(tbl, "i", 0, 1),
add = Table.AddColumn(i, "new", each Table.Column(i, col){[i]}*10),
del = Table.RemoveColumns(add, "i")
in
del

Related

Find matching value in query based on text

I am building a table in power query and I want to find the matching value from a column in a row. Does anyone know how to do this? I import my source data with:
leagueDataSource = #"League Data All",
this gives me this table:
I then have a variable called:
leagueName = "Albania - Superliga",
and want to create another variable called activeSeason. How do I match the variable leagueName with the value in active Season ?
Found the answer myself :)
leagueName = "name",
a = List.PositionOf(leagueDataSource[League], leagueName, 0),
leagueID = Number.ToText(leagueDataSource[Active Season]{a}),

Power Query (M) _ Dynamically update a column list for List.Sum function

I'm not sure if even possible but the goal is to dynamically update a query based on the user selecting a date. I have a table in my Excel file while updates a value which feeds to PeriodString variable (below)
/*Parameter name = PeriodString */
let
Source = Excel.CurrentWorkbook(){[Name="PeriodString"]}[Content],
StrPeriod = Source[Value]{0}
in
StrPeriod
The part of the code I want to update is the [ ..months selected ].
=List.Sum({[FYOpening],[January],[February],[March],[April],[May]})
With the below variable
=List.Sum({PeriodStr})
I tried using Table.Column as I realize I have to convert the value to a list of selectable columns but I cant' get it to work.
=List.Sum({Table.Column(PeriodString{0},PeriodString[0])})
Expression.Error: We cannot convert the value "[FY Opening],[Januar..." to type List.
Details:
Value=[FY Opening],[January],[February],[March],[April],[May]
Type=[Type]
Let me know if possible / alternatives.
If you need exactly value like "[Col1],[Col2],[Col3]" for PeriodString, then use such code:
let
Source = #table({"a".."e"},{{1..5}, {6..10}}),
PeriodString = "[b],[d],[e]",
sum = Table.AddColumn(Source, "sum", each List.Sum(Expression.Evaluate("{"&PeriodString&"}", [_=_])))
in
sum
I'd prefer to use PQ list instead:
let
Source = #table({"a".."e"},{{1..5}, {6..10}}),
list = {"b","d","e"},
sum = Table.AddColumn(Source, "sum", each List.Sum(Record.ToList(Record.SelectFields(_, list))))
in
sum

Power Query: Function to search a column for a list of keywords and return only rows with at least one match

I am making a simple Google-like search function in Power Query.
Let's say I have a column called Description in a table called Database. The user then inputs some search queries like "dog, cat, animals". I want to filter Database for rows that contain at least one of these keywords. They keywords can change each time, depending on what the user types in a named range in Excel.
I know you can filter a column in Power Query for multiple keywords, like this:
FilterRows = Table.SelectRows(LastStep, each Text.Contains([English], "dog") or Text.Contains([English], "cat")),
but those keywords are static, and the column is also static. I want to be able to control both the keywords and the column name as variables. I think I need to write a function but I am not sure how to start.
Your question requires several moving parts.
First, I would get the keywords from a named range "Keywords" into a table like this:
{KeywordTbl}
let
GetKeywords = if Excel.CurrentWorkbook(){[Name="Keywords"]}[Content]{0}[Column1] = null then null else Text.Split(Excel.CurrentWorkbook(){[Name="Keywords"]}[Content]{0}[Column1], ", "),
ConvertToTable = Table.FromList(GetKeywords,null,{"Keywords"})
in
ConvertToTable
Secondly, store the column name where you want to search in an Excel named range called "ColName". Then pull the named range into Power Query like this:
{ColName}
let
GetColName = Excel.CurrentWorkbook(){[Name="ColName"]}[Content]{0}[Column1]
in
GetColName
Then I would write a function that takes 4 variables, the table and column you want to look in, and the table and column containing the keywords:
{SearchColForKeywords}
(LookInTbl as table, KeywordTbl as table, LookInCol as text, KeywordCol as text) =>
let
RelativeMerge = Table.AddColumn(LookInTbl, "RelativeJoin",
(Earlier) => Table.SelectRows(KeywordTbl,
each Text.Contains(Record.Field(Earlier, LookInCol), Record.Field(_, KeywordCol), Comparer.OrdinalIgnoreCase))),
ExpandRelativeJoin = Table.ExpandTableColumn(RelativeMerge, "RelativeJoin", {KeywordCol}, {"Keywords found"}),
FilterRows = Table.SelectRows(ExpandRelativeJoin, each [Keywords found] <> null and [Keywords found] <> ""),
// Concatenate multiple keyword founds line into one line
GroupAllData = Table.Group(FilterRows, {"Word ID"}, {{"AllData", each _, type table [First column=text, Second column=text, ... your other columns=text]}}),
AddCol = Table.AddColumn(GroupAllData, "Keywords found", each [AllData][Keywords found]),
ExtractValues = Table.TransformColumns(AddCol, {"Keywords found", each Text.Combine(List.Transform(_, Text.From), ", "), type text}),
DeleteAllData = Table.RemoveColumns(ExtractValues,{"AllData"}),
MergeQueries = Table.NestedJoin(DeleteAllData, {"Word ID"}, FilterRows, {"Word ID"}, "DeleteAllData", JoinKind.LeftOuter),
ExpandCols = Table.ExpandTableColumn(MergeQueries, "DeleteAllData", {"First Col name", "Second col name", ... "Your Other column names here"}),
DeleteKeywordsFound = Table.RemoveColumns(ExpandCols,{"Keywords found"})
in
DeleteKeywordsFound
FYI, half of this function has been developed by a user named lmkeF on PowerBI community. The full discussion is here. I merely improved on his solution.
Finally, I will use that function in another query like this:
StepName = SearchColForKeywords(MainTbl, KeywordTbl, ColName, "Keywords"),
You may customize the 4 variable names.

Filter a column using IF statement in powerquery

I'm having trouble filtering my columns in powerquery. I'm using a parameter to filter my 'Island' NI or SI, however I am struggling to find a way to keep all the data when no parameter is inputted.
I would like to leave the column full (NI and SI) if no input is provided. I've added two pictures on imgur below. I'm fairly new here.
I've tried using an if statement but couldn't figure out the right piece of code or if it's even possible to do so.
Thanks
enter image description here. enter image description here
Note: see the very end for a version that does not need to hard-code values.
First let's test if the search filter is a valid value. If not, don't filter anything.
if IslandChoice is "NI" or "SI"
then filter using IslandChoice
else
show everything
The other route is
if IslandChoice is not "NI" and is not "SI"
show everything
else
filter using IslandChoice
It depends if you want invalid values to show nothing, or everything.
First check "is this filter a valid choice?
[Island] = "SI" or [Island] = "WI"
If either one is true, then it's good. If both are false, then it's an invalid key.
filter_if_valid = Table.SelectRows(
Source,
(row) =>
if IslandChoice = "SI" or IslandChoice = "NI" then
row[Island] = IslandChoice
else
true
)
How does it work, and where is each ?
each is a normal function. It's sugar, a shortcut that declares a function which has a single arugment. (If you need more, use a function declaration)
see more: Details on the grammar of each is in the docs
each implicitly creates the variable named _
Because each abstracts a couple of things, it's harder to tell what's happening.
each creates a variable named _ which references the current row.
each declares a function without the arguments part
Functions follow the form
(arguments) => return_expression
Written using each it removes the (arguments) => portion.
each return_expression
return_expression can be a simple [Column] = 10 test. (Or it can create many local variables using a let .. in expression inside the function).
These statements are all exactly equivelent functions
each [Island] = IslandChoice
each _[Island] = IslandChoice
(_) => _[Island] = IslandChoice
(row) => row[Island] = IslandChoice
They all
declare a function without a name
accept exactly 1 argument
read the column [Island] in the current table row
returns true if they are equal, else false
original filter
after replacing each you had:
always_filter_table = Table.SelectRows(
Source,
(row) =>
row[Island] = IslandChoice
),
new filter
filter_if_valid = Table.SelectRows(
Source,
(row) =>
if IslandChoice = "SI" or IslandChoice = "NI" then
row[Island] = IslandChoice
else
true
)
filter without hard-coding values
This will run alone without any requirements. Create a new -> blank query
let
Source = #table(
{"Island", "Point Of Connection"},
{
{"SI", "STK0331"},
{"SI", "TIM0111"},
{"NI", "ZEJ2395"},
{"NI", "XER9345"},
{"WI", "QXF9785"}
}
),
is_valid_filter = (value as any, valid_values as list) =>
List.Contains( valid_values, value ),
// use one or the other if you want all existing values to be valid
static_valid_filters = {"SI", "NI"},
dynamic_valid_filters = List.Distinct(
Source[Island]
),
IslandChoice = "NI",
// if filter is a valid value, filter using it.
// otherwise show all.
filtered_when_valid = Table.SelectRows(
Source,
(row) =>
if is_valid_filter(
IslandChoice, static_valid_filters
) then
row[Island] = IslandChoice
else
true
)
in
filtered_when_valid

Trying to pull data from a SODA API into Excel

The API call looks like this:
https://data.edmonton.ca/resource/3pdp-qp95.json?house_number=10008&street_name=103%20STREET%20NW
and returns data in json:
[{"account_number":"3070208","garage":"N","house_number":"10008","latitude":"53.539158992619","longitude":"-113.497760691896","neighbourhood":"DOWNTOWN","street_name":"103 STREET NW","tax_class":"Non Residential","total_asmt":"1717000"}]
I have an excel table with specific house_number and street_name pairs and I want to capture the total_asmt column for each pair.
I've been able to create a power query which pulls the very first data point into a new sheet:
let
Parameter = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Removed Other Columns" = Table.SelectColumns(Parameter,{"house_number", "street_name"}),
X = #"Removed Other Columns"[house_number]{0},
Y = #"Removed Other Columns"[street_name]{0},
Source = Json.Document(Web.Contents("https://data.edmonton.ca/resource/3pdp-qp95.json?house_number="& X &"&street_name=" & Y)),
in
Source
I can't figure out how to iterate through all the value I have in X and Y or how to capture specific rows from the JSON data. Any help would be appreciated!
Thanks,
Aaleem
I think your best best is to not do it.
Why are you wasting your time scraping this data one address at a time when you could have the entire city's data in under a minute.
JSON: https://data.edmonton.ca/resource/3pdp-qp95.json
CSV: https://data.edmonton.ca/api/views/q7d6-ambg/rows.csv?accessType=DOWNLOAD
XML: https://data.edmonton.ca/api/views/q7d6-ambg/rows.xml?accessType=DOWNLOAD
...among others. Heck, they even have !
And when you're done with that one, they have a few hundred other interesting datasets.
The trick was to create a function inside powerquery, and then use the query as part of a table. Create the function as below and then under the data tab select your table using "From Table/Range" from there it is pretty straight forward.
let a_value= (x as number,y as text)=> //this creates the function
let //this is essentially the query I wanted with some minor changes from above
x_text = Number.ToText(x, "D", ""),
Source = Json.Document(Web.Contents("https://data.edmonton.ca/resource/3pdp-qp95.json?house_number="&x_text&"&street_name="&y)),
Source1 = Source{0},
total_asmt = Source1[total_asmt]
in
total_asmt
in a_value //closes the function

Resources