Applying function to each row in Query Custom Column - excel

Summary of problem:
I have a PowerQuery Table in Excel that contains 13 columns. The 13th Column is a custom column "Task Start Week Number". I want the PowerQuery to apply a formula to each of the rows generated for this Query. The formula is as follows:
=IFS(AND('Program Dates'!$B$2<WEEKNUM(New_Items_to_Save[Start Date]),
WEEKNUM(New_Items_to_Save[Start Date])<54),
'Program Dates'!$G$2-('Program Dates'!$D$2-(-53+WEEKNUM(New_Items_to_Save[Start Date]))),
WEEKNUM(New_Items_to_Save[Start Date])<'Program Dates'!$B$2,
'Program Dates'!$G$2-('Program Dates'!$D$2-(-53+WEEKNUM(New_Items_to_Save[Start Date])))+53)
What I've done here is reference a cell which contains the formula, that way I can just run the GetValue() function for a named range. I can't get this to work and I don't know what I'm doing wrong.
Thank you in advance for your help!
Context:
This is the query table I need to add the calculation to.
The last column is the custom column, and those values should be calculated using the following cells:
This is the source of the other info needed to calculate the week number of the program, with reference arrows shown.
Note: The dates referenced in the function have already been converted using the WEEKNUM() operation. I am comparing Week# to Week#, not Date to Week#
Function Logic:
AND: if the date falls within the range of the current year ie. week# is less than 54, but after the start of the program, then perform this calc.
IFS: otherwise, if week# is before the end of the program ie. 2023, then perform this calculation.
Edit:
Here is the PowerQuery function I want to call for each of the new cells in this custom column:
Parameter2 = Date.WeekOfYear(StartWeek)
let
GetWeek = ()
if GetValue("Start_Week") < Parameter2 < 54
then (GetValue("Program_Duration") - GetValue("End_Week") + 53 - Parameter2))
else
(GetValue("Program_Duration") - GetValue("End_Week") + 53 - Parameter2 +53))
in
GetWeek
I don't know if I need the let statement or if I should just put it in a function
f(x) => [equation]
and then call "...each f([column name])" in power query?

I think that there are actually three different parts to your question, and maybe your confusion is coming from combining them all together.
The way I see it is in these parts:
How to create a custom function.
How to apply a function to a new column.
How to apply a function to an existing column.
How to create a custom function
There are two main ways to create a custom function in Power Query:
Using the UI (follow steps here):
Step
Description
Image
1
Write your query
2
Parameterise your query
3
Create your function
Using only code (follow steps here):
Example to filter a table:
let fun_FilterTable = (tbl_InputTable as table, txt_FilterValue as text) as table =>
let
Source = tbl_InputTable,
Filter = Table.SelectRows(DayCount, each Text.Contains([Column], txt_FilterValue))
in
Filter
in
fun_FilterTable
Example to check if one string contains another:
let fun_CheckStringContains = (txt_String as text, txt_Check as text) as nullable logical =>
let
Source = txt_String,
Check = Text.Contains(Source, txt_Check)
in
Check
in
fun_CheckStringContains
More resources:
Using custom functions
Custom Functions Made Easy in Power BI Desktop
PowerQuery best practices
DataFlow best practices
How to apply a function to a new column
Also has two different ways to achieve:
Custom Column (follow steps here):
Step
Description
Image
1
Create custom column
2
Add function
Custom Function (follow steps here):
Step
Description
Image
1
Invoke custom function
Sources:
Add a custom column
Using custom functions
Custom Functions Made Easy in Power BI Desktop
How to apply a function to an existing column
Also has two different ways to achieve (unfortunately, only possible with pure code):
Using Transformation:
Example to uppercase an entire column:
let
Source = Table,
#"Uppercased text" = Table.TransformColumns(Source, {{"Column", each Text.Upper(_), type nullable text}})
in
#"Uppercased text"
Example to add a prefix to all rows in one column:
let
Source = Table,
#"Added prefix" = Table.TransformColumns(Source, {{"Column", each "test_" & _, type text}})
in
#"Added prefix"
Example to coerce column to date in Australian format:
let
Source = Table,
#"Fix date" = Table.TransformColumns(Source, {{"DateColumn", each Date.From(_, "en-AU"), type date}})
in
#"Fix date"
Using Replacement
Example to replace some text:
let
Source = Table,
#"Replaced value" = Table.ReplaceValue(Source, "Admin", "Administrator", Replacer.ReplaceText, {"Column"})
in
#"Replaced value"
Example to replace with values from another column
let
Source = Table,
#"Replaced value" = Table.ReplaceValue(Source, each [FixThisColumn], each [OtherColumn], Replacer.ReplaceText, {"FixThisColumn"})
in
#"Replaced value"
Your Specific Problem
Without some dummy data to use, I have created some here. Please note, in future, please provide some data in a minimum reproducible example (see here), so that we can easily recreate the scenario from your example.
Data:
ID
ProgramStartDate
ProgramEndDate
1
1/Jan/2020
1/Dec/2021
2
1/Jan/2022
1/Mar/2023
3
1/Mar/2022
1/Dec/2022
4
1/Sep/2021
1/Dec/2023
5
1/Jan/2023
1/Dec/2023
I think that you should be using a combination of the PowerQuery in-build date functions (see here) and some of the PowerQuery conditional processes (see here).
My code would look something like this:
let
Source = Table.FromColumns({{1,2,3,4,5},{"1/Jan/2020","1/Jan/2022","1/Mar/2022","1/Sep/2021","1/Jan/2023"},{"1/Dec/2021","1/Mar/2023","1/Dec/2022","1/Dec/2023","1/Dec/2023"}},{"ID","ProgramStartDate","ProgramEndDate"}),
fix_Types = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"ProgramStartDate", type date}, {"ProgramEndDate", type date}}),
add_Today = Table.AddColumn(fix_Types, "DateToday", each Date.From(DateTime.LocalNow()), type date),
add_CheckCurrentYear = Table.AddColumn(add_Today, "IsInCurrentYear", each Date.IsInCurrentYear([DateToday]), type logical),
add_CheckProgramRunning = Table.AddColumn(add_CheckCurrentYear, "ProgramIsCurrent", each [DateToday]>[ProgramStartDate] and [DateToday]<[ProgramEndDate], type logical),
add_ConditionalCheck = Table.AddColumn(add_CheckProgramRunning, "DoSomething", each if [IsInCurrentYear] and [ProgramIsCurrent] then "Do Something" else null, type text)
in
add_ConditionalCheck
And the final output would look something like this:
ID
ProgramStartDate
ProgramEndDate
DateToday
IsInCurrentYear
ProgramIsCurrent
DoSomething
1
1/01/2020
1/12/2021
22/12/2022
TRUE
FALSE
null
2
1/01/2022
1/03/2023
22/12/2022
TRUE
TRUE
Do Something
3
1/03/2022
1/12/2022
22/12/2022
TRUE
FALSE
null
4
1/09/2021
1/12/2023
22/12/2022
TRUE
TRUE
Do Something
5
1/01/2023
1/12/2023
22/12/2022
TRUE
FALSE
null
This should help you work towards resolving your issue.

Related

Error: Splitting rows into separate rows on all columns in Power Query

I had a problem spliting data in rows, used the solution provided by horseyride in the following link
Splitting rows into separate rows on all columns in Power Query.
Basically I am loocking to separete a row as breaks there are.
Many thanks #horseyride. The solution works in a simular problem. However, it's poping up the following error:
Expression.Error: We cannot convert a value of type Table to type Text.
Details:
Value=[Table]
Type=[Type]
My table is this one:
let
Source = Pdf.Tables(File.Contents("C:\Users\gmall\OneDrive\EF personales\EF\Temporales\IBK_Sueldo_PEN.pdf"), [Implementation="1.3"]),
Table002 = Source{[Id="Table002"]}[Data],
TableTransform = Table.Combine(List.Transform(List.Transform(Table.ToRecords(Source),
(x) => List.Transform(Record.ToList(x),each Text.Split(_,"#(lf)"))),
each Table.FromColumns(_,Table.ColumnNames(Source))))
in
TableTransform
Please let me know how to solve this issue:
Expression.Error: We cannot convert a value of type Table to type Text.
Details:
Value=[Table]
Type=[Type]
You need to use Table002 in step3 since that is the prior step name, not Source, which was the prior step name in my other answer
let
Source = Pdf.Tables(File.Contents("C:\Users\gmall\OneDrive\EF personales\EF\Temporales\IBK_Sueldo_PEN.pdf"), [Implementation="1.3"]),
Table002 = Source{[Id="Table002"]}[Data],
TableTransform = Table.Combine(List.Transform(List.Transform(Table.ToRecords(Table002),
(x) => List.Transform(Record.ToList(x),each Text.Split(_,"#(lf)"))),
each Table.FromColumns(_,Table.ColumnNames(Table002))))
in
TableTransform

Text.Contains for multiple values power query

I am attempting to create the following query:
The idea is to check if each row in the source query contains any of the following keywords in the Search list and return the Found words is present.
Importantly I need this to be dynamic i.e. the search list could be a single word or could be 100+ words. Therefore I need to work around just stitching a bunch of Text. Contains with or statements is possible.
In effect, I want to create something like
Text.Contains([Column1], {any value in search list}) then FoundWord else null
Data:
Physical hazards Flam. Liq. 3 - H226 Eliminate all sources of ignition.
Health hazards STOT SE 3 - H336. Avoid inhalation of vapours and contact with skin and eyes.
Environmental hazards Not Classified. Avoid the spillage or runoff entering drains, sewers or watercourses.
Personal precautions Keep unnecessary and unprotected personnel away from the spillage.
clothing as described in Section 8 of this safety data sheet. Provide adequate ventilation.
Search List:
Hazards
Eliminate
ventilation
Avoid
try this code for query Table2 after creating query lookfor
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
Findmatch = Table.AddColumn(Source, "Found", (x) => Text.Combine(Table.SelectRows(lookfor, each Text.Contains(x[Column1],[Column1], Comparer.OrdinalIgnoreCase))[Column1],", "))
in Findmatch

Extract CAS Number from Downloaded Data

I have downloaded a CSV file from Pubchem containing over 5000+ records. One of the columns contains a bunch of computed synonyms where CAS Number is the records I wish to extract. Unfortunately, the CAS number isn't necessarily in the same position in this list, making splitting by delimiter more difficult. Below is the source data example and the desired output I am trying to achieve.
An older answer to a post a while back used a Regex function to extract strings of Numbers with a given length.
fnRegexExtr
let fx=(text,regex)=>
Web.Page(
"<script>
var x='"&text&"';
var y=new RegExp('"&regex&"','g');
var b=x.match(y);
document.write(b);
</script>")[Data]{0}[Children]{0}[Children]{1}[Text]{0}
in
fx
Unsure if this is possible here and unfamiliar with Regex but I'm wondering if it is possible to modify this function to extract CAS numbers. The difficulty is that CAS Numbers can be in various formats CAS Numbers are up to 10 digits long using the format xxxxxxx-yy-z.
If anyone has any alternative solutions to extracting CAS numbers with this somewhat complex data feel free to post.
Data:
cid and cmpdname can be anything.
1-Aminopropan-2-ol|1-AMINO-2-PROPANOL|78-96-6|Isopropanolamine|Monoisopropanolamine
1-chloro-2,4-dinitrobenzene|2,4-Dinitrochlorobenzene|97-00-7|Dinitrochlorobenzene|DNCB|Chlorodinitrobenzene|CDNB
1,2-dichloroethane|Ethylene dichloride|107-06-2|Ethylene chloride|Ethane, 1,2-dichloro-|Glycol dichloride|Dutch liquid|Dutch oil|Ethane dichloride|Aethylenchloride
1,2,4-trichlorobenzene|120-82-1|Benzene, 1,2,4-trichloro-|unsym-Trichlorobenzene|Hostetex L-pec|Trojchlorobenzene
CHLOROACETALDEHYDE|2-chloroacetaldehyde|107-20-0|Chloroethanal|2-Chloroethanal|Acetaldehyde, chloro-|Chloroaldehyde|Monochloroacetaldehyde|2-Chloro-1-ethanal
In PQ, this will pull out the contents of any item that does not contain a letter in cmpdsynonym, which I think is basically what you are looking for
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom.3", each List.RemoveNulls(List.Transform(Text.Split([cmpdsynonym],"|"), each if _ = Text.Remove (_,{"A".."Z","a".."z"}) then _ else null)){0})
in #"Added Custom"
Here's one way of doing it in PQ, using fnRegexExtr to return the CAS; and a simple Text.Split to return the chemical compound name:
let
//Read in data and set data type as text
Source = Excel.CurrentWorkbook(){[Name="Compounds"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//Transform to desired output
Result = Table.FromColumns(
{List.Transform(#"Changed Type"[Column1], each Text.Split(_,"|")){0}}
& {List.Transform(#"Changed Type"[Column1],each fnRegexExtr(_, "\\b\\d{1,7}-\\d{2}-\\d"))},
type table[Compound=text, CAS=text]
)
in
Result
Original
Results

Power M query syntax to get the value for a named cell in Excel

I am still learning about Power Query and Power M and I'm trying to get the value of a specific "named" cell in Excel and use this in Power M. It is just a single cell and
=Record.Field(Excel.CurrentWorkbook(){[Name="weekone"]}[Content]{0},Excel.CurrentWorkbook(){[Name="weekone"]}[Content]{0})
Maybe I am not understanding the syntax of how to reach information in a particular field correctly, or I am getting mixed up on how to use the Record.Field() function.
Any help or guidance that can be provided would be greatly appreciated! Thanks!
Record.Field gives the value of a field in a record.
It takes the record as the first argument and the name of the field as the second argument.
In a step by step approach it will be clearer:
let
Source = Excel.CurrentWorkbook(){[Name="weekone"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type date}}),
FirstRecord = #"Changed Type"{0},
RecordValue = Record.Field(FirstRecord,"Column1")
in
RecordValue
Or, in 1 line:
= DateTime.Date(Record.Field(Excel.CurrentWorkbook(){[Name="weekone"]}[Content]{0},"Column1"))
This would be an alternative:
= DateTime.Date(Excel.CurrentWorkbook(){[Name="weekone"]}[Content]{0}[Column1])
My preference would be:
= DateTime.Date(Table.FirstValue(Excel.CurrentWorkbook(){[Name="weekone"]}[Content]))

Excel Power Query M: passing a header as function parameter?

In Excel Power Query, in the M language:
How do I pass a column name as a parameter to a function? As a (contrived) example, supposed I have a table:
Fish Fowl
1
2 1
1
2
I want a function which will take the table and one of the column-names, and return the sum of that column. I tried this implementation
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
SumType = (Tbl, ColumnName) =>
List.Sum(Tbl[ColumnName]),
FowlSum = SumType(ChType, "Fowl")
in
FowlSum
But it fails with an error: "The column 'ColumnName' wasn't found."
So how can I pass a column name (as a string) to a function, and then access that column within the functions?
Use Table.Column. In your case the line would look like List.Sum(Table.Column(Tbl, ColumnName)).

Resources