Create Power Query Column Recursively - excel

EMI Calculation:
Number.Round((rate/12*pv)/(1-Number.Power(1+rate/12,-nper)),0)
where PV is initial loan amount
Rate is the initial rate of interest at the beginning
and nper is 180 = Tenure in months
here the EMI = 107485
Keeping EMI Fixed, now i need to compute monthly interest and principal
The interest calculated formula will be
Previous balance * Rate of interest/12 * no. of days / no of days in the previous month
So if there is no rate change the interest will be prev bal * rate of interest /12 else it will be for the no of days new interest rate applicable.
Principal = EMI-Interest paid
New Balance = Previous Balance - Principal
Amount
What i am not able to achieve is :
Every time Interest is calculated, i want to refer to the previous balance which is not available.
I have tried creating Recursive functions to compute Interest, principal and balance columns:
Below function computes EMI value at the beginning
PMT
(pv,rate,nper)=>
let
payment = Number.Round((rate/12*pv)/(1-Number.Power(1+rate/12,-nper)),0)
in
payment
Below function computes Interest amount (Index Value is the row number / payment no)
Get_InterestValue
(indexValue,table,rate,no_of_days,no_of_days_in_Month)=>
let
prev_idx=indexValue-1,
previousbal = get_previous_balance(indexValue,table),
interest=previousbal*rate/12/no_of_days_in_Month*no_of_days
in
interest
Below function computes Principal amount paid
GetPrincipalPaid
(indexvalue,table,EMI_amount,Interest_amount)=>
let
principal_amount= EMI_amount-Interest_amount
in
principal_amount
Below function computes Previous balance
get_previous_balance
(index_value,table)=>
let
prev_idx=index_value-1,
prev_bal=if prev_idx=0 then fParameter("ParameterTable","Initial_Loan_Amount") else Table.SelectRows(table, each ([Index] = prev_idx)){0}[Bal]
in
prev_bal
fParameter function reads the intial values table
The columns gives errors

I have found a solution as in following code:
Now I am facing performance issue. Below solution works good with 10 Months or smaller. once u go for a higher Tenure, the calculations are very slow and doesnt generate any output for long time or probably crashes before calculations are completed,
is there any way i can speed up the query processing?
PMT Function:
(pv,rate,nper)=>
let
payment = Number.Round((rate/12*pv)/(1-Number.Power(1+rate/12,-nper)),0)
in
payment
fParameter Function:
let Parameter=(TableName,ParameterLabel) =>
let
Source=Excel.CurrentWorkbook(){[Name=TableName]}[Content],
value=Source{[Parameter=ParameterLabel]}[Value]
in
value
in Parameter
Payment Schedule Query:
let
Custom1 = List.Generate(()=>Date.Month(fParameter("ParameterTable","StartDate")),each _<=fParameter("ParameterTable","Tenure_in_Months")*1.5+Date.Month(fParameter("ParameterTable","StartDate"))-1,each _+1),
#"Converted to Table" = Table.FromList(Custom1, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Month", each if Number.Mod([Column1],12)=0 then 12 else Number.Mod([Column1],12)),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Year1", each Date.Year(fParameter("ParameterTable","StartDate"))+Number.RoundDown([Column1]/12)),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "Year", each if [Month]=12 then [Year1]-1 else [Year1]),
#"Renamed Columns1" = Table.RenameColumns(#"Added Custom2",{{"Column1", "PaymentMonth"}}),
#"Added Custom3" = Table.AddColumn(#"Renamed Columns1", "Payment Date", each #date([Year],[Month],Date.Day(fParameter("ParameterTable","StartDate")))),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom3",{"Payment Date"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Other Columns",{{"Payment Date", type date}}),
#"Added Custom4" = Table.AddColumn(#"Changed Type", "EMI", each EMIValue),
#"Merged Queries" = Table.NestedJoin(#"Added Custom4",{"Payment Date"},Loan_Rates,{"Date"},"Loan_Rates",JoinKind.FullOuter),
#"Expanded Loan_Rates" = Table.ExpandTableColumn(#"Merged Queries", "Loan_Rates", {"Date", "Loan Rate"}, {"Date", "Loan Rate"}),
#"Added Custom5" = Table.AddColumn(#"Expanded Loan_Rates", "New_Date", each if [Date] = null then [Payment Date] else [Date]),
#"Sorted Rows" = Table.Sort(#"Added Custom5",{{"New_Date", Order.Ascending}}),
#"Filled Down" = Table.FillDown(#"Sorted Rows",{"Loan Rate"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Payment Date", "Date"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"New_Date", "Payment Date"}}),
#"Reordered Columns" = Table.ReorderColumns(#"Renamed Columns",{"Payment Date", "EMI", "Loan Rate"}),
#"Added Index" = Table.AddIndexColumn(#"Reordered Columns", "Index", 0, 1),
#"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1),
#"Merged Queries1" = Table.NestedJoin(#"Added Index1",{"Index"},#"Added Index1",{"Index.1"},"Added Index1",JoinKind.LeftOuter),
#"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries1", "Added Index1", {"Payment Date"}, {"Payment Date.1"}),
#"Added Custom7" = Table.AddColumn(#"Expanded Added Index1", "Payment.Date.2", each if [Index.1]=1 then fParameter("ParameterTable","Disbursement Date") else [Payment Date.1]),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom7",{{"Payment.Date.2", type date}}),
#"Removed Columns2" = Table.RemoveColumns(#"Changed Type1",{"Payment Date.1"}),
#"Renamed Columns2" = Table.RenameColumns(#"Removed Columns2",{{"Payment.Date.2", "Payment Date.1"}}),
#"Added Custom6" = Table.AddColumn(#"Renamed Columns2", "Days", each [Payment Date]-[Payment Date.1]),
#"Extracted Days" = Table.TransformColumns(#"Added Custom6",{{"Days", Duration.Days, Int64.Type}}),
#"Added Custom8" = Table.AddColumn(#"Extracted Days", "No Of Days in Month", each Date.EndOfMonth([Payment Date.1])-Date.StartOfMonth([Payment Date.1])+#duration(1,0,0,0)),
#"Removed Columns1" = Table.RemoveColumns(#"Added Custom8",{"Index", "Payment Date.1"}),
#"Extracted Days1" = Table.TransformColumns(#"Removed Columns1",{{"No Of Days in Month", Duration.Days, Int64.Type}}),
#"Reordered Columns1" = Table.ReorderColumns(#"Extracted Days1",{"Payment Date", "Days", "EMI", "Loan Rate"}),
#"Renamed Columns3" = Table.RenameColumns(#"Reordered Columns1",{{"Index.1", "Index"}}),
#"Reordered Columns2" = Table.ReorderColumns(#"Renamed Columns3",{"Index", "Payment Date", "EMI", "No Of Days in Month", "Days", "Loan Rate"}),
#"Replaced Value" = Table.ReplaceValue(#"Reordered Columns2",null,0,Replacer.ReplaceValue,{"EMI"}),
#"Invoked Custom Function" = Table.AddColumn(#"Replaced Value", "Interest", each Get_InterestValue([Index], Table.Buffer(#"Replaced Value"))),
#"Added Custom9" = Table.AddColumn(#"Invoked Custom Function", "Principal", each [EMI]-[Interest]),
#"Added Custom11" = Table.AddColumn(#"Added Custom9", "Balance", each fParameter("ParameterTable","Initial_Loan_Amount")-List.Sum(List.FirstN(#"Added Custom9"[Principal],[Index]))),
#"Filtered Rows" = Table.SelectRows(#"Added Custom11", each [Interest] > 0)
in
#"Filtered Rows"
Loan_Rates table Query == This reads change of loan rates over a period:
let
Source = Excel.CurrentWorkbook(){[Name="Loan_Rates"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Loan Rate", Percentage.Type}})
in
#"Changed Type"
Get_InterestValue Function
(indexValue,table)=>
let
prev_idx=indexValue-1,
rate=Table.SelectRows(table, each ([Index] = indexValue)){0}[Loan Rate],
no_of_days_in_Month=Table.SelectRows(table, each ([Index] = indexValue)){0}[No Of Days in Month],
no_of_days=Table.SelectRows(table, each ([Index] = indexValue)){0}[Days],
previousbal = get_previous_balance(indexValue,table),
interest=previousbal*rate/12/no_of_days_in_Month*no_of_days
in
interest
GetPrincipalPaid Function:
(indexValue,table)=>
let
EMI_Amount=Table.SelectRows(table, each ([Index] = indexValue)){0}[EMI],
principal_amount= EMI_Amount-Get_InterestValue(indexValue,table)
in
principal_amount
GetBalance Function:
(indexValue,table)=>
let
bal = get_previous_balance(indexValue,table)-GetPrincipalPaid(indexValue,table)
in
bal
get_previous_balance function:
(indexValue,table)=>
let
prev_idx=indexValue-1,
prev_bal=if prev_idx=0 then fParameter("ParameterTable","Initial_Loan_Amount") else GetBalance(prev_idx,table)
in
prev_bal
EMIValue Function:
let
Source = PMT(fParameter("ParameterTable","Initial_Loan_Amount"), fParameter("ParameterTable","Rate_int_PA"), fParameter("ParameterTable","Tenure_in_Months")),
#"Converted to Table" = #table(1, {{Source}}),
#"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "EMI"}}){0}[EMI]
in
#"Renamed Columns"

Related

Extracting Text Between boundaries by applying logical parameters

First I think this is a complicated question to follow. Please see the Steps of my M code which I think will make it clearer.
So I am trying to achieve the following:
The idea is any text in the input box can be limited to relevant sections using the parameters table If the Text in the parameters box is Contained in the Text being searched. For me, at least the question is more complicated than it first appears. If required /interested Please see my explanation below:
Desired Output
Fundamentally I want a way of filtering the Input text, using the parameters box such that each line returned is contained only within the relevant sections of start1-End1, Start2-End2.
Ideally, you can use any part of the text to set the limits. So I could say Everything between SECTION1-2, and between lines 2 and 5. will returns lines 2-5.
Or you could say Everything between SECTION 1-3, lines 4-8. Will return lines 4-8. Note you could even say SECTION1-3, Lines 4 -SECTION 4 which would return lines 4 up to 14.
Finally you could even overlap sections and These should still be captured separately and the lines where overlap occurs should repeat in the output.
M Code:
Parameters:
let
Source = Excel.CurrentWorkbook(){[Name="Parameters"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Start1", type text}, {"End1", type text}, {"Start2", type text}, {"End2", type text}}),
#"Filled Down" = Table.FillDown(#"Changed Type",{"Start1", "End1"}),
#"Duplicated Column1" = Table.DuplicateColumn(#"Filled Down", "Start1", "Start1 - Copy"),
#"Duplicated Column2" = Table.DuplicateColumn(#"Duplicated Column1", "End1", "End1 - Copy"),
#"Duplicated Column3" = Table.DuplicateColumn(#"Duplicated Column2", "Start2", "Start2 - Copy"),
#"Duplicated Column4" = Table.DuplicateColumn(#"Duplicated Column3", "End2", "End2 - Copy"),
#"Added Custom" = Table.AddColumn(#"Duplicated Column4", "Custom", each "X"),
#"Merged Columns" = Table.CombineColumns(#"Added Custom",{"Start1", "Custom"},Combiner.CombineTextByDelimiter("+++", QuoteStyle.None),"Start1"),
#"Duplicated Column" = Table.DuplicateColumn(#"Merged Columns", "End1", "End1 - Copy.1"),
#"Merged Columns3" = Table.CombineColumns(#"Duplicated Column",{"Start1", "End1"},Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Search1"),
#"Added Custom1" = Table.AddColumn(#"Merged Columns3", "Custom", each "X"),
#"Added Custom3" = Table.AddColumn(#"Added Custom1", "Custom.1", each "X"),
#"Merged Columns1" = Table.CombineColumns(#"Added Custom3",{"Start2", "Custom"},Combiner.CombineTextByDelimiter("+++", QuoteStyle.None),"Start2"),
#"Merged Columns4" = Table.CombineColumns(#"Merged Columns1",{"End2", "Custom.1"},Combiner.CombineTextByDelimiter("+++", QuoteStyle.None),"End2"),
#"Merged Columns2" = Table.CombineColumns(#"Merged Columns4",{"Start2", "End2", "End1 - Copy.1"},Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Search2"),
#"Added Custom2" = Table.AddColumn(#"Merged Columns2", "Custom", each Table.FromColumns({Text.Split([Search1], ","), Text.Split([Search2], ",")})),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom2",{"Start1 - Copy", "End1 - Copy","Start2 - Copy","End2 - Copy","Custom"}),
#"Expanded Custom" = Table.ExpandTableColumn(#"Removed Other Columns", "Custom", {"Column1", "Column2"}, {"Search1", "Search2"}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Expanded Custom", "Search1", Splitter.SplitTextByEachDelimiter({"+++"}, QuoteStyle.None, true), {"Search1", "Filter1"}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Split Column by Delimiter", "Search2", Splitter.SplitTextByEachDelimiter({"+++"}, QuoteStyle.None, true), {"Search2", "Filter2"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"Search1", type text}, {"Filter1", type text}, {"Search2", type text}, {"Filter2", type text}}),
#"Filled Down1" = Table.FillDown(#"Changed Type1",{"Search1"}),
#"Sorted Rows" = Table.Sort(#"Filled Down1",{{"Filter1", Order.Ascending}}),
#"Replaced Value" = Table.ReplaceValue(#"Sorted Rows",null,"",Replacer.ReplaceValue,{"Filter1", "Search2", "Filter2"})
in
#"Replaced Value"
Text:
let
Source = Excel.CurrentWorkbook(){[Name="TextToSearch"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Text", type text}}),
Search1 = Table.AddColumn(#"Changed Type", "Search1", (x) => Table.SelectRows(Parameters, each Text.Contains(x[Text],[Search1], Comparer.OrdinalIgnoreCase))),
#"Expanded Search1" = Table.ExpandTableColumn(Search1, "Search1", {"Search1", "Filter1"}, {"Search1", "Filter1"}),
#"Filled Down" = Table.FillDown(#"Expanded Search1",{"Search1", "Filter1"}),
#"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Filter1] = "X")),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Text", "Search1"}),
Search2 = Table.AddColumn(#"Removed Other Columns", "Search2", (x) => Table.SelectRows(Parameters, each Text.Contains(x[Search1],[Search1], Comparer.OrdinalIgnoreCase) and Text.Contains(x[Text],[Search2], Comparer.OrdinalIgnoreCase))),
#"Removed Other Columns1" = Table.SelectColumns(Search2,{"Text", "Search2"}),
#"Expanded Search2" = Table.ExpandTableColumn(#"Removed Other Columns1", "Search2", {"Start1 - Copy", "End1 - Copy", "Start2 - Copy", "End2 - Copy", "Search2", "Filter2"}, {"Start1 - Copy", "End1 - Copy", "Start2 - Copy", "End2 - Copy", "Search2.1", "Filter2"}),
#"Filled Down1" = Table.FillDown(#"Expanded Search2",{"Search2.1", "Filter2"}),
#"Filtered Rows1" = Table.SelectRows(#"Filled Down1", each ([Filter2] = "X")),
#"Removed Other Columns2" = Table.SelectColumns(#"Filtered Rows1",{"Start1 - Copy", "End1 - Copy", "Start2 - Copy", "End2 - Copy", "Text"}),
#"Filled Down2" = Table.FillDown(#"Removed Other Columns2",{"Start1 - Copy", "End1 - Copy", "Start2 - Copy", "End2 - Copy", "Text"}),
#"Renamed Columns" = Table.RenameColumns(#"Filled Down2",{{"Start1 - Copy", "Start1"}, {"End1 - Copy", "End1"}, {"Start2 - Copy", "Start2"}, {"End2 - Copy", "End2"}})
in
#"Renamed Columns"
Real Example:
https://1drv.ms/x/s!AsrLaUgt0KCLvUgQQctfMtFe057l?e=AkbeP3
Here you go.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCnZ1DvH091MwVNKBs42A7JzMvFSwIJhhohSrE60E5MEE4EpMwTLIOmFsY7gpBnCWIYpqUyTVpnCT4aqNjJRiYwE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Start1 = _t, End1 = _t, Start2 = _t, End2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Start1", type text}, {"End1", type text}, {"Start2", type text}, {"End2", type text}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Start1", "End1"}),
#"Filled Down" = Table.FillDown(#"Replaced Value",{"Start1", "End1"}),
#"Added Custom" = Table.AddColumn(#"Filled Down", "Custom", each let
input = Input[Text],
s1 = List.PositionOf(input, [Start1]),
e1 = List.PositionOf(input, [End1]),
r1 = if s1=e1 then List.Range(input,s1) else List.Range(input,s1,e1-s1+1),
s2 = List.PositionOf(r1, [Start2]),
e2 = List.PositionOf(r1, [End2]),
r2 = List.Range(r1,s2,e2-s2+1)
in r2),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom")
in
#"Expanded Custom"
Here it is with partial matches.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8kjNUdJRCnZ1DvH091MwArJzMvNSFQxhDBOlWJ1oJSAPJgBXYgqWQdYJYxvDTTGAswxRVJsiqTaFmwxXbWSkFBsLAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Start1 = _t, End1 = _t, Start2 = _t, End2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Start1", type text}, {"End1", type text}, {"Start2", type text}, {"End2", type text}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Start1", "End1"}),
#"Filled Down" = Table.FillDown(#"Replaced Value",{"Start1", "End1"}),
#"Added Custom" = Table.AddColumn(#"Filled Down", "Custom", each let
input = Input[Text],
s1 = List.PositionOf(input, List.FindText(input,[Start1]){0}),
e1 = List.PositionOf(input, List.FindText(input,[End1]){0}),
r1 = if s1=e1 then List.Range(input,s1) else List.Range(input,s1,e1-s1+1),
s2 = List.PositionOf(r1, List.FindText(input,[Start2]){0}),
e2 = List.PositionOf(r1, List.FindText(input,[End2]){0}),
r2 = List.Range(r1,s2,e2-s2+1)
in r2),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom")
in
#"Expanded Custom"

A better way to extract Subheading numbers using power query

I am attempting to extract section heading numbers from a column in excel using power query.
I have already achieved this by matching with an existing list. However, I wonder if there is a better way to achieve this in fewer steps.
M Code:
let
Source = Excel.CurrentWorkbook(){[Name="Table7"]}[Content],
#"Trimmed Text1" = Table.TransformColumns(Source,{{"Column1", PowerTrim, type text}}),
SectionNumbers = Table.AddColumn(#"Trimmed Text1", "SectionNumber", (x) => Text.Combine(Table.SelectRows(SectionNumbers, each Text.Contains(x[Column1],[SectionNumbers], Comparer.OrdinalIgnoreCase))[SectionNumbers],", ")),
#"Split Column by Delimiter2" = Table.SplitColumn(SectionNumbers, "SectionNumber", Splitter.SplitTextByEachDelimiter({","}, QuoteStyle.None, true), {"SectionNumber.1", "SectionNumber.2"}),
#"Added Custom1" = Table.AddColumn(#"Split Column by Delimiter2", "Custom", each if [SectionNumber.2] = null then [SectionNumber.1] else [SectionNumber.2]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Column1", "Custom"})
in
#"Removed Other Columns"
The Section numbers being matched to can be generated using:
SectionNumbers
let
Source = {1..16},
#"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Custom", each {1..9}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Added Custom1" = Table.AddColumn(#"Expanded Custom", "Custom.1", each "."),
#"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Added Custom1", {{"Custom", type text}}, "en-GB"),{"Custom", "Custom.1"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
#"Merged Columns1" = Table.CombineColumns(Table.TransformColumnTypes(#"Merged Columns", {{"Column1", type text}}, "en-GB"),{"Column1", "Merged"},Combiner.CombineTextByDelimiter(".", QuoteStyle.None),"SectionNumbers")
in
#"Merged Columns1"
Essentially I would like a way of extracting any decimal at the start of a cell, either 15.0. or 15.0, or even 15.0.1 etc.
I have considered using regex i.e. \d+\.\d+[.] which should work however I have many rows and find that regex sometimes is computationally intensive in this case, so it takes much longer to load than the Above M Code.
Another power query method
Since you know your section numbers you could:
Generate a (buffered) list of all the section numbers
see if the first space-separated part of the string in column 1 exists in the Section Number list.
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
//create list of all serial numbers
SerialNumbers = List.Buffer(
let
Part1 = List.Transform({1..16}, each Text.From(_) & "."),
Part2 =List.Transform({1..10}, each Text.From(_) & "."),
sn = List.Accumulate(Part1,{}, (state, current)=> state &
List.Generate(
()=>[s=current & Part2{0}, idx=0],
each [idx] < List.Count(Part2),
each [s=current & Part2{[idx]+1}, idx=[idx]+1],
each [s]))
in
sn),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each
let
x = Text.Split([Column1]," "){0}
in
if List.Contains(SerialNumbers,x) then x else null, type text)
in
#"Added Custom"
How about
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if Text.PositionOfAny([Column1], {"0".."9"})>=0 then Text.BeforeDelimiter(Text.From([Column1])," ") else null)
in #"Added Custom"

Power Query: Expression.Error: There weren't enough elements in the enumeration to complete the operation. (LIST)

What I am trying to achieve is to obtain "matches/pairs" from two tables. One (source 1)is data table with Date/Time and Pressure value columns and the other (source 2) is like Date/Time and Info value Columns. Second table has so called "pairs" , start and stop in certain time. I want to get exact matches when is found in source 1 or approximate match when is not exact as in source 1 (seconds can be a problem).
Lets say you are matching/lookup two tables, give me everything that falls between for instance 15.01.2022 06:00:00 and 15.01.2022 09:15:29.
Where I have a problem is more likely exact match and seconds. It is skipping or cant find any pair if the seconds are not matching. So my question is how to make if not seconds then lookup for next availablee match, can be a minute too as long as they are in the given range (start stop instances).
That is a reason I am getting this Expression error. Or is there a way to skip that error and proceed with Query??
Link to download the data:
https://docs.google.com/spreadsheets/d/1Jv5j7htAaEFktN0ntwOZCV9jesF43tEP/edit?usp=sharing&ouid=101738555398870704584&rtpof=true&sd=true
On the code below is what I am trying to do:
let
//Be sure to change the table names in the Source= and Source2= lines to be the actual table names from your workbook
Source = Excel.CurrentWorkbook(){[Name="Parameters"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date/Time", type datetime}, {"P7 [mbar]", Int64.Type}}),
//get start/stop times table
Source2 = Excel.CurrentWorkbook(){[Name="Log_Original"]}[Content],
typeIt = Table.TransformColumnTypes(Source2, {"Date/Time", type datetime}),
#"Filtered Rows" = Table.SelectRows(typeIt, each ([#"Date/Time"] <> null)),
#"Added Index" = Table.AddIndexColumn(#"Filtered Rows", "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "NextLineStart", each if Text.Contains([Info],"start", Comparer.OrdinalIgnoreCase) = true
and Text.Contains(#"Added Index"[Info]{[Index]+1},"start",Comparer.OrdinalIgnoreCase) = true
then "delete"
else null),
#"Filtered Rows1" = Table.SelectRows(#"Added Custom", each ([NextLineStart] = null)),
#"Removed Columns1" = Table.RemoveColumns(#"Filtered Rows1",{"Index", "NextLineStart"}),
//create a list of all the relevant start/stop times
filterTimes = List.Combine(
List.Generate(
()=> [times = List.DateTimes(#"Removed Columns1"[#"Date/Time"]{0},
Duration.TotalSeconds(#"Removed Columns1"[#"Date/Time"]{1}-#"Removed Columns1"[#"Date/Time"]{0})+1,
#duration(0,0,0,1)), IDX = 0],
each [IDX] < Table.RowCount(#"Removed Columns1"),
each [times = List.DateTimes(#"Removed Columns1"[#"Date/Time"]{[IDX]+2},
Duration.TotalSeconds(#"Removed Columns1"[#"Date/Time"]{[IDX]+3}-#"Removed Columns1"[#"Date/Time"]{[IDX]+2})+1,
#duration(0,0,0,1)), IDX = [IDX]+2],
each [times]
)
),
//filter the table using the list
filterTimesCol = Table.FromList(filterTimes,Splitter.SplitByNothing()),
filteredTable = Table.Join(#"Changed Type","Date/Time",filterTimesCol,"Column1",JoinKind.Inner),
#"Removed Columns" = Table.RemoveColumns(filteredTable,{"Column1"}),
#"Added Custom1" = Table.AddColumn(#"Removed Columns", "Custom", each DateTime.ToText([#"Date/Time"],"dd-MMM-yy")),
#"Filtered Rows2" = Table.SelectRows(#"Added Custom1", each [#"Date/Time"] > #datetime(2019, 01, 01, 0, 0, 0)),
#"Sorted Rows" = Table.Sort(#"Filtered Rows2",{{"Date/Time", Order.Ascending}})
in
#"Sorted Rows"
I set up the below to return a sorted table with all results between the start and ending date/times. You can then select the first or middle or bottom row of each table if you want from this point. Its hard to tell from your question if you are looking for the value closest to the start value, closest to the end value or something inbetween. You can wrap my Table.Sort with a Table.FirstN or Table.LastN to pick up the first or last row.
I left most of your starting code alone
let Source = Table.Buffer(T1),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date/Time", type datetime}, {"P7 [mbar]", Int64.Type}}),
//get start/stop times table
Source2 = T2,
typeIt = Table.TransformColumnTypes(Source2, {"Date/Time", type datetime}),
#"Filtered Rows" = Table.SelectRows(typeIt, each ([#"Date/Time"] <> null)),
#"Added Index" = Table.AddIndexColumn(#"Filtered Rows", "Index", 0, 1),
// shift Info up one row for comparison
shiftedList = List.RemoveFirstN( #"Added Index"[Info],1),
custom1 = Table.ToColumns( #"Added Index") & {shiftedList},
custom2 = Table.FromColumns(custom1,Table.ColumnNames( #"Added Index") & {"NextInfo"}),
#"Filtered Rows2" = Table.SelectRows(custom2, each not (Text.Contains([Info],"start", Comparer.OrdinalIgnoreCase) and Text.Contains([NextInfo],"start", Comparer.OrdinalIgnoreCase))),
#"Added Custom3" = Table.AddColumn(#"Filtered Rows2", "Type", each if Text.Contains(Text.Lower([Info]),"start") then "start" else if Text.Contains(Text.Lower([Info]),"finished") then "finished" else null),
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom3",{"Info", "NextInfo"}),
#"Added Custom1" = Table.AddColumn(#"Removed Columns2", "Custom", each if [Type]="start" then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"Custom"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Index"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[Type]), "Type", "Date/Time"),
#"Added Custom2" = Table.AddColumn(#"Pivoted Column","Table",(i)=>Table.Sort(Table.SelectRows(T1, each [#"Date/Time"]>=i[start] and [#"Date/Time"]<=i[finished]),{{"Date/Time", Order.Ascending}}) , type table )
in #"Added Custom2"

Power Query - Group based on two filters

I'm working on a simplification of some of my reports by using Power Query.
I have a table with this structure:
In the first column we can see the Sales Order Number.
In the second column we can see the Message type. If it is a XR or PR Message.
And in the last column we can find the Key for the status:
A = Active
B = Active
C = Closed
With this logic I would like to get this result table:
This way I can see as a cross table the type and the state per sales order.
How is this possible by using Power Query?
Are you sure about your results for Sales Order 103?
Try the following M code, amending where necessary (Source step, for example):
let
Source = Excel.CurrentWorkbook(){[Name = "Table1"]}[Content],
XRCol = Table.AddColumn(
Source,
"XR",
each Text.BeforeDelimiter([Message], "-") = "XR",
type logical
),
PRCol = Table.AddColumn(
XRCol,
"PR",
each Text.BeforeDelimiter([Message], "-") = "PR",
type logical
),
XRActCol = Table.AddColumn(PRCol, "XR Active", each [XR] and [Key] <> "C", type logical),
PRActCol = Table.AddColumn(XRActCol, "PR Active", each [PR] and [Key] <> "C", type logical),
RemoveCols = Table.RemoveColumns(PRActCol, {"Message", "Key"}),
Group = Table.Group(
RemoveCols,
{"Sales Order"},
{
{"XR", each List.Max([XR]), type logical},
{"PR", each List.Max([PR]), type logical},
{"XR Active", each List.Max([XR Active]), type logical},
{"PR Active", each List.Max([PR Active]), type logical}
}
)
in
Group
If I see it correctly your output PR and XR is mixed up in Sales order 103.
Try if this works for you:
let
Source = Excel.Workbook(File.Contents("C:\Users\maitting\Documents\Mappe1.xlsx"), null, true),
Tabelle1_Sheet = Source{[Item="Tabelle1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Tabelle1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Sales Order", Int64.Type}, {"Message", type text}, {"Key", type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type", "Message", Splitter.SplitTextByDelimiter("-", QuoteStyle.Csv), {"Message"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Message", type text}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type1","A","Active",Replacer.ReplaceText,{"Key"}),
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value","B","Active",Replacer.ReplaceText,{"Key"}),
#"Replaced Value2" = Table.ReplaceValue(#"Replaced Value1","C","Closed",Replacer.ReplaceText,{"Key"}),
#"Merged Columns" = Table.CombineColumns(#"Replaced Value2",{"Message", "Key"},Combiner.CombineTextByDelimiter(" - ", QuoteStyle.None),"Merged"),
#"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Merged Columns", {"Sales Order"}, "Attribute", "Value"),
#"Pivoted Column" = Table.Pivot(#"Unpivoted Columns", List.Distinct(#"Unpivoted Columns"[Value]), "Value", "Attribute", List.Count),
#"Added Conditional Column" = Table.AddColumn(#"Pivoted Column", "XR",
each if [#"XR - Active"] = 1 then 1
else if [#"XR - Closed"] = 1 then 1
else 0),
#"Added Conditional Column1" = Table.AddColumn(#"Added Conditional Column", "PR",
each if [#"PR - Active"] = 1 then 1
else if [#"PR - Closed"] = 1 then 1
else 0),
#"Removed Columns" = Table.RemoveColumns(#"Added Conditional Column1",{"XR - Closed", "PR - Closed"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"Sales Order", "XR", "PR", "XR - Active", "PR - Active"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Reordered Columns",{{"XR", type logical}, {"PR", type logical}, {"XR - Active", type logical}, {"PR - Active", type logical}})
in
#"Changed Type2"
Output:
I've just added Sales order 104 with PR Closed to handle all possible cases.
My version
let Source = Excel.CurrentWorkbook(){[Name = "Table1"]}[Content],
RemoveEndingColumn = Table.TransformColumns(Source,{{"Message", each Text.BeforeDelimiter(_, "-"), type text}}),
#"Create MessageKey" = Table.AddColumn(RemoveEndingColumn, "MessageKey", each [Message]&" "&[Key]),
StackColumns = Table.RenameColumns(Table.SelectColumns(#"Create MessageKey",{"Sales Order", "Message"}),{{"Message", "Column"}})& Table.RenameColumns(Table.SelectColumns(#"Create MessageKey",{"Sales Order", "MessageKey"}),{{"MessageKey", "Column"}}),
#"Added Custom1" = Table.AddColumn(StackColumns, "TrueFalse", each true, type logical),
#"Pivoted Column" = Table.Pivot(#"Added Custom1", List.Distinct(#"Added Custom1"[Column]), "Column", "TrueFalse")
in #"Pivoted Column"

Data Not Retrieve in power query excel 2013?

I am creating a Calendar MS Excel 2013 through Power Query the other table data not shown in the workbook because I create a connection model and the data exist in data model so I am creating a blank query to and paste this power query to create a calendar please guide me where I am wrong
Thanks
let Source =
{Number.From(List.Min(Excel[Date]))..Number.From(List.Max(Excel[Date]))},
"Converted to Table" = Table.FromList(Source,
Splitter.SplitByNothing(), null, null, ExtraValues.Error), "Renamed
Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1",
"Date"}}), "Changed Type" = Table.TransformColumnTypes(#"Renamed
Columns",{{"Date", type date}}), "Added Custom" =
Table.AddColumn(#"Changed Type", "Year", each Date.Year([Date])),
"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Year",
Int64.Type}}), "Added Custom1" = Table.AddColumn(#"Changed Type1",
"Month", each Date.Month([Date])), "Changed Type2" =
Table.TransformColumnTypes(#"Added Custom1",{{"Month", Int64.Type}}),
"Added Custom2" = Table.AddColumn(#"Changed Type2", "MonthName", each
Date.MonthName([Date])), "Changed Type3" =
Table.TransformColumnTypes(#"Added Custom2",{{"MonthName", type
text}}), "Sorted Rows" = Table.Sort(#"Changed Type3",{{"Date",
Order.Ascending}}) in "Sorted Rows"

Resources