source link
I am trying to come up with a solution to the following problem.
Problem:
In my dataset I have certain quantity of item in demand (need), and purchase orders that re-supply that item(Supply). I need to determine for each demand , what is the first date where we will have enough supply to fill the demand.
For example, if we look at our 1st demand, which require 5 units, according to the cumulative Sum column, 18/12/23 will be the first date when we would have enough qty supplied to satisfy the first demand. The problem appears when we have more the one demand for an item.
If we stay with same item What I would like to do is to update the cumulative Sum when we meet the enough quantity ( as cumulative Sum = cumulative Sum- qty(demand) or 6(cumulative supply)-5(demand) = 1 ) so the cumulative Sum for the next demand will be 100 +1 = 101 and not 100 + 6 = 106. Thereby we can simply rely on the cumulative Sum (updated) to retrieve the first date where we will have enough supply to fill the demand.
I'm not sure if something like this is possibly in Power Query but any help is greatly appreciated.
Hopefully that all makes sense. Thx.
Revised
In powerquery try this as code for Demand
let Source = Excel.CurrentWorkbook(){[Name="DemandDataRange"]}[Content],
#"SupplyGrouped Rows" = Table.Group(Supply, {"item"}, {{"data", each
let a = Table.AddIndexColumn( _ , "Index", 0, 1),
b=Table.AddColumn(a, "CumTotal", each List.Sum(List.FirstN(a[Qty],[Index]+1)))
in b, type table }}),
#"SupplyExpanded data" = Table.ExpandTableColumn(#"SupplyGrouped Rows", "data", { "Supply date", "CumTotal"}, {"Supply date", "CumTotal"}),
#"Grouped Rows" = Table.Group(Source, {"item"}, {{"data", each
let a= Table.AddIndexColumn(_, "Index", 0, 1),
b=Table.AddColumn(a, "CumTotal", each List.Sum(List.FirstN(a[Qty],[Index]+1)))
in b, type table }}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"Qty", "Date", "Index", "CumTotal"}, {"Qty", "Date", "Index", "CumTotal"}),
x=Table.AddColumn(#"Expanded data","MaxDate",(i)=>try Table.SelectRows( #"SupplyExpanded data", each [item]=i[item] and [CumTotal]>=i[CumTotal] )[Supply date]{0} otherwise null, type date ),
#"Removed Columns" = Table.RemoveColumns(x,{"Index", "CumTotal"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Columns",{{"Date", type date}})
in #"Changed Type"
Given my understanding of what you want for results, the following Power Query M code should return that.
If you just want to compare the total supply vs total demand, then only check the final entries instead of the first non-negative.
Read the code comments, statement names and explore the Applied Steps to understand the algorithm.
let
//Read in the data tables
//could have them in separate querries
Source = Excel.CurrentWorkbook(){[Name="Demand"]}[Content],
Demand = Table.TransformColumnTypes(Source,{{"item", type text}, {"Qty", Int64.Type}, {"Date", type date}}),
//make demand values negative
#"Transform Demand" = Table.TransformColumns(Demand,{"Qty", each _ * -1}),
Source2 = Excel.CurrentWorkbook(){[Name="Supply"]}[Content],
Supply = Table.TransformColumnTypes(Source2,{{"item", type text},{"Qty", Int64.Type},{"Supply date", type date}}),
#"Rename Supply Date Column" = Table.RenameColumns(Supply,{"Supply date","Date"}),
//Merge the tables and sort by Item and Date
Merge = Table.Combine({#"Rename Supply Date Column", #"Transform Demand"}),
#"Sorted Rows" = Table.Sort(Merge,{{"item", Order.Ascending}, {"Date", Order.Ascending}}),
//Group by Item
//Grouped running total to find first positive value
#"Grouped Rows" = Table.Group(#"Sorted Rows", {"item"}, {
{"First Date", (t)=> let
#"Running Total" = List.RemoveFirstN(List.Generate(
()=>[rt=t[Qty]{0}, idx=0],
each [idx]<Table.RowCount(t),
each [rt=[rt]+t[Qty]{[idx]+1}, idx=[idx]+1],
each [rt]),1),
#"First non-negative" = List.PositionOfAny(#"Running Total", List.Select(#"Running Total", each _ >=0), Occurrence.First)
in t[Date]{#"First non-negative"+1}, type date}})
in
#"Grouped Rows"
Supply
Demand
Results
I did this in Excel formula rather than using powerquery - there will be a powerquery equivalent but I'm not very fluent in DAX yet.
You need a helper column - could do without it but everything's much more readable if you have it.
In sheet Supply (2), cell E2, enter the formula:
=SUMIFS(Supply!B:B; Supply!C:C;"<=" & C2;Supply!A:A;A2)-SUMIFS(Dem!B:B;Dem!C:C;"<=" & C2;Dem!A:A;A2)
and copy downwards. This can be described as Total supply up to that date minus total demand up to that date. In some cases this will be negative (where there's more demand than supply).
Now you need to find the date of the first non-negative value for that.
First create a unique list of the items - I put it on the same sheet in the range G2:G6. Then in H2, the formula:
=MINIFS(C:C;A:A;G2;E:E;">=" & 0)
and copy downwards.
I have data within a table that occasionally has been inputted with text to say something like not available or No Data etc. I wish to replace each instance a cell contains no that this is then replaced with null across n number of columns. I don't know every type of word that has been entered but it looks as though each cell to be converted to null contains no as characters so I will go with this.
i.e.
Is there any way to combine `if text.contains([n columns],"no") then null else [n columns]
In powerquery, this removes the content of any cell containing (No,NO,no,nO) and converts to a null
Click select the first column, right click, Unpivot other columns
click select Value column and transform ... data type .. text
right click Value column and transform ... lower case
we really don't want that so change this in the formula bar
= Table.TransformColumns(#"Changed Type1",{{"Value", Text.Lower, type text}})
to resemble this instead (which also ignore the Case of the No)
= Table.TransformColumns(#"Changed Type1",{{"Value", each if Text.Contains(_,"no", Comparer.OrdinalIgnoreCase) then null else _, type text}})
click select attribute column
Transform ... pivot column
values column:Value, Advanced ... don’t aggregate
sample full code:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"Column"}, "Attribute", "Value"),
#"Changed Type1" = Table.TransformColumnTypes(#"Unpivoted Other Columns",{{"Value", type text}}),
#"CheckForNo" = Table.TransformColumns(#"Changed Type1",{{"Value", each if Text.Contains(_,"no", Comparer.OrdinalIgnoreCase) then null else _, type text}}),
#"Pivoted Column" = Table.Pivot(#"CheckForNo", List.Distinct(#"Lowercased Text"[Attribute]), "Attribute", "Value")
in #"Pivoted Column"
I am attempting to extract cells through a combination of index(match) and right(len)-find() functions from an array of data with text. In my formula, I am searching for instances of "* DS#1 ", excel returns those but also returns instances with " DS#11 *". How do I get excel to return only DS#1?
I have attempted to use an if statement with no success, if(formula="* 11 *","",formula).
Below is a link to an example of the data. The first cell highlighted in yellow should not be returning that text, it should be "". The second cell highlighted in yellow is appropriate to return that data.
example data
=RIGHT(INDEX($V:$AC,MATCH("DS#1",$AC:$AC,0),1),LEN(INDEX($V:$AC,MATCH(FW$1,$AC:$AC,0),1))-FIND($AG2,INDEX($V:$AC,MATCH(FW$1,$AC:$AC,0),1))+1)
Here some example on how to find a value and check for the following char.
Formula in D2:
=INDEX(A2:A6,MATCH(1,INDEX((ISNUMBER(SEARCH("DS#1",B2:B6)))*(NOT(ISNUMBER(MID(B2:B6,SEARCH(C2,B2:B6)+LEN(C2),1)*1))),0),0))
Here is a formula you can adapt to your ranges which will return a list from the range rngDS that contain findDS. I used named ranges, but you can adapt them to your own ranges.
Not sure if this is what you want since you chose to not post examples of your data or desired results.
The routine finds the findDS string and then checks to be sure that the following character is non-numeric.
C1: =IFERROR(INDEX(rngDS,AGGREGATE(15,6,1/(NOT(ISNUMBER(-MID(rngDS,SEARCH(findDS,rngDS)+LEN(findDS),1))+ISERROR(MID(rngDS,SEARCH(findDS,rngDS)+LEN(findDS),1))))*ROW(rngDS),ROWS($1:1))),"")
and fill down
It would be very difficult to come up a formula based solution especially when you need to first differentiate DS1, DS#1. DS#11, DS#11X etc. then look for the text string after each DS code, not to mention these confusing codes may (or may not) be positioned in random orders in the text string.
A better approach would be using Power Query which is available in Excel 2010 and later versions. My solution is using Excel 2016.
Presume you have the following two tables:
You can use From Table function in the Data tab to add both tables to the Power Query Editor.
Once added, make a duplicate copy of Table 1. I have renamed the duplicate as Table1 (2) - Number Ref. Then you should have three un-edited queries:
If your source data is a larger table containing some other information, you can google how to add a worksheet to the editor and how to remove unnecessary columns and remove duplicated values.
Firstly, let's start working with Table1.
Here are the steps:
Use Replace Values function to remove all # from the text string, and then replace all DS with DS# in the text string, so all DS codes are in the format of DS#XXX. Eg. DS8 will be changed to DS#8. This step may not be necessary if DS8 is a valid code as well as DS#8;
Use Split Column function to split the text strings by the word DS, and put each sub text string into a new row, then you should have the following:
Use Split Column function again to split the text strings by 1 Character from the left and you should have the following:
Filter the first column to show hash tag # only and then remove the first column, then you should have the following:
use Replace Values function repeatedly to remove the following characters/symbols from the text strings: (, ), HT, JH, SK, //, and replace dash - with space . I presume these are irrelevant in the comment but you can leave them if needed. Then you should have:
use Split Column function again to split the text string by the first space on the left, then you should have:
Then you can Trim and Clean the second column to further tidy up the comments, rename the columns as DS#, Comments, and Number Ref consecutively, and change the format of the third column to Text. Then you should have:
The last step is to add a custom column called Match ID to combine the value from first and third column into one text string as shown below:
Secondly, let's work on Table1 (2) - Number Ref
Here are the steps:
Remove the first column so leave the Number Ref column as the single column;
Transpose the column, and Promote the first row as header. Then you should have:
The purpose of this query is to transform all Number Reference into column headers, then append this query with the next query (Table2) to achieve the desired result which I will explain in the next section.
Lastly, let's work on the third query Table2.
Here are the steps:
Append this table with the Number Ref table from previous step;
highlight the whole table and use Replace Values function to replace all null with number 1. Then highlight the first column and use Unpivot Other Columns function to transform the table as below:
then Remove the last column (Value), and add a new custom column called Match ID to combine the DS code with the Number Reference. Then you should have:
Merge the table with Table1 using Match ID as shown below:
Expand the newly merged column Table1 to show Comments;
Use Split Column function to split the first column by Non-digit to digit, change the format of the digit column to whole number, and then sort Attribute column and the digit column ascending consecutively, then you should have:
Use Split Column function again to split the Match ID column by dash sign -, and remove the first three columns, rename the remaining three columns as DS#, Number Ref and Comments consecutively, then you should have:
Close & Load this table to a new worksheet as desired, which may look like this:
In conclusion, It is entirely up to you how you would like to structure the table in Power Query. You can pre-filter the Number Reference in the editor and load only relevant results to a worksheet, you can load the full table to a worksheet and use VLOOKUP or INDEX to retrieve the data as desired, or you can load the third query to data model from where you can create pivot tables to play around.
Here are the codes behind the scene for reference only. All steps are using built-in functions of the editor without any advanced manual coding.
Table1
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Comments", type text}}),
#"Replaced Value8" = Table.ReplaceValue(#"Changed Type","#","",Replacer.ReplaceText,{"Comments"}),
#"Replaced Value9" = Table.ReplaceValue(#"Replaced Value8","DS","DS#",Replacer.ReplaceText,{"Comments"}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(#"Replaced Value9", {{"Comments", Splitter.SplitTextByDelimiter("DS", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Comments"),
#"Split Column by Position" = Table.SplitColumn(#"Split Column by Delimiter", "Comments", Splitter.SplitTextByPositions({0, 1}, false), {"Comments.1", "Comments.2"}),
#"Filtered Rows" = Table.SelectRows(#"Split Column by Position", each ([Comments.1] = "#")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Comments.1"}),
#"Replaced Value1" = Table.ReplaceValue(#"Removed Columns",")","",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value2" = Table.ReplaceValue(#"Replaced Value1","-"," ",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value3" = Table.ReplaceValue(#"Replaced Value2","(","",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value4" = Table.ReplaceValue(#"Replaced Value3","HT","",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value5" = Table.ReplaceValue(#"Replaced Value4","JH","",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value6" = Table.ReplaceValue(#"Replaced Value5","SK","",Replacer.ReplaceText,{"Comments.2"}),
#"Replaced Value7" = Table.ReplaceValue(#"Replaced Value6","//","",Replacer.ReplaceText,{"Comments.2"}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Replaced Value7", "Comments.2", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, false), {"Comments.2.1", "Comments.2.2"}),
#"Trimmed Text" = Table.TransformColumns(#"Split Column by Delimiter1",{{"Comments.2.2", Text.Trim, type text}}),
#"Cleaned Text" = Table.TransformColumns(#"Trimmed Text",{{"Comments.2.2", Text.Clean, type text}}),
#"Renamed Columns" = Table.RenameColumns(#"Cleaned Text",{{"Comments.2.1", "DS#"}, {"Comments.2.2", "Comments"}}),
#"Changed Type1" = Table.TransformColumnTypes(#"Renamed Columns",{{"Number Ref", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type1", "Match ID", each "DS#"&[#"DS#"]&"-"&[Number Ref])
in
#"Added Custom"
Table1 (2) - Number Ref
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Comments", type text}, {"Number Ref", Int64.Type}}),
#"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"Number Ref"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Removed Other Columns",{{"Number Ref", type text}}),
#"Transposed Table" = Table.Transpose(#"Changed Type1"),
#"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
#"Changed Type2" = Table.TransformColumnTypes(#"Promoted Headers",{{"388", type any}, {"1", type any}})
in
#"Changed Type2"
Table2
let
Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"List", type text}}),
#"Appended Query" = Table.Combine({#"Changed Type", #"Table1 (2) - Number Ref"}),
#"Replaced Value" = Table.ReplaceValue(#"Appended Query",null,"1",Replacer.ReplaceValue,{"List", "388", "1"}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Replaced Value", {"List"}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Value"}),
#"Added Custom" = Table.AddColumn(#"Removed Columns", "Match ID", each [List]&"-"&[Attribute]),
#"Merged Queries" = Table.NestedJoin(#"Added Custom", {"Match ID"}, Table1, {"Match ID"}, "Table1", JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Comments"}, {"Comments"}),
#"Split Column by Character Transition" = Table.SplitColumn(#"Expanded Table1", "List", Splitter.SplitTextByCharacterTransition((c) => not List.Contains({"0".."9"}, c), {"0".."9"}), {"List.1", "List.2"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Character Transition",{{"List.2", Int64.Type}}),
#"Sorted Rows" = Table.Sort(#"Changed Type1",{{"Attribute", Order.Ascending}, {"List.2", Order.Ascending}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Sorted Rows", "Match ID", Splitter.SplitTextByDelimiter("-", QuoteStyle.Csv), {"Match ID.1", "Match ID.2"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"List.1", type text}, {"Match ID.1", type text}, {"Match ID.2", type text}}),
#"Removed Other Columns" = Table.SelectColumns(#"Changed Type2",{"Match ID.1", "Match ID.2", "Comments"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Other Columns",{{"Match ID.1", "DS#"}, {"Match ID.2", "Number Ref"}})
in
#"Renamed Columns"
Cheers :)
Based on the creative answers from this post, I was able to take some of these ideas and form a solution. There are two parts to the solution I found. The first is to replace the strings that I am trying to filter out with a random text that doesn't appear in any instance in my data. Since I had a range of data I needed to replace (DS11 through DS19), I used a VBA function to avoid a large nested function.
Once I had the strings I am trying to filter out replaced, I added an If(Isnumber(search()) function to display "" when the replaced text is returned.
Function REPLACETEXTS(strInput As String, rngFind As Range, rngReplace As Range) As String
Dim strTemp As String
Dim strFind As String
Dim strReplace As String
Dim cellFind As Range
Dim lngColFind As Long
Dim lngRowFind As Long
Dim lngRowReplace As Long
Dim lngColReplace As Long
lngColFind = rngFind.Columns.Count
lngRowFind = rngFind.Rows.Count
lngColReplace = rngFind.Columns.Count
lngRowReplace = rngFind.Rows.Count
strTemp = strInput
If Not ((lngColFind = lngColReplace) And (lngRowFind = lngRowReplace)) Then
REPLACETEXTS = CVErr(xlErrNA)
Exit Function
End If
For Each cellFind In rngFind
strFind = cellFind.Value
strReplace = rngReplace(cellFind.Row - rngFind.Row + 1, cellFind.Column - rngFind.Column + 1).Value
strTemp = Replace(strTemp, strFind, strReplace)
Next cellFind
REPLACETEXTS = strTemp
End Function
I want to filter a Power Query column by a range of values. I see that I can manually select or deselect values from the column, but I need to do this automatically absed on a range of values already in the sheet.
Let's say the table I'm querying contains 100 unique names. I only want to import rows that match a list of 20 names I have in my sheet.
How can I do this with Power Query?
Edit: I can make an excel function to concatenate a list of names into the format needed for the query like this: (= Table.SelectRows(#"Changed Type", each ([Ticket Assignee] ="Name 1" or [Ticket Assignee] ="Name 2"))). But still need a way to reference this from the query.
Edit2:
My query:
This errors out with:
Expression.Error: We expected a FieldsSelector value.
let
names_Source = Excel.CurrentWorkbook(){[Name="namesTable"]}[Content],
names_Values = names_Source{1},
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type datetime}, {"Ticket Assignee", type text}, {"# Solved", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Ticket Assignee] <> null)),
#"Filtered Rows1" = Table.SelectRows(#"Changed Type", each Record.HasFields(names_Values, [Ticket Assignee]))
in
#"Filtered Rows1"
Edit3: Modified Query:
Expression.Error: We cannot convert the value null to type Record.
Details:
Value=
Type=Type
let
names_Source = Excel.CurrentWorkbook(){[Name="namesTable"]}[Content],
names_Values = Record.FromList(names_Source[Zendesk Name], names_Source[Zendesk Name]),
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type datetime}, {"Ticket Assignee", type text}, {"# Solved", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([Ticket Assignee] <> null)),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each Record.HasFields(names_Values, [Ticket Assignee]))
in
#"Filtered Rows1"
Edit4: My table has blank values that were erroring out as null. I added a line to filter out the null values from my names source
If the list of items you want to check against is in an Excel worksheet, then you can import the table using From Table and use List.Contains to see if your values are in that table. If the worksheet table is in WorksheetQuery and the names are in the column Names, then your step would look like:
= Table.SelectRows(PreviousStep, each List.Contains(WorksheetQuery[Names], [Ticket Assignee]))
If that ends up being too slow, you can try to convert the column into a record and then using Record.HasFields instead.
Why don't to use join operations? It seems like it works at least not slower, but more clear, in my mind:
let
names_Source = Excel.CurrentWorkbook(){[Name="namesTable"]}[Content],
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Joind = Table.NestedJoin(Source,{"Ticket Assignee"},names_Source,{"Zendesk Name"},"NewColumn",JoinKind.RightOuter),
Filtered = Table.RemoveColumns(Joind,{"NewColumn"})
in
Filtered