Excel: arrange prescription journey based on dates and IDs - excel

I am trying to arrange patient's journey based on first regimen then second regimen and so on.
However, after sorting data based on date:
I tried using IF formula as follow but it does not work correctly ( it worked for ID with three rows without having A5 in the formula):
=IF(AND(A2=A3,A3=A4,A4=A5,(C5-C4<=30),(C4-C3<=30),D3>(C4+30),D2>(C3+30)),B2&", "&B3&", "&B4&", "&B5,
IF(AND(A2=A3,A3=A4,A4=A5,(C4-C3<=30),D2>(C3+30)),B2&", "&B3&", "&B4,
IF(AND(A2=A3,A3=A4,A4=A5,(C4-C3<=30),D2<(C3+30)),B3&", "&B4,
IF(AND(A2=A3,A3=A4,A4=A5,(C4-C3>30),D2<(C3+30)), B3, "N")
I need to have similar results as follow:
Is there any way to have a formula helping to do so, or any other way to have similar results.

I had been working on Power Query code so I will present that first.
You can adapt the same algorithm to use in VBA, if you prefer. I would probably be using nested dictionaries and/or a class module to accomplish it effectively
To use Power Query
Select some cell in your Data Table
Data => Get&Transform => from Table/Range or from within sheet
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
M Code
let
//Change next line to reflect your data source
Source = Excel.CurrentWorkbook(){[Name="Drugs"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"CATEGORY", type text},
{"F.DATE", type date}, {"L.DATE", type date}}),
//Group by ID, then run fnJourney custom function on each subtable to return results
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {
{"Count", each fnJourney(_)}}),
//Expand results
#"Expanded Count" = Table.ExpandListColumn(#"Grouped Rows", "Count"),
#"Expanded Count1" = Table.ExpandRecordColumn(#"Expanded Count", "Count", {"Year", "Reg"}),
//Pivot on Year column with no aggregation
#"Year Headers" = List.Sort(List.Distinct(Table.TransformColumnTypes(#"Expanded Count1", {{"Year", type text}}, "en-US")[Year])),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Expanded Count1", {{"Year", type text}}, "en-US"),
#"Year Headers", "Year", "Reg"),
#"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column", List.Transform(#"Year Headers", each {_, type text})),
//Join with original table
join = Table.NestedJoin(#"Changed Type", "ID", #"Changed Type1","ID", "Joined", JoinKind.LeftOuter),
//add shifted ID column to decide if the joined table should be retained or deleted
#"Shifted ID" = Table.FromColumns(Table.ToColumns(join) & {{null} & List.RemoveLastN(join[ID])},
type table[ID=Int64.Type, CATEGORY=text, F.DATE=date, L.DATE=date, joined=table, shiftedID=Int64.Type]),
#"Added Custom" = Table.AddColumn(#"Shifted ID", "Custom", each if [shiftedID] <> [ID] then [joined] else null, type nullable table),
//Remove shifted and Joined columns
//Then expand the tables in the Custom Column
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"joined", "shiftedID"}),
#"Expanded Custom" = Table.ExpandTableColumn(#"Removed Columns", "Custom", #"Year Headers"),
#"Changed Type2" = Table.TransformColumnTypes(#"Expanded Custom", List.Transform(#"Year Headers", each {_, type text}))
in
#"Changed Type2"
Custom Function CodeCreate new Blank query and paste code below
//Rename this query fnJourney
(tbl as table) =>
let
//Source = Drugs,
Source = tbl,
Years = {List.Min(List.Transform(Source[F.DATE], each Date.Year(_)))..
List.Max(List.Transform(Source[L.DATE], each Date.Year(_)))},
inJourney = List.Generate(
()=>[yr=Years{0},
reg=Text.Combine(Table.SelectRows(Source,
each Years{0} >= Date.Year([F.DATE])
and Years{0} <= Date.Year([L.DATE]))[CATEGORY],", "),
idx=0],
each [idx] < List.Count(Years),
each [yr=Years{[idx]+1},
reg=Text.Combine(Table.SelectRows(Source,
(r)=>Years{[idx]+1} >= Date.Year(r[F.DATE])
and Years{[idx]+1} <= Date.Year(r[L.DATE]))[CATEGORY],", "),
idx=[idx]+1],
each Record.FromList({[yr], [reg]},{"Year","Reg"})
)
in
inJourney
Before
After

Related

Excel Power Query; Expand table into columns

I have two tables that I'd like to merge, but I can't quite get the merge to work the way I'd like it to. The first is a list of days and some data, the second is a ranking of items sold that day along with the number sold, etc. I'd like to get the top 'x' items added as columns to my daily data. I got the merge to work, and so now I have a column that says 'table.' However if I expand the table, each item turns into its own row. Instead I'd like table item 2 to repeat all of its columns with a postfix.
date
total_sales
items
1/1/21
$50000
table
1/2/21
$40000
table
date
total_sales
items_1
units_1
items_2
units_2
1/1/21
$50000
pens
15
pencils
10
1/2/21
$40000
erasers
35
pens
5
etc.
I can do this with visual basic but I don't think that's the best way to go about it. Thanks for your help! Also, is there a specific term for this operation that I could have searched for?
Let's suppose your second table looks like this:
First, let's group by date where for each group we sort and index the subtable. (This is the hardest step.)
Once we have the index for each group, expand that column back so we're back to the start except with the new index column.
From here, you can filter the index to get only the top N and then unpivot the [items] and [units] columns.
Merge the [index] and [Attribute]
and then pivot on [Merged]
Ta da! Now your data is shaped so that you can easily merge it with the first table.
Here's the full code for this example. (You can paste this into the Advanced Editor and look through Applied Steps.)
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMtQ31DcyMDJU0lEqSM0rBlKGpkqxOugSyZk5YDkDqJwRTC61KLE4tQgkZ2yKLgc1EMM8hB5sWqBWGSnFxgIA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [date = _t, items = _t, units = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"date", type date}, {"items", type text}, {"units", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"date"}, {{"indexed", each Table.AddIndexColumn(Table.Buffer(Table.Sort(_, {{"units", Order.Descending}})), "Index", 1, 1, Int64.Type), type table}}),
#"Expanded indexed" = Table.ExpandTableColumn(#"Grouped Rows", "indexed", {"items", "units", "Index"}, {"items", "units", "Index"}),
#"Filtered Rows" = Table.SelectRows(#"Expanded indexed", each [Index] < 3),
#"Unpivoted Columns1" = Table.UnpivotOtherColumns(#"Filtered Rows", {"date", "Index"}, "Attribute", "Value"),
#"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Unpivoted Columns1", {{"Index", type text}}, "en-US"),{"Attribute", "Index"},Combiner.CombineTextByDelimiter("_", QuoteStyle.None),"Merged"),
#"Pivoted Column1" = Table.Pivot(#"Merged Columns", List.Distinct(#"Merged Columns"[Merged]), "Merged", "Value"),
#"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column1",{{"items_1", type text}, {"units_1", Int64.Type}, {"items_2", type text}, {"units_2", Int64.Type}})
in
#"Changed Type1"
The only place I'm doing special M language magic (not just clicking stuff in the GUI) is this step which I created by applying a couple of steps to one of the sub-tables and pasting the GUI-generated code back into the group by step (plus Table.Buffer wrapper to make sure the sorting "sticks").
#"Grouped Rows" =
Table.Group(
#"Changed Type",
{"date"},
{
{"indexed", each
Table.AddIndexColumn(
Table.Buffer(
Table.Sort(_, {{"units", Order.Descending}})
),
"Index", 1, 1, Int64.Type
), type table
}
}
),

Transpose repeating data from rows into columns Excel

I have data set of basic housing data in the following format:
Existing data format:
That format is the same and reapeats for hundrets of properties. I would like to transform that that into a table format like the following example:
Property Type
Price
Location
Region
Additional info
Area
House
252000
London
Kensington
4500 square meters
...
...
...
...
...
etc
In other words I want to make the text before ":" symbol column name with the text after it the data that goes into into the corresponding cell and to repeat that for hundrets of sites. Usually there is missing(no data) in Additional info but sometimes there is.
I am not shure which is the best program to do this. So far in my mind comes Excel but if there is an easier way I will be glad to use it.
As per my below screenshot Excel 365 I have used following formulas.
C2=FILTERXML("<t><s>"&SUBSTITUTE(INDEX($A:$A,SEQUENCE(COUNTA($A:$A)/4,1,1,4)),": ","</s><s>")&"</s></t>","//s[last()]")
D2=FILTERXML("<t><s>"&SUBSTITUTE(INDEX($A:$A,SEQUENCE(COUNTA($A:$A)/4,1,2,4)),": ","</s><s>")&"</s></t>","//s[last()]")
E2=FILTERXML("<t><s>"&SUBSTITUTE(SUBSTITUTE(INDEX($A:$A,SEQUENCE(COUNTA($A:$A)/4,1,3,4)),",","</s><s>"),":","</s><s>")&"</s></t>","//s[2]")
F2=FILTERXML("<t><s>"&SUBSTITUTE(SUBSTITUTE(INDEX($A:$A,SEQUENCE(COUNTA($A:$A)/4,1,3,4)),",","</s><s>"),":","</s><s>")&"</s></t>","//s[last()-1]")
H2=FILTERXML("<t><s>"&SUBSTITUTE(INDEX($A:$A,SEQUENCE(COUNTA($A:$A)/4,1,4,4)),": ","</s><s>")&"</s></t>","//s[last()]")
If you are not in Excel 365 then can try-
=FILTERXML("<t><s>"&SUBSTITUTE(INDEX($A:$A,ROW($A1)+(ROW($A1)-1)*3),": ","</s><s>")&"</s></t>","//s[last()]")
Basically =ROW(A1)+(ROW(A1)-1)*3 will generate a sequence of row numbers and INDEX($A:$A,ROW($A1)+(ROW($A1)-1)*3) will return value from Column A as per that sequence. Then FILTERXML() will return expected value specified in xPath parameter.
To know, how FILTERXML() works yo can read this article from JvdV. This is a fantastic article for FILTERXML() lover.
You can obtain your desired output using Power Query, available in Windows Excel 2010+ and Office 365 Excel
Select some cell in your original table
Data => Get&Transform => From Table/Range
When the PQ UI opens, navigate to Home => Advanced Editor
Make note of the Table Name in Line 2 of the code.
Replace the existing code with the M-Code below
Change the table name in line 2 of the pasted code to your "real" table name
Examine any comments, and also the Applied Steps window, to better understand the algorithm and steps
Note: The fnPivotAll function is a custom function that enables a method of creating a non-aggregated Pivot Table where there are multiple values per Pivot Column. From the UI, you add this as a New Query from Blank, and just paste that M-code in place of what's there
M-Code (for main query)
let
//Read in data
//Change table name in next line to your actural table name
Source = Excel.CurrentWorkbook(){[Name="Table1_2"]}[Content],
//Split by comma into new rows
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(Source, {{"Column1",
Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv),
let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Column1"),
//Remove the blank rows
#"Filtered Rows" = Table.SelectRows(#"Split Column by Delimiter", each ([Column1] <> "" and [Column1] <> " ")),
//Split by the rightmost colon only into new columns
#"Split Column by Delimiter1" = Table.SplitColumn(#"Filtered Rows", "Column1",
Splitter.SplitTextByEachDelimiter({":"}, QuoteStyle.Csv, true), {"Column1.1", "Column1.2"}),
//Split by the remaining colon into new rows
// So as to have empty rows under "Additional data"
//Then Trim the columns to remove leading/trailing spaces
#"Split Column by Delimiter2" = Table.ExpandListColumn(Table.TransformColumns(#"Split Column by Delimiter1", {{"Column1.1", Splitter.SplitTextByDelimiter(":", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Column1.1"),
#"Changed Type" = Table.TransformColumnTypes(#"Split Column by Delimiter2",{{"Column1.1", type text}, {"Column1.2", type text}}),
#"Trimmed Text" = Table.TransformColumns(#"Changed Type",{{"Column1.1", Text.Trim, type text}, {"Column1.2", Text.Trim, type text}}),
//Create new column processing "Additional Data" to show a blank
// and Price to just show the numeric value, splitting from "EUR"
#"Added Custom" = Table.AddColumn(#"Trimmed Text", "Custom", each if [Column1.1] = "Additional data" then " "
else if [Column1.1] = "Price" then Text.Split([Column1.2]," "){1} else [Column1.2]),
//Remove unneeded column
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Column1.2"}),
//non-aggregated pivot
pivot = fnPivotAll(#"Removed Columns","Column1.1","Custom"),
//set data types (frequently a good idea in PQ
#"Changed Type1" = Table.TransformColumnTypes(pivot,{
{"Property type", type text},
{"Location", type text},
{"region", type text},
{"Additional data", type text},
{"Area", type text},
{"Price", Currency.Type}})
in
#"Changed Type1"
M-Code (for custom function)
be sure to rename this query: fnPivotAll
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/
(Source as table,
ColToPivot as text,
ColForValues as text)=>
let
PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))),
#"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _),
TableFromRecordOfLists = (rec as record, fieldnames as list) =>
let
PartialRecord = Record.SelectFields(rec,fieldnames),
RecordToList = Record.ToList(PartialRecord),
Table = Table.FromColumns(RecordToList,fieldnames)
in
Table,
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)),
#"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames),
#"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames)
in
#"Expanded Values"

How do I filter for the concentrate of all set filters in PowerBI?

I have data in this form:
In PowerBI, I added another column where I keep only code combinations that consist of one single code. I did this with something like IF(LEN(Code > 1), "", Code). Result:
This enables me to create a slicer that contains only single codes. I also added a table that shows Codes.
Now, when I select codes in the slicer, I want the table to show these codes, plus the exact combination of it.
For example, when I select A and B, the table should show me A and B and A, B. I don't want it to show A, B, C although it contains A and B.
If I filter for A and B and C, however, I want it to show A and B and C and A, B, C - but not A, B.
How can I achieve that?
All entries in Codes are saved as strings.
You need a disconnected (not connected to the base table) table for your slicer. Now, if I consider your base table name is - your_table_name, you can create the new slicer table with this below code-
slicer =
SELECTCOLUMNS(
FILTER(
your_table_name_8,
LEN(your_table_name_8[codes]) = 1
),
"code",your_table_name_8[codes]
)
After creating the new slicer table, check in the model is there any auto relation detected between 2 tables or not. If you find any relation, just Remove the relation.
Now, create your slicer from the newly created table and create this below measure in your base table your_table_name-
show_or_hide =
VAR current_code = MIN(your_table_name_8[codes])
VAR comma_separated_list =
CALCULATE (
CONCATENATEX (
VALUES(slicer[code]),
slicer[code],
","
)
)
RETURN
IF(
current_code = comma_separated_list || (LEN(current_code) = 1 && CONTAINSSTRING(comma_separated_list,current_code)),
1,
0
)
lets see the outcome-
Finally, you can apply a visual level filter using the new measure show_or_hide and apply a filter so that value with True only shown in the visual.
Reorder Combination
Let we have this following table-
Now this following code from Advanced Query Editor will give the the expected output-
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WctRx0nFWitUBsZx1nJRiYwE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 1, 1, Int64.Type),
#"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Index", "Column1"}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Reordered Columns", "Column1", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), {"Column1.1", "Column1.2", "Column1.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column1.1", type text}, {"Column1.2", type text}, {"Column1.3", type text}}),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type1", {"Index"}, "Attribute", "Value"),
#"Added Custom" = Table.AddColumn(#"Unpivoted Other Columns", "column_name", each "Column-" & [Value]),
#"Changed Type2" = Table.TransformColumnTypes(#"Added Custom",{{"column_name", type text}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type2",{"Attribute"}),
#"Reordered Columns1" = Table.ReorderColumns(#"Removed Columns",{"Index", "column_name", "Value"}),
#"Sorted Rows" = Table.Sort(#"Reordered Columns1",{{"Index", Order.Ascending}, {"column_name", Order.Ascending}}),
#"Pivoted Column" = Table.Pivot(#"Sorted Rows", List.Distinct(#"Sorted Rows"[column_name]), "column_name", "Value", List.Min),
#"Merged Columns" = Table.CombineColumns(#"Pivoted Column",{"Column-A", "Column-B", "Column-C"},Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"Merged")
in
#"Merged Columns"
Please check what applied in steps one by one for better understanding. Here is the output with ascending order in the combination-
Index column added for keep tracking the row from start to end. If you have already a similar column, you can use that column as well.

How to get the column names in excel

I am currently working on converting an excel table with >100 calculated columns into a DAX table with similar calculated columns.
Excel formula refers to the columns by their column coordinates, e.g. sum = A2+B2
whereas DAX refers the column name in the formula, e.g. sum = Principal +Interest
Now, when I am trying to convert excel version of sum=A2+B2, is there any clever way for me to find out A=Principal and B= Interest in the spreadsheet.
I am talking about a table which has >50 non calculated columns and >100 calculated columns to be converted to comparable DAX model. Every time I look into a formula for the calculated columns in excel,
e.g.
Interest = =IF(AND(AR7="Pending",OR(V7="Completed",V7="Approved", LEFT(V7,4)="OPEN")),IF(U7*1>0,U7*1,N7)/VLOOKUP(F7,Tax!A:B,2,0),0)
I need to go back and forth in the table to find out what are the corresponding column for each coordinate in this.
Is there any clever way for me to find out what column names correspond to which column coordinate in the above formula, for example AR=Status, V=Application, U=Amount.....
Thank you in advance.
Thanks everyone. I think I managed to solve it using Power query. The following generates a table for an excel table with columns from A-ZZ and their corresponding coordinates.
Table Name - Query1
let
Source = {"A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"},
#"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Added Custom" = Table.AddColumn(#"Converted to Table", "Custom", each {"A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"}),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Custom"),
#"Inserted Merged Column" = Table.AddColumn(#"Expanded Custom", "Merged", each Text.Combine({[Column1], [Custom]}, ""), type text),
#"Removed Columns" = Table.RemoveColumns(#"Inserted Merged Column",{"Column1", "Custom"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Merged", "Column1"}}),
Custom1 = #"Converted to Table"&#"Renamed Columns",
#"Added Index" = Table.AddIndexColumn(Custom1, "Index", 1, 1) in
#"Added Index"
Now if I have a table like following, I run COLUMN syntax on my data source columns which produces the column number and I can use the previous PQWRY table to this to get the names
then
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WUtLBQLE60UoupcnZCkCOe35+cSqQds4vB3EDMtNT8/OADI/8ouJUqAiIcs7ITM5OzQPrNQTyjYDYGIhNgNgUiM2A2FwpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}),
#"Removed Top Rows" = Table.Skip(#"Changed Type",1),
#"Transposed Table" = Table.Transpose(#"Removed Top Rows"),
#"Changed Type1" = Table.TransformColumnTypes(#"Transposed Table",{{"Column2", Int64.Type}}),
#"Merged Queries" = Table.NestedJoin(#"Changed Type1", {"Column2"}, Query1, {"Index"}, "Query1", JoinKind.LeftOuter),
#"Expanded Query1" = Table.ExpandTableColumn(#"Merged Queries", "Query1", {"Column1"}, {"Column1.1"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Query1",{"Column2"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column1.1", "Alphabetical_Coordinate"}}) in
#"Renamed Columns"
which gives me this
You can use VBA and the precedents member of the range object to get this. It will work for everything on the current sheet. I was not able to pick up the Tax!A:B and didn't have time to look further. Although, slightly incomplete, I at least wanted to provide this answer as a very good start
Sub formulaByField()
Dim formula As String
formula = Selection.formula
Debug.Print formula
Dim p As Range
For Each p In Selection.Precedents
'Debug.Print p.Address(False, False)
'Debug.Print Selection.Parent.Cells(1, p.Column).Value
formula = Replace(formula, p.Address(False, False), Selection.Parent.Cells(1, p.Column).Value)
Next
Debug.Print formula
End Sub
Code based on this setup:
Output
=IF(AND(A7="Pending",OR(B7="Completed",C7="Approved", LEFT(C7,4)="OPEN")),IF(D7*1>0,U7*1,E7)/VLOOKUP(F7,Tax!A:B,2,0),0)
=IF(AND(Duck="Pending",OR(Goose="Completed",Cow="Approved", LEFT(Cow,4)="OPEN")),IF(Pigeon*1>0,*1,Horse)/VLOOKUP(Pig,Tax!A:B,2,0),0)

Create a different pivot view in Power Query

I have the data structured in excel in the following format
What I want to do with that is to transform it into this. In simple words for each ID I want to record the difference in value from previous day, and if there is no value in previous day we just keep the current value.
As an intermediate step I am trying to transform the raw data into something like this but I am not sure how to go about it in simple Excel pivot tables, or Power query transformations.
There is something wrong with your sample because [v1-v2] is not the same method as [v5-v4, v3-v2, v8-v7] but I assume the latter ones were right
See if this works for you
Assumes data in 3 columns in a range named Table1 with column headers Dates, ID, Value
You can paste into PowerQuery using ... Advanced Editor ...
Creates a column with the value of yesterday for that ID and returns a null if nothing is found. Then does the subtraction, and pivots
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Dates", type date}, {"ID", type text}, {"Value", Int64.Type}}),
Yesterday = Table.AddColumn(#"Changed Type" , "Yesterday", (i) => List.Sum(Table.SelectRows( #"Changed Type", each ([ID] = i[ID] and Date.AddDays([Dates],1) = i[Dates]))[Value]), type number ),
#"Replaced Value" = Table.ReplaceValue(Yesterday,null,0,Replacer.ReplaceValue,{"Yesterday"}),
#"Added Custom" = Table.AddColumn(#"Replaced Value", "Custom", each [Value]-[Yesterday]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Value", "Yesterday"}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns", {{"Dates", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns", {{"Dates", type text}}, "en-US")[Dates]), "Dates", "Custom", List.Sum)
in #"Pivoted Column"

Resources