SCD-2 using delta live table - databricks

Delta live table now has the capability to do SCD Type 2 changes. But after going through this feature, I understood that this will work if I have only one new row with a new effective date. In the scenario where I have two new rows with two different effective dates, how do we handle such scenarios. For example, if I have a discount that is effective from today till next 3 months and one more discount which is effective after three months. In this case, whenever a change record comes, i will have to sort the rows to get the latest effective data and change that and not the future one
Thanks

Does those two discount have any unique key? If that is the case you can build the type2 using that. If not you can make the discount and discount end date as a key.

Related

How to Subtract a value from the oldest expiration date and move to the next once it hits zero?

I work for an organization and I am working on a way to better organize our part numbers that have expiration dates. I've started off by making an Excel sheet that has the part number multiple times based on the expiration date and then had the matching quantities of that expiration date. What im trying to do is what you always do in life, if something is going to go bad use up the one that will expire first. But now trying to do this in an excel table automatically. This is my Ex: my master log where they will put in the Part#, expiration date, and quantity, this is where most info will be pulled from.
I am now on to taking away our companies forecasted usage of these parts away from these quantities to show an overall usage over the months and pairing the intersecting expiration dates. I’ve got the Usage by using a pivot table to total these part numbers quantities. And I’ve made a chart mapping the overall usage to part quantity forecast with it being flagged if the expiration date interferes with the actual date. This is the forcasting for what we plan to use of each part #
This is me subtracting the usage number from the inventory of those part numbers and it being flagged when it expires.
=IF((IF((VLOOKUP($F60,Expires!$1:$1048576, 10, FALSE))<Sheet1!G$58, "Bad", "Good"))="good", T33, "Expired")
I used this function, my vlookup pulling from my first sheet and pulling the part numbers expiration date and if it is <the date up top (9/1,10/1 etc.) then it will be flagged as expired and wont even show the usage. T33 is simply pulling the fromual from another table which is just quantity- usage. Please lmk if this looks right my vlookup turns out pulls the latest expiration date from the multiple part #s.
But that is with total quantities, not separated based on the part number expiration dates. What I am wondering is how can I make a table like this but subtracts the usage from quantity of the part number that will expire first? Then after that quantity hits zero it takes away the usage from the next one due to expire? But also I want the Numbers to be able to auto populate from the sheet where the user enters the data (master log).
So this is what it will be doing and will look like the table above multiple of the same part number. Whats happening now
But needs to look like this Final table and add the part number to the list and quantity as they are entered in the master log. I was trying to do a Vlookup(MIN and take the part number from the other table based on the lowest expiration date first but I don’t think excel is capable of that, I’m not sure here.
Please help, thank you!

excel PowerPivot Auto Calculated Measures & Columns

After looking at a few similarish questions I figured I needed something more specific so asking here. I will start by explaining the situation:
The Setup
I have a Store which sells Cakes, Cookies and Wine. I have the weekly sales data of each product sorta like this:
Product ID
Product Name
Quantity
Value
Week Ending
1
Ginderbread
2
£4
13/01/22
2
Chocolate chip
5
£25
13/01/22
3
Red Wine Bottle
1
£10
13/01/22
4
Sponge Cake
3
£9
13/01/22
Currently every week's data is stored within the same table, with me using a Week filter to show only the week i'm interested in.
Using this Data I created PivotTables that shows the sales of each category, with the ability to drill down to show the specific products. Table looks something like this:
Category
Quantity
Value
Cakes
2
£4
Cookies
7
£29
Wine
1
£10
The issue
I now want to stick in a new calculated column that shows the Value as a %. E.g The total value for the previous table was £43, so Cookies is about 67%. If I drill down, it would show the Chocolate Chip record as 80% and Gingerbread as 20%
I imagine doing this would be easier if each individual week's data was on a different table, but I got a lot of weeks and I also want to do tables showing the sales for over a period of time. Plus I don't know of a way to merge the "value" and "quantity" columns, etc instead of having 1 for each week being shown.
any advice would be appreciated
Create an extra column in the source table (prior to filtering) entitled "perc" calculated as the corresponding value for each row divdied by the total value across all rows (se pic. / eqn. for first row below) --
=E2/$E$6
No calculated fields required - just include perc as the mesaure of interest in your pivot table, with value setting as 'sum':
The reason why this worked is because of the common denominator - which allows one to sum ratios on a 1:1 basis.
Devising a calculated field using the standard 'fields, items & sets' functionality for ordinary pivot tables would not be feasible / possible as far as I am aware. You would need to move into the realm of power pivots and data models - which is not too complicated (readily accesible directly from the field list per below) - however, I see this as unnecessary complication for the task at hand.
Side notes:
Using table names in your functions is sometimes more convenient when entering, albeit may appear tricky at first when reviewing - first eqn above becomes:
=[#Value]/Table1[[#Totals],[Value]]

turning rows to columns in Excel

I have an excel datasheet with more than 1,000,000 rows and 80 columns. the datasheet contains sales information of a chain store with more than 1700 store nationwide. each store is repeated 52(weeks in a year)* about 30 (products sold in that given week)* 2(two years). I want to convert the rows corresponding to products to columns. I can't do that using transverse because the products sold each week might not be exactly the same as those sold next week. do you have any solutions?
thanks
I just made a very simplified version of that excel file. the problem is that the products sold are not the same each week. there is a limited set of product, but only some of the items are sold each week
https://drive.google.com/open?id=1B2vjIL2hemfQNrCz0X6u_pzi7Euy6IWa3Lj0_HzDXDE
This isn't much of an answer yet - but it either will become one or I'll delete it, depending on the OP's response.
I'm thinking that transverse/transpose is the wrong term for what you're trying to do.
Perhaps you're just trying to better organize/visualize this data, something similar to one of these Pivot Tables:
or
These are just two of the infinite ways you can organize data in a Pivot Table.
Is that similar to what you're trying to do? If so I'll share some more info.
If this quantity of data is going to keep coming your way, way you really need is to start using an Access database to get this under control and be able to report on it properly (and easily, once it's setup).

Dynamic Sales Summary Depending on Month (can't use MAXIF/MINIF)

I have a student license of Office Pro 2016, but somehow it doesn't have the IFS, MINIF/MINIFS, MAXIF/MAXIFS functions that would make my life so much easier. I tried to illustrate what I'm doing in the image below:
Example
My actual data is much more complicated and long, but the idea is that I will periodically manually update data within the DATATABLE, and I want the SUMMARYTABLE to reflect the total pending sales by month. There are never more than two months pending at a time, and if only sales from one particular month are pending, I don't want the second row to appear.
I've already found a way to display the month in this case using regular MAX and MIN functions, but now I'm stumped as to how to display the sum total of sales within those months. Specifically, I don't know how to make a reference to a particular month in my criteria.
I've tried this formula:
=SUMIFS(DATATABLE[SALES],DATATABLE[SHOP]="A",DATATABLE[STATUS]="PENDING",MONTH(DATATABLE[DATE]),SHOPATOTALS[MONTH])
If this worked it would have been perfect, but the month portion isn't recognized as a proper range.

Excel PowerPivot DAX Calculated Field

I think I've got a relatively easy problem here on my hands, just having trouble getting it to work. Let me preface this by saying I'm new to DAX.
Consider the following PowerPivot Data Model :
It consists of a Sales Records table, which is joined to a Date lookup table, as well as a "Daily Expenses" table. The "Daily Expenses" table stores a single value which represents the averaged cost of running a specific trading location per day.
It's extremely easy to produce a PivotTable that provides me with the total Sales Amount per store per [insert date period]. What I'm after is a DAX formulation that will calculate the profit per store per day - ie. Total Sales minus DailyExpenses (operating cost):
In theory, this should be pretty easy, but I'm confused about how to utilise DAX row context.
Failed attempts:
Profit:=CALCULATE(SUM(SalesInformation[SaleAmount] - DailyStoreExpenses[DailyExpense]))
Profit:=CALCULATE(SUM(SalesInformation[SaleAmount] - DailyStoreExpenses[DailyExpense]), DailyStoreExpenses[StoreLocation])
Profit:=SUM(SalesInformation[SaleAmount] - RELATED(DailyStoreExpenses[DailyExpense])
etc
Any help appreciated.
Zam, unfortunately your failed attempts are not close :-)
The answer, and best practice, is use a 4th table called 'Stores' which contains a unique record per store - not only is this useful for bringing together data from your two fact tables but it can contain additional info about the stores which you can use for alternative aggregations e.g. Format, Location etc.
You should create a relationship between each of the Sales and Expenses tables and the Store table and then use measures like:
[Sales] = SUM(SalesInformation[SaleAmount])
[Expenses] = SUM(DailyStoreExpenses[DailyExpense])
[Profit] = [Sales] - [Expenses]
Provided you have the Date and Store tables correctly linked to the two 'Fact' tables (ie Sales and Expenses) then the whole thing should line up nicely.
Edit:
If you want to roll this up into weeks, years etc. and you have no kind of relationship between expenses and the calendar then you'll need to adjust your expenses measure accordingly:
[Expenses] = SUM(DailyStoreExpenses[DailyExpense]) * COUNTROWS(DateTable)
This will basically take the number of days in that particular filter context and multiply the expenses by it.

Resources