I'am working on this turtorial from PluralSight and have a problem converting a column with date & decimals to sperate columns.
According the turorial the formula if Value.Is([Value], type date) = true then [Value] else null
to create a custom column with name date
Anyone that knows a solution?
I have adapted the forumula for now
= Table.AddColumn(#"Changed Type", "Date", each if Text.Contains([Value], "/") then Date.From([Value],"en-US") else null)
Related
In my Power Query I have a column that shows different durations on certain items, but it displays an error when attempting to convert on time or duration.
As a solution next to my Excel Table I created a formula that alows to convert the duration in the format I wish to use, but I have not been able to translate the formula into a language that Power Query can understand (I am pretty new to Power Query).
This is how the data is pulled from source:
But I will like it to show like this:
The Excel Formula I am using to accomplish this is:
=IF(LEN([#Age])=7,"0"&[#Age],IF(LEN([#Age])=5,"00:"&[#Age],IF(LEN([#Age])=4,"00:0"&[#Age],IF(LEN([#Age])=3,"00:00"&[#Age],[#Age]))))
It will be nice to have it in the Power Query instead of the Excel sheet, as it serves as a learning oportunity.
I am self learning Power Query in Excel so any help is welcomed.
EDIT: In Case of the duration being more than 24:00:00, how will i approach it
Here is the error code it returns
You can add a custom column with the formula:
Duration.FromText(
Text.Combine(
List.LastN(
{"00"} & List.ReplaceValue(Text.Split([Age],":"),"","00",Replacer.ReplaceValue),
3),
":"))
The formula
Splits the text string by the colon into a List
Replacing blanks with {00} and also prepend the list with a {00} element
Retrieve the last three elements and combine them into a colon separated text string.
Use Duration.FromText function to convert to a duration.
Set the data type of the column to duration
In the PQ Editor, a duration will have the format of d.hh:mm:ss, but when you load it back into Excel, you can change that to [hh]:mm:ss
You can accomplish the above all in the PQ User Interface.
Here is M-Code that does the same thing:
let
Source = Excel.CurrentWorkbook(){[Name="Table16"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Age", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Duration", each Duration.FromText(
Text.Combine(
List.LastN(
{"00"} & List.ReplaceValue(Text.Split([Age],":"),"","00",Replacer.ReplaceValue),
3),
":"))),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Age"})
in
#"Removed Columns"
You can even do it (using M-Code in the Advanced Editor) without adding a column by using the Table.TransformColumns function:
let
Source = Excel.CurrentWorkbook(){[Name="Table16"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Age", type text}}),
#"Change to Duration" = Table.TransformColumns(#"Changed Type",
{"Age", each Duration.FromText(
Text.Combine(
List.LastN(
{"00"} & List.ReplaceValue(Text.Split(_,":"),"","00",Replacer.ReplaceValue),
3),
":")), type duration})
in
#"Change to Duration"
All result in:
Edit
With your modified data, now showing duration values of more than 23 hours (not allowed in a duration literal in PQ), the transformation will be different. We have to check the hours and break it into days and hours if it is more than 23.
Note: the below edit also assumes there will never be anything entered in the day location; and that entries for minutes and seconds will always be within range. If there might be day values, you will need to just add what's there to the "overflow" from the hours entry
So we change the Custom Column formula to check for that:
let
split = List.LastN({"00","00"} & List.ReplaceValue(Text.Split([Age],":"),"","00",Replacer.ReplaceValue),4),
s = Number.From(List.Last(split)),
m = Number.From(List.LastN(split,2){0}),
hTotal = Number.From(List.LastN(split,3){0}),
h = Number.Mod(hTotal,24),
d = Number.IntegerDivide(hTotal,24)
in #duration(d,h,m,s)
If you might have illegal values for minutes or seconds, you can add logig to check for that also
Also, if you will be loading this into Excel, and you might have total days >31, you will need to format it (in Excel), as [hh]:mm:ss as with the format d.hh:mm:ss Excel cannot display more than 31 days (although the proper value will be stored in the cell)
I have a file with 50,000 lines of data in 3 columns- Unique ID, Start Date, and End Date.
Using Power Pivot, I need to determine if any records with the same Unique ID have any overlapping dates. Each Unique ID appears about 5 times.
In excel, I would use a formula
SUMPRODUCT: =SUMPRODUCT(($B3<=$C$3:$C$13)*($C3>=$B$3:$B$13)*($A$3:$A$13=A3))>1
While this formula works really well in excel, with 50k+ records, this breaks my computer.
I was wondering, how would I perform that same calculation in power pivot/query.
Example of the data and calculation.
Thank you so much!
following a PowerQuery M-Code, this will solve your problem. Don't know how long it will take for 50k rows:
let
Quelle = Excel.CurrentWorkbook(){[Name="tab_Dates"]}[Content],
Change_Type = Table.TransformColumnTypes(Quelle,{{"Unique ID", type text}, {"Start Date", type date}, {"End Date", type date}}),
add_List_Dates = Table.AddColumn(Change_Type, "List_Dates", each List.Dates([Start Date], Duration.Days([End Date]-[Start Date])+1 , #duration(1,0,0,0))),
expand_List_Dates = Table.ExpandListColumn(add_List_Dates, "List_Dates"),
add_CountIF_ID_Date = Table.AddColumn(expand_List_Dates, "CountIF_ID_Date", (CountRows) =>
Table.RowCount(
Table.SelectRows(
expand_List_Dates,
each
([Unique ID] = CountRows[Unique ID] and [List_Dates] = CountRows[List_Dates])))),
Change_Type_2 = Table.TransformColumnTypes(add_CountIF_ID_Date,{{"CountIF_ID_Date", type text}}),
ChangeValue_CountIF_ID_Date = Table.ReplaceValue(Change_Type_2, each [CountIF_ID_Date], each if [CountIF_ID_Date] <> "1" then "FALSE" else "TRUE",Replacer.ReplaceText,{"CountIF_ID_Date"}),
Remove_Column_List_Dates = Table.RemoveColumns(ChangeValue_CountIF_ID_Date,{"List_Dates"}),
Remove_Duplicates = Table.Distinct(Remove_Column_List_Dates)
in
Remove_Duplicates
I am using a spreadsheet to log the tasks completed and in progress of a project. I was to generate some VBA code that can identify the latest delivery date within a task. However, in each task there are various sub tasks.
So the boundaries are the task which are whole numbers, and in between these whole numbers e.g. 46 and 46, are sub tasks.
The latest date needs to be calculated by examining the dates of the tasks between each whole number. E.g. 46.1,46.2,46.3 etc.
Would i be better by using the excel functions or would it be easier to use code?
e.g. the example of an excel function but in vba i would use.
Worksheets("Activity Overview").cells(n, "E").value = "=IFERROR(IF(AGGREGATE(14,7,'Sub Tasks'!S:S/(('Sub Tasks'!A:A>='Activity Overview'!A" & n & ")*('Sub Tasks'!A:A<'Activity Overview'!A" & n + 1 & ")),1),AGGREGATE(14,7,'Sub Tasks'!S:S/(('Sub Tasks'!A:A>='Activity Overview'!A" & n & ")*('Sub Tasks'!A:A<'Activity Overview'!" & n + 1 & ")),1),""""),"""")"
```
Use MAXIFS():
=MAXIFS(B:B,A:A,">="&E1,A:A,"<"&E1+1)
If one does not have MAXIFS then use AGGREGATE:
=AGGREGATE(14,7,$B$1:$B$6/(($A$1:$A$6>=E1)*($A$1:$A$6<E1+1)),1)
AGGREGATE is an array type formula and as such the references should be limited to the data range.
Here's a solution using SUMPRODUCT. It basically filters all values between the base value (>=49) and less than the base value plus one (<50).
You can also do this using Power Query aka Get & Transform available in Excel 2010+
Get Data from Table/Range
Change the Date column to Date format (it defaults to DateTime)
Create a custom column which is the Integer of the Task column and Name it Main Task
=Int64.From([Task])
Delete the original Task column
Group By the Main Task column
New Column Name Latest Date
Operation: Max
Column: Target Delivery Date
If there are any nulls in the Latest Date Column (from tasks with no delivery dates), you can either leave them, or filter them out.
Data
Results
M-Code
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Task", type number}, {"Target Delivery Date", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Main Task", each Int64.From([Task])),
#"Grouped Rows" = Table.Group(#"Added Custom", {"Main Task"}, {{"Latest Date", each List.Max([Target Delivery Date]), type date}}),
#"Filtered Rows" = Table.SelectRows(#"Grouped Rows", each ([Latest Date] <> null))
in
#"Filtered Rows"
I am looking to insert a remove column step which removes any column where the header (which is a date) is before a certain date (older than X years prior to the current date). I receive a large data dump which is just a list of client names and fees they pay each month from 2012 to today, headed by the month they pay each fee, but as time goes on I don't need the oldest of the data.
So far I have tried producing a list from the headers (based on a previous response from another board member - thankyou #horseyride!) and then removing the columns which dont meet the criteria FROM that list. However it keeps breaking.
This is the latest line in the advanced Editor
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Removed Columns", {{"Calendar Period", type text}}, "en-GB"), List.Distinct(Table.TransformColumnTypes(#"Removed Columns", {{"Calendar Period", type text}}, "en-GB")[#"Calendar Period"]), "Calendar Period", "Approved Invoice Amount", List.Sum)
This are the lines i am attempting to create:
"ColumnList" = List.Select(Table.ColumnNames(#"Pivoted Column"), each Text.Contains(_, " ")),,
"Delete Columns"= Table.Transform(#"Pivoted Column", Table.RemoveColumns(#"ColumnList", each {})as table)
in
#"Delete Columns"
the Second bit of code I cant seem to get right - that is what I believe it should look like for now. But essentially i want the table to remove any columns where their header (a date) is prior to X amount of years older than todays date.
EDIT - Screenshot of before and after IF the desired cut off was Dec 2012:
Example Data
Thank you in advance
Just use following code. For static date:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
final = Table.SelectColumns(Source, List.Select(Table.ColumnNames(Source),
each try Date.From(_) >= #date(2012,12,1) otherwise true))
in
final
For dynamic date (older than 3 years prior to the current date):
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
final = Table.SelectColumns(Source, List.Select(Table.ColumnNames(Source),
each try Date.From(_) >= Date.AddYears(Date.From(DateTime.FixedLocalNow()),-3)
otherwise true))
in
final
I'm a little rusty on PowerQuery.
I need to count "previous" entries in the same table.
For example, let's say we have a table of car sales.
For the purposes of PowerQuery, this table will be named tblCarSales
I need to add two aggregate columns.
The first aggregate column is the count of previous sales.
The Excel formula would be =COUNTIF([Sale Date],"<"&[#[Sale Date]])
The second aggregate column is the count of previous sales by make.
The Excel formula would be =COUNTIFS([Sale Date],"<"&[#[Sale Date]],[Make],[#Make])
How can this behavior be accomplished in PowerQuery, instead of using Excel formulas?
For example, I'm starting with the source statement:
let
Source = Excel.CurrentWorkbook(){[Name="tblCarSales"]}[Content]
in
Source
... where the source table only provides the Make, Model, and Sale Date columns.
You can do this sort of thing using List and Table functions. I'll show both.
let
Source = Excel.CurrentWorkbook(){[Name="tblCarSales"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Previous Sale Count",
(C) => List.Count(List.Select(Source[Sale Date],
each _ < C[Sale Date]))),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Previous Sale Count By Make",
(C) => Table.RowCount(Table.SelectRows(Source,
(S) => S[Sale Date] < C[Sale Date] and S[Make] = C[Make])))
in
#"Added Custom1"
We have to use the functions so that Power Query knows what context we're looking at the columns in. For further reading, check out this Power Query M Primer.