I have a large amount of reference data in excel, which I am trying to manipulate in a variety of ways. I'm having some problems with the way it is structured and sorting into a more manageable format.
Problem number 1:
I have three columns. Column A contains first a date, and then a designator of high or low. Column B contains times, Column C contains heights.
I would like to sort the data by column B (easy enough) EXCEPT I would like the date headings in Column A preserved. It's almost as though I have 365 tables, each with between 3 and 5 pieces of data - I'm looking to sort the 3 - 5 pieces of data within each date only.
This is what I have currently:
There's no issue with me taking the data and manipulating it some other way first - this is ultimately around me being able to take a batch of data (5x different reference points, each for 365 days) and develop a process to sanitise it and get it displayed in time order, as well as being able to get it into a usable format for problem 2 (I need to adjust some other data points by the sorted data once I have it).
This is what I would like it to look like (I manually went through each of these blocks and sorted them):
It is possible to do it in Excel as follows in cell E2:
=LET(rng, A1:C11, set, FILTER(rng, (INDEX(rng,,1) <>"")),
dates, SCAN("", INDEX(set,,1), LAMBDA(acc, item, IF(ISNUMBER(item), item, acc))),
in, FILTER(HSTACK(dates, set), INDEX(set,,2)<>""), inDates, INDEX(in,,1),
out, REDUCE("", UNIQUE(inDates), LAMBDA(acc, date,
LET(sorted, VSTACK(date, DROP(SORT(FILTER(in, inDates = date),3),,1), {"","",""}),
VSTACK(acc, sorted)
))), IFERROR(DROP(DROP(out,1),-1),"")
)
Here is the output:
You can avoid the clean-up process except for removing the last row as follow:
=LET(rng, A1:C11, set, FILTER(rng, (INDEX(rng,,1) <>"")),
dates, SCAN("", INDEX(set,,1), LAMBDA(acc, item, IF(ISNUMBER(item), item, acc))),
in, FILTER(HSTACK(dates, set), INDEX(set,,2)<>""), inDates, INDEX(in,,1),
out, REDUCE("", UNIQUE(inDates), LAMBDA(acc, date,
LET(sorted, VSTACK(HSTACK(date,"",""), DROP(SORT(FILTER(in, inDates = date),3),,1),
{"","",""}), IF(MAX(LEN(acc))=0, sorted, VSTACK(acc, sorted))
))), DROP(out, -1)
)
Explanation
Basically is to carry out the manual steps but using excel functions. The name set, is the same as the input data (rng) but we removed the empty rows. The name dates, is a column with the same size as rng, repeating all the dates. The condition in the SCAN function to identify a new date is ISNUMBER because dates are stored in Excel as whole numbers. The name in has the data in the format we want for doing the sorting and filter by date removing the date header and adding as the first column the dates.
Now we use DROP/REDUCE/VSTACK pattern (check the answer to the question: how to transform a table in Excel from vertical to horizontal but with different length provided by David Leal) to append each sorted data for a given unique date. We add the date as the first row, then sorted data, and finally an empty row to separate each group of data. Finally, we do a clean-up via IFERROR/DROP to remove the #N/A values and the first and the last empty row.
I have a table that has a series of Columns with data I need to split out. Example below
STATUS#10/16 12:00:00 (CODE)
I've been able to split it easy enough and when I originally tried to set the date on an older dataset it identified it as a date e.g. 16th Oct 2021 However I started to get errors on this date column and trying with different datasets (10/12, 10/13, 10/14) it is not finding the date. I tried the following query code but I'm receiving errors
[STATUS DATE] is split to 10/14, 10/15 etc
#date( Date.Year(DateTime.LocalNow), Date.Month(Text.End([STATUS DATE]), 2), Date.Day(Text.Start([STATUS DATE]),2))
However I'm getting a function error so I tried
Date.From(Date.Day(Text.Start([STATUS DATE]),2) & Date.Month(Text.End([STATUS DATE]),2) & Date.Year(DateTime.LocalNow)
I have also tried to do this from an example column however the query created is looking at the cell value e.g. if 10/14 then 14/10/2021 else if 13/10 then 14/10/2021. This method i feel is prone for error once I include a larger dataset.
Is there anyway I can determine the date value based on mm/dd format? But with year end in mind, make the YYYY be determined by current year unless we move into Jan and then I don't want the Oct, Nov, Dec value showing as 2022.
You don't really show what your original data looks like.
But if it is like:
Source
Then you can use this code in the Add Custom Column dialog:
let
split=Text.SplitAny([STATUS DATE],"#/ "),
mnth = Number.From(split{1}),
dy = Number.From(split{2})
in
#date(Date.Year(DateTime.LocalNow()),mnth,dy)
The Text.SplitAny function lets you input a list of delimiters and the text will split on all of them. So it is relatively simple to extract the month and day values
to create:
Split [STATUS DATE] one more time into [Month] and [Day] column, using the "/" as a separator. Then you don't have to bother with 1 or 2 digit numbers and you can simply use this formula:
#date(Date.Year(DateTime.LocalNow()), [Month], [Day])
DateTime.LocalNow() is a function, so you need to add the brackets.
[Month] and [Day] are numbers already, so you don't need the Date.Month() or Date.Day() functions.
I have a job that loads a CSV into a MySQL table. The field I am paring to a date using the t-map component has empty strings for a few of the records. When parsing the string to date, I need to convert those empty strings to the current date. I've tried this expression in the tmap component with no luck. My field name is "Payroll_Pay_Date" Here is the expression:
row1.Payroll_Pay_Date equals("")?TalendDate.getCurrentDate():TalendDate.parseDate("MM/dd/yyyy",row1.Payroll_Pay_date)
I've tried several itterations of this. Actually, what I really want to is to have the empty string be replaced with the "current date - 7 days" but I figure I would start with the above. Any advice would be much appreciated.
You could do this:
row1.Payroll_Pay_Date == null || row1.Payroll_Pay_Date.trim().isEmpty() ? TalendDate.getCurrentDate() : TalendDate.parseDate("MM/dd/yyyy",row1.Payroll_Pay_date)
Here I first test if the string is null or if it's empty or containing only whitespaces, in which case I return the current date. Otherwise I parse the date.
To get the current date minus 7 days, you could do:
TalendDate.addDate(TalendDate.getCurrentDate(), -7, "dd")
I have a report that use a cutoff date, based on this date I have to read records from 3 months ago, is there any way I can create parameters to use base on this date?
for instance I was considering to create period parameters from given date as 07/13/2018:
begperiod = 05-2018
endperiod = 07-2018.
Any idea? is there any other way to use this fields on the filters as parameters?
Your best bet would be to adjust the date in your Filters on the line sending the date to be compared to SQL. You'd want to put in the Value1 column something like this:
=DateAdd( #Date, 'm', -3)
What that does is it takes your date parameter and subtracts 3 months from it. The entire line should look something like this:
DACTableName.DateFieldName GreaterOrEqual =DateAdd( #StartDate, 'm', -3)
I created a saved search on sales orders to calculate days between order date and ship date, grouped and averaged by class (the formula is cased to distinguish between 'Wholesale' class and all others). The numeric formula is:
CASE WHEN {class} = 'Wholesale' THEN {actualshipdate} - {startdate} ELSE {actualshipdate} - {shipdate} END
The summary type for the formula result is average. The summary-level results have way too many decimal places. Is there a way to round the summarized results to a pre-defined number of digits?
I have tried using the ROUND() function, both inside the CASE statement and as a wrap-around. I've also looked through general preferences for rounding defaults and haven't found any.
In order to round the result, you should implement you own average calculation and rounding.
Add the following result field besides the class grouping field that you already have:
Field : Formula (Numeric)
Summary Type : Maximum
Formula : ROUND(SUM(CASE WHEN {class} = 'Wholesale' THEN {actualshipdate} - {startdate} ELSE {actualshipdate} - {shipdate} END) / COUNT({internalid})),2)
Basically, you are doing here your own analytics.
Since the results are grouped, each result fields contains the whole result set of this field which allows you to add analytical functions on this group.
The summary type is MAXIMUM, just in order to show one result, so it can also be min or avg, no difference.
The function is calculating the sum of your formula and dividing it by the records count to get the average.