Converting excel date serial to date in postgres - excel

So I've got a large mass of excel files that aren't formatted properly and because of this their dates aren't being read in as dates, but as the date serial they use (i think the number of days since 1900-1-1?). Ex: 41917.0054050926 is really 10/5/2014 12:07:47 AM.
All I need is the actual date, not the time. I know I can get the year by doing 41917/365.25, rounding and adding this to 1900, but i'm not sure how to get the day and the month. Is there anything built into postgres to handle this? If not, does anyone know an arithmetic way of finding the day and year?
Thanks!

I just had this issue and wrote a plpgsql function to handle it. Note that it will work on whole numbers only meaning it will only do dates, not date and time. Someone else can feel free to expand it.
CREATE OR REPLACE FUNCTION To_Timestamp_From_Excel (ExcelDate integer)
RETURNS timestamp without time zone AS $$
BEGIN
IF ExcelDate > 59 THEN
ExcelDate = ExcelDate - 1;
END IF;
RETURN date '1899-12-31' + ExcelDate;
END;
$$ LANGUAGE plpgsql;
You would need to call it like this if you have date and time encoded or if your field type is text:
UPDATE myTable set MyDateField = to_Timestamp_From_Excel(MyDateField::integer)

With data in A1, in B1 enter:
=ROUND(A1,0)+1
and format B1 as mm/dd/yyyy
For example:

Its a little bit messy and not 100% accurate, but I can do this in a custom PG procedure:
41917.0054050926/365.25 gets the year
41917.0054050926%365.25 gets the day of the year
I can then add the day of the year as an interval to the last day of the previous year.

I use an instruction like that:
select timestamp '1899/12/30' + interval '43404 day 43200'
to obtain '2018/10/30 12:00:00'
In Excel this date is 43404.5, then i have integer part before "day" inside interval, and fraction multiplied by 86400 (seconds per day).

Try this
CREATE OR REPLACE FUNCTION cdate(rhs anyelement) RETURNS timestamp without time zone AS
$BODY$
DECLARE retval timestamp without time zone;
BEGIN
IF pg_typeof($1)=ANY ('{int2,int,int8,real,float8,numeric}'::regtype[]) THEN
retval:=DATE_TRUNC('second',DATE('1899-12-30')+$1*INTERVAL '1 day');
ELSE
IF pg_typeof($1)=ANY ('{timestamp,date}'::regtype[]) THEN
retval:=$1::timestamp without time zone;
END IF;
END IF;
RETURN retval;
END;
$BODY$
LANGUAGE plpgsql IMMUTABLE STRICT;

Without creating the function, you can try:
Postgresql convert sql date to excel serialized date:
date_part('day', here_your_date - '1900-01-01'::date)+2 AS excel_serialized_date
Excel serialized to Postgresql date:
-- for e.g.: excel_serialized_date = 44492
(to_date('1900-01-01','YYYY-MM-DD') + excel_serialized_date::integer)-2 as exceldate_to_postgresqldate,
/* https://learn.microsoft.com/en-us/office/troubleshoot/excel/wrongly-assumes-1900-is-leap-year
therefore -2 is needed */
In the Excel quick verification by the formula:
=Text(44492,"YYYY-MM-DD")

Related

Calculation with respect to dates is incorrect in oracle

I have to calculate some forumales based on Dates coming from table. For reference the exact value with formula is provided in Excel.
The formula and Excel answer is below.
Formula in Excel :
=IF(
D12>=DATE(2016,10,1),
(S12-T12)+(S12-T12)*2.5%
+(
IF(
ROUNDDOWN(YEARFRAC(DATE(2016,4,1),D12),0)=0,
0,
EFFECT(
(ROUNDDOWN(YEARFRAC(DATE(2016,4,1),D12),0)*2.5)%,
ROUNDDOWN(YEARFRAC(DATE(2016,4,1),D12),0)
)
)
)
*
((S12-T12)+((S12-T12)*2.5%)),
((S12-T12))
)
-(
IF(
AND(
D12>=DATE(2018,4,1),
D12<=DATE(2018,12,21),
E12<=DATE(2018,12,21)
),
7.69%*(S12-T12),
0
)
)
And the answer for which calculation in Excel is:- 30,153
and my answer coming is 28700/-
The value for D12 = 03/05/18 (dd/mm/yy) format, E12 = 21/04/19
S12 = 30000, T12 = 2000,
Here is my calculation logic provided in oracle.
IF TO_DATE(V_FINALSRDATE, 'dd-mm-yy') >= TO_DATE('01-10-2016', 'dd-mm-yy')
THEN
v_STD_REVISED_AMT := (V_STANDRD_AMT - v_OD_Discount) + (V_STANDRD_AMT - v_OD_Discount) * 2.5/ 100;
dbms_output.put_line( 'Standard revised amount 1: ' || v_STD_REVISED_AMT);
ELSE
v_STD_REVISED_AMT := (V_STANDRD_AMT - v_OD_Discount);
dbms_output.put_line( 'Standard revised amount 2: ' || v_STD_REVISED_AMT);
END IF;
Do I need to add one more IFELS part? Where is my logic failing?
Please help as where my logic is failing.
You are not doing the full calculation; you have calculated the (S12-T12)+(S12-T12)*2.5% but you have missed the second part of the calculation.
You need to implement an Oracle version of the YEARFRAC function using US (NASD) 30/360 day count basis (since you are not passing a 3rd argument to YEARFRAC) and then add in the second half of your Excel formula into your PL/SQL calculation.
If you want it to have the same behaviour as Excel then you will also need to implement all the bugs that Excel has as the documentation notes that:
The YEARFRAC function may return an incorrect result when using the US (NASD) 30/360 basis, and the start_date is the last day in February.
However, since the exact nature of the bug is not detailed, you will need to work out what the errors are and implement them yourself. But since you are using a start_date of 2016-04-01 then this may not apply (unless you also need to generalise this function for use elsewhere).
Alternatively, since you appear to be rounding the year fraction down to the nearest whole number then you are only calculating the number of full years between the dates and, instead of YEARFRAC, you could use:
EXTRACT( YEAR FROM (V_FINALSRDATE - DATE '2016-04-01') YEAR TO MONTH)
Or
FLOOR( MONTHS_BETWEEN( V_FINALSRDATE, DATE '2016-04-01' ) / 12 )

What's wrong with my formula? Just want to show the correct days left

I am trying to determine what is wrong with my formula. Just want to show the correct days left.
Exp. Date = RSPOExpDate
Using Domino Designer 8.
temp := ((RSPOExpDate - #Today)/60)/60;
tempdays := #TextToNumber(temp)/24;
days := #Left(#Text(tempdays); ".");
#If(days = #Text(days));
Screenshot
Don't include in your view selection formula or column formulas any function which delivers current time or date like #Now or #Today for performance reasons. Those functions cause the view to refresh every time it is called.
In case you want to stay with it then change your formula to
#Integer((RSPOExpDate - #Today)/60/60/24)
In case RSPOExpDate can be empty use this
#If(RSPOExpDate = ""; ""; #Integer((RSPOExpDate - #Today)/60/60/24))
Update:
Write an agent that runs sometime in first hours of every day with the formula:
FIELD DaysLeft := #If(RSPOExpDate = ""; ""; #Integer((RSPOExpDate - #Today)/60/60/24))
Your column formula is then just
DaysLeft
or you show a red/green flag icon
#If((DaysLeft)<=90; 181; 182)
I think the problem lies in your last line. Just set it to days. That will return the calculated number of days remaining (or days passed as it will show a negative for that).
As Knut says, the performance for this will be pretty poor. It would be much better for performance to have an agent that runs nightly to stamp the number of days remaining on the document and show this field in that column. You can use the same formula as you have and just replace the last line with #SetField("DaysRemianing"; days);at the end of the formula
The Agent based approach already outlined is correct. Consider using #BusinessDays to make life easier.
#BusinessDays(startDates; endDates; daysToExclude; datesToExclude)
Something like:
#BusinessDays(RSPOExpDate; #Today)
https://www.ibm.com/support/knowledgecenter/en/SSVRGU_9.0.1/basic/H_BUSINESSDAYS_FUNCTION.html

Difference time view from Excel to VBA

I can't obtain the average time from start to end of some activities,
I tried 1K way but the result isn't correct, every time I've one day minus.
the image can explain better (that my english).
In this example the sum of my activities il 480:52:56 hours, in vba I've different result, for vba the date is "19/01/1900 00:52:56" like 456:52:56 hous
24 hours minus
why this difference? and how I can obtain the same result?
thanks for any suggestion
Dates are stored as serial numbers where first valid date has a value of 1. This value in excel reads as 01/01/1900, and in VBA as 31/12/1899. In excel, value 60 returns 29/02/1900 which doesn't exist in VBA, so from value 61 onwards all values will return the same date in VBA and excel.
/e: Also, maximum value is 2958465 (31/12/9999), values higher than that will return error rather than valid date
thanks to your comments I understand that the problem is for the minor dates of March 1, 1900 so I changed the select from:
Select [DataAttesa] as Data, avg(iif([totHours] > 1 and [totHours] < 61, dateadd("d",-1,CDate([totHours])) , [totHours])) as nr FROM [db_In$] Where TypeTrasp = "AOG" group by [DataAttesa] Order by [DataAttesa]asc
now, when I put the recordset.results on excel the value are correct.
Thank at all

date time conversion

I have 40241 as a date value. Which format is this in?
I think it is in seconds past midnight.
But I need a formula so that I can work out manually and verify!!
Thanks
If it is an Excel datestamp, then it's the number of days since 31st December 1899 (with 1900 treated as a leap year); which puts it as 4th March 2010... unless Excel was configured to use the Mac 1904 Calendar, in which case it's the number of days since 1st January 1904.
How to convert it depends on your preferred scripting language; or whether you can simply use Excel itself, and just set the format mask for that cell to one of the date formats
If it is "seconds past midnight", you can simply do this:
divide by 3600 (seconds in an hour); you get the number of hours;
take the remainder of that division, and divide it by 60; you get the minutes;
the remainder is the seconds.
Example:
40241/3600=11 (641)
641/60=10 (41)
So it is 11:10:41.
By the way, I suppose that it's a time value; if it was a datetime value it would probably be much bigger (like UNIX timestamps) or it would have a decimal part (like, IIRC, OLE dates).
It turns out that it's an Excel date; then, have a look at this KB article, it's all explained in detail; but if you just want to display it correctly, go on the properties of the cell (Ctrl+1) and set its data type to "Date" or "Date/Time" (or whatever it was, I don't have Excel at hand at the moment).
Excel stores dates in an interesting way. I've had this crop up on me too but I never had to move outside Excel so I could just use the format function in Excel.
You can read more here:
http://www.ozgrid.com/Excel/ExcelDateandTimes.htm
http://www.cpearson.com/excel/datetime.htm
Excel saves the date as an integer for the number of days since Jan 1st, 1900
Note: there is a bug in excel so you do the conversion and subtract one. If you see a decimal after it is the time.
Here is some java code to convert it if you want to verify it:
public static Date ExcelDateParse(int ExcelDate){
Date result = null;
try{
GregorianCalendar gc = new GregorianCalendar(1900, Calendar.JANUARY, 1);
gc.add(Calendar.DATE, ExcelDate - 1);
result = gc.getTime();
} catch(RuntimeException e1) {}
return result;
}
I condensed the apache solution for the date without time ( https://svn.apache.org/repos/asf/poi/trunk/src/java/org/apache/poi/ss/usermodel/DateUtil.java )
public static Date parseExcelDate(double date) {
int wholeDays = (int) Math.floor(date);
Calendar calendar = new GregorianCalendar();
int startYear = 1900;
int dayAdjust = wholeDays < 61 ? 0 : -1;
calendar.set(startYear, 0, wholeDays + dayAdjust, 0, 0, 0);
return calendar.getTime();
}
Other thread: Program in Java to convert a date to serial number as done in Excel

Excel date to Unix timestamp

Does anyone know how to convert an Excel date to a correct Unix timestamp?
Non of these worked for me... when I converted the timestamp back it's 4 years off.
This worked perfectly: =(A2-DATE(1970,1,1))*86400
Credit goes to: Filip Czaja http://fczaja.blogspot.ca
Original Post: http://fczaja.blogspot.ca/2011/06/convert-excel-date-into-timestamp.html
Windows and Mac Excel (2011):
Unix Timestamp = (Excel Timestamp - 25569) * 86400
Excel Timestamp = (Unix Timestamp / 86400) + 25569
MAC OS X (2007):
Unix Timestamp = (Excel Timestamp - 24107) * 86400
Excel Timestamp = (Unix Timestamp / 86400) + 24107
For Reference:
86400 = Seconds in a day
25569 = Days between 1970/01/01 and 1900/01/01 (min date in Windows Excel)
24107 = Days between 1970/01/01 and 1904/01/02 (min date in Mac Excel 2007)
If we assume the date in Excel is in A1 cell formatted as Date and the Unix timestamp should be in a A2 cell formatted as number the formula in A2 should be:
= (A1 * 86400) - 2209075200
where:
86400 is the number of seconds in the day
2209075200 is the number of seconds between 1900-01-01 and 1970-01-01 which are the base dates for Excel and Unix timestamps.
The above is true for Windows. On Mac the base date in Excel is 1904-01-01 and the seconds number should be corrected to: 2082844800
Here is a mapping for reference, assuming UTC for spreadsheet systems like Microsoft Excel:
Unix Excel Mac Excel Human Date Human Time
Excel Epoch -2209075200 -1462 0 1900/01/00* 00:00:00 (local)
Excel ≤ 2011 Mac† -2082758400 0 1462 1904/12/31 00:00:00 (local)
Unix Epoch 0 24107 25569 1970/01/01 00:00:00 UTC
Example Below 1234567890 38395.6 39857.6 2009/02/13 23:31:30 UTC
Signed Int Max 2147483648 51886 50424 2038/01/19 03:14:08 UTC
One Second 1 0.0000115740… — 00:00:01
One Hour 3600 0.0416666666… ― 01:00:00
One Day 86400 1 1 ― 24:00:00
* “Jan Zero, 1900” is 1899/12/31; see the Bug section below. † Excel 2011 for Mac (and older) use the 1904 date system.
As I often use awk to process CSV and space-delimited content, I developed a way to convert UNIX epoch to timezone/DST-appropriate Excel date format:
echo 1234567890 |awk '{
# tries GNU date, tries BSD date on failure
cmd = sprintf("date -d#%d +%%z 2>/dev/null || date -jf %%s %d +%%z", $1, $1)
cmd |getline tz # read in time-specific offset
hours = substr(tz, 2, 2) + substr(tz, 4) / 60 # hours + minutes (hi, India)
if (tz ~ /^-/) hours *= -1 # offset direction (east/west)
excel = $1/86400 + hours/24 + 25569 # as days, plus offset
printf "%.9f\n", excel
}'
I used echo for this example, but you can pipe a file where the first column (for the first cell in .csv format, call it as awk -F,) is a UNIX epoch. Alter $1 to represent your desired column/cell number or use a variable instead.
This makes a system call to date. If you will reliably have the GNU version, you can remove the 2>/dev/null || date … +%%z and the second , $1. Given how common GNU is, I wouldn't recommend assuming BSD's version.
The getline reads the time zone offset outputted by date +%z into tz, which is then translated into hours. The format will be like -0700 (PDT) or +0530 (IST), so the first substring extracted is 07 or 05, the second is 00 or 30 (then divided by 60 to be expressed in hours), and the third use of tz sees whether our offset is negative and alters hours if needed.
The formula given in all of the other answers on this page is used to set excel, with the addition of the daylight-savings-aware time zone adjustment as hours/24.
If you're on an older version of Excel for Mac, you'll need to use 24107 in place of 25569 (see the mapping above).
To convert any arbitrary non-epoch time to Excel-friendly times with GNU date:
echo "last thursday" |awk '{
cmd = sprintf("date -d \"%s\" +\"%%s %%z\"", $0)
cmd |getline
hours = substr($2, 2, 2) + substr($2, 4) / 60
if ($2 ~ /^-/) hours *= -1
excel = $1/86400 + hours/24 + 25569
printf "%.9f\n", excel
}'
This is basically the same code, but the date -d no longer has an # to represent unix epoch (given how capable the string parser is, I'm actually surprised the # is mandatory; what other date format has 9-10 digits?) and it's now asked for two outputs: the epoch and the time zone offset. You could therefore use e.g. #1234567890 as an input.
Bug
Lotus 1-2-3 (the original spreadsheet software) intentionally treated 1900 as a leap year despite the fact that it was not (this reduced the codebase at a time when every byte counted). Microsoft Excel retained this bug for compatibility, skipping day 60 (the fictitious 1900/02/29), retaining Lotus 1-2-3's mapping of day 59 to 1900/02/28. LibreOffice instead assigned day 60 to 1900/02/28 and pushed all previous days back one.
Any date before 1900/03/01 could be as much as a day off:
Day Excel LibreOffice
-1 -1 1899/12/29
0 1900/01/00* 1899/12/30
1 1900/01/01 1899/12/31
2 1900/01/02 1900/01/01
…
59 1900/02/28 1900/02/27
60 1900/02/29(!) 1900/02/28
61 1900/03/01 1900/03/01
Excel doesn't acknowledge negative dates and has a special definition of the Zeroth of January (1899/12/31) for day zero. Internally, Excel does indeed handle negative dates (they're just numbers after all), but it displays them as numbers since it doesn't know how to display them as dates (nor can it convert older dates into negative numbers). Feb 29 1900, a day that never happened, is recognized by Excel but not LibreOffice.
Because my edits to the above were rejected (did any of you actually try?), here's what you really need to make this work:
Windows (And Mac Office 2011+):
Unix Timestamp = (Excel Timestamp - 25569) * 86400
Excel Timestamp = (Unix Timestamp / 86400) + 25569
MAC OS X (pre Office 2011):
Unix Timestamp = (Excel Timestamp - 24107) * 86400
Excel Timestamp = (Unix Timestamp / 86400) + 24107
You're apparently off by one day, exactly 86400 seconds.
Use the number 2209161600
Not the number 2209075200
If you Google the two numbers, you'll find support for the above.
I tried your formula but was always coming up 1 day different from my server. It's not obvious from the unix timestamp unless you think in unix instead of human time ;-) but if you double check then you'll see this might be correct.
I had an old Excel database with "human-readable" dates, like 2010.03.28 20:12:30
Theese dates were in UTC+1 (CET) and needed to convert it to epoch time.
I used the =(A4-DATE(1970;1;1))*86400-3600 formula to convert the dates to epoch time from the A column to B column values.
Check your timezone offset and make a math with it. 1 hour is 3600 seconds.
The only thing why i write here an anwser, you can see that this topic is more than 5 years old is that i use the new Excel versions and also red posts in this topic, but they're incorrect. The DATE(1970;1;1). Here the 1970 and the January needs to be separated with ; and not with ,
If you're also experiencing this issue, hope it helps you.
Have a nice day :)
None of the current answers worked for me because my data was in this format from the unix side:
2016-02-02 19:21:42 UTC
I needed to convert this to Epoch to allow referencing other data which had epoch timestamps.
Create a new column for the date part and parse with this formula
=DATEVALUE(MID(A2,6,2) & "/" & MID(A2,9,2) & "/" & MID(A2,1,4))
As other Grendler has stated here already, create another column
=(B2-DATE(1970,1,1))*86400
Create another column with just the time added together to get total seconds:
=(VALUE(MID(A2,12,2))*60*60+VALUE(MID(A2,15,2))*60+VALUE(MID(A2,18,2)))
Create a last column that just adds the last two columns together:
=C2+D2
To make up for the daylight saving time (starting on March's last sunday until October's last sunday) I had to use the following formula:
=IF(
AND(
A2>=EOMONTH(DATE(YEAR(A2);3;1);0)-MOD(WEEKDAY(EOMONTH(DATE(YEAR(A2);3;1);0);11);7);
A2<=EOMONTH(DATE(YEAR(A2);10;1);0)-MOD(WEEKDAY(EOMONTH(DATE(YEAR(A2);10;1);0);11);7)
);
(A2-DATE(1970;1;1)-TIME(1;0;0))*24*60*60*1000;
(A2-DATE(1970;1;1))*24*60*60*1000
)
Quick explanation:
If the date ["A2"] is between March's last sunday and October's last sunday [third and fourth code lines], then I'll be subtracting one hour [-TIME(1;0;0)] to the date.
If you can create a custom function (User Defined Function : UDF) :
create a VBA module and copy these codes :
Function unix_time(Optional my_year, Optional my_month, Optional my_day, Optional my_hour, Optional my_minute, Optional my_second)
Dim now_date, now_time, ref_date, ref_time
my_year = IIf(IsMissing(my_year) = False, my_year, year(Now))
my_month = IIf(IsMissing(my_month) = False, my_month, month(Now))
my_day = IIf(IsMissing(my_day) = False, my_day, day(Now))
my_hour = IIf(IsMissing(my_hour) = False, my_hour, hour(Now))
my_minute = IIf(IsMissing(my_minute) = False, my_minute, minute(Now))
my_second = IIf(IsMissing(my_second) = False, my_second, second(Now))
now_date = DateSerial(my_year, my_month, my_day)
now_time = TimeSerial(my_hour, my_minute, my_second)
ref_date = DateSerial(1970, 1, 1)
ref_time = TimeSerial(0, 0, 0)'You can change this line based on your time zone. for example : Tehran -> TimeSerial(3,30,0)
now_date = now_date + now_time
ref_date = ref_date + ref_time
unix_time = (now_date - ref_date) * 86400
End Function
Then type in any cell you want: =unix_time()
Above code returns current date and time in unix format.
If you need custom date and time, pass 6 parameters to "unix_time()" function.
Example : =unix_time(year, month, day, hour, minute, second)
Here's a javascript implementation.
// Parses an Excel Date ("serial") into a corresponding javascript Date in UTC+0 timezone.
// (with time equal to 00:00)
//
// https://www.pcworld.com/article/3063622/software/mastering-excel-date-time-serial-numbers-networkdays-datevalue-and-more.html
// "If you need to calculate dates in your spreadsheets,
// Excel uses its own unique system, which it calls Serial Numbers".
//
export default function parseExcelDate(excelSerialDate, options) {
// https://support.microsoft.com/en-gb/help/214330/differences-between-the-1900-and-the-1904-date-system-in-excel
if (options && options.epoch1904) {
excelSerialDate += 1462
}
// "Excel serial date" is just
// the count of days since `01/01/1900`
// (seems that it may be even fractional).
//
// The count of days elapsed
// since `01/01/1900` (Excel epoch)
// till `01/01/1970` (Unix epoch).
// Accounts for leap years
// (19 of them, yielding 19 extra days).
const daysBeforeUnixEpoch = 70 * 365 + 19
// An hour, approximately, because a minute
// may be longer than 60 seconds, due to "leap seconds".
//
// Still, Javascript `Date` (and UNIX time in general) intentionally
// drops the concept of "leap seconds" in order to make things simpler.
// So it's fine.
// https://stackoverflow.com/questions/53019726/where-are-the-leap-seconds-in-javascript
//
// "The JavaScript Date object specifically adheres to the concept of Unix Time
// (albeit with higher precision). This is part of the POSIX specification,
// and thus is sometimes called "POSIX Time". It does not count leap seconds,
// but rather assumes every day had exactly 86,400 seconds. You can read about
// this in section 20.3.1.1 of the current ECMAScript specification, which states:
//
// "Time is measured in ECMAScript in milliseconds since 01 January, 1970 UTC.
// In time values leap seconds are ignored. It is assumed that there are exactly
// 86,400,000 milliseconds per day."
//
// The fact is, that the unpredictable nature of leap seconds makes them very
// difficult to work with in APIs. One can't generally pass timestamps around
// that need leap seconds tables to be interpreted correctly, and expect that
// one system will interpret them the same as another. For example, while your
// example timestamp 1483228826 is 2017-01-01T00:00:00Z on your system,
// it would be interpreted as 2017-01-01T00:00:26Z on POSIX based systems,
// or systems without leap second tables. So they aren't portable.
// Even on systems that have full updated tables, there's no telling what those
// tables will contain in the future (beyond the 6-month IERS announcement period),
// so I can't produce a future timestamp without risk that it may eventually change.
//
// To be clear - to support leap seconds in a programming language, the implementation
// must go out of its way to do so, and must make tradeoffs that are not always acceptable.
// Though there are exceptions, the general position is to not support them - not because
// of any subversion or active countermeasures, but because supporting them properly is much,
// much harder."
//
// https://en.wikipedia.org/wiki/Unix_time#Leap_seconds
// https://en.wikipedia.org/wiki/Leap_year
// https://en.wikipedia.org/wiki/Leap_second
//
const hour = 60 * 60 * 1000
return new Date(Math.round((excelSerialDate - daysBeforeUnixEpoch) * 24 * hour))
}

Resources