Apache POI Date Parsing One Second Off - apache-poi

I'm parsing an Excel spreadsheet with a date in it. The results from POI are off by 1 second compared to what's displayed in Excel.
The unformatted data in Excel is: 43261.5027743056
The cell in Excel has a format of: mm/dd/yyyy hh:mm:ss
The field in Excel displays as: 6/10/2018 12:04:00 PM
The POI parser (v 4.0.1 and 4.1.0 both) parse it as:
Value: 43261.502774305598
Format: mm/dd/yyyy\ hh:mm:ss
Result: 6/10/2018 12:03:59 PM
Here's my code:
private final DataFormatter formatter;
case NUMBER:
String n = value.toString();
if (this.formatString != null) {
thisStr = formatter.formatRawCellContents(Double.parseDouble(n), this.formatIndex, this.formatString);
}
else thisStr = n;
break;
Am I doing something wrong?

The problem is not the binary floating point problem. This also exists but it should not impact seconds of time.
The problem is that your value 43261.5027743056 is not really exact the date time 06/10/2018 12:04:00 but 06/10/2018 12:03:59.700. So it is 06/10/2018 12:03:59 plus 700 milliseconds. You could see this if you would formatting the cell using the format DD/MM/YYYY hh:mm:ss.000 in Excel.
For such values there is a discrepancy between Excel's date formatting and apache poi's DataFormatter, which uses Java's date format. When Excel shows the date time value 06/10/2018 12:03:59,700 without milliseconds, then it rounds to seconds internally. So 06/10/2018 12:03:59.700 is shown as 06/10/2018 12:04:00. Java's date formatters don't round but simply don't show the milliseconds. So 06/10/2018 12:03:59.700 is shown as 06/10/2018 12:03:59.
Apache poi's DateUtil provides methods which rounds seconds. But those methods seems not be used in DataFormatter.
As workaround we could override formatCellValue of DataFormatter to do so.
Complete example:
Excel:
Code:
import java.io.FileInputStream;
import org.apache.poi.util.LocaleUtil;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.formula.ConditionalFormattingEvaluator;
import java.util.Date;
class ExcelParseCellValues {
public static void main(String[] args) throws Exception {
Workbook workbook = WorkbookFactory.create(new FileInputStream("Excel.xlsx"));
DataFormatter dataFormatter = new DataFormatter() {
#Override
public String formatCellValue(Cell cell, FormulaEvaluator evaluator, ConditionalFormattingEvaluator cfEvaluator) {
CellType cellType = cell.getCellType();
if (cellType == CellType.FORMULA) {
if (evaluator == null) {
return cell.getCellFormula();
}
cellType = evaluator.evaluateFormulaCell(cell);
}
if (cellType == CellType.NUMERIC && DateUtil.isCellDateFormatted(cell, cfEvaluator)) { //we have a date
CellStyle style = cell.getCellStyle();
String dataFormatString = style.getDataFormatString();
if (!dataFormatString.matches(".*(s\\.0{1,3}).*")) { //the format string does not show milliseconds
boolean use1904Windowing = false;
if ( cell != null && cell.getSheet().getWorkbook() instanceof Date1904Support)
use1904Windowing = ((Date1904Support)cell.getSheet().getWorkbook()).isDate1904();
boolean roundSeconds = true; //we round seconds
Date date = DateUtil.getJavaDate(cell.getNumericCellValue(), use1904Windowing, LocaleUtil.getUserTimeZone(), roundSeconds);
double value = DateUtil.getExcelDate(date);
return super.formatRawCellContents(value, style.getDataFormat(), dataFormatString, use1904Windowing);
}
}
return super.formatCellValue(cell, evaluator, cfEvaluator);
}
};
CreationHelper creationHelper = workbook.getCreationHelper();
FormulaEvaluator formulaEvaluator = creationHelper.createFormulaEvaluator();
Sheet sheet = workbook.getSheetAt(0);
for (Row row : sheet) {
for (Cell cell : row) {
String cellValue = dataFormatter.formatCellValue(cell, formulaEvaluator);
System.out.print(cellValue + "\t");
}
System.out.println();
}
workbook.close();
}
}
Result:
Description of value Floatingpoint value DD/MM/YYYY hh:mm:ss.000 DD/MM/YYYY hh:mm:ss
Your example value 43261,5027743056 06/10/2018 12:03:59.700 06/10/2018 12:04:00
Exact Datetime 12:04 43261,5027777778 06/10/2018 12:04:00.000 06/10/2018 12:04:00
Exact minus 500 ms 43261,5027719907 06/10/2018 12:03:59.500 06/10/2018 12:04:00
Exact plus 500 ms 43261,5027835648 06/10/2018 12:04:00.500 06/10/2018 12:04:01
Exact minus 501 ms 43261,5027719792 06/10/2018 12:03:59.499 06/10/2018 12:03:59
Exact plus 501 ms 43261,5027835764 06/10/2018 12:04:00.501 06/10/2018 12:04:01

You're doing this when you parse the cell value as a double. Not all decimal values can be represented exactly as doubles. The nearest double to 43261.5027743056 is 43261.502774305597995407879352569580078125, which rounds to the value you're seeing.

Related

Flutter/Dart: Excel Date to Date Object

I am parsing an excel document in excel using the excel: ^1.1.5 package. In my sheet, i have a Date column and this Date is being received in my Flutter code in the following format
"44663"
rather than:
"2022/04/12"
How do I parse this to a format such as YY-MM-DD.
I have tried DateTime.parse(), but it throws an error that my date format is invalid.
I found the answer :
const gsDateBase = 2209161600 / 86400;
const gsDateFactor = 86400000;
final date = double.tryParse("44663");
if (date == null) return null;
final millis = (date - gsDateBase) * gsDateFactor;
print(DateTime.fromMillisecondsSinceEpoch(millis.toInt(), isUtc: true));

apache poi write HH:mm:ss string to Excel Time data type

i used apache poi to rewrite a CSV file to Excel, for any string like HH:mm:ss, i need to convert it to the appropriate Excel data type so that user can apply sum() function on that column. i tried different datatype, but when i open the excel file i cannot sum that column the sum always showed as "0:00:00", even though clicking on that column in excel it showed as 'Time' data type,
here is my code:
CreationHelper creationHelper = workbook.getCreationHelper();
HSSFCellStyle timeStyle = workbook.createCellStyle();
timeStyle.setDataFormat(creationHelper.createDataFormat().getFormat("h:mm:ss"));
cell.setCellValue(column);
if (isTime(column)) {
cell.setCellStyle(timeStyle);
}
private boolean isTime(String value) {
try {
dtf.parseDateTime(value);
return true;
} catch (IllegalArgumentException e) {
return false;
}
}
and here is my excel file
If the object column in your code is String, then the content of the cell will always be string (text) cell content after cell.setCellValue(column). This content the function SUM cannot work with. Those functions needs numeric content. In Excel date and time also is numeric content only formatted as date-time. With default settings 1 = 1 day = 01/01/1900 00:00:00. 1 hour = 1/24, 1 minute = 1/24/60, 1 second = 1/24/60/60.
If column is string of format "HH:MM:SS", then DateUtil.convertTime can be used to convert this string into a Excel valuable time.
Complete example which shows what not works and what works:
import java.io.FileOutputStream;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.DateUtil;
public class ExcelCalculateTimeValues {
public static void main(String[] args) throws Exception {
Workbook workbook = new HSSFWorkbook();
//Workbook workbook = new XSSFWorkbook();
CreationHelper createHelper = workbook.getCreationHelper();
CellStyle styletime = workbook.createCellStyle();
styletime.setDataFormat(createHelper.createDataFormat().getFormat("hh:mm:ss"));
Sheet sheet = workbook.createSheet();
sheet.createRow(0).createCell(0).setCellValue("Time sting");
sheet.getRow(0).createCell(1).setCellValue("Time");
String[][] tableData = new String[][]{
{"12:34:00", "22:45:00"},
{"23:45:05", "01:34:40"},
{"08:01:00", "13:23:00"},
{"15:41:12", "23:23:22"}
};
int r = 1;
for (String[] rowData : tableData) {
Row row = sheet.createRow(r++);
int c = 0;
for (String cellData : rowData) {
Cell cell = row.createCell(c);
if (c == 0 ) {
cell.setCellValue(cellData); //this sets string cell data
} else if (c == 1) {
cell.setCellValue(DateUtil.convertTime(cellData)); //this sets datetime cell data
}
cell.setCellStyle(styletime);
c++;
}
}
sheet.createRow(r).createCell(0).setCellFormula("SUM(A2:A"+r+")"); //cannot work because of string values in A2:A4
sheet.getRow(r).createCell(1).setCellFormula("SUM(B2:B"+r+")"); //will work
workbook.setForceFormulaRecalculation(true);
if (workbook instanceof HSSFWorkbook) {
workbook.write(new FileOutputStream("ExcelCalculateTimeValues.xls"));
} else if (workbook instanceof XSSFWorkbook) {
workbook.write(new FileOutputStream("ExcelCalculateTimeValues.xlsx"));
}
workbook.close();
}
}

Apache POI to excel - Zero Date

Please make sure you understand my problem before replying, it is not as simple as it looks. Please don't just do a google search and post the link to the results; I already looked.
I have a VB.Net application that we are replacing with a Java application. The purpose of the application is to write an excel sheet (.xls). The file is then sent over to a second party and they process the data in it. I am using the APACHE POI to write the file.
The final product is being rejected by the second party because two time fields are "not valid". After scratching my head for a while, I noticed that Java produced file and VB.Net produced file are handling 0 date values differently. Let's say the time is suppose to be 3:30 PM in military time, the data appears as 15:30 on both files. The problem is the date portion of the field:
VB.Net generated: 1/0/1900 15:30
Java generated: 1/1/1970 15:30
I can't seem to find a way to have the apache POI mimic the way excel handles 0 dates. The following are some of the things I tried.
I set my date/time variable in the java application as 1/0/1900 15:30. This gives me an error in the application.
I set my variable as a string and pass it to the worksheet and then set the format of the cell. I don't get an error, but the data stays as 'general' until I double click on the cell and press Enter. This process is suppose to be automated so this is not an option.
I set the formula of the cell to =TIMEVALUE("15:30"), but this was not accepted by the 2nd party.
Has anyone else ran into this problem? Can anyone think of a way around this? Having the second party change the way they read the file is not an option.
What you need to know is that Excel stores datetime values as floating point double values. There 0 = 00:00:00 and 1 = 24:00:00 = 01/01/1900 00:00:00. Also 0.5 = 12:00:00 and 1.5 = 36:00:00 = 01/01/1900 12:00:00. So in other words, Excels datetime values are starting with 0 and 1 is one day and is 01/01/1900. Also 1/24 is one hour, 1/24/60 is one minute and 1/24/60/60 is one second.
The problem using a Java Date is that the months in Calendar constructors are 0 based. So month 0 is January and new GregorianCalendar(1900, 0, 1, 15, 30, 0) will be 01/01/1900 15:30:00. And there is not a day 0, so new GregorianCalendar(1900, 0, 0, 15, 30, 0) will be 12/31/1899 15:30:00 and this will be -1 for Excel.
Because the problems with Excels date behavior are known, apache poi provides DateUtil.
Using this we can do:
import java.io.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.util.Calendar;
import java.util.GregorianCalendar;
class XSSFNullDateTest {
public static void main(String[] args) {
try {
Workbook wb = new XSSFWorkbook();
Sheet sheet = wb.createSheet("Sheet1");
CreationHelper creationHelper = wb.getCreationHelper();
CellStyle cellStyleTime = wb.createCellStyle();
cellStyleTime.setDataFormat(creationHelper.createDataFormat().getFormat("hh:mm:ss"));
//using a Calendar:
Calendar calendar = new GregorianCalendar(1900, 0, 1, 15, 30, 0);
System.out.println(calendar.getTime()); //01/01/1900 15:30:00
double doubleTime = DateUtil.getExcelDate(calendar, false);
System.out.println(doubleTime); //1.6458333333333335
Cell cell = sheet.createRow(0).createCell(0);
cell.setCellValue(doubleTime-1); //subtract 1 so we have day 0
cell = sheet.getRow(0).createCell(1);
cell.setCellValue(doubleTime-1); //subtract 1 so we have day 0
cell.setCellStyle(cellStyleTime);
//using a string:
doubleTime = DateUtil.convertTime("15:30:00");
System.out.println(doubleTime); //0.6458333333333334 = day 0 already
cell = sheet.createRow(1).createCell(0);
cell.setCellValue(doubleTime);
cell = sheet.getRow(1).createCell(1);
cell.setCellValue(doubleTime);
cell.setCellStyle(cellStyleTime);
OutputStream out = new FileOutputStream("XSSFNullDateTest.xlsx");
wb.write(out);
wb.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}

string automatically converted in spring

I'm working on a project in Spring using SpringMVC. I'm importing data from (.xls) files .
the problem is that:
I'm reading this value "945854955" as a String but saved in DB as "9.45854955E8"
this value "26929" saved as "26929.0"
this value "21/05/1987" saved as "31918.0"
/read Code
// import ...
#RequestMapping(value="/read")
public String Read(Model model,#RequestParam CommonsMultipartFile[] fileUpload)
throws IOException, EncryptedDocumentException, InvalidFormatException {
List<String> liste = new ArrayList();
Employe employe = new Employe();
String modelnom = null;
liste = extraire(modelnom); //See the second code
for (int m=0, i=29;i<liste.size();i=i+29) {
if(i % 29 == 0) {
m++;
}
employe.setNomEmploye(liste.get(29*m+1));
//...
employe.setDateNaissance((String)liste.get(29*m+8).toString()); // here i had the date problem
employe.setDateEntree((String)liste.get(29*m+9).toString()); // here i had the date problem
employe.setDateSortie((String)liste.get(29*m+10).toString()); // here i had the date problem
// ...
employe.setNumCpteBanc(liste.get(29*m+17)); // here i had the first & second case problem
employe.setNumCIMR(liste.get(29*m+19)); // here i had the first & second case problem
employe.setNumMUT(liste.get(29*m+20)); // here i had the first & second case problem
employe.setNumCNSS(liste.get(29*m+21)); // here i had the first & second case problem
boolean bool=true;
List<Employe> employes = dbE.getAll();// liste des employes
for (int n=0;n<employes.size();n++) {
if (employes.get(n).getMatriculeMY() == (int)mat ) {
bool= false;
}
}
if (bool) {
dbE.create(employe);
}
}
return "redirect";
}
extraire code
private List<String> extraire (String nomFichier) throws IOException {
List<String> liste = new ArrayList();
FileInputStream fis = new FileInputStream(new File(nomFichier));
HSSFWorkbook workbook = new HSSFWorkbook(fis);
HSSFSheet spreadsheet = workbook.getSheetAt(0);
Iterator < Row > rowIterator = null;
// recup une ligne
rowIterator = spreadsheet.iterator();
while (rowIterator.hasNext()) {
int i = 0;
row = (HSSFRow) rowIterator.next();
Iterator < Cell > cellIterator = row.cellIterator();
while ( cellIterator.hasNext()) {
Cell cell = cellIterator.next();
i++;
/**
* Pour verifier si une ligne est vide. (for verifing if the line is empty)
*/
if (i % 29 == 0 || i == 1) {
while ( cellIterator.hasNext() && cell.getCellType() == Cell.CELL_TYPE_BLANK) {
cell = cellIterator.next();
}
}
switch (cell.getCellType()) {
case Cell.CELL_TYPE_NUMERIC:
String cellule = String.valueOf(cell.getNumericCellValue());
liste.add(cellule);
break;
case Cell.CELL_TYPE_STRING:
liste.add(cell.getStringCellValue());
break;
case Cell.CELL_TYPE_BLANK:
cellule = " ";
liste.add(cellule);
break;
}
}
}
fis.close();
return liste;
}
}
Excel's tries to data type cells and sometimes when you explicitly specify the data type Excel may try and cast the cell. You can try to right click on the cell and select 'Format Cell', then select 'Text' as the type (Category). However, at parse time it may still get hosed up.
Your quickest solution might be to save the file as a CSV and use that. You can still edit it in Excel. Although you will need to do some validation to ensure Excel isn't trying to do the above conversions on CSV save as. There are a lot of good Java CSV parsers out there OpenCSV, Super CSV.
The most time consuming, but probably the most correct way, if you want to continue to use Excel, is build a middle ware layer that parses the row and correctly identifies and formats the cell values. Apache POI and HSSF & XSSF can be used. Be warned that to handle xls and xlsx requires two different sets of libraries and often enough abstraction to handle both.
See https://poi.apache.org/spreadsheet/
As an Example:
protected String getCellValue(final Cell cell){
if (null == cell) { return null; }
// For Excel binaries 97 and below, The method of setting the cell type to CELL_TYPE_STRING converts the
// Formatted to date to a short. To correct this we check that the cell type is numeric and the check that it is
// date formatted. If we don't check that it is Numeric first an IllegalAccessorException is thrown.
if(cell.getCellType() == Cell.CELL_TYPE_NUMERIC && isCellDateFormated(cell) {
// isCellDateFormated is seperate util function to look at the cell value in order to determine if the date is formatted as a double.
// is a date format.
return // do date format procedure.
}
cell.setTypeCell(Cell.CELL_TYPE_STRING);
return cell.toString();
}
Hope this helps.
============Update==================
Instead of calling methods like "getNumericCellValue()" try setting the cell type to String and using toString like the example above. Here is my test code.
Note the xls file has one row and 4 cells in csv: "abba,1,211,q123,11.22"
public void testExtract() throws Exception{
InputStream is = new FileInputStream("/path/to/project/Test/src/test/java/excelTest.xls");
HSSFWorkbook wb = new HSSFWorkbook(is);
HSSFSheet sheet = wb.getSheetAt(0);
Iterator<Row> rowIter = sheet.iterator();
while (rowIter.hasNext()){
HSSFRow row = (HSSFRow) rowIter.next();
Iterator<Cell> cellIter = row.cellIterator();
while (cellIter.hasNext()){
Cell cell = cellIter.next();
System.out.println("Raw to string: " + cell.toString());
// Check for data format here. If you set a date cell to string and to string the response the output is funky.
cell.setCellType(Cell.CELL_TYPE_STRING);
System.out.println("Formatted to string: " + cell.toString());
}
}
is.close();
}
Output is
Raw to string: abba
Formatted to string: abba
Raw to string: 1.0
Formatted to string: 1
Raw to string: 211.0
Formatted to string: 211
Raw to string: q1123
Formatted to string: q1123
Raw to string: 11.22
Formatted to string: 11.22

DateTime is rounded up to the next day using ExcelLibrary

The datetime I'm writing to Excel always get rounded up to the next day:
workSheet.Cells[0, 0] = new Cell(DateTime.Now, new CellFormat(CellFormatType.DateTime, #"HH:mm:ss"));
In the output Excel file the cell gets this value: 29/09/2013 00:00:00
The DateTime.Now from this example is 28/09/2013 19:42:23
I ended up passing the cell value as a string instead of as a DateTime:
workSheet.Cells[0, 0] = new Cell(DateTime.Now.ToString(#"HH:mm:ss:ff"),
new CellFormat(CellFormatType.DateTime, #"HH:mm:ss"));
If you are using the ExcelLibrary Project Source Code, you can fix this by:
Go to SharedResource Class in this location: [Project Source Code folder]\Office\BinaryFileFormat folder
Change the EncodeDateTime function as below:
public double EncodeDateTime(DateTime value)
{
double days = (value - BaseDate).Days;
//if (days > 365) days++;
return days;
}
Pass the DataTime object to the Cell with the prefered format:
worksheet.Cells[iIndex, j] = new Cell(((DateTime)cellValue), new CellFormat(CellFormatType.DateTime, #"dd/MM/yyyy"));
You need to convert the date format from OLE Automation to the .net format by using DateTime.FromOADate.
If oCell.Format.FormatType = CellFormatType.Date OrElse oCell.Format.FormatType = CellFormatType.DateTime Then
Dim d As Double = Double.Parse(oCell.Value)
Debug.print(DateTime.FromOADate(d))
End If

Resources