XLSX removing sheets OutOfMemory Exception - apache-poi

I am trying to load the XLSX file using POI library that has 5 sheets. Size of the file is 5 MB. Total records in all sheets are around 30,000.
Once the file is loaded i need to delete the 1 or more sheets on the fly based on sheet neame as input.
Here is the snippet.
public void generateReportWorkBook(String[] requestedReports) throws Exception {
// Read the file
String dailyTicketReport = ReportConstants.REPORT_PATH + ReportConstants.FILE_NAME + ReportConstants.XLSX_FILE_EXTN;
FileInputStream fis = null;
XSSFWorkbook book = null;
try {
fis = new FileInputStream(dailyTicketReport);
book = new XSSFWorkbook(fis);
for (int i = book.getNumberOfSheets() - 1; i >= 0; i--) {
XSSFSheet tmpSheet = book.getSheetAt(i);
if (!ArrayUtils.contains(requestedReports, tmpSheet.getSheetName())) {
book.removeSheetAt(i);
}
}
} catch (Exception e) {
logger.error("Error occured while removing the sheets from workbook");
throw e;
} finally {
IOUtils.closeQuietly(fis);
}
}
When i execute the program. I get OutofMemory Exception.
How can i remove the sheets without memory issue.

I too faced the same issue of OOM while parsing xlsx file...after two days of struggle, I finally found out the below code that was really perfect;
This code is based on sjxlsx. It reads the xlsx and stores in a HSSF sheet.
// read the xlsx file
SimpleXLSXWorkbook = new SimpleXLSXWorkbook(new File("C:/test.xlsx"));
HSSFWorkbook hsfWorkbook = new HSSFWorkbook();
org.apache.poi.ss.usermodel.Sheet hsfSheet = hsfWorkbook.createSheet();
Sheet sheetToRead = workbook.getSheet(0, false);
SheetRowReader reader = sheetToRead.newReader();
Cell[] row;
int rowPos = 0;
while ((row = reader.readRow()) != null) {
org.apache.poi.ss.usermodel.Row hfsRow = hsfSheet.createRow(rowPos);
int cellPos = 0;
for (Cell cell : row) {
if(cell != null){
org.apache.poi.ss.usermodel.Cell hfsCell = hfsRow.createCell(cellPos);
hfsCell.setCellType(org.apache.poi.ss.usermodel.Cell.CELL_TYPE_STRING);
hfsCell.setCellValue(cell.getValue());
}
cellPos++;
}
rowPos++;
}
return hsfSheet;

Related

I've created a .xlsx file using Apache POI and It can be opened in Linux. But In windows, that file is getting corrupted and can't open using MSExcel

Here's how I tried to do the task and it's working perfectly in Linux. But in windows, it's getting corrupted.
HSSFWorkbook book = new HSSFWorkbook();
HSSFSheet sheet = book.createSheet();
HSSFRow row = sheet.createRow(0);
for (int i = 0; i <tblAttendance.getColumns().size(); i++) {
row.createCell(i).setCellValue(tblAttendance.getColumns().get(i).getText());
}
for (int i = 0; i <tblItems.size(); i++) {
row=sheet.createRow(i+1);
for (int j = 0; j < tblAttendance.getColumns().size(); j++) {
row.createCell(j).setCellValue(tblAttendance.getColumns().get(j).getCellData(i).toString());
}
}
FileChooser fileChooser = new FileChooser();
File file = fileChooser.showSaveDialog(btnFilter.getScene().getWindow());
if (file!=null){
Path path = Paths.get(file.getAbsolutePath()+".xlsx");
try {
OutputStream outputStream = Files.newOutputStream(path);
book.write(outputStream);
outputStream.close();
new Alert(Alert.AlertType.CONFIRMATION,"Backup Success!",ButtonType.OK).show();
} catch (IOException e) {
new Alert(Alert.AlertType.ERROR,"Something went wrong! Please try again!",ButtonType.OK).show();
e.printStackTrace();
}

org.apache.poi.openxml4j.exceptions.InvalidOperationException: Can't open the specified file

My code is not working, it always shows the above mentioned exception.
but I can always see the tmp file being generated.
here is the code, can someone please suggest something:
FileInputStream fis =null;
try{
fis= new FileInputStream(new File("/home/amar/Desktop/new/abc.xls"));
Workbook wb = new org.apache.poi.xssf.usermodel.XSSFWorkbook(fis);
int numOfSheets = wb.getNumberOfSheets();
System.out.println("bhargo num of sheets is " + numOfSheets);
for(int i=0; i<numOfSheets; i++){
org.apache.poi.ss.usermodel.Sheet sheet = wb.getSheetAt(i);
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext())
{
Row row = rowIterator.next();
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = (Cell) cellIterator.next();
if (cell.getCellType() == cell.CELL_TYPE_STRING) {
System.out.println("bhargo cell value is " + cell.getStringCellValue().trim());
}
}
}
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally{
System.out.println("bhargo, closing the stream");
try {
fis.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
There are a number of issues here:
fis= new FileInputStream(new File("/home/amar/Desktop/new/abc.xls"));
Workbook wb = new org.apache.poi.xssf.usermodel.XSSFWorkbook(fis);
Firstly, as explained in the Apache POI documentation, don't use an InputStream if you have a file! It's slower and uses more memory
Secondly, XSSF is the code for working with .xlsx files, but your file is a .xls one, so that won't work.
Thirdly, Apache POI has code which will automatically work out what kind of file yours is, and create the appropriate workbook for you
Your code should therefore instead be
Workbook wb = WorkbookFactory.create(new File("/home/amar/Desktop/new/abc.xls"));
This will create the right kind of workbook, direct from the file
I was able to solve my problem.
I am working on linux so it was saving the file in the older version of excel

Selenium excel read and write to find row number

In my program I want to find the row number in the excel sheet matching the string I have passed as argument . It works fine for first and second row but problem is with the next rows. My code to find row number is as below :
public int findrownum(String sName, String value, int cNum) throws Exception{
File excel = new File(filepath);
FileInputStream fis = new FileInputStream(excel);
XSSFWorkbook wb = new XSSFWorkbook(fis);
XSSFSheet ws = wb.getSheet(sName);
boolean check = true;
int i=0;
while (check){
XSSFRow rowH = ws.getRow(i);
XSSFCell cell = rowH.getCell(cNum);
String cellvalue = cellToString(cell);
if (cellvalue.equals(value)){
check = false;
}
else {
i = i+1;
}
}
return i;
}
}
I want to read third row that is the string with name registration from the excel
Sl No test case name result timestamp
1 login Pass 03/03/2014 12:11:43 PM
2 Registration
Please let me know what changes needs to be done in the code .
Thanks
I used the similar logic as mentioned by #eric in JUNIT now i am able to find the row number .But now its giving error while i try to read the data using this row number . My code to read data is as below . Please let me know what changes needs to be done public String dataread(String sName, int rNum, String cName) throws Exception{
File excel = new File(filepath);
FileInputStream fis = new FileInputStream(excel);
XSSFWorkbook wb = new XSSFWorkbook(fis);
XSSFSheet ws = wb.getSheet(sName);
XSSFRow rowH = ws.getRow(rNum-1);
int totalRows = ws.getLastRowNum();
int i =0;
for(i=0;i<=totalRows;i++)
{
XSSFCell cell = rowH.getCell(i);
String value = cellToString(cell);
if (value.equals(cName)){
System.out.println(i);
break;
}
}
XSSFRow row = ws.getRow(rNum);
XSSFCell cell = row.getCell(i);
String value = cellToString(cell) return value;
}
In general From this Documentation you can use the getHeight() to get in which your cursor instead of writing up your own loop. Obviously this would reduce the execution time as well. Also the code which you have written could have caused the exception,as there is no more physical rows.
ws.getRow(i); can cause a fatal error if i>height of the last row
Hope the following code helps. The assumption is the data in the cell is string data. Also this is with apache poi api.
public static String getcellValue(int testRowNo, int colNo)
{
String projectPath = System.getProperty("user.dir");
String excelPath = projectPath + "/TestSet.xlsx";
File excel = new File(excelPath);
FileInputStream fis = null;
Workbook workBook = null;
String cellValue = null;
try
{
fis = new FileInputStream(excel);
workBook = WorkbookFactory.create(fis);
Sheet workSheet = workBook.getSheet(sheetName);
int totalRows = workSheet.getLastRowNum();
Row row = null;
cellValue = workSheet.getRow(testRowNo).getCell(colNo).getStringCellValue();
} catch (InvalidFormatException e)
{
e.printStackTrace();
} catch (IOException e)
{
e.printStackTrace();
}finally
{
try
{
fis.close();
} catch (IOException e)
{
e.printStackTrace();
}
}
return cellValue;
}
public static int getrowNumber(String sheetName, String cellData)
{
String projectPath = System.getProperty("user.dir");
String excelPath = projectPath + "/TestSet.xlsx";
File excel = new File(excelPath);
FileInputStream fis = null;
Workbook workBook = null;
String cellValue = null;
try
{
fis = new FileInputStream(excel);
workBook = WorkbookFactory.create(fis);
Sheet workSheet = workBook.getSheet(sheetName);
int totalRows = workSheet.getLastRowNum();
Row row = null;
int testRowNo = 0;
for(int rowNo =1; rowNo<=totalRows; rowNo++)
{
row = workSheet.getRow(rowNo);
testRowNo = testRowNo +1;
if(row.getCell(0).getStringCellValue().equalsIgnoreCase(cellData))
{
break;
}
}
} catch (InvalidFormatException e)
{
e.printStackTrace();
} catch (IOException e)
{
e.printStackTrace();
}finally
{
try
{
fis.close();
} catch (IOException e)
{
e.printStackTrace();
}
}
return testRowNo;
}

Excel Found unreadable content -POI XSSFWorkbook setcellvalue

I am trying to setcellvalue for xlsx file , Program works fine with no error , but while opening the xlsx file it throws an error saying Excel Found unreadable content
public boolean setCellData(String sheetName,String colName,int rowNum, String data){
try{
fis = new FileInputStream(path);
workbook = new XSSFWorkbook(path);
if(rowNum<=0)
return false;
int index = workbook.getSheetIndex(sheetName);
int colNum=-1;
if(index==-1)
return false;
sheet = workbook.getSheetAt(index);
row=sheet.getRow(0);
for(int i=0;i<row.getLastCellNum();i++){
//System.out.println(row.getCell(i).getStringCellValue().trim());
if(row.getCell(i).getStringCellValue().trim().equals(colName))
colNum=i;
}
if(colNum==-1)
return false;
sheet.autoSizeColumn(colNum);
row = sheet.getRow(rowNum-1);
if (row == null)
row = sheet.createRow(rowNum-1);
cell = row.getCell(colNum);
if (cell == null)
cell = row.createCell(colNum);
// cell style
//CellStyle cs = workbook.createCellStyle();
//cs.setWrapText(true);
//cell.setCellStyle(cs);
cell.setCellValue(data);
fileOut = new FileOutputStream(path);
workbook.write(fileOut);
fileOut.close();
}
catch(Exception e){
e.printStackTrace();
return false;
}
return true;
}
You have initialized FileInputStream, but not used that. Please replace
fis = new FileInputStream(path);
workbook = new XSSFWorkbook(path);
with
fis = new FileInputStream(path);
workbook = new XSSFWorkbook(fis);
this works on my end and not showing any error.

how to read data from csv file into C# console Application

using System;
namespace jagged_array
{
class Program
{
static void Main(string[] args)
{
string[][] Members = new string[10][]{
new string[]{"amit","amit#gmail.com", "9999999999"},
new string[]{"chandu","chandu#gmail.com","8888888888"},
new string[]{"naveen","naveen#gmail.com", "7777777777"},
new string[]{"ramu","ramu#gmail.com", "6666666666"},
new string[]{"durga","durga#gmail.com", "5555555555"},
new string[]{"sagar","sagar#gmail.com", "4444444444"},
new string[]{"yadav","yadav#gmail.com", "3333333333"},
new string[]{"suraj","suraj#gmail.com", "2222222222"},
new string[]{"niharika","niharika#gmail.com","11111111111"},
new string[]{"anusha","anusha#gmail.com", "0000000000"},
};
for (int i =0; i < Members.Length; i++)
{
System.Console.Write("Name List ({0}):", i + 1);
for (int j = 0; j < Members[i].Length; j++)
{
System.Console.Write(Members[i][j] + "\t");
}
System.Console.WriteLine();
}``
Console.ReadKey();
}
}
}
The above is the code for my C# console program in which i used jagged array and i assigned values manually but now my requirement is 'without assigning manually into array i want the same details to import into my program from an csv file(which is at some location in my disc). So how to do it what functions should i make use , please help me with some example. Thank you.
static void Main()
{
string csv_file_path=#"C:\Users\Administrator\Desktop\test.csv";
DataTable csvData = GetDataTabletFromCSVFile(csv_file_path);
Console.WriteLine("Rows count:" + csvData.Rows.Count);
Console.ReadLine();
}
private static DataTable GetDataTabletFromCSVFile(string csv_file_path)
{
DataTable csvData = new DataTable();
try
{
using(TextFieldParser csvReader = new TextFieldParser(csv_file_path))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
while (!csvReader.EndOfData)
{
string[] fieldData = csvReader.ReadFields();
//Making empty value as null
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
}
csvData.Rows.Add(fieldData);
}
}
}
catch (Exception ex)
{
}
return csvData;
}
Treat the CSV file like an excel workbook and you will find a lot of examples on the web for what you need to do.
ExcelFile ef = new ExcelFile();
// Loads file.
ef.LoadCsv("filename.csv");
// Selects first worksheet.
ExcelWorksheet ws = ef.Worksheets[0];
I won't go into details, but you can read lines text from a file with File.ReadAllLines.
Once you have those lines, you can split them into parts using String.Split (at least this will work if the CSV file contains very simple information as in your example).

Resources