Find cells by their column name and change data type - apache-poi

I've a simple method to read csv and convert it to Excel:
public static void main(String[] args) throws Exception {
CSVReader csvReader = new CSVReader(new FileReader("P:\\employees.csv"));
SXSSFWorkbook sxssfWorkbook = new SXSSFWorkbook();
SXSSFSheet sxssfSheet = sxssfWorkbook.createSheet("Sheet");
String[] dataRow = null;
int rowNum = 0;
while ((dataRow = csvReader.readNext()) != null) {
Row currentRow = sxssfSheet.createRow(rowNum);
for (int i = 0; i < dataRow.length; i++) {
String cellValue = dataRow[i];
currentRow.createCell(i).setCellValue(cellValue);
}
rowNum++;
}
sxssfWorkbook.write(new FileOutputStream("P:\\employees.xlsx"));
}
But there's a problem with cell data type. All my data now represents as text. I want to find columns by their name (for example age, paid_total), not by index, and set numeric (float) data type for these columns. Something like this (sorry for sql-like style, for me it's a simplier to describe): WHEN columnName IN ('age', 'paid_total') SET allColumnType AS NUMERIC. How can I do this? Or it's only possible with indexes?

CSV files always are plain text files without data types. But if you exactly know which column should be which data type, then a type safe Excel sheet can be created. This can be achieved by column indes as well as by column header. To detect types by column header, those headers wolud must be into a separate data structure. But this will always be benefical.
Let's take the example employees.csv from here: https://gist.github.com/kevin336/acbb2271e66c10a5b73aacf82ca82784.
Then following should work:
import java.io.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.xssf.streaming.*;
import org.apache.poi.ss.SpreadsheetVersion;
import org.apache.poi.ss.util.AreaReference;
import org.apache.poi.ss.util.CellReference;
import com.opencsv.CSVReader;
import java.time.format.DateTimeFormatterBuilder;
import java.time.format.DateTimeFormatter;
import java.time.LocalDate;
class CreateExcelFromCSVDifferentDataTypes {
public static void main(String[] args) throws Exception {
try (
SXSSFWorkbook sxssfWorkbook = new SXSSFWorkbook(); FileOutputStream fileout = new FileOutputStream("./employees.xlsx");
CSVReader csvReader = new CSVReader(new FileReader("./employees.csv"));
) {
sxssfWorkbook.setCompressTempFiles(true);
CellStyle dateStyle = sxssfWorkbook.createCellStyle();
dateStyle.setDataFormat(sxssfWorkbook.getCreationHelper().createDataFormat().getFormat("dd-MMM-yy"));
SXSSFSheet sxssfSheet = sxssfWorkbook.createSheet("Sheet");
sxssfSheet.setRandomAccessWindowSize(100);
String[] strHeaders = null;
String[] dataRow = null;
int rowNum = 0;
while ((dataRow = csvReader.readNext()) != null) {
if (rowNum == 0) strHeaders = dataRow;
Row currentRow = sxssfSheet.createRow(rowNum);
for (int i = 0; i < dataRow.length; i++) {
String cellValue = dataRow[i];
if (rowNum > 0 && "HIRE_DATE".equals(strHeaders[i])) {
DateTimeFormatter formatter= new DateTimeFormatterBuilder().parseCaseInsensitive().appendPattern("dd-MMM-yy").toFormatter(java.util.Locale.ENGLISH);
LocalDate localDate = LocalDate.parse(cellValue, formatter);
currentRow.createCell(i).setCellValue(localDate);
currentRow.getCell(i).setCellStyle(dateStyle);
} else if (rowNum > 0 && "SALARY".equals(strHeaders[i])) {
double d = Double.valueOf(cellValue);
currentRow.createCell(i).setCellValue(d);
} else {
currentRow.createCell(i).setCellValue(cellValue);
}
}
rowNum++;
}
sxssfWorkbook.write(fileout);
sxssfWorkbook.dispose();
}
}
}

Related

Empty flowfile after trying to write content as Excel

I have following code to read .csv and transform it to Excel:
#Grab("org.apache.poi:poi:5.0.0")
#Grab("org.apache.poi:poi-ooxml:5.0.0")
#Grab("com.opencsv:opencsv:4.6")
import java.io.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.xssf.streaming.*;
import org.apache.poi.ss.SpreadsheetVersion;
import org.apache.poi.ss.util.AreaReference;
import org.apache.poi.ss.util.CellReference;
import com.opencsv.CSVReader;
def flowFile = session.get()
if(!flowFile)
return
flowFile = session.write(flowFile, { inputStream, outputStream ->
try {
SXSSFWorkbook sxssfWorkbook = new SXSSFWorkbook();
CSVReader csvReader = new CSVReader(new InputStreamReader(inputStream));
sxssfWorkbook.setCompressTempFiles(true);
SXSSFSheet sxssfSheet = sxssfWorkbook.createSheet("Sheet");
sxssfSheet.setRandomAccessWindowSize(100);
String[] strHeaders = null;
String[] dataRow = null;
int rowNum = 0;
while ((dataRow = csvReader.readNext()) != null) {
if (rowNum == 0) strHeaders = dataRow;
Row currentRow = sxssfSheet.createRow(rowNum);
for (int i = 0; i < dataRow.length; i++) {
String cellValue = dataRow[i];
currentRow.createCell(i).setCellValue(cellValue);
}
rowNum++;
}
int lastRow = rowNum -1;
int lastCol = strHeaders.length -1;
AreaReference areaReference = new AreaReference(new CellReference(0, 0), new CellReference(lastRow, lastCol), SpreadsheetVersion.EXCEL2007);
XSSFWorkbook xssfWorkbook = sxssfSheet.getWorkbook().getXSSFWorkbook();
XSSFSheet xssfSheet = xssfWorkbook.getSheet(sxssfSheet.getSheetName());
XSSFTable xssfTable = xssfSheet.createTable(areaReference);
//xssfTable.updateHeaders(); // this cannot work since xssfSheet does not contain any data until now
for (int i = 0; i < strHeaders.length; i++) {
String columnHeader = strHeaders[i];
if (xssfTable.getCTTable().getTableColumns().getTableColumnList().size() > i)
xssfTable.getCTTable().getTableColumns().getTableColumnList().get(i).setName(columnHeader);
}
xssfTable.getCTTable().addNewTableStyleInfo();
XSSFTableStyleInfo style = (XSSFTableStyleInfo)xssfTable.getStyle();
style.setName("TableStyleLight9");
style.setShowColumnStripes(false);
style.setShowRowStripes(true);
xssfTable.getCTTable().addNewAutoFilter().setRef(areaReference.formatAsString());
sxssfWorkbook.write(outputStream)
} catch (Exception ex) {
ex.printStackTrace();
}
} as StreamCallback)
session.transfer(flowFile, REL_SUCCESS)
But it returns empty flowfile after ExecuteGroovyScript.
Before I tested this in IntelliJ IDEA with Java. But reading from file and saving to file:
FileOutputStream fileout = new FileOutputStream("P:\\test.xlsx");
CSVReader csvReader = new CSVReader(new FileReader("P:\\test.csv"));
So, I guess something wrong with inputstream/outputstream?
P.S. To test some values from cell I decided to write some value to attributes:
...
sxssfWorkbook.write(outputStream);
flowFile.putAttribute('test', sxssfSheet.getRow(5).getCell(1).toString())
...
But flowfile doesn't contain this attribute.
What did I wrong?

Insert fields in File record using Groovy

I have below code in Groovy. Basically what I'm trying is to read the set of Input records and merge them into 1 or more records with common key combination.
The Key combination is as shown below. After reading the input file, I have written the key and fields into HashMap ( see code). But now I need to check the key in the input file , if the key is seen then I have write the output record otherwise I just need to write a output record as without merging. My questions
what is the command to insert a field in Output record ?.
import java.util.Properties;
import java.io.InputStream;
import java.util.HashMap;
import java.util.Map;
for( int i = 0; i < dataContext.getDataCount(); i++ ) {
InputStream is = dataContext.getStream(i);
Properties props = dataContext.getProperties(i);
reader = new BufferedReader(new InputStreamReader(is));
/* This is how to declare HashMap */
def forcastMap = [:]
String Key;
String Shipfrom = "";
String Item = "";
String Fcast = "";
String Shipto = "";
String Planned_Arrival_Date = "";
String Qty = "";
String PrevKey = "";
List<String> line = null
while ((line = reader.readLine()) != null)
{
if(line.length() > 20) //Make sure it is a data line so we can do substring manipulation
{
Shipfrom = line.substring(35,12)
Item = line.substring(50,50)
Fcast = line.substring(10,50)
Shipto = line.substring(75,10)
Planned_Arrival_Date = line.substring(85,8)
Qty = line.substring(90,12)
Key = (Shipfrom + Item + Fcast + Shipto)
forcastMap.put(Key,Planned_Arrival_Date,Qty)
if key != PrevKey {
}
}
}
//dataContext.storeStream(is, props);
}

Selenium Webdriver How to select records from table by fetching Excel Input

im struggeling for below scenario.
Application displayed records of 100 suppliers in one table have three columns namely as ID,Company name and Subscription name.
i want to take input from my excel sheet say company name"xyz" and using that input i have to click on subscription name details link so application will navigates me next page.
Sample code i have created as below:
`public static void main(String[] args) throws BiffException, IOException, Exception {
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
//Workbook location
Workbook wBook = Workbook.getWorkbook(new File("C:\Users\amit.bhagwat\Documents\TestData\SampleData.xls"));
//get sheet
jxl.Sheet Sheet = wBook.getSheet(0);
//loop
for(int i=1; i<Sheet.getRows(); i++)
{
driver.get("http://206.132.42.243/Web");
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
driver.findElement(By.xpath("//input[#id='UserName']")).sendKeys(Sheet.getCell(0, i).getContents());
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
driver.findElement(By.xpath("//input[#id='Password']")).sendKeys(Sheet.getCell(1, i).getContents());
driver.findElement(By.xpath("//input[#id='Password']")).sendKeys(Sheet.getCell(1, i).getContents());
Thread.sleep(40);
driver.findElement(By.xpath("//input[#name='Login']")).click();
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
driver.findElement(By.xpath("//a[contains(text(),'Task')]")).click();
driver.manage().timeouts().implicitlyWait(30,TimeUnit.SECONDS);
driver.findElement(By.xpath("//a[contains(text(),'Data Checking')]")).click();
jxl.Sheet Sheet2 = wBook.getSheet(0);
WebElement kancheck = driver.findElement(By.name("Grant & Brown"));
kancheck.click();
System.out.println(kancheck.isSelected());
driver.findElement(By.xpath("//a[contains(text(),'Data Checking')]")).sendKeys(Sheet2.getCell(1, i).getContents());
Thread.sleep(40);` enter code here
As far as I could understand, you are trying to read the file from a remote location and then read the information from it. It would be a good practice if you can use Apache POI library to read contents at run-time.
In my project, I read all the contents from an excel sheet usingApache POI library to set the values of my variables. Here is a code snippet on how i achieved it. Hopefully this will guide you to a proper solution. :)
public void readExcelDoc() throws FileNotFoundException, IOException
{
POIFSFileSystem fs = new POIFSFileSystem(new FileInputStream("excelDoc//scripts.xls"));
HSSFWorkbook wb = new HSSFWorkbook(fs);
HSSFSheet sheet = wb.getSheetAt(0);
HSSFRow row = null;
HSSFCell cell = null;
int rows = 0; // No of rows
// rows = sheet.getPhysicalNumberOfRows();
rows = sheet.getLastRowNum();
int cols = 2; // No of columns
int tmp = 0;
// This trick ensures that we get the data properly even if it doesn't start from first few rows
for(int i = 0; i < 10 || i < rows; i++) {
row = sheet.getRow(i);
if(row != null) {
tmp = sheet.getRow(i).getPhysicalNumberOfCells();
if(tmp > cols) cols = tmp;
}
}
int testRowNo = 0;
String rowName = "Test Name";
String columnValue = " ";
//Iterate through Row and columns here. Excluding 1st row for title names
for(int r = 1; r <= rows; r++) {
row = sheet.getRow(r);
if(row != null) {
//Browse through columns using c
for(int c = 0; c < cols; c++) {
if(c==0) //Only taking data from Cell 0; Ignoring any other inputs
{
cell = row.getCell((short)c);
try
{
if(cell.getStringCellValue().contains(rowName))
{
testRowNo =row.getRowNum();
}
if(testRowNo > 0 )
{
if(cell.getColumnIndex() == 0 && row.getRowNum() > testRowNo && cell.getStringCellValue().length() !=0)
{
try{
String cellValue = cell.getStringCellValue().toLowerCase();
//System.out.println(cellValue);
scriptType.add(cellValue);
}
catch(IllegalStateException e)
{
e.printStackTrace();
scriptType.add(cell.getStringCellValue());
}
}
}
}
catch(NullPointerException e)
{
}
}
if(c==1)
{
cell = row.getCell((short)c); //this sets the column number
if(testRowNo == 0)
{
try{
String cellValue = cell.getStringCellValue();
//System.out.println(cellValue);
columnValue = cellValue;
}
catch(IllegalStateException e)
{
String cellValue = cell.toString();
columnValue = cellValue;
}
catch(NullPointerException e)
{
String cellValue = nodata;
columnValue = cellValue;
}
}
}
if(c==2)
{
cell = row.getCell((short)c); //this sets the column number
if(testRowNo == 0)
{
try{
String cellValue = cell.getStringCellValue();
//System.out.println(cellValue);
inputParameters.put(cellValue, columnValue);
}
catch(IllegalStateException e)
{
String cellValue = cell.toString();
inputParameters.put(cellValue, columnValue);
}
catch(NullPointerException e)
{
String cellValue = nodata;
inputParameters.put(cellValue, columnValue);
}
}
}
}
}
}
System.out.println("---------The parameters set from excel are : ---------");
#SuppressWarnings("rawtypes")
Iterator iterator = inputParameters.keySet().iterator();
while (iterator.hasNext()) {
String key = iterator.next().toString();
String value = inputParameters.get(key).toString();
System.out.println(key + " : " + value);
}
}

Selenium IDE: read set of data from excel

This is actually not a question but solution proposal: I found workaround to read set of data from excel. in this case there is no need for multiple users or data variation but read parameters to create a validation environment.
ok, solution is to save excel file to html format and then let the Selenium IDE to read parameters from that. Users needs only to agree the same filename to be used.
1) you should add "Apache POI" jar files in order to read your excel through java.
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import org.apache.poi.xssf.usermodel.XSSFCell;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
public class ExcelReadExample {
#SuppressWarnings({ "unchecked", "rawtypes" })
public static void main(String[] args) throws Exception {
String filename = "E:\\data.xls";
List sheetData = new ArrayList();
FileInputStream fis = null;
try {
fis = new FileInputStream(filename);
XSSFWorkbook workbook = new XSSFWorkbook(fis);
XSSFSheet sheet = workbook.getSheetAt(0);
Iterator rows = sheet.rowIterator();
while (rows.hasNext()) {
XSSFRow row = (XSSFRow) rows.next();
Iterator cells = row.cellIterator();
List data = new ArrayList();
while (cells.hasNext()) {
XSSFCell cell = (XSSFCell) cells.next();
data.add(cell);
}
sheetData.add(data);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fis != null) {
fis.close();
}
}
showExelData(sheetData);
}
private static void showExelData(List sheetData) {
int sum = 0;
for (int i = 0; i < sheetData.size(); i++) {
List<XSSFCell> list = (List) sheetData.get(i);
for (int j = 0; j < list.size(); j++) {
XSSFCell cell = (XSSFCell) list.get(j);
if(cell.getCellType()==0)
{
sum += cell.getNumericCellValue();
}
}
System.out.println("");
System.out.println("Sum Value is:" +sum);
}
}
}
Change the file path.
i hve mentioned my sheet name as "input" change it as per yours
Happy excelling :D

How to Export DataTable into Excel

I have list of objects and i converted those into data table now i am unable to export those into excel
Below is the sample code
class Program
{
static void Main(string[] args)
{
Student s1 = new Student("Student-A",100);
Student s2 = new Student("Student-B", 90);
Student s3 = new Student("Student-C", 80);
List<Student> studentList = new List<Student>() { s1,s2,s3};
ListToDataTable converter = new ListToDataTable();
DataTable dt = converter.ToDataTable(studentList);
Console.WriteLine();
}
}
Below is the student class which has two properties
class Student
{
public string Name { get; set; }
public int? Score { get; set; }
public Student(string name,int? score)
{
this.Name = name;
this.Score = score;
}
}
Below is the class used for converting list of objects to data table
public class ListToDataTable
{
public DataTable ToDataTable<T>(List<T> items)
{
DataTable dataTable = new DataTable(typeof(T).Name);
PropertyInfo[] Props = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo prop in Props)
{
dataTable.Columns.Add(prop.Name);
}
foreach (T item in items)
{
var values = new object[Props.Length];
for (int i = 0; i < Props.Length; i++)
{
values[i] = Props[i].GetValue(item, null);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
}
Try to create a simple CSV file from your DataTable.
You can use the following DataTable extension, after you have converted your list to a DataTable.
public static string ToCSV(this DataTable table)
{
var result = new StringBuilder();
for (int i = 0; i < table.Columns.Count; i++)
{
result.Append(table.Columns[i].ColumnName);
result.Append(i == table.Columns.Count - 1 ? "\n" : ",");
}
foreach (DataRow row in table.Rows)
{
for (int i = 0; i < table.Columns.Count; i++)
{
result.Append(row[i].ToString());
result.Append(i == table.Columns.Count - 1 ? "\n" : ",");
}
}
return result.ToString();
}
Example of usage :
// replace with your data table here
DataTable dt = new DataTable();
var bytes = Encoding.GetEncoding("iso-8859-1").GetBytes(dt.ToCSV());
MemoryStream stream = new MemoryStream(bytes);
StreamReader reader = new StreamReader(stream);
Response.Clear();
Response.Buffer = true;
Response.AddHeader("content-disposition", string.Format("attachment;filename={0}.csv", "filename"));
Response.ContentType = "application/text";
Response.ContentEncoding = Encoding.Unicode;
Response.Output.Write(reader.ReadToEnd());
Response.Flush();
Response.End();
Since you have this tagged as interop, I went that route (no need to create a csv file, just export directly to excel).
This solution is not the prettiest, but it works. I've also changed it some as you can export your studentList directly to excel (no need to convert it to a dataTable first).
First thing, in your solution, you need to add a reference to "Microsoft.Office.Interop.Excel". To do this, right click on "References" in Solution Explorer, then "Add Reference", then click on the ".NET" tab, then scroll down to find it.
Once that is done, update your code as follows:
using System;
using System.Collections.Generic;
using Excel = Microsoft.Office.Interop.Excel;
static void Main()
{
var s1 = new Student("Student-A", 100);
var s2 = new Student("Student-B", 90);
var s3 = new Student("Student-C", 80);
var studentList = new List<Student> { s1, s2, s3 };
// Create an excel sheet
var xlApp = new Excel.Application { Visible = true }; // Create instance of Excel and make it visible.
xlApp.Workbooks.Add(Excel.XlSheetType.xlWorksheet); // Create a workbook (WB)
var xlWS = (Excel.Worksheet)xlApp.ActiveSheet; // Reference the active worksheet (WS)
xlWS.Name = "Exported Student"; // Name the worksheet
// Add Header fields to Excel [row, column]
var r = 1;
xlWS.Cells[r, 1] = "Name";
xlWS.Cells[r, 2] = "Score";
// Copy data from StudentList to Excel
foreach (Student student in studentList)
{
r++;
xlWS.Cells[r, 1] = student.Name;
xlWS.Cells[r, 2] = student.Score;
}
}
This will automatically export your studentList to an excel sheet. There wasn't a need for the ListToDataTable class.

Resources