I need to create an excel file for upload scenario in jmeter. The excel has 3 columns and number of rows is a dynamic value coming from parameter file.
The row values cannot have same data for different excel. So I am using random string to create data. By hard coding number of rows I am able to create file with below code using apache poi but facing issues to handle dynamic number of rows. Can somebody please provide solution?
Below is the code which is working fine for creating 5 rows.
def path = FileServer.getFileServer().getBaseDir;
def separator = File.separator;
def sourceFileName = "CreateDynamicExcel";
HSSFWorkbook workbook = new HSSFWorkbook();
HSSFSheet sheet = workbook.createSheet("Billing");
Object[] dataTypes = [
["Column1Header","Column2Header","Column3Header"],
["${__RandomString(10,abcdefghij,)}","${__Random(100000000,199999999,)}","${__RandomString(10,abcdefghijklmnopqrst,)}"],
["${__RandomString(10,abcdefghij,)}","${__Random(100000000,199999999,)}","${__RandomString(10,abcdefghijklmnopqrst,)}"],
["${__RandomString(10,abcdefghij,)}","${__Random(100000000,199999999,)}","${__RandomString(10,abcdefghijklmnopqrst,)}"],
["${__RandomString(10,abcdefghij,)}","${__Random(100000000,199999999,)}","${__RandomString(10,abcdefghijklmnopqrst,)}"]];
int rowNum = 0;
for (Object[] datatype:datatypes)
HSSFRow = sheet.createRow(rowNum++);
int colNum = 0;
for(Object filed:datatype){
HSSFCell cell = row.createCell(colNumn+=);
if(filed.instanceof(String){
cell.setCellValue((String) filed);
}
if(filed.instanceof(Integer){
cell.setCellValue((Integer) filed);
}
}
try{
FileOutputStream out = new FileOutputStream(new File(path+separator+sourceFileName+".xls"));
workbook.write(out);
out.close();
}
catch(FileNotFoundException e){
e.printStacktrace();
}
I don't think you should be inlining JMeter Functions or Variables in Groovy scripts because:
It conflicts with Groovy GString Template Engine syntax
Only first occurrence will be cached and used for subsequent iterations
So you can use the following expressions instead:
org.apache.commons.lang3.RandomStringUtils.randomAlphanumeric(10)
org.apache.commons.lang3.RandomUtils.nextInt(100000000, 199999999)
etc.
In case of any problems - take a look at jmeter.log file, in case of any issues you should find the root cause or at least a clue there
Related
I have the following code that I use to get the sheet names of an excel file(.xlsx)
XSSFWorkbook workBookXlsx = new XSSFWorkbook(new FileInputStream(pathToFile));
ArrayList<String> sheetNames = new ArrayList<>();
int numberOfSheets = workBookXlsx.getNumberOfSheets();
for (int i = 0; i < numberOfSheets; i++) {
sheetNames.add(workBookXlsx.getSheetAt(i).getSheetName());
}
workBookXlsx = null;
The issue I have with the above code is that it takes a lot of memory(~700MB) & a long time(5-6s) to create the XSSFWorkbook for a file of size 9MB. Even setting the workBookXlsx to null doesn't release the memory taken by the javaw(I know gc may or maynot be called & JVM wont release memory just because I have set a variable to null)
I did go through the documentation of Workbook, XSSFWorkbook & from what I understood, there is no method that will help me get the sheet names with low memory imprint.
The one solution I have found is to manually unzip the .xlsx file and read the contents of the .\xl\woorkbook.xml to get the sheet names and the r:id
Is there an API for getting the sheet names in an .xlsx file without large memory imprint?
To show what #Gagravarr probably meant with his comment:
The XSSFReader contains a method XSSFReader.getSheetsData which "Returns an Iterator which will let you get at all the different Sheets in turn. Each sheet's InputStream is only opened when fetched from the Iterator. It's up to you to close the InputStreams when done with each one.". But as often this is not the whole truth. In truth it returns a XSSFReader.SheetIterator which has a method XSSFReader.SheetIterator.getSheetName to get the sheet names.
Example:
import java.io.InputStream;
import java.io.FileInputStream;
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.xssf.eventusermodel.XSSFReader;
import java.util.Iterator;
public class ExcelXSSFGetSheetNamesXSSFReader {
public static void main(String[] args) throws Exception {
OPCPackage pkg = OPCPackage.open(new FileInputStream("Example.xlsx"));
XSSFReader r = new XSSFReader( pkg );
Iterator<InputStream> sheets = r.getSheetsData();
if (sheets instanceof XSSFReader.SheetIterator) {
XSSFReader.SheetIterator sheetiterator = (XSSFReader.SheetIterator)sheets;
while (sheetiterator.hasNext()) {
InputStream dummy = sheetiterator.next();
System.out.println(sheetiterator.getSheetName());
dummy.close();
}
}
pkg.close();
}
}
Conclusion: Currently you cannot work with apache poi only by trusting the API documentation. Instead you must always have a look at the source code.
im trying use for loop on this to loop from datasheet, but its only reading one row, dont know where i did wrong any Ideas?
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.usermodel.DataFormatter
cellDataFormatter = new DataFormatter()
//Create formula evaluator
fEval = new XSSFFormulaEvaluator(context.srcWkSheet.getWorkbook())
//Increment the rowcounter then read in the next row of items
RC = context.rowCounter;
if(RC<=context.srcWkSheet.getLastRowNum()){//Check if we've reached the last row
for(int i =0; i < RC; i++)
{
curTC = testRunner.testCase
sourceRow = context.srcWkSheet.getRow(i)//Get a spreadsheet row
//Step through cells in the row and populate property data
data1Cell = sourceRow.getCell(0)
curTC.setPropertyValue("data1",cellDataFormatter.formatCellValue(data1Cell ,fEval))
data2Cell = sourceRow.getCell(1)
curTC.setPropertyValue("data2",cellDataFormatter.formatCellValue(data2Cell ,fEval))
data3Cell = sourceRow.getCell(2)
curTC.setPropertyValue("data3",cellDataFormatter.formatCellValue(data3Cell ,fEval))
//Rename test cases for readability in the TestSuite log
curTC.getTestStepAt(0).setName("data1-" + curTC.getPropertyValue("BC"))
//Go back to first test request with newly copied properties
testRunner.gotoStep(0)
}
}
From the API documentation for testRunner.gotoStep(0):
Transfers execution of this TestRunner to the TestStep with the specified index in the TestCase
Execution will continue after the indexed step. You are probably expecting it will return back to your loop, which is incorrect!
You probably meant something like: curTC.getTestStepAt(0).run(context.testRunner, context); API documentation.
You could also have an issue with the excel file you are providing. XSSFFormulaEvaluator I believe is only for old style *.xls excel format. Could be an issue if you're feeding *.xlsx format excel file.
In SoapUI NG Pro there's a DataSource test step that simply allows you to point to a file (xls or xlsx) and feed in the data
http://www.soapui.org/data-driven-tests/functional-tests.html
When trying to get the Input the data from the Excel Sheet while working with OATS tool, it always gets into the catch block of the function. The below is the script written. Please help us resolve this issue.
public String getInputfromExcel(int argColumnNumber,int argRowNumber)throws Exception
{
String inputExcelName = dataPath+".xlsx";
String cellContent = "12";
try
{
Workbook workbook = Workbook.getWorkbook(new File(inputExcelName));
Sheet sheet = workbook.getSheet(0);
Cell a1 = sheet.getCell(argColumnNumber, argRowNumber);
cellContent = (a1.getContents()).toString();
System.out.println(cellContent.toString());
workbook.close();
}
catch (Exception e)
{
addReport("Getting Input From Excel", "Fail","Exception while reading value from excel sheet");
}
return cellContent;
}
Axel has brought up the point. On a further note, if I remember correctly, the function sheet.getCell(arg1, arg2) has first argument as rowNumber and 2nd as columnNumber (both the values are 0 based index).
Its old quetion....but just posting answer it might be helpful for someone needy.
In Oracle Application Testing Suite. NO NEED of external JARs to read/write data.
You can enable DataTable module in the tool
Complete explanation given here, http://www.testinghive.com/how-to-read-write-excel-in-oats/
//Define Sheet name to be read, and provide comma seperated to read multiple sheets
String sheetName = "Sheet1";
//Mention excel sheet path
String strFile= "C:\\Demo\\test.xls";
//Defined array list to add Sheets to read
List sheetList = new ArrayList();
sheetList.add(sheetName);
// Iports Sheet1
datatable.importSheets(strFile, sheetList, true, true);
//get rowcount
info("Total rows :"+datatable.getRowCount());
int rowcount=datatable.getRowCount();
//Loop to read all rows
for (int i=0;i<rowcount;i++)
{
//Set current row fromw here you need to start reading, in this case start from first row
datatable.setCurrentRow(sheetName, i);
String strCompany=(String) datatable.getValue(sheetName,i,"Company");
String strEmpFName=(String) datatable.getValue(sheetName,i,"FirstName");
String strEmpLName=(String) datatable.getValue(sheetName,i,"LastName");
String strEmpID=(String) datatable.getValue(sheetName,i,"EmpID");
String strLocation=(String) datatable.getValue(sheetName,i,"Location");
//prints first name and last name
System.out.println("First Name : "+strEmpFName+", Last Name : "+strEmpLName);
//Sets ACTIVE column in excel sheet to Y
String strActive="Y";
datatable.setValue(sheetName, i, datatable.getColumn(sheetName, datatable.getColumnIndex("Active")), strActive);
}
//Updates sheet with updated values ie ACTIVE column sets to Y
datatable.exportToExcel("C:\\Demo\\test1.xlsx");
I need to generate csv files and I stumbled on a module in our project itself which uses Apache POI to generate excel sheets aleady. So I thought I could use the same to generate csv. So I asked google brother, but he couldnt find anything for sure that says Apache POI can be used for CSV file generation. I was checking on the following api too and it only talks about xls sheets and not csv anywhere. Any ideas?
http://poi.apache.org/apidocs/org/apache/poi/ss/usermodel/Workbook.html
Apache Poi will not output to CSV for you. However, you have a couple good options, depending on what kind of data you are writing into the csv.
If you know that none of your cells will contain csv markers such as commas, quotes, line endings, then you can loop through your data rows and copy the text into a StringBuffer and send that to regular java IO.
Here is an example of writing an sql query to csv along those lines: Poi Mailing List: writing CSV
Otherwise, rather than figure out how to escape the special characters yourself, you should check out the opencsv project
If you check official web site Apache POI, you can find lots of example there. There is also an example that shows how you can have csv formatted output by using apache POI.
ToCSV example
Basic strategy:
1) Apache Commons CSV is the standard library for writing CSV values.
2) But we need to loop through the Workbook ourselves, and then call Commons CSV's Printer on each cell value, with a newline at the end of each row. Unfortunately this is custom code, it's not automatically available in XSSF. But it's easy:
// In this example we construct CSVPrinter on a File, can also do an OutputStream
Reader reader = Files.newBufferedReader(Paths.get(SAMPLE_CSV_FILE_PATH));
CSVPrinter csvPrinter = new CSVPrinter(reader, CSVFormat.DEFAULT);
if (workbook != null) {
XSSFSheet sheet = workbook.getSheetAt(0); // Sheet #0
Iterator<Row> rowIterator = sheet.rowIterator();
while (rowIterator.hasNext()) {
Row row = rowIterator.next();
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = cellIterator.next();
csvPrinter.print(cell.getStringCellValue()); // Call Commons CSV here to print
}
// Newline after each row
csvPrinter.println();
}
}
// at the end, close and flush CSVPrinter
csvPrinter.flush();
csvPrinter.close();
An improved and tested version of gene b's response is this:
/**
* Saves all rows from a single Excel sheet in a workbook to a CSV file.
*
* #param excelWorkbook path to the Excel workbook.
* #param sheetNumber sheet number to export.
* #param csvFile CSV file path for output.
* #throws IOException if failed to read the Excel file or create/write to a CSV file.
*/
public static void excelToCsv(String excelWorkbook, int sheetNumber, String csvFile) throws IOException {
try (Workbook workbook = WorkbookFactory.create(new File(excelWorkbook), null, true); // Read-only: true
BufferedWriter writer = new BufferedWriter(new FileWriter(csvFile));
CSVPrinter csvPrinter = new CSVPrinter(writer, CSVFormat.DEFAULT)) {
Sheet sheet = workbook.getSheetAt(sheetNumber);
DataFormatter format = new DataFormatter();
for (Row row : sheet) {
for (int c = 0; c < row.getLastCellNum(); c++) {
// Null cells returned as blank
Cell cell = row.getCell(c, Row.MissingCellPolicy.CREATE_NULL_AS_BLANK);
String cellValue = format.formatCellValue(cell);
csvPrinter.print(cellValue);
}
csvPrinter.println();
}
csvPrinter.flush();
}
}
The following improvements were made:
NullPointerException won't be thrown if a cell in an Excel Row was never edited. A blank value will be written to the CSV instead.
Excel values are rendered using DataFormatter allowing the CSV to match the visual representation of the Excel sheet.
try-with-source used for auto-close of the file objects.
The workbook is opened in the read-only mode.
I've been looking on the web for 30 minutes now and can't find any explanation about that. Here is my problem :
I wrote an application with poi to parse some data from 200 excel files or so and put some of it into a new file. I do some cell evaluation with FormulaEvaluator to know the content of the cells before choosing to keep them or not.
Now, when i test it on a test file with only values in the cells, the program works perfectly but when i use it on my pile of files I get this error :
"could not resolve external workbook name"
Is there any way to ignore external workbook references or set up the environment so that it wont evaluate formula with external references?
Because the ones I need don't contain references...
Thank you
Can you not just catch the error, and skip over that cell?
You're getting the error because you've asked POI to evaluate a the formula in a cell, and that formula refers to a different file. However, you've not told POI where to find the file that's referenced, so it objects.
If you don't care about cells with external references, just catch the exception and move on to the next cell.
If you do care, you'll need to tell POI where to find your files. You do this with the setupEnvironment(String[],Evaluator[]) method - pass it an array of workbook names, and a matching array of evaluators for those workbooks.
In order for POI to be able to evaluate external references, it needs access to the workbooks in question. As these don't necessarily have the same names on your system as in the workbook, you need to give POI a map of external references to open workbooks, through the setupReferencedWorkbooks(java.util.Map<java.lang.String,FormulaEvaluator> workbooks) method.
I have done please see below code that is working fine at my side
public static void writeWithExternalReference(String cellContent, boolean isRowUpdate, boolean isFormula)
{
try
{
File yourFile = new File("E:\\Book1.xlsx");
yourFile.createNewFile();
FileInputStream myxls = null;
myxls = new FileInputStream(yourFile);
XSSFWorkbook workbook = new XSSFWorkbook(myxls);
FormulaEvaluator mainWorkbookEvaluator = workbook.getCreationHelper().createFormulaEvaluator();
XSSFWorkbook workbook1 = new XSSFWorkbook(new File("E:\\elk\\lookup.xlsx"));
// Track the workbook references
Map<String,FormulaEvaluator> workbooks = new HashMap<String, FormulaEvaluator>();
workbooks.put("Book1.xlsx", mainWorkbookEvaluator);
workbooks.put("elk/lookup.xlsx", workbook1.getCreationHelper().createFormulaEvaluator());
workbook2.getCreationHelper().createFormulaEvaluator());
// Attach them
mainWorkbookEvaluator.setupReferencedWorkbooks(workbooks);
XSSFSheet worksheet = workbook.getSheetAt(0);
XSSFRow row = null;
if (isRowUpdate) {
int lastRow = worksheet.getLastRowNum();
row = worksheet.createRow(++lastRow);
}
else {
row = worksheet.getRow(worksheet.getLastRowNum());
}
if (!isFormula) {
Cell cell = row.createCell(row.getLastCellNum()==-1 ? 0 : row.getLastCellNum());
cell.setCellValue(Double.parseDouble(cellContent));
} else {
XSSFCell cell = row.createCell(row.getLastCellNum()==-1 ? 0 : row.getLastCellNum());
System.out.println(cellContent);
cell.setCellFormula(cellContent);
mainWorkbookEvaluator.evaluateInCell(cell);
cell.setCellFormula(cellContent);
// mainWorkbookEvaluator.evaluateInCell(cell);
//System.out.println(cell.getCellFormula() + " = "+cell.getStringCellValue());
}
workbook1.close();
myxls.close();
FileOutputStream output_file =new FileOutputStream(yourFile,false);
//write changes
workbook.write(output_file);
output_file.close();
} catch (Exception e) {
e.printStackTrace();
}
}