Timeout when generate excel for download - apache-poi

I am trying to export Excel using poi and servlet. code like this
SXSSFWorkbook workbook = genrateExcel(id)
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
workbook.write(outputStream);
byte[] outArray = outputStream.toByteArray();
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
response.setContentLength(outArray.length);
response.setHeader("Expires:", "0");
response.setHeader("Content-Disposition", "attachment; filename=" + URLEncoder.encode(excelFileName, "UTF-8"));
OutputStream outStream = response.getOutputStream();
outStream.write(outArray);
outStream.flush();
workbook.dispose();
workbook.close();
But sometimes generating costs too much time,so i get a 504 timeout error.
I wonder how can I hold the connection until file is generated ?
UPDATE:
thanks Gagravarr. I update the code to this
ExecutorService threadPool = Executors.newSingleThreadExecutor();
Future<SXSSFWorkbook> future = threadPool.submit(new Callable<SXSSFWorkbook>() {
#Override
public SXSSFWorkbook call() {
return genrateExcel(id);
}
});
SXSSFWorkbook workbook = null;
try {
response.setHeader("Expires:", "0");
response.setHeader("Content-Disposition", "attachment; filename=" + URLEncoder.encode(excelFileName, "UTF-8"));
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
OutputStream outStream = response.getOutputStream();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
int times = 0;
while (times < 50) {
if (future.isDone()) {
workbook = future.get();
break;
}
outStream.flush();
Thread.sleep(5000);
times++;
}
workbook = workbook == null ? new SXSSFWorkbook() : workbook;
workbook.write(outputStream);
byte[] outArray = outputStream.toByteArray();
response.setContentLength(outArray.length);
outStream.write(outArray);
outStream.flush();
workbook.dispose();
workbook.close();
} catch (Exception e) {
e.printStackTrace();
}
But got a ClientAbortException: java.io.IOException meanwhile Chrome report the page isnot working ERR_INCOMPLETE_CHUNKED_ENCODING is sth wrong?

Related

Returning excel file using spring boot controller

I was trying to make a rest endpoint in Spring Boot which reads from DB, generates an excel file(Using Apache POI) which is returned to the user using HttpServletResponse but when I invoke this, the excel is getting created but it's not downloading. I had some other code earlier in place which was working fine but I accidentally removed that and now I'm stuck. Any help/leads are appreciated.
#RequestMapping(path = "/save", method = RequestMethod.GET)
public ResponseEntity<String> saveToXls(#RequestParam String id, #RequestParam String appName, HttpServletResponse response) {
AppInstance appInstance = appInstanceRepo.get(id);
List<DownloadDetail> downloadDetailList = downloadDAO.searchByInstanceId(id);
//List<DownloadDetail> downloadDetailList = appInstance.getDownloads();
System.out.print("LIST SIZE:" + downloadDetailList.size());
String fileName = appName + " report";
File myFile = new File(fileName + ".xls");
FileOutputStream fileOut;
downloadDetailList.forEach(downloadDetail -> System.out.print(downloadDetail.getSid()));
try {
try (HSSFWorkbook workbook = new HSSFWorkbook()) {
HSSFSheet sheet = workbook.createSheet("lawix10");
HSSFRow rowhead = sheet.createRow((short) 0);
rowhead.createCell((short) 0).setCellValue("SID");
rowhead.createCell((short) 1).setCellValue("Download Time");
rowhead.createCell((short) 2).setCellValue("OS Version");
int i = 0;
for (DownloadDetail downloadDetail : downloadDetailList) {
System.out.print("In loop -2");
HSSFRow row = sheet.createRow((short) i);
row.createCell((short) 0).setCellValue(downloadDetail.getSid());
row.createCell((short) 1).setCellValue(downloadDetail.getDownloadTime());
row.createCell((short) 2).setCellValue(downloadDetail.getOsVersion());
i++;
}
fileOut = new FileOutputStream(myFile);
workbook.write(fileOut);
}
fileOut.close();
byte[] buffer = new byte[10240];
response.addHeader("Content-disposition", "attachment; filename=test.xls");
response.setContentType("application/vnd.ms-excel");
try (
InputStream input = new FileInputStream(myFile);
OutputStream output = response.getOutputStream();
) {
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
}
response.flushBuffer();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
return null;
}
EDIT:
I tried to do it another way as shown below:
try (InputStream is = new FileInputStream(myFile)) {
response.addHeader("Content-disposition", "attachment; filename=test.xls");
response.setContentType("application/vnd.ms-excel");
IOUtils.copy(is, response.getOutputStream());
}
response.flushBuffer();
This also doesn't seem to cut it.
This is a my example. Probably the issue is how you manage the OutputStream:
ServletOutputStream os = response.getOutputStream();
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment; filename=\""+fileName+".xls\"");
workbook = excelStrategyMap.get(strategy).export(idList, status, params);
workbook.write(os);
workbook.close();
os.flush();
response.flushBuffer();
Once you get the workbook file, set the file name and file type. and add the response header and content type as mentioned below.
Then write the file to the response and flush it's buffer.
XSSFWorkbook file = excelUploadService.downloadDocument();
String filename = "Export.xlsx";
String filetype = "xlsx";
response.addHeader("Content-disposition", "attachment;filename=" + filename);
response.setContentType(filetype);
// Copy the stream to the response's output stream.
file.write(response.getOutputStream());
response.flushBuffer();
In the client side, get the response from the REST API and set the content type received by the response object. Using FileSaver library save the file into your local file system.
Here is the documentation for FileSaver js -- File saver JS Library
var type = response.headers("Content-Type");
var blob = new Blob([response.data], {type: type});
saveAs(blob, 'Export Data'+ '.xlsx');
#GetMapping(value = "/", produces = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
#ResponseBody
public byte[] generateExcel() {
byte[] res = statisticsService.generateExcel();
return res;

spring mvc export excel failed sometimes and data displays garblend on the page

There is a data report as the following in my project, I want to export it as excel:
But the wierd thing is that sometimes it exports successfully and sometimes failed. I have tried many times and it turns out that when the excel sheet is over 17 lines, it will lead to this:
the page redirects to a new page with garblend data.
the post request has been changed to get request.
It's sure that the excel is successfully created even if it exports failed, because I have written it to disk and checked the file. The following is the controller, what goes wrong with me?
#RequestMapping("/download")
public void download(HttpServletRequest request, HttpServletResponse response, ReportCondition condition){
try {
List<HashMap<String, String>> mapList = reportFormService.find(condition);
if(mapList == null || mapList.size() == 0){
logger.info("No reports...");
}
Map<String, Date> dateMap = DateConditionUtil.getStartEndDate1(condition.getStartDate(), condition.getEndDate());
String startDate = DateFormatUtils.format(dateMap.get("startDate"), "yyyyMMdd");
String endDate = DateFormatUtils.format(dateMap.get("endDate"), "yyyyMMdd");
String[] titleArr = new String[]{"序号","日期","应用系统","短信服务商","请求发送总数","请求成功数量","实际短信条数","实际成功条数","费用","失败数量","成功率"};
String[] fieldArr = new String[]{"ID","RECORDDATE","APPNAME","PROVIDERNAME","SENDCOUNT","SUCCESSCOUNT","SENDSUM","SUCCESSSUM","TOTALFEE","FAILURECOUNT","SUCCESSRATE"};
ByteArrayOutputStream os = new ByteArrayOutputStream();
WorkBookUtil.createExcel(titleArr, fieldArr, mapList, os);
try(BufferedInputStream bis = new BufferedInputStream(new ByteArrayInputStream(os.toByteArray()));
BufferedOutputStream bos = new BufferedOutputStream(response.getOutputStream())){
byte[] buff = new byte[1024];
int bytes;
while (-1 != (bytes = bis.read(buff, 0, buff.length))) {
bos.write(buff, 0, bytes);
}
bos.flush();
}catch (Exception e){
logger.error(e);
}
response.setCharacterEncoding("utf-8");
response.setHeader("content-disposition", "attachment;filename=" + String.format("%s-%s.xls", startDate, endDate));
response.setContentType("application/vnd.ms-excel;charset=utf-8");
} catch (IOException e){
logger.error("Export failed", e);
}
}
You should set the response headers first before you create the .xls file. So try this:
response.setCharacterEncoding("utf-8");
response.setHeader("content-disposition", "attachment;filename=" + String.format("%s-%s.xls", startDate, endDate));
response.setContentType("application/vnd.ms-excel;charset=utf-8");
ByteArrayOutputStream os = new ByteArrayOutputStream();
WorkBookUtil.createExcel(titleArr, fieldArr, mapList, os);
// ...
Try adding your response headers before writing the bytes out to the user. It would help to add a content-length header as well. You can get that after your call to createExcel with os.toByteArray().length. In addition close() your bos after you flush() it.

why downloading to file is not working in jsf? [duplicate]

This question already has answers here:
How to provide a file download from a JSF backing bean?
(5 answers)
Closed 5 years ago.
i made a call to download() method to save json into xml with extension ".svg". The jsondata is global variable store json.
public void download(){
File file = exportFile(jsondata);
HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse();
writeOutContent(response, file, file.getName());
FacesContext.getCurrentInstance().responseComplete();
FacesContext.getCurrentInstance().renderResponse();
}
and the exportFile(jsondata) is
public File exportFile(String jsonData){
File xmlFile = null;
try {
DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
Document doc = docBuilder.newDocument();
JSONObject jsonObject = new JSONObject(jsonData);
Element root = doc.createElement("web");
doc.appendChild(root);
Element rootElement1 = doc.createElement("class");
rootElement1.appendChild(doc.createTextNode(jsonObject.getString("class")));
root.appendChild(rootElement1);
JSONArray jsonArray1 = (JSONArray) jsonObject.get("nodes");
Element rootElement2 = doc.createElement("nodes");
root.appendChild(rootElement2);
for (int i = 0; i < jsonArray1.length(); i++) {
Element staff = doc.createElement("node");
rootElement2.appendChild(staff);
JSONObject childObject = (JSONObject) jsonArray1.get(i);
Iterator<String> keyItr = childObject.keys();
while (keyItr.hasNext()) {
String key = keyItr.next();
Element property = doc.createElement(key);
property.appendChild(doc.createTextNode(childObject.getString(key)));
staff.appendChild(property);
}
}
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
//for pretty print
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
DOMSource source = new DOMSource(doc);
xmlFile = new File("file.svg");
//write to console or file
// StreamResult console = new StreamResult(System.out);
StreamResult file = new StreamResult(xmlFile);
//write data
// transformer.transform(source, console);
transformer.transform(source, file);
} catch (Exception pce) {
pce.printStackTrace();
}
return xmlFile;
}
finally to write this one file writeOutContent()
public void writeOutContent(final HttpServletResponse res, final File content, final String theFilename) {
if (content == null) {
System.out.println("content is null");
return;
}
try {
res.setHeader("Content-Disposition", "attachment; filename=\"" + theFilename + "\"");
System.out.println("res " + res.getHeader("attachment; filename=\"" + theFilename + "\""));
res.setContentType("application/octet-stream");
FileInputStream fis = new FileInputStream(content);
OutputStream os = res.getOutputStream();
int bt = fis.read();
while (bt != -1) {
os.write(bt);
bt = fis.read();
}
os.flush();
fis.close();
os.close();
} catch (Exception ex) {
Logger.getLogger(DownloadFile.class.getName()).log(Level.SEVERE, null, ex);
}
}
i can see the xml in console but what am doing wrong that its not downloading?? please help me.
thanks in advance.
i got the mistake. it was not in above code. if we make through commandLink then it won't work but if make call through commandButton then it worked. if you want know know more read difference between commandButton vs commandLink

Downloading a file using JSF

Hi I am trying to download a file from server but at the end of my process I end up with only numbers and some weird characters on my browser. It's not downloading the file. I am using seam and JSF 1.2.
Here is my code:
public void writeBytesToResponse(UploadDefinition _instance, String path) {
FacesContext context = FacesContext.getCurrentInstance();
HttpServletResponse response = (HttpServletResponse) context .getExternalContext().getResponse();
try {
byte[] bytes = getFile(path);
response.reset();
response.setContentType(ContentType.PDF.getLabel());
response.setContentLength(bytes.length);
response.setHeader("Content-disposition", "attachment; filename=\"" + _instance.getFileName() + "\"");
OutputStream outputStream = response.getOutputStream();
outputStream.write(bytes);
outputStream.flush();
outputStream.close();
context.responseComplete();
} catch (Exception ex) {
ex.printStackTrace();
}
}
#SuppressWarnings("resource")
public byte[] getFile(String filePath) throws FileNotFoundException, IOException {
File file = new File(filePath);
InputStream is = new FileInputStream(file);
long length = file.length();
if (length > Integer.MAX_VALUE) {
throw new IOException("File is too large " + file.getName());
}
byte[] bytes = new byte[(int) length];
int offset = 0;
int numRead = 0;
while (offset < bytes.length && (numRead = is.read(bytes, offset, bytes.length - offset)) >= 0) {
offset += numRead;
}
if (offset < bytes.length) {
throw new IOException("Could not completely read file " + file.getName());
}
is.close();
return bytes;
}
Here I call the method:
public void downloadFile() {
writeBytesToResponse(ud, path);
}
I got the answer, I was using <a4j:commandButton> on jsf and I changed it to <h:commandButton> then it worked. The point is not to use ajax.

How to reduce memory while we Zip a big excel file in java

I am trying to zip a 30Mb excel file.
Wrapping th fileinputstream and fileoutputstream reduced some of the memory consumption.
Please suggest if you have a better option than this approach.
Do you think, flushing the ZipOutputStream after zos.write() inside while loop will be of any advantage?
Following is the code which i use.
public void zipBigFile()
throws IOException, ParseException {
String outFileName= "D:/Out/Big.zip";
File file = new File(outFileName);
FileOutputStream out = new FileOutputStream(file);
ZipOutputStream zos = new ZipOutputStream(new BufferedOutputStream(out));
byte[] buffer = new byte[1024 * 8];
String inFileName = "D:/In/Big_File.xlsx";
FileInputStream fis = new FileInputStream(inFileName);
BufferedInputStream bis = new BufferedInputStream(fis);
ZipEntry zipEntry = new ZipEntry("Big_File_Zipped.xlsx");
zos.putNextEntry(zipEntry);
int length;
while ((length = bis.read(buffer)) > 0) {
zos.write(buffer, 0, length);
}
zos.closeEntry();
fis.close();
fis = null;
buffer = null;
zos.flush();
zos.close();
out.close();
out = null;
zos = null;
}

Resources