I am trying to zip a 30Mb excel file.
Wrapping th fileinputstream and fileoutputstream reduced some of the memory consumption.
Please suggest if you have a better option than this approach.
Do you think, flushing the ZipOutputStream after zos.write() inside while loop will be of any advantage?
Following is the code which i use.
public void zipBigFile()
throws IOException, ParseException {
String outFileName= "D:/Out/Big.zip";
File file = new File(outFileName);
FileOutputStream out = new FileOutputStream(file);
ZipOutputStream zos = new ZipOutputStream(new BufferedOutputStream(out));
byte[] buffer = new byte[1024 * 8];
String inFileName = "D:/In/Big_File.xlsx";
FileInputStream fis = new FileInputStream(inFileName);
BufferedInputStream bis = new BufferedInputStream(fis);
ZipEntry zipEntry = new ZipEntry("Big_File_Zipped.xlsx");
zos.putNextEntry(zipEntry);
int length;
while ((length = bis.read(buffer)) > 0) {
zos.write(buffer, 0, length);
}
zos.closeEntry();
fis.close();
fis = null;
buffer = null;
zos.flush();
zos.close();
out.close();
out = null;
zos = null;
}
Related
I was trying to make a rest endpoint in Spring Boot which reads from DB, generates an excel file(Using Apache POI) which is returned to the user using HttpServletResponse but when I invoke this, the excel is getting created but it's not downloading. I had some other code earlier in place which was working fine but I accidentally removed that and now I'm stuck. Any help/leads are appreciated.
#RequestMapping(path = "/save", method = RequestMethod.GET)
public ResponseEntity<String> saveToXls(#RequestParam String id, #RequestParam String appName, HttpServletResponse response) {
AppInstance appInstance = appInstanceRepo.get(id);
List<DownloadDetail> downloadDetailList = downloadDAO.searchByInstanceId(id);
//List<DownloadDetail> downloadDetailList = appInstance.getDownloads();
System.out.print("LIST SIZE:" + downloadDetailList.size());
String fileName = appName + " report";
File myFile = new File(fileName + ".xls");
FileOutputStream fileOut;
downloadDetailList.forEach(downloadDetail -> System.out.print(downloadDetail.getSid()));
try {
try (HSSFWorkbook workbook = new HSSFWorkbook()) {
HSSFSheet sheet = workbook.createSheet("lawix10");
HSSFRow rowhead = sheet.createRow((short) 0);
rowhead.createCell((short) 0).setCellValue("SID");
rowhead.createCell((short) 1).setCellValue("Download Time");
rowhead.createCell((short) 2).setCellValue("OS Version");
int i = 0;
for (DownloadDetail downloadDetail : downloadDetailList) {
System.out.print("In loop -2");
HSSFRow row = sheet.createRow((short) i);
row.createCell((short) 0).setCellValue(downloadDetail.getSid());
row.createCell((short) 1).setCellValue(downloadDetail.getDownloadTime());
row.createCell((short) 2).setCellValue(downloadDetail.getOsVersion());
i++;
}
fileOut = new FileOutputStream(myFile);
workbook.write(fileOut);
}
fileOut.close();
byte[] buffer = new byte[10240];
response.addHeader("Content-disposition", "attachment; filename=test.xls");
response.setContentType("application/vnd.ms-excel");
try (
InputStream input = new FileInputStream(myFile);
OutputStream output = response.getOutputStream();
) {
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
}
response.flushBuffer();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
return null;
}
EDIT:
I tried to do it another way as shown below:
try (InputStream is = new FileInputStream(myFile)) {
response.addHeader("Content-disposition", "attachment; filename=test.xls");
response.setContentType("application/vnd.ms-excel");
IOUtils.copy(is, response.getOutputStream());
}
response.flushBuffer();
This also doesn't seem to cut it.
This is a my example. Probably the issue is how you manage the OutputStream:
ServletOutputStream os = response.getOutputStream();
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment; filename=\""+fileName+".xls\"");
workbook = excelStrategyMap.get(strategy).export(idList, status, params);
workbook.write(os);
workbook.close();
os.flush();
response.flushBuffer();
Once you get the workbook file, set the file name and file type. and add the response header and content type as mentioned below.
Then write the file to the response and flush it's buffer.
XSSFWorkbook file = excelUploadService.downloadDocument();
String filename = "Export.xlsx";
String filetype = "xlsx";
response.addHeader("Content-disposition", "attachment;filename=" + filename);
response.setContentType(filetype);
// Copy the stream to the response's output stream.
file.write(response.getOutputStream());
response.flushBuffer();
In the client side, get the response from the REST API and set the content type received by the response object. Using FileSaver library save the file into your local file system.
Here is the documentation for FileSaver js -- File saver JS Library
var type = response.headers("Content-Type");
var blob = new Blob([response.data], {type: type});
saveAs(blob, 'Export Data'+ '.xlsx');
#GetMapping(value = "/", produces = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
#ResponseBody
public byte[] generateExcel() {
byte[] res = statisticsService.generateExcel();
return res;
I am trying to export Excel using poi and servlet. code like this
SXSSFWorkbook workbook = genrateExcel(id)
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
workbook.write(outputStream);
byte[] outArray = outputStream.toByteArray();
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
response.setContentLength(outArray.length);
response.setHeader("Expires:", "0");
response.setHeader("Content-Disposition", "attachment; filename=" + URLEncoder.encode(excelFileName, "UTF-8"));
OutputStream outStream = response.getOutputStream();
outStream.write(outArray);
outStream.flush();
workbook.dispose();
workbook.close();
But sometimes generating costs too much time,so i get a 504 timeout error.
I wonder how can I hold the connection until file is generated ?
UPDATE:
thanks Gagravarr. I update the code to this
ExecutorService threadPool = Executors.newSingleThreadExecutor();
Future<SXSSFWorkbook> future = threadPool.submit(new Callable<SXSSFWorkbook>() {
#Override
public SXSSFWorkbook call() {
return genrateExcel(id);
}
});
SXSSFWorkbook workbook = null;
try {
response.setHeader("Expires:", "0");
response.setHeader("Content-Disposition", "attachment; filename=" + URLEncoder.encode(excelFileName, "UTF-8"));
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
OutputStream outStream = response.getOutputStream();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
int times = 0;
while (times < 50) {
if (future.isDone()) {
workbook = future.get();
break;
}
outStream.flush();
Thread.sleep(5000);
times++;
}
workbook = workbook == null ? new SXSSFWorkbook() : workbook;
workbook.write(outputStream);
byte[] outArray = outputStream.toByteArray();
response.setContentLength(outArray.length);
outStream.write(outArray);
outStream.flush();
workbook.dispose();
workbook.close();
} catch (Exception e) {
e.printStackTrace();
}
But got a ClientAbortException: java.io.IOException meanwhile Chrome report the page isnot working ERR_INCOMPLETE_CHUNKED_ENCODING is sth wrong?
I have a Word document, where I have to replace text and save this document in Oracle database, in PDF format. Have someone any idea how to resolve this?
String inputFilename = "/root/GeneratorUmow/web/WEB-INF/umowy/kkb/wniosekozaciag.docx";
try(InputStream is = new FileInputStream(inputFilename)) {
dbd = new DokumentyBlobDAO();
db.setNazwaPliku("Wniosek_o_zaciagniecie.pdf");
db.setTypDokuemntu(876990);
if(du.getTypKlienta() == S_ID_Lead_Partner_Typ_Klienta_KKB) {
POIFSFileSystem fs = null;
fs = new POIFSFileSystem(is);
HWPFDocument doc = new HWPFDocument(fs); //funkcja zamyka inputstream
Range range = doc.getRange();
range.replaceText("my&&nazwa&&firmy", "KAKAOWY SZATAN");
//konwersja na pdf
FileOutputStream fos = new FileOutputStream(pathToDirectory + File.separator + folderName+File.separator + "wniosekzaciagniecie.docx");
doc.write(fos);
String inputChangeFile = pathToDirectory+ File.separator +folderName+File.separator+"wniosekzaciagniecie.docx";
InputStream inputstream = new FileInputStream(inputFilename);
length = inputChangeFile.length();
System.out.println("dlugosc lancucha " + length);
XWPFDocument document = new XWPFDocument(inputstream);
System.out.println("message3");
PdfOptions options = PdfOptions.create();
PdfOptions options = PdfOptions.create().fontEncoding("windows-1250");
System.out.println("message4");
OutputStream out = new FileOutputStream(new File(pathToDirectory+ File.separator +folderName+File.separator+"kakaowyszal.pdf"));
PdfConverter.getInstance().convert(document, out, options);
System.out.println("message5");
}
}
catch( Exception ex){
System.out.println("err" + ex);
return -1;
}
i am having an issue when trying to read saved List from binary file back to a List.
the file is encrypted, without encryption i had no problem.
writing method:
private void WriteEncodedFile(FileStream fileStream, MemoryStream memoryStream)
{
StreamReader sr = new StreamReader(memoryStream);
BinaryWriter bw = new BinaryWriter(fileStream);
memoryStream.Seek(0, SeekOrigin.Begin);
string data = sr.ReadToEnd();
var bytes = Encoding.UTF8.GetBytes(data);
for (int i = 0; i < bytes.Length; i++) bytes[i] ^= 0x5a;
bw.Write(Convert.ToBase64String(bytes));
bw.Close();
fileStream.Close();
}
Reading method:
private void ReadEncodedFile(FileStream fileStream, MemoryStream memoryStream)
{
BinaryReader br = new BinaryReader(fileStream);
StreamWriter sw = new StreamWriter(memoryStream);
fileStream.Seek(0, SeekOrigin.Begin);
memoryStream.Seek(0, SeekOrigin.Begin);
string base64 = br.ReadString();
byte[] bytes = Convert.FromBase64String(base64);
for (int i = 0; i < bytes.Length; i++) bytes[i] ^= 0x5a;
string decodedData = Encoding.UTF8.GetString(bytes);
sw.Write(decodedData);
}
when reading the content i can see it in the "decodedData".
however the StreamWriter seems not to write it into the MemoryStream.
any idea?
thanks.
I think you are simply not flushing/closing the stream before checking.
sw.Flush();
or
sw.Close();
if done
and
fileStream.Close(); for good practice too
Edit:
Considering the comment, you need to seek to start of memory stream again before deserialising
i.e. memoryStream.Seek(0, SeekOrigin.Begin); where-ever you do it
I have created an Encryption method to encrypt the data into test.txt Text file. The encryption is Working fine and its Reading the encryted text file into textBox1.
I m using 2 buttons and 2 textBox for doing this.
Now I want to Read the decryption text from test.txt in textBox2
but when i click on the button to read data from test.txt to textBox2.
I am getting an exception like (CryptographicException was unhandled) Bad Data.
Here is the code I used for Dycryption.
private void button2_Click(object sender, EventArgs e)
{
FileStream stream = new FileStream("C:\\test.txt",
FileMode.Open, FileAccess.Read);
DESCryptoServiceProvider cryptic = new DESCryptoServiceProvider();
cryptic.Key = ASCIIEncoding.ASCII.GetBytes("ABCDEFGH");
cryptic.IV = ASCIIEncoding.ASCII.GetBytes("ABCDEFGH");
CryptoStream crStream = new CryptoStream(stream,
cryptic.CreateDecryptor(), CryptoStreamMode.Read);
StreamReader reader = new StreamReader(crStream);
//string data = reader.ReadToEnd();
textBox2.Text = reader.ReadLine();
//stream.Close();
}
Codes that i used for Encrytion is here:
private void button1_Click(object sender, EventArgs e)
{
FileStream stream = new FileStream("C:\\test.txt", FileMode.OpenOrCreate, FileAccess.Write);
DESCryptoServiceProvider cryptic = new DESCryptoServiceProvider();
cryptic.Key = ASCIIEncoding.ASCII.GetBytes("ABCDEFGH");
cryptic.IV = ASCIIEncoding.ASCII.GetBytes("ABCDEFGH");
CryptoStream crStream = new CryptoStream(stream,
cryptic.CreateEncryptor(), CryptoStreamMode.Write);
byte[] data = ASCIIEncoding.ASCII.GetBytes(textBox2.Text);
crStream.Write(data, 0, data.Length);
crStream.Close();
stream.Close();
string text = System.IO.File.ReadAllText(#"C://test.txt");
textBox1.Text = text.ToString();
}
I don't know why you want to read it from test.txt file. when your displaying it in textbox. but check below code you can able to encrypt and decry-pt and meanwhile able to display and save in text boxes.
protected void Button1_Click(object sender, EventArgs e)
{
FileStream stream = new FileStream("C:\\test.txt", FileMode.OpenOrCreate, FileAccess.Write);
string datastring= Encrypt("ABCDEFGH", "ABCDEFGH");
byte[] data = ASCIIEncoding.ASCII.GetBytes(datastring);
stream.Write(data, 0, data.Length);
stream.Close();
string text = System.IO.File.ReadAllText(#"C:\\test.txt");
TextBox1.Text = text.ToString();
}
protected void Button2_Click(object sender, EventArgs e)
{
StreamReader reader =null;
try
{
FileStream stream = new FileStream("C:\\test.txt",
FileMode.Open, FileAccess.Read);
string fileContents;
using ( reader = new StreamReader(stream))
{
fileContents = reader.ReadToEnd();
}
string data = Decrypt(fileContents, "ABCDEFGH");
TextBox2.Text = data;
stream.WriteByte(Convert.ToByte(data));
stream.Close();
}
catch (Exception ex)
{
}
finally
{
reader.Close();
}
}
public static string Encrypt(string message, string password)
{
// Encode message and password
byte[] messageBytes = ASCIIEncoding.ASCII.GetBytes(message);
byte[] passwordBytes = ASCIIEncoding.ASCII.GetBytes(password);
// Set encryption settings -- Use password for both key and init. vector
DESCryptoServiceProvider provider = new DESCryptoServiceProvider();
ICryptoTransform transform = provider.CreateEncryptor(passwordBytes, passwordBytes);
CryptoStreamMode mode = CryptoStreamMode.Write;
// Set up streams and encrypt
MemoryStream memStream = new MemoryStream();
CryptoStream cryptoStream = new CryptoStream(memStream, transform, mode);
cryptoStream.Write(messageBytes, 0, messageBytes.Length);
cryptoStream.FlushFinalBlock();
// Read the encrypted message from the memory stream
byte[] encryptedMessageBytes = new byte[memStream.Length];
memStream.Position = 0;
memStream.Read(encryptedMessageBytes, 0, encryptedMessageBytes.Length);
// Encode the encrypted message as base64 string
string encryptedMessage = Convert.ToBase64String(encryptedMessageBytes);
return encryptedMessage;
}
public static string Decrypt(string encryptedMessage, string password)
{
// Convert encrypted message and password to bytes
byte[] encryptedMessageBytes = Convert.FromBase64String(encryptedMessage);
byte[] passwordBytes = ASCIIEncoding.ASCII.GetBytes(password);
// Set encryption settings -- Use password for both key and init. vector
DESCryptoServiceProvider provider = new DESCryptoServiceProvider();
ICryptoTransform transform = provider.CreateDecryptor(passwordBytes, passwordBytes);
CryptoStreamMode mode = CryptoStreamMode.Write;
// Set up streams and decrypt
MemoryStream memStream = new MemoryStream();
CryptoStream cryptoStream = new CryptoStream(memStream, transform, mode);
cryptoStream.Write(encryptedMessageBytes, 0, encryptedMessageBytes.Length);
cryptoStream.FlushFinalBlock();
// Read decrypted message from memory stream
byte[] decryptedMessageBytes = new byte[memStream.Length];
memStream.Position = 0;
memStream.Read(decryptedMessageBytes, 0, decryptedMessageBytes.Length);
// Encode deencrypted binary data to base64 string
string message = ASCIIEncoding.ASCII.GetString(decryptedMessageBytes);
return message;
}