Can't seem to output a good file with ClosedXML, when I open the file I'm getting
We found a problem with some content in 'filename.xlsx' Do you want us to recover as much as we can? If you trust the source of this workbook, click Yes.
using (var workbook = new XLWorkbook())
{
var worksheet = workbook.AddWorksheet("name");
worksheet.Row(1).Cell(i + 1).SetValue("test");
worksheet.Row(k + 2).Cell(column.Order + 1).SetValue("test value");
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
memoryStream.Seek(0, SeekOrigin.Begin);
return memoryStream.GetBuffer();
}
}
It was either using ToArray() instead of GetBuffer() that fixed the problem. Cheers
using (var workbook = new XLWorkbook())
{
var worksheet = workbook.AddWorksheet("name");
worksheet.Row(1).Cell(i + 1).SetValue("test");
worksheet.Row(k + 2).Cell(column.Order + 1).SetValue("test value");
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
memoryStream.Position = 0;
return memoryStream.ToArray();
}
}
Related
I have an ASP.NET Core app, with a model, the aim is to allow user to upload an excel file and then save the file to the model/table. I have the below method
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<IActionResult> Upload(IFormFile file)
{
string webRootPath = _hostEnvironment.WebRootPath;
var uploads = Path.Combine(webRootPath, "Upload");
var files = HttpContext.Request.Form.Files;
var extension = Path.GetExtension(files[0].FileName);
using (var filesStream = new FileStream(Path.Combine(uploads, file.FileName), FileMode.Create))
{
files[0].CopyTo(filesStream);
}
var list = new List<User>();
using (var stream = new MemoryStream())
{
await file.CopyToAsync(stream);
using (var package = new ExcelPackage(stream))
{
ExcelWorksheet worksheet = package.Workbook.Worksheets[0];
var rowcount = worksheet.Dimension.Rows;
for (int row = 2; row <= rowcount; row++)
{
list.Add(new User
{
Name = worksheet.Cells[row, 1]?.Value?.ToString().Trim(),
Address1 = worksheet.Cells[row, 2]?.Value?.ToString().Trim(),
PostCode = worksheet.Cells[row, 3]?.Value?.ToString().Trim(),
Mobile = worksheet.Cells[row, 4]?.Value?.ToString().Trim(),
});
}
}
}
foreach (var user in list)
{
_db.User.AddAsyncy(user);
}
_db.SaveChangesAsyncy();
return View();
}
This code works fine by processing an excel file uploaded by a user but the problem I'm having is that when the file is large say above 3 mb, it takes well over 8 minutes to upload.
Any idea how to speed this up please? Thanks.
There are two things you can do to increase speed.
1)Instead of reading excel file with ExcelWorksheet class go with a library called ExcelDataReader which can read around 600k records under a minute.
sample code
Model
class Person
{
public int id,
public string name
}
//and excel file has both columns in model ,the we can read with below code
using ExcelDataReader;
System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
var fileName = "./Person.xlsx";
var timer = new Stopwatch();
timer.Start();
int counter=0;
List<Person> persons = new List<Person>();
using (var stream = System.IO.File.Open(fileName, FileMode.Open, FileAccess.Read))
{
using (var reader = ExcelReaderFactory.CreateReader(stream))
{
while (reader.Read()) //Each row of the file
{
var person = new Person
{
id = reader.GetValue(0).ToString(),
name = reader.GetValue(1).ToString()
}
persons.Add(person)
counter++;
}
timer.Stop();
duration = timer.ElapsedMilliseconds / 1000;
//to check performace print duration and persons list
}
}
https://github.com/ExcelDataReader/ExcelDataReader
2)Once you read and store data in a list, you can store that data in DataTable class and insert into database using Oracle.ManagedDataAccess.Client Nuget package instead of EFcore. This method is fast. Please go through below link for doing this with Oracle database.
https://www.c-sharpcorner.com/article/two-ways-to-insert-bulk-data-into-oracle-database-using-c-sharp/
var db_timer = new Stopwatch();
db_timer.Start();
DataTable dt = new DataTable();
dt.Columns.Add("id");
dt.Columns.Add("name");
for (int i = 0; i < counter; i++)
{
DataRow dr = dt.NewRow();
dr["id"] = persons[i].id;
dr["name"] = persons[i].name;
dt.Rows.Add(dr);
}
using (var connection = new OracleConnection(oracleConString))
{
connection.Open();
int[] ids = new int[dt.Rows.Count];
string[] names = new string[dt.Rows.Count];
for (int j = 0; j < dt.Rows.Count; j++)
{
ids[j] = Convert.ToString(dt.Rows[j]["id"]);
names[j] = Convert.ToString(dt.Rows[j]["name"]);
}
OracleParameter id = new OracleParameter();
id.OracleDbType = OracleDbType.Int32;
id.Value = ids;
OracleParameter name = new OracleParameter();
name.OracleDbType = OracleDbType.Varchar2;
name.Value = names;
OracleCommand cmd = connection.CreateCommand();
cmd.CommandText = "INSERT INTO TEST(id,name) VALUES (:1,:2)";
cmd.ArrayBindCount = ids.Length;
cmd.Parameters.Add(id);
cmd.Parameters.Add(name);
cmd.ExecuteNonQuery();
}
just sample code you can user timer to check how much time it is taking to execute.
I have the below code to save the data in excel sheet as .xls
public ActionResult ExportToExcel()
{
DataTable tbl = CopyGenericToDataTable(res);
tbl.TableName = "InvalidInvoices";
using (XLWorkbook wb = new XLWorkbook())
{
wb.Worksheets.Add(tbl);
wb.Style.Alignment.Horizontal = XLAlignmentHorizontalValues.Center;
wb.Style.Font.Bold = true;
Response.Clear();
Response.Buffer = true;
Response.Charset = "";
Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
Response.AddHeader("content-disposition", "attachment;filename= "+fileName + ".xls");
using (MemoryStream MyMemoryStream = new MemoryStream())
{
wb.SaveAs(MyMemoryStream);
MyMemoryStream.WriteTo(Response.OutputStream);
Response.Flush();
Response.End();
}
}
}
Above is code which download the xls excel sheet at client side. It works fine the data gets saved in excel sheet.
Problem is if I try to upload this same file using below code -
if (files != null)
{
HttpPostedFileBase upload = files.FirstOrDefault();
Stream stream = upload.InputStream;
DataSet result = new DataSet();
if (upload != null && upload.ContentLength > 0)
{
if (upload.FileName.EndsWith(".xls") || upload.FileName.EndsWith(".xlsx"))
{
// ExcelDataReader works with the binary Excel file, so it needs a FileStream
// to get started. This is how we avoid dependencies on ACE or Interop:
// We return the interface, so that
IExcelDataReader reader = null;
if (upload.FileName.EndsWith(".xls"))
{
reader = ExcelReaderFactory.CreateBinaryReader(stream);
}
else if (upload.FileName.EndsWith(".xlsx"))
{
reader = ExcelReaderFactory.CreateOpenXmlReader(stream);
}
reader.IsFirstRowAsColumnNames = false;
result = reader.AsDataSet();
reader.Close();
}
}
}
In above code I am getting error in ExcelReaderFactory.CreateBinaryReader(stream);
In stream it has the values in bytes too just on using createBinaryreader of excelreaderfactory reader has error message as 'Invalid file signature'.
Any help will be highly appreciated.
ClosedXML generates .xlsx files, not .xls files.
Check your code:
Response.AddHeader("content-disposition", "attachment;filename= "+fileName + ".xls");
I am trying to load the XLSX file using POI library that has 5 sheets. Size of the file is 5 MB. Total records in all sheets are around 30,000.
Once the file is loaded i need to delete the 1 or more sheets on the fly based on sheet neame as input.
Here is the snippet.
public void generateReportWorkBook(String[] requestedReports) throws Exception {
// Read the file
String dailyTicketReport = ReportConstants.REPORT_PATH + ReportConstants.FILE_NAME + ReportConstants.XLSX_FILE_EXTN;
FileInputStream fis = null;
XSSFWorkbook book = null;
try {
fis = new FileInputStream(dailyTicketReport);
book = new XSSFWorkbook(fis);
for (int i = book.getNumberOfSheets() - 1; i >= 0; i--) {
XSSFSheet tmpSheet = book.getSheetAt(i);
if (!ArrayUtils.contains(requestedReports, tmpSheet.getSheetName())) {
book.removeSheetAt(i);
}
}
} catch (Exception e) {
logger.error("Error occured while removing the sheets from workbook");
throw e;
} finally {
IOUtils.closeQuietly(fis);
}
}
When i execute the program. I get OutofMemory Exception.
How can i remove the sheets without memory issue.
I too faced the same issue of OOM while parsing xlsx file...after two days of struggle, I finally found out the below code that was really perfect;
This code is based on sjxlsx. It reads the xlsx and stores in a HSSF sheet.
// read the xlsx file
SimpleXLSXWorkbook = new SimpleXLSXWorkbook(new File("C:/test.xlsx"));
HSSFWorkbook hsfWorkbook = new HSSFWorkbook();
org.apache.poi.ss.usermodel.Sheet hsfSheet = hsfWorkbook.createSheet();
Sheet sheetToRead = workbook.getSheet(0, false);
SheetRowReader reader = sheetToRead.newReader();
Cell[] row;
int rowPos = 0;
while ((row = reader.readRow()) != null) {
org.apache.poi.ss.usermodel.Row hfsRow = hsfSheet.createRow(rowPos);
int cellPos = 0;
for (Cell cell : row) {
if(cell != null){
org.apache.poi.ss.usermodel.Cell hfsCell = hfsRow.createCell(cellPos);
hfsCell.setCellType(org.apache.poi.ss.usermodel.Cell.CELL_TYPE_STRING);
hfsCell.setCellValue(cell.getValue());
}
cellPos++;
}
rowPos++;
}
return hsfSheet;
I use Apache POI and I have a lot of data that I want to write to *.xls file in parts.
Now I use:
FileOutputStream fileOutputStream = null;
File tmpFile = new File("blabla.xls");
Workbook workbook = null;
try {
for(int i = 1; i <= pageNumber; i++) {
workbook = xlsGenerator.generateWorkbook(someData, i);
fileOutputStream = new FileOutputStream(tmpFile);
workbook.write(fileOutputStream);
}
}
But it doesn't work. It always replace old data instead of appending new data to workbook. So, are there ways to write in parts?
using System;
namespace jagged_array
{
class Program
{
static void Main(string[] args)
{
string[][] Members = new string[10][]{
new string[]{"amit","amit#gmail.com", "9999999999"},
new string[]{"chandu","chandu#gmail.com","8888888888"},
new string[]{"naveen","naveen#gmail.com", "7777777777"},
new string[]{"ramu","ramu#gmail.com", "6666666666"},
new string[]{"durga","durga#gmail.com", "5555555555"},
new string[]{"sagar","sagar#gmail.com", "4444444444"},
new string[]{"yadav","yadav#gmail.com", "3333333333"},
new string[]{"suraj","suraj#gmail.com", "2222222222"},
new string[]{"niharika","niharika#gmail.com","11111111111"},
new string[]{"anusha","anusha#gmail.com", "0000000000"},
};
for (int i =0; i < Members.Length; i++)
{
System.Console.Write("Name List ({0}):", i + 1);
for (int j = 0; j < Members[i].Length; j++)
{
System.Console.Write(Members[i][j] + "\t");
}
System.Console.WriteLine();
}``
Console.ReadKey();
}
}
}
The above is the code for my C# console program in which i used jagged array and i assigned values manually but now my requirement is 'without assigning manually into array i want the same details to import into my program from an csv file(which is at some location in my disc). So how to do it what functions should i make use , please help me with some example. Thank you.
static void Main()
{
string csv_file_path=#"C:\Users\Administrator\Desktop\test.csv";
DataTable csvData = GetDataTabletFromCSVFile(csv_file_path);
Console.WriteLine("Rows count:" + csvData.Rows.Count);
Console.ReadLine();
}
private static DataTable GetDataTabletFromCSVFile(string csv_file_path)
{
DataTable csvData = new DataTable();
try
{
using(TextFieldParser csvReader = new TextFieldParser(csv_file_path))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
while (!csvReader.EndOfData)
{
string[] fieldData = csvReader.ReadFields();
//Making empty value as null
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
}
csvData.Rows.Add(fieldData);
}
}
}
catch (Exception ex)
{
}
return csvData;
}
Treat the CSV file like an excel workbook and you will find a lot of examples on the web for what you need to do.
ExcelFile ef = new ExcelFile();
// Loads file.
ef.LoadCsv("filename.csv");
// Selects first worksheet.
ExcelWorksheet ws = ef.Worksheets[0];
I won't go into details, but you can read lines text from a file with File.ReadAllLines.
Once you have those lines, you can split them into parts using String.Split (at least this will work if the CSV file contains very simple information as in your example).