MVC Linq throws intermittent null reference exception during export to Excel - asp.net-mvc-5

I'm using the same code in 6 Areas, each a separate database. Been working fine for a long time, now fails in 3 of the 6 with a null reference exception. Classes all the same, all database tables hold data. First method is the view which displays the data, second called by a link in the view to export the data to Excel. 3 cases display and export correctly. In 2 out of the other 3 cases the data is displayed correctly in the view, and the null exception is thrown at the point the data should be being retrieved in the export method. In the last case the null exception is thrown when trying to return the view. Simply can't understand how it works in some instances but not in others.
public ActionResult PVS(string studySite, string currentFilter, int? page)
{
ViewBag.SelectedID = studySite;
if (studySite != null)
page = 1;
else
studySite = currentFilter;
ViewBag.CurrentFilter = studySite;
IQueryable<PVS> schedule = db.PVS.OrderBy(v => v.SiteNumber).ThenBy(v => v.Participant);
if (studySite != null)
{
string selectedID = studySite;
images = schedule.Where(v => v.SiteNumber == selectedID);
}
int pageSize = 10;
int pageNumber = (page ?? 1);
return View(schedule.ToPagedList(pageNumber, pageSize));
}
public void ExportPVSToExcel()
{
var grid = new System.Web.UI.WebControls.GridView();
var pvs = db.PVS.OrderBy(p => p.SiteNumber).ThenBy(p => p.Participant).ToList();
grid.DataSource = pvs;
grid.DataBind();
// create an Excel worksheet for the data export
ExcelPackage excel = new ExcelPackage();
var workSheet = excel.Workbook.Worksheets.Add("PVS");
var totalCols = grid.Rows[0].Cells.Count;
var totalRows = grid.Rows.Count;
var headerRow = grid.HeaderRow;
for (var i = 1; i <= totalCols; i++)
{
workSheet.Cells[1, i].Value = headerRow.Cells[i - 1].Text;
}
for (var j = 1; j <= totalRows; j++)
{
for (var i = 1; i <= totalCols; i++)
{
var data = pvs.ElementAt(j - 1);
workSheet.Cells[j + 1, i].Value = data.GetType().GetProperty(headerRow.Cells[i - 1].Text).GetValue(data, null);
}
}
using (var memoryStream = new MemoryStream())
{
Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
Response.AddHeader("content-disposition", "attachment; filename=PVS.xlsx");
excel.SaveAs(memoryStream);
memoryStream.WriteTo(Response.OutputStream);
Response.Flush();
Response.End();
}
}

Related

How to speed up Excel file upload EPPlus

I have an ASP.NET Core app, with a model, the aim is to allow user to upload an excel file and then save the file to the model/table. I have the below method
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<IActionResult> Upload(IFormFile file)
{
string webRootPath = _hostEnvironment.WebRootPath;
var uploads = Path.Combine(webRootPath, "Upload");
var files = HttpContext.Request.Form.Files;
var extension = Path.GetExtension(files[0].FileName);
using (var filesStream = new FileStream(Path.Combine(uploads, file.FileName), FileMode.Create))
{
files[0].CopyTo(filesStream);
}
var list = new List<User>();
using (var stream = new MemoryStream())
{
await file.CopyToAsync(stream);
using (var package = new ExcelPackage(stream))
{
ExcelWorksheet worksheet = package.Workbook.Worksheets[0];
var rowcount = worksheet.Dimension.Rows;
for (int row = 2; row <= rowcount; row++)
{
list.Add(new User
{
Name = worksheet.Cells[row, 1]?.Value?.ToString().Trim(),
Address1 = worksheet.Cells[row, 2]?.Value?.ToString().Trim(),
PostCode = worksheet.Cells[row, 3]?.Value?.ToString().Trim(),
Mobile = worksheet.Cells[row, 4]?.Value?.ToString().Trim(),
});
}
}
}
foreach (var user in list)
{
_db.User.AddAsyncy(user);
}
_db.SaveChangesAsyncy();
return View();
}
This code works fine by processing an excel file uploaded by a user but the problem I'm having is that when the file is large say above 3 mb, it takes well over 8 minutes to upload.
Any idea how to speed this up please? Thanks.
There are two things you can do to increase speed.
1)Instead of reading excel file with ExcelWorksheet class go with a library called ExcelDataReader which can read around 600k records under a minute.
sample code
Model
class Person
{
public int id,
public string name
}
//and excel file has both columns in model ,the we can read with below code
using ExcelDataReader;
System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
var fileName = "./Person.xlsx";
var timer = new Stopwatch();
timer.Start();
int counter=0;
List<Person> persons = new List<Person>();
using (var stream = System.IO.File.Open(fileName, FileMode.Open, FileAccess.Read))
{
using (var reader = ExcelReaderFactory.CreateReader(stream))
{
while (reader.Read()) //Each row of the file
{
var person = new Person
{
id = reader.GetValue(0).ToString(),
name = reader.GetValue(1).ToString()
}
persons.Add(person)
counter++;
}
timer.Stop();
duration = timer.ElapsedMilliseconds / 1000;
//to check performace print duration and persons list
}
}
https://github.com/ExcelDataReader/ExcelDataReader
2)Once you read and store data in a list, you can store that data in DataTable class and insert into database using Oracle.ManagedDataAccess.Client Nuget package instead of EFcore. This method is fast. Please go through below link for doing this with Oracle database.
https://www.c-sharpcorner.com/article/two-ways-to-insert-bulk-data-into-oracle-database-using-c-sharp/
var db_timer = new Stopwatch();
db_timer.Start();
DataTable dt = new DataTable();
dt.Columns.Add("id");
dt.Columns.Add("name");
for (int i = 0; i < counter; i++)
{
DataRow dr = dt.NewRow();
dr["id"] = persons[i].id;
dr["name"] = persons[i].name;
dt.Rows.Add(dr);
}
using (var connection = new OracleConnection(oracleConString))
{
connection.Open();
int[] ids = new int[dt.Rows.Count];
string[] names = new string[dt.Rows.Count];
for (int j = 0; j < dt.Rows.Count; j++)
{
ids[j] = Convert.ToString(dt.Rows[j]["id"]);
names[j] = Convert.ToString(dt.Rows[j]["name"]);
}
OracleParameter id = new OracleParameter();
id.OracleDbType = OracleDbType.Int32;
id.Value = ids;
OracleParameter name = new OracleParameter();
name.OracleDbType = OracleDbType.Varchar2;
name.Value = names;
OracleCommand cmd = connection.CreateCommand();
cmd.CommandText = "INSERT INTO TEST(id,name) VALUES (:1,:2)";
cmd.ArrayBindCount = ids.Length;
cmd.Parameters.Add(id);
cmd.Parameters.Add(name);
cmd.ExecuteNonQuery();
}
just sample code you can user timer to check how much time it is taking to execute.

Using Epplus to import data from an Excel file to SQL Server database table

I've tried implementing thishttps://www.paragon-inc.com/resources/blogs-posts/easy_excel_interaction_pt6 on an ASP.NET MVC 5 Application.
//SEE CODE BELOW
[HttpPost]
public ActionResult Upload(HttpPostedFileBase file)
{
var regPIN = DB.AspNetUsers.Where(i => i.Id == user.Id).Select(i => i.registrationPIN).FirstOrDefault();
if (file != null && file.ContentLength > 0)
{
var extension = Path.GetExtension(file.FileName);
var excelFile = Path.Combine(Server.MapPath("~/App_Data/BulkImports"),regPIN + extension);
if (System.IO.File.Exists(excelFile))
{
System.IO.File.Delete(excelFile);
}
else if (file.ContentType == "application/vnd.ms-excel" || file.ContentType == "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
{
file.SaveAs(excelFile);//WORKS FINE
//BEGINING OF IMPORT
FileInfo eFile = new FileInfo(excelFile);
using (var excelPackage = new ExcelPackage(eFile))
{
if (!eFile.Name.EndsWith("xlsx"))//Return ModelState.AddModelError()
{ ModelState.AddModelError("", "Incompartible Excel Document. Please use MSExcel 2007 and Above!"); }
else
{
var worksheet = excelPackage.Workbook.Worksheets[1];
if (worksheet == null) { ModelState.AddModelError("", "Wrong Excel Format!"); }// return ImportResults.WrongFormat;
else
{
var lastRow = worksheet.Dimension.End.Row;
while (lastRow >= 1)
{
var range = worksheet.Cells[lastRow, 1, lastRow, 3];
if (range.Any(c => c.Value != null))
{ break; }
lastRow--;
}
using (var db = new BlackBox_FinaleEntities())// var db = new BlackBox_FinaleEntities())
{
for (var row = 2; row <= lastRow; row++)
{
var newPerson = new personalDetails
{
identificationType = worksheet.Cells[row, 1].Value.ToString(),
idNumber = worksheet.Cells[row, 2].Value.ToString(),
idSerial = worksheet.Cells[row, 3].Value.ToString(),
fullName = worksheet.Cells[row, 4].Value.ToString(),
dob = DateTime.Parse(worksheet.Cells[row, 5].Value.ToString()),
gender = worksheet.Cells[row, 6].Value.ToString()
};
DB.personalDetails.Add(newPerson);
try { db.SaveChanges(); }
catch (Exception) { }
}
}
}
}
}//END OF IMPORT
ViewBag.Message = "Your file was successfully uploaded.";
return RedirectToAction("Index");
}
ViewBag.Message = "Error: Your file was not uploaded. Ensure you upload an excel workbook file.";
return View();
}
else
{
ViewBag.Message = "Error: Your file was not uploaded. Ensure you upload an excel workbook file.";
return View();
}
}
See Picture Error
Any help would be greatly appreciated mates.
you can do like this:
public bool readXLS(string FilePath)
{
FileInfo existingFile = new FileInfo(FilePath);
using (ExcelPackage package = new ExcelPackage(existingFile))
{
//get the first worksheet in the workbook
ExcelWorksheet worksheet = package.Workbook.Worksheets[1];
int colCount = worksheet.Dimension.End.Column; //get Column Count
int rowCount = worksheet.Dimension.End.Row; //get row count
string queryString = "INSERT INTO tableName VALUES"; //Here I am using "blind insert". You can specify the column names Blient inset is strongly not recommanded
string eachVal = "";
bool status;
for (int row = 1; row <= rowCount; row++)
{
queryString += "(";
for (int col = 1; col <= colCount; col++)
{
eachVal = worksheet.Cells[row, col].Value.ToString().Trim();
queryString += "'" + eachVal + "',";
}
queryString = queryString.Remove(queryString.Length - 1, 1); //removing last comma (,) from the string
if (row % 1000 == 0) //On every 1000 query will execute, as maximum of 1000 will be executed at a time.
{
queryString += ")";
status = this.runQuery(queryString); //executing query
if (status == false)
return status;
queryString = "INSERT INTO tableName VALUES";
}
else
{
queryString += "),";
}
}
queryString = queryString.Remove(queryString.Length - 1, 1); //removing last comma (,) from the string
status = this.runQuery(queryString); //executing query
return status;
}
}
Details: http://sforsuresh.in/read-data-excel-sheet-insert-database-table-c/

Generating a HTML table from an Excel file using EPPlus?

I would like to generate a HTML table from an excel file. The EPPlus package provides a .net API for manipulating excel file. I would be happy to know whether it is possible to generate a HTML table code from an Excel file using EPPlus? I couldn't find anything on the documentation, but intuition tells me that there should be a way to do it
Thank you!
If you are looking for something built into EPPlus, I havent seen anything that will export directly HTML.
Best thing would be to bring in the Excel file to EPPlus and extract the data to a collection or DataTable. That should be pretty straight forward.
From there, there is plenty of documentation on how to get that to an html table. Quick search turned up this as the first hit:
Datatable to html Table
I wrote some code, you can try this.
static void Main(string[] args)
{
ExcelPackage p = new ExcelPackage(new System.IO.FileInfo("X.XLSX"));
var sheet = p.Workbook.Worksheets["HMTD"];
var noOfCol = sheet.Dimension.End.Column;
var noOfRow = sheet.Dimension.End.Row;
StringBuilder s = new StringBuilder();
s.Append("<table>");
for (int i = 1; i < noOfRow; i++)
{
s.Append("<tr>");
for (int j = 1; j < noOfCol; j++)
{
int colspan = 1;
int rowspan = 1;
if (!sheet.Cells[i, j].Merge || (sheet.Cells[i, j].Merge && isFirstMergeRange(sheet, sheet.Cells[i, j].Address, ref colspan, ref rowspan)))
{
s.Append("<td rowspan='" + rowspan + "' colspan='" + colspan + "'>");
if(sheet.Cells[i,j] != null && sheet.Cells[i,j].Value != null)
s.Append(sheet.Cells[i,j].Value.ToString());
s.Append("</td>");
}
}
s.Append("</tr>");
}
s.Append("</table>");
System.IO.File.WriteAllText("duc.html",s.ToString());
Console.ReadKey();
}
private static bool isFirstMergeRange(ExcelWorksheet sheet, string address, ref int colspan, ref int rowspan)
{
colspan = 1;
rowspan = 1;
foreach (var item in sheet.MergedCells)
{
var s = item.Split(':');
if (s.Length > 0 && s[0].Equals(address)){
ExcelRange range = sheet.Cells[item];
colspan = range.End.Column - range.Start.Column;
rowspan = range.End.Row - range.Start.Row;
if(colspan == 0) colspan = 1;
if(rowspan == 0) rowspan = 1;
return true;
}
}
return false;
}
Based on the answer from Duc Tran Minh, it works fine for me after I modified the code like this:
static void Main(string[] args)
{
ExcelPackage p = new ExcelPackage(new System.IO.FileInfo("X.XLSX"));
var sheet = p.Workbook.Worksheets["HMTD"];
var noOfCol = sheet.Dimension.End.Column;
var noOfRow = sheet.Dimension.End.Row;
StringBuilder s = new StringBuilder();
s.Append("<table>");
for (int i = 1; i <= noOfRow; i++)
{
s.Append("<tr>");
for (int j = 1; j <= noOfCol; j++)
{
int colspan = 1;
int rowspan = 1;
if (!sheet.Cells[i, j].Merge || (sheet.Cells[i, j].Merge && isFirstMergeRange(sheet, sheet.Cells[i, j].Address, ref colspan, ref rowspan)))
{
s.Append("<td rowspan='" + rowspan + "' colspan='" + colspan + "'>");
if(sheet.Cells[i,j] != null && sheet.Cells[i,j].Value != null)
s.Append(sheet.Cells[i,j].Value.ToString());
s.Append("</td>");
}
}
s.Append("</tr>");
}
s.Append("</table>");
System.IO.File.WriteAllText("duc.html",s.ToString());
Console.ReadKey();
}
bool isFirstMergeRange(ExcelWorksheet sheet, string address, ref int colspan, ref int rowspan)
{
colspan = 1;
rowspan = 1;
foreach (var item in sheet.MergedCells)
{
var s = item.Split(':');
if (s.Length > 0 && s[0].Equals(address))
{
ExcelRange range = sheet.Cells[item];
colspan = range.End.Column - range.Start.Column + 1;
rowspan = range.End.Row - range.Start.Row + 1;
return true;
}
}
return false;
}

Update a SavedQuery (View) from the SDK

I am trying to change all the Business Unit references I got after importing a solution to the ones in the Acceptance environment.
QueryExpression ViewQuery = new QueryExpression("savedquery");
String[] viewArrayFields = { "name", "fetchxml" };
ViewQuery.ColumnSet = new ColumnSet(viewArrayFields);
ViewQuery.PageInfo = new PagingInfo();
ViewQuery.PageInfo.Count = 5000;
ViewQuery.PageInfo.PageNumber = 1;
ViewQuery.PageInfo.ReturnTotalRecordCount = true;
EntityCollection retrievedViews = service.RetrieveMultiple(ViewQuery);
//iterate though the values and print the right one for the current user
int oldValues = 0;
int accValuesUpdated = 0;
int prodValuesUpdated = 0;
int total = 0;
foreach (var entity in retrievedViews.Entities)
{
total++;
if (!entity.Contains("fetchxml"))
{ }
else
{
string fetchXML = entity.Attributes["fetchxml"].ToString();
for (int i = 0; i < guidDictionnary.Count; i++)
{
var entry = guidDictionnary.ElementAt(i);
if (fetchXML.Contains(entry.Key.ToString().ToUpperInvariant()))
{
Console.WriteLine(entity.Attributes["name"].ToString());
oldValues++;
if (destinationEnv.Equals("acc"))
{
accValuesUpdated++;
Console.WriteLine();
Console.WriteLine("BEFORE:");
Console.WriteLine();
Console.WriteLine(entity.Attributes["fetchxml"].ToString());
string query = entity.Attributes["fetchxml"].ToString();
query = query.Replace(entry.Key.ToString().ToUpperInvariant(), entry.Value.AccGuid.ToString().ToUpperInvariant());
entity.Attributes["fetchxml"] = query;
Console.WriteLine();
Console.WriteLine("AFTER:");
Console.WriteLine();
Console.WriteLine(entity.Attributes["fetchxml"].ToString());
}
else
{
prodValuesUpdated++;
string query = entity.Attributes["fetchxml"].ToString();
query = query.Replace(entry.Key.ToString().ToUpperInvariant(), entry.Value.ProdGuid.ToString().ToUpperInvariant());
entity.Attributes["fetchxml"] = query;
}
service.Update(entity);
}
}
}
}
Console.WriteLine("{0} values to be updated. {1} shall be mapped to acceptance, {2} to prod. Total = {3} : {4}", oldValues, accValuesUpdated, prodValuesUpdated, total, retrievedViews.Entities.Count);
I see that the new value is corrected, but it does not get saved. I get no error while updating the record and publishing the changes in CRM does not help.
Any hint?
According to your comments, it sounds like the value you're saving the entity as, is the value that you want it to be. I'm guessing your issue is with not publishing your change. If you don't publish it, it'll still give you the old value of the FetchXml I believe.
Try calling this method:
PublishEntity(service, "savedquery");
private void PublishEntity(IOrganizationService service, string logicalName)
{
service.Execute(new PublishXmlRequest()
{
ParameterXml = "<importexportxml>"
+ " <entities>"
+ " <entity>" + logicalName + "</entity>"
+ " </entities>"
+ "</importexportxml>"
});
}

Getting a random UserProfile In SharePoint 2010

I am trying to retrieve a random number of users from the UserProfileManager.
But I am encountering errors when deploying to the live servers. I can't seem to see what is causing the error. My code is below:
for (int i = 0; i < NumberOfUserLimit; i++)
{
UserProfile up = profileManager.GetUserProfile(random.Next(1, NumberOfUserLimit));
if (up["FirstName"] != null && up["FirstName"].Value != null && !String.IsNullOrEmpty(up["FirstName"].Value.ToString()))
{
DataRow drUserProfile;
drUserProfile = dtUserProfile.NewRow();
drUserProfile["DisplayName"] = up.DisplayName;
drUserProfile["FirstName"] = up["FirstName"].Value;
drUserProfile["LastName"] = up["LastName"].Value;
drUserProfile["Department"] = up["Department"].Value;
drUserProfile["Location"] = up["SPS-Location"].Value;
drUserProfile["HireDate"] = up["SPS-HireDate"].Value;
drUserProfile["ContactNumber"] = up["Office"].Value;
if (up["PictureURL"] != null && up["PictureURL"].Value != null && !String.IsNullOrEmpty(up["PictureURL"].Value.ToString()))
{
string cleanAccountName = up["AccountName"].Value.ToString().Replace(#"\", "_");
string pictureUrl = String.Format("https://my.someintranet.com/User Photos/Profile Pictures/{0}_MThumb.jpg", cleanAccountName);
drUserProfile["Image"] = pictureUrl;
}
else
{
drUserProfile["Image"] = "~/_layouts/images/O14_person_placeHolder_96.png";
}
drUserProfile["MySiteUrl"] = up.PublicUrl;
dtUserProfile.Rows.Add(drUserProfile);
}
}
My code works when I apply a simple foreach to my code above instead of the "for loop":
foreach (UserProfile up in profileManager)
Which proves I can return userprofiles.
Any help is appreciated.
profileManager.GetUserProfile(long recordId)
expects a recordId from userprofile table. It is not an index, so you cannot use "random".
If you want to check RecordId, you can take a look at SQL tables of ProfileDB. Table "UserProfile_Full" has MasterRecordId column. Your parameter in GetUserProfile has to match of the user profile's MasterRecordId.
you can use the following code to get your random profiles:
IEnumerator profiles = profileManager.GetEnumerator();
int index = new Random().Next(1, 100);
while (index >= 0 && profiles.MoveNext())
index--;
UserProfile currentProfile = (UserProfile)profiles.Current
Code that handles Random better
public class TestClass
{
private random = new Random();
private long totalNumberOfProfiles; //ProfileManager.Count not always returns count correctly
public TestClass()
{
//this does not have to be in constructor but point is to have it cached (reasonably)
IEnumerator profiles = profileManager.GetEnumerator();
long counter = 0;
while (profiles.MoveNext())
counter++;
this.totalNumberOfProfiles = counter;
}
public fillInDataSet()
{
//something is here...
IEnumerator profiles = profileManager.GetEnumerator();
int index = random.Next(1, totalNumberOfProfiles);
while (index >= 0 && profiles.MoveNext())
index--;
UserProfile currentProfile = (UserProfile)profiles.Current
//something is here...
}
}

Resources