I have written a web app in asp.net that will import data from an excel spreadsheet to SQL (code below). If I run it from VS on my local machine using IIS Express it works fine. However, when I deploy it as a web app on our Azure web server as an IIS App Pool, it is only importing part of the data from excel (16250 rows out of 73,000). I have the app pool set to enable 32 bit applications (the microsoft.ace.oledb library on that server is 32 bit). I found the last record it loaded in SQL (it only partially loaded the row) and moved that to the top of the excel spreadsheet and re-imported. That record imported fine this time but still only got partial import into SQL (16219 rows this time). So it appears the data is OK but I cannot figure out why it is not loading all the data. I do not get any errors when running the process.
Code:
string excelstring = "provider=microsoft.ace.oledb.16.0;data source=" + rptpath + filename +
";extended properties=" + "\"excel 12.0;hdr=yes;\"";
try
{
OleDbConnection oledbconnection = new OleDbConnection();
oledbconnection.ConnectionString = excelstring;
oledbconnection.Open();
DataTable dtSheet = oledbconnection.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
foreach (DataRow drSheet in dtSheet.Rows)
{
sheetname = drSheet["TABLE_NAME"].ToString();
}
sqlstring = "SELECT * FROM [" + sheetname + "]";
OleDbCommand oledbcommand = new OleDbCommand(sqlstring, oledbconnection);
oledbcommand.CommandTimeout = 240;
DbDataReader dr = oledbcommand.ExecuteReader();
SqlBulkCopy bulkInsert = new SqlBulkCopy(connsql);
bulkInsert.DestinationTableName = tablename;
bulkInsert.BulkCopyTimeout = 240;
bulkInsert.WriteToServer(dr);
oledbconnection.Close();
}
catch (Exception Ex)
{
ClientScript.RegisterStartupScript(GetType(), "showError", "alert('" + Ex.Message + "');", true);
errormessage = Ex.Message;
return "error";
}
Related
Hi I want to establish a connection with the Microsoft azure databricks delta table inside my spring boot application.I have the cluster url,username and the password(token) of the delta table from which I need to pull the data to my application. Kindly shed some light on this
You can access cluster & underlying tables using the JDBC (see documentation). You need to get the corresponding driver, and add it to your application, and then just use normal JDBC API, like this:
String jdbcConnectPassthroughCluster = "jdbc:spark://<server-hostname>:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/0/xxxx-xxxxxx-xxxxxxxx;AuthMech=3;UID=token;PWD=";
String PATH = "<personal token>"
String JDBC_DRIVER = "com.simba.spark.jdbc.Driver";
String DB_URL = jdbcConnectPassthroughCluster + PAT;
Class.forName(JDBC_DRIVER);
System.out.println("Getting connection");
Connection conn = DriverManager.getConnection(DB_URL);
Statement stmt = conn.createStatement();
System.out.println("Going to execute query");
ResultSet rs = stmt.executeQuery("select * from table");
System.out.println("Query is executed");
int i = 0;
while(rs.next()) {
System.out.println("Row " + i + "=" + rs.getLong(1));
i++;
}
rs.close();
stmt.close();
conn.close();
I am generating Excel files using OpenXML, the file opens without any issues on my desktop with Excel 2016 installed. The file also passes validation using Open XML Productivity Tool 2.5.
The problem is that the file is rejected from the SQL Server Integration Services (SSIS) processing. The error is "External table is not in the expected format"
As far as I know SSIS is using OLEDB to process the files.
I have tried to read the file locally using the following code and was able to reproduce the same error.
The code is as follows:
var fileName = "C:\\Temp\\myExcel.xlsx";
var connectionString = String.Empty;
if (Path.GetExtension(fileName) == ".xlsx")
{
connectionString = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=\"Excel 12.0;HDR=YES;\"", fileName);
}
else //.xls
{
connectionString = string.Format("Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties=\"Excel 8.0;HDR=YES;IMEX=1;\"", fileName);
}
using (OleDbConnection conn = new OleDbConnection(connectionString))
{
conn.Open();
var adapter = new OleDbDataAdapter("select * from [test$]", conn);
var excelDataSet = new DataSet();
adapter.Fill(excelDataSet, "anyNameHere");
var data = excelDataSet.Tables["anyNameHere"];
foreach (DataRow row in data.Rows)
{
Console.Out.WriteLine(row[NAME].ToString());
}
}
All I was able to find in the net that this error is due to wrong connection string (Microsoft.Jet.OLEDB.4.0 and Excel 8.0 instead of Microsoft.ACE.OLEDB.12.0 and Excel 12.0) but I already have this.
This is definitely something wring with the file itself. The problem disappears if I open the file in Excel, and use save as.
I was trying to unpack the original and resaved files and compare the contents but the difference is huge, when saving excel adds lots of styles, themes, and xmls references. I really would not like to follow that path.
Do you know what parts are important for OleDB provider when reading Excel??
I tried many ways to fix this error :
"Failed to establish a database connection. Check connection string, username and password."
Code :
function AcessaVendas () {
var URL = 'jdbc:sqlserver://servername:49159'
var USER = 'visitant'
var PASS = '*******'
var conn = Jdbc.getConnection(URL, USER, PASS);
var stmt = conn.createStatement();
stmt.setMaxRows(10000);
var start = new Date();
var rs = stmt.executeQuery( myquery());
var ss = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Dados");
ss.getRange('a2:h1000').clearContent();
var cell = ss.getRange('a2');
var row = 0;
while (rs.next()) {
for (var col = 0; col < 11; col++) {
cell.offset(row, col).setValue(rs.getString(col + 1));
}
row++;
}
rs.close();
stmt.close();
conn.close();
var end = new Date();
Logger.log('Time elapsed: ' + (end.getTime() - start.getTime()));
}
First i have tried changing the azure servername for the server ip, then i tried adding the database name like,
db = "databasename"
URL = 'jdbc:mysql://servername:49159'+'/'+ db;
I also thought that maybe the database user that i have created in the sql server management studio could be the problem, because he has db_datareader role but i tried using the user with db_owner role and it also didn't work.
I also tried to put every Google App Server IPs whitelist range in the firewall of the azure server(one at once, or do i have to put ervery ip range at once?).
Could someone give me a help?
I have a simple Windows Azure cloud service that does random number generation running on my local machine. I have then published this to azure
public string GetNumber()
{
Random number = new Random();
return "number = " + number.Next(1, 100).ToString() + " number ";
}
When I call this method from my client everything works fine.
ie
ServiceReference1.RandomNumbersClient client = new ServiceReference1.RandomNumbersClient();
Console.WriteLine("Starting .... ");
Console.WriteLine(client.GetNumber());
Now, the next step is I have added some code that requires "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework.NETFramework\v4.5.2\System.Speech.dll" to be available in the project. This is easy on my local server as I just add the reference to the dll in my VS project.
New method ....
public string GetNumberAsString()
{
Random number = new Random();
SpeechSynthesizer synth = new SpeechSynthesizer();
synth.SetOutputToDefaultAudioDevice();
synth.Speak("This example demonstrates a basic use of Speech Synthesizer");
return "number = " + number.Next(1, 100).ToString() + " number ";
}
Problem now is when I publish to Azure. How do I make the DLL available to my app running in Azure?
When I call the GetNumber() method it works fine but any call to GetNumberAsString() fails because of the additional code.
ServiceReference1.RandomNumbersClient client = new ServiceReference1.RandomNumbersClient();
Console.WriteLine("Starting .... ");
Console.WriteLine(client.GetNumberAsString());
How do I actually inspect the logs for the Webservice running on Azure so I can make a start at finding out what the problem is.
I am trying to create a new user -> set password and enable account .
earlier i was using 1 single object , but after looking at a few posts i decided to use 'using' for 3 different operations
string strDisplayName = txtFirstName.Text + " " + txtLastName.Text;
string strUser = txtLoginName.Text;
string pw = "pass#123";
using (var objADAM = new DirectoryEntry("LDAP://" + adlink + "/CN=Users,DC=SS,DC=COM", "ss\\luser", "pass#123", AuthenticationTypes.Secure))
{
const long ADS_OPTION_PASSWORD_PORTNUMBER = 6;
const long ADS_OPTION_PASSWORD_METHOD = 7;
const int ADS_PASSWORD_ENCODE_CLEAR = 1;
string strPort = "389";
int intPort = Int32.Parse(strPort);
using (var objUser = objADAM.Children.Add("CN=" + strUser, "user"))
{
objUser.Properties["sAMAccountName"].Add(strUser);
objUser.CommitChanges();
}
}
using (var user = new DirectoryEntry("LDAP://" + adlink + "/CN=" + strUser + ",CN=Users,DC=SS,DC=COM", "ss\\rluser", "pass#123"))
{
user.Invoke("SetPassword", new object[] { "password" });
user.CommitChanges();
}
using (var user = new DirectoryEntry("LDAP://" + adlink + "/CN=" + strUser + ",CN=Users,DC=SS,DC=COM", "ss\\rluser", "pass#123"))
{
//Enable account and change password on first logon flag
user.Properties["userAccountControl"].Value = 0x200;
user.Properties["pwdLastSet"].Value = 0;
user.CommitChanges();
}
I must mention, that i am outside the domian, and trying to connect to a remote AD on another domain . The credential's passed however are the ADMIN
The user creation goes on smoothly (after some hiccups with port opening & LDAP connections) , but the issue occurs when the invoke ->setpassword is called .
The error is :"the RPC server is unavailable " , just to make sure i am not doing something wrong in my code, i downloaded a LDAP admin tool and tried to reset the password of an existing user ->same error
steps
-checked the RPC service running
-opened RPC ports -135 ,blah blah..basically every port there is to open :|
any help is appreciated .
Thanks
Rajat
For example:
DirectoryEntry de = new DirectoryEntry();
de.Path = "LDAP://dnsname.domain.com:389/OU=Companies;
Microsoft recommends accessing using DNS.
if the machine you are accessing is connected to a different domain, you must specify it as "ip dnsname" in the hosts file in the "C:\Windows\System32\drivers\etc " directory.
using adlink is string domain, because AD method Invoke using domain name: "abc.com"