I've developed a .Net console application to run as a webjob under Azure App Service.
This console app is using WinSCP to transfer files from App Service Filesystem to an on-prem FTP Server.
The connectivity between App Service & the on-perm FTP server is Okay.
Most of the time the job succeeds ,the files are synced, and log files written as well.
Sometimes, the job fails, no files synced, and no Log is NOT being written as well.
The Exception that is being fired intermittently on ALL of our Azure environments (Dev, Test, Prod):
WinSCP process terminated with exit code -1073741819 (C0000005). There was no output. Response log file D:\local\Temp\wscp550C.03E988EE.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.
Any Clues ?
My code snippet ..
///Session Options
var sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
HostName = host,
UserName = userName,
Password = password,
SshHostKeyFingerprint = sshHostKeyFingerprint
};
///Opening Session & Sync Files
using (var session = new Session())
{
var timestmp = DateTime.Now.ToString("MMddyyyyHHmmss") + ".txt";
session.SessionLogPath = ConfigurationManager.AppSettings["SessionLogPath"] + timestmp;
session.XmlLogPath = ConfigurationManager.AppSettings["XmlLogPath"] + timestmp;
session.XmlLogPreserve = true;
session.FileTransferred += FileTransferred;
session.Open(sessionOptions);
var syncResult = session.SynchronizeDirectories(SynchronizationMode.Remote, localFolder, remoteFolder, false,false);
syncResult.Check();
}
WinSCP process terminated with exit code -1073741819 (C0000005). There was no output. Response log file D:\local\Temp\wscp550C.03E988EE.tmp was not created. This could indicate lack of write permissions to the log folder or problems starting WinSCP itself.
The error messages shows that you don’t give an write permission on production. Please give all write permission in Production Environment.
WinSCP could not create/write to log file that it uses to report back to the .NET assembly . Make sure your process has write permissions to the temporary folder. Alternatively, you can use (undocumented) property Session.XmlLogPath to change the log file location.
Refer here
C0000005 means "(memory) access violation". That probably indicates that WinSCP has crashed. It might mean a bug in WinSCP. Make sure you have the latest version. Otherwise, you better report this on WinSCP support forum, as software bugs do not really belong to Stack Overflow.
Related
I am using Serilog to write to a file and try to get more information about an error that is occurring in my production cluster...
In my local dev cluster the log files are created fine but they are not created in the VM's on my production cluster. I think this may be security related
Has anyone ever had this?
My production cluster has 5 nodes with a Windows 2016 VM on each
Even more strange is that this works on a single node cluster in Azure
public static ILogger ConfigureLogging(string appName, string appVersion)
{
AppDomain.CurrentDomain.ProcessExit += (sender, args) => Log.CloseAndFlush();
var configPackage = FabricRuntime.GetActivationContext().GetConfigurationPackageObject("Config");
var environmentName = configPackage.GetSetting("appSettings", "Inspired.TradingPlatform:EnvironmentName");
var loggerConfiguration = new LoggerConfiguration()
.WriteTo.File(#"D:\SvcFab\applog-" + appName + ".txt", shared: true, rollingInterval: RollingInterval.Day)
.Enrich.WithProperty("AppName", appName)
.Enrich.WithProperty("AppVersion", appVersion)
.Enrich.WithProperty("EnvName", environmentName);
var log = loggerConfiguration.CreateLogger();
log.Information("Starting {AppName} v{AppVersion} application", appName, appVersion);
return Log.Logger = log;
}
Paul
I wouldn't recommend logging into local files in Service Fabric, since your node may be moved to another VM any time and you won't have access to these files. Consider using another sinks which write to external system (database, message bus or logging system like loggly)
It is likely a permission issue. Your service might be trying to log to a folder where it does not have permission.
By default, your services will run under same user as the Fabric.exe process, that run as NetworkService, you can find more information about this on this link.
I would not recommend this approach, because many reasons, a few of them are:
Your services might be moved around the cluster so your files will be incomplete
You have to log on multiple machines to find the logs
The node might be gone with files (Scale up + Down, Failure, Disk error)
Multiple instances on same node trying to access the same file
and so on...
On Service Fabric, the recommended way is to use EventSource(or ETW) + EventFlow + Application Insights. They run smoothly together and bring you many features.
If you want to use stay on Serilog, I would recommend you use Serilog + Application Insights instead, it will give you move flexibility on your monitoring. Take a look at the Application Insights sink for serilog here.
This was actually user error! I was connecting to a different cluster of VMs than the one my service fabric was connected to! Whoops!
I can't get SqlDataProvider to work when executed in a fsx script which is running in an Azure Web Site.
I have started from the samples that Tomas Petrecek has here: https://github.com/tpetricek/Dojo-Suave-FsHome.
In short it is a FSX script that is executed using the IIS httpPlatformHandler so that all http requests to my Azure Web site is forwarded to my F# script.
The F# Script use Suave to handle the requests.
When I tried adding some database access to my HTTP handlers I got into problems.
The problematic code looks like this:
[<Literal>]
let connStr = "Server=(localdb)\\v11.0;Initial Catalog=My_Database;Integrated Security=true;"
[<Literal>]
let resolutionFolder = __SOURCE_DIRECTORY__
FSharp.Data.Sql.Common.QueryEvents.SqlQueryEvent |> Event.add (printfn "Executing SQL: %s")
// the following line fails when executing in azure
type db = SqlDataProvider<connStr, Common.DatabaseProviderTypes.MSSQLSERVER, ResolutionPath = resolutionFolder>
let saveData someDataToSave =
let ctx = db.GetDataContext(Environment.GetEnvironmentVariable("SQLAZURECONNSTR_QUERIES"))
.....
/// code using the context here
This works just fine when I run it locally, but when I deploy it to the azure site it will fail at the line where the type dbis created.
The error message is (line 70 is the line that has the type db = ...:
D:\home\site\wwwroot\app.fsx(70,11): error FS3033: The type provider
'FSharp.Data.Sql.SqlTypeProvider' reported an error: A network-related
or instance-specific error occurred while establishing a connection to
SQL Server. The server was not found or was not accessible. Verify
that the instance name is correct and that SQL Server is configured to
allow remote connections. (provider: SQL Network Interfaces, error: 52
- Unable to locate a Local Database Runtime installation. Verify that SQL Server Express is properly installed and that the Local Database
Runtime feature is enabled.)
The design-time database in the connStr is not available in the azure site, but I thought this is why we have the GetDataContext overload that takes a connection string to be used at run-time?
Is it because it is running as a script and not as compiled code that it is trying to access the database when creating the TypeProvider?
If yes, does it mean that my only option is to compile and provide the database code as a compiled assembly that I load and use in my Suave FSX script?
Reading the connection string from a config file does not work very well as this is in a azure site. I really need to get the connection string from an environment variable (which is set in the azure management interface).
Hmm, this is a bit unfortunate - as #Fyodor mentioned in the comments, the problem is that the script-based deployment to Azure actually compiles the script on the Azure machine - and so you need to have a statically-resolved connection string that works on Azure.
There are two options:
Use compiled project instead. If you compile your F# code locally and deploy the compiled code to Azure it will work. Sadly, there are no good samples for that.
Do some clever trick to make the connection string accessible to the script at compile time.
Send a PR to the SQL provider so that you can give it the name of an environment variable and it reads the connection string from there.
I think (3) would actually be quite nice and useful feature.
I'm not necessarily sure what the best way to do (2) would be. But I think you might be able to modify app.azure.fsx so that it creates a file (say connection.fsx) that contains something like:
module Connection
let [<Literal>] ConnString = "<Contents of SQLAZURECONNSTR_QUERIES>"
Then app.fsx could load this script and use Connection.ConnString in the argument of SQL type provider.
I would like to create a view in SQL Server 2008 that displays the data contained in an Excel file.
I do not want to use the import data as these data are updated.
I found this: http://support.microsoft.com/kb/321686/en
I made these commands to enable some options :
sp_configure 'show advanced options', 1
RECONFIGURE
GO
sp_configure 'ad hoc distributed queries', 1
RECONFIGURE
GO
But when I try to read the Excel file with this query:
SELECT * FROM INTO XLImport3 OPENDATASOURCE ('Microsoft.Jet.OLEDB.4.0', 'Data Source=C:\Documents and Settings\jconnor\Desktop\Test.xlsx') ... [Sheet1$]
I back (it's a translation from french) :
The OLE DB provider " Microsoft.Jet.OLEDB.4.0 " for linked server " (null) " returned >message " Unspecified error " .
Msg 7303 , Level 16 , State 1, Line 1
Unable to initialize the object data source OLE DB provider " Microsoft.Jet.OLEDB.4.0 " for >linked server " (null) " .
Does someone have any clue ?
Thank you in advance
Kal,
So your main error is likely this;
OLE DB provider "MICROSOFT.JET.OLEDB.4.0" for linked server "(null)" returned message "Unspecified error".
Msg 7303, Level 16, State 1, Line 1
Cannot initialize the data source object of OLE DB provider "MICROSOFT.JET.OLEDB.4.0" for linked server "(null)".
I would check some permissions.
Check the permissions on the Temp folder
This is needed because the provider uses the temp folder while retrieving the data. The folder can be one of the below based on whether you use a local system account or network domain account.
For network accounts, folder is
\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp
and for local system account its \Windows\ServiceProfiles\LocalService\AppData\Local\Temp
Right click on this folder and give it read write access to the account (or group) executing the code. That solved the error for me.
Also
http://blogs.msdn.com/b/spike/archive/2008/07/23/ole-db-provider-microsoft-jet-oledb-4-0-for-linked-server-null-returned-message-unspecified-error.aspx
This is because the SQL Server Service is trying to write the temp DSN to the temp folder for the login that started the service, in this case the Admin/Admin login.
The temp folder is something like: C:\Documents and Settings\Admin\Local Settings\Temp
.15 As mentioned, the OleDbProvider will always execute in the context of the user who initialized it, in this case User/User.
.16 User/User has no rights on this folder (C:\Documents and Settings\Admin\Local Settings\Temp).
If running FileMon when the SQL is executed, we can see the following:
(Actually, try using Process Monitor - http://technet.microsoft.com/en-us/sysinternals/bb896645)
sqlservr.exe:000 QUERY INFORMATION C:\DOCUME~1\Admini~1\LOCALS~1\Temp ACCESS DENIED Attributes: Error
So to summarize so far:
The SQL Server service is started as Admin/Admin, when the select is made, the OleDb provider is invoked by User/User.
Now the OleDb provider attempts to create a temporary DSN in the temp directory. This will be the temp directory for the SQL Server service (Admin/Admin)
but the user (in this case User/User) does not have write permissions on this folder. And the error occurs.
There are two ways to resolve this.
Option 1:
Log out of the machine and log in as the account that starts the SQL Server Service (in this case Admin/Admin) then start a command prompt
and type “set t” (no quotes), this will show something like:
TEMP=C:\DOCUME~1\Admin\LOCALS~1\Temp
TMP=C:\DOCUME~1\Admin\LOCALS~1\Temp
these are the environment variables set for %TEMP% and %TMP%, so go to that folder and right click and select Properties -> Security,
then add the user, in this case User/User, note that the default for the user is Read&Execute/List Folder Content/Read, this not enough, you have to select Write as well.
Log out, and log in again as User/User and rerun the command from SSMS. This time it should work.
Option 2:
Log on as Admin and change the TEMP and TMP variable to, for example, C:\Temp, basically this moves the Temp directory out of the Documents and Settings folder.
However, you must restart the SQL server for this to take effect.
So basically, what happens is that when SQL Server starts, it uses the Temp folder of the startup account (Admin/Admin) but the MICROSOFT.JET.OLEDB.4.0 will always execute
as the user who calls the SQL command (User/User) and this will fail unless this user does not have Write access to that temp folder.
Without knowing all setups out there, perhaps option 2 is the preferred solution since with option 1, you will have to add ALL the users that will invoke the provider which may not be practical.
Also, when changing the startup account for the SQL Server service, then the TEMP directory for that account will be used, and you will see the error again until you, again, give write permissions for all the users on this TEMP folder...or a user group (preferred).
I'm getting the following error when attempting to run DotNetNuke 7.1 from IIS.
Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.NullReferenceException: Object reference not set to an instance of an object.
Source Error:
Line 572: //first call GetProviderPath - this insures that the Database is Initialised correctly
Line 573: //and also generates the appropriate error message if it cannot be initialised correctly
Line 574: string strMessage = DataProvider.Instance().GetProviderPath();
Line 575: //get current database version from DB
Line 576: if (!strMessage.StartsWith("ERROR:"))
I've tried running it from Visual Studio 2012 after downloading and extracting the source code to a folder, then running, but I get the same error (also, VS loads about 13 instances of it's built in webserver which can't be correct).
Clearly, there is something wrong with the database. From what I've read in the past, there should have been a start up configuration page (for configuring settings the first time you run the project).
I did look at the local version of IIS (running on Windows 8) and it created the site fine there, however, for some reason the internal webserver attempts to run (and the option to run on an external IIS is greyed out).
Anyone run into this problem with DNN Community edition? I've tried running as admin and setting permissions with no luck at all.
Any way to fix this?
Ok, the key is to delete the Database.mdf file completely.
Then create a new empty database of your choice in SQL Server (2008 or greater).
Create a new user account with db_owner access (as it must be able to create tables, etc).
Change the connection strings in the release.config and development.config to connect to the database.
DELETE the web.config file.
RENAME either config file to "web.config"
Set the default project to the web project in VS
set the default page to default.aspx
Run
I made the erroneous assumption that running the app would rename the config file for me (not sure why I assumed that).
SOLVED!
I am attempting to obtain a data feed from yahoo finance. I am doing this with the following code:
System.Net.WebRequest request = System.Net.WebRequest.Create(http://download.finance.yahoo.com/download/quotes.csv?format=sl&ext=.csv&symbols=^ftse,^ftmc,^ftas,^ftt1x,^dJA);
request.UseDefaultCredentials = true;
// set properties of the request
using (System.Net.WebResponse response = request.GetResponse())
{
using (System.IO.StreamReader reader = new System.IO.StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
I have placed this code into a console application and, using Console.WriteLine on the output I receive the information I require. I have used the 'Run as..' command to execute this using a specific domain account.
When I use this code from within a Page load I receive the following error message "No connection could be made because the target machine actively refused it 76.13.114.90:80".
This seems to suggest that the call is reaching yahoo (is this true?) and that there is something missing.
This would suggest there is an identity difference in the calls between the console application and application pool.
Environment is: Windows Server 2003, IIS 6.0, .net 4.0
"Target machine actively refused it" indicates that the TCP connection itself is not succeeding. This could be due to the fact that the Proxy settings when run under IIS are not the same as those that apply when you run in the console.
You can fix this by setting a WebProxy on your request, that points to the proxy server being used in the environment.
Yes, an active refusal is indication that the target machine is receiving the request and the information in the headers is either incorrect or insufficient to process the request. It is entirely possible that if you had to run this call using a "run as" command in console that the application pool's identity user does not have the appropriate permission or username. You can attempt to change the identity user to this specific domain account to see if that alleviates the problem, but you may have to isolate this particular function into its own application pool in order to protect the rest of the website from having this specification.