I am setting up an ASP.NET WebApi application running on the Azure web application environment. Most of my controllers and endpoints work fine, but there are about 10 routes that return 500 errors "An error has occured". It's not random. It's the same routes every time and I can find no pattern (not all the HTTP methods, from different controllers where other routes in that same controller work fine, and so forth)
When these errors occur, no error logging gets triggered as far as I can tell in the app. (I am using Raygun.IO if that matters). I tried adding a global.asax file with following lines in the Application_Error function:
Exception ex = Server.GetLastError();
new RaygunClient().SendInBackground(null, ex);
but as far as I can see, it doesn't get triggered when these 500 errors occur. The only thing I have found in the Azure server logs is the following warning in the Failed Requests log:
However, I don't see any errors in the trace previous to that point. I'm also not finding any other errors in any of my Azure logs that I can relate to these failing routes.
It's .NET 4.6 (tried 4.5.1 and 4.5.2 with no difference) WebApi using an OWIN startup class. Also tried updating all my packages to no effect.
Check to be sure you don't have routes that conflict. When a URI matches two or more controller actions, and thus Web API can't pick one, you will get a 500 error.
If you have a consistent repro, then you can use remote debugging to attach to your web app and debug it (https://azure.microsoft.com/en-us/documentation/articles/web-sites-dotnet-troubleshoot-visual-studio/). If you can repro it through the browser, you can also turn the custom errors page off so that it shows the stack trace through the browser (<customErrors mode="off"/> under <server.web>).
Also, you can enable better diagnostic logs using the web app settings. Information on how to do so is here: https://azure.microsoft.com/en-us/documentation/articles/web-sites-configure/ under the section "Enabling diagnostic logs".
Also, as Brent said, URL conflicts can cause some nasty errors which aren't immediately obvious.
I figured out what was going on. There were actual errors occurring (mostly SQL related), but the way that we had configured the functions and error logging in those functions, the actual errors were being swallowed up and lost, overriding the error logging behavior of Raygun. I went through and ripped out the Try/Catch blocks we were using for error trapping and logging and the real errors finally emerged into the Raygun dashboard.
Related
This is a similar post to Azure Web App Trace logs not appearing in log, however the original poster seems to have abandoned the question without resolving/accepting an answer.
I am trying to trace an issue that only happens on the Azure web app (now called app service). I'm unable to perform any remote debugging due to our company policies, so tracing is our best tool.
However, I've tried following various tutorials, but I still can't seem to get any of my trace information logged.
I've tried:
Setting the Application Logging (Filesystem) Level to Verbose, Information, Error -- nothing.
Looking for the logs in
the FTP server at /LogFiles/Application
the KUDU interface at https://.scm.azurewebsites.net and again, navigated to /LogFiles/Application
portal's Monitoring > Live stream (the section under Diagnostic Logs for the website)
Nada. I've even waited a few hours (thinking it might be a delay), and still nothing.
I setup a very basic hello world ASPX and all it does (in the Page_Load) is try to write 'hello' to the trace log using
Trace.TraceError
Trace.TraceInformation
Trace.TraceWarning
Trace.WriteLine
Console.Out.WriteLine
Console.Error.WriteLine
Some weird stuff I've also tried
setting my debug=true in my web.config
setting CustomErrors from RemoteOnly to Off
trying to use System.Diagnostics.TextWriterTraceListener
Anyone have any ideas I might try?
Exceptions in your live web app are reported by Application Insights. You can correlate failed requests with exceptions and other events at both the client and server, so that you can quickly diagnose the causes. You may refer this article: https://learn.microsoft.com/en-us/azure/application-insights/app-insights-asp-net-exceptions.
If you use NLog, log4Net or System.Diagnostics.Trace for diagnostic tracing in your ASP.NET application, you can have your logs sent to Azure Application Insights, where you can explore and search them. Your logs will be merged with the other telemetry coming from your application, so that you can identify the traces associated with servicing each user request, and correlate them with other events and exception reports. You may refer this article: https://learn.microsoft.com/en-us/azure/application-insights/app-insights-asp-net-trace-logs.
I had an issue on my test server that was throwing an exception, I noticed that it logged nothing to application insights so first tried to debug on my local machine. I managed to replicate the issue, and low and behold in my DEV application insights I have server exceptions logged.
So I go back to my Test Application insights and again replicate the issue but still no errors logged (Even after an hour or so).
However I can see it logs them as "Failed Requests", just not Server Exceptions. Without the server exceptions I can't see the stacktrace/error message.
I am using the Log4Net app insights extension to log these exceptions. It's obviously not a code issue because the same code is being run in both cases. So I looked at configuration issues.
My log4net.config and applicationinsights.config have no transforms from Dev to Test (Or to prod). They stay the same.
My Web.config only has a transform to remove the debug=true as is standard.
I do set some environment properties in Azure (Using Azure Websites), but the only one that is actually application changing is the AppInsightsInstrumentation key to switch it to a different AppInsights. However I know that this key is correct because I'm logging some things, just not others.
Here's the answer (Kinda)
I was logging the exception in a Basecontroller, this worked fine on my local machine and I could debug through.
What happens though is when I push it remotely, .net's out of the box error handling kicks in. Notably the global "HandleErrorAttribute" (Check your FilterConfig.cs). This seems to capture and swallow the exception. There is lots of other talks about this if you wish to override this behaviour, mostly when talking about other logging tools like Elmah.
The reason I only see this in my test environment is because CustomErrors defaults to remoteOnly. So I only see it not logging when it's not on my machine.
Because I'm handling errors myself, I can remove this line from my FilterConfig.cs
filters.Add(new HandleErrorAttribute());
And server exceptions should be logged now (I also handle showing the user a nice error page in my BaseController for now)
My application is deployed on IIS 7. I want to check the number of failures as my logic is getting failed at some point and getting errors.Is there any general weblogs in IIS.I can only see system errors in the event logs. Is there any web logs?
Manually trawling the standard W3C logs is ok if you're chasing down requests for certain content types, but they won't tell you an awful lot about why your web application is failing and responding with many 4XX and 5XX status codes. You'll get a status code, but that's about it.
Failed Request Tracing:
Your "go to" diagnostic tool should be the Failed Request Tracing feature that is built into IIS7+.
FRT is one of my favourite features of IIS7/8 for tracking down problems with production sites, especially when debugging apps built on the WebAPI and Ajaxy type stuff.
For more information see:
http://www.iis.net/learn/troubleshoot/using-failed-request-tracing
For example, last week FRT helped me get to the bottom of an issue with a client's hosted site. A particular part of the site (which uses the WebAPI) was failing with a 405 Method Not Allowed status code when making a HTTP DELETE request and despite the DELETE verb being permitted.
Using FRT I was able to generate trace of the failing request which showed me this:
Expanding the "View Trace" entries revealed this error:
The solution for our customer was to disable (it's not used) the WebDAV native module which doesn't permit non-Windows authenticated requests with certain verbs (such as DELETE) to complete. Even if the WebDAV module isn't handling the request it's still in the request pipeline inspecting and validating request headers.
Failed Request Tracing is a really invaluable diagnostic tool, you should learn how to use it.
The HTTPERR Logs:
You should also check the HTTPERR logs located in:
C:\Windows\System32\LogFiles\HTTPERR
If you get 503 - Service Unavailable errors they're a good place to look for clues as to what went wrong if an application pool fails catastrophically, and often.
The is a folder named 'logs' in your 'inetpub' folder where all the logs live. You can look at the Logging tab under IIS in IIS Manager to see the name of the specific log you should check for your site.
my question should be quite simple, but unfortunately I had no luck in solving it.
Basically, I have some Web API controllers hosted by OWIN and deployed on Azure.
I really need to track down exceptions that occur in each middleware (for example OAuthAuthorizationServerProvider or SignalR Persistent Connections), but I definitely don't have a clue on how to achieve it.
I tried Elmah, but it doesn't seem to work properly with OWIN due to lacking HttpContext.
I tried using log4net, but I'm only able to log exceptions thrown by Web API Controllers using a custom ExceptionFilterAttribute.. others are ignored.
I tried to define a custom LoggerFactory and to assign it in Startup, using app.SetLoggerFactory(new MyLoggerFactory()),
but exception thrown by other middlewares are not logged.
I tried to get at least a meaningful error message sent to the client, but despite <customErrors mode="Off"/> and <deployment retail="false"/>, Azure refuses to return anything but {"message":"an error has occurred"}.. I tried both Azure Web Sites and Azure Cloud Services.
I saw some cloud alternatives that should work with OWIN, like Elmah.io or Raygun.io, but I don't need their cloud features and it is definitely not worth paying hundreds $ per year just to log some exceptions.
What should be the best way to log any possible exception thrown by my application?
Thanks for your help
have you take a look at this link ? http://www.asp.net/web-api/overview/error-handling/web-api-global-error-handling
Because you can't catch all the exceptions using an exceptionFilter, they propose to use a IExceptionLogger and IExceptionHandler to allow global error handling in Web Api 2.
After that, if it's not fit your need, you can construct an OwinMiddleWare that you will place in first position (before the Authenticate stage), this middleware could :
create a requestId in the header of the response
analyse the response code, before sending response, and if it's not a IsSuccessStatusCode, you could log the exception message to a DB and replace the content of the response to send a simple error message to the client using the requestId (to allow you to find the related exception in your db)
hope this help
I'm trying to figure out why I'm getting 500 errors in setting up a website in IIS.
So far I've tried the following steps:
Enabled Failed Request Tracing (Doesn't write logs for this site, but
works for other sites)
Enabled detailed error messages. Still Getting the default 500 page
with no additional information.
Give app pool full permission to the project directory.
Made sure app pool was running on classic .NET 2 (old app)
Running the site under a permutation of (Classic/Integrated, .NET
2/4)
Enabled anonymous authentication
So my thinking is, somehow, the site fails before the logging modules are ran.
I suspect this is the case because I see no new entities in Event Viewer, IIS Advanced Logs folder, Or in Failed Request Tracing folder. My only source of information (besides 500 error) is a new entry in the IIS log:
2012-12-04 13:06:05 127.0.0.7 GET / - 80 - 127.0.0.1 Mozilla/5.0+(compatible;.....)
To verify this, is there a way to check which stage of the pipeline a request failed? Is it possible to run the logging modules before the failure occurs?
There is a trace event logger for HTTP.sys. With this you can determine if the request is even making it to the right app pool in IIS. Direction on usage
As a last resort, Microsoft offers a tool called Debug Diagnostic. When you have no other option, use this. It will produce a crash dump of the app pool of your choice. Not easy to go through, but it’s a lead. Direction on usage