how to get azure webrole to log all 404s - azure

Question

I have been trying to get logging working with azure for my MVC project but so far haven't had much success.
I have a Diagnostics connection string in my ServiceConfiguration.Cloud.cscfg file which points to my blob storage:
...
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.DiagnosticsConnectionString" value="**ConectionString**" />
</ConfigurationSettings>
My web.config has tracing set up
...
<tracing>
<traceFailedRequests>
<remove path="*"/>
<add path="*">
<traceAreas>
<add provider="ASP" verbosity="Verbose" />
<add provider="ASPNET" areas="Infrastructure,Module,Page,AppServices" verbosity="Verbose" />
<add provider="ISAPI Extension" verbosity="Verbose" />
<add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module" verbosity="Verbose" />
</traceAreas>
<failureDefinitions timeTaken="00:00:15" statusCodes="400-599" />
</add>
</traceFailedRequests>
</tracing>
</system.webServer>
My WebRole.cs has the following in
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;
namespace MvcWebRole1
{
public class WebRole : RoleEntryPoint
{
public override bool OnStart()
{
// Get the factory configuration so that it can be edited
DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
// Set scheduled transfer interval for infrastructure logs to 1 minute
config.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1);
// Specify a logging level to filter records to transfer
config.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
// Set scheduled transfer interval for user's Windows Azure Logs to 1 minute
config.Logs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1);
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.DiagnosticsConnectionString", config);
//RoleEnvironment.Changing += this.RoleEnvironmentChanging;
return base.OnStart();
}
}
}
But the I am not seeing any diagnostics logs
The mam folder just contains an MACommanda.xml and a MASecret, vsdeploy folder is empty and the wad-control-container has a file for each deployment.
Am I missing something / doing something wrong?
I have been trying to follow the guides from http://msdn.microsoft.com/en-us/library/windowsazure/gg433048.aspx in particular http://channel9.msdn.com/learn/courses/Azure/Deployment/DeployingApplicationsinWindowsAzure/Exercise-3-Monitoring-Applications-in-Windows-Azure
Update:
I found the following which could be part of the problem
IIS7 Logs Are Not Collected Properly -
http://msdn.microsoft.com/en-us/library/hh134842
although that should only account for the 404s not working, with a failure definition of 15 seconds my 17 second sleep in my controller action should have still been logged

Answer 1

Here is how I was finally able to get all the logging working on the Azure web role:
In the WebRole.cs include the following:
// Get the default initial configuration for DiagnosticMonitor.
var config = DiagnosticMonitor.GetDefaultInitialConfiguration();
// Filter the logs so that only error-level logs are transferred to persistent storage.
config.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = config.Logs.ScheduledTransferLogLevelFilter =
config.WindowsEventLog.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
// Schedule a transfer period of 30 minutes.
config.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = config.Logs.ScheduledTransferPeriod = config.WindowsEventLog.ScheduledTransferPeriod =
config.Directories.ScheduledTransferPeriod = config.PerformanceCounters.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
// Specify a buffer quota.
config.DiagnosticInfrastructureLogs.BufferQuotaInMB = config.Logs.BufferQuotaInMB = config.WindowsEventLog.BufferQuotaInMB =
config.Directories.BufferQuotaInMB = config.PerformanceCounters.BufferQuotaInMB = 512;
// Set an overall quota of 8GB maximum size.
config.OverallQuotaInMB = 8192;
// WindowsEventLog data buffer being added to the configuration, which is defined to collect event data from the System and Application channel
config.WindowsEventLog.DataSources.Add("System!*");
config.WindowsEventLog.DataSources.Add("Application!*");
// Use 30 seconds for the perf counter sample rate.
TimeSpan perfSampleRate = TimeSpan.FromSeconds(30D);
config.PerformanceCounters.DataSources.Add(new PerformanceCounterConfiguration()
{
CounterSpecifier = #"\Memory\Available Bytes",
SampleRate = perfSampleRate
});
config.PerformanceCounters.DataSources.Add(new PerformanceCounterConfiguration()
{
CounterSpecifier = #"\Processor(_Total)\% Processor Time",
SampleRate = perfSampleRate
});
config.PerformanceCounters.DataSources.Add(new PerformanceCounterConfiguration()
{
CounterSpecifier = #"\ASP.NET\Applications Running",
SampleRate = perfSampleRate
});
// Start the DiagnosticMonitor using the diagnosticConfig and our connection string.
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString",config);
return base.OnStart();
In the web.config under system.WebServer add the following:
<tracing>
<traceFailedRequests>
<remove path="*"/>
<add path="*">
<traceAreas>
<add provider="ASPNET" areas="Infrastructure,Module,Page,AppServices" verbosity="Verbose" />
<add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module" verbosity="Verbose" />
</traceAreas>
<failureDefinitions statusCodes="400-599" />
</add>
</traceFailedRequests>
</tracing>
In the service definition file add the following under the web role:
<LocalResources>
<LocalStorage name="DiagnosticStore" sizeInMB="8192" cleanOnRoleRecycle="false"/>
</LocalResources>
That should enable all your logging in the MVC application.

Answer 2

Can you try by removing the "timeTaken" attribute from "failureDefinitions" node? Ref: http://msdn.microsoft.com/en-us/library/aa965046(v=VS.90).aspx.

Answer 3

Check the deployment's diagnostics settings by viewing the appropriate file in wad-control-container.
I note that you are not setting what in my experience is all the required values for DiagnosticInfrastructureLogs or for Logs, including bufferQuotaInMB and scheduledTransferLogLevelFilter.
Try this:
// Get the factory configuration so that it can be edited
DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
config.DiagnosticInfrastructureLogs.bufferQuotaInMB = 512;
config.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1D);
config.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
config.Logs.bufferQuotaInMB = 512;
config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
config.Logs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1D);
Try that to start. Ensure as well that you have added trace listeners.