Application Insights - Logging exceptions - azure

I wrote a custom logger for Application Insights in my app. I don't see any exceptions or ANY events when viewing App Insights in Azure Portal. Here is the logger class code, when I debug the code I do see a key assigned to the InstrumentationKey property, any ideas what I am doing wrong here? Do I need to attach other info to the client or configuration?
public class AppInsightsLogger:ILogger
{
private TelemetryClient ai;
public AppInsightsLogger()
{
ai = new TelemetryClient();
if (string.IsNullOrEmpty(ai.InstrumentationKey))
{
// attempt to load instrumentation key from app settings
var appSettingsTiKey = AppSettings.InsightsKey;
if (!string.IsNullOrEmpty(appSettingsTiKey))
{
TelemetryConfiguration.Active.InstrumentationKey = appSettingsTiKey;
ai.InstrumentationKey = appSettingsTiKey;
}
else
{
throw new Exception("Could not find instrumentation key for Application Insights");
}
}
}
public void LogException(Exception ex)
{
ai.TrackException(ex);
}
}

I created a new Console Application, installed latest stable ApplicationInsights SDK and pretty much kept your example, with minor but important difference - I either let it wait before shutting down after calling TrackException or added TelemetryClient.Flush()
namespace logtest
{
class Program
{
static void Main(string[] args)
{
AppInsightsLogger logger = new AppInsightsLogger();
logger.LogException(new InvalidOperationException("Is data showing?"));
// either wait for a couple of minutes for the batch to be sent of add ai.Flush() after ai.TrackException() to send the batch immediately
Console.ReadLine();
}
}
public class AppInsightsLogger
{
private TelemetryClient ai;
public AppInsightsLogger()
{
ai = new TelemetryClient();
if (string.IsNullOrEmpty(ai.InstrumentationKey))
{
// attempt to load instrumentation key from app settings
var appSettingsTiKey = "<ikey>";
if (!string.IsNullOrEmpty(appSettingsTiKey))
{
TelemetryConfiguration.Active.InstrumentationKey = appSettingsTiKey;
ai.InstrumentationKey = appSettingsTiKey;
}
else
{
throw new Exception("Could not find instrumentation key for Application Insights");
}
}
}
public void LogException(Exception ex)
{
ai.TrackException(ex);
// ai.Flush();
}
}
}
First I could see telemetry item sent in Visual Studio debug output window:
Then I could see telemetry leaving my machine in Fiddler, I also could see it was successfully accepted by our data collection endpoint.
And finally I could see it in the portal:

Nowadays, the question and the accepted answer are outdated. The contemporary approach is to add Microsoft.ApplicationInsights.AspNet NuGet package, which has logging out-of-the-box.
Please refer to the official docs.
The gist from the docs:
ApplicationInsightsLoggerProvider is enabled by default.
Captured log types can be configured in appsettings.json (or programmatically if you really need it)
"Logging": {
"LogLevel": {
"Default": "Warning"
},
"ApplicationInsights": {
"LogLevel": {
"Default": "Error"
}
}
}

For a version 3 webjob, I tried instantiating a TelemetryClient and calling TrackException(), as well as calling Flush() and waiting up to 180 seconds, but none of that worked. What does work is using the ILogger logger object and passing the exception into LogError().

Related

Where in Azure Portal are the logger.Log statements?

I have a .NET Core WebAPI app. The app is deployed on Azure as an App Service.
In the code, I have enabled Application Insights like so
public static IWebHost BuildWebHost(string[] args) =>
WebHost
.CreateDefaultBuilder(args)
.UseApplicationInsights()
.UseStartup<Startup>()
.ConfigureLogging((hostingContext, logging) =>
{
logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging")).SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Error);
logging.AddApplicationInsights(" xxxxx-xxxx-xxxx-xxxx--xxxxxxxxxxx").SetMinimumLevel(LogLevel.Trace);
})
.Build();
In the constructor of a controller and inside an method of a controller I have these logging statements.
_logger.LogInformation("ApiController, information");
_logger.LogWarning("ApiController, warning");
_logger.LogCritical("ApiController, critical");
_logger.LogWarning("ApiController, error");
_logger.LogDebug("ApiController, debug");
In the Azure Portal I have Application Insights for my App Service enabled. Here a picture from the portal.
App Insights in Azure Portal
But where do I see the logging statements in the Azure portal?
When I go to Application Insights -> Logs and I query by
search *
I can see the requests made to the API but not the logging statements.
Application Insights Log
Where are the log statements?
First, it is not good practice to configure the log level in code. You can easily configure the log level in the appsettings.json file. So in Program.cs -> public static IWebHost BuildWebHost method, change the code to below:
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseApplicationInsights()
.UseStartup<Startup>()
.Build();
Then in appsettings.json(also right click the file -> properties -> set "copy to output directory" to "Copy if newer"):
{
"Logging": {
"IncludeScopes": false,
"ApplicationInsights": {
"LogLevel": {
"Default": "Trace"
}
},
"Console": {
"LogLevel": {
"Default": "Warning"
}
}
},
"ApplicationInsights": {
"InstrumentationKey": "the key"
}
}
In Controller, like ValuesController:
public class ValuesController : Controller
{
private readonly ILogger _logger;
public ValuesController(ILoggerFactory loggerFactory)
{
_logger = loggerFactory.CreateLogger<ValuesController>();
}
// GET api/values
[HttpGet]
public IEnumerable<string> Get()
{
_logger.LogInformation("ApiController, information");
_logger.LogWarning("ApiController, warning");
_logger.LogCritical("ApiController, critical");
_logger.LogWarning("ApiController, error");
_logger.LogDebug("ApiController, debug");
return new string[] { "value1", "value2" };
}
}
Run the project, and wait for a few minutes(application insights would always take 3 to 5 minutes or more to display the data). Then nave to azure portal -> application insights logs, remember that all the logs written by ILogger are stored in "traces" table. Just write the query like "traces" and specify a proper time range, you should see all the logs like below:

how distinguish traces from different instances .net core application in Application Insights

I work on .NET Core 2.2 console application that uses Microsoft.Extensions.Logging and is configured to send logs to Azure Application Insights using Microsoft.ApplicationInsights.Extensibility by:
services.AddSingleton(x =>
new TelemetryClient(
new TelemetryConfiguration
{
InstrumentationKey = "xxxx"
}));
...
var loggerFactory = serviceProvider.GetService<ILoggerFactory>();
loggerFactory.AddApplicationInsights(serviceProvider, logLevel);
It works ok: I can read logs in Application Insights. But the application can be started simultanously in few instances (in different Docker containers). How can I distinguish traces from different instances? I can use source FileName, but I don't know how I should inject it.
I tried to use Scope:
var logger = loggerFactory.CreateLogger<Worker>();
logger.BeginScope(dto.FileName);
logger.LogInformation($"Start logging.");
It's interesting that my configuration is almost identical as in example: https://github.com/MicrosoftDocs/azure-docs/issues/12673
But in my case I can't see the property "FileName" in Application Insights.
For console project, if you want to use the custom ITelemetryInitializer, you should use this format: .TelemetryInitializers.Add(new CustomInitializer());
Official doc is here.
I test it at my side, and it works. The role name can be set.
Sample code is below:
static void Main(string[] args)
{
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = "xxxxx";
configuration.TelemetryInitializers.Add(new CustomInitializer());
var client = new TelemetryClient(configuration);
ServiceCollection services = new ServiceCollection();
services.AddSingleton(x => client);
var provider = services.BuildServiceProvider();
var loggerFactory = new LoggerFactory();
loggerFactory.AddApplicationInsights(provider, LogLevel.Information);
var logger = loggerFactory.CreateLogger<Program>();
logger.LogInformation("a test message 111...");
Console.WriteLine("Hello World!");
Console.ReadLine();
}
Check the role name in azure portal:
If you really have no way to distinguish them you can use a custom telemetry initializer like this:
public class CustomInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
telemetry.Context.Cloud.RoleName = Environment.MachineName;
}
}
and/or you can add a custom property:
public class CustomInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if(telemetry is ISupportProperties)
{
((ISupportProperties)telemetry).Properties["MyIdentifier"] = Environment.MachineName;
}
}
}
In this example I used Environment.MachineName but you can of course use something else if needed. Like this work Id parameter of yours.
the wire it up using:
services.AddSingleton<ITelemetryInitializer, CustomInitializer>();

Is it possible to exclude a url from Application Insights?

We have an Azure web role deployed that has been using Application Insights (ver. 1.0.0.4220), however, we're going over our data quota. Is it possible to configure Application Insights ignore a specific URL?
We have a status web service that gets a huge amount of traffic but never throws any errors. If I could exclude this one service URL I could cut my data usage in half.
Out of the box it is not supported. Sampling feature is coming but that would not be configurable by specific url. You can implement your own channel that would have your custom filtering. Basically your channel will get event to be sent, you check if you want to send it or not and then if yes pass to standard AI channel. Here you can read more about custom channels.
There are two things that changed since this blog post has been written:
channel should implement only ITelemetryChannel interface (ISupportConfiguration was removed)
and instead of PersistenceChannel you should use Microsoft.ApplicationInsights.Extensibility.Web.TelemetryChannel
UPDATE: Latest version has filtering support: https://azure.microsoft.com/en-us/blog/request-filtering-in-application-insights-with-telemetry-processor/
My team had a similiar situation where we needed to filter out urls that were successful image requests (we had a lot of these which made us hit the 30k datapoints/min limit).
We ended up using a modified version of the class in Sergey Kanzhelevs blog post to filter these out.
We created a RequestFilterChannel class which is an instance of ServerTelemetryChannel and extended the Send method. In this method we test each telemetry item to be sent to see if it is an image request and if so, we prevent it from being sent.
public class RequestFilterChannel : ITelemetryChannel, ITelemetryModule
{
private ServerTelemetryChannel channel;
public RequestFilterChannel()
{
this.channel = new ServerTelemetryChannel();
}
public void Initialize(TelemetryConfiguration configuration)
{
this.channel.Initialize(configuration);
}
public void Send(ITelemetry item)
{
if (item is RequestTelemetry)
{
var requestTelemetry = (RequestTelemetry) item;
if (requestTelemetry.Success && isImageRequest((RequestTelemetry) item))
{
// do nothing
}
else
{
this.channel.Send(item);
}
}
else
{
this.channel.Send(item);
}
}
public bool? DeveloperMode
{
get { return this.channel.DeveloperMode; }
set { this.channel.DeveloperMode = value; }
}
public string EndpointAddress
{
get { return this.channel.EndpointAddress; }
set { this.channel.EndpointAddress = value; }
}
public void Flush()
{
this.channel.Flush();
}
public void Dispose()
{
this.channel.Dispose();
}
private bool IsImageRequest(RequestTelemetry request)
{
if (request.Url.AbsolutePath.StartsWith("/image.axd"))
{
return true;
}
return false;
}
}
Once the class has been created you need to add it to your ApplicationInsights.config file.
Replace this line:
<TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel"/>
with a link to your class:
<TelemetryChannel Type="XXX.RequestFilterChannel, XXX" />
Alternatively, you can disable the automated request collection and keep only exception auto-collection, just remove the RequestTrackingModule line from applicationinsights.config.
If you still need to collect some of the requests, not just filter all out, you can then call TrackRequest() (in the object of TelemetryClient class) from your code in the appropriate place after you know that you certainly need to log this request to AI.
Update: Filtering feature has been released some time ago and allows for exclusion of certain telemetry items way easier.

Azure: WebRole OnStart() is never called

I have a simple WebRole class in my Azure solution:
public class WebRole : RoleEntryPoint
{
public override bool OnStart()
{
DiagnosticMonitor.Start("DiagnosticsConnectionString");
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
RoleEnvironment.Changing += RoleEnvironmentChanging;
CloudStorageAccount.SetConfigurationSettingPublisher(
(configName, configSettingPublisher) =>
{
string connectionString = RoleEnvironment.GetConfigurationSettingValue(configName);
configSettingPublisher(connectionString);
}
);
return base.OnStart();
}
// ...
}
For some reason, I can't get breakpoints in OnStart() to be hit when I run the project. Why might this be?
One possibility is that your startup project is your web application instead of the cloud project. Make sure that the cloud project is the startup project, and then verify your app is running in the development fabric.

Spec fails when run by mspec.exe, but passes when run by TD.NET

I wrote about this topic in another question.
However, I've since refactored my code to get rid of configuration access, thus allowing the specs to pass. Or so I thought. They run fine from within Visual Studio using TestDriven.Net. However, when I run them during rake using the mspec.exe tool, they still fail with a serialization exception. So I've created a completely self-contained example that does basically nothing except setup fake security credentials on the thread. This test passes just fine in TD.Net, but blows up in mspec.exe. Does anybody have any suggestions?
Update: I've discovered a work-around. After researching the issue, it seems the cause is that the assembly containing my principal object is not in the same folder as the mspec.exe. When mspec creates a new AppDomain to run my specs, that new AppDomain has to load the assembly with the principal object in order to deserialize it. That assembly is not in the same folder as the mspec EXE, so it fails. If I copied my assembly into the same folder as mspec, it works fine.
What I still don't understand is why ReSharper and TD.Net can run the test just fine? Do they not use mspec.exe to actually run the tests?
using System;
using System.Security.Principal;
using System.Threading;
using Machine.Specifications;
namespace MSpecTest
{
[Subject(typeof(MyViewModel))]
public class When_security_credentials_are_faked
{
static MyViewModel SUT;
Establish context = SetupFakeSecurityCredentials;
Because of = () =>
SUT = new MyViewModel();
It should_be_initialized = () =>
SUT.Initialized.ShouldBeTrue();
static void SetupFakeSecurityCredentials()
{
Thread.CurrentPrincipal = CreatePrincipal(CreateIdentity());
}
static MyIdentity CreateIdentity()
{
return new MyIdentity(Environment.UserName, "None", true);
}
static MyPrincipal CreatePrincipal(MyIdentity identity)
{
return new MyPrincipal(identity);
}
}
public class MyViewModel
{
public MyViewModel()
{
Initialized = true;
}
public bool Initialized { get; set; }
}
[Serializable]
public class MyPrincipal : IPrincipal
{
private readonly MyIdentity _identity;
public MyPrincipal(MyIdentity identity)
{
_identity = identity;
}
public bool IsInRole(string role)
{
return true;
}
public IIdentity Identity
{
get { return _identity; }
}
}
[Serializable]
public class MyIdentity : IIdentity
{
private readonly string _name;
private readonly string _authenticationType;
private readonly bool _isAuthenticated;
public MyIdentity(string name, string authenticationType, bool isAuthenticated)
{
_name = name;
_isAuthenticated = isAuthenticated;
_authenticationType = authenticationType;
}
public string Name
{
get { return _name; }
}
public string AuthenticationType
{
get { return _authenticationType; }
}
public bool IsAuthenticated
{
get { return _isAuthenticated; }
}
}
}
Dan,
thank you for providing a reproduction.
First off, the console runner works differently than the TestDriven.NET and ReSharper runners. Basically, the console runner has to perform a lot more setup work in that it creates a new AppDomain (plus configuration) for every assembly that is run. This is required to load the .dll.config file for your spec assembly.
Per spec assembly, two AppDomains are created:
The first AppDomain (Console) is created
implicitly when mspec.exe is
executed,
a second AppDomain is created by mspec.exe for the assembly containing the specs (Spec).
Both AppDomains communicate with each other through .NET Remoting: For example, when a spec is executed in the Spec AppDomain, it notifies the Console AppDomain of that fact. When Console receives the notification it acts accordingly by writing the spec information to the console.
This communiciation between Spec and Console is realized transparently through .NET Remoting. One property of .NET Remoting is that some properties of the calling AppDomain (Spec) are automatically included when sending notifications to the target AppDomain (Console). Thread.CurrentPrincipal is such a property. You can read more about that here: http://sontek.vox.com/library/post/re-iprincipal-iidentity-ihttpmodule-serializable.html
The context you provide will run in the Spec AppDomain. You set Thread.CurrentPrincipal in the Because. After Because ran, a notification will be issued to the Console AppDomain. The notification will include your custom MyPrincipal that the receiving Console AppDomain tries to deserialize. It cannot do that since it doesn't know about your spec assembly (as it is not included in its private bin path).
This is why you had to put your spec assembly in the same folder as mspec.exe.
There are two possible workarounds:
Derive MyPrincipal and MyIdentity from MarshalByRefObject so that they can take part in cross-AppDomain communication through a proxy (instead of being serialized)
Set Thread.CurrentPrincipal transiently in the Because
(Text is required for formatting to work -- please ignore)
Because of = () =>
{
var previousPrincipal = Thread.CurrentPrincipal;
try
{
Thread.CurrentPrincipal = new MyPrincipal(...);
SUT = new MyViewModel();
}
finally
{
Thread.CurrentPrincipal = previousPrincipal;
}
}
ReSharper, for example, handles all the communication work for us. MSpec's ReSharper Runner can hook into the existing infrastructure (that, AFAIK, does not use .NET Remoting).

Resources