Events not flowing to Application Insights when using secrets.json in localhost - asp.net-core-2.0

Question

I used this Application Insights getting started guide for setting up my application. I'm running VS2017 version 15.5.7 and my application is using AspNetCore 2.0.0.
When I F5 debug my application using IIS Express from Visual Studio, I see Application Insights events within the debugger Events window. However, the same events are not flowing to Application Insights in Azure; I've configured the InstrumentationKey within secrets.json as you can see below. I've confirmed the key is loaded into my application configuration by setting a breakpoint.
As another debugging data point, I've confirmed events do successfully flow to Application Insights when running in a Azure Web App. This test uses the exact same code. Instead of loading the InstrumentationKey from secrets.json, however, I've configured APPINSIGHTS_INSTRUMENTATIONKEY environment variable to have the key (via ApplicationSettings pane of the Web App in the portal).
I'm at a loss for why my events are not flowing to Application Insights via localhost debugger, but they are when deployed to a Web App.
Program.cs
public static void Main( string[] args )
{
BuildWebHost( args ).Run();
}
public static IWebHost BuildWebHost( string[] args ) =>
WebHost.CreateDefaultBuilder( args )
.UseStartup<Startup>()
.UseApplicationInsights()
.Build();
Startup.cs
var configurationBuilder = new ConfigurationBuilder()
.SetBasePath( hostingEnvironment.ContentRootPath )
.AddJsonFile( $"appsettings.{hostingEnvironment.EnvironmentName}.json", optional: true, reloadOnChange: true );
if ( hostingEnvironment.IsDevelopment() )
{
configurationBuilder.AddUserSecrets<Startup>( optional: true );
}
this.Configuration = configurationBuilder
.AddEnvironmentVariables()
.Build();
secrets.json
{
"ApplicationInsights": {
"InstrumentationKey": "omitted for StackOverflow"
}
}
my.csproj
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.1.1" />
<PackageReference Include="Microsoft.ApplicationInsights.Web" Version="2.5.1" />
<PackageReference Include="Microsoft.AspNetCore" Version="2.0.0" />
launchSettings.json
{
"iisSettings": {
"windowsAuthentication": false,
"anonymousAuthentication": true,
"iisExpress": {
"applicationUrl": "http://localhost:2322/",
"sslPort": 44330
}
},
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"launchUrl": "https://localhost:44330/",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},
"FrontDoor": {
"commandName": "Project",
"launchBrowser": true,
"launchUrl": "https://localhost:44330/api/config",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "http://localhost:2323"
}
}
}

Answer 1

when you see appinsights events in the debugger output window, do those events say (unconfigured) in them? if that is present in the output window, that would mean that the sdk is running in debug, but not finding the instrumentation key in your appsettings.json (or in code) note that the application insights sdks don't look in usersecrets for this setting, only appsettings! (edit 5/28/2020: see OP's edit below, while this was the issue 2 years ago, if you use AddApplicationInsightsTelemetry instead of the obsolete UseApplicationInsights this limitation does not apply)
if the events in the debugger window do not say (unconfigured), the next step is to use something like Fiddler to watch your outbound http traffic, to make sure you're seeing outbound calls by the SDK going to dc.services.visualstudio.com, and that those outbound calls are succeeding. (and at this step, you can verify that the instrumentation key think you are using are using is the one the sdk is using to send the events)
if the events are being sent, and are using the ikey you have set, the last step is to verify that the ikey you're using is the one for the resource you're looking at in the portal. every once in a while, someone has an ikey for "dev" somewhere, and then is looking at a "prod" resource in the portal and not seeing the events they expect.
if you've gotten this far, with the events being sent, sent to the ikey you want, and verified the ikey is for the resource you expect it to be, then verify that there aren't any service outages or delays that might be affecting your data, which you can find at http://aka.ms/aistatus otherwise, you should see events normally in seconds~minutes depending on where in the world you are and what region your resource is in azure, etc.
OP edit for completeness of answer:
Indeed, I was hitting #1 and as John hinted in the comments, I needed to let AI SDK know about my iKey in another way. Instead of using UseApplicationInsights() in Program.cs, I've switched to using AddApplicationInsightsTelemetry(configuration) within StartUp.ConfigureServices(). The configuration passed along has my iKey loaded from secrets.json.

Answer 2

This is a known limitation, being fixed in the coming beta.
https://github.com/microsoft/ApplicationInsights-dotnet/issues/1882
Until then workaround is to use the AddApplicationInsights(IConfiguration) overlaod.

Answer 3

Starting from Microsoft.ApplicationInsights.AspNetCore version 2.15.0 this problem has been fixed.
For lower versions the overload services.AddApplicationInsightsTelemetry(Configuration) would read also from secrets.json.
User secrets and other configuration providers
If you want to store the instrumentation key in ASP.NET Core user
secrets or retrieve it from another configuration provider, you can
use the overload with a
Microsoft.Extensions.Configuration.IConfiguration parameter. For
example, services.AddApplicationInsightsTelemetry(Configuration);.
Starting from Microsoft.ApplicationInsights.AspNetCore version 2.15.0,
calling services.AddApplicationInsightsTelemetry() will automatically
read the instrumentation key from
Microsoft.Extensions.Configuration.IConfiguration of the application.
There is no need to explicitly provide the IConfiguration.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/asp-net-core