I have a .Net core Web App API which accept request from front-end and then send HTTP POST request to Azure search to get search results.
I just use the build-in application insights logging log basic info in request and dependencies sources with zero logging code.
Now I want to extend the default Application Insights dependencies table to add the request body to Azure search.
What is the easiest way with minimum code?
As per your requirement, you can try the code below:
public class RequestBodyInitializer : ITelemetryInitializer
{
readonly IHttpContextAccessor httpContextAccessor;
public RequestBodyInitializer(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public void Initialize(ITelemetry telemetry)
{
if (telemetry is RequestTelemetry requestTelemetry)
{
if ((httpContextAccessor.HttpContext.Request.Method == HttpMethods.Post ||
httpContextAccessor.HttpContext.Request.Method == HttpMethods.Put) &&
httpContextAccessor.HttpContext.Request.Body.CanRead)
{
const string jsonBody = "JsonBody";
if (requestTelemetry.Properties.ContainsKey(jsonBody))
{
return;
}
//Allows re-usage of the stream
httpContextAccessor.HttpContext.Request.EnableRewind();
var stream = new StreamReader(httpContextAccessor.HttpContext.Request.Body);
var body = stream.ReadToEnd();
//Reset the stream so data is not lost
httpContextAccessor.HttpContext.Request.Body.Position = 0;
requestTelemetry.Properties.Add(jsonBody, body);
}
}
}
and add this to the Startup > ConfigureServices
services.AddSingleton<ITelemetryInitializer, RequestBodyInitializer>();
Related
I created a http trigger V1 azure function on net framework 4.8, and used ILogger for logging. The code is like this.
I checked the Application Insight and queried for traces table. This table contains columns named customDimensions and operation_ParentId. May I ask is there anyway to add custom property in customDimensions column, or set a new Guid value for operation_ParentId? I know that I can use TelemetryClient sdk to create a custom telemetry client for logging. Just curious if there is any easy way which doesn't need to create a new telemetry client, because azure function offers bulit-in integration with application insight.
Also, since azure function runtimes automatically tracks requests, is there any way to change the operation_ParentId and customDimensions for requests table as well? Thanks a lot!
To get both the headers and App Insights to get the custom operation Id, two things must be overridden.
The first is an Activity that wraps the HttpClient, which is responsible for controlling the correlation headers and the other is App Insights' dependency tracing.
Although you can disable Actions completely in your HttpClients, you can just remove the one in the client by setting Activity.Current = null to limit side effects.
var operationId = "CR" + Guid.NewGuid().ToString();
var url = "https://www.microsoft.com";
using (var client = new HttpClient())
{
using (var requestMessage =
new HttpRequestMessage(HttpMethod.Get, url))
{
//Makes the headers configurable
Activity.Current = null;
//set correlation header manually
requestMessage.Headers.Add("Request-Id", operationId);
await client.SendAsync(requestMessage);
}
}
The next step is to remove the App Insights default tracking for this request. Again, you can disable dependency tracking completely, or you can filter out the default telemetry for this request. Processors are registered inside the Startup class just like initializers.
services.AddApplicationInsightsTelemetryProcessor<CustomFilter>();
public class CustomFilter : ITelemetryProcessor
{
private ITelemetryProcessor Next { get; set; }
// next will point to the next TelemetryProcessor in the chain.
public CustomFilter(ITelemetryProcessor next)
{
this.Next = next;
}
public void Process(ITelemetry item)
{
// To filter out an item, return without calling the next processor.
if (!OKtoSend(item)) { return; }
this.Next.Process(item);
}
// Example: replace with your own criteria.
private bool OKtoSend(ITelemetry item)
{
var dependency = item as DependencyTelemetry;
if (dependency == null) return true;
if (dependency.Type == "Http"
&& dependency.Data.Contains("microsoft.com")
//This key is just there to help identify the custom tracking
&& !dependency.Context.GlobalProperties.ContainsKey("keep"))
{
return false;
}
return true;
}
}
Finally, you must inject a telemetry client and call TelemetryClient.TrackDependency() in the method that makes the remote call.
var operationId = "CR" + Guid.NewGuid().ToString();
//setup telemetry client
telemetry.Context.Operation.Id = operationId;
if (!telemetry.Context.GlobalProperties.ContainsKey("keep"))
{
telemetry.Context.GlobalProperties.Add("keep", "true");
}
var startTime = DateTime.UtcNow;
var timer = System.Diagnostics.Stopwatch.StartNew();
//continue setting up context if needed
var url = "https:microsoft.com";
using (var client = new HttpClient())
{
//Makes the headers configurable
Activity.Current = null;
using (var requestMessage =
new HttpRequestMessage(HttpMethod.Get, url))
{
//Makes the headers configurable
Activity.Current = null;
//set header manually
requestMessage.Headers.Add("Request-Id", operationId);
await client.SendAsync(requestMessage);
}
}
//send custom telemetry
telemetry.TrackDependency("Http", url, "myCall", startTime, timer.Elapsed, true);
Refer here more information.
Note: The above is possible by disabling the built-in dependency tracking and App Insights and handling it on your own. But the better approach is let .NET Core & App Insights do the tracking.
Recently we encountered an issue where our servers times out because of huge traffic surge, and those request telemetries were logged into AI as success with response code zero. Is there any way to configure response code zero to be termed as failure. Since request telemetries are captured automatically by AI so we dont have any handle on that
You can do it by using ITelemetryInitializer in your .NET core project.
To be termed as failure when response code is zero, you can set the Success property of request telemetry data as false. The sample code as below(using .NET core 2.2 for this test). And please make sure you're using the latest version of Microsoft.ApplicationInsights.AspNetCore 2.13.1.
Here is the custom ITelemetryInitializer:
public class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (telemetry is RequestTelemetry)
{
var requestTelemetry = telemetry as RequestTelemetry;
//you can change the ResponseCode to "0" in your project
if (requestTelemetry.ResponseCode == "200")
{
// set Success property to false
requestTelemetry.Success = false;
}
}
}
}
then register it in the Startup.cs -> ConfigureServices method:
public void ConfigureServices(IServiceCollection services)
{
//your other code
//here, register the custom ITelemetryInitializer
services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
After executing the code, in azure portal -> your application insights -> Logs, you can see the Success property of request are made as false:
I have a .NET core API that performs HTTP connections to other API. I am able to visualize the outgoing HTTP request in Application Insights, under Dependency Event Types, but it has only basic information. I'm looking on how to add more information about the outgoing HTTP call (like the HTTP headers for instance).
I've looked into https://learn.microsoft.com/en-us/azure/azure-monitor/app/api-custom-events-metrics#trackdependency but I didn't find any concrete way of doing this.
As it has been said, the solution proposed by IvanYang is using the recived request instead of the dependency request.
I've built this ITelemetryInstance for that:
public void Initialize(ITelemetry telemetry)
{
var dependecyTelemetry = telemetry as DependencyTelemetry;
if (dependecyTelemetry == null) return;
if (dependecyTelemetry.TryGetOperationDetail("HttpRequest", out object request)
&& request is HttpRequestMessage httpRequest)
{
foreach (var item in httpRequest.Headers)
{
if (!dependecyTelemetry.Properties.ContainsKey(item.Key))
dependecyTelemetry.Properties.Add(item.Key, string.Join(Environment.NewLine, item.Value));
}
}
if (dependecyTelemetry.TryGetOperationDetail("HttpResponse", out object response)
&& response is HttpResponseMessage httpResponse)
{
var responseBody = httpResponse.Content.ReadAsStringAsync().GetAwaiter().GetResult();
if (!string.IsNullOrEmpty(responseBody))
dependecyTelemetry.Properties.Add("ResponseBody", responseBody);
}
}
This will record all the headers sent to the dependency and also the response received
The other solution given doesn't actually work the way you think it should, since it's attaching the header from the incoming HTTP request to the outgoing dependency request, which is misleading. If you want to attach dependency data to dependency logs then you need to wrap the dependency in a custom dependency wrapper, eg here I'm logging the outgoing payload of the dependency so I can see what's being sent by my system:
Activity activity = null;
IOperationHolder<DependencyTelemetry> requestOperation = null;
if ((request.Method == HttpMethod.Post || request.Method == HttpMethod.Put) && _httpContextAccessor?.HttpContext != null)
{
var bodyContent = await request.Content?.ReadAsStringAsync();
if (!string.IsNullOrEmpty(bodyContent))
{
activity = new Activity("Wrapped POST/PUT operation");
activity.SetTag("RequestBody", bodyContent);
requestOperation = _telemetryClient.StartOperation<DependencyTelemetry>(activity);
}
}
// perform dependency function
httpResponseMessage = await base.SendAsync(request, cancellationToken);
if (activity != null && requestOperation != null)
{
_telemetryClient.StopOperation(requestOperation);
}
I think what you're looking for is ITelemetryInitializer, which can add custom property for dependency telemetry.
And for .net core web project, you can refer to this link.
I write a demo as below:
1.Create a custom ITelemetryInitializer class to collect any dependency data:
public class MyTelemetryInitializer: ITelemetryInitializer
{
IHttpContextAccessor httpContextAccessor;
public MyTelemetryInitializer(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public void Initialize(ITelemetry telemetry)
{
//only add custom property to dependency type, otherwise just return.
var dependencyTelemetry = telemetry as DependencyTelemetry;
if (dependencyTelemetry == null) return;
if (!dependencyTelemetry.Context.Properties.ContainsKey("custom_dependency_headers_1"))
{
//the comment out code use to check the fields in Headers if you don't know
//var s = httpContextAccessor.HttpContext.Request.Headers;
//foreach (var s2 in s)
//{
// var a1 = s2.Key;
// var a2 = s2.Value;
//}
dependencyTelemetry.Context.Properties["custom_dependency_headers_1"] = httpContextAccessor.HttpContext.Request.Headers["Connection"].ToString();
}
}
}
2.Then in the Startup.cs -> ConfigureServices method:
public void ConfigureServices(IServiceCollection services)
{
//other code
//add this line of code here
services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
3.Test result, check if the custom property is added to azure portal -> Custom Properties:
I have implemented a custom authentication scheme in a web service based on the ASP.NET Core webhost. I want to add Application Insights to this service.
When I successfully authenticate the user, I do something like this
telemetry.Context.User.Id = authenticatedUserName;
the telemetry object is the TelemetryClient I get from dependency injection.
Now, the problem is that the user ID does not show up among the requests, and I am not sure why.
This works
customEvents | where user_Id != "" and name == "MyCustomEvent"
but not this
request | where user_Id != ""
or this
dependencies | where user_Id != ""
Is there somewhere else where I should set the user ID for the request? I'd rather not create a custom event just for this.
I also tried setting the User property on the HttpContext object, but it does not seem to have any effect.
You should use ITelemetryInitializer for your purpose.
The following is my test steps(asp.net core 2.1):
Step 1:Add the Aplication Insights telemetry by right click your project -> Add -> Application Insights telemetry. The screenshot as below:
Step 2:Add a new class which implements the ITelemetryInitializer:
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
namespace WebApplication33netcore
{
public class MyTelemetryInitializer: ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var request = telemetry as RequestTelemetry;
if (request != null)
{
//set the user id here with your custom value
request.Context.User.Id = "ivan111";
}
}
}
}
Step 3:Register your telemetry initializer in ConfigureServices method in Startup.cs. For details, refer to here:
public void ConfigureServices(IServiceCollection services)
{
services.Configure<CookiePolicyOptions>(options =>
{
options.CheckConsentNeeded = context => true;
options.MinimumSameSitePolicy = SameSiteMode.None;
});
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
//Add the following code to register your telemetry initializer
services.AddSingleton<ITelemetryInitializer>(new MyTelemetryInitializer());
}
Step 4:Check the test result:
In visual studio Application Insights Search:
Then check it in Analytics:
Actually, the answer was surprisingly simple.
HttpContext ctx = ...
var requestTelemetry = ctx.Features.Get<RequestTelemetry>()
requestTelemetry.Context.User.Id = authenticationResult.UserName;
I know that the services get wired-up by instantiating the BasicAppHost, and the IoC by using the ConfigureContainer property, but where is the right place to add the filters? The test in question never fire the global filter:
[TestFixture]
public class IntegrationTests
{
private readonly ServiceStackHost _appHost;
public IntegrationTests()
{
_appHost = new BasicAppHost(typeof(MyServices).Assembly)
{
ConfigureContainer = container =>
{
//
}
};
_appHost.Plugins.Add(new ValidationFeature());
_appHost.Config = new HostConfig { DebugMode = true };
_appHost.GlobalRequestFilters.Add(ITenantRequestFilter);
_appHost.Init();
}
private void ITenantRequestFilter(IRequest req, IResponse res, object dto)
{
var forTennant = dto as IForTenant;
if (forTennant != null)
RequestContext.Instance.Items.Add("TenantId", forTennant.TenantId);
}
[TestFixtureTearDown]
public void TestFixtureTearDown()
{
_appHost.Dispose();
}
[Test]
public void CanInvokeHelloServiceRequest()
{
var service = _appHost.Container.Resolve<MyServices>();
var response = (HelloResponse)service.Any(new Hello { Name = "World" });
Assert.That(response.Result, Is.EqualTo("Hello, World!"));
}
[Test]
public void CanInvokeFooServiceRequest()
{
var service = _appHost.Container.Resolve<MyServices>();
var lead = new Lead
{
TenantId = "200"
};
var response = service.Post(lead); //Does not fire filter.
}
}
ServiceStack is set at 4.0.40
Updated
After perusing the ServiceStack tests (which I highly recommend BTW) I came across a few example of the AppHost being used AND tested. It looks like the "ConfigureAppHost" property is the right place to configure the filters, e.g.
ConfigureAppHost = host =>
{
host.Plugins.Add(new ValidationFeature());
host.GlobalRequestFilters.Add(ITenantRequestFilter);
},
ConfigureContainer = container =>
{
}
Updated1
And they still don't fire.
Updated2
After a bit of trial and error I think it's safe to say that NO, the filters are not hooked up while using the BasicAppHost. What I have done to solve my problem was to switch these tests to use a class that inherits from AppSelfHostBase, and use the c# servicestack clients to invoke the methods on my service. THIS does cause the global filters to be executed.
Thank you,
Stephen
No the Request and Response filters only fire for Integration Tests where the HTTP Request is executed through the HTTP Request Pipeline. If you need to test the full request pipeline you'd need to use a Self-Hosting Integration test.
Calling a method on a Service just does that, i.e. it's literally just making a C# method call on a autowired Service - there's no intermediate proxy magic intercepting the call in between.