I have followed the guidance at https://learn.microsoft.com/en-us/windows/apps/trace-processing/extensibility to get my hands on the .NET Runtime events. When I get a EventContext instance with the unparsed data I have no convenient way to parse things further?
Ideally there should be a parser generator for manifest based events like it is the case with TraceEvent. Something like
TraceProcessorGen -generateEvents c:\Windows\Microsoft.NET\Framework\v4.0.30319\CLR-ETW.man
would be a big help. I am not inclinded to manually write the parsing code with hundreds of hard coded offsets for dozens of events.
class ClrDataSource : IFilteredEventConsumer
{
public IReadOnlyList<Guid> ProviderIds { get; } = new Guid[] { new Guid("e13c0d23-ccbc-4e12-931b-d9cc2eee27e4") };
public int Count { get; private set; }
public void Process(EventContext eventContext)
{
ReadOnlySpan<byte> data = eventContext.Event.Data;
// What do do next?
}
}
TraceEvent from Vance Morrison has an easy way to explore manifest based events where the Payload and PayloadNames are already preparsed based on their manifest. This is not very performant but for many cases and explorative research very helpful.
(I am a developer at Microsoft who works on the TraceProcessor project.)
IFilteredEventConsumer is a way to get at the unparsed events in the trace, and it's true that we have not added support for using a manifest file to simplify that parsing.
However, parsed events for that provider should be available in the IGenericEventDataSource like this:
using (ITraceProcessor trace = TraceProcessor.Create(tracePath))
{
Guid[] providerIds = new[] { Guid.Parse("e13c0d23-ccbc-4e12-931b-d9cc2eee27e4") };
IPendingResult<IGenericEventDataSource> pendingEventsData = trace.UseGenericEvents(providerIds);
trace.Process();
IGenericEventDataSource eventData = pendingEventsData.Result;
foreach (IGenericEvent genericEvent in eventData.Events)
{
// Process event here
}
}
Within each IGenericEvent, there is a property called Fields, which should let you access events either by integer index or by name.
Related
In a Controller-Service-Datalayer architecture, I'm searching for a way to verify that my controller methods perform exactly one call to the service layer like this:
#DeleteMapping(value = "/{id}")
public ResponseEntity<String> deleteBlubber(#PathVariable("id") long blubberId) {
service.deleteBlubber(blubberId);
return new ResponseEntity<>("ok", HttpStatus.OK);
}
This should not be allowed:
#DeleteMapping(value = "/{id}")
public ResponseEntity<String> deleteBlubber(#PathVariable("id") long blubberId) {
service.deleteOtherStuffFirst(); // Opens first transaction
service.deleteBlubber(blubberId); // Opens second transaction - DANGER!
return new ResponseEntity<>("ok", HttpStatus.OK);
}
As you can see from the comments, the reason for this is to make sure that each request is handled in one transaction (that is started in the service layer), not multiple transactions.
It seems that ArchUnit can only check meta-data from classes and methods and not what's actually going on in a method. I would have to be able to count the request to the service classes, which seems to not be possible in ArchUnit.
Any idea if this might be possible? Thanks!
With JavaMethod.getMethodCallsFromSelf() you have access to all methods calls of a given method. This could be used inside a custom ArchCondition like this:
methods()
.that().areDeclaredInClassesThat().areAnnotatedWith(Controller.class)
.should(new ArchCondition<JavaMethod>("call exactly one service method") {
#Override
public void check(JavaMethod item, ConditionEvents events) {
List<JavaMethodCall> serviceCalls = item.getMethodCallsFromSelf().stream()
.filter(call -> call.getTargetOwner().isAnnotatedWith(Service.class))
.toList();
if (serviceCalls.size() != 1) {
String message = serviceCalls.stream().map(JavaMethodCall::getDescription).collect(joining(" and "));
events.add(SimpleConditionEvent.violated(item, message));
}
}
})
I need your help.
I have created a new screen, where I am calling all invoices pending release.
I have problems to release, I send a message where you request (you want to release).
It shows me the infinite message.
Only once should you ask me, then you should go out and follow the normal process.
public ProcessDocNew()
{
// Acuminator disable once PX1008 LongOperationDelegateSynchronousExecution [Justification]
Document.SetProcessDelegate(
delegate (List<ARInvoice> list)
{
List<ARRegister> newlist = new List<ARRegister>(list.Count);
foreach (ARInvoice doc in list)
{
newlist.Add(doc);
}
ProcessDoc(newlist, true);
}
);
Document.SetProcessCaption(ActionsMensje.Process);
Document.SetProcessAllCaption(ActionsMensje.ProcessAll);
}
public virtual void ProcessDoc(List<ARRegister> list, bool isMassProcess)
{
string title = "Test";
string sms = "¿Stamp?";
var Graph = PXGraph.CreateInstance<ARInvoiceEntry>();
ARInvoice document = Document.Current;
PEFEStampDocument timbrar = new PEFEStampDocument();/*This is a class where it is, all my method*/
if (isMassProcess == true)
{
Document.Ask(title, sms, MessageButtons.YesNo, MessageIcon.Question);
{
PXLongOperation.StartOperation(Graph, delegate
{
timbrar.Stamp(document, Graph); /*here I have my release method*/
});
}
}
}
public static class ActionsMensje
{
public const string Process = "Process";
public const string ProcessAll = "Process All";
}
I await your comments
Only once should you ask me, then you should go out and follow the
normal process.
That is not how the processing pattern works. The process delegate is called for each record and is therefore not a valid location to display a message that should be shown only once.
You would need to add a custom action to achieve that behavior. The scenario you're looking for should be implemented with a processing filter checkbox and processing filter to comply with best practices:
Documentation on processing screens implementation is available here:
https://help-2019r2.acumatica.com/Help?ScreenId=ShowWiki&pageid=a007b57b-af69-4c0f-9fd1-f5d98351035f
I am using the latest version of CsvHelper library and yet Azure WebJob making troubles reading my csv file.
It says not valid DateTime format. Though I am 101% sure that my csv has the right DateTime format.
I believe that the CsvHelper causes the issue but I don't really know..
If more information is needed please let me know.
For such issue, it would be diffcult to us to answer if we don't have a sample that can reproduce the issue. So here I will just share you an effective way to troubleshoot by yourself.
First of all, the issue is not caused by web job, so you should debug it locally first.
1, Create a simple console app for test.
2, Download the source code of CsvHelper library, add the source project in same solution in VS then reference CsvHelper project like below:
3, In your test app, input some code with same functionality you use in web job. The below is what i use for demo. Foo class contains a DateTime property:
class Program
{
static void Main(string[] args)
{
using (var reader = new StreamReader(#"C:\Users\toml\Desktop\test.csv"))
using (var csv = new CsvReader(reader))
{
csv.Configuration.BadDataFound = null;
var records = csv.GetRecords<Foo>();
foreach (var item in records)
{
Console.WriteLine(item.Time);
}
}
Console.ReadKey();
}
}
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
public DateTime Time { get; set; }
}
4, According the stack trace, the error occurs on ConvertFromString method which is in TypeConversion folder -> DateTimeConverter.cs file.
Open this file and set breakpoint like below:
The last method that throws exception is DateTime.Parse( text, formatProvider, dateTimeStyle ).
Now Run the project and check if each parametor, espectially "text", is expected.
It will provide more information for you to debug.
We have an Azure web role deployed that has been using Application Insights (ver. 1.0.0.4220), however, we're going over our data quota. Is it possible to configure Application Insights ignore a specific URL?
We have a status web service that gets a huge amount of traffic but never throws any errors. If I could exclude this one service URL I could cut my data usage in half.
Out of the box it is not supported. Sampling feature is coming but that would not be configurable by specific url. You can implement your own channel that would have your custom filtering. Basically your channel will get event to be sent, you check if you want to send it or not and then if yes pass to standard AI channel. Here you can read more about custom channels.
There are two things that changed since this blog post has been written:
channel should implement only ITelemetryChannel interface (ISupportConfiguration was removed)
and instead of PersistenceChannel you should use Microsoft.ApplicationInsights.Extensibility.Web.TelemetryChannel
UPDATE: Latest version has filtering support: https://azure.microsoft.com/en-us/blog/request-filtering-in-application-insights-with-telemetry-processor/
My team had a similiar situation where we needed to filter out urls that were successful image requests (we had a lot of these which made us hit the 30k datapoints/min limit).
We ended up using a modified version of the class in Sergey Kanzhelevs blog post to filter these out.
We created a RequestFilterChannel class which is an instance of ServerTelemetryChannel and extended the Send method. In this method we test each telemetry item to be sent to see if it is an image request and if so, we prevent it from being sent.
public class RequestFilterChannel : ITelemetryChannel, ITelemetryModule
{
private ServerTelemetryChannel channel;
public RequestFilterChannel()
{
this.channel = new ServerTelemetryChannel();
}
public void Initialize(TelemetryConfiguration configuration)
{
this.channel.Initialize(configuration);
}
public void Send(ITelemetry item)
{
if (item is RequestTelemetry)
{
var requestTelemetry = (RequestTelemetry) item;
if (requestTelemetry.Success && isImageRequest((RequestTelemetry) item))
{
// do nothing
}
else
{
this.channel.Send(item);
}
}
else
{
this.channel.Send(item);
}
}
public bool? DeveloperMode
{
get { return this.channel.DeveloperMode; }
set { this.channel.DeveloperMode = value; }
}
public string EndpointAddress
{
get { return this.channel.EndpointAddress; }
set { this.channel.EndpointAddress = value; }
}
public void Flush()
{
this.channel.Flush();
}
public void Dispose()
{
this.channel.Dispose();
}
private bool IsImageRequest(RequestTelemetry request)
{
if (request.Url.AbsolutePath.StartsWith("/image.axd"))
{
return true;
}
return false;
}
}
Once the class has been created you need to add it to your ApplicationInsights.config file.
Replace this line:
<TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel"/>
with a link to your class:
<TelemetryChannel Type="XXX.RequestFilterChannel, XXX" />
Alternatively, you can disable the automated request collection and keep only exception auto-collection, just remove the RequestTrackingModule line from applicationinsights.config.
If you still need to collect some of the requests, not just filter all out, you can then call TrackRequest() (in the object of TelemetryClient class) from your code in the appropriate place after you know that you certainly need to log this request to AI.
Update: Filtering feature has been released some time ago and allows for exclusion of certain telemetry items way easier.
I upgraded to Azure SDK 2.5 and switched to semantic logging with EventSources.
Logging works locally with a custom EventListener.
When deployed, logs are written to a storage table, but only the EventId, Pid, Tid etc. are populated, the really interesting fields (Message, Task, Keyword, Opcode) are left blank.
The diagnostics infrastructure log is full of errors with regards to ETW, but I don't know what to make of them:
Failed to load backup EventSource manifest file C:\Resources\{13b7ec61-6424-d4d3-9972-a83e58d8d6bb}\directory\f71b19461fcf494d89d3717b3a13cadf. something.WorkerRole.DiagnosticStore\WAD0103\Configuration\EventSource_Manifest_fe06b63d-39aa-5419-0529-18c4dacf4f68_Ver_20.backup.xml;
EventSource events will be logged without a proper schema until provider sends the manifest packets
Load manifest file failed for C:\Resources\{13b7ec61-6424-d4d3-9972-a83e58d8d6bb}\directory\f71b19461fcf494d89d3717b3a13cadf.something. WorkerRole. DiagnosticStore\WAD0103\Configuration\EventSource_Manifest_fe06b63d-39aa-5419-0529-18c4dacf4f68_Ver_20.xml
Failed to manage manifest version for file C:\Resources\{13b7ec61-6424-d4d3-9972-a83e58d8d6bb}\directory\f71b19461fcf494d89d3717b3a13cadf. something. WorkerRole.DiagnosticStore\WAD0103\Configuration\EventSource_Manifest_fe06b63d-39aa-5419-0529-18c4dacf4f68_Pid_3436.xml
Failed to process EventSource manifest event GUID:fe06b63d-39aa-5419-0529-18c4dacf4f68, event id:0xFFFE
Change in the number of events lost since the last sample: EventsCaptured=2 EventsLogged=1 EventsLost=0
I do not use a manifest file and specify the EventSource via class / attribute name:
<EtwEventSourceProviderConfiguration scheduledTransferPeriod="PT3M" scheduledTransferLogLevelFilter="Information" provider="something.Core">
<DefaultEvents eventDestination="CoreEvents" />
</EtwEventSourceProviderConfiguration>
I must be missing something, but I do not know what.
The remaining diagnostic services all work (infrastructure logs, performance counter etc.).
The EventId that is being logged is the correct one, but all the important information of the log is missing, I suppose because of an incomplete configuration?
Edit: here is my EventSource code. I won't post the entire thing because it's quite large. I use another type that calls the EventSource methods and handles formatting of parameters (if the source is enabled in that level). Most method arguments are of type string, there are no objects or other complex types passed around (that handles the other type).
[EventSource(Name = "something.Core")]
public sealed class CoreEventSource : EventSource {
private static readonly CoreEventSource SoleInstance = new CoreEventSource();
static CoreEventSource() {}
private CoreEventSource() {}
public static CoreEventSource Instance {
get { return SoleInstance; }
}
public static EventKeywords AllKeywords = (EventKeywords)(-1);
public class Keywords {
public const EventKeywords None = (EventKeywords)(1 << 1);
public const EventKeywords Infrastructure = (EventKeywords)(1 << 2);
[...]
}
public class Tasks {
public const EventTask None = EventTask.None;
// generic operations
public const EventTask Create = (EventTask)11;
public const EventTask Update = (EventTask)12;
public const EventTask Delete = (EventTask)13;
public const EventTask Get = (EventTask)14;
public const EventTask Put = (EventTask)15;
public const EventTask Remove = (EventTask)16;
public const EventTask Process = (EventTask)17;
}
[Event(1, Message = "Initialization of {0} failed: {1}.", Level = EventLevel.Critical, Keywords = Keywords.Infrastructure)]
public void CriticalInitializationFailure(string component, string details, string exception) {
this.WriteEvent(1, component, details, exception);
}
[Event(2, Message = "[Role '{0}'] Startup: {1}", Level = EventLevel.Informational, Keywords = Keywords.Infrastructure)]
public void RoleStartup(string roleName, string message) {
this.WriteEvent(2, roleName, message);
}
[Event(3, Message = "[Role '{0}'] Stop failed: {1}.", Level = EventLevel.Error, Keywords = Keywords.Infrastructure)]
public void RoleStopFailed(string roleName, string details, string exception) {
this.WriteEvent(3, roleName, details, exception);
}
[Event(4, Message = "An unhandled exception occurred.", Level = EventLevel.Critical, Keywords = Keywords.Infrastructure)]
public void UnhandledException(string exception) {
this.WriteEvent(4, exception);
}
[Event(5, Message = "An unobserved exception occurred in a faulted task.", Level = EventLevel.Critical, Keywords = Keywords.Infrastructure)]
public void UnobservedTaskException(string exception) {
this.WriteEvent(5, exception);
}
[...]
}
Turns out there were quite a few problems with my EventSource. The first thing I'd recommend to anyone working with ETW is to use the Microsoft TraceEvent Library from NuGet, even if you use System.Diagnostics.Tracing, because it comes with a tool that will verify your EventSource code and notify you about problems.
I had to fix the following:
EventSource names must not contain a period .
Task/Opcode pairs must be unique within an EventSource
One must not declare a None field in a custom Keywords or Tasks enumeration
Hope this is of some use to anyone who encounters a similar problem.
Another thing that should be taken care of (which fixed our case)
- EventSources should only have a Name or a Guid, not both.
In our case, having both caused
- The EtwEventSourceProvider to not log anything
- The EtwEventManifestProvider to log the same way you outlined, with empty data points.