On some machines (probably only on Windows 7 and 2008 R2 but not on Windows 10) I have problem using SemanticLogging. When I run it I'm receiving fallowing output:
Event Trace Session prefix: Microsoft-SemanticLogging-Etw
Sink name: ConsoleEventSink
Event sources:
Name: 8943bf09-be18-551a-efe5-612ee62ded5e
Performance, Level: LogAlways, MatchAnyKeyword: None
Sink name: PerformaceSINK
Event sources:
Name: 8943bf09-be18-551a-efe5-612ee62ded5e
Performance, Level: LogAlways, MatchAnyKeyword: None
Service started.
Press enter to end ...
ERROR: Exception in Command Processing for EventSource Performance: Object
reference not set to an instance of an object.;
All that happens in specific scenerio:
I'm starting process which writes Events
then I run SemanticLogging-svc.exe -c
after few moments error occurs
But when I change order and first start SemanticLogging-svc.exe and after that I run "event writer", everything goes how it should.
But when I setup all as described in first scenerio and after error I will try to collect data using PerfView magic happens and SemanticLogging starts collecting data.
Using PerfView I have checked Microsoft-SemanticLogging-Etw source but nothing there.
SemanticLogging-svc.config:
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw SemanticLogging-svc.xsd">
<traceEventService />
<sinks>
<consoleSink name="ConsoleEventSink">
<sources>
<eventSource name="PerformanceEventSource" level="LogAlways" />
</sources>
<customEventTextFormatter type="ServiceTelemetry.EventFormatter.CsvEventFormatter, ServiceTelemetry"/>
</consoleSink>
<rollingFlatFileSink
name="PerformanceEventSourceSINK"
fileName=".\logs\%ComputerName%_Performance.log"
rollFileExistsBehavior="Increment"
rollInterval="Midnight"
timeStampPattern="yyyyMMdd">
<sources>
<eventSource name="PerformanceEventSource" level="LogAlways" />
</sources>
<customEventTextFormatter type="ServiceTelemetry.EventFormatter.CsvEventFormatter, ServiceTelemetry"/>
</rollingFlatFileSink>
</sinks>
</configuration>
EventFormatter:
namespace ServiceTelemetry.EventFormatter
{
public class CsvEventFormatter : IEventTextFormatter
{
public void WriteEvent(EventEntry eventEntry, TextWriter writer)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < eventEntry.Payload.Count; i++)
{
sb.AppendFormat("{0};", eventEntry.Payload[i]);
}
writer.WriteLine(sb.ToString());
}
}
}
EventSource:
namespace ServiceTelemetry.EventSources
{
[EventSource(Name = "Performance")]
public sealed class PerformanceEventSource : EventSource
{
[Event(1, Level = EventLevel.LogAlways, Task = TaskCodes.GetResource, Opcode = OperationCodes.Compleated)]
public void GetResourceSuccess(string Session, string ResourceName, long ElapsedMilliseconds)
{
if (IsEnabled())
{
WriteEvent(1, Session, ResourceName, ElapsedMilliseconds);
}
}
public static PerformanceEventSource Log = new PerformanceEventSource();
private PerformanceEventSource()
{
}
}
}
It was necessary to install manifest first, then you can start your EventWriter and you can collect data any time you start SematicLogger.
Unfortunately system throws error but for now I'm good with that.
EventSource .net 4.0 GenerateManifest
Related
I couldn't find any information on how to do it. Basically FluentFTP is using System.Diagnostics to log their messages.
FluentFtp expose the following static method:
FtpTrace.AddListener(TraceListener listener);
However I don't know if there is any way to implement (or use existing implementation, which?) TraceListener in the way it relays everything to log4net engine.
Any hints or ideas?
Thanks, Radek
You can attach a listener to the OnLogEvent method that FluentFTP exposes.
private static readonly log4net.ILog Log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
public static void UploadFTP(FileInfo localFile, string remoteFileLocation, string remoteServer, NetworkCredential credentials)
{
FtpClient client = new FtpClient(remoteServer, credentials);
client.RetryAttempts = 3;
client.OnLogEvent = OnFTPLogEvent;
client.Connect();
if (!client.UploadFile(localFile.FullName, remoteFileLocation, FtpExists.Overwrite, false, FtpVerify.Retry | FtpVerify.Throw))
{
throw new Exception($"Could not Upload File {localFile.Name}. See Logs for more information");
}
}
private static void OnFTPLogEvent(FtpTraceLevel ftpTraceLevel, string logMessage)
{
switch (ftpTraceLevel)
{
case FtpTraceLevel.Error:
Log.Error(logMessage);
break;
case FtpTraceLevel.Verbose:
Log.Debug(logMessage);
break;
case FtpTraceLevel.Warn:
Log.Warn(logMessage);
break;
case FtpTraceLevel.Info:
default:
Log.Info(logMessage);
break;
}
}
The method OnFTPLogEvent will be called every-time the OnLogEvent action will be called allowing you to extend any logging you have already built into your application.
Basically FluentFTP is using System.Diagnostics.TraceListener so in order to make it logging to your log4net log you need to write your own simple class that would redirect logs to log4net logger. Like the following:
using System.Diagnostics;
using log4net;
namespace YourApp.Logging
{
public class Log4NetTraceListener : TraceListener
{
private readonly ILog _log;
public Log4NetTraceListener(string provider)
{
_log = LogManager.GetLogger(provider);
}
public override void Write(string message)
{
if(_log == null)
return;
if(!string.IsNullOrWhiteSpace(message))
_log.Info(message);
}
public override void WriteLine(string message)
{
if(_log == null)
return;
if (!string.IsNullOrWhiteSpace(message))
_log.Info(message);
}
}
}
Then, in your app.config file add the following entry:
<system.diagnostics>
<trace autoflush="true"></trace>
<sources>
<source name="FluentFTP">
<listeners>
<clear />
<add name="FluentLog" />
</listeners>
</source>
</sources>
<sharedListeners>
<add name="FluentLog" type="YourApp.Logging.Log4NetTraceListener, YourApp" initializeData="FluentLog" />
</sharedListeners>
</system.diagnostics>
That should enable FluentFtp logs and merge it with your application log4net log.
I am working with an Azure Service Bus Queue (or potentially a topic if required), and would like to know how a Web Job can be used with the Queue.
When a message comes onto the queue it represents a process that will run within the web job (or be started from the webjob). This process might be quick, 30 seconds, or it might be slow, 1 hour etc.
Can I use a single Web Job for this and somehow say that it should be running no more than 10 of these processes at a time?
Yes you can use a WebJob. I have created a simple WebJob with Storage Queue to just guide how it can be done. The below workflow will run only ten process at a time and keep all the other requests in memory of ConcurrentQueue. You will have to implement the logic to dequeue it and consume it
public class Functions
{
public delegate void CompletedProcessHandler(object sender, CompletedProcessHandlerArgs args);
static readonly Dictionary<int, CustomProcess> _dictionary =
new Dictionary<int, CustomProcess>();
static readonly ConcurrentQueue<ProcessEntity> _remaining =
new ConcurrentQueue<ProcessEntity>();
// This function will get triggered/executed when a new message is written
// on an Azure Queue called queue.
public static void ProcessQueueMessage([QueueTrigger("testqueue")] ProcessEntity msg,
TextWriter log)
{
if (_dictionary.Count <= 10)
{
var newProcess = new CustomProcess((_dictionary.Last().Key) + 1,
msg.Duration);
}
else
{
_remaining.Enqueue(msg);
}
}
public static void CompletedProcess(object sender, CompletedProcessHandlerArgs args)
{
_dictionary[Int32.Parse(args.ProcessID)].Dispose();
_dictionary.Remove(Int32.Parse(args.ProcessID));
}
}
public class CustomProcess : IDisposable
{
public event Functions.CompletedProcessHandler OnProcessCompleted;
private CancellationTokenSource _token;
private string _id;
private Timer _timer;
public CustomProcess(int i, int duration)
{
_timer = new Timer { Enabled = true, Interval = duration * 1000 };
_timer.Elapsed += Timer_Elapsed;
_id = i.ToString();
_token = new CancellationTokenSource();
Task.Factory.StartNew(() => WriteMessages());
_timer.Start();
OnProcessCompleted += Functions.CompletedProcess;
}
private void Timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
_token.Cancel();
OnProcessCompleted?.Invoke(this, new CompletedProcessHandlerArgs(_id));
}
private void WriteMessages()
{
while (!_token.Token.IsCancellationRequested)
{
Console.WriteLine("Test Message from process " + _id);
}
}
public void Dispose()
{
_token.Dispose();
_timer.Dispose();
}
}
public class CompletedProcessHandlerArgs : EventArgs
{
public string ProcessID { get; set; }
public CompletedProcessHandlerArgs(string ID)
{
ProcessID = ID;
}
}
public class ProcessEntity
{
public int Duration { get; set; }
}
In the app.config of the web job you need to provide the two app settings
<add name="AzureWebJobsDashboard"
connectionString="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey]" />
<add name="AzureWebJobsStorage"
connectionString="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey]" />
The Program file is the default one from the Visual Studio template
public class Program
{
// Please set the following connection strings in app.config for this WebJob to run:
// AzureWebJobsDashboard and AzureWebJobsStorage
static void Main()
{
var host = new JobHost();
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
}
WebJob will keep dequeue the message the moment it comes. Since you want only 10 to run at a time you will have to enqueue the message in memory and wait for running process to complete before you start a new one
As #Rick has mentioned you can set the is_Singleton property to true in settings.job file of the web job
Yes, you can trigger a web job with an Azure Service Bus Queue or Topic. A good example to look at to get you going would be the Service Bus quick start project template in Visual Studio.
In particular, you want to look at the ServiceBusTrigger attribute that the Web Jobs SDK provides.
As for the scalability of the web job, this will scale according to your web app instances. So, if you had say 5 instances of your web app with always on enabled, then you would have 5 instances of your web job. As an additional comment on this, if you wanted just one instance of the web job in an environment of 5 web app instances, then you could set the is_singleton property to true in the settings.job file.
I've bumped into a following problem with Azure Diagnostic Monitor:
When I create a new AppDomain in OnStart() event in WorkerRole entry point the diagnostics works only in the parent AppDomain. I've tried initializing Diagnostics Monitor in the child AppDomain but it doesn't help. (Traces are collected only from the parent domain)
Example repro code:
public class WorkerRole : RoleEntryPoint
{
public override void Run()
{
// This is a sample worker implementation. Replace with your logic.
InitializeDiagnostics();
Trace.TraceInformation("WorkerRole1 entry point called", "Information");
while (true)
{
Thread.Sleep(10000);
Trace.TraceInformation("Parent domain working", "Information");
}
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
InitializeDiagnostics();
var setup = new AppDomainSetup();
setup.ApplicationBase = AppDomain.CurrentDomain.BaseDirectory;
setup.ConfigurationFile = AppDomain.CurrentDomain.SetupInformation.ConfigurationFile;
var newDomain = System.AppDomain.CreateDomain("NewApplicationDomain",null, setup);
foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies().Where(x => !x.GlobalAssemblyCache))
{
newDomain.Load(assembly.GetName());
}
newDomain.Load(typeof (Worker).Assembly.FullName);
var worker = newDomain.CreateInstanceAndUnwrap(this.GetType().Assembly.FullName, typeof (Worker).FullName) as Worker;
worker.DoWork();
return base.OnStart();
}
public void InitializeDiagnostics()
{
var roleInstanceDiagnosticManager = new RoleInstanceDiagnosticManager(RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"),
RoleEnvironment.DeploymentId,
RoleEnvironment.CurrentRoleInstance
.Role.Name,
RoleEnvironment.CurrentRoleInstance.Id);
var dmc = roleInstanceDiagnosticManager.GetCurrentConfiguration();
var dictionaryConfiguration = new DirectoryConfiguration();
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", dmc);
}
}
public class Worker : MarshalByRefObject
{
public void DoWork()
{
new Task(() =>
{
while (true)
{
Thread.Sleep(1000);
Trace.TraceInformation(AppDomain.CurrentDomain.FriendlyName + " Worker working...", "Information");
}
}).Start();
}
}
}
App config:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.diagnostics>
<trace>
<listeners>
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
name="AzureDiagnostics">
<filter type="" />
</add>
</listeners>
</trace>
</system.diagnostics>
</configuration>
Expected output:
Lots of logged messages:
"{Domain Name} Wokrer working..."
Some
"Parent domain working"
Actual output:
"Parent domain working"
I'm using Azure SDK 2.0. Have any of you came across a similar issue ?
Ok, finally solved it. Upgrading Azure SDK to 2.3 did the thing... It's interesting that messages still doesn't appear in Compute emulator console, but after upgrade they are correctly logged to WADLog table.
Im having some problems with NServiceBus, I can get the pubsub example working fine, but now I'm trying to integrate it into a production project and I cant get the thing to work!
My publisher code is exactly the same as the publisher example (I've just imported the project to rule out any other issues) but I then create a void function and call it from my WPF app and I get a "you cant call bus without creating an instance of bus" error
public void RunTest()
{
var eventMessage = new MarketPriceMessage();
eventMessage.Ticker = "IBM";
eventMessage.DataType = "Bid";
eventMessage.Value = (decimal)23.23423;
eventMessage.EventId = Guid.NewGuid();
eventMessage.Time = DateTime.Now; // > 30 ? (DateTime?)DateTime.Now : null;
eventMessage.Duration = TimeSpan.FromSeconds(99999D);
Bus.Publish(eventMessage);
}
Any ideas as to whats going on there and where I'm going wrong?
Following #Adam's comments below this is the code I'm using internally in my WPF App:
public partial class App : Application
{
public IBus bus { get; set; }
protected override void OnStartup(StartupEventArgs e)
{
base.OnStartup(e);
NServiceBus.Configure.With()
.Log4Net()
.SpringBuilder()
.XmlSerializer()
.MsmqTransport()
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()
.Start();
}
}
}
and
namespace WpfApplication2
{
class EndpointConfig : IConfigureThisEndpoint, AsA_Publisher { }
}
and
namespace WpfApplication2
{
public class SubscriptionAuthorizer : IAuthorizeSubscriptions
{
public bool AuthorizeSubscribe(string messageType, string clientEndpoint, string clientWindowsIdentity, IDictionary<string, string> headers)
{
return true;
}
public bool AuthorizeUnsubscribe(string messageType, string clientEndpoint, string clientWindowsIdentity, IDictionary<string, string> headers)
{
return true;
}
}
}
App Config
<configuration>
<configSections>
<section name="MsmqTransportConfig" type="NServiceBus.Config.MsmqTransportConfig, NServiceBus.Core"/>
<section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core"/>
</configSections>
<MsmqTransportConfig
InputQueue="WpfApplication2InputQueue"
ErrorQueue="error"
NumberOfWorkerThreads="1"
MaxRetries="5"/>
<UnicastBusConfig>
<!--DistributorControlAddress="" DistributorDataAddress="" ForwardReceivedMessagesTo="">-->
<MessageEndpointMappings>
</MessageEndpointMappings>
</UnicastBusConfig>
When I'm stepping through my code I can see that bus is a null object.
I am including the references as normal
I'm not too familiar with WPF, but it looks like there is an Application.Startup event that may work. You need to "manually" configure the bus as shown here in the docs
If you're not using Autofac or some other container, the problem is you skipped the assignment to your bus variable. I normally put this in Global.asax Application_Startup, but this way should work too.
If you are using a container, and you register the class that implements your ServiceContract, you can get away with having a local IBus constructor/property injected when it's instantiated.
public IBus bus { get; set; }
protected override void OnStartup(StartupEventArgs e)
{
base.OnStartup(e);
bus = NServiceBus.Configure.With() // keep a reference to the returned bus.
.Log4Net()
.SpringBuilder()
.XmlSerializer()
.MsmqTransport()
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()
.Start();
}
I've created a simple WF4 console app and set up log4net identically to my other apps. However, when I fire up the console and use the ILog object inside WF4 (I actually pass it into the workflow), no information is presented using my ColoredConsoleAppender. What am I doing wrong?
Workflow trace output is written to trace listeners and as far as I am aware log4net doesn't log the output written to a trace listener by default. I am no expert on log4net so there might be an easier way but creating a TraceListener that just passes all data on to log4net is not hard, the following code worked just fine in a quick test.
public class Log4netTraceListener : TraceListener
{
private static readonly ILog _log = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
public override void TraceData(TraceEventCache eventCache, string source, TraceEventType eventType, int id, params object[] data)
{
base.TraceData(eventCache, source, eventType, id, data);
}
public override void TraceData(TraceEventCache eventCache, string source, TraceEventType eventType, int id, object data)
{
var logger = LogManager.GetLogger(source);
switch (eventType)
{
case TraceEventType.Critical:
logger.Fatal(data);
break;
case TraceEventType.Error:
logger.Error(data);
break;
case TraceEventType.Information:
logger.Info(data);
break;
case TraceEventType.Verbose:
logger.Debug(data);
break;
case TraceEventType.Warning:
logger.Warn(data);
break;
default:
base.TraceData(eventCache, source, eventType, id, data);
break;
}
}
public override void Write(string message)
{
_log.Info(message);
}
public override void WriteLine(string message)
{
_log.Info(message);
}
Next you need to make sure the activity trace information is send to this TraceListener using the following code in you app.config.
<system.diagnostics>
<sources>
<source name="System.Activities"
switchValue="Verbose">
<listeners>
<add name="Test"
type="WorkflowConsoleApplication17.Log4netTraceListener, WorkflowConsoleApplication17"/>
</listeners>
</source>
</sources>
</system.diagnostics>
Create an Extension for your workflow that your activities can get from the context.
var wf = new WorkflowApplication(myActivity);
var log = new MyLogForNetExtensionLol();
wf.Extensions.Add(log);
then, within the activity:
var log = context.GetExtension<ILog>();
log.Write("Worked!");