I'm using log4net with AdoNetAppender. It's seems that the AdoNetAppender has a Flush method. Is there anyway I can call that from my code?
I'm trying to create an admin page to view all the entries in the database log, and I will like to setup log4net with bufferSize=100 (or more), then I want the administrator to be able to click an button on the admin page to force log4net to write the buffered log entries to the database (without shutting down log4net).
Is that possible?
Assuming you're using log4net out of the box, you can dig your way down & flush the appender like this:
public void FlushBuffers()
{
ILog log = LogManager.GetLogger("whatever");
var logger = log.Logger as Logger;
if (logger != null)
{
foreach (IAppender appender in logger.Appenders)
{
var buffered = appender as BufferingAppenderSkeleton;
if (buffered != null)
{
buffered.Flush();
}
}
}
}
Edit: I wrote the above under the assumption that you wanted to flush the appenders for a specific ILog (probably a bad assumption now that I re-read the question), but as Stefan points out in a comment below, you can simplify the code a little if you want to flush all appenders across the whole repository as follows:
public void FlushBuffers()
{
ILoggerRepository rep = LogManager.GetRepository();
foreach (IAppender appender in rep.GetAppenders())
{
var buffered = appender as BufferingAppenderSkeleton;
if (buffered != null)
{
buffered.Flush();
}
}
}
Today simpler option is available:
LogManager.Flush();
Flushes logging events buffered in all configured appenders in the default repository.
https://logging.apache.org/log4net/release/sdk/html/M_log4net_LogManager_Flush.htm
It is highly recommended to add a timeout, like
LogManager.Flush(3000);
Even if Flush is used or ImmediateFlush is set, changes are not reflected immediately in the log file. In order for the FileSystemWatcher events to fire, you can do this
if (appender.ImmediateFlush)
{
using (FileStream fs = new FileStream(appender.File,
FileMode.Open, FileAccess.Read, ileShare.ReadWrite))
{ }
}
Related
I am new with log4net and am wondering how to properly be able to log. I have seen examples of people doing Logs.Enter, Logs.Exit. It properly logs and helps users when testing/debugging. Examples of how to define is greatly appreciated
You define a log4net log like so:
private static log4net.ILog log = log4net.LogManager.GetLogger(ConfigurationManager.AppSettings["LogName"]);
I defined it so that it is defined in app config. Afterwards, I define methods to determine the type of log i.e. Error, Info, Enter, Exit etc.
Enter/Exit method example:
[Conditional("ENTEREXIT")]
public static void Enter()
{
_log.Info(GetMethodName() + ": In");
}
[Conditional("ENTEREXIT")]
public static void Exit()
{
_log.Info(GetMethodName() + ": Out");
}
Save this in a separate class called Logs.cs
Then, whenever you want to log, you simple do Logs.Enter and then stop logging with Logs.Exit. You can do Logs.Info or Logs.Error without the Enter/Exit Logs.
I have three queues and one worker that I want monitoring the three queues (or only two of them)
One queue is qPirate
One queue is qShips
One queue is qPassengers
The idea is that workers will either be looking at all 3 of them, 2 of them, or one of them, and doing different things depending on what the message says.
The key though is that say a message is failing because ship1 is offline, all queues in qships will refresh, workers that are looking at that and other queues will get hung up slightly from it as they will try to process the messages for that queue while only looking at the other queues a little bit, while the other workers that are looking at the other 2 queues and skipping qships will continue to process through messages without holdup or delays.
public static void GotMessage([ServiceBusTrigger("%LookAtAllQueuesintheservicebus%")] BrokeredMessage message)
{
var handler = new MessageHandler();
var manager = new MessageManager(
handler,
"PirateShips"
);
manager.ProcessMessageViaHandler(message);
}
Looking around online I'm guessing this isn't something that's possible, but it seems like it would be? Thanks in advance either way!
Edit1: I'll add the Job Host as well to attempt to clarify things a bit
JobHostConfiguration config = new JobHostConfiguration()
{
DashboardConnectionString = "DefaultEndpointsProtocol=https;AccountName=PiratesAreUs;AccountKey=Yarr",
StorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=PiratesAreUs;AccountKey=Yarr",
NameResolver = new QueueNameResolver()
};
ServiceBusConfiguration serviceBusConfig = new ServiceBusConfiguration()
{
ConnectionString = "Endpoint=AllPirateQueuesLocatedHere;SharedAccessKeyName=PiratesAreUs;SharedAccessKey=Yarr"
};
serviceBusConfig.MessageOptions.AutoComplete = false;
serviceBusConfig.MessageOptions.AutoRenewTimeout = TimeSpan.FromMinutes(1);
serviceBusConfig.MessageOptions.MaxConcurrentCalls = 1;
config.UseServiceBus(serviceBusConfig);
JobHost host = new JobHost(config);
host.RunAndBlock();
Also the QueueNameResolverClass is simply
public class QueueNameResolver : INameResolver
{
public string Resolve(string name)
{
return name;
}
}
I don't appear to have anyway to have the NameResolver be multiple queues, while I can say that I want the jobhost to look at a certain ServiceBus, I don't know how to tell it to look at all the queues within the ServiceBus.
In other words, I want multiple servicebustriggers on this worker so that if a message gets sent to qpirate1 and qships1 which are both located in service bus AllPirateQueuesHere, the worker can pick up the message in qpirate1, process it, then pick up the message in qships1 and process it.
Figured out the answer... This is possible and its simpler than I thought I'm not sure why I didn't connect the dots but I'm still curious why there isn't more documentation about this. Apparently it's simply make a function per queue you want a worker to look at multiple queues. So if you had three queues you'd want something like the below (you can handle each message differently).
public static void GotMessage1([ServiceBusTrigger("%qPirate1%")] BrokeredMessage message)
{
var handler = new MessageHandler();
var manager = new MessageManager(
handler,
"Pirates"
);
manager.ProcessMessageViaHandler(message);
}
public static void GotMessage2([ServiceBusTrigger("%qShip1%")] BrokeredMessage message)
{
var handler = new MessageHandler();
var manager = new MessageManager(
handler,
"Ships"
);
manager.ProcessMessageViaHandler(message);
}
public static void GotBooty([ServiceBusTrigger("%qBooty%")] BrokeredMessage message)
{
var handler = new MessageHandler();
var manager = new MessageManager(
handler,
"Booty"
);
manager.ProcessMessageViaHandler(message);
}
We are using InMemoryTransientMessageService to chain several one-way notification between services. We can not use Redis provider, and we do not really need it so far. Synchronous dispatching is enough.
We are experimenting problems when using a publish inside a service that is handling another publish. In pseudo-code:
FirstService.Method()
_messageQueueClient.Publish(obj);
SecondService.Any(obj)
_messageQueueClient.Publish(obj);
ThirdService.Any(obj)
The SecondMessage is never handled. In the following code of ServiceStack TransientMessageServiceBase, when the second message is processed, the service "isRunning" so it does not try to handled the second:
public virtual void Start()
{
if (isRunning) return;
isRunning = true;
this.messageHandlers = this.handlerMap.Values.ToList().ConvertAll(
x => x.CreateMessageHandler()).ToArray();
using (var mqClient = MessageFactory.CreateMessageQueueClient())
{
foreach (var handler in messageHandlers)
{
handler.Process(mqClient);
}
}
this.Stop();
}
I'm not sure about the impact of changing this behaviour in order to be able to nest/chain message publications. Do you think it is safe to remove this check? Some other ideas?
After some tests, it seems there is no problem in removing the "isRunning" control. All nested publications are executed correctly.
I'm implementing my own logging framework. Following is my BaseLogger which receives the log entries and push it to the actual Logger which implements the abstract Log method.
I use the C# TPL for logging in an Async manner. I use Threads instead of TPL. (TPL task doesn't hold a real thread. So if all threads of the application end, tasks will stop as well, which will cause all 'waiting' log entries to be lost.)
public abstract class BaseLogger
{
// ... Omitted properties constructor .etc. ... //
public virtual void AddLogEntry(LogEntry entry)
{
if (!AsyncSupported)
{
// the underlying logger doesn't support Async.
// Simply call the log method and return.
Log(entry);
return;
}
// Logger supports Async.
LogAsync(entry);
}
private void LogAsync(LogEntry entry)
{
lock (LogQueueSyncRoot) // Make sure we ave a lock before accessing the queue.
{
LogQueue.Enqueue(entry);
}
if (LogThread == null || LogThread.ThreadState == ThreadState.Stopped)
{ // either the thread is completed, or this is the first time we're logging to this logger.
LogTask = new new Thread(new ThreadStart(() =>
{
while (true)
{
LogEntry logEntry;
lock (LogQueueSyncRoot)
{
if (LogQueue.Count > 0)
{
logEntry = LogQueue.Dequeue();
}
else
{
break;
// is it possible for a message to be added,
// right after the break and I leanve the lock {} but
// before I exit the loop and task gets 'completed' ??
}
}
Log(logEntry);
}
}));
LogThread.Start();
}
}
// Actual logger implimentations will impliment this method.
protected abstract void Log(LogEntry entry);
}
Note that AddLogEntry can be called from multiple threads at the same time.
My question is, is it possible for this implementation to lose log entries ?
I'm worried that, is it possible to add a log entry to the queue, right after my thread exists the loop with the break statement and exits the lock block, and which is in the else clause, and the thread is still in the 'Running' state.
I do realize that, because I'm using a queue, even if I miss an entry, the next request to log, will push the missed entry as well. But this is not acceptable, specially if this happens for the last log entry of the application.
Also, please let me know whether and how I can implement the same, but using the new C# 5.0 async and await keywords with a cleaner code. I don't mind requiring .NET 4.5.
Thanks in Advance.
While you could likely get this to work, in my experience, I'd recommend, if possible, use an existing logging framework :) For instance, there are various options for async logging/appenders with log4net, such as this async appender wrapper thingy.
Otherwise, IMHO since you're going to be blocking a threadpool thread during your logging operation anyway, I would instead just start a dedicated thread for your logging. You seem to be kind-of going for that approach already, just via Task so that you'd not hold a threadpool thread when nothing is logging. However, the simplification in implementation I think benefits just having the dedicated thread.
Once you have a dedicated logging thread, you then only need have an intermediate ConcurrentQueue. At that point, your log method just adds to the queue and your dedicated logging thread just does that while loop you already have. You can wrap with BlockingCollection if you need blocking/bounded behavior.
By having the dedicated thread as the only thing that writes, it eliminates any possibility of having multiple threads/tasks pulling off queue entries and trying to write log entries at the same time (painful race condition). Since the log method is now just adding to a collection, it doesn't need to be async and you don't need to deal with the TPL at all, making it simpler and easier to reason about (and hopefully in the category of 'obviously correct' or thereabouts :)
This 'dedicated logging thread' approach is what I believe the log4net appender I linked to does as well, FWIW, in case that helps serve as an example.
I see two race conditions off the top of my head:
You can spin up more than one Thread if multiple threads call AddLogEntry. This won't cause lost events but is inefficient.
Yes, an event can be queued while the Thread is exiting, and in that case it would be "lost".
Also, there's a serious performance issue here: unless you're logging constantly (thousands of times a second), you're going to be spinning up a new Thread for each log entry. That will get expensive quickly.
Like James, I agree that you should use an established logging library. Logging is not as trivial as it seems, and there are already many solutions.
That said, if you want a nice .NET 4.5-based approach, it's pretty easy:
public abstract class BaseLogger
{
private readonly ActionBlock<LogEntry> block;
protected BaseLogger(int maxDegreeOfParallelism = 1)
{
block = new ActionBlock<LogEntry>(
entry =>
{
Log(entry);
},
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = maxDegreeOfParallelism,
});
}
public virtual void AddLogEntry(LogEntry entry)
{
block.Post(entry);
}
protected abstract void Log(LogEntry entry);
}
Regarding the loosing waiting messages on app crush because of unhandled exception, I've bound a handler to the event AppDomain.CurrentDomain.DomainUnload. Goes like this:
protected ManualResetEvent flushing = new ManualResetEvent(true);
protected AsyncLogger() // ctor of logger
{
AppDomain.CurrentDomain.DomainUnload += CurrentDomain_DomainUnload;
}
protected void CurrentDomain_DomainUnload(object sender, EventArgs e)
{
if (!IsEmpty)
{
flushing.WaitOne();
}
}
Maybe not too clean, but works.
I can log info messages without a problem, but can't figure out how to log verbose messages.
Any help would be welcomed.
My problem is:
loggingEvent.Level can be checked in the Format function. The possible values are amongst others, Info, Debug, Error, Verbose. There are more, but these are the ones I'll be using mostly.
The actual log object only has the following methods:
Log.Info
Log.Debug
Log.Warn
Log.Error
As you can see - no verbose!
So how can I Log a verbose message, this is different to debug
Thanks in advance
You can add a Verbose (or Trace level) to log4net by using extension methods. This is what I'm using:
public static class ILogExtentions
{
public static void Trace(this ILog log, string message, Exception exception)
{
log.Logger.Log(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType,
log4net.Core.Level.Trace, message, exception);
}
public static void Trace(this ILog log, string message)
{
log.Trace(message, null);
}
public static void Verbose(this ILog log, string message, Exception exception)
{
log.Logger.Log(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType,
log4net.Core.Level.Verbose, message, exception);
}
public static void Verbose(this ILog log, string message)
{
log.Verbose(message, null);
}
}
Usage example:
public class ClientDAO
{
private static readonly log4net.ILog log = log4net.LogManager.GetLogger(typeof(ClientDAO));
public void GetClientByCode()
{
log.Trace("your verbose message here");
//....
}
}
Source:
http://www.matthewlowrance.com/post/2010/07/14/Logging-to-Trace-Verbose-etc-with-log4net.aspx
You cannot figure out, because, AFAIK there is no "verbose" level in log4net. Is there one in log4j?
Following are the levels
ALL
DEBUG
INFO
WARN
ERROR
FATAL
OFF
Informational messages are the ones where you specify what you are doing currently in your application. Those messages spit out by OS commands or tools when you say -verbose, would be these kind of messages.
Debug messages are mostly for programmers and they allow you to write information such as variable creation, life-cycle, exception stack traces etc. Something that only the programmer/ support staff would be interested in.
[Edit]
Just thought of this. You can very well add a switch or config element to your application named "verbose" and then spit out the informational messages if set to true. Or wrap the logging in a helper method, which will log in log4net as well as send the same message to console. Also, you can use the ConsoleAppender to log messages to console. I have never used it though. Is this what you were looking for?
Hope this helps.
Apache log4net has the following log levels:
DEBUG < INFO < WARN < ERROR < FATAL
For messages considered more verbose than informational messages (INFO), the DEBUG level is the option to go for. Writing debug messages should be as simple as:
myLog.Debug("This is a pretty verbose message");
If you write extremely many debug messages and/or the messages are costly to produce (eg involves heavy string concatenation), consider adding a conditional around the logging:
if (myLog.IsDebugEnabled)
{
myLog.Debug("This is a pretty verbose message");
}
If you find yourself doing this often and want to DRY up your code, consider using extension methods for deferred message formatting, which will turn the above statement into this:
Log.Debug( () => "This is a pretty verbose message" );
In case off someone still need the answer (without using System.Reflection)
It's not necessary to set DeclaringType, just set null (auto resolve in Lo4Net)
public bool IsVerboseEnable { get { return _log.Logger.IsEnabledFor(Level.Verbose); } }
public string Verbose(string text)
{
_log.Logger.Log(null, Level.Verbose, text, null);
return text;
}
Tested & Validated
Code use in log4net
public virtual void Log(Type callerStackBoundaryDeclaringType, Level level, object message, Exception exception)
{
try
{
if (this.IsEnabledFor(level))
{
this.ForcedLog((callerStackBoundaryDeclaringType != null) ? callerStackBoundaryDeclaringType : Logger.declaringType, level, message, exception);
}
}
catch (Exception exception2)
{
LogLog.Error(Logger.declaringType, "Exception while logging", exception2);
}
}
I did not try it, but I think it should be quite straight-forward: Internally log4net knows a level "verbose"; it is only the ILog interface that does not expose it. Therefore it should be quite simple to add a IsVerboseEnabled and Verbose() method to this interface. Of course you need to be willing to change the log4net source code...
I've tested log4net with BasicConfigurator, and writing log messages generated output for all levels from EMERGENCY down to DEBUG, but not for TRACE or VERBOSE.
I needed to execute the code below for them to start logging.
var logRepository = LogManager.GetRepository(Assembly.GetEntryAssembly());
((Hierarchy)logRepository).Root.Level = Level.All;