How to log a message when the BufferingForwardingAppender is flushed? - log4net

Question

I have a BufferingForwardingAppender configured to forward the last 10 messages to a RollingFileAppender when an ERROR occurs. Can I get a custom message to appear in the log before each batch of messages?
I want to indicate that this is a new ERROR event and its context, so that the log is readable?
My log4net configuration is kinda like this:
<appender name="ErrorBufferingAppender" type="log4net.Appender.BufferingForwardingAppender">
<bufferSize value="10" />
<lossy value="true" />
<evaluator type="log4net.Core.LevelEvaluator">
<threshold value="ERROR" />
</evaluator>
<appender-ref ref="ErrorFileAppender" />
</appender>
<appender name="ErrorFileAppender" type="log4net.Appender.RollingFileAppender">
<param name="File" value="logs\Errors.txt"/>
<!-- other important parameters -->
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%-4thread] %-5level %logger{1} - %message%newline"/>
</layout>
</appender>
And my desired output is kinda like this
---------- ERROR ----------
2011-05-09 16:59:12,327 [8 ] INFO AppLogger - DiskStorageMonitor.StartMonitoring()
2011-05-09 16:59:12,331 [8 ] INFO AppLogger - DiskStorageMonitor.Initializing File System Watcher...
2011-05-09 16:59:12,341 [8 ] INFO AppLogger - DiskStorageMonitor - LocalSlideDataStorageDrive: C:
2011-05-09 16:59:12,370 [8 ] INFO AppLogger - Transition: From 'MinimumDiskSpaceQuotaState' to 'IdleState'.
2011-05-09 16:59:26,697 [8 ] INFO AppLogger - Transition: From 'IdleState' to 'SlideHolderMacroState'.
2011-05-09 16:59:26,702 [8 ] INFO AppLogger - Transition: From 'SlideHolderMacroState' to 'WaitingForMacroCmdResponseState'.
2011-05-09 16:59:26,781 [12 ] INFO YStageManager - Moving to: -1.25
2011-05-09 16:59:26,782 [14 ] INFO XStageManager - Moving to: -142
2011-05-09 16:59:30,800 [12 ] ERROR RecoveringErrorHandler - An error occurred. Retrying... (snipped exception message & stacktrace)
---------- ERROR ----------
2011-05-09 16:59:30,808 [12 ] WARN MacroCameraRecoverySteps - An error occurred while calling the macro camera. Resetting and retrying...
2011-05-09 16:59:30,809 [12 ] INFO MacroCameraManager - Resetting the Macro Camera.
2011-05-09 16:59:30,886 [12 ] ERROR GetMacroImageCommandHandler - Macro image processing failed for the slide in position 0. (snipped exception message & stacktrace)

Answer 1

I've got a solution that will probably work for you but it is a bit of a work-around. Basically, set two appenders to write to the same file. When you write an error (which will trigger the Buffer to clear to your log file), have the other appender write the "-------Error--------" text first with a newline before the text.
You will have to play around with it a bit to get it to work, but this should be a workable solution. Apache does allow multiple appenders to write to the same file. Just make sure you put the +MinimalLock on your appenders so they don't lock the file. Here is a SO link that explains how two appenders can write to the same file:
Can Log4net have multiple appenders write to the same file?

Answer 2

Thanks to BiggsTRC for the great answers. This is my working log4net configuration and sample output.
This appender responds to any ERROR message by writing the context "header" in the log. The log is minimally locked to allow this and the buffering appender to write to the file. The ignoresException tag set to false actually tells log4net that this layout will write the exception (rather than it writing automatically below the message). But really I don't want it to, that's coming in the other appender.
<appender name="ErrorSectionAppender" type="log4net.Appender.FileAppender">
<filter type="log4net.Filter.LevelMatchFilter">
<levelToMatch value="ERROR" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
<file value="logs\Errors.txt" />
<appendToFile value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.PatternLayout">
<ignoresException value="false" />
<conversionPattern value="%newline---------- ERROR # %date ----------%newline" />
</layout>
</appender>
This appender buffers all messages until an ERROR message appears, then it dumps the last 10 (bufferSize) messages and the error message to the ErrorFileAppender.
<appender name="ErrorBufferingAppender" type="log4net.Appender.BufferingForwardingAppender">
<bufferSize value="10" />
<lossy value="true" />
<evaluator type="log4net.Core.LevelEvaluator">
<threshold value="ERROR" />
</evaluator>
<appender-ref ref="ErrorFileAppender" />
</appender>
This is the actual error file appender. The conversion pattern is standard. Notice the minimal file lock here as well.
<appender name="ErrorFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="logs\Errors.txt"/>
<appendToFile value="true"/>
<staticLogFileName value="true"/>
<rollingStyle value="Size"/>
<maxFileSize value="1048576"/>
<maxSizeRollBackups value="-1"/>
<countDirection value="1"/>
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%-4thread] %-5level %logger{1} - %message%newline"/>
</layout>
</appender>
The output of these appenders looks like this
---------- ERROR # 2011-05-13 13:51:22,632 ----------
2011-05-13 13:51:04,937 [9 ] INFO OmnyxScannerLogger - Transition: From 'No State' to 'CommunicationSyncState'.
2011-05-13 13:51:05,363 [12 ] DEBUG IMacroImageProcessor - hello, 10
2011-05-13 13:51:05,369 [19 ] INFO XStageManager - Connecting
2011-05-13 13:51:05,369 [12 ] INFO ZStageManager - Connecting
2011-05-13 13:51:05,369 [16 ] INFO YStageManager - Connecting
2011-05-13 13:51:05,393 [9 ] INFO OmnyxScannerLogger - Transition: From 'CommunicationSyncState' to 'SystemInitializeState'.
2011-05-13 13:51:08,528 [17 ] INFO YStageManager - Homing
2011-05-13 13:51:08,539 [12 ] INFO ZStageManager - Homing
2011-05-13 13:51:22,632 [14 ] ERROR RecoveringErrorHandler - An error occurred. Retrying...
Omnyx.Scanner.CliContracts.ScannerException: Injected: The stage failed to home.
at Omnyx.Scanner.Simulation.ErrorInjection.ErrorInjectionInterceptor.Intercept(IInvocation invocation) in D:\Simulation\ErrorInjection\ErrorInjectionInterceptor.cs:line 42
at Castle.DynamicProxy.AbstractInvocation.Proceed()
at Castle.Proxies.IXStageManagerProxy.Home()
---------- ERROR # 2011-05-13 13:51:33,758 ----------
2011-05-13 13:51:29,034 [9 ] INFO OmnyxScannerLogger - DiskStorageMonitor.StartMonitoring()
2011-05-13 13:51:29,037 [9 ] INFO OmnyxScannerLogger - DiskStorageMonitor.Initializing File System Watcher...
2011-05-13 13:51:29,046 [9 ] INFO OmnyxScannerLogger - DiskStorageMonitor - LocalSlideDataStorageDrive: C:
2011-05-13 13:51:29,062 [9 ] INFO OmnyxScannerLogger - -- LocalSlideDataStorageOverhead: 0
2011-05-13 13:51:29,075 [9 ] INFO OmnyxScannerLogger - Transition: From 'MinimumDiskSpaceQuotaState' to 'IdleState'.
2011-05-13 13:51:31,727 [9 ] INFO OmnyxScannerLogger - Transition: From 'IdleState' to 'SlideHolderMacroState'.
2011-05-13 13:51:31,732 [9 ] INFO OmnyxScannerLogger - Transition: From 'SlideHolderMacroState' to 'WaitingForMacroCmdResponseState'.
2011-05-13 13:51:31,792 [12 ] INFO YStageManager - Moving to: -1.25
2011-05-13 13:51:31,793 [20 ] INFO XStageManager - Moving to: -142
2011-05-13 13:51:33,758 [12 ] ERROR RecoveringErrorHandler - An error occurred. Retrying...
Omnyx.Scanner.CliContracts.ScannerException: Injected: The macro camera failed.
at Omnyx.Scanner.Simulation.ErrorInjection.ErrorInjectionInterceptor.Intercept(IInvocation invocation) in D:\Simulation\ErrorInjection\ErrorInjectionInterceptor.cs:line 42
at Castle.DynamicProxy.AbstractInvocation.Proceed()
at Castle.Proxies.IMacroCameraManagerProxy.TakeTopLitAndBackLitPictures()