Log4net - dynamically switch appender between AdoNetAppender and RollingFileAppender - log4net

I am using AdoNetAppender (SQL server) in my asp.net application and would like use to RollingFileAppender incase of any connection issue with SQL. Is there any way to configure to use RollingFileAppender only when there is an issue with AdoNetAppender?
Thanks
por

There is no built in support for this kind of failover scenario in log4net, the problem being that appenders are quite isolated from each other in the log4net architecture.
A common setup though is to have both appenders logging in parallel, only that the file appender only keeps, say, a weeks worth of data. Should the AdoNetAppender fail you will always have the latest data in files.
But I definitively see the case here for an appender that could have a priority list of sub-appenders doing some simple failover in case of failure. This should not be too hard to implement either building on the AppenderSkeleton.

I've implemented such an appender and blogged about it here and here (mirror). The code can be found here.
I've extended AppenderSkeleton and created a new Appender called FailoverAppender that has two members of type AppenderSkeleton.
A default appender called "PrimaryAppender" - used by default, until it fails.
A failover appender called "FailoverAppender" - used only after the primary fails.
The actual type of the PrimaryAppender and the FailoverAppender are configured using log4net's xml configuration syntax (see an example below).
A snippet:
public class FailoverAppender : AppenderSkeleton
{
private AppenderSkeleton _primaryAppender;
private AppenderSkeleton _failOverAppender;
....
}
In the implementation of the Append method, I send by default LoggingEvents only to the PrimaryAppender and surround it with a try-catch. If the PrimaryAppender throws (fails), I signal a flag and send the LoggingEvent to the FailoverAppender.
The next LoggingEvents will be sent directly and only to the FailoverAppender.
protected override void Append(LoggingEvent loggingEvent)
{
if (LogToFailOverAppender)
{
_failOverAppender?.DoAppend(loggingEvent);
}
else
{
try
{
_primaryAppender?.DoAppend(loggingEvent);
}
catch
{
ActivateFailOverMode();
Append(loggingEvent);
}
}
}
In addition, I created a custom ErrorHandler that will propagate inner-appender exceptions to signal that an appender has failed internally, which will let LoggingEvents to be sent only to the FailoverAppender.
class FailOverErrorHandler : IErrorHandler
{
public FailOverAppender FailOverAppender { get; set; }
public FailOverErrorHandler(FailOverAppender failOverAppender)
{
FailOverAppender = failOverAppender;
}
public void Error(string message, Exception e, ErrorCode errorCode)
=> FailOverAppender.ActivateFailOverMode();
public void Error(string message, Exception e)
=> FailOverAppender.ActivateFailOverMode();
public void Error(string message)
=> FailOverAppender.ActivateFailOverMode();
}
Configuration example:
<!--This custom appender handles failovers. If the first appender fails, it'll delegate the message to the back appender-->
<appender name="FailoverAppender" type="MoreAppenders.FailoverAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline"/>
</layout>
<!--This is a custom test appender that will always throw an exception -->
<!--The first and the default appender that will be used.-->
<PrimaryAppender type="MoreAppenders.ExceptionThrowerAppender" >
<ThrowExceptionForCount value="1" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline"/>
</layout>
</PrimaryAppender>
<!--This appender will be used only if the PrimaryAppender has failed-->
<FailOverAppender type="log4net.Appender.RollingFileAppender">
<file value="log.txt"/>
<rollingStyle value="Size"/>
<maxSizeRollBackups value="10"/>
<maximumFileSize value="100mb"/>
<appendToFile value="true"/>
<staticLogFileName value="true"/>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline"/>
</layout>
</FailOverAppender>
</appender>

Related

Custom PatternLayoutConverter with log4net.Ext.Json?

I have the following log4net configuration:
<log4net>
<appender name="Console" type="log4net.Appender.ConsoleAppender">
<layout type='log4net.Layout.SerializedLayout, log4net.Ext.Json'>
<renderer type='log4net.ObjectRenderer.JsonDotNetRenderer, log4net.Ext.Json.Net'>
<DateFormatHandling value="IsoDateFormat" />
<NullValueHandling value="Ignore" />
</renderer>
<converter>
<name value="preparedMessage" />
<type value="JsonLogs.CustomLayoutConverter" />
</converter>
<default />
<remove value='message' />
<remove value='ndc' />
<member value='message:messageObject' />
<member value='details:preparedMessage' />
</layout>
</appender>
<appender name="Console2" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<converter>
<name value="preparedMessage" />
<type value="JsonLogs.CustomLayoutConverter" />
</converter>
<conversionPattern value="%level %thread %logger - %preparedMessage%newline" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="Console" />
<appender-ref ref="Console2" />
</root>
</log4net>
with the following implementation of my custom PatternLayoutConverter:
namespace JsonLogs
{
using System.IO;
using log4net.Core;
using log4net.Layout.Pattern;
public class CustomLayoutConverter : PatternLayoutConverter
{
#region Methods
protected override void Convert(TextWriter writer, LoggingEvent loggingEvent)
{
if (loggingEvent.MessageObject is string stringMessage)
{
writer.Write(new { message = stringMessage });
}
else
{
writer.Write(loggingEvent.RenderedMessage);
}
}
#endregion
}
}
For some reason, the converter works perfectly fine with the Console2 appender(which is not JSON driven) but it doesn't work with the Console appender whose output is JSON.
Example of the output:
Console -> {"date":"2018-12-09T12:25:28.0529041+03:00","level":"INFO","appname":"JsonLogs.exe","logger":"JsonLogs.Program","thread":"1","message":"Test","details":"preparedMessage"}
Console2 -> INFO 1 JsonLogs.Program - { message = Test }
My goal is to have details always in JSON that's why I introduced my own converter to catch primitive values and wrap them in a custom object.
Is my configuration wrong? Or I'm missing something? Could you help me, please, to figure this out?
Thank you
The issue seems to be a bug of log4net.Ext.Json. I'm going to report it on their GitLab.
So far, I ended up with my custom log4net layout which looks like this
public class CustomLayout : PatternLayout
{
#region Public Methods and Operators
public override void Format(TextWriter writer, LoggingEvent loggingEvent)
{
var message = loggingEvent.MessageObject.GetType().IsPrimitive || loggingEvent.MessageObject is string || loggingEvent.MessageObject is decimal || loggingEvent.MessageObject is BigInteger
? new { message = loggingEvent.MessageObject }
: loggingEvent.MessageObject;
writer.WriteLine(JsonConvert.SerializeObject(new
{
timestamp = loggingEvent.TimeStampUtc,
threadId = loggingEvent.ThreadName,
details = message,
logger = loggingEvent.LoggerName,
level = loggingEvent.Level.DisplayName,
user = loggingEvent.UserName
}));
}
#endregion
}
it meets my needs and does exactly what I want.
The exact place of this problem is AddMember Method and its implementation. Here is SerializedLayout source code for that:
public virtual void AddMember(string value)
{
var arrangement = log4net.Util.TypeConverters.ArrangementConverter.GetArrangement(value, new ConverterInfo[0]);
m_arrangement.AddArrangement(arrangement);
}
As you can see the second parameter of GetArrangment is empty array of ConverterInfo, Though there must be our custom attached ones (by AddConverter method or by xml).
As the solution you can implement your own subclass that will derive from SerializedLayout with overridden AddMember like this:
public override void AddMember(string value)
{
var customConverter = new ConverterInfo("lookup", typeof(CustomPatternConverter));
var arrangement = log4net.Util.TypeConverters.ArrangementConverter.GetArrangement(value, new ConverterInfo[] { customConverter });
m_arrangement.AddArrangement(arrangement);
}
Hope it helps as it did with my case!

Log4j create a file for each logger

I have a lot of selenium tests that create a logger per class, it might not be the best way but it's code written by somebody else and I dont have time to rewrite it. I would like each Test of have it's own logfile so that it's easier to see what went wrong.
Is there a way to have log4j create a file for each logger that is created?
Yes, you can do this with log4j1 but I believe the only way is through programmatically adding the file appender to the logger.
Here is some sample code:
package test;
import org.apache.log4j.Appender;
import org.apache.log4j.FileAppender;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;
public class Main {
private static final Logger logger = Logger.getLogger(Main.class);
private static final Logger logFoo = Logger.getLogger("test.Foo");
private static final Logger logBar = Logger.getLogger("test.Bar");
public static void main(String[] args) {
logger.addAppender(createFileAppender("logs/main.log"));
logFoo.addAppender(createFileAppender("logs/foo.log"));
logBar.addAppender(createFileAppender("logs/bar.log"));
logger.info("This is the main logger");
logFoo.info("this is the foo logger");
logBar.info("This is the bar logger");
}
private static Appender createFileAppender(String logName) {
FileAppender fa = new FileAppender();
fa.setName("FileLogger");
fa.setFile(logName);
fa.setLayout(new PatternLayout("%d %-5p [%c{1}] %m%n"));
fa.setThreshold(Level.DEBUG);
fa.setAppend(true);
fa.activateOptions();
return fa;
}
}
Here is a sample log4j.xml config file:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="consoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%c{1}] %m %n" />
</layout>
</appender>
<logger name="test" additivity="false">
<level value="DEBUG" />
<appender-ref ref="consoleAppender" />
</logger>
</log4j:configuration>
Note that you probably don't need to specify the console appender, I did it just to make sure things were working. You may not even need to specify any loggers, but I didn't test with that configuration.
The output of the above is 3 log files each containing the one message that was provided to the corresponding logger, and console output of all log messages.

log4net Azure Blob Appender

When saving logs into Azure Blob Storage, is there a way to create a different folder per day?
Right now I'm using this configuration, and it works fine
<appender name="AzureAppender2" type="log4net.Appender.AzureBlobAppender, log4net.Appender.Azure">
<param name="ContainerName" value="testcon" />
<param name="DirectoryName" value="myfolder/logs.txt" />
<param name="ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=testcon;AccountKey="rftgdfgdfgfdg78=="/>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
</layout>
</appender>
I've already tried this
<param name="DirectoryName" value=%date/" />
But it doesn't work
What I want is to be able to dynamically use different folders per day: something like
DirectoryName = 2016-05-13
DirectoryName = 2016-05-12
DirectoryName = 2016-05-11
Is this achievable?
Thanks
Here's the code for the appender on GitHub.
Here's the DirectoryName class property that maps from the configuration value:
private string _directoryName;
public string DirectoryName
{
get
{
if (String.IsNullOrEmpty(_directoryName))
throw new ApplicationException(Resources.DirectoryNameNotSpecified);
return _directoryName;
}
set
{
_directoryName = value;
}
}
And the relevant Filename method that actually creates the file name for the blob:
private static string Filename(LoggingEvent loggingEvent, string directoryName)
{
return string.Format("{0}/{1}.{2}.entry.log.xml",
directoryName,
loggingEvent.TimeStamp.ToString("yyyy_MM_dd_HH_mm_ss_fffffff",
DateTimeFormatInfo.InvariantInfo),
Guid.NewGuid().ToString().ToLower());
}
So it looks like directoryName only accepts static values. Good thing it's open source ...

DailyRollingFileAppender not work

I use Log4j to write some log my program.
I find and read many question and answer in this site, but i can't solve my problem.
Here my code:
1. log4j.xml
<appender name="rollingfileAppender" class="org.apache.log4j.DailyRollingFileAppender">
<param name="append" value="true"/>
<param name="file" value="logs/process.log"/>
<param name="DatePattern" value="'.'yyyy-MM-dd-HH"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss:SSS} %-5p [%c{1}] %m%n"/>
</layout>
</appender>
<root>
<level value="DEBUG"/>
<appender-ref ref="rollingfileAppender"/>
<appender-ref ref="stdout"/>
</root>
2. My java code
package TestPacket;
import org.apache.log4j.Logger;
import org.apache.log4j.xml.DOMConfigurator;
public class TestLog4jXML {
static Logger logger = org.apache.log4j.Logger.getLogger(TestLog4jXML.class.getName());
public TestLog4jXML() {
}
public static void main(String[] args) {
try {
DOMConfigurator.configure("log4j1.xml");
logger.trace("Entering application.");
logger.debug("Debug");
logger.info("info");
logger.warn("warn");
logger.error("error");
logger.fatal("fatal");
lungtng();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void lungtng()
{
logger.fatal("some text here");
logger.info("hello picaso");
}
}
And i run my program, with eclipse, windows os.
But the log file name only: process.log not in daily format: process.log.yyyy-MM-dd-HH
Who can explain this to me?
It appears that if you're running windows this is a known bug:
see here: http://do.whileloop.org/2014/02/14/log4j-rolling-file-appenders-in-windows/
See here: https://issues.apache.org/bugzilla/show_bug.cgi?id=29726
And also here: http://www.coderanch.com/t/424837/java/java/Log-log-file-rolled-day
The solutions seems to be to use the extras here:
http://logging.apache.org/log4j/extras/
Try using this appender with the correct policies defined:
http://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/RollingFileAppender.html
The org.apache.log4j.DailyRollingFileAppender will create new log file for each day, each hour or each minute but file name of the current log always will be in the format that you've specified in the "file" parameter. In your example it's "process.log". The file names of the logs for the previous hours will be in format "process.log.yyyy-MM-dd-HH".

How to use aspnet-session pattern layout?

I have adonet appender and I defined additional column. I want to get the userId from the asp.net session and do log.
According to this page there is %aspnet-session{key} pattern which I use like this:
<parameter>
<parameterName value="#userId" />
<dbType value="String" />
<size value="255" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%aspnet-session{current_member}" />
</layout>
</parameter>
and I got the following result in the database:
/LM/W3SVC/1/ROOT/trunk-1-129718741958458380spnet-session{current_member}
What I am doing wrong here?
As of log4net version 1.2.11.0, you can use %aspnet-request{ASP.NET_SessionId} of the ASP.NET pattern layout converters like so:
<conversionPattern value="%date %level %logger [%aspnet-request{ASP.NET_SessionId}] - %message%newline" />
See the release notes:
[LOG4NET-87] - Support ASP.Net related PatternConverters to allow items from the HttpContext.Current.Session, Cache, Request, etc. to be captured.
/LM/W3SVC/1/ROOT/trunk-1-129718741958458380spnet-session{current_member}
is getting logged because in your log4net definition you have something like this right?
<parameter>
<parameterName value="#user" />
<dbType value="String" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%aspnet-session{current_memeber}" />
</layout>
</parameter>
and log4net is taking "%a" out of "%aspnet-session{current_memeber}" and thinking you want the application domain. %a is the log4net pattern that converts to the application domain. This is really annoying and i recently ran into this and do not know a way around it.
See here:
https://logging.apache.org/log4net/log4net-1.2.13/release/sdk/log4net.Layout.PatternLayout.html
Found a solution to this problem. If you use log4net 1.2.11 or greater declaring your parameter like this should work.
<parameter>
<parameterName value="#session" />
<dbType value="String" />
<size value="2147483647"/>
<layout type="log4net.Layout.PatternLayout">
<converter>
<name value ="AspNetSessionPatternConverter"/>
<type value="log4net.Layout.Pattern.AspNetSessionPatternConverter"/>
</converter>
<conversionPattern value="%aspnet-session{gwsession}" />
</layout>
</parameter>
I don't know if %a being short circuited to appdomain is a bug or if you're supposed to declare your params like this with the aspnetsessionpatternconverter, but it would be nice if the log4net documentation was updated. Hope this helps someone else.
You need to use log4net 1.2.11 or higher to get access to the ASP.NET pattern converters.
You'll get the LM/W3SVC/1/ROOT/trunk-1-129718741958458380spnet-session{current_member} message when you use an older version of log4net.
I found a solution to my problem.
I just refactored it to serve my needs:
public class Log4NetAspNetProperty
{
private const string PropertyNamePrefix = "log4net_app_";
private const string PropertyDefaultValue = null;
private readonly string propertyName;
private readonly object propertyValue;
public string Name { get { return propertyName; } }
private Log4NetAspNetProperty(string propertyName, object propertyValue)
{
if (String.IsNullOrWhiteSpace(propertyName)) throw new ArgumentNullException("propertyName");
this.propertyName = propertyName;
this.propertyValue = propertyValue;
if (HttpContext.Current != null)
HttpContext.Current.Items[GetPrefixedPropertyName()] = propertyValue;
}
public override string ToString()
{
if (HttpContext.Current == null)
return PropertyDefaultValue;
var item = HttpContext.Current.Items[GetPrefixedPropertyName()];
return item != null ? item.ToString() : PropertyDefaultValue;
}
private static string GetPrefixedPropertyName()
{
return String.Format("{0}{1}", PropertyNamePrefix, PropertyDefaultValue);
}
public static void CurrentUserId(object userId)
{
var property = new Log4NetAspNetProperty("CurrentUserId", userId);
log4net.ThreadContext.Properties[property.Name] = property;
}
public static void CurrentUrl(object url)
{
var property = new Log4NetAspNetProperty("CurrentUrl", url);
log4net.ThreadContext.Properties[property.Name] = property;
}
}

Resources