Rotating GC Log files with qos Logback - garbage-collection

I am trying to rotate my gc.log file every time my application starts up.
I am using this file appender in my logback.xml file.
...
<appender name="GCFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.directory}/gc.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.directory}/gc.log.%d{yyyyMMdd}_%d{HHmmss,aux}.gz</fileNamePattern>
<TimeBasedFileNamingAndTriggeringPolicy class="com.ga.omni.utility.StartupTriggeringPolicy" />
<maxHistory>50</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%msg%n</pattern>
</encoder>
</appender>
...
<root level="INFO">
<appender-ref ref="FILE" />
<appender-ref ref="GCFILE" />
</root>
(the "FILE" ref is a reference to our default logging file for the app.)
The appender references a TimeBasedFileNamingAndTriggeringPolicy named StartupTriggeringPolicy:
#NoAutoStart //won't be autostarted by Joran at config time
public class StartupTriggeringPolicy<E> extends DefaultTimeBasedFileNamingAndTriggeringPolicy<E> {
Logger log = LoggerFactory.getLogger(StartupTriggeringPolicy.class);
public StartupTriggeringPolicy() {
log.info("StartupTriggeringPolicy constructor called");
}
#Override
public void start() {
log.info("StartupTriggeringPolicy start() called... initialting gc.log rollover");
super.start();
//only check this once, on startup.
nextCheck = 0L;
isTriggeringEvent(null, null);
try {
tbrp.rollover();
log.info("StartupTriggeringPolicy start() called... gc.log successfully rolled over.");
} catch (RolloverFailure e) {
log.warn("Error rolling over gc.log file in StartupTriggeringPolicy.start()");
//Do nothing
}
}
}
The trouble that I'm facing is that the app starts up, but the StartupTriggeringPolicy never seems to get instantiated. None of the logs from the constructor or start() method are written, and if I put breakpoints in those methods, the breakpoints don't get hit.
Any suggestions would be greatly apperciated!

Related

Custom PatternLayoutConverter with log4net.Ext.Json?

I have the following log4net configuration:
<log4net>
<appender name="Console" type="log4net.Appender.ConsoleAppender">
<layout type='log4net.Layout.SerializedLayout, log4net.Ext.Json'>
<renderer type='log4net.ObjectRenderer.JsonDotNetRenderer, log4net.Ext.Json.Net'>
<DateFormatHandling value="IsoDateFormat" />
<NullValueHandling value="Ignore" />
</renderer>
<converter>
<name value="preparedMessage" />
<type value="JsonLogs.CustomLayoutConverter" />
</converter>
<default />
<remove value='message' />
<remove value='ndc' />
<member value='message:messageObject' />
<member value='details:preparedMessage' />
</layout>
</appender>
<appender name="Console2" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<converter>
<name value="preparedMessage" />
<type value="JsonLogs.CustomLayoutConverter" />
</converter>
<conversionPattern value="%level %thread %logger - %preparedMessage%newline" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="Console" />
<appender-ref ref="Console2" />
</root>
</log4net>
with the following implementation of my custom PatternLayoutConverter:
namespace JsonLogs
{
using System.IO;
using log4net.Core;
using log4net.Layout.Pattern;
public class CustomLayoutConverter : PatternLayoutConverter
{
#region Methods
protected override void Convert(TextWriter writer, LoggingEvent loggingEvent)
{
if (loggingEvent.MessageObject is string stringMessage)
{
writer.Write(new { message = stringMessage });
}
else
{
writer.Write(loggingEvent.RenderedMessage);
}
}
#endregion
}
}
For some reason, the converter works perfectly fine with the Console2 appender(which is not JSON driven) but it doesn't work with the Console appender whose output is JSON.
Example of the output:
Console -> {"date":"2018-12-09T12:25:28.0529041+03:00","level":"INFO","appname":"JsonLogs.exe","logger":"JsonLogs.Program","thread":"1","message":"Test","details":"preparedMessage"}
Console2 -> INFO 1 JsonLogs.Program - { message = Test }
My goal is to have details always in JSON that's why I introduced my own converter to catch primitive values and wrap them in a custom object.
Is my configuration wrong? Or I'm missing something? Could you help me, please, to figure this out?
Thank you
The issue seems to be a bug of log4net.Ext.Json. I'm going to report it on their GitLab.
So far, I ended up with my custom log4net layout which looks like this
public class CustomLayout : PatternLayout
{
#region Public Methods and Operators
public override void Format(TextWriter writer, LoggingEvent loggingEvent)
{
var message = loggingEvent.MessageObject.GetType().IsPrimitive || loggingEvent.MessageObject is string || loggingEvent.MessageObject is decimal || loggingEvent.MessageObject is BigInteger
? new { message = loggingEvent.MessageObject }
: loggingEvent.MessageObject;
writer.WriteLine(JsonConvert.SerializeObject(new
{
timestamp = loggingEvent.TimeStampUtc,
threadId = loggingEvent.ThreadName,
details = message,
logger = loggingEvent.LoggerName,
level = loggingEvent.Level.DisplayName,
user = loggingEvent.UserName
}));
}
#endregion
}
it meets my needs and does exactly what I want.
The exact place of this problem is AddMember Method and its implementation. Here is SerializedLayout source code for that:
public virtual void AddMember(string value)
{
var arrangement = log4net.Util.TypeConverters.ArrangementConverter.GetArrangement(value, new ConverterInfo[0]);
m_arrangement.AddArrangement(arrangement);
}
As you can see the second parameter of GetArrangment is empty array of ConverterInfo, Though there must be our custom attached ones (by AddConverter method or by xml).
As the solution you can implement your own subclass that will derive from SerializedLayout with overridden AddMember like this:
public override void AddMember(string value)
{
var customConverter = new ConverterInfo("lookup", typeof(CustomPatternConverter));
var arrangement = log4net.Util.TypeConverters.ArrangementConverter.GetArrangement(value, new ConverterInfo[] { customConverter });
m_arrangement.AddArrangement(arrangement);
}
Hope it helps as it did with my case!

Log4j create a file for each logger

I have a lot of selenium tests that create a logger per class, it might not be the best way but it's code written by somebody else and I dont have time to rewrite it. I would like each Test of have it's own logfile so that it's easier to see what went wrong.
Is there a way to have log4j create a file for each logger that is created?
Yes, you can do this with log4j1 but I believe the only way is through programmatically adding the file appender to the logger.
Here is some sample code:
package test;
import org.apache.log4j.Appender;
import org.apache.log4j.FileAppender;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;
public class Main {
private static final Logger logger = Logger.getLogger(Main.class);
private static final Logger logFoo = Logger.getLogger("test.Foo");
private static final Logger logBar = Logger.getLogger("test.Bar");
public static void main(String[] args) {
logger.addAppender(createFileAppender("logs/main.log"));
logFoo.addAppender(createFileAppender("logs/foo.log"));
logBar.addAppender(createFileAppender("logs/bar.log"));
logger.info("This is the main logger");
logFoo.info("this is the foo logger");
logBar.info("This is the bar logger");
}
private static Appender createFileAppender(String logName) {
FileAppender fa = new FileAppender();
fa.setName("FileLogger");
fa.setFile(logName);
fa.setLayout(new PatternLayout("%d %-5p [%c{1}] %m%n"));
fa.setThreshold(Level.DEBUG);
fa.setAppend(true);
fa.activateOptions();
return fa;
}
}
Here is a sample log4j.xml config file:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="consoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%c{1}] %m %n" />
</layout>
</appender>
<logger name="test" additivity="false">
<level value="DEBUG" />
<appender-ref ref="consoleAppender" />
</logger>
</log4j:configuration>
Note that you probably don't need to specify the console appender, I did it just to make sure things were working. You may not even need to specify any loggers, but I didn't test with that configuration.
The output of the above is 3 log files each containing the one message that was provided to the corresponding logger, and console output of all log messages.

How to create separate Log4j2 rolling file appenders and loggers programatically for multiple threads

I am running my TestNG tests on multiple threads (Appium tests on multiple devices simultaneously) and want to write the test logs on different threads in different files. Here the threads are created automatically before the start of the test flow.
So I want to create separate appender and separate logger programatically so that each appender would be attached to its own thread only and then the loggers created in one thread would have the appender created in that thread only.
Please let me know how to achieve it step by step.
First, this feels like an XY Problem especially because you have not provided any reasoning as to why you want to go with a programmatic solution.
It is possible to achieve separate log files on a thread by thread basis without programmatically creating loggers and appenders. Since I believe this to be a more optimal solution I will provide a demo of how it can be done.
Explanation
This solution will use the RoutingAppender and ThreadContext to create appenders for each Thread. The example that follows will use a simple FileAppender but you can swap in a RollingFileAppender very easily. Since you did not state that you have any requirement to use different log levels for each thread the following example will not implement this feature. Finally, the example will not use TestNG as it has some overhead to set up; instead a simple class with a main that creates two threads will be used.
Example Implementation
log4j2.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Routing name="MyRoutingAppender">
<Routes pattern="$${ctx:threadName}">
<Route>
<File
fileName="logs/${ctx:threadName}/log.txt"
name="appender-${ctx:threadName}">
<PatternLayout>
<Pattern>[%date{ISO8601}][%-5level][%t] %m%n</Pattern>
</PatternLayout>
</File>
</Route>
</Routes>
</Routing>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="[%date{ISO8601}][%-5level][%t] %m%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="Thread" level="TRACE" additivity="false">
<AppenderRef ref="STDOUT" />
<AppenderRef ref="MyRoutingAppender" />
</Logger>
<Root level="WARN">
<AppenderRef ref="STDOUT" />
</Root>
</Loggers>
</Configuration>
Java class that generates logs using 2 concurrent threads:
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
public class MultiThreadLog4j2SepFilesMain {
//Create a lock to use for synchronizing the getting of the logger
private static final Object lock = new Object();
public static void main(String[] args) {
Thread t1 = new Thread(new Runnable(){
public void run() {
//Set up the context before getting logger
ThreadContext.put("threadName", Thread.currentThread().getName());
//Get the logger for this thread
Logger log = null;
synchronized(lock){
log = LogManager.getLogger(Thread.currentThread().getName());
}
//Generate some logs
log.info("here's the first thread");
//Wait a while so that threads interleave
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//Generate more logs
log.debug("some debug in first thread");
log.info("finishing first thread");
}}, "Thread.1"); //Use a name that will allow us to use Thread.getName when getting the logger inside the thread
Thread t2 = new Thread(new Runnable(){
public void run() {
//Set up the context before getting logger
ThreadContext.put("threadName", Thread.currentThread().getName());
//Get logger for this thread
Logger log = null;
synchronized(lock){
log = LogManager.getLogger(Thread.currentThread().getName());
}
//Generate some logs
log.info("here's the second thread");
log.debug("some debug in second thread");
}}, "Thread.2"); //Use a name that will allow us to use Thread.getName when getting the logger inside the thread
//Start both threads
t1.start();
t2.start();
}
}

DailyRollingFileAppender not work

I use Log4j to write some log my program.
I find and read many question and answer in this site, but i can't solve my problem.
Here my code:
1. log4j.xml
<appender name="rollingfileAppender" class="org.apache.log4j.DailyRollingFileAppender">
<param name="append" value="true"/>
<param name="file" value="logs/process.log"/>
<param name="DatePattern" value="'.'yyyy-MM-dd-HH"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss:SSS} %-5p [%c{1}] %m%n"/>
</layout>
</appender>
<root>
<level value="DEBUG"/>
<appender-ref ref="rollingfileAppender"/>
<appender-ref ref="stdout"/>
</root>
2. My java code
package TestPacket;
import org.apache.log4j.Logger;
import org.apache.log4j.xml.DOMConfigurator;
public class TestLog4jXML {
static Logger logger = org.apache.log4j.Logger.getLogger(TestLog4jXML.class.getName());
public TestLog4jXML() {
}
public static void main(String[] args) {
try {
DOMConfigurator.configure("log4j1.xml");
logger.trace("Entering application.");
logger.debug("Debug");
logger.info("info");
logger.warn("warn");
logger.error("error");
logger.fatal("fatal");
lungtng();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void lungtng()
{
logger.fatal("some text here");
logger.info("hello picaso");
}
}
And i run my program, with eclipse, windows os.
But the log file name only: process.log not in daily format: process.log.yyyy-MM-dd-HH
Who can explain this to me?
It appears that if you're running windows this is a known bug:
see here: http://do.whileloop.org/2014/02/14/log4j-rolling-file-appenders-in-windows/
See here: https://issues.apache.org/bugzilla/show_bug.cgi?id=29726
And also here: http://www.coderanch.com/t/424837/java/java/Log-log-file-rolled-day
The solutions seems to be to use the extras here:
http://logging.apache.org/log4j/extras/
Try using this appender with the correct policies defined:
http://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/RollingFileAppender.html
The org.apache.log4j.DailyRollingFileAppender will create new log file for each day, each hour or each minute but file name of the current log always will be in the format that you've specified in the "file" parameter. In your example it's "process.log". The file names of the logs for the previous hours will be in format "process.log.yyyy-MM-dd-HH".

log4net issue in web service

We are successfully using log4net in our UI Layer but when we are testing in Webservice layer it does not work.
Here is the code in UI Layer:
public partial class _Default : System.Web.UI.Page
{
ILog logger = log4net.LogManager.GetLogger(typeof(_Default));
protected void Page_Load(object sender, EventArgs e)
{
ServiceReference1.IService1 is1 = new ServiceReference1.Service1Client();
is1.GetData(1);
logger.Info("Hello Nine Thanks for use Log4Net,This is info message");
logger.Debug("Hello Nine Thanks for use Log4Net,This is Debug message");
logger.Error("Hello Nine Thanks for use Log4Net,This is Error message");
logger.Warn("Hello Nine Thanks for use Log4Net,This is Warn message");
logger.Fatal("Hello Nine Thanks for use Log4Net,This is Fatal message");
}
}
Here is the web.config settings for the UI Layer:
<log4net>
<appender name="FileAppender" type="log4net.Appender.FileAppender">
<param name="File" value="MyloggerSite2.log"/> <!-- This is logging in app root folder -->
<param name="AppendToFile" value="true"/>
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-2p %c [%x] - %m%n"/>
</layout>
</appender>
<root>
<level value="All"/>
<appender-ref ref="FileAppender"/>
</root>
</log4net>
Here is the code in ServiceLayer which does not work although it it is the same as above for the most:
public class Service1 : IService1
{
public string GetData(int value)
{
ILog logger = log4net.LogManager.GetLogger(typeof(Service1));
logger.Info("Hello Nine Thanks for use Log4Net,This is info message");
logger.Debug("Hello Nine Thanks for use Log4Net,This is Debug message");
logger.Error("Hello Nine Thanks for use Log4Net,This is Error message");
logger.Warn("Hello Nine Thanks for use Log4Net,This is Warn message");
logger.Fatal("Hello Nine Thanks for use Log4Net,This is Fatal message");
return string.Format("You entered: {0}", value);
}
}
Please let me know if you have any suggestions.
Thanks,
N
Do you configure log4net for instance by having an attribute like this in your web service:
[assembly: log4net.Config.XmlConfigurator(Watch=true)]
I have just added this attribute manually in AssemblyInfo.cs:
[assembly: log4net.Config.XmlConfigurator(Watch = true)]
It works.
Thanks.
Just add below assembly in web config file of your service's project.
[assembly: log4net.Config.XmlConfigurator(Watch = true)]

Resources