log4j: Rolling logs into .gz file with DailyRollingFileAppender using TimeBasedRollingPolicy - log4j

The current system is working as expected and the log files are rolled once an hr according to the logic below:
log4j.appender.oozie=org.apache.log4j.DailyRollingFileAppender
log4j.appender.oozie.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.oozie.File=${oozie.log.dir}/oozie.log
log4j.appender.oozie.Append=true
log4j.appender.oozie.layout=org.apache.log4j.PatternLayout
log4j.appender.oozie.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - %m%n
I can understand that rolling of log file directly into a compressed file(.gz or .zip) is possible with RollingFileAppender(rolling based on file size) using TimeBasedRollingPolicy. I'm using DailyRollingFileAppender(time based rolling) and would like to achieve compression with that. I modified my properties as follows. But that doesnt work.
log4j.appender.oozie=org.apache.log4j.DailyRollingFileAppender
log4j.appender.oozie.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.oozie.File=${oozie.log.dir}/oozie.log
log4j.appender.oozie.Append=true
log4j.appender.oozie.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.oozie.RollingPolicy.FileNamePattern=foo.%d{yyyy-MM-dd-HH}.gz
log4j.appender.oozie.layout=org.apache.log4j.PatternLayout
log4j.appender.oozie.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - %m%n
Any pointers would be highly appreciated. Here are some relevant links.
Configuring RollingFileAppender in log4j
http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html

Pardon my surprise, but at the exact link you gave above it says in the accepted answer:
Note that TimeBasedRollingPolicy can only be configured with xml, not log4j.properties
Have you tried rewriting your configuration to XML format?

please try the following, it works for me:
log4j.rootLogger=CONSOLE,file
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE..layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.file=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.file.Threshold=WARN
log4j.appender.file.Encoding=UTF-8
log4j.appender.file.File=/path to your logs/logs/log_file.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.file.RollingPolicy.FileNamePattern=/path to your logs/logs/log_file%d{yyyy-MM-dd-HH-mm-ss}.log
log4j.appender.file.layout.ConversionPattern=%d{MM-dd#HH:mm:ss} %-5p (%13F:%L) %3x - %m%n

Related

Why I have question instead line number when logging with log4j

I try to configure logger to write line number but instead of it I have question.
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p [%t]:%L - %m%n
I figured out that it is not log4j problem but setting of my project. The most probably missing debug information in my compiled artifacts. (Line number not showing?) But how can I set that in IntellijIdea 2019.1?

File name is generating as "Application.log.1" on reaching maxFileSize

below is Log4J.properties configuration ( Using jar "log4j-1.2.17.jar")
log4j.logger.devpinoyLogger=DEBUG, dest1
log4j.appender.dest1=org.apache.log4j.RollingFileAppender
log4j.appender.dest1.maxFileSize=1KB
log4j.appender.dest1.maxBackupIndex=3
log4j.appender.dest1.layout=org.apache.log4j.PatternLayout
log4j.appender.dest1.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} %c%m%n
log4j.appender.dest1.File=log\\Application.log
log4j.appender.dest1.Append=false
but once maxFileSize is reached backup log file is generated as Application.log.1
I want file to be generated as Application1.log
I got the solution using RollingFileAppender.
log4j.logger.devpinoyLogger=DEBUG, dest1
log4j.appender.dest1=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.dest1.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.dest1.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.dest1.triggeringPolicy.MaxFileSize=100
log4j.appender.dest1.rollingPolicy.FileNamePattern=C:\\TRB Workspace\\TRBAutomationFramework\\log\\Application-%i.log
log4j.appender.dest1.rollingPolicy.ActiveFileName=C:\\TRB Workspace\\TRBAutomationFramework\\log\\Application.log
log4j.appender.dest1.layout=org.apache.log4j.PatternLayout
log4j.appender.dest1.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} %c %m%n
log4j.appender.dest1.Append=false
Also required "apache-log4j-extras-1.2.17.jar"
I did not test, but as explained by Koning, you may try to add this to your properties:
log4j.appender.dest1.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.dest1.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.dest1.rollingPolicy.FileNamePattern=log\\Application-%i.log
log4j.appender.dest1.rollingPolicy.ActiveFileName=log\\Application.log

log4j appender with a file url

I have a problem where log4j is not logging to one of my files and i'm not sure why. I have some code that runs scripts, the scripts can add logging which is logged to a file using log4j, I am trying to create an appender that only logs for a particular script.
log4j.logger.com.my.class=INFO, JS_LOG
log4j.appender.JS_LOG.layout=org.apache.log4j.PatternLayout
log4j.appender.JS_LOG.Encoding=UTF-8
log4j.appender.JS_LOG.File=${log.outputdir}/js_service.log
log4j.appender.JS_LOG.MaxFileSize=2MB
log4j.appender.JS_LOG.MaxBackupIndex=10
log4j.appender.JS_LOG.Append=true
log4j.appender.JS_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.JS_LOG.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n
the above appender works and i get this in my log file
2012-04-18 11:25:52,043 [<MD> Inc Msg Dispatch-1 New] INFO com.my.class.file:/myfile - info
when the script logs something it logs to a logger of the form
com.my.class.file:/myfile this appears in the log using the above config in my log4j.properties file.
if I change the above to:
log4j.logger.com.my.class.file:/myfile=INFO, JS_LOG
log4j.appender.JS_LOG.layout=org.apache.log4j.PatternLayout
log4j.appender.JS_LOG.Encoding=UTF-8
log4j.appender.JS_LOG.File=${log.outputdir}/js_service.log
log4j.appender.JS_LOG.MaxFileSize=2MB
log4j.appender.JS_LOG.MaxBackupIndex=10
log4j.appender.JS_LOG.Append=true
log4j.appender.JS_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.JS_LOG.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n
nothing is logged to the file. I would have thought this should work? I retrieve the file
I managed to solve this problem, its something to do with the file: in my log4j.properties I changed the appender to file_ then in my code I do some manipulation to the file url to make it match the appender.

Log4j separate logs

I am trying to write separate logs for different packages using log4j. I am able to write separate logs for different levels using a customize appender.
You need two appender like this:
log4j.appender.MYLOGFILE=org.apache.log4j.FileAppender
log4j.appender.MYLOGFILE.File=/abc.log
log4j.appender.MYLOGFILE.Append=true
log4j.appender.MYLOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.MYLOGFILE.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n
log4j.appender.MYLOGFILE1=org.apache.log4j.FileAppender
log4j.appender.MYLOGFILE1.File=/abcde.log
log4j.appender.MYLOGFILE1.Append=true
log4j.appender.MYLOGFILE1.layout=org.apache.log4j.PatternLayout
log4j.appender.MYLOGFILE1.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n
Then you define your packages:
log4j.logger.mypackage=MYLOGFILE
log4j.additivity.mypackage=false
log4j.logger.secondpackage=MYLOGFILE1
log4j.additivity.secondpackage=false
The last line(additivity) is needed to not have your package inherit the global appender. Doing so will result in the log messages from mypackage also printed at the default appender.

log4j - filtering by class instead of level

In my app, I need to download 3 different artifacts from a remote ftp server. After I download each artifact, I validate the artifacts. And, if there are any validation errors, I need to log them aka
validation errors with Artifact A => A.log
validation errors with Artifact B => B.log
validation errors with Artifact C => C.log
My log4j.properties looks like:
log4j.rootLogger=DEBUG, rollingfile
log4j.logger.artifactALogger=INFO, AppenderA
log4j.logger.artifactBLogger=INFO, AppenderB
log4j.logger.artifactCLogger=INFO, AppenderC
log4j.appender.AppenderA.File=A.log
log4j.appender.AppenderA.MaxFileSize=5000KB
log4j.appender.AppenderA.MaxBackupIndex=1
log4j.appender.AppenderA.layout=org.apache.log4j.PatternLayout
log4j.appender.AppenderA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.AppenderB.File=B.log
log4j.appender.AppenderB.MaxFileSize=5000KB
log4j.appender.AppenderB.MaxBackupIndex=1
log4j.appender.AppenderB.layout=org.apache.log4j.PatternLayout
log4j.appender.AppenderB.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.AppenderC.File=C.log
log4j.appender.AppenderC.MaxFileSize=5000KB
log4j.appender.AppenderC.MaxBackupIndex=1
log4j.appender.AppenderC.layout=org.apache.log4j.PatternLayout
log4j.appender.AppenderC.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
As you can see, I have 1 appender per logger and the configuration for each appender is exactly the same (except for the name of the log file)...alot of duplication! Down the line, if I end up with more artifacts to download, I would need to declare another logger and another appender for each artifact. Something just doesn't seem right to me doing it this way. Maybe, I'm just not using log4j the way it was designed to be used? I'm currently filtering log output by class rather than by log level. Should I be using a different logging API that is more suitable?
We can create a custom logger for log4j for this purpose. We had a similar requirement once and ended up creating a custom logger.

Resources