Log4j2 main ERROR AmazonRollingRandomAccessFile contains an invalid element or attribute - log4j

I'm trying to use Log4j2 in my application and my relevant configuration looks like below code:
*.*.log4j2.appender.file.type = RollingRandomAccessFile;
*.*.log4j2.appender.file.name = file;
*.*.log4j2.appender.file.fileName = "$ROOT/var/output/logs/$APP";
*.*.log4j2.appender.file.filePattern = "$ROOT/var/output/logs/$APP.%d{yyyy-MM-dd-HH}";
*.*.log4j2.appender.file.layout.type = PatternLayout;
*.*.log4j2.appender.file.layout.pattern = "%d{DATE} [%p] %X{requestId} (%t) %c: %m%n";
But, when i run my program i get this error:
955 main ERROR RollingRandomAccessFile contains an invalid element or attribute "fileName"
Can anyone help me on this issue?
Your help is much appreciated.
Best Regards.

I suspect that the *.*.log4j2 prefix may be related. Can you try without this, like in the example in the log4j2 manual?
appender.rolling.type = RollingRandomAccessFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}
appender.rolling.filePattern = target/rolling2/test1-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d %p %C{1.} [%t] %m%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 2
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=100MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 5
rootLogger.level = info
rootLogger.appenderRef.rolling.ref = RollingFile

Related

log4j2 configuration change from size based to time based

We're looking to change our size based trigger to time based trigger policy (every 5 minutes). But our configuration looks very different from the current log4j2. Here's our current configuration:
log4j.rootLogger=INFO,file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=${spark.yarn.app.container.log.dir}/spark.log
log4j.appender.file.MaxFileSize=10MB
log4j.appender.file.MaxBackupIndex=100
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d %p %t %c - %m%n
log4j.appender.file.encoding=UTF-8
If I change it to this, I'm not sure if it'll work or if I'm missing anything?
log4j.appender.rolling.type = RollingFile
log4j.appender.rolling.name = RollingFile
log4j.appender.rolling.fileName = spark.log
log4j.appender.rolling.filePattern = ${spark.yarn.app.container.log.dir}/spark-%d{yyyy-MM-dd-HH-mm}-%i.log.gz
log4j.appender.rolling.layout.type = PatternLayout
log4j.appender.rolling.layout.pattern = %d %p %t %c - %m%n
log4j.appender.rolling.policies.type = Policies
log4j.appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
log4j.appender.rolling.policies.time.interval = 5
log4j.appender.rolling.policies.time.modulate = true

How to stop python log printing in console

I am using the following code. Still my logs are being printed both in console and log file. Can anyone please help me to identify how to stop printing log in the console
logger = logging.getLogger('MyLog')
logger.handlers = []
hdlr = logging.handlers.RotatingFileHandler(LOG_FILE_PATH, maxBytes = ConfConst.MAX_LOG_FILE_SIZE , backupCount= ConfConst.MAX_LOG_FILE_BACKUP_COUNT)
formatter = logging.Formatter('[%(asctime)s] p%(process)s {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s','%m-%d %H:%M:%S')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logging.StreamHandler(stream=None)
logger.setLevel(logging.INFO)
Use this it may help you
logger = logging.getLogger()
logger.disabled = True
... whatever you want ...
logger.disabled = False

Write Spark worker logs to stdout/stderr

Hi I am trying to redirect Spark workers logs to stdout/stderr.
I have added custom log4j.properties file that looks like this:
log4j.rootLogger = INFO, stdout, stderr
# configure stdout
log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Threshold = TRACE
log4j.appender.stdout.filter.filter1=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.stdout.filter.filter1.levelMin = TRACE
log4j.appender.stdout.filter.filter1.levelMax = INFO
log4j.appender.stdout.Target = System.out
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern = %d{yy/MM/dd HH:mm:ss,SSS} %p %c: %m%n
# configure stderr
log4j.appender.stderr = org.apache.log4j.ConsoleAppender
log4j.appender.stderr.Threshold = WARN
log4j.appender.stderr.Target = System.err
log4j.appender.stderr.layout = org.apache.log4j.PatternLayout
log4j.appender.stderr.layout.ConversionPattern = %d{yy/MM/dd HH:mm:ss,SSS} %p %c: %m%n
# Settings to quiet third party logs that are too verbose
log4j.logger.org.apache.hadoop.util.NativeCodeLoader = ERROR
log4j.logger.org.eclipse.jetty=WARN
log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO
log4j.logger.org.apache.spark=WARN
log4j.logger.org.spark-project.jetty.server=WARN
Still logs are not redirected to stdout/stderr but to stdout and stderr files on the disk.
Anyone knows how can I make it work?
EDIT: reading from this blog post it seems like it is just the way it works

Why log4j2 not rolling over after reaching the max and keeps overwriting the default log file?

I am using the below configuration to generate the log rolling based on size. The logs are created, however the rolling stops after creating the max files (10). The default file is keep on overwriting and all the rolled over files never gets updated.
appender.sample.type = RollingFile
appender.sample.name = SampleRollingFile
appender.sample.fileName = ${basedir}/logs/${machine}_My_Sample.0.log
appender.sample.filePattern = ${basedir}/logs/${machine}_My_Sample.%i.log
appender.sample.layout.type = PatternLayout
appender.sample.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [T:%T] %c{1}:%L - %m%n
appender.sample.policies.type = Policies
appender.sample.policies.size.type = SizeBasedTriggeringPolicy
appender.sample.policies.size.size = 100MB
appender.sample.strategy.type = DefaultRolloverStrategy
appender.sample.strategy.max = 10
But when I changed my default initial file name to My_Sample.log (removing the .0 index) it works perfectly fine. My application requirement is initial file with .0 and the rolling the logs from 1 to 10.
I added the below properties to get the desired result. By enabling tracing (status = TRACE) in the log4j2 they use the index 0 to play around with the renaming and purge. To avoid the collision, I started the index from 1 and the continuous rolling is happening !!
appender.sample.strategy.fileIndex = 1
appender.sample.strategy.min = 1

Very simple log4j2 properties configuration file using Console and Rolling File appender

I'd like a log4j2 properties file configuration with a console and a rolling file appender using log4j2 that can be used different application. The log configuration should rotate the log in production environment.
I think there is no such industry standard for logging or log4j2 configuration. Everyone change the configuration as per the need of the application.
Below is one sample log4j2 configuration file having ConsoleAppender and RollingFileAppender -
status = warn
name= properties_configuration
# Give directory path where log files should get stored
property.basePath = ./log/
# ConsoleAppender will print logs on console
appender.console.type = Console
appender.console.name = consoleLogger
appender.console.target = SYSTEM_OUT
appender.console.layout.type = PatternLayout
# Specify the pattern of the logs
appender.console.layout.pattern = %d{yyyy-MM-dd HH:mm:ss.SSS} %level [%t] [%c] [%M] [%l] - %msg%n
# RollingFileAppender will print logs in file which can be rotated based on time or size
appender.rolling.type = RollingFile
appender.rolling.name = fileLogger
appender.rolling.fileName= ${basePath}app.log
appender.rolling.filePattern= ${basePath}app_%d{yyyyMMdd}.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d{yyyy-MM-dd HH:mm:ss.SSS} %level [%t] [%c] [%M] [%l] - %msg%n
appender.rolling.policies.type = Policies
# Rotate log file each day and keep 30 days worth
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basePath = ${basePath}
appender.rolling.strategy.delete.maxDepth = 1
appender.rolling.strategy.delete.ifLastModified.type = IfLastModified
# Delete files older than 30 days
appender.rolling.strategy.delete.ifLastModified.age = 30d
# Mention package name here in place of example. Classes in this package or subpackages will use ConsoleAppender and RollingFileAppender for logging
logger.example.name = example
logger.example.level = debug
logger.example.additivity = false
logger.example.appenderRef.rolling.ref = fileLogger
logger.example.appenderRef.console.ref = consoleLogger
# Configure root logger for logging error logs in classes which are in package other than above specified package
rootLogger.level = error
rootLogger.additivity = false
rootLogger.appenderRef.rolling.ref = fileLogger
rootLogger.appenderRef.console.ref = consoleLogger
Try this out:
# Declare loggers
name=LoggingConfig
appenders=a_console, a_rolling
rootLogger.level=info
rootLogger.appenderRefs=ar_console,ar_rolling
rootLogger.appenderRef.ar_console.ref=StdoutAppender
rootLogger.appenderRef.ar_rolling.ref=DailyRollingAppender
# Console logger
appender.a_console.type=Console
appender.a_console.name=StdoutAppender
appender.a_console.layout.type=PatternLayout
appender.a_console.layout.pattern=%d{ISO8601} [%t] %-5p (%F\:%L) - %m%n
# File logger
appender.a_rolling.type=RollingFile
appender.a_rolling.name=DailyRollingAppender
appender.a_rolling.layout.pattern=%d{ISO8601} [%t] %-5p (%F\:%L) - %m%n
appender.a_rolling.fileName=log4j2-sample.log
appender.a_rolling.filePattern=log4j2-sample-%d{yyyy-MM-dd}.log
appender.a_rolling.layout.type=PatternLayout
appender.a_rolling.policies.type=Policies
appender.a_rolling.policies.time.type=TimeBasedTriggeringPolicy
appender.a_rolling.policies.time.interval=1
This is the simplest config if you just need to use console appender.
name=config
appenders=console
appender.console.type=Console
appender.console.name=STDOUT
appender.console.layout.type=PatternLayout
#appender.console.layout.pattern =%d{HH:mm:ss} [%t] %c{1} [%-5level] - %msg%n
appender.console.layout.pattern=%d{dd-MM-yyyy HH:mm:ss} [%-5p] (%F:%L) - %m%n
rootLogger.level=info
rootLogger.appenderRefs=stdout
rootLogger.appenderRef.stdout.ref=STDOUT
#if you want to make package specific configuration
#logger.org.apache=warn

Resources