In my log4j file, I have already added the following log level for ContextHandler,
but, ContextHandler stuff still prints the log.
log4j.logger.org.eclipse.jetty.server.handler.ContextHandler=WARN
log4j.logger.org.mortbay.jetty.handler.ContextHandler=WARN
I would ask how to suppress ContextHandler logs.
Adding this to your log4j.properties should help:
log4j.category.org.spark_project.jetty=WARN
Update: Since Spark 3.0 it's changed to
log4j.category.org.sparkproject.jetty=WARN (thanks #ivan-vergiliev)
Related
I have looked into:
log4j2.properties in /etc/spark/conf
yarn-site.xml
yarn-env.sh (via YARN_LOG_DIR it is not getting set. In fact while running a job there is no env variable YARN_LOG_DIR in my executors)
log4j.properties in /etc/hadoop/conf
Where can I find and modify the yarn.nodemanager.log-dirs property?
To find this, we need to traverse some of Hadoop's source code:
yarn.nodemanager.log-dirs defaults to ${yarn.log.dir}/userlogs.
yarn.log.dir defaults to $HADOOP_LOG_DIR
$HADOOP_LOG_DIR defaults to ${HADOOP_HOME}/logs
So, have a look at $HADOOP_HOME/logs/userlogs to see whether you find something in there!
If you want to edit it, you can do either of the following:
edit $HADOOP_HOME
edit $HADOOP_LOG_DIR
add -Dyarn.log.dir=<your_chosen_value> to your spark application
add -Dyarn.nodemanager.log-dirs=<your_chosen_value> to your spark application
I'm working on a spark streaming job which runs on standalone mode. The executors by default append the logs in $SPARK_HOME/work/app_idxxxx/stderr and stdout files. Now the problem comes when app runs for a long time say a month or more and it generates a lot of logs inside stderr file. I would like to rollup the stderr daily for a week and archive(delete) that after that. I changed the log4j.properties with org.apache.log4j.RollingFileAppender and directed the logs to a file instead of stderr but the file doesn't respect the rolling and keeps growing.
Creating a cron job to do that is also not working since spark has a pointer to that specific file and changing the name probably not working.
I could't find any documentations for these specific logs. I really appreciate for any help.
After digging more, I finally found how to resolve the issue and I post it here so that the next person don't go through all this suffer and trial/error.
The setting for those logs are in two different places. One in $SPARK_HOME/conf/spark-default.conf add these three lines below in each executor:
spark.executor.logs.rolling.time.interval daily
spark.executor.logs.rolling.strategy time
spark.executor.logs.rolling.maxRetainedFiles 7
The other file that you need to change in each executor is $SPARK_HOME/conf/spark-env.sh add the following line:
SPARK_WORKER_OPTS="$SPARK_WORKER_OPTS -Dspark.worker.cleanup.enabled=true -Dspark.worker.cleanup.interval=1800
-Dspark.worker.cleanup.appDataTtl=864000
-Dspark.executor.logs.rolling.strategy=time
-Dspark.executor.logs.rolling.time.interval=daily
-Dspark.executor.logs.rolling.maxRetainedFiles=7 "
export SPARK_WORKER_OPTS
After these changes it started working properly. Hope this helps some people :)
if you are in standalone mode, just export an environment is enough:
export SPARK_WORKER_OPTS="-Dspark.executor.logs.rolling.strategy=time -Dspark.executor.logs.rolling.time.interval=daily -Dspark.executor.logs.rolling.maxRetainedFiles=7"
you can also refer to: http://apache-spark-user-list.1001560.n3.nabble.com/Executor-Log-Rotation-Is-Not-Working-td18024.html
When I launch my tests in localhost:9000/#tests# in console I can see long-long log with DEBUG log-level. I want to get rid of that. I tried to put to my application.conf
line like:
application.log=INFO
or even this line:
application.log.system.out=off
But it does not work. Maybe I should put log4j setting specially for test context somehow..?
You can simply prefix those two rules with the "test" id.
%test.application.log.system=off
%test.application.log=ERROR
And by the way log levels are : trace -> debug -> info -> warn -> error -> fatal
So when displaying info, you are displaying a lot of information.
What's your log4j.properties configuration? Most likely you have Debug enabled in there.
If you have this setting enabled:
application.log.path=/log4j.properties
try to disable it and run the tests to verify this is changing the logs. If that's the issue then modify that line to:
%test.application.log.path=/log4j-test.properties
%dev.application.log.path=/log4j-dev.properties
and create the 2 versions of log4j.properties, so you have the right logging level in each.
I am trying to track down a problem with GWT. I get an error which I want to track down to the source but log4j truncates the stacktrace by indicating something like "... 26 more" hence I cannot determine the exact location for the problem. I tried finding out if there is anyway to prevent that truncation but have been unable to find any option that will accomplish that. Is this truncation a configurable feature?
I am running Tomcat 6.0.31, Spring 3.0 (for backend), GWT 2.1.0.
Thanks
It isn't log4j truncating the stack trace, it's standard Java. And it's actually only making the trace more readable because those lines that were omitted were already output by the 'enclosing' exception. See Throwable javadoc.
I just spent the better part of 3 hours trying to get my Rails application logging with Log4j. I've finally got it working, but I'm not sure if what I did is correct. I tried various methods to no avail until my very last attempt. So I'm really looking for some validation here, perhaps some pointers and tips as well -- anything would be appreciated to be honest. I've summarized all my feeble methods into three attempts below. I'm hoping for some enlightenment on where I went wrong with each attempt -- even if it means I get ripped up.
Thanks for the help in advance!
System Specs
Rails 3.0
Windows Server 2008
Log4j 1.2
Tomact 6.0.29
Java 6
Attempt 1 - Configured Tomcat to Use Log4J
I basically followed the guide on the Apache Tomcat website here. The steps are:
Create a log4j.properties file in $CATALINA_HOME/lib
Download and copy the log4j-x.y.z.jar into $CATALINA_HOME/lib
Replace $CATALINA_HOME/bin/tomcat-juli.jar with the tomcat-juli.jar from the Apache Tomcat Extras folder
Copy tomcat-juli-adapters.jar from the Apache Tomcat Extras folder into $CATALINA_HOME/lib
Delete $CATALINA_BASE/conf/logging.properties
Start Tomcat (as a service)
Expected Results According to the Guide
I should have seen a tomcat.log file in my $CATALINA_BASE/logs folder.
Actual Results
No tomcat.log
Saw three of the standard logs instead
jakarta_service_20101231.log
stderr_20101231.log
stdout_20101231.log
Question
Shouldn't I have at least seen a tomcat.log file?
Attempt 2 - Use default Tomcat logging (commons-logging)
Reverted all the changes from the previous setup
Modified $CATALINA_BASE/conf/logging.properties by doing the following:
Adding a setting for my application in the handlers line: 5rails3.org.apache.juli.FileHandler
Adding Handler specific properties
5rails3.org.apache.juli.FileHandler.level = FINE
5rails3.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
5rails3.org.apache.juli.FileHandler.prefix = rails3.
Adding Facility specific properties
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rails3].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/rails3].handlers = 4host-manager.org.apache.juli.FileHandler
Modified my web.xml by adding the following context parameter as per the Logging section of the jruby-rack README (I also modified my warbler.rb accordingly, but I opted to change the web.xml directly to test things faster).
<context-param>
<param-name>jruby.rack.logging</param-name>
<param-value>commons_logging</param-value>
</context-param>
Restarted Tomcat
Results
A log file was created (rails3.log), however there was no log information in the file.
Attempt 2A - Use Log4j with existing set up
I decided to give Log4j another whirl with this new web.xml setting.
Copied the log4j.jar into my WEB-INF/lib folder
Created a log4j.properties file and put it into WEB-INF/classes
log4j.rootLogger=INFO, R
log4j.logger.javax.servlet=DEBUG
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=${catalina.base}/logs/rails3.log
log4j.appender.R.MaxFileSize=5036KB
log4j.appender.R.MaxBackupIndex=4
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss} [%t] %-5p %c %x - %m%n
Restarted Tomcat
Results
Same as Attempt 2
NOTE: I used log4j.logger.javax.servlet=DEBUG because I read in the jruby-rack README that all logging output is automatically redirected to the javax.servlet.ServletContext#log method. So I thought this would capture it, but I was obviously wrong.
Question
Why didn't this work?
Isn't Log4J using the commons_logging API?
Attempt 3 - Tried out slf4j (WORKED)
A bit uncertain as to why Attempt 2A didn't work, I thought to myself, maybe I can't use commons_logging for the jruby.rack.logging parameter because it's probably not using commons_logging API... (but I was still not sure). I saw slf4j as an option. I have never heard of it and out of curiosity, I decided to look it up. After reading about it briefly, I thought it was as good of a shot as any and decided to try it out following the instructions here.
Continuing from the setup of Attempt 2A:
Copied slf4j-api-1.6.1.jar and slf4j-simple-1.6.1.jar into my WEB-INF/lib folder
I also copied slf4j-log4j12-1.6.1.jar into my WEB-INF/lib folder
Restarted Tomcat
And VIOLA! I now have logging information going into my rails3.log file.
So the big question is:
WTF?
Even though logging seems to be working now, I'm really not sure if what I did is right. So like I said earlier, I'm really looking for some validation more or less. I'd also appreciate any pointers/tips/advice if you have any. Thanks!
We also did various experiments and finally settled with slf4j option. Coming from Java background we knew slf4j, so we didn't go further.
<context-param>
<param-name>jruby.rack.logging</param-name>
<param-value>slf4j</param-value>
</context-param>
btw, there is no need to copy slf4j-simple-1.6.1.jar into tomcat/lib or WEB-INF/lib when using slf4j-log4jxxx.jar
It's possible that your early tests with commons-logging were showing zero bytes files because of bufferSize.
If your entries get flushed to the file when you stop Tomcat then this is indeed the case and you are falling foul of the default 8Kb buffer.
To disable, for immediate flushing try...
org.apache.juli.FileHandler.bufferSize = -1
on the specific handler...
So for localhost...
2localhost.org.apache.juli.FileHandler.bufferSize = -1