I want to log in to multiple log files(flume and console). How to set log4j as package level?ie com.mypackage.myclass into flume and other packages into console..
First of all you need to configure log4j to have two named loggers, one that sends to the Console appender, and one that sends to Flume. You can then write use a proxy class for making your logging calls that routes the log4j calls to the different loggers depending on the package the caller is in. You can do this by accessing the stack of the current thread, like so:
public class Logger
{
public static org.apache.log4j.Logger getLogger()
{
// this will get the calling frame, 0=Thread, 1=this, 2=caller
StackTraceElement stackElement = Thread.currentThread().getStackTrace()[2];
if(stackElement.getClassName().startsWith("the.package.that.goes.to.flume"))
{
return org.apache.log4j.Logger.getLogger("Flume");
}
else
{
return org.apache.log4j.Logger.getLogger("Console");
}
}
}
}
The code above is assuming you have named your two loggers 'Flume' and 'Console'.
When ever you make a logging call in your app, use Logger.getLogger() rather than going to log4j directly.
Check this blog post
http://veerasundar.com/blog/2009/07/log4j-tutorial-adding-log4j-logging-to-your-project/
It has a complete PDF for download on how to add log4j to project.
You need to define categories for different packages.Everything is explained in above PDF.
Hope it helps.
Related
I can't be able to add console appender using logger.addAppender method with log4j-over-slf4j 1.7.x dependency. Moreover, I am unable to set target of that particular Console Appender(i.e., SYSTEM_OUT/SYSTEM_ERR).
I have initialized a console appender object and tried to push that reference into addAppender method by typecasting that reference to Appender. But in that case I am unable to set Target/WriterLocation(i.e., SYSTEM_OUT/SYSTEM_ERR) for console appender reference. I have used the below code snippet-
ConsoleAppender ca = new ConsoleAppender();
ca.setWriter(new OutputStreamWriter(System.out)); // this line is not compatible with log4j-over-slf4j jar
ca.setLayout(new PatternLayout("%-5p [%t]: %m%n"));
logger.addAppender(ca);
Please help me to sort out this problem.
What you are doing doesn't make a lot of sense. log4j-over-slf4j will route calls like logger.debug(), logger.info(), etc to slf4j and then presumably to some other logging framework to be logged. Your code is trying to manipulate log4j 1 objects which won't be involved in logging since you are routing log events to SLF4J (which is why many of them aren't supported by log4j-over-slf4j).
In order to help you we would know what logging implementation you really want to use.
I am trying to use the default active web library to log everything to a separate log file. Right now I'm running everything under IntelliJ (via mvn jetty:run) and all the logging is coming out to the console only.
I tried added a log4j.properties file in the WEB-INF directory; didn't work (I have not added log4j dependency to my pom as I don't want it in there).
Looking a slf4j, I cannot find any properties or config file that let's me define how I would log to a specific log file. And, I'm not sure what logging AW uses, so it's hard to see what I need to configure.
Stuck at this point, and just googling and reading thru the slf4j site to try to get this working.
I general, if you want logging done by Log4j and Slf4j, you will need to add appropriate dependencies. Here is the configuration from one of our projects:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
this will bring appropriate deps.
Here is the contents of the log4j.properties file.
log4j.rootLogger=INFO, ${logger-name}, SPLUNK
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Target=System.out
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.SPLUNK=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SPLUNK.File=${catalina.home}/logs/worker-splunk.log
log4j.appender.SPLUNK.Append=true
log4j.appender.SPLUNK.Encoding=UTF-8
# This is a filter that will filter out junk we do not want to sent to Splunk
log4j.appender.SPLUNK.filter.1=app.utils.SplunkLogFilter
log4j.appender.SPLUNK.layout=org.javalite.logging.JsonLog4jLayout
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.File=${catalina.home}/logs/worker.log
log4j.appender.FILE.Append=true
log4j.appender.FILE.Encoding=UTF-8
log4j.appender.FILE.layout=org.javalite.logging.JsonLog4jLayout
The logger-name is a Maven- filtered property. Locally it is resolved to CONSOLE and when the app is built, it resolves to FILE. This way, we can observe log on the console during development.
The class SplunkLogFilter looks like this:
import org.apache.log4j.spi.Filter;
import org.apache.log4j.spi.LoggingEvent;
public class SplunkLogFilter extends Filter {
private static final String[] EXCLUDED_LOGGERS = new String[]{"***ServiceImpl", "app.utils.ProcessUtil"};
private static final String[] EXCLUDED_MESSAGES = new String[]{"****Command"};
#Override
public int decide(LoggingEvent event) {
String loggerName = event.getLoggerName();
for (String excludedLogger : EXCLUDED_LOGGERS) {
if(loggerName.equals(excludedLogger)){
return Filter.DENY;
}
}
String message = event.getMessage().toString();
for (String excludedMessage : EXCLUDED_MESSAGES) {
if(message.contains(excludedMessage)){
return Filter.DENY;
}
}
return Filter.NEUTRAL;
}
}
So, we are logging into two files in parallel, where one is shipped into Splunk.
The Splunk file is smaller, so we pay less for Splunk, but we retain full files just in case.
I am trying to follow this advice to debug my Spray routes, i.e. using logRequestResponse. I am using debug logging level. However, no debug messages appear on the console.
Firstly, the logRequestResponse directive takes an implicit LoggingContext parameter. LoggingContext will log to an implicit akka actor system if there is one in scope (see its Scaladoc for what it will do in other cases). There is an implicit actor system in scope in Spray Testkit, so if you are using that, that's probably the one that will be in scope. That loads from the standard locations (application.conf, reference.conf) so if you want to customise the config, you can do
override def testConfig = ...
Wherever you load the config from, it should contain at least
akka {
loglevel = "DEBUG"
}
to make debug logging show up.
Also, check the config file of your underlying slf4j logging implementation (e.g. log4j) to see where the log statements are being written and what are the minimum log levels in that config file, because that also affects the verbosity of the logging.
Our middleware team assignes logger names to each application and that is how they know where to direct our socket appenders to.
I would like to use the standard Logger.getLogger(Clazz.class) paradigm but that does not work with the above constraint. Also we can't log library statements out to our socket appender which would come in handy a lot.
Is there a fairly painless way to map everything from all loggers to this middleware assigned logger?
I think our middleware group messed up in how the configured the enterprise logging system. It looks like there is a setApplication property on the SocketAppender that should be used instead. Regardless, this is what we have to deal with...
You'd like to redirect your "regular" loggers' output to the "middleware logger" directly, i.e. without setting the middleware logger's appender on all the "regular" loggers, right?
If this is the case, try writing your own appender:
class MiddlewareRedirectingAppender extends AppenderSkeleton {
private Logger middlewareLogger = Logger.getLogger("your 'middleware' logger name");
public void doAppend(LoggingEvent event) {
// implement whatever filtering, etc. you want
middlewareLogger.log(...);
}
}
Attach this appender to your "regular" loggers, or just to the root logger (depending on how your "middleware" logger behaves).
Disclaimer: this is just a loose idea, I haven't tested it.
I am using log4j for logging, and a property file for configuration. Currently, my log files are too big (3.5 GB is too large for a log file). So think I need to use RollingFileAppender - but when I do so the log file continues to grow overly large. I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
For the record, my current configuration looks like this:
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.layout=com.mycompany.util.log.Log4JSimpleLayout
log4j.appender.MAIN_LOG.DatePattern='.'yyyy-MM-dd
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=99
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
An alternative to RollingFileAppender would also be a fine solution.
I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
This seems to work fine for me #mcherm. See below.
Are you sure that you are using the log4j.properties that you think you are? Try changing the .File to another path to see if log output goes to the new file. What version of log4j are you using? I'm running 1.2.15.
Hope this helps.
I created the following test program:
package com.j256.ormlite;
import org.apache.log4j.Logger;
public class Foo {
private static Logger logger = Logger.getLogger(Foo.class);
public static void main(String[] args) {
for (int x = 0; x < 10000000; x++) {
logger.error("goodness this shouldn't be happening to us right here!!!!");
}
}
}
My log4j.properties file holds:
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG.layout=com.j256.ormlite.Log4JSimpleLayout
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=5
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
Notice that I removed the DatePattern which wasn't valid for my RollingFileAppender. My layout is:
package com.j256.ormlite;
import org.apache.log4j.spi.LoggingEvent;
public class Log4JSimpleLayout extends org.apache.log4j.Layout {
#Override
public String format(LoggingEvent event) {
return "log message = " + event.getMessage().toString() + "\n";
}
#Override
public boolean ignoresThrowable() {
return true;
}
public void activateOptions() {
}
}
Running with -Dcatalina.base=/tmp/ I get files in /tmp/logs/ which go up to index #5 and are 10mb in size. If I tune the MaxFileSize or the MaxBackupIndex, it adjusts appropriately.
Your issue might be with the fact that you are specifying a DatePattern.
The DatePattern is meant to be used with the DailyRollingFileAppender to specify the date that the log file should roll.
I don't believe it can be used in conjunction with the MaxFileSize and MaxBackupIndex attributes.
Log4j lets you roll files based on file size or date but not both.
When we need log files to be rolled on a daily basis, we should be using DailyRollingFileAppender instead of RollingFileAppender.
You do not need to specify the MaxFileSize limit instead only DatePattern is enough for rolling files based on frequency.
I have tried the below configuration in log4j.properties file for rolling log files every minute.
log4j.appender.infoAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.infoAppender.Threshold=INFO
log4j.appender.infoAppender.DatePattern='.' yyyy-MM-dd HH-mm
log4j.appender.infoAppender.File=C:/logs/info.log
Start by setting the -Dlog4j.debug JVM parameter. That prints out a few useful lines of debug information showing which config file it's found and is using, etc. That should give you some clues to what's going wrong.
See http://logging.apache.org/log4j/1.2/manual.html