I am trying to use the default active web library to log everything to a separate log file. Right now I'm running everything under IntelliJ (via mvn jetty:run) and all the logging is coming out to the console only.
I tried added a log4j.properties file in the WEB-INF directory; didn't work (I have not added log4j dependency to my pom as I don't want it in there).
Looking a slf4j, I cannot find any properties or config file that let's me define how I would log to a specific log file. And, I'm not sure what logging AW uses, so it's hard to see what I need to configure.
Stuck at this point, and just googling and reading thru the slf4j site to try to get this working.
I general, if you want logging done by Log4j and Slf4j, you will need to add appropriate dependencies. Here is the configuration from one of our projects:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
this will bring appropriate deps.
Here is the contents of the log4j.properties file.
log4j.rootLogger=INFO, ${logger-name}, SPLUNK
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Target=System.out
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.SPLUNK=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SPLUNK.File=${catalina.home}/logs/worker-splunk.log
log4j.appender.SPLUNK.Append=true
log4j.appender.SPLUNK.Encoding=UTF-8
# This is a filter that will filter out junk we do not want to sent to Splunk
log4j.appender.SPLUNK.filter.1=app.utils.SplunkLogFilter
log4j.appender.SPLUNK.layout=org.javalite.logging.JsonLog4jLayout
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.File=${catalina.home}/logs/worker.log
log4j.appender.FILE.Append=true
log4j.appender.FILE.Encoding=UTF-8
log4j.appender.FILE.layout=org.javalite.logging.JsonLog4jLayout
The logger-name is a Maven- filtered property. Locally it is resolved to CONSOLE and when the app is built, it resolves to FILE. This way, we can observe log on the console during development.
The class SplunkLogFilter looks like this:
import org.apache.log4j.spi.Filter;
import org.apache.log4j.spi.LoggingEvent;
public class SplunkLogFilter extends Filter {
private static final String[] EXCLUDED_LOGGERS = new String[]{"***ServiceImpl", "app.utils.ProcessUtil"};
private static final String[] EXCLUDED_MESSAGES = new String[]{"****Command"};
#Override
public int decide(LoggingEvent event) {
String loggerName = event.getLoggerName();
for (String excludedLogger : EXCLUDED_LOGGERS) {
if(loggerName.equals(excludedLogger)){
return Filter.DENY;
}
}
String message = event.getMessage().toString();
for (String excludedMessage : EXCLUDED_MESSAGES) {
if(message.contains(excludedMessage)){
return Filter.DENY;
}
}
return Filter.NEUTRAL;
}
}
So, we are logging into two files in parallel, where one is shipped into Splunk.
The Splunk file is smaller, so we pay less for Splunk, but we retain full files just in case.
Related
I am trying to figure out how to use the ExtendedLog4NetLogger.cs
to change the log file path dynamically at runtime or using the LoggingFacility?
This should be something similar to using log4net directly like this:
log4net.GlobalContext.Properties["LogName"] = logName;
How would I access the ExtendedLogger if I register log4net integration like this:
container.AddFacility<LoggingFacility>(f => f.UseLog4Net());
Update: I use the following code to register the Extended Logger
container.AddFacility<LoggingFacility>(LoggerImplementation.ExtendedLog4net).WithConfig(configFile).ToLog(Logger));
I get no runtime exceptions and the logger is not a null instance but I don't see the log file created at all using the global properties, I also set the config value to this for the appender: <file type="log4net.Util.PatternString" value="%property{LogName}" />
If I just set the file property in the config file to full path it does work. I am wondering if it is not working because configuration is done before setting the global variable.
extendedlogger.GlobalProperties["logName"] = logName;
To enable extended logger you need to do:
container.AddFacility<LoggingFacility>(f => f.LogUsing(LoggerImplementation.ExtendedLog4net));
1. How would I access the ExtendedLogger if I register log4net integration like this:
Using the dependency injection you can expect IExtendedLogger object in the place where you need it.
2. I am wondering if it is not working because configuration is done before setting the global variable.
That's right. You need to reconfigure log4net after setting a property.
Here is an example:
using Castle.Core.Logging;
using log4net.Config;
class MyClass {
private readonly IExtendedLogger _extendedLogger;
public MyClass(IExtendedLogger extendedLogger) {
_extendedLogger = extendedLogger;
}
public void MyFunction() {
_extendedlogger.GlobalProperties["logName"] = logName;
XmlConfigurator.Configure();
_extendedlogger.Error("my error message");
}
}
I want to log in to multiple log files(flume and console). How to set log4j as package level?ie com.mypackage.myclass into flume and other packages into console..
First of all you need to configure log4j to have two named loggers, one that sends to the Console appender, and one that sends to Flume. You can then write use a proxy class for making your logging calls that routes the log4j calls to the different loggers depending on the package the caller is in. You can do this by accessing the stack of the current thread, like so:
public class Logger
{
public static org.apache.log4j.Logger getLogger()
{
// this will get the calling frame, 0=Thread, 1=this, 2=caller
StackTraceElement stackElement = Thread.currentThread().getStackTrace()[2];
if(stackElement.getClassName().startsWith("the.package.that.goes.to.flume"))
{
return org.apache.log4j.Logger.getLogger("Flume");
}
else
{
return org.apache.log4j.Logger.getLogger("Console");
}
}
}
}
The code above is assuming you have named your two loggers 'Flume' and 'Console'.
When ever you make a logging call in your app, use Logger.getLogger() rather than going to log4j directly.
Check this blog post
http://veerasundar.com/blog/2009/07/log4j-tutorial-adding-log4j-logging-to-your-project/
It has a complete PDF for download on how to add log4j to project.
You need to define categories for different packages.Everything is explained in above PDF.
Hope it helps.
Is there a way to figure out where is Log4J picking the configuration file from?
I tried to change my log4j.xml and the changes were not reflected in Log4j behaviour. I deleted log4j.xml and funnily enough, Log4J is still working with the old behaviour. So it must be picking some configuration file which is available in my namespace. But the question is how can I figure out which one. Is there a way to do that? there are so many different dependencies on jars etc, so one of them must contains a log4j.xml or log4j.properties that is overriding my changes.
Any ideas?
When running your application you can set the system property -Dlog4j.debug. log4j will produce debug output in this case and tells you about how it resolves the path to the log4j.xml, e.g.:
log4j: Trying to find [log4j.xml] using context classloader sun.misc.Launcher$AppClassLoader#11b86e7.
log4j: Using URL [file:/C:/develop/workspace/foobar/target/classes/log4j.xml] for automatic log4j configuration.
log4j: Preferred configurator class: org.apache.log4j.xml.DOMConfigurator
To set a path explicit use the system property -Dlog4j.configuration=file:c:/pathToFile as described here.
Activating -Dlog4j.debug like suggested by FrVaBe in his answer is a good solution. However, I wanted to print or log the location on startup without activating log4j's debug mode. I created a little util method, that is mostly a copy of log4j's default init routine, which returns the URL of the configuration file.
Maybe it's useful for someone else:
/**
* Get the URL of the log4j config file. This is mostly copied from org.apache.log4j.LogManager's default init routine.
*
* #return log4j config url
*/
public static URL getLog4JConfigurationUrl(){
/** Search for the properties file log4j.properties in the CLASSPATH. */
String override = OptionConverter.getSystemProperty(LogManager.DEFAULT_INIT_OVERRIDE_KEY, null);
// if there is no default init override, then get the resource
// specified by the user or the default config file.
if (override == null || "false".equalsIgnoreCase(override)){
String configurationOptionStr = OptionConverter.getSystemProperty(LogManager.DEFAULT_CONFIGURATION_KEY, null);
URL url;
// if the user has not specified the log4j.configuration
// property, we search first for the file "log4j.xml" and then
// "log4j.properties"
if (configurationOptionStr == null){
url = Loader.getResource("log4j.xml");
if (url == null){
url = Loader.getResource(LogManager.DEFAULT_CONFIGURATION_FILE);
}
} else {
try {
url = new URL(configurationOptionStr);
} catch (MalformedURLException ex) {
// so, resource is not a URL:
// attempt to get the resource from the class path
url = Loader.getResource(configurationOptionStr);
}
}
if (url == null)
throw new RuntimeException("log4j configuration could not be found!");
return url;
}
throw new RuntimeException("default init is overridden, log4j configuration could not be found!");
}
PS: if you know a way to ask log4j for it's currently used configuration file, let me know.
I am using log4j for logging, and a property file for configuration. Currently, my log files are too big (3.5 GB is too large for a log file). So think I need to use RollingFileAppender - but when I do so the log file continues to grow overly large. I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
For the record, my current configuration looks like this:
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.layout=com.mycompany.util.log.Log4JSimpleLayout
log4j.appender.MAIN_LOG.DatePattern='.'yyyy-MM-dd
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=99
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
An alternative to RollingFileAppender would also be a fine solution.
I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
This seems to work fine for me #mcherm. See below.
Are you sure that you are using the log4j.properties that you think you are? Try changing the .File to another path to see if log output goes to the new file. What version of log4j are you using? I'm running 1.2.15.
Hope this helps.
I created the following test program:
package com.j256.ormlite;
import org.apache.log4j.Logger;
public class Foo {
private static Logger logger = Logger.getLogger(Foo.class);
public static void main(String[] args) {
for (int x = 0; x < 10000000; x++) {
logger.error("goodness this shouldn't be happening to us right here!!!!");
}
}
}
My log4j.properties file holds:
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG.layout=com.j256.ormlite.Log4JSimpleLayout
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=5
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
Notice that I removed the DatePattern which wasn't valid for my RollingFileAppender. My layout is:
package com.j256.ormlite;
import org.apache.log4j.spi.LoggingEvent;
public class Log4JSimpleLayout extends org.apache.log4j.Layout {
#Override
public String format(LoggingEvent event) {
return "log message = " + event.getMessage().toString() + "\n";
}
#Override
public boolean ignoresThrowable() {
return true;
}
public void activateOptions() {
}
}
Running with -Dcatalina.base=/tmp/ I get files in /tmp/logs/ which go up to index #5 and are 10mb in size. If I tune the MaxFileSize or the MaxBackupIndex, it adjusts appropriately.
Your issue might be with the fact that you are specifying a DatePattern.
The DatePattern is meant to be used with the DailyRollingFileAppender to specify the date that the log file should roll.
I don't believe it can be used in conjunction with the MaxFileSize and MaxBackupIndex attributes.
Log4j lets you roll files based on file size or date but not both.
When we need log files to be rolled on a daily basis, we should be using DailyRollingFileAppender instead of RollingFileAppender.
You do not need to specify the MaxFileSize limit instead only DatePattern is enough for rolling files based on frequency.
I have tried the below configuration in log4j.properties file for rolling log files every minute.
log4j.appender.infoAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.infoAppender.Threshold=INFO
log4j.appender.infoAppender.DatePattern='.' yyyy-MM-dd HH-mm
log4j.appender.infoAppender.File=C:/logs/info.log
Start by setting the -Dlog4j.debug JVM parameter. That prints out a few useful lines of debug information showing which config file it's found and is using, etc. That should give you some clues to what's going wrong.
See http://logging.apache.org/log4j/1.2/manual.html
We have a modular application where modules have their own log4j logs (i.e. communication log and error log). The appenders and categories for these are all configured in the core log4j XML, but not all modules are always installed.
The DailyRollingFileAppender creates its file regardless of use and that exposes the full set of modules although not present and as some of them are customer specific we'd like to hide logs not in use.
Is there a way to make DailyRollingFileAppender create its file on first use instead of automatically at startup?
I had the same problem, so I have extended the standard FileAppender class and I have created a new LazyFileAppender that is a FileAppender that lazily initialize the log file(creates it only when the first write operation happens).
The LazyFileAppender and some other additions to the standard log4j library can be found into a simple library that I have created : log4j-additions .
You can look at the source to develop your own extension or you can use it as is ...
In Log4j 2, both FileAppender and RollingFileAppender has the parameter "createOnDemand" which can be used to configure to create the log file only when a log event passed to the appender.
Example:
<RollingFile name="LogFile" fileName="test.log" filePattern="test-%i.log.gz" createOnDemand="true">
<Policies>
<SizeBasedTriggeringPolicy size="1MB"/>
</Policies>
<DefaultRolloverStrategy max="5"/>
</RollingFile>
More details here: https://logging.apache.org/log4j/2.x/manual/appenders.html#RollingRandomAccessFileAppender
The file appenders have no option to lazily create the log files - the setFile method automatically creates the file if it doesn't already exist: ostream = new FileOutputStream(fileName, append);
You'll have to extend the appender and overwrite the file initialisation code yourself to get the behaviour you're after.
Extend the standard FileAppender class was unsuccessful for me. So I have found an other solution using appenders programmatically to create log files on demand only (and with timestamp in the name file). I have written these two methods :
public void startLog() {
SimpleDateFormat sdf_long = new SimpleDateFormat("yyyy_MM_dd_HH_mm_ss");
FileAppender fa = new FileAppender();
fa.setName("foo");
fa.setFile(sdf_long.format(new Date()) + ".log");
fa.setLayout(new PatternLayout("%d{HH:mm:ss.SSS} %m%n"));
fa.setThreshold(Level.DEBUG);
fa.setAppend(true);
fa.activateOptions();
Logger.getRootLogger().addAppender(fa);
}
public void stopLog() {
Logger.getRootLogger().getAppender("foo").close();
Logger.getRootLogger().removeAppender("foo");
}
My log4j.properties file only configures the console appender. When I want to start logging I call the startLog() method. When I want to log in an other file I call stopLog() first and then startLog() method.