I am using log4j for logging, and a property file for configuration. Currently, my log files are too big (3.5 GB is too large for a log file). So think I need to use RollingFileAppender - but when I do so the log file continues to grow overly large. I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
For the record, my current configuration looks like this:
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.layout=com.mycompany.util.log.Log4JSimpleLayout
log4j.appender.MAIN_LOG.DatePattern='.'yyyy-MM-dd
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=99
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
An alternative to RollingFileAppender would also be a fine solution.
I believe I have just misconfigured it; does anyone have a working example of configuring RollingFileAppender?
This seems to work fine for me #mcherm. See below.
Are you sure that you are using the log4j.properties that you think you are? Try changing the .File to another path to see if log output goes to the new file. What version of log4j are you using? I'm running 1.2.15.
Hope this helps.
I created the following test program:
package com.j256.ormlite;
import org.apache.log4j.Logger;
public class Foo {
private static Logger logger = Logger.getLogger(Foo.class);
public static void main(String[] args) {
for (int x = 0; x < 10000000; x++) {
logger.error("goodness this shouldn't be happening to us right here!!!!");
}
}
}
My log4j.properties file holds:
log4j.appender.MAIN_LOG=org.apache.log4j.RollingFileAppender
log4j.appender.MAIN_LOG.File=${catalina.base}/logs/webtop.log
log4j.appender.MAIN_LOG.layout=com.j256.ormlite.Log4JSimpleLayout
log4j.appender.MAIN_LOG.MaxFileSize=10MB
log4j.appender.MAIN_LOG.MaxBackupIndex=5
log4j.appender.MAIN_LOG.append=true
log4j.rootCategory=ALL, MAIN_LOG
Notice that I removed the DatePattern which wasn't valid for my RollingFileAppender. My layout is:
package com.j256.ormlite;
import org.apache.log4j.spi.LoggingEvent;
public class Log4JSimpleLayout extends org.apache.log4j.Layout {
#Override
public String format(LoggingEvent event) {
return "log message = " + event.getMessage().toString() + "\n";
}
#Override
public boolean ignoresThrowable() {
return true;
}
public void activateOptions() {
}
}
Running with -Dcatalina.base=/tmp/ I get files in /tmp/logs/ which go up to index #5 and are 10mb in size. If I tune the MaxFileSize or the MaxBackupIndex, it adjusts appropriately.
Your issue might be with the fact that you are specifying a DatePattern.
The DatePattern is meant to be used with the DailyRollingFileAppender to specify the date that the log file should roll.
I don't believe it can be used in conjunction with the MaxFileSize and MaxBackupIndex attributes.
Log4j lets you roll files based on file size or date but not both.
When we need log files to be rolled on a daily basis, we should be using DailyRollingFileAppender instead of RollingFileAppender.
You do not need to specify the MaxFileSize limit instead only DatePattern is enough for rolling files based on frequency.
I have tried the below configuration in log4j.properties file for rolling log files every minute.
log4j.appender.infoAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.infoAppender.Threshold=INFO
log4j.appender.infoAppender.DatePattern='.' yyyy-MM-dd HH-mm
log4j.appender.infoAppender.File=C:/logs/info.log
Start by setting the -Dlog4j.debug JVM parameter. That prints out a few useful lines of debug information showing which config file it's found and is using, etc. That should give you some clues to what's going wrong.
See http://logging.apache.org/log4j/1.2/manual.html
Related
I am trying to use the default active web library to log everything to a separate log file. Right now I'm running everything under IntelliJ (via mvn jetty:run) and all the logging is coming out to the console only.
I tried added a log4j.properties file in the WEB-INF directory; didn't work (I have not added log4j dependency to my pom as I don't want it in there).
Looking a slf4j, I cannot find any properties or config file that let's me define how I would log to a specific log file. And, I'm not sure what logging AW uses, so it's hard to see what I need to configure.
Stuck at this point, and just googling and reading thru the slf4j site to try to get this working.
I general, if you want logging done by Log4j and Slf4j, you will need to add appropriate dependencies. Here is the configuration from one of our projects:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
this will bring appropriate deps.
Here is the contents of the log4j.properties file.
log4j.rootLogger=INFO, ${logger-name}, SPLUNK
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Target=System.out
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.SPLUNK=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SPLUNK.File=${catalina.home}/logs/worker-splunk.log
log4j.appender.SPLUNK.Append=true
log4j.appender.SPLUNK.Encoding=UTF-8
# This is a filter that will filter out junk we do not want to sent to Splunk
log4j.appender.SPLUNK.filter.1=app.utils.SplunkLogFilter
log4j.appender.SPLUNK.layout=org.javalite.logging.JsonLog4jLayout
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.File=${catalina.home}/logs/worker.log
log4j.appender.FILE.Append=true
log4j.appender.FILE.Encoding=UTF-8
log4j.appender.FILE.layout=org.javalite.logging.JsonLog4jLayout
The logger-name is a Maven- filtered property. Locally it is resolved to CONSOLE and when the app is built, it resolves to FILE. This way, we can observe log on the console during development.
The class SplunkLogFilter looks like this:
import org.apache.log4j.spi.Filter;
import org.apache.log4j.spi.LoggingEvent;
public class SplunkLogFilter extends Filter {
private static final String[] EXCLUDED_LOGGERS = new String[]{"***ServiceImpl", "app.utils.ProcessUtil"};
private static final String[] EXCLUDED_MESSAGES = new String[]{"****Command"};
#Override
public int decide(LoggingEvent event) {
String loggerName = event.getLoggerName();
for (String excludedLogger : EXCLUDED_LOGGERS) {
if(loggerName.equals(excludedLogger)){
return Filter.DENY;
}
}
String message = event.getMessage().toString();
for (String excludedMessage : EXCLUDED_MESSAGES) {
if(message.contains(excludedMessage)){
return Filter.DENY;
}
}
return Filter.NEUTRAL;
}
}
So, we are logging into two files in parallel, where one is shipped into Splunk.
The Splunk file is smaller, so we pay less for Splunk, but we retain full files just in case.
I am trying to figure out how to use the ExtendedLog4NetLogger.cs
to change the log file path dynamically at runtime or using the LoggingFacility?
This should be something similar to using log4net directly like this:
log4net.GlobalContext.Properties["LogName"] = logName;
How would I access the ExtendedLogger if I register log4net integration like this:
container.AddFacility<LoggingFacility>(f => f.UseLog4Net());
Update: I use the following code to register the Extended Logger
container.AddFacility<LoggingFacility>(LoggerImplementation.ExtendedLog4net).WithConfig(configFile).ToLog(Logger));
I get no runtime exceptions and the logger is not a null instance but I don't see the log file created at all using the global properties, I also set the config value to this for the appender: <file type="log4net.Util.PatternString" value="%property{LogName}" />
If I just set the file property in the config file to full path it does work. I am wondering if it is not working because configuration is done before setting the global variable.
extendedlogger.GlobalProperties["logName"] = logName;
To enable extended logger you need to do:
container.AddFacility<LoggingFacility>(f => f.LogUsing(LoggerImplementation.ExtendedLog4net));
1. How would I access the ExtendedLogger if I register log4net integration like this:
Using the dependency injection you can expect IExtendedLogger object in the place where you need it.
2. I am wondering if it is not working because configuration is done before setting the global variable.
That's right. You need to reconfigure log4net after setting a property.
Here is an example:
using Castle.Core.Logging;
using log4net.Config;
class MyClass {
private readonly IExtendedLogger _extendedLogger;
public MyClass(IExtendedLogger extendedLogger) {
_extendedLogger = extendedLogger;
}
public void MyFunction() {
_extendedlogger.GlobalProperties["logName"] = logName;
XmlConfigurator.Configure();
_extendedlogger.Error("my error message");
}
}
The functionality that I need is writing a header line at the beginning of the configured log file. The log file should, in addition, get rolled over based on a time pattern (I'm talking logback 1.0.7).
So, I'm thinking of writing an Appender - although I'm not sure whether it's not a custom Layout that I actually need.
1) Appender
Per logback's documentation, the right approach is to extend AppenderSkeleton, but then how would I combine this with the RollingFileAppender (to make the file rollover?)
On the other hand, if I extend RollingFileAppender, what method do I override to just decorate the existing functionality? How do I tell it to write that particular String only at the beginning of the file?
2) Layout
Analogously, the approach seems to be extending LayoutBase, and providing an implementation for doLayout(ILoggingEvent event).
But again, I don't know how to just decorate the behaviour - just adding a new line in the file, rather than disrupting its functionality (because I still want the rest of the logs to show up properly).
The getFileHeader() in LayoutBase looks promising, but how do I use it? Is it even intended to be overridden by custom layouts? (probably yes, since it's part of the Layout interface, but then how?)
Thank you!
Here I am answering my own question, just in case someone else comes across the same problem.
This is how I eventually did it (don't know however if it's the orthodox way):
Instead of extending AppenderSkeleton, I extended RollingFileAppender (to keep the rollover functionality), and overrode its openFile() method. In here I could manipulate the log file and write the header in it, after letting it do whatever it needed to do by default. Like this:
public void openFile(String fileName) throws IOException {
super.openFile(fileName);
File activeFile = new File(getFile());
if (activeFile.exists() && activeFile.isFile() && activeFile.length() == 0) {
FileUtils.writeStringToFile(activeFile, header);
}
}
I configured the header in logback.xml, as simple as this: <header> value </header>. This injects it in the header field of my new appender.
Seems to work without problems, but please do post if you know a better way!
Your solution has a problem: it deletes the first line of log of each new file. I think this is because you write the header whereas the file is open by logback.
I've found another solution that does not have this problem:
public void openFile(String fileName) throws IOException
{
super.openFile(fileName);
File activeFile = new File(getFile());
if (activeFile.exists() && activeFile.isFile() && activeFile.length() == 0)
{
lock.lock();
try
{
new PrintWriter(new OutputStreamWriter(getOutputStream(), StandardCharsets.UTF_8), true).println("your header");
}
finally
{
lock.unlock();
}
}
}
I want to log in to multiple log files(flume and console). How to set log4j as package level?ie com.mypackage.myclass into flume and other packages into console..
First of all you need to configure log4j to have two named loggers, one that sends to the Console appender, and one that sends to Flume. You can then write use a proxy class for making your logging calls that routes the log4j calls to the different loggers depending on the package the caller is in. You can do this by accessing the stack of the current thread, like so:
public class Logger
{
public static org.apache.log4j.Logger getLogger()
{
// this will get the calling frame, 0=Thread, 1=this, 2=caller
StackTraceElement stackElement = Thread.currentThread().getStackTrace()[2];
if(stackElement.getClassName().startsWith("the.package.that.goes.to.flume"))
{
return org.apache.log4j.Logger.getLogger("Flume");
}
else
{
return org.apache.log4j.Logger.getLogger("Console");
}
}
}
}
The code above is assuming you have named your two loggers 'Flume' and 'Console'.
When ever you make a logging call in your app, use Logger.getLogger() rather than going to log4j directly.
Check this blog post
http://veerasundar.com/blog/2009/07/log4j-tutorial-adding-log4j-logging-to-your-project/
It has a complete PDF for download on how to add log4j to project.
You need to define categories for different packages.Everything is explained in above PDF.
Hope it helps.
Is there a way to figure out where is Log4J picking the configuration file from?
I tried to change my log4j.xml and the changes were not reflected in Log4j behaviour. I deleted log4j.xml and funnily enough, Log4J is still working with the old behaviour. So it must be picking some configuration file which is available in my namespace. But the question is how can I figure out which one. Is there a way to do that? there are so many different dependencies on jars etc, so one of them must contains a log4j.xml or log4j.properties that is overriding my changes.
Any ideas?
When running your application you can set the system property -Dlog4j.debug. log4j will produce debug output in this case and tells you about how it resolves the path to the log4j.xml, e.g.:
log4j: Trying to find [log4j.xml] using context classloader sun.misc.Launcher$AppClassLoader#11b86e7.
log4j: Using URL [file:/C:/develop/workspace/foobar/target/classes/log4j.xml] for automatic log4j configuration.
log4j: Preferred configurator class: org.apache.log4j.xml.DOMConfigurator
To set a path explicit use the system property -Dlog4j.configuration=file:c:/pathToFile as described here.
Activating -Dlog4j.debug like suggested by FrVaBe in his answer is a good solution. However, I wanted to print or log the location on startup without activating log4j's debug mode. I created a little util method, that is mostly a copy of log4j's default init routine, which returns the URL of the configuration file.
Maybe it's useful for someone else:
/**
* Get the URL of the log4j config file. This is mostly copied from org.apache.log4j.LogManager's default init routine.
*
* #return log4j config url
*/
public static URL getLog4JConfigurationUrl(){
/** Search for the properties file log4j.properties in the CLASSPATH. */
String override = OptionConverter.getSystemProperty(LogManager.DEFAULT_INIT_OVERRIDE_KEY, null);
// if there is no default init override, then get the resource
// specified by the user or the default config file.
if (override == null || "false".equalsIgnoreCase(override)){
String configurationOptionStr = OptionConverter.getSystemProperty(LogManager.DEFAULT_CONFIGURATION_KEY, null);
URL url;
// if the user has not specified the log4j.configuration
// property, we search first for the file "log4j.xml" and then
// "log4j.properties"
if (configurationOptionStr == null){
url = Loader.getResource("log4j.xml");
if (url == null){
url = Loader.getResource(LogManager.DEFAULT_CONFIGURATION_FILE);
}
} else {
try {
url = new URL(configurationOptionStr);
} catch (MalformedURLException ex) {
// so, resource is not a URL:
// attempt to get the resource from the class path
url = Loader.getResource(configurationOptionStr);
}
}
if (url == null)
throw new RuntimeException("log4j configuration could not be found!");
return url;
}
throw new RuntimeException("default init is overridden, log4j configuration could not be found!");
}
PS: if you know a way to ask log4j for it's currently used configuration file, let me know.