I am trying to clean up the logging configuration one of our applications currently uses for it's log4net implementation. The application is called with an argument containing the filepath to an XML file. That XML file includes various configuration information, one of which is the logger name to use.
Currently, every configuration file has it's own logger, and every logger has their own appenders. As an example we would have:
logger job1
logger job2
logger job3
and
appender consoleJob1
appender consoleJob2
appender consoleJob3
appender rollingFileJob1
appender rollingFileJob2
appender rollingFileJob3
appender smtpJob1
appender smtpJob2
appender smtpJob3
Even though each of those appenders have nearly identical configurations. With hundreds of configuration files, this logging configuration section of the app.config is quite large. I believe I can consolidate most of these down to just a handful of generic appenders (Console, rollingFile, smtpToIT, smptpToSupport etc), and change each logger to use the generic appenders.
A big roadblock to how I visualize this working, is that each appender would need to use the logger's name in the configuration somewhere. Is there a variable or setting I can use in an appender that will allow the appender to use the logger's name? For instance, the RollingFileAppender should log to '\log[loggername].txt' The smtpAppender should have a subject of 'Log for [loggername] on MM\DD\YYYY'.
I took a look at http://logging.apache.org/log4net/release/config-examples.html, and believe I understand how the date can be added but I don't see anything about accessing the logger name from within the appender.
Is there anyway to access a property of the logger being used within the appender? Further, am I understanding how log4net is meant to be configured, with appenders being re-used among multiple loggers? I hadn't heard of log4net until a few weeks ago so I might be going about this all wrong.
I found the solution elsewhere, in an answer to a different question. Unfortunately I don't remember where I found it and only came back to my original question because of Newtopian's answer.
You can set a Global property through the following line:
log4net.GlobalContext.Properties["id"] = "SomeText";
Then refer to it in the appender as I did below:
<appender name="NewAuto2ProcessingTestFILE" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="Log/%property{id}NewAuto2ProcessingTest.txt" />
<appendToFile value="true" />
<maxSizeRollBackups value="10" />
<staticLogFileName value="true" />
<datePattern value=".yyyy-MM-dd.\\t\\x\\t" />
<rollingStyle value="Date" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%d [%t] %-5p - %m%n" />
</layout>
</appender>
Was looking for the same thing, unfortunately it seems it's not possible out of the box, check the doc here about the accepted variables in the pattern String.
You'd probably would have to create your own appender, or your own formatter, to gain access to more variables (I suspect both).
Related
I have a WCF service running on a single server, using Log4net to track usage via INFO and WARN level log entries. Using a RollingFileAppender with the following very standard config:
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="\\mylocation\data\PRD\myApp\MyService"/>
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="-yyyy-MM-dd'.log'" />
<staticLogFileName value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
</layout>
</appender>
<root>
<level value="ALL" />
<appender-ref ref="RollingLogFileAppender" />
<appender-ref ref="ADONetAppender_SqlServer" />
</root>
I also use an ADONetAppender, which receives redirected "WARN" level data and writes it to a DB table in SQL server via a stored procedure. The config for this is a bit long, so I have omitted it for readability.
I have this setup in our Dev and TST environments where it has been running fine. In the PRD environment, it seems to generate duplicate log files. The first is named according to my specified pattern i.e. "logfile-yyyy-mm-dd.log". The second file looks like an addition to the first, with the date pattern duplicated i.e. "logfile-yyyy-mm-dd.log.-yyyy-mm-dd.log".
Making this more interesting is that entries contained in the two files overlap by time. File 1 might have entries from 8am to 12am, and file 2 will also contain entries from the same time period. The entries are not duplicates, they are generated by different users of the service. The copies of Files 1 and 2 can be pretty much any size, so this is not an issue of reaching a size or date/time threshold and generating the next required log file.
The DB table entries contain all the expected rows, some contained in each of the log files. These rows are generated only by WARN level logging, and some WARNings appear in each log file.
I have bounced this off some log4net savvy folks in our shop, but nobody has a good idea of what might be causing this duplicate file behavior. Any ideas from Stackland appreciated.
Your date pattern shouldn't have .log after it. I also am unsure why you have two appenders declared in the root. You should be able to get rid of the root altogether, given the rest of your config it has no purpose (assuming you don't have more that isn't in the example).
I've found that this happens when the file being logged to is locked by another thread or process.
I'm assuming Log4Net creates the other file because it can't log to the configured logfile and thus creates a new file and logs to it, but I'd have to go through the log4net code to be sure of that assumption.
Try adding
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
to the appender element to minimize the amount of locking. Just be aware that there is additional overhead associated with using MinimalLock: http://logging.apache.org/log4net/release/sdk/log4net.Appender.FileAppender.MinimalLock.html
I had the same exact problem.
I confirm that was a permission issue. In my case the logfiles were generated by two different accounts (from a scheduled task and sometimes by a manual run via console), and in this case the file name had a duplicate date pattern.
After resetting the permissions to both users (the scheduled process user and the interactive users) the issue did not repeat anymore.
Greets,
Michele
I've got this settings for log4net in the log4net.config to allow multiple threads to write to the same file:
<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender">
<!-- Minimal locking to allow multiple threads to write to the same file -->
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
<file value="log\UI.log"/>
<appendToFile value="true"/>
<rollingStyle value="Date"/>
<maxSizeRollBackups value="30"/>
<datePattern value="-yyyyMMdd"/>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%newline%date [%thread] %-5level [%property{identity}] %logger{3} - %message%newline"/>
</layout>
</appender>
But after midnight the new created log file is being overwritten all the time and thus there is only the last one event in the file. After server restart it all goes right again till the next midnight.
So can anyone say whether this is a config issue or this is just a log4net issue?
The problem was solved by removing the locking model key since I have only one process (IIS, w3wp.exe) which uses the same logger.
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
Since it was said here:
If you use RollingFileAppender things become even worse as several process may try to start rolling the log file concurrently. RollingFileAppender completely ignores the locking model when rolling files, rolling files is simply not compatible with this scenario.
I think you will get unpredictable results.
My guess is that your use of a - sign on the datePattern is confusing the framework so that after the first roll any log triggers a roll event.
What happens when you try this with
<datePattern value="yyyyMMdd" />
per the example here.
To change the rolling period adjust the DatePattern value. For
example, a date pattern of "yyyyMMdd" will roll every day. See
System.Globalization.DateTimeFormatInfo for a list of available
patterns.
My Application is using logging in two manner....1) programatic 2) log4j.xml
I have to create the logs files in two ( 1 using programatic and other using log4j.xml ) locations.
Programatic way ( have one more properties file in which all the things are mention like log level and all....lets say..thorugh this...file is getting created..name as "SAS_VP.log") :
Enumeration loggers = Logger.getRootLogger().getLoggerRepository().getCurrentLoggers();
......
Logger temp = (Logger)iter.next();
temp.setLevel(level);
log4j.xml
<appender name="FILE" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/LOGS/SAM/SAM_VJ.log"/>
<param name="Threshold" value="DEBUG"/>
<param name="MaxFileSize" value="10000KB"/>
<param name="MaxBackupIndex" value="10"/>
<param name="Append" value="false"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd MMM yyyy HH:mm:ss,SSS} [%t] %5p [%F(% M):%L] %m%n"/>
</layout>
</appender>
<root>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</root>
ISSUE :
Log level which i set programatically overwrite the log level of log4j.xml.Like in log4j.xml is have set the level "Debug" and Programatically I have set the level as "ERROR" then the file (SAM_VJ.log) which is created by log4j.xml only contains ERROR level logs.
How to solve this issue...I want that...my both logging ( programmatic and log4j ) should be indepedent.
Is there anything in log4j in which...if i have set the log level of package "com.sas" is "Debug" then nobody can modify that...something like mutable type
<logger name="com.sam">
<priority value="DEBUG"/>
</logger>
Looking for your suggesstion....
I'm not sure you should really ask for such a functionality.
You're talking about a way to configure the log4j framework, and yes, it supports 3 different ways of configuration:
properties file
xml configuration
programmatically, via your java code
Its ok to me that the programmatic configuration allows to change the configured state of log4j loggers/appenders whatsoever.
Your xml configuration should be loaded during the system startup, and then you apply your java code that overrides the configuration.
If you have a logic of supplying a configuration in Java, why don't you improve your logic and define an error level (from your example) only if you really wish to do it.
Its impossible to configure the same logger to work both with DEBUG level (and up) AND ERROR level (and up).
This is a feature in fact, and not a drawback, since it allows to change the behavior of LOG4j on the running system (without a restart) which is useful for issue tracking.
Of course you can WRAP your loggers so that they won't allow setLevel, but, I really don't see why would you do that.
Hope, this helps
Why not just use two appenders with different log levels instead?You can add levelMax and levelMin params to tell log4j the appender just log those levels.
<param name="LevelMax" value="warn" />
<param name="LevelMin" value="info" />
I am confused as to why INFO statements are making it to the Console. Here is the general setup:
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out"/>
<param name="Threshold" value="DEBUG"/>
<layout .../>
</appender>
<appender name="REST_LOG" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="/logs/rest.log" />
<param name="Threshold" value="INFO" />
....
</appender>
<category name="xyz.web">
<priority value="WARN" />
<appender-ref ref="CONSOLE" />
</category>
<category name="xyz.web.rest">
<priority value="INFO" />
<appender-ref ref="REST_LOG" />
</category>
So I want INFO and above statements to only do to REST_LOG and WARN statements and above to go to REST_LOG and CONSOLE. What I am seeing is INFO statements from xyz.web.rest in the REST_LOG as expected but also seeing INFO statements from xyz.web.rest in CONSOLE which I wasn't expecting.
Can somebody explain what is going on?
You should set additivity to false on the xyz.web.rest logger.
Without the additivity you will see all the INFO message logged with logger xyz.web.rest twice in the console, because the logger inherit the appender of Root logger
and of the xyz.web logger.
See the Appenders and Layouts section of the Log4j documentation
Named Hierarchy
A logger is said to be an ancestor of another logger
if its name followed by a dot is a prefix of the descendant logger
name. A logger is said to be a parent of a child logger if there are
no ancestors between itself and the descendant logger.
For example, the logger named "com.foo" is a parent of the logger
named "com.foo.Bar". Similarly, "java" is a parent of "java.util" and
an ancestor of "java.util.Vector". This naming scheme should be
familiar to most developers.
The root logger resides at the top of the logger hierarchy. It is exceptional in two ways:
it always exists,
it cannot be retrieved by name.
Appenders and Layouts
Each enabled logging request for a given logger will be forwarded to
all the appenders in that logger as well as the appenders higher in
the hierarchy. For example, if a console appender is added to the
root logger, then all enabled logging requests will at least print on
the console.
If in addition a file appender is added to a logger, say C, then
enabled logging requests for C and C's children will print on a file
and on the console. It is possible to override this default behavior
so that appender accumulation is no longer additive by setting the
additivity flag to false.
The thresholds are hierarchical. DEBUG includes all INFO, WARN, ERROR. So it is natural that if you define console with a threshold of DEBUG that it would receive INFO level messages.
If you only want console to receive WARN and ERROR, set its threshold to WARN.
According to the Log4Net documentation, the RollingFileAppender will only roll the log file when a message is logged. I need to log to this file, but import it every day into another database. I cannot use a database appender because I need the files and I have to translate the data from the log file to the database (it isn't a direct copy). The problem is if there is no log activity after midnight, the log doesn't roll. The importer looks for the previous days file (and I can't change this code), so if there is no activity and the log hasn't rolled, the importer doesn't find the file. Is there anyway to force the log to roll at midnight without having another thread that wakes up and forces it to roll? Could a custom appender do this? I would like to avoid this if possible.
Write a Windows Service that fires an event just after midnight that writes a dummy log entry using the same configuration.
You have to think about this from the point of the question "what code paths lead to the rollover routine?". Once you know how that routine is reached you can decide how to trigger it.
Could a custom appender do it? Sure, but no code in the appender will run until you log via it so you're back to square one.
As for the question "Is there anyway to force the log to roll at midnight without having another thread that wakes up and forces it to roll?", I would say that that question is equivalent to "Is it possible to force the log to roll at midnight without any code being run?". I'm not trying to be funny about it, or to insult you, I'm just trying to restate the question in a way which will hopefully answer it for you. :-)
The easiest way to solve this is to have something wake up and log to force the file to rotate.
According to the RollingFileAppender documentation you can set it to roll on a daily basis, see this configuration:
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="logfile" />
<appendToFile value="true" />
<rollingStyle value="Date" />
<datePattern value="yyyyMMdd" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
</layout>
</appender>