The old log files didn't get removed when using logback RollingFileAppender - log4j

My logback.xml file as following, and I set MaxHistory=1 in order to delete old log files and only keep the log for one day. But I found the old files didn't get removed. I still could see them as:app.log.2019-02-11 app.log.2019-02-12
<configuration>
<property name="APP_NAME" value="logbacktest-logs" />
<property name="LOG_HOME" value="/tmp/${APP_NAME}" />
<property name="ENCODER_PATTERN" value="%d %C.%method:%L _ %msg%n"/>
<contextName>${APP_NAME}</contextName>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${ENCODER_PATTERN}</pattern>
</encoder>
</appender>
<appender name="APP_APPEND" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/app.log.%d{yyyy-MM-dd}</fileNamePattern>
<MaxHistory>1</MaxHistory>
</rollingPolicy>
<encoder>
<pattern>${ENCODER_PATTERN}</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
<appender-ref ref="APP_APPEND" />
</root>
</configuration>

This is a part of my log
12:11:32,358 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
which means that say your application is not up during that time, then it doesn't get a chance to delete it
I had faced a same issue when I had set the maxHistory to 5, it deleted 5 log files sometimes but when my application died it could not delete the logs because it didn't get a chance to delete.
Refering the logback docs, it says you can use this:
<cleanHistoryOnStart> true </cleanHistoryOnStart>
If set to true, archive removal will be executed on appender start up. By default this property is set to false.
Archive removal is normally performed during roll over. However, some applications may not live long enough for roll over to be triggered. It follows that for such short-lived applications archive removal may never get a chance to execute. By setting cleanHistoryOnStart to true, archive removal is performed at appender start up.
Modify your piece of logback-spring.xml to:
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/app.log.%d{yyyy-MM-dd}</fileNamePattern>
<maxHistory>1</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>

Related

Log4net file rollingFile not deleting old files

I have log4Net set up with the following config settings in a web service web.config file:
<log4net>
<!-- RollingFile is set to be a File Appender -->
<appender name="RollingFile" type="log4net.Appender.RollingFileAppender">
<file value="c:\temp\Sync.log" />
<appendToFile value="true" />
<maximumFileSize value="50MB" />
<maxSizeRollBackups value="10" />
<datePattern value="yyyyMMdd" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date %-5level %-50logger - %message%newline%newline" />
</layout>
</appender>
<!-- Set root logger level to DEBUG to have all log -->
<root>
<level value="INFO" />
<appender-ref ref="RollingFile" />
</root>
The issue I am having is when I look in temp I see 17 files
Sync.Log
Sync.log20180424.log
(other sync files with dates)
...
the last one being
Sync.log20180405
I am confused as to why it is not removing the ones beyond 10 days as stated by the maxSizeRollBackups.
I know that by default the is rollingStyle = composite.
With my current config, how would it behave if I let it go?
And as a second - what am I missing in my current settings.
Thanks. I know there are similar questions out there but in reading them I feel like I am still missing something.
It turns out that what I was trying to do is not possible.
As stated in other places:
A maximum number of backup files when rolling on date/time boundaries is not supported.
Because of this, I took the date portion out so that the backups roll by size only. After doing that I did not have the same issues. I don't like the file name as much but it is still easy enough to track them based on their file properties having the date last edited.

Control SQL logging level in Catalina.out

Our app runs on Tomcat 8 and Linux. We have log4j.xml shipped with the war that controls the level of logging for our applications logging. The log4j also defines logging levels for SQL.
In hibernate.cfg.xml, the "hibernate.show_sql" is set to true.
<property name="hibernate.show_sql">true</property>
In $CATALINA_BASE/conf/logging.properties
All references to ConsoleAppender:
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
Are removed, so nothing is sent to std out except for the SQL statements.
The: "hibernate.show_sql">true, sends all SQL statements being processed to catalina.out – please note that catalina.out is a file created only on Linux/Unix machines.
I read that "hibernate.show_sql">true, logs the SQL statements at DEBUG level. The problem is that I want to log those statements at ERROR level but don’t know how to control the level?
I read that that level for SQL logging can be controlled by log4j.xml, but changing it in log4j.xml has no effect on how the catalina.out is being logged. Does anyone know how can I control the SQL logging in catalina.out to be only at ERROR level?
Below is log4j.xml.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
<appender name="mylog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="${catalina.base}/logs/mylog.log" />
<param name="Threshold" value="TRACE" />
<param name="Append" value="true" />
<param name="MaxFileSize" value="300MB" />
<param name="MaxBackupIndex" value="10" />
<layout class="it.openutils.log4j.FilteredPatternLayout">
<param name="ConversionPattern"
value="%d{ISO8601} %5p [%t] Executor:%X{Executor} Type:%X{Type} A:%X{Account} C:%X{Campaign} %c{1}:%L - %m%n" />
</layout>
</appender>
<logger name="org.hibernate.SQL" additivity="false">
<level value="ERROR" />
<appender-ref ref="mylog" />
</logger>
<logger name="org.hibernate">
<level value="ERROR"/>
</logger>
<logger name="org.hibernate.jdbc.JDBCContext">
<level value="ERROR"/>
</logger>
<root>
<level value="info" />
<appender-ref ref="mylog" />
</root>
Since I was not able to find anyway to set the level to ERROR for Catalina.out, I decided to turn off the SQL logging all together. So I changed all levels in log4j.xml back to INFO and set this property to false in hibernate.cfg.xml.
<property name="hibernate.show_sql">false</property>
After a while logging, I noticed that there is nothing being logged in Catalina.out unless an error happens, so in this case, the errors and exceptions are being logged into Catalina.out and not the regular SQL statements, which is (almost) something I was trying to achieve at the first place. I do not know why that’s the case thou and where/how those logging levels are defined. But hope that helps somebody.

Create multiple logfiles during application run using log4net

I'm using log4net in a c# windows service, so the main application runs every time. I configured log4net to log into a date specific directory. Inside this directory log4net creates a logfile with a timestamp in the filename. This works fine. The probem is, that it only creates a new logfile, when the windows service restarts. This causes the logfile to get very big, expecially if the service runs over several days.
The question is:
Is there a possibility to force log4net to start with a new logfile throughout my code.
This is my log4net config:
<log4net>
<appender name="FileAppender" type="log4net.Appender.FileAppender">
<file type="log4net.Util.PatternString" value="${ProgramData}\Sirona\Log\AcqSrv\%date{yyMMdd}\AcqSrvAll-%date{HHmmss}.log" />
<appendToFile value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%2thread] [%-5level] [%logger] - %message%newline" />
</layout>
</appender>
<root>
<level value="ALL" />
<appender-ref ref="FileAppender" />
</root> <!-- Do not want NHibernate log -->
</log4net>
Best regards
Emu
I think your problem is solved by the RollingFileAppender: http://logging.apache.org/log4net/release/config-examples.html
http://logging.apache.org/log4net/release/sdk/log4net.Appender.RollingFileAppender.html
It works exactly as a FileAppender but once it reach a maximum size (configurable) it creates another log file.
You can also instruct it to keep only the latest X log files.
Hope it helps.

log4net not logging from Autocad Plugin on deployment

I am developing an AutoCAD 2012 plugin using .net API"s of autocad. The Autocad plugin's are dll which are loaded into the Autocad runtime when the Autocad is started. I have been able to successfully log from log4net using Visual Studio developement environment. However, when I run the plugin outside of Visual Studio, that is when I deploy my plugin, the log4net does not log any messages or even create a log file. Here is the logging related code and the configuration file.
log4net_autocad.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" requirePermission="false"/>
</configSections>
<log4net>
<appender name="AutocadRollingFileAppender" type="log4net.Appender.RollingFileAppender">
<param name="File" value="${NIRVANA_SOFTWARE_INSTALL_PATH}Relay_Manager_Autocad.log"/>
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="0" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p [%C.%M] %m%n"/>
</layout>
</appender>
<logger name="AutocadFile">
<level value="All"/>
<appender-ref ref="AutocadRollingFileAppender"/>
</logger>
</log4net>
</configuration>
myPlugin.cs: Code to congifure logging
public class MyPlugin : IExtensionApplication {
private static log4net.ILog log;
void IExtensionApplication.Initialize() {
...
log4net.Config.XmlConfigurator.Configure(new System.IO.FileInfo("log4net_autocad.config"));
log = log4net.LogManager.GetLogger("AutocadFile");
}
void IExtensionApplication.Initialize() will execute when the Autocad is started. Couple of things that I have checked:
Checked that the Autocad Plugin is properly loaded in Autocad, which
means that log4net initialization code has run.
Checked that log4net config file (log4net_autocad.config) is copied
in the bin directory of the deployed application
The strange thing is that when I load the plugin in developement environment(VS 2010), the plugin logs properly. So why is the log4net not logging (or even creating a log file) in deployment.
EDIT:
I checked some additional stuff
In the plugin just before I log, I checked following through alert dialogs.
The log.Logger.Name properly returns the name of the logger that I am using.
log.Logger.IsEnabledFor(log4net.Core.Level.All) returns false, even though I have configured Level Value=All in the config file.
EDIT-2
I tried to define config file as explained in the referring url of the answer. I am quoting the approach as outlined in the document.
2.] Application Configuration File: [AppName].exe.config, [AppName].dll.config
The application-level configuration, [AppName].exe.config or [AppName].dll.config, is where
most of this document will spend its time. This configuration file will contain all applicationlevel settings and can even be used to define the use of and default values for settings
associated with the Roaming-User and Local-User configuration files. These files are
typically stored in the same directory as the application executable, but can be placed
elsewhere if necessary. There will be much more discussion on this to come.
The above approach suits my application, but I still can't get the logger to log anything at all.I defined a log4net config file with name RelayAnalysis_Autocad.dll.config and supplied it to the log4net for configuration. I think there is something else to it that I am not able to understand. I have started doubting whether we can log using log4net from autocad plugin?
I also removed the environment path and hardcoded it so as to eliminate any issues regarding the reading of environment variable. Still no luck.
On deploy, you should store the plugin configuration in one of the default config files:
User.config
acad.exe.config
machine.config
Probably, the User.config is the best file to carefully change during setup. Moreover you should design and code the changes so that they can be reverted on uninstall.
Doublecheck that you have properly replaced the ${NIRVANA_SOFTWARE_INSTALL_PATH} during setup and that the folder has proper permissions.
Furthermore, I would try to either add to the AssemblyInfo.cs the following line
[assembly: log4net.Config.XmlConfigurator(ConfigFile="AbsolutePathToTheConfig")]
(in which case you shouldn't call XmlConfigurator.Configure) or use an absolute path in the call to
log4net.Config.XmlConfigurator.Configure(new FileInfo("AbsolutePathToTheConfig"));
Finally, note that the log4net documentation about XmlConfigurator.Configure states that the configuration file used can be a XML not matching the .NET config rules.
The configuration file must be valid XML. It must contain at least one element called log4net that holds the log4net configuration data.
Thus, in the log4net_autocad.config file you can skip the <configuration> element:
<log4net>
<appender name="AutocadRollingFileAppender" type="log4net.Appender.RollingFileAppender">
<param name="File" value="Absolute\Path\To\Relay_Manager_Autocad.log"/>
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="0" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p [%C.%M] %m%n"/>
</layout>
</appender>
<logger name="AutocadFile">
<level value="All"/>
<appender-ref ref="AutocadRollingFileAppender"/>
</logger>
</log4net>

Need to modify this log4net to write to a file, with ability to write to many files

So I already have nhibernate working which uses log4net, and my web.config currently looks like:
<log4net debug="false">
<appender name="console" type="log4net.Appender.ConsoleAppender, log4net">
<layout type="log4net.Layout.PatternLayout,log4net">
<param name="ConversionPattern" value="%d{ABSOLUTE} %-5p %c{1}:%L - %m%n"/>
</layout>
</appender>
<!-- Setup the root category, add the appenders and set the default priority -->
<root>
<priority value="INFO"/>
<appender-ref ref="console"/>
</root>
</log4net>
Now my current requirement is the ability to create a seperate log file for each major module that I have in my application.
Right now it seems to be just writing to console, but I need to add the ability to write to files.
What do I have to modify in my configuration to be able to do this?
I want to be able to, in my code, say I want to write to logfile "logfile_1", and if it doesn't exist, create it, otherwise just append to it.
Really simple, rinse and repeat as required for number of files you want, you can also associate namespaces with different files if you want them separated out that way but I find it easier just to use Log2Console to view the file all together.
<appender name="FileAppender" type="log4net.Appender.FileAppender">
<file value="k:\temp\log2console.log" />
<appendToFile value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.XmlLayoutSchemaLog4j" />
</appender>
Don't forget to add the appender-ref to your root section.

Resources