Is it possible to configure a Camel route to send a message to a specific log4j logger? For example, I have the following logger:
<logger name="com.me.log.mylogger" additivity="false">
<level value="debug" />
<appender-ref ref="file_appender_messages" />
</logger>
file_appender_messages is just a RollingFileAppender.
I then try to log it using the following in my Camel context:
<to uri="log:com.me.log.mylogger?level=INFO" />
But it outputs on the command line instead of the log file specified in file_appender_messages:
25-Oct-2012 11:46:44 org.apache.camel.processor.CamelLogger log
INFO: [MESSAGE BODY]
I would like to be able to use dffferent loggers for messages from different sources. I could do it in my message processors but ideally it could be configured in the route xml. Can it be done?
Camel uses slf4j since some time. So you have to first configure slf4j to use log4j as backend. In maven add the following dependencies:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
I fixed this by defining the Logger as a Bean in my application XML file
<bean id="myLogger" class="org.apache.log4j.Logger" factory-method="getLogger">
<constructor-arg value="com.me.log.mylogger"/>
</bean>
Then in my route, when I want to log the message I just direct it to the relevant method on the Bean
<to uri="bean:myLogger?method=info" />
Related
This is what I am trying to do
Objective: I have a jar file which I am installing on my cluster and then invoking as part of a job. I want to redirect logs from my application (jar) to Cloud Logging.
I have followed this logback guide.
Everything works fine when I run code from my local machine, I can see logs in Cloud Logging. However, as soon as I create the jar and run it on Databricks, logs are not being redirected to Cloud Logging, rather it's being captured by console only.
I think this is what's happening: As Apache Spark uses log4j and it gets initialized before my jar is being loaded. When my jar is being loaded by Spark, sl4j finds the log4j implementation as it finds that jar in classpath rather than taking the logback implementation.
Is there a way I can fix this issue?
Code:
import org.slf4j.{Logger, LoggerFactory}
object App {
def test_slf4j() = {
import org.slf4j.{Logger, LoggerFactory}
val logger: Logger =LoggerFactory.getLogger(getClass.getName)
logger.info("using slf4j ")
}
def main(args:Array[String]):Unit ={
test_slf4j()
}
}
loback.xml
<configuration>
<appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
<!-- Optional : filter logs at or above a level -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<log>application.log</log> <!-- Optional : default java.log -->
<resourceType>gae_app</resourceType> <!-- Optional : default: auto-detected, fallback: global -->
<enhancer>com.example.logging.logback.enhancers.ExampleEnhancer</enhancer> <!-- Optional -->
<flushLevel>INFO</flushLevel> <!-- Optional : default ERROR -->
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<Pattern>
%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n
</Pattern>
</encoder>
</appender>
<logger name="org.apache" level="INFO" additivity="false">
<appender-ref ref="CLOUD" />
<appender-ref ref="CONSOLE" />
</logger>
<root level="info">
<appender-ref ref="CLOUD" />
<appender-ref ref="CONSOLE" />
</root>
</configuration>
I'm using log4j 2.17.1.
Log4j is rolling files daily but will sometimes write to files it has already rolled. In some cases it is going back several days.
Example:
app.log.2022-01-03 has been overwritten with data from 2022-01-04.
app.log.2022-01-04 has been overwritten with data from 2022-01-10.
app.log.2022-01-11 has been overwritten with data from 2022-01-17.
Is there anything wrong with my configuration here? I just want it to roll everyday.
<Configuration>
<Appenders>
<RollingFile name="A1" append="true" fileName="/var/log/app/app.log">
<PatternLayout
pattern="%d{yyyy-MM-dd hh:mm:ss} [%t] %-5p %c %x %m%n" />
<FilePattern>/var/log/app/app.log.%d{yyyy-MM-dd}</FilePattern>
<Policies>
<TimeBasedTriggeringPolicy />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="A1" />
</Root>
</Loggers>
</Configuration>
I believe my issue was caused by multiple LoggerContexts attempting to write to the same RollingFileAppender.
I resolved this problem by following the steps in the Log4j Web Application usage:
https://logging.apache.org/log4j/2.x/manual/webapp.html
I had several web applications deployed to a single JBoss instance. Each app had their own copy of the log4j jars. I moved the log4j jars out of the wars and into a JBoss module to get them into the server classloader.
Then I followed the directions on the Logging Separation page:
https://logging.apache.org/log4j/2.x/manual/logsep.html
and I configured a single LoggerContext for JBoss:
Place the logging jars in the container's classpath and set the system property log4j2.contextSelector to org.apache.logging.log4j.core.selector.BasicContextSelector. This will create a single LoggerContext using a single configuration that will be shared across all applications.
I am using Log4j Http appender to send data to Splunk using mule cloudhub. During the build it thorws the error:
ERROR Appenders contains an invalid element or attribute Http
and I am not seeing the data in Splunk.
The error happens with Log4j Configuration:
<Http name="Splunktest" url="myurl" token="mytoken"
disableCertificateValidation="true"></Http>
During the maven build it is throwing the mentioned error. Mule runtime version 3.8.4
Did anyone else face the same error?
Entire Log4j for reference
<!--These are some of the loggers you can enable.
There are several more you can find in the documentation.
Besides this log4j configuration, you can also use Java VM environment variables
to enable other logs like network (-Djavax.net.debug=ssl or all) and
Garbage Collector (-XX:+PrintGC). These will be append to the console, so you will
see them in the mule_ee.log file. -->
<Appenders>
<RollingFile name="file" fileName="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}splunk.log"
filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}splunk-%i.log">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n" />
<SizeBasedTriggeringPolicy size="10 MB" />
<DefaultRolloverStrategy max="10"/>
</RollingFile>
<Http name="Splunktest" url="myurl" token="mytoken" disableCertificateValidation="true"></Http>
</Appenders>
<Loggers>
<!-- Http Logger shows wire traffic on DEBUG -->
<AsyncLogger name="org.mule.module.http.internal.HttpMessageLogger" level="WARN"/>
<!-- JDBC Logger shows queries and parameters values on DEBUG -->
<AsyncLogger name="com.mulesoft.mule.transport.jdbc" level="WARN"/>
<!-- CXF is used heavily by Mule for web services -->
<AsyncLogger name="org.apache.cxf" level="WARN"/>
<!-- Apache Commons tend to make a lot of noise which can clutter the log-->
<AsyncLogger name="org.apache" level="WARN"/>
<!-- Reduce startup noise -->
<AsyncLogger name="org.springframework.beans.factory" level="WARN"/>
<!-- Mule classes -->
<AsyncLogger name="org.mule" level="INFO"/>
<AsyncLogger name="com.mulesoft" level="INFO"/>
<!-- Reduce DM verbosity -->
<AsyncLogger name="org.jetel" level="WARN"/>
<AsyncLogger name="Tracking" level="WARN"/>
<AsyncRoot level="INFO">
<AppenderRef ref="file" />
</AsyncRoot>
<AsyncLogger name="splunk.logger" level="INFO" >
<AppenderRef ref="Splunktest" />
</AsyncLogger>
</Loggers>
The Http appender is not included in the log4j2 version used by the mule runtime 3.8.4.
As far as I know the latest version used in runtime 3.X.X is log4j2 2.8.2
and as you can see from the code here it doesn't define any Http appender.
The Http appender has been introduced in log4j2 2.10.0 ( code here) so you have 2 options:
bundle in you application the log4j2 version 2.10.0 and try to configure the classloader override as explained here
extract the Http appender class and it's dependencies from the version 2.10.0, package as a jar and import in your project, see picture below:
Hope this helps ...
Setting up a new standalone app and want to use log4j. Am using Maven on Netbeans (6.5), and was able to add the log4j jar/dependency, but it can't seem to find my log4j.xml file and I'm not sure how to tell Netbeans to find it.
Here's my error:
log4j:WARN No appenders could be found for logger (com.domain.project).
log4j:WARN Please initialize the log4j system properly.
log4j.xml path is src/main/resources/com/domain/product/gui/resource/log4j.xml
Note: the location of the log4j.xml was chosen to be consistent with legacy code (i.e. so future maintainers can find it), but this could be changed if necessary.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.domain.project</groupId>
<artifactId>project-log</artifactId>
<packaging>jar</packaging>
<version>0.1-SNAPSHOT</version>
<name>project-log</name>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.9</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
Here's my very simple log4j.xml:
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="ConsoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<!-- The default pattern: Date Priority [Category] Message\n -->
<param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>
</layout>
</appender>
<category name="com.domain.project">
<priority value="debug"/>
</category>
<root>
<priority value="info"/>
<appender-ref ref="ConsoleAppender"/>
</root>
</log4j:configuration>
And my call to set up logging:
static Logger log = Logger.getLogger("com.domain.project");
How do I tell Netbeans to set up a Maven classpath to this configuration file?
Ilane
I recently faced the same problem when I wanted to get the logging working when the application was launched from within the IDE.
My solution is to use the additionalArguments of the nbm-maven-plugin as illustrated below.
Notice the use of -J-Dlog4j.configuration=... the extra -J ensures that the option is passed to the application, otherwise it only affects the JVM of the maven launcher.
<build>
<pluginManagement
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>nbm-maven-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<brandingToken>${brandingToken}</brandingToken>
<cluster>${brandingToken}</cluster>
<additionalArguments>-J-Dlog4j.configuration=file:${basedir}/log4j.properties</additionalArguments>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
Location of log4j.xml is rather strange. I would place it in src/main/resources and the problem would disappear. If you need some legacy tools to find the same file where they need it, try to copy it there in generate-resources phase. For example, with maven-antrun-plugin
So log4j comes with two existing log rollers: RollingFileAppender, and DailyRollingFileAppender. Has anyone heard of an appender that does both of what the former do?
I need an appender that will roll log files based on filesize, but also append the current date to it.
I've been thinking about creating my own appender, but if there is already one that has been created, why not save the time and use that one?
Looks like you want a mix of the http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html and the http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html.
You'll have to code by yourself. The good news is: you'll just have "merge" those classes functionality, no "low level" new code required.
I know this post is a bit late but you could try out the TimeAndSizeRollingAppender. It's freely available under the Apache 2.0 license, download it from www.simonsite.org.uk.
The Log4J Extras from the "companions" project provide an array of policies for rolling, including what you're looking for.
APIDoc: http://logging.apache.org/log4j/extras/apidocs/index.html
Homepage: http://logging.apache.org/log4j/extras/
Using Log4j
As #JavaJigs said, Log4j's extras can be used.
First of all, if you're using Maven, add it as dependency keeping in mind to use the same log4j version, just to avoid any kind of conflict.
<!-- you should already have something like this -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- add this one, please note the version is the same than log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>apache-log4j-extras</artifactId>
<version>1.2.17</version>
</dependency>
Then, set up an appender to use both kinds of rollings. This is a quick and dirty example that rolls every minute and / or when the size of the log files exceeds 1000 bytes.
<appender name="rollout5" class="org.apache.log4j.rolling.RollingFileAppender">
<rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="ActiveFileName" value="log4j-current.log" />
<param name="FileNamePattern" value="log4j-%d{HH-mm}.%i.log.gz" />
</rollingPolicy>
<triggeringPolicy class="org.apache.log4j.rolling.SizeBasedTriggeringPolicy">
<param name="MaxFileSize" value="1000" />
</triggeringPolicy>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p - %m%n" />
</layout>
</appender>
Add the appender to the logger as you usually do.
If you run the application you should obtain something like that...
log4j-current.log
log4j-18-13.1480266729211.log.gz
log4j-18-12.1480266729095.log.gz
log4j-18-12.1480266729123.log.gz
Obviously the numbers of files and their names depend on how your application logs.
As you can see, the %i placeholder gets replaced with a sort of random yet ever increasing number. I was not able to find a way to have it replaced with number starting from 0 though. Nevertheless, such files listed in alphabetic order should match their historical order.
Other ideas
I know you mentioned explicitly log4j. But, if you can, why not to evaluate moving to log4j2? In this log4j2 piece of doc there are a couple of examples that seem to suit your needs.
Too late to reply. But hopefully this will help someone.
Add log4j dependency to your pom.xml file
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
Grab a copy of this FileAppender implementation. Thanks to the author of this file.
http://haobangshou.googlecode.com/svn/trunk/hbs/APPLICATION/server/common/src/com/hbs/common/appender/TimeSizeRollingFileAppender.java
Place this log4j.xml file in your resources folder
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration>
<appender name="consoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%p] %c{1} %m%n"/>
</layout>
</appender>
<appender name="debug" class="com.myproject.log.TimeSizeRollingFileAppender">
<param name="File" value="log/debug.log"/>
<param name="MaxBackupIndex" value="300"/>
<param name="Encoding" value="GB2312"/>
<!--CHANGE THIS TO A LARGER SIZE EG : 20MB. USE 1MB TO TEST IF THE SETTING WORKS.-->
<param name="MaxFileSize" value="1MB"/>
<param name="DatePattern" value="'.'yyyy-MM-dd"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %p [%c] - [%m]%n"/>
</layout>
</appender>
<!--CHANGE name TO ROOT PACKAGE NAME OF YOUR PROJECT THAT YOU WANT TO LOG.-->
<logger name="com.myproject" additivity="false">
<level value="debug"/>
<appender-ref ref="consoleAppender"/>
<appender-ref ref="debug"/>
</logger>
<logger name="org.springframework" additivity="false">
<level value="debug"/>
<appender-ref ref="consoleAppender"/>
<appender-ref ref="debug"/>
</logger>
<logger name="org.hibernate" additivity="false">
<level value="debug"/>
<appender-ref ref="consoleAppender"/>
<appender-ref ref="debug"/>
</logger>
<root>
<priority value="INFO"></priority>
<appender-ref ref="consoleAppender"/>
<appender-ref ref="debug"/>
</root>
To do a quick test:
import org.apache.log4j.Logger;
public class Main {
static public void main(String[] args) {
Logger log = Logger.getLogger(Main.class);
for(int i = 0; i < 10000; i ++)
log.info("Testing log");
}
}
Done!!
We use the class you see here. It works as you described and extends FileAppender.