My log4j properties as follows,
#Application Logger
log4j.rootLogger=DEBUG,file
#log4j.rootLogger=DEBUG,file,console
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=../common/logs/api/log4j-app.log
log4j.appender.file.maxFileSize=6MB
log4j.appender.file.maxBackupIndex=5
log4j.appender.file.layout=org.apache.log4j.PatternLayout
#log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c<strong><strong>{1}</strong></strong>:%L - %m%n
log4j.appender.file.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.file.Append=false
#Manual WebLogs
log4j.logger.webLogger=INFO, web
log4j.appender.web=org.apache.log4j.RollingFileAppender
log4j.appender.web.maxFileSize=900KB
log4j.appender.web.maxBackupIndex=6
log4j.appender.web.layout=org.apache.log4j.PatternLayout
#log4j.appender.web.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.web.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.web.File=../common/logs/web/manual.log
log4j.appender.web.Append=false
#Manual ApiLogs
log4j.logger.apiLogger=INFO, api ,console
log4j.appender.api=org.apache.log4j.RollingFileAppender
log4j.appender.api.maxFileSize=900KB
log4j.appender.api.maxBackupIndex=6
log4j.appender.api.layout=org.apache.log4j.PatternLayout
#log4j.appender.api.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.api.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.api.File=../common/logs/api/manual.log
log4j.appender.api.Append=false
#Manual ApiSecurityLogs
log4j.logger.apiSecurityLogger=INFO, apiSecurity
log4j.appender.apiSecurity=org.apache.log4j.RollingFileAppender
log4j.appender.apiSecurity.maxFileSize=900KB
log4j.appender.apiSecurity.maxBackupIndex=6
log4j.appender.apiSecurity.layout=org.apache.log4j.PatternLayout
#log4j.appender.apiSecurity.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.apiSecurity.layout.ConversionPattern=%d{ISO8601} : %L -- %m%n
log4j.appender.apiSecurity.File=../common/logs/apiSecurity/manualSecurity.log
log4j.appender.apiSecurity.Append=false
# Appender which writes to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} %c %m%n
We are using the above log4j.properties to create web logs and api logs.
web related logs goes to web/manual.log.
api related logs goes to api/manual.log.
api security related logs goes to apiSecurity/manualSecurity.log
for api manual.log and log4j-app.log files are creating as
manual.log, manual.log.1, manual.log.2 ...etc.,
log4j-app.log, log4j-app.log.1, log4j-app.log.2 ... so on.
I want all the log details in a single file as manual.log and log4j-app.log, please let me know how to get it?
Appenders type needs to be changed from RollingFileAppender to FileAppender.
So after changing it, your log4j.properties would look like:
#Application Logger
log4j.rootLogger=DEBUG,file
#log4j.rootLogger=DEBUG,file,console
log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File=log4j-app.log
#log4j.appender.file.maxFileSize=6MB
#log4j.appender.file.maxBackupIndex=5
log4j.appender.file.layout=org.apache.log4j.PatternLayout
#log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c<strong><strong>{1}</strong></strong>:%L - %m%n
log4j.appender.file.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.file.Append=false
#Manual WebLogs
log4j.logger.webLogger=INFO, web
log4j.appender.web=org.apache.log4j.FileAppender
#log4j.appender.web.maxFileSize=900KB
#log4j.appender.web.maxBackupIndex=6
log4j.appender.web.layout=org.apache.log4j.PatternLayout
#log4j.appender.web.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.web.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.web.File=../common/logs/web/manual.log
log4j.appender.web.Append=false
#Manual ApiLogs
log4j.logger.apiLogger=INFO, api ,console
log4j.appender.api=org.apache.log4j.FileAppender
#log4j.appender.api.maxFileSize=900KB
#log4j.appender.api.maxBackupIndex=6
log4j.appender.api.layout=org.apache.log4j.PatternLayout
#log4j.appender.api.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.api.layout.ConversionPattern=%d{ISO8601} %5p : %L -- %m%n
log4j.appender.api.File=manual.log
log4j.appender.api.Append=false
#Manual ApiSecurityLogs
log4j.logger.apiSecurityLogger=INFO, apiSecurity
log4j.appender.apiSecurity=org.apache.log4j.RollingFileAppender
log4j.appender.apiSecurity.maxFileSize=900KB
log4j.appender.apiSecurity.maxBackupIndex=6
log4j.appender.apiSecurity.layout=org.apache.log4j.PatternLayout
#log4j.appender.apiSecurity.layout.ConversionPattern=%d{ISO8601} %5p %F <Method>%M : %L -- %m%n
log4j.appender.apiSecurity.layout.ConversionPattern=%d{ISO8601} : %L -- %m%n
log4j.appender.apiSecurity.File=../common/logs/apiSecurity/manualSecurity.log
log4j.appender.apiSecurity.Append=false
# Appender which writes to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{dd/MM/yyyy HH:mm:ss} %c %m%n
Related
I am using rsyslog server running on localhost(centos) and remote machine(ubuntu).I am able to send the logs from localhost to remote server using TCP connection and UDP connection able to see the logs in remote server.
My localhost config :
# rsyslog configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on
# File to store the position in the journal
$IMJournalStateFile imjournal.state
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
*.* ##192.168.122.50:514
My remote server config:
/etc/rsyslog.conf Configuration file for rsyslog.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
# Default logging rules can be found in /etc/rsyslog.d/50-default.conf
#################
#### MODULES ####
#################
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support
$ModLoad imtcp
$InputTCPServerRun 514
$AllowedSender TCP, 192.168.0.0/8
#$ModLoad immark # provides --MARK-- message capability
$template TmplAuth, "/var/log/client_logs/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplMsg, "/var/log/client_logs/%HOSTNAME%/%PROGRAMNAME%.log"
authpriv.* ?TmplAuth
*.info;mail.none;authpriv.none;cron.none ?TmplMsg
# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
# Enable non-kernel facility klog messages
$KLogPermitNonKernelFacility on
###########################
#### GLOBAL DIRECTIVES ####
###########################
#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# Filter duplicated messages
$RepeatedMsgReduction on
#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog
#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
authpriv.* /var/log/secgw-siglogs;RSYSLOG_FileFormat
$template RemoteStore, "/var/log/remote/%HOSTNAME%/%timegenerated:1:10:date-rfc3339%"
:source, !isequal, "localhost" -?RemoteStore
:source, isequal, "last" ~$template RemoteStore, "/var/log/remote/%HOSTNAME%/%timegenerated:1:10:date-rfc3339%"
:source, !isequal, "localhost" -?RemoteStore
:source, isequal, "last" ~
Now i have to send some user specific logs not all kernel logs,auth logs which are present in /var/log location,Is there any configuration need to be modified ?
I try to modify the error log to show runtime duration.
I added %T/%D at the end of the LogFormat at httpd.conf
like this:
<IfModule log_config_module>
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
# NOTE: "combined" and "common" are required by WHM
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%T%D %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T/%D" combined
# access_log format can be set in WHM under 'Basic cPanel & WHM Setup'
CustomLog logs/access_log combined
</IfModule>
I restarted the httpd service but the change didn't take place.
At Cpanel it is specified to use the combined format.
Can someone help me please?
I have a apache server running on CentOS 6.5 server. In log files 25% of records are like this
000.000.000.000 - - [24/Jul/2014:04:15:10 +0000] "GET /address/of/the/page.html HTTP/1.1" 200 20341 "-" "Mozilla/4.0
But rest of 75% of logs are like this, without IP
- - [24/Jul/2014:04:15:10 +0000] "GET /address/of/the/page.html HTTP/1.1" 200 20341 "-" "Mozilla/4.0
Can somebody tell me what is the problem?
Thanks!
By default Apache prints the REMOTE_ADDR header value in the access log, it may be that in your environment that value is blank in some cases. You can try printing the X-Forwarded-For header in the access log and see if that has the values that you need.
Typically if the requests are passing through proxy servers/load balancers its a good idea to check the value of the XFF header for the correct client IP.
Hope this help.
-Avijit
Complementing Avi's response, you will need to open the file /etc/apache2/apache2.conf and change the following lines:
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
to:
LogFormat "%{X-Forwarded-For}i %v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %O" common
LogFormat "%{X-Forwarded-For}i %{Referer}i -> %U" referer
LogFormat "%{X-Forwarded-For}i %{User-agent}i" agent
Then, restart apache service with the command: sudo service apache2 restart
and the client ip will appear on apache log.
Check the result with:
cat /var/log/apache2/access.log
When I don't specify a logfile in the virtual host sections of my conf-file the logs are written in the file specified in httpd.conf (=access_log).
A log-entry would look like this:
SOMEIP - - [22/Jan/2013:18:34:08 +0100] "GET / HTTP/1.1" 200 1752 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/SOMEIP Safari/537.17"
SOMEIP - - [22/Jan/2013:18:34:08 +0100] "GET /img/homepage_bg.png HTTP/1.1" 304 - "http://DOMAIN/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) C$
But when I define a log file in the virtual host section the new log file contains different information:
SOMEIP - - [22/Jan/2013:18:33:34 +0100] "GET / HTTP/1.1" 200 1752
SOMEIP - - [22/Jan/2013:18:33:34 +0100] "GET /img/homepage_bg.png HTTP/1.1" 304 -
i define the log file like this:
CustomLog logs/DOMAIN-access_log common
Why does a custom log contain less information than the general log where all virtual hosts log in by default?
You need to define the alias "common" with a log format that includes the user-agent.
LogFormat "%h %l %u %t \"%r\" %>s %b "%{User-agent}i" common
You didn't say what flavour of Linux you're using. Any decently configured Apache (for example the Debian-based ones like Ubuntu, Mint, etc.) will already have a fitting LogFormat containing the user-agent in their configuration. Look for all the lines matching LogFormat. You should find something like this:
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
Just use the combined or even the vhost_combined parameter for your logfile:
CustomLog logs/DOMAIN-access_log combined
You should also look at the documentation for the Custom Log Formats.
I have a system that uses both Apache Camel and Karaf.
I need some specific processor to log into a new log instead of the default one ,karaf.log.
I have reached that there is a file called "org.ops4j.pax.logging.cfg" that is responsible for logging configuration .
Here it's the file before I mess with it :
################################################################################
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
################################################################################
# Root logger
log4j.rootLogger=INFO, out, osgi:VmLogAppender
log4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer
# CONSOLE appender not used by default
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
# File appender
log4j.appender.out=org.apache.log4j.RollingFileAppender
log4j.appender.out.layout=org.apache.log4j.PatternLayout
log4j.appender.out.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.out.file=${karaf.data}/log/karaf.log
log4j.appender.out.append=true
log4j.appender.out.maxFileSize=1MB
log4j.appender.out.maxBackupIndex=10
# Sift appender
log4j.appender.sift=org.apache.log4j.sift.MDCSiftingAppender
log4j.appender.sift.key=bundle.name
log4j.appender.sift.default=karaf
log4j.appender.sift.appender=org.apache.log4j.FileAppender
log4j.appender.sift.appender.layout=org.apache.log4j.PatternLayout
log4j.appender.sift.appender.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %m%n
log4j.appender.sift.appender.file=${karaf.data}/log/$\\{bundle.name\\}.log
log4j.appender.sift.appender.append=true
Now I have added an extra File appender :
# Additional File appender
log4j.rootLogger=INFO, out, osgi:VmLogAppender # I removed the first log4j.rootLogger
log4j.appender.new=org.apache.log4j.RollingFileAppender
log4j.appender.new.layout=org.apache.log4j.PatternLayout
log4j.appender.new.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.new.file=${karaf.data}/log/new.log
log4j.appender.new.append=true
log4j.appender.new.maxFileSize=1MB
log4j.appender.new.maxBackupIndex=10
And from Camel , I logged from DSL :
process(exceptionProcessor).
process(doSmth).log(LoggingLevel.INFO,"new","That is a new file to log into")
The result now that it logs erverything on BOTH "new.log" & "karaf.log" file .
The question is : how could I log into"new.log" just in case a specific processor .
Others will be logged into the karaf.log !!
The solution for my problem is :
1- u should update "org.ops4j.pax.logging.cfg" and add a new file appender .
In my case my new file appender is "new"
################################################################################
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
################################################################################
# Root logger
log4j.rootLogger=INFO, out,osgi:VmLogAppender
log4j.logger.new=INFO,new
log4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer
# CONSOLE appender not used by default
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
# new File appender
log4j.appender.new=org.apache.log4j.RollingFileAppender
log4j.appender.new.layout=org.apache.log4j.PatternLayout
log4j.appender.new.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.new.file=${karaf.data}/log/new.log
log4j.appender.new.append=true
log4j.appender.new.maxFileSize=1MB
log4j.appender.new.maxBackupIndex=10
# File appender
log4j.appender.out=org.apache.log4j.RollingFileAppender
log4j.appender.out.layout=org.apache.log4j.PatternLayout
log4j.appender.out.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.out.file=${karaf.data}/log/karaf.log
log4j.appender.out.append=true
log4j.appender.out.maxFileSize=1MB
log4j.appender.out.maxBackupIndex=10
# Sift appender
log4j.appender.sift=org.apache.log4j.sift.MDCSiftingAppender
log4j.appender.sift.key=bundle.name
log4j.appender.sift.default=karaf
log4j.appender.sift.appender=org.apache.log4j.FileAppender
log4j.appender.sift.appender.layout=org.apache.log4j.PatternLayout
log4j.appender.sift.appender.layout.ConversionPattern=%d{ABSOLUTE} | %-5.5p | %-16.16t | %-32.32c{1} | %-32.32C %4L | %m%n
log4j.appender.sift.appender.file=${karaf.data}/log/$\\{bundle.name\\}.log
log4j.appender.sift.appender.append=true
2- I use DSL to log :
process(doSmth).log(LoggingLevel.INFO,"new","HelloZ my new Logger ${exception.stacktrace}")
Enjoy :)