I've CentOS 7.4 with logrotate 3.8.6 installed. I've a custom logrotate file under /etc/logrotate.d/ to rotate some logs on a Tomcat (e.g., catalina.out) which is installed in the same machine.
/opt/test/apache-tomcat-8.5.15-client/logs/catalina.out {
copytruncate
daily
rotate 30
olddir /opt/test/apache-tomcat-8.5.15-client/logs/backup
compress
missingok
maxsize 50M
dateext
dateformat .%Y-%m-%d
}
I want the log to be rotated daily or if the size reaches 50MB. When this happens log files are compressed and copied into a backup folder and are kept for 30 days before being deleted.
I already ran logrotate manually in debug mode with the following command and no errors were displayed (and the expected zipped log files were created):
/usr/sbin/logrotate -d /etc/logrotate.d/openncp-tomcat-backoffice 2> /tmp/logrotate.debug
In /var/lib/logrotate/logrotate.status there are no issues, the files are shown as rotated but they're not in fact:
"/var/log/yum.log" 2017-11-27-19:0:0
"/opt/test/apache-tomcat-8.5.15-server/logs/catalina.out" 2017-12-15-3:41:1
"/var/log/boot.log" 2017-12-15-3:41:1
"/var/log/up2date" 2017-11-27-19:0:0
I've the default /etc/logrotate.conf:
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
dateext
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0600 root utmp
rotate 1
}
# system-specific logs may be also be configured here.
I also have the default /etc/cron.daily/logrotate:
#!/bin/sh
/usr/sbin/logrotate -s /var/lib/logrotate/logrotate.status /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0
I ask for your guidance on configuring this appropriately.
The problem was related to the SELinux file type of the log files, which were located in a directory different from /var/log, meaning that the logrotate process didn't have access to perform its tasks. I found this other SO thread as well as this Redhat page that helped to solve the issue. I found the Redhat documentation very helpful, so I provide here 2 links:
5.9.3. CHECKING THE DEFAULT SELINUX CONTEXT
5.6.2. PERSISTENT CHANGES: SEMANAGE FCONTEXT
To answer your question (as in the title) about having daily and maxsize, note that by default logrotate runs once a day anyway so that means maxsize is hardly useful in that situation. The file will be rotated anyway (assuming you don't have SELinux in the way, of course).
maxsize is useful with weekly and monthly, of course, because logrotate still checks the files daily.
Note that the fact that logrotate runs daily is just because on many systems it is installed that way by default.
$ ls -l /etc/cron.daily
...
-rwxr-xr-x 1 root root 372 Aug 21 2017 logrotate
...
Moving that file to /etc/cron.hourly is safe and it now will be useful to have the daily option turned on:
$ sudo mv -i /etc/cron.daily/logrotate /etc/cron.hourly/logrotate
WARNING: a system like Ubuntu is likely to re-install the daily file on the next update of the logrotate package, which is rather infrequent, but can happen. I do not know of a clean way to avoid that issue. The ugly way is to create an empty file of the same name which will prevent the packager from adding the file from the package.
$ sudo touch /etc/cron.daily/logrotate
Or edit and put a comment such as:
# placeholder to prevent installer from changing this file
Related
Here's the logrotate config file under /etc/logrotate.d/mylog
/var/www/www.mysite.com/logs/* {
daily
rotate 365
compress
delaycompress
missingok
notifempty
create 0664 apache apache
}
I set this yesterday at 4 pm, and today at 10am i didn't see any compressed file.
I tried logrotate with -d and -f option
With -d option, i got this output
reading config file mysite
reading config info for /var/www/www.mysite.com/logs/*
Handling 1 logs
rotating pattern: /var/www/www.mysite.com/logs/* after 1 days (365 rotations)
empty log files are not rotated, old logs are removed
considering log/var/www/www.mysite.com/logs/access-2018-08-08.log
log does not need rotating
With -f option, nothing appeared.
But i noticed it added the .1 suffix to each of my log.
Anyone please explain what is this strange behavior please?
Logrotate does not have a built-in scheduling system.
If you want to clean up logs manually once, sure you can run it manually:
logrotate /etc/logrotate.d/mylog
And if you check logrotate man page for -f behaviour:
-f, --force
Tells logrotate to force the rotation, even if it doesn't think this is necessary.
Sometimes this is useful after adding new entries to a logrotate config file,
or if old log files have been removed by hand, as the new files will be created,
and logging will continue correctly.
-d, --debug
Turns on debug mode and implies -v. In debug mode, no changes will be made to
the logs or to the logrotate state file.
So with -f option you forcefully rotate all logs thus the .1 suffix.
And with -d option it is a dry run so you only preview the changes.
Now if you want to run it daily, you have to make use of cron.
By default after installation logrotate should have a default configuration file /etc/logrotate.conf and a default cron job /etc/cron.daily/logrotate
You can either make use of them or create your own configuration file and cron job.
Say if you place your custom configuration at /etc/logrotate.d/mylog then you can place the following cron job at /etc/cron.d/logrotate-cron to run it daily.
0 0 * * * root /usr/sbin/logrotate /etc/logrotate.d/mylog --state /etc/logrotate.status
You can check the file /etc/logrotate.status to make sure it ran.
The configuration of my rsyslog file in logrotate :
/opt/mapvariable/log/myapp
{
rotate 24
hourly
maxsize 10k
compress
ifempty
postrotate
reload rsyslog >/dev/null 2>&1 || true
endscript
}
I have copied logrotate from cron.daily to cron.hourly.
Then I executed following commands :
sudo logrotate -f /etc/logrotate.conf
sudo logrotate -f /etc/logrotate.conf
Still, it's not working. Any guidance will be much helpful.
Thank You.
Define logs in first line like:
/opt/mapvariable/log/mapapp/*.log
{
...
}
It will run this on all files end with .log or give the log file name instead of .log. Comment the post-rotate section to troubleshoot. Rotation of logs required for empty files too with ifempty? Check the size of logs file too.
I enabled the error_log = /var/log/php5-fpm.log under /etc/php5/fpm/php-fpm.conf on my ubuntu 12.04 running nginx and php5-fpm.
But I noticed that php5-fpm.log does not logrotate. I tried to understand some of the configuration I found from the internet but I'm reluctant to test it on my production server.
Here are some of the config that I found:
/var/log/php5-fpm.log {
rotate 12
weekly
missingok
notifempty
compress
delaycompress
postrotate
invoke-rc.d php5-fpm reopen-logs > /dev/null
endscript
}
This is the link of the config. As I understand, all I need is to create a file called php5-fpm under /etc/logrotate.d/, so it will look like /etc/logrotate.d/php5-fpm and with the above code.
I also found another sample from this link with the following code:
/var/log/php5-fpm.log {
daily
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
[ ! -f /var/run/php5-fpm.pid ] || kill -USR1 `cat /var/run/php5-fpm.pid`
endscript
}
Since I'm new to logrotate configuration, I want to make sure that what I will do is correct.
So, which of the two configuration is correct? The first one or the second one? And is it correct that I will create a file only at /etc/logrotate.d/php5-fpm and put the code in there?
Sorry if this is a newbie question, I just can't find the complete explanation on how to do this.
Just to clarify for others coming via Google:
1)
invoke-rc.d php5-fpm reopen-logs > /dev/null
This is something that must be supported by your distribution. The option "reopen-logs" does not come with the default init script provided by the PHP source package. So you likely might not be able to use this.
2)
[ ! -f /var/run/php5-fpm.pid ] || kill -USR1 `cat /var/run/php5-fpm.pid`
This is the correct option and also officially being supported by PHP-FPM, see:
https://github.com/php/php-src/blob/b7a7b1a624c97945c0aaa49d46ae996fc0bdb6bc/sapi/fpm/fpm/fpm_events.c#L94
You will be able to see from the source code that this "signal" was extra made for logrotating and should be preferred over "USR2" which should only be used for reloading configs.
I have an account on remote network file system and files older that 12 weeks are purged. I was wondering if it is a good idea to touch every file once in a week using crontab.
Some files on that filesystem are being updated by different numerical programs running on the cluster. Would there be any conflict if a file is being updated and was touched at the same time?
This is what I am planning to do:
crontab file
0 0 * * 0 find /globle/scratch/sd/pratik -exec touch { } \;
I configured RSnapshot on a WD My Book Live (2TB) and its working (at least that's what the logs say). I used the reporting tool rsnapreport.pl from /usr/share/doc/rsnapshot/examples/utils/rsnapreport.pl.gz to get human readable mail reports about the crontab triggered backup jobs.
While the backup jobs seem to work, the reports are a obviously missing information as you can see in this snipplet:
SOURCE TOTAL FILES FILES TRANS TOTAL MB MB TRANS LIST GEN TIME FILE XFER TIME
--------------------------------------------------------------------------------------------------------------------
rsync://server:/vmail 13950 137 3687.81 20.31 0.052 seconds 0.000 seconds
ERRORS
/shares/rsnapshot/daily.0/ NO STATS DATA
The Question now is:
Beside the error at the bottom, which is my first and major problem and question, the FILE XFER TIME is also 0 for all backup jobs (I guess that the issues correlate).
I followed all instructions (see below) - what am I missing?
So what did I do so far:
*) The NAS runs Debian Squeeze (incl. squeeze-backports), Kernel Version is 2.6.32, PPC Architecture.
*) rsync version 3.0.3-2 (preinstalled), with /etc/rsyncd.conf:
pid file=/var/run/rsyncd.pid
lock file=/var/run/rsync.lock
log file=/var/log/rsync.log
[rsync]
path=/shares/rsync
uid=root
gid=share
read only=no
list=yes
auth users=root
*) Installed rsnapshot 1.3.1-1 with /etc/rsnapshot.conf:
config_version 1.2
snapshot_root /shares/rsnapshot/
cmd_rm /bin/rm
cmd_rsync /usr/bin/rsync
cmd_logger /usr/bin/logger
interval daily 7
interval weekly 4
interval monthly 3
verbose 3
loglevel 3
logfile /var/log/rsnapshot.log
lockfile /var/run/rsnapshot.pid
rsync_long_args --delete --numeric-ids --relative --delete-excluded --stats
backup rsync://server:/vmail/ backupOfServer/vmail/
backup ...
backup ...
backup ...
*) unpacked the report script and followed instructions in the script (most of which you can see in the config above):
# this script prints a pretty report from rsnapshot output
# in the rsnapshot.conf you must set
# verbose >= 3
# and add --stats to rsync_long_args
# then setup crontab 'rsnapshot daily 2>&1 | rsnapreport.pl | mail -s"SUBJECT" backupadm#adm.com
# don't forget the 2>&1 or your errors will be lost to stderr
*) and set up cron.d/rsnapshot:
MAILTO="user1#foo,user2#foo"
30 3 * * * root /usr/bin/rsnapshot daily 2>&1 | /root/rsnapreport.pl
0 3 * * 1 root /usr/bin/rsnapshot weekly 2>&1 | /root/rsnapreport.pl
30 2 1 * * root /usr/bin/rsnapshot monthly 2>&1 | /root/rsnapreport.pl
If you need any detailed or additional information, don't hesitate. We are happy to have daily reports of the backup at all, just the errors at the bottom are making us nervous.
Best Regards and thanks in advance,
Peter
The reason for this error was, that I did not uncomment the cmd_cp parameter. RSnapshot therefore used its build-in copy mechanism, which uses rsync.
This call of rsync was echoed to the output. The report script scans the output for calls to rsync and looks for transfer statistics, but the initial "copy" command does not produce such stats - therefore the error saying "NO STATS" for the source /daily.0
The solution is, to read the configuration file and follow the instructions:
# LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
#cmd_cp /bin/cp
Uncommenting the last line fixes the error... RTFM ;)
The "NO STATS DATA" error is also reported if you miss out on the:
rsync_long_args --stats
The "NO STATS DATA" error is also reported if you backing up something containing "rsync" in its path like /etc/default/rsync.
For example in this case the command rsnapshot daily 2>&1 | /bu/script/rsnapreport.pl | mail -s "[BU Report]date" me#example.com will return the following errors:
Use of uninitialized value $source in hash element at
/bu/script/rsnapreport.pl line 95, <> line 3991. Use of uninitialized
value $source in hash element at /bu/script/rsnapreport.pl line 96, <>
line 3991. ...
This is due to the rsnapreport.pl script which parse stats info from the rsync outpout and from the "rsync" string in it.
To simply resolve this problem add in your /etc/rsnapshot.conf the line corresponding to the problematic rsync string found in the rsync outpout:
For example and if you don't need to backup etc/default/rsync:
exclude etc/default/rsync
If you need to backup the files with path containing "rsync" you have to modify the rsnapreport.pl script.