Logrotate duplicates files - linux

I have a test application which runs every hour and uses unique log file at each execution. To clean the logs the following logrotate configuration has been set:
{
# Daily rotation with 1 week of backlog
daily
rotate 7
maxage 7
dateext
compress
}
The first day the log file compressed (which is ok) but a empty file is left and every other day that files is "emptied" and compressed. And that makes 6 files of every logfiles which fills the inodes table of the FS. Here's two examples :
-rw-r--r-- 1 root root 1752 Feb 11 01:36 J20190211013601_Status.txt-20190212.gz
-rw------- 1 root root 20 Feb 12 03:33 J20190211013601_Status.txt-20190213.gz
-rw------- 1 root root 20 Feb 13 03:37 J20190211013601_Status.txt-20190214.gz
-rw------- 1 root root 20 Feb 14 03:10 J20190211013601_Status.txt-20190215.gz
-rw------- 1 root root 20 Feb 15 03:12 J20190211013601_Status.txt-20190216.gz
-rw------- 1 root root 20 Feb 16 03:36 J20190211013601_Status.txt-20190217.gz
-rw------- 1 root root 20 Feb 17 03:44 J20190211013601_Status.txt-20190218.gz
-rw------- 1 root root 0 Feb 18 03:24 J20190211013601_Status.txt
-rw-r--r-- 1 root root 1752 Feb 11 02:36 J20190211023601_Status.txt-20190212.gz
-rw------- 1 root root 20 Feb 12 03:33 J20190211023601_Status.txt-20190213.gz
-rw------- 1 root root 20 Feb 13 03:37 J20190211023601_Status.txt-20190214.gz
-rw------- 1 root root 20 Feb 14 03:10 J20190211023601_Status.txt-20190215.gz
-rw------- 1 root root 20 Feb 15 03:12 J20190211023601_Status.txt-20190216.gz
-rw------- 1 root root 20 Feb 16 03:36 J20190211023601_Status.txt-20190217.gz
-rw------- 1 root root 20 Feb 17 03:44 J20190211023601_Status.txt-20190218.gz
-rw------- 1 root root 0 Feb 18 03:24 J20190211023601_Status.txt
How can i correct this, in order to delete the files after being compressed
Thanks for time and help,

This is how logrotate is supposed to function; your issue stems from the fact that you're using unique filenames every time your appplication runs.
When logrotate runs for the first time on each log, it's moving the log file from "J20190211023601_Status.txt" to "J20190211023601_Status.txt-20190212.gz" and then creating a new file named J20190211023601_Status.txt.
Logrotate has no inherent idea that those filenames are unique and thus will never be populated again; all it sees is a log it's rotated in the past, so it must be rotated again as per your configuration.
Your easiest solution here is to pass the nocreate directive for this logrotation; this will prevent that new log file from being created and subsequently rotated while still respecting the 7-day age limit on previously rotated files:
{
daily
maxage 7
dateext
compress
nocreate
}

Related

How to view logs of container before restart

I have a container which was restart 14 hours ago. The container is running since 7 weeks. I want to inspect the container logs during a certain interval. When i run below command, I see there is no output
docker container logs pg-connect --until 168h --since 288h
When i run below commands i only see logs since the container was restarted.
docker logs pg-connect
Any idea how to retrieve older logs for the container?
More info if helps
> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9f08fb6fb0fb kosta709/alpine-plus:0.0.2 "/connectors-restart…" 7 weeks ago Up 14 hours connectors-monitor
7e919a253a29 debezium/connect:1.2.3.Final "/docker-entrypoint.…" 7 weeks ago Up 14 hours pg-connect
>
>
> docker logs 7e919a253a29 -n 2
2022-08-26 06:37:10,878 INFO || WorkerSourceTask{id=relations-0} Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]
2022-08-26 06:37:10,878 INFO || WorkerSourceTask{id=relations-0} flushing 0 outstanding messages for offset commit [org.apache.kafka.connect.runtime.WorkerSourceTask]
> docker logs 7e919a253a29 |head
org.apache.kafka.common.KafkaException: Producer is closed forcefully.
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:766)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortIncompleteBatches(RecordAccumulator.java:753)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:279)
at java.base/java.lang.Thread.run(Thread.java:834)
2022-08-24 16:13:06,567 ERROR || WorkerSourceTask{id=session-0} failed to send record to barclays.public.session: [org.apache.kafka.connect.runtime.WorkerSourceTask]
org.apache.kafka.common.KafkaException: Producer is closed forcefully.
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:766)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortIncompleteBatches(RecordAccumulator.java:753)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:279)
>
> ls -lart /var/lib/docker/containers/7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1/
total 90720
drwx------ 2 root root 6 Jul 1 10:39 checkpoints
drwx--x--- 2 root root 6 Jul 1 10:39 mounts
drwx--x--- 4 root root 150 Jul 1 10:40 ..
-rw-r----- 1 root root 10000230 Aug 24 16:13 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.9
-rw-r----- 1 root root 10000163 Aug 24 16:13 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.8
-rw-r----- 1 root root 10000054 Aug 24 16:16 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.7
-rw-r----- 1 root root 10000147 Aug 24 16:42 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.6
-rw-r----- 1 root root 10000123 Aug 24 16:42 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.5
-rw-r----- 1 root root 10000019 Aug 24 16:42 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.4
-rw-r----- 1 root root 10000159 Aug 24 16:42 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.3
-rw-r----- 1 root root 10000045 Aug 24 16:42 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.2
-rw-r--r-- 1 root root 199 Aug 25 16:30 hosts
-rw-r--r-- 1 root root 68 Aug 25 16:30 resolv.conf
-rw-r--r-- 1 root root 25 Aug 25 16:30 hostname
-rw------- 1 root root 7205 Aug 25 16:30 config.v2.json
-rw-r--r-- 1 root root 1559 Aug 25 16:30 hostconfig.json
-rw-r----- 1 root root 10000085 Aug 25 16:31 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log.1
drwx--x--- 4 root root 4096 Aug 25 16:31 .
-rw-r----- 1 root root 2843232 Aug 26 06:38 7e919a253a296494b74361e258e49d8c3ff38f345455316a15e1cb28cf556fa1-json.log
As stated by [the official guide][1]:
The docker logs command batch-retrieves logs present at the time of execution.```
To solve this issue you should instrument the container software to log its output to a persistent (rotated if you want) log file.
[1]: https://docs.docker.com/engine/reference/commandline/logs/

Installation Of Cassandra

I have download cassandra via terminal but problem is where are the other folders like data, conf, lib, doc etc.
i can see only some files as shown in figure i.e Click here
where is the other folders ?
By "download cassandra via terminal" and your screenshot, I'll assume that you installed Cassandra via apt-get.
From the Apache Cassandra project Wiki, section on Installation from Debian packages:
The default location of configuration files is /etc/cassandra.
The default location of log and data directories is /var/log/cassandra/ and /var/lib/cassandra.
As for the lib directory, check how your $CASSANDRA_HOME is being set:
$ grep CASSANDRA_HOME /etc/init.d/cassandra
CASSANDRA_HOME=/usr/share/cassandra
$ ls -al /usr/share/cassandra/
total 8312
drwxr-xr-x 3 root root 4096 Dec 13 07:57 .
drwxr-xr-x 372 root root 12288 Nov 28 08:51 ..
-rw-r--r-- 1 root root 5962385 Jun 1 2016 apache-cassandra-3.6.jar
lrwxrwxrwx 1 root root 24 Jun 1 2016 apache-cassandra.jar -> apache-cassandra-3.6.jar
-rw-r--r-- 1 root root 1902216 Jun 1 2016 apache-cassandra-thrift-3.6.jar
-rw-r--r-- 1 root root 875 May 31 2016 cassandra.in.sh
drwxr-xr-x 3 root root 12288 Dec 13 07:57 lib
-rw-r----- 1 root root 82123 Oct 20 2015 metrics-core-2.2.0.jar
-rw-r----- 1 root root 9639 Oct 20 2015 metrics-graphite-2.2.0.jar
-rw-r--r-- 1 root root 509144 Jun 1 2016 stress.jar
Note that Cassandra's lib directory is shown in the middle of the directory listing above.

npm erased Plesk 12 on Ubuntu 16.04

My issue is pretty simple to explain and I guess hard to solve: I commited the stupidity of installing npm on a cloud based server with Ubuntu 16.04 with Plesk 12.
After reading this article I realized it was too late and after trying to connect to my Plesk GUI got an 403 error.
Also if I execute plesk on the CLI, it shows:
user#server:~$ plesk repair
plesk: command not found
So, I erased Plesk... All my hosted sites on Plesk are reachable with their databases in their URLs or by SSH.
The hosting provider told me there is no way to restore Plesk without losing everything unless I made a backup, but I didn't. Maybe there is an alternative... Do you know it?
Edit: Content of /var/lib/psa/dumps:
user#server:/var/lib/psa/dumps$ ls -lrta
total 5708
drwxr-xr-x 3 root root 4096 ene 4 2017 ..
-rw------- 1 root root 206315 ene 4 2017 mysql.preupgrade.12.0.18-12.0.18.20170104-173632.dump.gz
-rw------- 1 root root 3417 ene 4 2017 mysql.preupgrade.apsc.12.0.18-12.0.18.20170104-173633.dump.gz
-rw------- 1 root root 208481 ene 4 2017 mysql.preupgrade.12.0.18-12.5.30.20170104-174155.dump.gz
-rw------- 1 root root 3002 ene 4 2017 mysql.preupgrade.apsc.12.0.18-12.5.30.20170104-174156.dump.gz
-rw------- 1 root root 220725 ene 23 2017 mysql.preupgrade.12.5.30-12.5.30.20170123-062554.dump.gz
-rw------- 1 root root 3002 ene 23 2017 mysql.preupgrade.apsc.12.5.30-12.5.30.20170123-062556.dump.gz
-rw------- 1 root root 236736 feb 8 06:27 mysql.preupgrade.12.5.30-12.5.30.20170208-062713.dump.gz
-rw------- 1 root root 3003 feb 8 06:27 mysql.preupgrade.apsc.12.5.30-12.5.30.20170208-062715.dump.gz
-rw------- 1 root root 262580 feb 24 06:26 mysql.preupgrade.12.5.30-12.5.30.20170224-062621.dump.gz
-rw------- 1 root root 4603 feb 24 06:26 mysql.preupgrade.apsc.12.5.30-12.5.30.20170224-062623.dump.gz
-rw------- 1 root root 258785 mar 22 06:26 mysql.preupgrade.12.5.30-12.5.30.20170322-062626.dump.gz
-rw------- 1 root root 4898 mar 22 06:26 mysql.preupgrade.apsc.12.5.30-12.5.30.20170322-062627.dump.gz
-rw------- 1 root root 251339 abr 17 06:25 mysql.preupgrade.12.5.30-12.5.30.20170417-062540.dump.gz
-rw------- 1 root root 4899 abr 17 06:25 mysql.preupgrade.apsc.12.5.30-12.5.30.20170417-062543.dump.gz
-rw------- 1 root root 244219 may 16 06:25 mysql.preupgrade.12.5.30-12.5.30.20170516-062533.dump.gz
-rw------- 1 root root 4373 may 16 06:25 mysql.preupgrade.apsc.12.5.30-12.5.30.20170516-062535.dump.gz
-rw------- 1 root root 248044 jun 1 06:25 mysql.preupgrade.12.5.30-12.5.30.20170601-062529.dump.gz
-rw------- 1 root root 4381 jun 1 06:25 mysql.preupgrade.apsc.12.5.30-12.5.30.20170601-062530.dump.gz
-rw------- 1 root root 273341 jul 17 06:25 mysql.preupgrade.12.5.30-12.5.30.20170717-062542.dump.gz
-rw------- 1 root root 4379 jul 17 06:25 mysql.preupgrade.apsc.12.5.30-12.5.30.20170717-062544.dump.gz
-rw------- 1 root root 367277 jul 20 06:26 mysql.daily.dump.8.gz
-rw------- 1 root root 367218 jul 21 06:25 mysql.daily.dump.7.gz
-rw------- 1 root root 368954 jul 22 06:25 mysql.daily.dump.6.gz
-rw------- 1 root root 369279 jul 23 06:25 mysql.daily.dump.5.gz
-rw------- 1 root root 368767 jul 24 06:25 mysql.daily.dump.4.gz
-rw------- 1 root root 369629 jul 25 06:26 mysql.daily.dump.3.gz
-rw------- 1 root root 370169 jul 26 06:25 mysql.daily.dump.2.gz
-rw------- 1 root root 368027 jul 27 06:25 mysql.daily.dump.1.gz
-rw------- 1 root root 368128 jul 28 06:26 mysql.daily.dump.0.gz
drwxr-xr-x 2 psaadm psaadm 4096 jul 29 01:32 .
-rw------- 1 root root 20 jul 29 01:32 mysql.plesk.core.prerm.12.5.30.20170729-013250.dump.gz
First of all, I recommend to change the hosting provider, as their answer is incorrect and it seems, that they don't have much experience with Plesk, nor do they seem to have much knowledge about Plesk.
Second, your statements are a bit unclear, as you state:
although I made a backup
and directly afterwards you stated:
and I didn't
Could you pls. clarify WHAT you did and what you forgot to do?
Third, pls. have a look at => "/var/lib/psa/dumps" and inform us about the possible content.
Fourth, if you have dumps located at "/var/lib/psa/dumps", you will always have the choice to re-install Plesk and to re-import the latest "psa", "mysql" and "horde" - databases from your dumps. How to restore a Plesk database dump is explained at:
=> https://support.plesk.com/hc/en-us/articles/213904125
If you have further Plesk related questions, I recommend a new thread at the official Plesk Community Forum ( => https://talk.plesk.com ) , where experienced Plesk users will help you with Plesk - related questions/issues/errors/problems! ;-)

Searching shared libraries in Linux

please explain me why on x86_64 Scientific Linux no file under /etc/ld.so.conf.d contains the directory /usr/lib64?
The list of directories to be searched by program loader is stored in the file /etc/ld.so.conf. On my distributive, this file stores this row: include ld.so.conf.d/.conf*
And the above directory consists of:
[root#dev_host build]$ ls -la /etc/ld.so.conf.d
total 36
drwxr-xr-x. 2 root root 4096 Aug 29 23:13 .
drwxr-xr-x. 103 root root 12288 Sep 18 03:41 ..
-rw-r--r--. 1 root root 17 Mar 20 2012 atlas-x86_64.conf
-r--r--r--. 1 root root 324 May 7 23:40 kernel-2.6.32-431.17.1.el6.x86_64.conf
-r--r--r--. 1 root root 324 Nov 22 2013 kernel-2.6.32-431.el6.x86_64.conf
-rw-r--r--. 1 root root 17 Feb 12 2014 mysql-x86_64.conf
lrwxrwxrwx 1 root root 31 Aug 11 14:46 postgresql-pgdg-libs.conf -> /etc/alternatives/pgsql-ld-conf
-rw-r--r--. 1 root root 22 Sep 7 2011 qt-x86_64.conf
I examined all these files - there is no /usr/lib64. Why? Is it stored in /etc/ld.so.cache?

Size of kernel built is much much larger than the built-in one

I got latest kernel source from kernel.org(using git), and followed the steps as described in this page to build the kernel. The kernel boots successfully, however, I have no idea what was done incorrectly in the configuration process that initrd.img-3.16.0 is so much larger than the build in one(initrd.img-3.13.0-32-generic)
I copied the configuration file .config from /boot/ and used "yes '' | make oldconfig" for the kernel configuration.
the file size total 191M
-rw-r--r-- 1 root root 1.2M Jul 14 21:29 abi-3.13.0-32-generic
-rw-r--r-- 1 root root 162K Jul 14 21:29 config-3.13.0-32-generic
-rw-r--r-- 1 root root 167K Aug 4 19:48 config-3.16.0
-rw-r--r-- 1 root root 20M Jul 28 15:14 initrd.img-3.13.0-32-generic
-rw-r--r-- 1 root root 151M Aug 4 19:48 initrd.img-3.16.0
-rw-r--r-- 1 root root 173K Mar 12 05:31 memtest86+.bin
-rw-r--r-- 1 root root 174K Mar 12 05:31 memtest86+.elf
-rw-r--r-- 1 root root 175K Mar 12 05:31 memtest86+_multiboot.bin
-rw------- 1 root root 3.3M Jul 14 21:29 System.map-3.13.0-32-generic
-rw-r--r-- 1 root root 3.4M Aug 4 19:48 System.map-3.16.0
-rw------- 1 root root 5.6M Jul 14 21:29 vmlinuz-3.13.0-32-generic
-rw-r--r-- 1 root root 5.7M Aug 4 19:48 vmlinuz-3.16.0
Thanks!
William
follow below steps to obtain the right kernel configuration
Copy /boot/.config to the kernel source code directory
make menuconfig
Exit and save configuration
make
and then continue with the other options for install
Note : Since you are using make oldconfig, this would enable many of the options not related to the platform but related to the CPU architecture.
This steps should help you solve this issue

Resources