I have to delete the log files older than 7 days even if they were modified within a period of 7 days. But the only solution I can find anywhere is based on find command using mtime option as below:
find /path/to/files -mtime +7 -exec rm {} \;
What is the possible solution to this problem.
If you're using a filesystem that tracks file birth time and a current enough Linux kernel, glibc and GNU coreutils, you can do something like
weekago=$(date -d "last week" +%s)
for file in /path/to/files/*; do
birthdate=$(stat -c %W "$file")
if [[ $birthdate -gt 0 && $birthdate -lt $weekago ]]; then
printf "%s\n" "$file"
# Uncomment when you're sure you're getting just the files you want
# rm -- "$file"
fi
done
Related
I have a remote server that copies 30-some backup files to a local server every day and I want to remove the old backups if and only if a newer backup successfully copied.
With different codes I tried, I managed to erase older files, but I got the problem that if it found one new backup, it deleted ALL older ones.
I have something like (picture this with 20 virtual machines):
vm001-2019-08-01.bck
vm001-2019-07-28.bck
vm002-2019-08-01.bck
vm003-2019-07-29.bck
vm004-2019-08-01.bck
vm004-2019-07-31.bck
vm004-2019-07-30.bck
vm004-2019-07-29.bck
...
And I'd want to erase all but keep only the most recent ones.
i.e.: erase:
vm001-2019-07-28.bck
vm002-2019-07-29.bck
vm004-2019-07-31.bck
vm004-2019-07-30.bck
vm004-2019-07-29.bck
and keep only:
vm001-2019-08-01.bck
vm002-2019-08-01.bck
vm003-2019-07-29.bck
vm004-2019-08-01.bck
the problem I had is that if I have any recent backup of any machine, files like vm-003-2019-07-29 get deleted, because they are older, even if they are of different machines.
I know there are several variants of this question in the site, but I can't quite get this to work.
I've been trying variants of this code:
#!/bin/bash
for i in ./*.bck
do
echo "found" "$i"
if [[ -n $(find "$i" -type f -mmin -1440) ]]
then
echo "$i"
find "$i" -type f -mmin +1440 -exec rm -f "$i" {} +
fi
done
(The echos are for debugging purposes only)
At this time, this code finds the newer and the older files, but doesn't delete anything. If I put find "$i" -type f -mmin +1440 -exec echo "$i" {} +, it never prints anything, as if find $i is not finding anything, but when I run it as a solo command in the terminal, it does (minus the -exec part).
I've tested this script generating files with different timestamps using touch -d, but I had no success.
Unless you add the -name test before the filename find is going to consider "$i" to be the name of a directory to search in. So your find command should be:
find -name "$i" -type f -mmin -1440
which will search in the current directory. Or
find /path/to/dir -name "$i" -type f -mmin -1440
which will search in a directory named "/path/to/dir".
But, based on BashFAQ/099, I would do this to delete all but the newest file for each VM (untested):
#!/bin/bash
declare -A newest # associative array to store name of newest file for each VM
for f in *
do
vm=${f%%-*} # extracts vm name from filename (i.e. vmm001 from vm001-2019-08-01.bck)
if [[ -f $f && $f -nt ${newest["$vm"]} ]]
then
newest["$vm"]=$f
fi
done
for f in *
do
vm=${f%%-*}
if [[ -f $f && $f != ${newest["$vm"]} ]]
then
rm "$f"
fi
done
This is set up to run against files in the current directory. It assumes that the files are named as shown in the question (the VM name is separated from the rest of the file name by a hyphen). In order to use an associative array, Bash 4 or higher is required.
I have this code:
#!/bin/bash
path="/home/asdf"
dateminusoneday=$(date +%m --date='-1 month')
date=$(date +"%Y-$dateminusoneday-%d")
list=$(find /home/asdf | grep -P '\d{4}\-\d{2}\-\d{2}' -o)
listArray=($list)
for i in "${listArray[#]}"
do
echo $i
if [[ $i < $date ]]; then
echo "delete file"
else
echo "no need delete this file" fi done
I need to delete the smallest files that date. but I do not get it
What would be the most optimal way?
thanks all.
From your code I see that you are trying to delete files older than one month. If I am not mistaken and you can accept that (1 month)==(30 days) you can use such one-liner:
find "$path" -mtime +30 -delete
If you want exactly 1 mont (not 30 days) you can use:
#!/bin/bash
path="/home/asdf"
number_of_days=$((($(date '+%s')-$(date -d '1 month ago' '+%s'))/86400))
find "$path" -mtime +$number_of_days -delete
I'm trying to write a script to create mysqldumps daily in a directory as well as check all the backups in that directory and remove any older than 7 days that is going to run on cron.
So my functions work correctly, it's just my last echo command that is not doing what I want it to. This is what I have so far:
DBNAME=database
DATE=`date +\%Y-\%m-\%d_\%H\%M`
SQLFILE=$DBNAME-${DATE}.sql
curr_dir=$1
#MAIN
mysqldump -u root -ppassword --databases $DBNAME > $SQLFILE
echo "$SQLFILE has been successfully created."
#Remove files older than 7 days
for filepath in "$curr_dir"*
do
find "$filepath" -mtime +7 -type f -delete
echo "$filepath has been deleted."
done
exit
So the backup creations and removal of old files both work. But, my problem is that echo "$filepath has been deleted." is printing all files in the directory instead of just the files older than 7 days that were deleted. Where am I going wrong here?
EDIT (Full solution):
This is the full solution that wound up working for me using everyone's advice from the answers and comments. This works for cron jobs. I had to specify the main function's output filepath because the files were being created in the root directory instead of the path specified in Argument $1.
Thank you everyone for the help! The if statement also checks whether or not $1 is the specified directory I want files to be deleted in.
#Variables
DBNAME=database
DATE=`date +\%Y-\%m-\%d_\%H\%M`
SQLFILE=$DBNAME-${DATE}.sql
curr_dir=$1
#MAIN
mysqldump -u root -ppassword --databases $DBNAME > /path/to/db-backups/directory/$SQLFILE
echo "$SQLFILE has been successfully created."
#Remove files older than 7 days
for filepath in "$curr_dir"*
do
if [[ $1 = "/path/to/db-backups/directory" ]]; then
find "$filepath" -mtime +7 -type f -delete -exec sh -c 'printf "%s has been deleted.\n" "$#"' _ {} +
fi
done
exit
You can merge the echo into the find:
find "$filepath" -mtime +7 -type f -delete -exec echo '{}' "has been deleted." \;
The -delete option is just a shortcut for -exec rm '{}' \; and all the -exec commands are run in the sequence you specify them in.
I run some kind of server on my linux machine and I use simple bash script to delete files every 3 and some files every 7 days. I use find command for doing that.But my files are saved periodically, meaning that the last modification day is the current day. So files never get deleted. Only worked for me the first time, because it met the conditions. I can't find a way to delete those files using a creation date, not modification date.
Here's my simple script:
#!/bin/sh
while true
do
java -server file.jar nogui
echo ">$(tput setaf 3)STARTING REBOOT$(tput sgr0) $(tput setaf 7)(Ctrl+C To Stop!)$(tput sgr0)"
find /folder/* -mtime +7 -exec rm -rf {} \;
find /folder/* -mtime +3 -exec rm -rf {} \;
find /logs/* -mtime +1 -exec rm -rf {} \;
echo ">Rebooting in:"
for i in 5 4 3 2 1
do
echo ">$i..."
sleep 1
done
done
If someone could help me with this, I would be really thankful!
Just an idea-don't shoot... :-)
If the files are not system files automatically generated by some process but is lets say server log files, you could possibly echo inside the file the creation date (i.e at the end or beginning) and grep that value later to decide if must be removed or kept.
No, it is not possible. Standard Linux filesystems do not track creation time at all. (ctime, sometimes mistaken for creation time, is metadata change time -- as compared to mtime, which is data modification time).
That said, there are certainly ugly hacks available. For instance, if you have the following script invoked by incron (or, less efficiently, cron) to record file creation dates:
#!/bin/bash
mkdir -p .ctimes
for f in *; do
if [[ -f $f ]] && [[ ! -e .ctimes/$f ]]; then
touch ".ctimes/$f"
fi
done
...then you can look for files in the .ctimes directory that are older than three days, and delete both the markers and the files they stand for:
#!/bin/bash
find .ctimes -mtime +3 -type f -print0 | while IFS= read -r -d '' filename; do
realname=${filename#.ctimes/}
rm -f -- "$filename" "$realname"
done
If you are on ext4 Filesystem there is some hope. You can retrieve it using stat and debugfs utilities. ext4 stores creation time with inode table entry i_crtime which is 'File creation time, in seconds since the epoch' per docs. Reference Link.
I am working on a backup script and I've got a problem. I would like to backup my documents to a ftp server. Because I don't like encfs so I try to realise this by using z-zip and encrypted archives. This is working well but I would like to create a new archive only when a file inside a subdirectory has changed so lftp is only uploading the changed ones.
My codesnippet looks like this:
cd /mnt/HD_a2/documents
for i in */
do 7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
done
How can I change this code so it's only creating a new archive when a file inside "i" has been changed within the last 7 days? (This script is executed by cron every 7 days)
for i in */
do
if [ `find "$i" -type f -mtime -7 | wc -l` -gt 0 ]
then 7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
fi
done
So, Barmar`s answer is almost right, but it does not count files correctly. I've looked around for other similar questions and it seems like it is a common mistake(please note that it is not critical for his solution, but it might confuse other programmers if they touch it, because it does not what most people will expect), no one is accounting the fact that filenames can contain newlines. So here is a bit better version that gives you the right file count:
for i in */
do
fileCount=$(find "$i" -type f -mtime -8 -exec printf . \;)
if (( ${#fileCount} > 0 )); then
7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
fi
done
But what if you have thousands of files? That would build a string that is exactly as long as the number of your files.
So we can use this:
for i in */
do
fileCount=$(find "$i" -type f -mtime -8 -exec printf . \; | wc -c)
if (( fileCount > 0 )); then
7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
fi
done
Or this:
for i in */
do
fileCount=$(find "$i" -type f -mtime -8 -exec echo \; | wc -l)
if (( fileCount > 0 )); then
7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
fi
done
Hoping that useless string data is not going to be cached.
But we only need to know if there is AT LEAST one file, we don't have to count all of them! This way we can stop immediately if something was found.
for i in */
do
fileFound=$(find "$i" -type f -mtime -8 | head -n 1)
if [[ $fileFound ]]; then
7za a -t7z /mnt/HD_a2/encrypted/ul_doc/"${i%/}.7z" -p1234 -mhe "$i"
fi
done
That's it. This solution will work MUCH faster because it does not have to look for other files and it does not even have to check their mtime. Try running this code without head and you'll notice a significant difference - from several hours to less than a second for my home folder. (I'm not even sure if it will ever manage to finish it on my pc, I have millions of small files in my home folder...)