In a directory there are files which are generated daily.
Format of files, if its generated on 16th Apr 2012 is TEST_20120416.
So I need to delete all the files which are older than 7 days. I tried doing this
#!/bin/ksh
find /data/Test/*.* -mtime -7 -exec rm -rf {} \;
exit 0
Now the problem is above code is deleting based on modification time but according to requirement file should delete based on creation time.Kindly help me out in deleting files based on filename(filename has timestamp).
As you fortunately have creation date encoded in filename, this should work:
#!/bin/sh
REFDATE=$(date --date='-7 days' +%Y%m%d)
PREFIX=TEST_
find /data/Test/ -name $PREFIX\* | while read FNAME; do
if [ ${FNAME#$PREFIX} -lt $REFDATE ]; then
rm $FNAME
fi
done
It will print warnings if you have some other files with names starting with TEST_ in which case some more filtering may be needed.
find /data/Test/*.* -ctime -7 -delete
'find /data/Test/.' will find the all the files in the /data/Test folder and argument '-ctime -7' will limit the search to the creation time to last 7 days and -delete option will delete such files
Related
This question already has answers here:
Shell script to delete directories older than n days
(5 answers)
Closed 9 years ago.
I want to delete scripts in a folder from the current date back to 10 days.
The scripts looks like:
2012.11.21.09_33_52.script
2012.11.21.09_33_56.script
2012.11.21.09_33_59.script
The script will run in every 10 day with Crontab, that's why I need the current date.
find is the common tool for this kind of task :
find ./my_dir -mtime +10 -type f -delete
EXPLANATIONS
./my_dir your directory (replace with your own)
-mtime +10 older than 10 days
-type f only files
-delete no surprise. Remove it to test your find filter before executing the whole command
And take care that ./my_dir exists to avoid bad surprises !
Just spicing up the shell script above to delete older files but with logging and calculation of elapsed time
#!/bin/bash
path="/data/backuplog/"
timestamp=$(date +%Y%m%d_%H%M%S)
filename=log_$timestamp.txt
log=$path$filename
days=7
START_TIME=$(date +%s)
find $path -maxdepth 1 -name "*.txt" -type f -mtime +$days -print -delete >> $log
echo "Backup:: Script Start -- $(date +%Y%m%d_%H%M)" >> $log
... code for backup ...or any other operation .... >> $log
END_TIME=$(date +%s)
ELAPSED_TIME=$(( $END_TIME - $START_TIME ))
echo "Backup :: Script End -- $(date +%Y%m%d_%H%M)" >> $log
echo "Elapsed Time :: $(date -d 00:00:$ELAPSED_TIME +%Hh:%Mm:%Ss) " >> $log
The code adds a few things.
log files named with a timestamp
log folder specified
find looks for *.txt files only in the log folder
type f ensures you only deletes files
maxdepth 1 ensures you dont enter subfolders
log files older than 7 days are deleted ( assuming this is for a backup log)
notes the start / end time
calculates the elapsed time for the backup operation...
Note: to test the code, just use -print instead of -print -delete. But do check your path carefully though.
Note: Do ensure your server time is set correctly via date - setup timezone/ntp correctly . Additionally check file times with 'stat filename'
Note: mtime can be replaced with mmin for better control as mtime discards all fractions (older than 2 days (+2 days) actually means 3 days ) when it deals with getting the timestamps of files in the context of days
-mtime +$days ---> -mmin +$((60*24*$days))
If you can afford working via the file data, you can do
find -mmin +14400 -delete
I'm new to bash, I have a task to delete all files older than 30 days, I can figure this out based on the files name Y_M_D.ext 2019_04_30.txt.
I know I can list all files with ls in a the folder containing the files. I know I can get todays date with $ date and can configure that to match the file format $ date "+%Y_%m_%d"
I know I can delete files using rm.
How do I tie all this together into a bash script that deletes files older than 30 days from today?
In pseudo-python code I guess it would look like:
for file in folder:
if file.name to date > 30 day from now:
delete file
I am by no means a systems administrator, but you could consider a simple shell script along the lines of:
# Generate the date in the proper format
discriminant=$(date -d "30 days ago" "+%Y_%m_%d")
# Find files based on the filename pattern and test against the date.
find . -type f -maxdepth 1 -name "*_*_*.txt" -printf "%P\n" |
while IFS= read -r FILE; do
if [ "${discriminant}" ">" "${FILE%.*}" ]; then
echo "${FILE}";
fi
done
Note that this is will probably be considered a "layman" solution by a professional. Maybe this is handled better by awk, which I am unfortunately not accustomed to using.
Here is another solution to delete log files older than 30 days:
#!/bin/sh
# A table that contains the path of directories to clean
rep_log=("/etc/var/log" "/test/nginx/log")
echo "Cleaning logs - $(date)."
#loop for each path provided by rep_log
for element in "${rep_log[#]}"
do
#display the directory
echo "$element";
nb_log=$(find "$element" -type f -mtime +30 -name "*.log*"| wc -l)
if [[ $nb_log != 0 ]]
then
find "$element" -type f -mtime +30 -delete
echo "Successfull!"
else
echo "No log to clean !"
fi
done
allows to include multiple directory where to delete files
rep_log=("/etc/var/log" "/test/nginx/log")
we fill the var: we'r doing a search (in the directory provided) for files which are older than 30 days and whose name contains at least .log. Then counts the number of files.
nb_log=$(find "$element" -type f -mtime +30 -name "*.log*"| wc -l)
we then check if there is a result other than 0 (posisitive), if yes we delete
find "$element" -type f -mtime +30 -delete
For delete file older than X days you can use this command and schedule it in /etc/crontab
find /PATH/TO/LOG/* -mtime +10 | xargs -d '\n' rm
or
find /PATH/TO/LOG/* -type f -mtime +10 -exec rm -f {} \
In my script i want to check if file named "Flag" was created more than 10 minutes from now, and if it was, delete it.
Got something like this:
ZMIENNA=`find /home/maciej/testy/ -mmin +10 -name Flag`
if [ -N $ZMIENNA ]
then
rm /home/maciej/testy/Flag
fi
Thank you for replying.
You can just use find to delete a file, for example to delete file1 in the current folder:
find . -name file1 -mmin +10 -exec rm {} \;
According to the manual the option -mmin of find is based on the last modification date:
-mmin n
File's data was last modified n minutes ago.
As stated by this post : How to get file creation date/time in Bash/Debian?
the creation time is not stored as file metadata on POSIX systems.
I am using following simple script script to take backup of all of my websites via tar
TIME=`date +%b-%d-%y`
FILENAME=backup-$TIME.tar.gz
#Parent backup directory
backup_parent_dir="/backup/httpdocs"
#Create backup directory and set permissions
backup_date=`date +%Y_%m_%d_%H_%M`
backup_dir="${backup_parent_dir}/${backup_date}"
echo "Backup directory: ${backup_dir}"
mkdir -p "${backup_dir}"
chmod 755 "${backup_dir}"
SRCDIR=/var/www/sites #Location of Important Data Directory (Source of backup).
tar -cpzf $backup_dir/$FILENAME $SRCDIR
Now, this is wokring fine but I need 2 things if I can do via same script
Can I exclude some folder within /var/www/sites directory, like if I don't want /var/www/sites/abc.com/logs folder to be backup. Can I define it and some other sub-directories within this script?
This script takes all sites in tar format in specified folder /backup/httpdocs through cronjob which runs daily during night and for old tarballs(older than 7 days) I have to delete them manually, so it there any possibility through same script so when it runs, it checks if there is any backup exists older than 7 days and it deletes it automatically?
EDIT:
Thanks everyone, this is what I am using now which takes backup excluding log files and delete anything older than 7 days
#!/bin/bash
#START
TIME=`date +%b-%d-%y` # This Command will add date in Backup File Name.
FILENAME=backup-$TIME.tar.gz # Here i define Backup file name format.
# Parent backup directory
backup_parent_dir="/backup/httpdocs"
# Create backup directory and set permissions
backup_date=`date +%Y_%m_%d_%H_%M`
backup_dir="${backup_parent_dir}/${backup_date}"
echo "Backup directory: ${backup_dir}"
mkdir -p "${backup_dir}"
chmod 755 "${backup_dir}"
SRCDIR=/var/www/vhosts # Location of Important Data Directory (Source of backup).
tar -cpzf $backup_dir/$FILENAME $SRCDIR --exclude=$SRCDIR/*/logs
find ${backup_parent_dir} -name '*' -type d -mtime +2 -exec rm -rfv "{}" \;
#END
Exclude from tar:
tar -cpzf $backup_dir/$FILENAME --exclude=/var/www/sites/abc.com/logs $SRCDIR
Find and delete old backups:
find ${backup_parent_dir} -type f -name 'backup-*.tar.gz' -mtime +7 -delete
The filter of find is conservative: selecting names that match backup-*.tar.gz probably renders the -type f (files only) option useless. I added it just in case you also have directories with such names. The -mtime +7 option is to be checked by you because older than 7 days is not accurate enough. Depending on what you have in mind it may be +6, +7 or +8. Please have a look at the find man page and decide by yourself. Note that the selection of backups to delete is not based on their names but on their date of last modification. If you modify them after they are created it may not be what you want. Let us know.
Use the --exclude option
tar -cpzf $backup_dir/$FILENAME $SRCDIR --exclude=$SRCDIR/*/logs
Name your backup files with an identifier that is derived from the day of the week. This will ensure the new file for each day will overwrite any existing file.
Day:
a Day of the week - abbreviated name (Mon)
A Day of the week - full name (Monday)
u Day of the week - number (Monday = 1)
d Day of the month - 2 digits (05)
e Day of the month - digit preceded by a space ( 5)
j Day of the year - (1-366)
w Same as 'u'
From: http://ss64.com/bash/date.html
For example:
TARGET_BASENAME= `date +%u`
option 1 when not many files/directories to exclude
tar -cpzf $backup_dir/$FILENAME --exclude=$SRCDIR/dir_ignore --exclude=$SRCDIR/*.log $SRCDIR
or if you have many entries to exclude much better done it in file
tar -cpzf $backup_dir/$FILENAME -X /path/to/exclude.txt $SRCDIR
where /path/to/exclude.txt file looks like
/var/www/dir_to_ignore
/var/www/*.log
you cannot use variables,but can use wildcards
second question answered very good by both guys before,i personalty love
find ${backup_parent_dir} -type f -name 'backup-*.tar.gz' -mtime +7 -delete
I'd need a shell command to show of the last modified and new files on the whole server (recursive) and echoing them into a txt file in the root.
Has anybody something like this?
I tried
find / - mmtime 30 -printf "%AD %Ar - %p\n" 2> /dev/null | sort -r > /lastmodified.txt
to post all names of all modified files from the last 30 days into a txt file in root, but it shows me only the files of the server itself and not the directories where my websites are uploaded to.
Thank you in advance - I am not an expert, and this is what I found so far. It is relative urgent as I need this to fix hacked files which has happend last week.
From http://content.hccfl.edu/pollock/Unix/FindCmd.htm:
find . -mtime 0 # find files modified within the past 24 hours
find . -mtime -1 # find files modified within the past 24 hours
find . -mtime 1 # find files modified between 24 and 48 hours ago
find . -mtime +1 # find files modified more than 48 hours ago
Make sure that you have only one 'm' and a minus sign in -mtime -30, as suggested in chorobas comment, to get last 30 days. -mtime 30 would give just files exactly 30 days ago.
You may want to use option -daystart to get files of last 30 days starting from midnight instead of just 30*24 hours ago. Use %TD and %Tr instead of %AD and %Ar to get modification times (instead of access times).
The final command would then be:
find / -daystart -mtime -30 -printf "%TD %Tr - %p\n" 2> /dev/null | sort -r > /lastmodified.txt
Note that the sort will break in January as 12 is sorted before 01. If you want to make sure the dates are always in order, use for example time definition %T+ (2012-11-29+21:07:41.0000000000) or %Tu/%Tm/%Td %TH:%TM (12/11/29 21:07)
What about inotify-tools
https://github.com/rvoicilas/inotify-tools/wiki#wiki-getting
http://linux.die.net/man/1/inotifywait
inotifywait example 2
#!/bin/sh
EVENT=$(inotifywait --format '%e' ~/file1)
[ $? != 0 ] && exit
[ "$EVENT" = "MODIFY" ] && echo 'file modified!'
[ "$EVENT" = "DELETE_SELF" ] && echo 'file deleted!'
# etc...