I am new in the world of Linux:please, accept my apology for this question:
I have written a script which check the size of a folder and ALL of its subfolder and if the size is greater than X bytes then files which have been modified more than X days ago will be deleted.
I have put folder size zero for test purpose, therefore, the script should have perform the deletion process, but, it does NOT.
This script does do what I expect and I do not know why.
Thanks for your help.
Script is:
#!/bin/sh
# 100GB SIZE LIMIT
SIZE="0"
# check the current size
CHECK="`du /media/nssvolumes/TEST/MB/`"
if ["$CHECK" -gt "$SIZE"]; then echo "$ACTION"
ACTION="`find /media/nssvolumes/TEST/MB/ -mindepth 0 -maxdepth 3 -mtime +1 -type f -exec rm -f {} \;`"
else exit
fi
Well, here we have a bunch of errors.
Supposing your OS is Linux, try running du from the command line to realise what it returns, because it is NOT a number.
Second problem - using of [] requires spaces.
Also, integers does not require ""
You're printing $ACTION before its value is defined.
exit is not needed to be called if script has no more code to execute
So the fixed script is:
#!/bin/sh
# 100GB SIZE LIMIT
SIZE=0
MY_DIR="/media/nssvolumes/TEST/MB/"
# check the current size
CHECK=$(du -bs $MY_DIR|awk '{print $1}')
if [ $CHECK -gt $SIZE ]; then
echo "ACTION"
find $MY_DIR -mindepth 0 -maxdepth 3 -mtime +1 -type f -exec rm {} \;
fi
Related
I'm working on a bash code which would enter the command df-h, loop through the disk usage fields and print the number using awk and if any of the disk usage reaches over 85 percent, it would find the files older than 3 days within the log path indicated in variable and remove them.
However upon trying to run the script, it constantly complains that the command was not found on line 6.
This is the code that I'm working on
files =$(find /files/logs -type f -mtime +3 -name '*.log)
process =$(df-h | awk '{print $5+0}')
for i in process
do
if $i -ge 85
then
for k in $files
do
rm -rf $k
done
fi
done;
Its so irritating because I feel that I'm so close to the solution and yet I still cant figure out as to whats wrong with the script that it refuses to work
you are searching files in /files/logs so you are probably only interested in this partition (root partition?)
your if statement did not respect the proper syntax... should be if [[ x -gt y ]]; then....
there was no need to loop thhrough the files collected by find since you can use -exec directly in find (see man find)
#!/bin/bash
# find the partition that contains the log files (this example with root partition)
PARTITION="$(mount | grep "on / type" | awk '{print $1}')"
# find the percentage used of the partition
PARTITION_USAGE_PERCENT="$(df -h | grep "$PARTITION" | awk '{print $5}' | sed s/%//g)"
# if partition too full...
if [[ "$PARTITION_USAGE_PERCENT" -gt 85 ]]; then
# run command to delete extra files
printf "%s%s%s\n" "Disk usage is at " "$PARTITION_USAGE_PERCENT" "% deleting log files..."
find /files/logs -type f -mtime +3 -name '*.log' -exec rm {} \;
fi
I have a remote server that copies 30-some backup files to a local server every day and I want to remove the old backups if and only if a newer backup successfully copied.
With different codes I tried, I managed to erase older files, but I got the problem that if it found one new backup, it deleted ALL older ones.
I have something like (picture this with 20 virtual machines):
vm001-2019-08-01.bck
vm001-2019-07-28.bck
vm002-2019-08-01.bck
vm003-2019-07-29.bck
vm004-2019-08-01.bck
vm004-2019-07-31.bck
vm004-2019-07-30.bck
vm004-2019-07-29.bck
...
And I'd want to erase all but keep only the most recent ones.
i.e.: erase:
vm001-2019-07-28.bck
vm002-2019-07-29.bck
vm004-2019-07-31.bck
vm004-2019-07-30.bck
vm004-2019-07-29.bck
and keep only:
vm001-2019-08-01.bck
vm002-2019-08-01.bck
vm003-2019-07-29.bck
vm004-2019-08-01.bck
the problem I had is that if I have any recent backup of any machine, files like vm-003-2019-07-29 get deleted, because they are older, even if they are of different machines.
I know there are several variants of this question in the site, but I can't quite get this to work.
I've been trying variants of this code:
#!/bin/bash
for i in ./*.bck
do
echo "found" "$i"
if [[ -n $(find "$i" -type f -mmin -1440) ]]
then
echo "$i"
find "$i" -type f -mmin +1440 -exec rm -f "$i" {} +
fi
done
(The echos are for debugging purposes only)
At this time, this code finds the newer and the older files, but doesn't delete anything. If I put find "$i" -type f -mmin +1440 -exec echo "$i" {} +, it never prints anything, as if find $i is not finding anything, but when I run it as a solo command in the terminal, it does (minus the -exec part).
I've tested this script generating files with different timestamps using touch -d, but I had no success.
Unless you add the -name test before the filename find is going to consider "$i" to be the name of a directory to search in. So your find command should be:
find -name "$i" -type f -mmin -1440
which will search in the current directory. Or
find /path/to/dir -name "$i" -type f -mmin -1440
which will search in a directory named "/path/to/dir".
But, based on BashFAQ/099, I would do this to delete all but the newest file for each VM (untested):
#!/bin/bash
declare -A newest # associative array to store name of newest file for each VM
for f in *
do
vm=${f%%-*} # extracts vm name from filename (i.e. vmm001 from vm001-2019-08-01.bck)
if [[ -f $f && $f -nt ${newest["$vm"]} ]]
then
newest["$vm"]=$f
fi
done
for f in *
do
vm=${f%%-*}
if [[ -f $f && $f != ${newest["$vm"]} ]]
then
rm "$f"
fi
done
This is set up to run against files in the current directory. It assumes that the files are named as shown in the question (the VM name is separated from the rest of the file name by a hyphen). In order to use an associative array, Bash 4 or higher is required.
I am an Intern and was given a task of creating a BASH script to delete files in a directory older than 60 days and then exports a text file containing the number of files deleted as well as the amount of data removed. I am still attempting to learn BASH and have a one liner to remove files older than 30 days;
`find $DIR -type f -mtime -60 -exec rm -rf {}\;`
I am still actively attempting to learn BASH, so extra notes on any responses would be greatly appreciated!
P.S. I found Bash Academy , but looks like the site is incomplete, any recommendations for further reading in my quest to learn bash will also be greatly appreciated!
I would use the below script, say deleter.sh for the purpose :
#!/bin/bash
myfunc()
{
local totalsize=0
echo " Removing files listed below "
echo "${#}"
sizes=( $(stat --format=%s "${#}") ) #storing sizes in an array.
for i in "${sizes[#]}"
do
(( totalsize += i )) #calculating total size.
done
echo "Total space to be freed : $totalsize bytes"
rm "${#}"
if [ $? -eq 0 ] #$? is the return value
then
echo "All files deleted"
else
echo "Some files couldn't be deleted"
fi
}
export -f myfunc
find "$1" -type f -not -name "*deleter.sh" -mtime +60\
-exec bash -c 'myfunc "$#"' _ {} +
# -not -name "*deleter.sh" to prevent self deletion
# Note -mtime +60 for files older than 60 days.
Do
chmod +x ./deleter.sh
And run it as
./deleter '/path/to/your/directory'
References
Find [ manpage ] for more info.
stat --format=%s gives size in bytes which we store in an array. See [ stat ] manpage.
feedback appreciated
I have folders containing large numbers of files (e.g. 1000+) of various sizes which I want to move in to smaller groups of, say, 100 files per folder.
I wrote an Apple Script which counted the files, created a numbered subfolder, and then moved 100 files in to the new folder (the number of files could be specified) which looped until there were less than specified number of files which it moved in to the last folder it created.
The problem was that it ran horrendously slowly. I'm looking for either an Apple Script or shell script I can run on my MacBook and/or Linux box which will efficiently move the files in to smaller groups.
How the files are grouped is not particularly significant, I just want fewer files in each folder.
This should get you started:
DIR=$1
BATCH_SIZE=$2
SUBFOLDER_NAME=$3
COUNTER=1
while [ `find $DIR -maxdepth 1 -type f| wc -l` -gt $BATCH_SIZE ] ; do
NEW_DIR=$DIR/${SUBFOLDER_NAME}${COUNTER}
mkdir $NEW_DIR
find $DIR -maxdepth 1 -type f | head -n $BATCH_SIZE | xargs -I {} mv {} $NEW_DIR
let COUNTER++
if [ `find $DIR -maxdepth 1 -type f| wc -l` -le $BATCH_SIZE ] ; then
mkdir $NEW_DIR
find $DIR -maxdepth 1 -type f | head -n $BATCH_SIZE | xargs -I {} mv {} $NEW_DIR
fi
done
The nested if statement gets the last remaining files. You can add some additional checks as you see needed after you modify for your use.
This is a tremendous kludge, but it shouldn't be too terribly slow:
rm /tmp/counter*
touch /tmp/counter1
find /source/dir -type f -print0 |
xargs -0 -n 100 \
sh -c 'n=$(echo /tmp/counter*); \
n=${n#/tmp/counter}; \
counter="/tmp/counter$n"; \
mv "$counter" "/tmp/counter$((n+1))"; \
mkdir "/dest/dir/$n"; \
mv "$#" "/dest/dir/$n"' _
It's completely indiscriminate as to which files go where.
The most common way to solve the problem of directories with too many files in them is to subdivide by the the first couple characters of the name. For example:
Before:
aardvark
apple
architect
...
zebra
zork
After:
a/aardvark
a/apple
a/architect
b/...
...
z/zebra
z/zork
If that isn't subdividing well enough, then go one step further:
a/aa/aardvark
a/ap/apple
a/ar/architect
...
z/ze/zebra
z/zo/zork
This should work quite quickly, because the move command that your script executes can use simple glob expansion to select all the files to move, ala mv aa* a/aa, as opposed to having to individually run a move command on each file (which would be my first guess as to why the original script was slow)
i'm trying to write a shell script, for linux and solaris, that finds some specific files older than X days and then deletes them. the trick is that during this process there are a couple of files that must not be deleted.
for example from the following list of files i need to delete *.zip and keep *.log and *.something.*
1.zip
2.zip
3.log
prefix.something.suffix
finding the files and feeding them to rm was easy, but i'm having difficulties in excluding the files from the deletion list.
experimenting around i discovered one can benefit from multiple complex expressions grouped with logical operators like this:
find -L path -type f \( -name '*.log' \) -a ! \( -name '*.zip' -o -name '*something*' \) -mtime +3
cheers,
G
or you could do this:
find /appl/ftp -type f -mtime +30 |grep -vf [exclude_file] | xargs rm -rf;
I needed to find a way to provide a hard coded list of exclude files to not remove, but remove everything else that was older than 30 days. Here is a little script to perform a remove of all files older that 30 days, except files that are listed in the [exclude_file].
EXCL_FILES=`/bin/cat [exclude_file]`;
RM_FILE=`/usr/bin/find [path] -type f -mtime +30`;
for I in $RM_FILES;
do
for J in $EXCL_FILES;
do
grep $J $I;
if [[ $? == 0 ]]; then
/bin/rm $I;
if [[ $? != 0 ]]; then echo "PROBLEM: Could not remove $I"; exit 1; fi;
fi;
done;
done;