I have a script that scans all the directories and subdirectories for those with "RC" in their name and delete all older then 40 days but always leave the last one even if it is older than 40 days.
The problem I am heaving is that if I run the script by hand ./cronJob.sh it works as it should. But when I put it on a crontab list it does not delete directories, but only outputs two lines in log.
#!/bin/bash
datum=$(date -I)
MOUNTLOG=/var/log/softwareRC/
FIND=/bin/find;
deleteDir(){
echo "-------- START $parent --------" >> $MOUNTLOG/$datum.log
dname=$(/usr/bin/dirname $1)
temp="${dname%\s.*}"
temp=(${temp[#]})
parent="${temp[0]}"
dirNum="$($FIND $parent -maxdepth 1 -name *RC* -type d -print | wc -l)"
najnovejsi="$($FIND $parent -maxdepth 1 -name *RC* -type d -print | sort | tail -n 1)"
if [ $dirNum -gt 1 ]; then
$FIND "$parent" -path "$najnovejsi" -prune -o -name *RC* -mtime +40 -print -exec rm -r "{}" \; >> $MOUNTLOG/$datum.log
fi;
echo "-------- END $parent --------" >> $MOUNTLOG/$datum.log
}
declare -i skipDir=1
while true
do
oldest=$($FIND -type d -name *RC* -mtime +40 -printf '%T+ %p\n' | sort -r | tail -n $skipDir | head -n 1)
# echo najstarejsi $oldest
dironly=$(echo $oldest | cut -d' ' -f 2-)
deleteDir "$dironly"
# echo $skipDir $dironly
/bin/sleep 1
if [ "$dironly" = "$testna" ]; then
break
else
testna=$(echo $oldest | cut -d' ' -f 2-)
let "skipDir++"
fi;
# echo primerjava $testna
done
Crontab job
0 2 * * * /mnt/local/TempDrive/Software_RC/.cleanOld.sh
Log output
[root#SambaServer softwareRC]# cat 2017-03-11.log
-------- START --------
-------- END --------
Add this line to your script:
#!/bin/bash
exec > $MOUNTLOG/$datum.log 2>&1
datum=$(date -I)
If there is an error message from the shell or one of the executed commands, it will show up in the log file.
0 2 * * * sh /mnt/local/TempDrive/Software_RC/cleanOld.sh
And check file permission and owner of the file
Thank you very much for your help and sorry for late reply. I have figured out what was wrong.
The problem was in the below line. I had to enter the whole path to the location where the script is ran from.
Before:
oldest=$($FIND -type d -name *RC* -mtime +40 -printf '%T+ %p\n' | sort -r | tail -n $skipDir | head -n 1)
After:
oldest=$($FIND /mnt/local/TempDrive/Software_RC -type d -name *RC* -mtime +40 -printf '%T+ %p\n' | sort -r | tail -n $skipDir | head -n 1)
This is the working code.
#!/bin/bash
datum=$(date -I)
MOUNTLOG=/var/log/softwareRC/
exec > $MOUNTLOG/$datum.log 2>&1
FIND=/bin/find;
deleteDir(){
echo "-------- START $parent --------" >> $MOUNTLOG/$datum.log
dname=$(/usr/bin/dirname $1)
temp="${dname%\s.*}"
temp=(${temp[#]})
parent="${temp[0]}"
dirNum="$($FIND $parent -maxdepth 1 -name *RC* -type d -print | wc -l)"
najnovejsi="$($FIND $parent -maxdepth 1 -name *RC* -type d -print | sort | tail -n 1)"
if [ $dirNum -gt 1 ]; then
$FIND "$parent" -path "$najnovejsi" -prune -o -name *RC* -mtime +40 -print -exec rm -r "{}" \; >> $MOUNTLOG/$datum.log
fi;
echo "-------- END $parent --------" >> $MOUNTLOG/$datum.log
}
declare -i skipDir=1
while true
do
oldest=$($FIND /mnt/local/TempDrive/Software_RC -type d -name *RC* -mtime +40 -printf '%T+ %p\n' | sort -r | tail -n $skipDir | head -n 1)
dironly=$(echo $oldest | cut -d' ' -f 2-)
deleteDir "$dironly"
/bin/sleep 1
if [ "$dironly" = "$testna" ]; then
break
else
testna=$(echo $oldest | cut -d' ' -f 2-)
let "skipDir++"
fi;
done
Related
I was wondering if there was a single command which would recursively find the directories in which their newest file in older than 3 days. Other solutions seem to only print the newest file in all subdirectories, I was wondering if there was a way to do it recursively and print all the subdirectories? I tried find -newermt "aug 27, 2022" -ls but this only gets me directories that have files younger than the date specified, not the youngest for each directory.
A long one-liner to sort files by date, get uniq directory names, list by modification keeping first
find ~/.config -type f -newermt "aug 29, 2022" -print0 | xargs -r0 ls -l --time-style=+%s | sort -r -k 6 | gawk '{ print $7}' | xargs -I {} dirname {} | sort | uniq | xargs -I {} bash -c "ls -lt --time-style=full-iso {} | head -n2" | grep -v 'total '
With comments
find ~/.config -type f -newermt "aug 29, 2022" -print0 |\
xargs -r0 ls -l --time-style=+%s | sort -r -k 6 |\ # newer files sorted by reverse date
gawk '{ print $7}' | xargs -I {} dirname {} |\ # get directory names
sort | uniq | \ # get uniq directory names
xargs -I {} bash -c "ls -lt --time-style=full-iso {} | head -n2" |\# list each directory by time, keep first
grep -v 'total '
If I'm understanding the requirements correctly, would you please try:
#!/bin/bash
find dir -type d -print0 | while IFS= read -r -d "" d; do # traverse "dir" recursively for subdirectories assigning "$d" to each directory name
if [[ -n $(find "$d" -maxdepth 1 -type f) \
&& -z $(find "$d" -maxdepth 1 -type f -mtime -3) ]]; then # if "$d" contains file(s) and does not contain files newer than 3 days
echo "$d" # then print the directory name "$d"
fi
done
A one liner version:
find dir -type d -print0 | while IFS= read -r -d "" d; do if [[ -n $(find "$d" -maxdepth 1 -type f) && -z $(find "$d" -maxdepth 1 -type f -mtime -3) ]]; then echo "$d"; fi; done
Please modify the top directory name dir according to your file location.
I don't know if I'm wording it correctly, but I'm counting file types and outputting the results into a file, and instead of there just being numbers, I'm trying to identify what each number is. Sooo basically right now I have:
$ find . -type f -iname *.jpg* | wc -l > Test.md
$ find . -type f -iname *.png* | wc -l >> Test.md
$ find . -type f -iname *.tiff* | wc -l >> Test.md
and when I cat Test.md I get:
$ cat Test.md
13
10
8
and what I'm trying to do is:
JPG: 13
PNG: 10
TIFF: 8
So just add the string without a newline before the count.
: > Test.md # truncate the file
echo -n "JPG: " >> Test.md
find . -type f -iname '*.jpg*' | wc -l >> Test.md
echo -n "PNG: " >> Test.md
find . -type f -iname '*.png*' | wc -l >> Test.md
echo -n "TIFF: " >> Test.md
find . -type f -iname '*.tiff*' | wc -l >> Test.md
or like, grab the output of wc with command substitution, and pass to echo to do some formatting:
echo "JPG: $(find . -type f -iname '*.jpg*' | wc -l)" > Test.md
echo "PNG: $(find . -type f -iname '*.png*' | wc -l)" >> Test.md
echo "TIFF: $(find . -type f -iname '*.tiff*' | wc -l)" >> Test.md
Note: quote the *.jpg* argument for find inside single (or double) quotes to prevent filename expansion on the argument. find needs the argument with *, not literal filenames after the shell expansion.
What I would do using a here-doc :
cat<<EOF>Test.md
JPG: $(find . -type f -iname '*.jpg*' | wc -l)
PNG: $(find . -type f -iname '*.png*' | wc -l)
TIFF: $(find . -type f -iname '*.tiff*' | wc -l)
EOF
I also need the directory name to be outputs as well. What I was able to do is to output the total number of lines in all directories with directory name.
find . -name '*.c' | xargs wc -l | xargs -I{} dirname {} | xargs -I{} dirname {}
I have jumbled up a mixture of bash commands mostly GNU-specific, make sure you have them, GNU grep and GNU Awk
find . -type f -print0 | xargs -0 grep -c ';$' | \
awk -F":" '$NF>0{cmd="dirname "$1; while ( ( cmd | getline result ) > 0 ) {printf "%s\t%s\n",result,$2} close(cmd) }'
The idea is grep -c returns the pattern count in format, file-name:count, which I am passing it to GNU Awk to filter those files whose count is greater than zero and print the directory of the file containing it and the count itself.
As a fancy one-liner as they call it these days,
find . -type f -print0 | xargs -0 grep -c ';$' | awk -F":" '$NF>0{cmd="dirname "$1; while ( ( cmd | getline result ) > 0 ) {printf "%s\t%s\n",result,$2} close(cmd) }'
Here is a script:
#!/usr/bin/env bash
for dir in */; do (
cd "$dir"
count=$(find . -name '*.c' -print0 | xargs -0 grep '[;]$' | wc -l)
echo -e "${count}\t${dir}"
) done
If you want numbers for each sub-directory:
#!/usr/bin/env bash
for dir in $(find . -type d); do (
cd "$dir"
count=$(find . -maxdepth 1 -name '*.c' -print0 | \
xargs -0 grep '[;]$' | wc -l)
echo -e "${count}\t${dir}"
) done
Using -maxdepth 1 makes sure the calculation is only done in the current directory, not its sub-directories. So each file is counted once.
I have two commands which both return numbers.
For example:
cat `find -name \*.cd -print` | wc -l
cat `find -name \*.c -print` | wc -l
Let's say that the first one returns 10, the other 5.
What would the command which return the sum of them, without changing these commands, look like?
I need something like this:
cat `find -name \*.cd -print` | wc -l + cat `find -name \*.c -print` | wc -l
and it should return 15 in this case.
How I can do that?
That command will execute yours two commands and print the sum of the results.
echo $(($(cat `find -name \*.cd -print` | wc -l) + $(cat `find -name \*.c -print` | wc -l)))
EDIT:
As #Karoly Horvath commented it would be more readable if it's not a oneliner:
cd_count=$(cat `find -name \*.cd -print` | wc -l)
c_count=$(cat `find -name \*.c -print` | wc -l)
echo $(($cd_count + $c_count))
It's better to combine the two searches:
cat $(find -regex '.*\.cd?$') | wc -l
or
find -regex '.*\.cd?$' | xargs cat | wc -l
or, if you filenames can contain spaces:
find -regex '.*\.cd?$' -print0 | xargs -0 cat | wc -l
$ expr $(echo 5) + $(echo 10)
15
Just replace the echo statements with your commands.
I am trying to list all directories and place its number of files next to it.
I can find the total number of files ls -lR | grep .*.mp3 | wc -l. But how can I get an output like this:
dir1 34
dir2 15
dir3 2
...
I don't mind writing to a text file or CSV to get this information if its not possible to get it on screen.
Thank you all for any help on this.
This seems to work assuming you are in a directory where some subdirectories may contain mp3 files. It omits the top level directory. It will list the directories in order by largest number of contained mp3 files.
find . -mindepth 2 -name \*.mp3 -print0| xargs -0 -n 1 dirname | sort | uniq -c | sort -r | awk '{print $2 "," $1}'
I updated this with print0 to handle filenames with spaces and other tricky characters and to print output suitable for CSV.
find . -type f -iname '*.mp3' -printf "%h\n" | uniq -c
Or, if order (dir-> count instead of count-> dir) is really important to you:
find . -type f -iname '*.mp3' -printf "%h\n" | uniq -c | awk '{print $2" "$1}'
There's probably much better ways, but this seems to work.
Put this in a shell script:
#!/bin/sh
for f in *
do
if [ -d "$f" ]
then
cd "$f"
c=`ls -l *.mp3 2>/dev/null | wc -l`
if test $c -gt 0
then
echo "$f $c"
fi
cd ..
fi
done
With Perl:
perl -MFile::Find -le'
find {
wanted => sub {
return unless /\.mp3$/i;
++$_{$File::Find::dir};
}
}, ".";
print "$_,$_{$_}" for
sort {
$_{$b} <=> $_{$a}
} keys %_;
'
Here's yet another way to even handle file names containing unusual (but legal) characters, such as newlines, ...:
# count .mp3 files (using GNU find)
find . -xdev -type f -iname "*.mp3" -print0 | tr -dc '\0' | wc -c
# list directories with number of .mp3 files
find "$(pwd -P)" -xdev -depth -type d -exec bash -c '
for ((i=1; i<=$#; i++ )); do
d="${#:i:1}"
mp3s="$(find "${d}" -xdev -type f -iname "*.mp3" -print0 | tr -dc "${0}" | wc -c )"
[[ $mp3s -gt 0 ]] && printf "%s\n" "${d}, ${mp3s// /}"
done
' "'\\0'" '{}' +