I am trying to use a txt file to store folder locations to use in find command. But keep getting folder not found works with only one folder location in file
with "$addfolder"
found=$(find "$addfolder" ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
and replacing \"
addfolder="$addfolder $Folder"
folder.txt :-
Main/Public
Main/General
Not Used
Old Backup Files
#!/bin/bash
addfolder=""
filename="Settings/folders.txt"
#Read Folder.txt for locations
while read -r Folder; do
if [ ! "$Folder" == "" ];then
if [ -d "$Folder" ]; then
addfolder="$addfolder \"$Folder\""
echo "$addfolder"
fi
fi
done < "$filename"
if [ "$addfolder" == "" ]; then
exit
fi
echo -e "\e[36mEnter Filename To Find :-\e[0m"
read -p "" ffind
echo -e "\e[92mSearching:\e[0m"
found=$(find $addfolder ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
echo -e "\e[33m$found\e[0m"
echo "Press Enter To Exit"
read -s -n 1 -p ""
Regular variables should only hold single strings.
To hold lists of strings, use an array:
#!/bin/bash
addfolder=()
filename="Settings/folders.txt"
#Read Folder.txt for locations
while IFS= read -r Folder; do
if [ ! "$Folder" == "" ];then
if [ -d "$Folder" ]; then
addfolder+=( "$Folder" )
echo "${addfolder[#]}"
fi
fi
done < "$filename"
if [ "${#addfolder[#]}" == 0 ]; then
exit
fi
echo -e "\e[36mEnter Filename To Find :-\e[0m"
read -p "" ffind
echo -e "\e[92mSearching:\e[0m"
found=$(find "${addfolder[#]}" ! -path "*/.bak/*" -type f -iname "*$ffind*" | sort)
echo -e "\e[33m$found\e[0m"
echo "Press Enter To Exit"
read -s -n 1 -p ""
Related
I have a recursive multiple folder structures. Inside of there are multiple zip, img files. But when I extracted it, new zip,img files come out again. I want to write a loop for this.
Scan whole recursive folders, find multiple zip files and extract them. After extraction scan folders again if system found new zip files, extract it again and again continue until no left zip file in the folders.
sample.tar.gz
1.img
Thefolder
samplefolder
t.img
2.img
samplefolder2
different.gz
3.img
4.img
There are more folders also under other img files.
I tried to write for loop for this but couldn't work it. After extraction 1.img script getting error.
Error Output:
ID = 277952352
Everything is Ok
Folders: 3
Files: 1
Size: 345424152
Compressed: 671024244
+ rm -f /home/sample/1.img
+ recursiveExtract /home/sample/Thefolder
+ for file in "$path"/*
+ '[' -d /home/sample/Thefolder ']'
+ recursiveExtract /home/sample/Thefolder
Segmentation fault
The code:
#!/bin/bash
set -x
target_file=$1
path="$(realpath "$1")"
recursiveExtract () { # $1=directory
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'img' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
b=$(ls $path | grep img | wc -l)
if [[ "$b" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tar' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.tar}"
fi
c=$(ls $path | grep tar | wc -l)
if [[ "$c" -eq 0 ]]; then
break
fi
done
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
7z x $file -o$path -r # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
d=$(ls $path | grep gz | wc -l)
if [[ "$d" -eq 0 ]]; then
break
fi
done
}
recursiveExtract "$1"
Latest version of my code:
#!/bin/bash
# set -x
recursiveExtract () { # $1=directory
path="$(realpath "$1")"
echo "GZ------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'gz' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.gz}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.gz}"
fi
# d=$(ls $path | grep gz | wc -l)
d=$(find $path -type f -name "*.gz" | wc -l)
echo "WC GZ-----------------:" $d
if [[ "$d" -eq 0 ]]; then
break
fi
done
echo "IMG------------------"
for file in "$path"/*; do
if [ -f "$file" -a "${file##*.}" = 'img' ]; then
echo "File:" $file
echo "Path:" $path
# 7z e "${file%.img}" -o"$file" # variation 1
7z x $file -o$path -r -aou # variation 2
rm -f "$file" # comment this if you want to keep the zip files.
recursiveExtract "${file%.img}"
fi
# b=$(ls $path | grep img | wc -l)
b=$(find $path -type f -name "*.img" | wc -l)
echo "WC IMG-----------------:" $b
if [[ "$b" -eq 0 ]]; then
break
fi
done
}
while true
do
d=$(find $1 -type f -name "*.gz" | wc -l)
b=$(find $1 -type f -name "*.img" | wc -l)
if [[ "$d" -eq 0 ]] && [[ "$b" -eq 0 ]]; then
break
else
recursiveExtract "$1"
fi
done
This shoud do the job
#!/bin/bash
find ./ -name "*.zip*" > current_zips.txt
while [[ `wc -l "current_zips.txt" | cut -d' ' -f1` > 0 ]]; do
find ./ -name "*.zip*" -exec unzip {} \;
while IFS= read file_; do rm $file_; done < "current_zips.txt" # for file_ in $(cat current_zips.txt);do rm $file_;done
find ./ -name "*.zip*" > current_zips.txt
done
Use the following script to generate test data
#!/bin/bash
#create some data
echo "data 0" > file0.txt
mkdir folder1 folder2
echo "data 1" > folder1/file1.txt
echo "data 2" > folder2/file1.txt
#zip data
zip file0.zip file0.txt
zip -r folder1.zip folder1
zip -r folder2.zip folder2
zip -r data.zip *.zip
#delete original data
rm -rf file* folder*
I think you want something like this:
recursiveExtract.bash
#!/bin/bash
# efi,gz,img,tar
recursiveExtract () { # $1=file/directory
local path="$(realpath "$1")" # This variable must be local.
for file in "$path"/*; do
if [ -d "$file" ]; then
recursiveExtract "$file"
elif [ -f "$file" -a "${file##*.}" = 'tgz' ]; then # file.tgz == file.tar.gz
local anotherFile="${file%.*}"'.tar'
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
#######################
# -aoa == Overwrite output files #
# -bd == Don't show progress bar #
# -bso0 == Silent #
#######################
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
elif [ -f "$file" -a "${file##*.}" = 'gz' ]; then
local anotherFile="${file%.*}"
if [ -n "$anotherFile" -a \( "${anotherFile##*.}" = 'tar' -o "${anotherFile##*.}" = 'img' \) ]; then # file.tar.gz, file.img.gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%/*}" -aoa -bd -bso0
7z x "$anotherFile" -o"${anotherFile%.*}" -aoa -bd -bso0
rm -f "$anotherFile" # DO NOT COMMENT THIS LINE.
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${anotherFile%.*}"
else # gz
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "{file%.*}"
fi
elif [ -f "$file" -a \( "${file##*.}" = 'img' -o "${file##*.}" = 'tar' \) ]; then # file.img or file.tar
echo 'Extracting: '"$file"
7z x "$file" -o"${file%.*}" -aoa -bd -bso0
rm -f "$file" # Comment this if you want to keep the archieves.
recursiveExtract "${file%.*}"
fi
done
}
recursiveExtract "$1"
Give the script executable permission with chmod +x recursiveExtract.bash then you can run it like:
$ ./recursiveExtract.bash <directory/file>
In your case, maybe like this:
$ ./recursiveExtract.bash 'sample.img.gz'
I have created a script to zip and move log files from one directory to another directory to free space. This is the script:
#!/bin/bash
logsDirectory="/test//logs/"
email=""
backupDirectory="/test/backup"
pid="/data/test/scripts/backup.pid"
usage=$(df | grep /data/logs | awk '{ print $2 }')
space=450000000
getBackup ()
{
if [[ ! -e $pid ]] then
if [[ $usage -le $space ]]
then
touch $pid
find $backupDirectory -mtime +15 -type f -delete;
for i in $(find $logsDirectory -type f -not -path "*/irws/*")
do
/sbin/fuser $i > /dev/null 2>&1
if [ $? -ne 0 ]
then
gzip $i
mv -v $i.gz $backupDirectory
else
continue
fi
done
[[ ! -z $email ]] && echo "Backup is ready" | mas"Backup" $email
rm -f $pid
fi
fi
}
getBackup
I am getting this error:
gzip: /data/logs/log01.log.gz already has .gz suffix -- unchanged
mv: cannot stat `/data/logs/log01.log.gz': No such file or directory
I got the error every time I ran the script in my DEV and PROD (CentOS servers) environments. To analyse it, I ran the same script in a VM (Ubuntu) in my laptop, and I don't get the error there.
My questions:
How can I prevent this error?
What I have done wrong in the script?
Your script contains a number of common clumsy or inefficient antipatterns. Here is a refactoring. The only real change is skipping any *.gz files.
#!/bin/bash
logsDirectory="/test//logs/"
email=""
backupDirectory="/test/backup"
pid="/data/test/scripts/backup.pid"
# Avoid useless use of grep -- awk knows how to match a regex
# Better still run df /data/logs
usage=$(df /data/logs/ | awk '{ print $2 }')
space=450000000
getBackup ()
{
# Quote variables
if [[ ! -e "$pid" ]]; then
if [[ "$usage" -le "$space" ]]; then
touch "$pid"
find "$backupDirectory" -mtime +15 -type f -delete;
# Exclude *.gz files
# This is still not robust against file names with spaces or wildcards in their names
for i in $(find "$logsDirectory" -type f -not -path "*/irws/*" -not -name '*.gz')
do
# Avoid useless use of $?
if /sbin/fuser "$i" > /dev/null 2>&1
then
gzip "$i"
mv -v "$i.gz" "$backupDirectory"
# no need for do-nothing else
fi
done
[[ ! -z "$email" ]] &&
echo "Backup is ready" | mas"Backup" "$email"
rm -f "$pid"
fi
fi
}
getBackup
With a slightly more intrusive refactoring, the proper fix to the find loop would perhaps look something like
find "$logsDirectory" -type f \
-not -path "*/irws/*" -not -name '*.gz' \
-exec sh -c '
for i; do
if /sbin/fuser "$i" > /dev/null 2>&1
then
gzip "$i"
mv -v "$i.gz" "$backupDirectory"
fi
done' _ {} +
where the secret sauce is to have find ... -exec + pass in the arguments to the sh -c script in a way which does not involve exposing the arguments to the current shell at all.
What I have done wrong in the script?
Your script tries to zip every file but the gzip command is rejecting files already zipped
How can I prevent this error?
Have the script check whether the file is zipped or not and only gzip if it corresponds (1). Alternatively, you could force re-compression even if it is already compressed (2).
Going with option number 1):
getBackup ()
{
if [[ ! -e $pid ]] then
if [[ $usage -le $space ]]
then
touch $pid
find $backupDirectory -mtime +15 -type f -delete;
for i in $(find $logsDirectory -type f -not -path "*/irws/*")
do
/sbin/fuser $i > /dev/null 2>&1
if [ $? -ne 0 ]
then
if [[ $i =~ \.gz$ ]]
# File is already zipped
mv -v $i $backupDirectory
else
gzip $i
mv -v $i.gz $backupDirectory
fi
else
continue
fi
done
[[ ! -z $email ]] && echo "Backup is ready" | mas"Backup" $email
rm -f $pid
fi
fi
}
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I keep getting these errors running my script and i just cannot work it out...
the error that keeps coming up is;
rm: cannot remove ~/my-documents/article:': Is a directory. The directory its referring to is $2...here is my script.
#! /bin/sh
SRC=$1
DES=$2
if [ $# -ne 2 ]; then
echo "1. Please enter the source directory"
echo "2. Please enter the destination directory"
echo "thankyou"
exit
fi
if [ ! -d $1 ]; then
echo "$1 is not a directory please enter a valid directory"
echo "thankyou"
exit
fi
#gives the user a error warning the source directory is invalid
if [ -d $2 ]; then
echo "output directory exists"
else
echo "Output directory does not exist, creating directory"
mkdir $2
fi
#creates the destination directory if one doesn't exist
IFILE=$GETFILES;
FINDFILE=$FINDFILE;
find $1 -name "*.doc" > FINDFILE
find $1 -name "*.pdf" > FINDFILE
find $1 -name "*.PDF" > FINDFILE
#finds doc, pdf & PDF files and sends data to findfile.
while read -r line;
do
cp $line $2
done < FINDFILE
#files read and copied to destination directory
IFILE=$2/$GETFILES;
ls -R $1 | egrep -i ".doc | .pdf" > IFILE;
LCOUNT=0
DIFFCOUNT=0
FOUND=0
ARCHIVE=1
BASE="${line%.*}"
EXTENSION="${line##*.}"
COUNT=$COUNT;
ls $2 | grep ${line%%.*} \; | wc -l
if [[ $COUNT -eq 0 ]];
then
cp $1/$line $2;
else
echo "there is already a file in the output so need to compare"
COMP=$2/$line
fi
while [[ $FOUND -eq 0 ]] && [[ $LCOUNT -lt $COUNT ]];
do
echo "diffcount is $DIFFCOUNT"
###compares the file from the input directory to the file in
###the output directory
if [ $DIFFCOUNT -eq 0 ];
then
echo "file has already been archived no action required"
FOUND=$FOUND [ $FOUND+1 ]
else
LCOUNT=$LCOUNT [ $LCOUNT+1 ]
COMP="OUT"/"$BASE"_"$LCOUNT"."$EXTENSION"
echo "line count for next compare is $LCOUNT"
echo "get the next file to compare"
echo "the comparison file is now $COMP"
fi
if [ $LCOUNT -ne $COUNT ]; then
ARCHIVE=$ [ $ARCHIVE+1 ]
else
ARCHIVE=0
fi
if [ $ARCHIVE -eq 0 ];
then
NEWOUT="OUT"/"$BASE"_"$LCOUNT"."$EXTENSION";
echo "newfile name is $NEWOUT"
cp $1/$LINE $NEWOUT
fi
done < $IFILE
rm $IFILE
OFILE=$2/DOCFILES;
ls $2 | grep ".doc" > $OFILE;
while read -r line;
do
BASE=${line%.*}
EXTENSION=${line##*.}
NEWEXTENSION=".pdf"
SEARCHFILE=$BASE$NEWEXTENSION
find $2 -name "$SEARCHFILE" -exec {} \;
done < $OFILE
rm $OFILE
### this will then remove any duplicate files so only
### individual .doc .pdf files will exist
a plain call to rm can only remove files, not directories.
$ touch /tmp/myfile
$ rm /tmp/myfile
$ mkdir /tmp/mydir
$ rm /tmp/mydir
rm: cannot remove ‘/tmp/mydir/’: Is a directory
You can remove directories by specifying the -d (to delete empty directories) or the -r (to delete directories and content recursively) flag:
$ mkdir /tmp/mydir
$ rm -r /tmp/mydir
$
this is well described in man rm.
apart from that, you seem to ignore quoting:
$ rm $OFILE
might break badly if the value of OFILE contains spaces, use quotes instead:
$ rm "${OFILE}"
and never parse the output of ls:
ls $2 | grep ".doc" > $OFILE
(e.g. if your "$2" is actually "/home/foo/my.doc.files/" it will put all files in this directory into $OFILE).
and then you iterate over the contents of this file?
instead, just use loop with file-globbing:
for o in "${2}"/*.doc
do
## loop code in here
done
or just do the filtering with find (and don't forget to call an executable with -exex):
find "$2" -name "$SEARCHFILE" -mindepth 1 -maxdepth 1 -type f -exec convertfile \{\} \;
I'm missing some images that should have been archived when this script runs. I think this may be to do with my indentations or my Md5 sum. I have tried everything I can think of.
here is the code with out the correct indentations:
#!/bin/sh
if [ ! -d "$1" ]; then
echo Directory "$1" cannot be found. Please try again.
exit
fi
if [ $# -eq 1 ]; then
echo "usage: Phar image_path archive_path"
exit
fi
if [ -d "$2" ]; then
echo "archive exists"
else
echo "the directory 'archive' does't exist. Creating directory 'archive'."
mkdir -p ~/archive
fi
find $1 -iname "IMG_[0-9][0-9][0-9][0-9].JPG" | cat > list.txt
[ -f ~/my-documents/md5.txt ] && rm md5.txt || break
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
sort -k 1,1 -u md5.txt | cat > uniquemd5.txt
cut -d " " -f 2- uniquemd5.txt > uniquelist.txt
sort uniquelist.txt -r -o uniquelist.txt
for line in $(cat uniquelist.txt)
do
file=$(basename $line) path="$2/file"
if [ ! -f $path ];
then
cp $line $2
else
cp $line $path.JPG
fi
rm uniquelist.txt md5.txt uniquemd5.txt list.txt
done
This loop
while read line;
do md5sum $line | xargs >> md5.txt
done < list.txt
should probably be
while read line;
do md5sum "$line"
done < list.txt > md5.txt
Quote parameter expansions, and it's unclear why you needed.
I have a different HTML files in a folder. How to rename the files so that they have the names of:
1.html
2.html
3.html
...
This can make it:
i=1
for file in /your/folder/*
do
mv $file ${i}.html
i=$((i+1)) #((i++)) was giving errors (see comments)
done
It loops through all files in /your/folder and renames them according to the number $i that keeps increasing.
Here is my script
#!/bin/sh
#
# batchrename - renames files like 01.ext, 02.ext ...
#
# format : batchrename <list of files>
# or: -r <extension> <<list of files> or <dir>>
# -r - recoursively
counter=0
extrec=""
if [ "$#" -lt "1" ]; then
echo -e "\n\t\tUsage:\n\tbatchrename [opt]\nopt:"
echo -e "-r <ext> <folder> (or file list) -- renames recoursively ALL files"
echo -e "\tin folder <folder> (or by file list given) with extension .<ext>"
echo -e "<folder> -- renames ALL files in folder given"
echo -e "<file list> -- renames ALL files of given filelist.\n\n"
exit 0
fi
Name="$*"
if [ "$1" = "-r" ]; then
extrec="$2"
shift
shift
Name="$*"
[ "$Name" = "" ] && Name="./"
fi
echo -e "\n\t\t\tRENAMING"
for file in $Name
do
file=`echo "$file" | sed "s/<>/ /g"`
if [ -d "$file" ];then
echo -e "\nDiving into \033[38m $file \033[39m"
cd "$file"
if [ "$extrec" != "" ]; then
batchrename -r $extrec `ls -1 | sed "s/\ /<>/g"`
else
batchrename `ls -1 | sed "s/\ /<>/g"`
fi
cd ../
continue
fi
ext=`ext "$file"`
if [ "$ext" = "ion" ]; then
continue
fi
if [ "$extrec" = "" -o "$ext" = "$extrec" ];then
counter=`expr $counter + 1`
echo -e "Progress: $counter files\r\c"
mv "$file" "rnmd$counter.$ext"
fi
done
echo -e "\n\n\t\t\tENDING"
digits=`echo $counter|awk '{print length ($0)}'`
cnt=1
while [ $digits -gt $cnt ]
do
f=`ls -S -1|grep "rnmd[0-9]\{$cnt\}\."`
rename rnmd rnmd0 $f
cnt=`expr $cnt + 1`
done
if [ "$counter" -gt "0" ]; then
rename rnmd "" rnmd*
fi
echo -e "\n\t\t\tDone !!!\n"
After renaming all your files will looks like 001.file, 002.file, ... and so on. Amount of leading zeros depends on amount of files. So, after renaming ls will show right order of files!
It use intermediate script ext:
#!/bin/sh
#
# ext - returns file suffix (case-unsensitive)
#
File="$*"
if [ -d "$File" ]; then
echo ""
exit 0
fi
EXT=`echo $File|sed 's/.\{1,\}\.//g'`
if [ "$EXT" = "$File" ]; then
EXT=""
fi
echo $EXT| tr '[:upper:]' '[:lower:]'
Here is a similar code, just to add to rename with the same prefix and append an incremental value
declare -i x=1
for f in $(find -type f); do
mv -v $f ${f%/*}/change_me_$x ;
x=$x+1;
done