Below is a snippet from a larger script that exports a list of the subdirectories of a user-specified directory, and prompts the user before making directories with the same names in another user-specified directory.
COPY_DIR=${1:-/}
DEST_DIR=${2}
export DIRS="`ls --hide="*.*" -m ${COPY_DIR}`"
export DIRS="`echo $DIRS | sed "s/\,//g"`"
if [ \( -z "${DIRS}" -a "${1}" != "/" \) ]; then
echo -e "Error: Invalid Input: No Subdirectories To Output\n"&&exit
elif [ -z "${DEST_DIR}" ]; then
echo "${DIRS}"&&exit
else
echo "${DIRS}"
read -p "Create these subdirectories in ${DEST_DIR}?" ANS
if [ ${ANS} = "n|no|N|No|NO|nO" ]; then
exit
elif [ ${ANS} = "y|ye|yes|Y|Ye|Yes|YE|YES|yES|yeS|yEs|YeS" ]; then
if [ ${COPYDIR} = ${DEST_DIR} ]; then
echo "Error: Invalid Target: Source and Destination are the same"&&exit
fi
cd "${DEST_DIR}"
mkdir ${DIRS}
else
exit
fi
fi
However, the command ls --hide="*.*" -m ${COPY_DIR} also prints files in the list as well. Is there any way to reword this command so that it only prints out directories? I tried ls -d, but that doesn't work, either.
Any ideas?
You should never rely on the output of ls to provide filenames. See the following for reasons not to parse ls: http://mywiki.wooledge.org/ParsingLs
You can build a list of directories safely using GNU find's -print0 option and appending the results to an array.
dirs=() # create an empty array
while read -r -d $'\0' dir; do # read up to the next \0 and store the value in "dir"
dirs+=("$dir") # append the value in "dir" to the array
done < <(find "$COPY_DIR" -type d -maxdepth 1 -mindepth 1 ! -name '*.*') # find directories that do not match *.*
The -mindepth 1 prevents find from matching the $COPY_DIR itself.
Related
I need to move files with names like source (new file).c to another directory and rename it if the file already exist there.
I tried a lot of things like
for file in $(find ~/path/ -type f -name "*.c"); do
or
IFS=$'\0' for file in $(find ~/path/ -type f -name "*.c"); do
Update.1 For rename condition i try if [ -f /this/is/the/path/${file} ]; then
or if [ -f "$file" ]; then
or if [ -f "$HOME/some/path/$file" ]; then
i want user input like read -p "some messege" msg but I can't figure it out and the if statement don't work as well and i don't know why...
fix it When I run the script, I get errors for split names. Example:
mv: cannot stat '(new': No such file or directory
Can someone help me with this?
Update.2 solution for find name with whitespaces: find ... | while IFS= read -r name; do your command done
Update.3 solution for if condition how don't work: check correct awnser
My regrets
Would you please try the following:
#!/bin/bash
path="/path/to/the/source" # original directory
dest="/path/to/the/dest" # destination directory
find "$path" -type f -name "*.c" -print0 | while IFS= read -r -d "" file; do
f=${file##*/} # extracts filename by removing everything before "/"
if [[ -f $dest/$f ]]; then
# if the file already exists
for ((;;)); do # then enter a infinite loop until proper filename is given
IFS= read -p "'$f' already exists. Input new name: " f2 < /dev/tty
if [[ ! $f2 =~ [[:alnum:]_] ]]; then
echo "'$f2' is not a valid filename."
elif [[ ! -f $dest/$f2 ]]; then
# a proper filename is input
break
else # the filename still exists
f=$f2
fi
done
fi
mv -- "$file" "$dest/$f2"
done
find ... -print0 uses a null character as a filename delimiter
to protect filenames which contain blank (whitespace, tab, newline ...)
character.
read -d "" corresponds to -print0 and split the input on the null characters.
read < /dev/tty avoids the conflict with the outermost pipeline
fed by the find command.
I need to write script in loop which will count the number of files and directories and indicates which grater and by how much. Like etc: there are 10 more files than directories.
I was trying something like that but it just show files and directories and I don't have idea how to indicates which is greater etc. Thanks for any help
shopt -s dotglob
count=0
for dir in *; do
test -d "$dir" || continue
test . = "$dir" && continue
test .. = "$dir" && continue
((count++))
done
echo $count
for -f in *; do
"$fname"
done
Here is a recursive dir walk I used for something a while back. Added counting of dirs and files:
#!/bin/sh
# recursive directory walk
loop() {
for i in *
do
if [ -d "$i" ]
then
dir=$((dir+1))
cd "$i"
loop
else
file=$((file+1))
fi
done
cd ..
}
loop
echo dirs: $dir, files: $file
Paste it to a script.sh and run with:
$ sh script.sh
dirs: 1, files: 11
You can use the find command to make things simplier.
The following command will list all the files in the given path:
find "path" -mindepth 1 -maxdepth 1 -type f
And also using the -type d you will get the directories.
Piping find into the wc -l will give you the number instead of the actual file and directory names, so:
root="${1:-.}"
files=$( find "$root" -mindepth 1 -maxdepth 1 -type f | wc -l)
dirs=$( find "$root" -mindepth 1 -maxdepth 1 -type d | wc -l)
if [ $files -gt $dirs ]; then
echo "there are $((files - dirs)) more files"
elif [ $files -lt $dirs ]; then
echo "there are $((dirs - files)) more dirs"
else
echo "there are the same"
fi
Use could use find to get the number of files/folders in a directory. Use wc -l to count the number of found paths, which you could use to calculate/show the result;
#!/bin/bash
# Path to search
search="/Users/me/Desktop"
# Get number of files
no_files=$(find "$search" -type f | wc -l )
# Number of folders
no_folders=$(find "$search" -type d | wc -l )
echo "Files: ${no_files}"
echo "Folders: ${no_folders}"
# Caculate dif
diff=$((no_files - $no_folders))
# Check if there are more folders or files
if [ "$diff" -gt 0 ]; then
echo "There are $diff more files then folders!"
else
diff=$((diff * -1 ) # Invert negative number to positive (-10 -> 10)
echo "There are $diff more folders then files!"
fi;
Files: 13
Folders: 2
There are 11 more files then folders!
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I keep getting these errors running my script and i just cannot work it out...
the error that keeps coming up is;
rm: cannot remove ~/my-documents/article:': Is a directory. The directory its referring to is $2...here is my script.
#! /bin/sh
SRC=$1
DES=$2
if [ $# -ne 2 ]; then
echo "1. Please enter the source directory"
echo "2. Please enter the destination directory"
echo "thankyou"
exit
fi
if [ ! -d $1 ]; then
echo "$1 is not a directory please enter a valid directory"
echo "thankyou"
exit
fi
#gives the user a error warning the source directory is invalid
if [ -d $2 ]; then
echo "output directory exists"
else
echo "Output directory does not exist, creating directory"
mkdir $2
fi
#creates the destination directory if one doesn't exist
IFILE=$GETFILES;
FINDFILE=$FINDFILE;
find $1 -name "*.doc" > FINDFILE
find $1 -name "*.pdf" > FINDFILE
find $1 -name "*.PDF" > FINDFILE
#finds doc, pdf & PDF files and sends data to findfile.
while read -r line;
do
cp $line $2
done < FINDFILE
#files read and copied to destination directory
IFILE=$2/$GETFILES;
ls -R $1 | egrep -i ".doc | .pdf" > IFILE;
LCOUNT=0
DIFFCOUNT=0
FOUND=0
ARCHIVE=1
BASE="${line%.*}"
EXTENSION="${line##*.}"
COUNT=$COUNT;
ls $2 | grep ${line%%.*} \; | wc -l
if [[ $COUNT -eq 0 ]];
then
cp $1/$line $2;
else
echo "there is already a file in the output so need to compare"
COMP=$2/$line
fi
while [[ $FOUND -eq 0 ]] && [[ $LCOUNT -lt $COUNT ]];
do
echo "diffcount is $DIFFCOUNT"
###compares the file from the input directory to the file in
###the output directory
if [ $DIFFCOUNT -eq 0 ];
then
echo "file has already been archived no action required"
FOUND=$FOUND [ $FOUND+1 ]
else
LCOUNT=$LCOUNT [ $LCOUNT+1 ]
COMP="OUT"/"$BASE"_"$LCOUNT"."$EXTENSION"
echo "line count for next compare is $LCOUNT"
echo "get the next file to compare"
echo "the comparison file is now $COMP"
fi
if [ $LCOUNT -ne $COUNT ]; then
ARCHIVE=$ [ $ARCHIVE+1 ]
else
ARCHIVE=0
fi
if [ $ARCHIVE -eq 0 ];
then
NEWOUT="OUT"/"$BASE"_"$LCOUNT"."$EXTENSION";
echo "newfile name is $NEWOUT"
cp $1/$LINE $NEWOUT
fi
done < $IFILE
rm $IFILE
OFILE=$2/DOCFILES;
ls $2 | grep ".doc" > $OFILE;
while read -r line;
do
BASE=${line%.*}
EXTENSION=${line##*.}
NEWEXTENSION=".pdf"
SEARCHFILE=$BASE$NEWEXTENSION
find $2 -name "$SEARCHFILE" -exec {} \;
done < $OFILE
rm $OFILE
### this will then remove any duplicate files so only
### individual .doc .pdf files will exist
a plain call to rm can only remove files, not directories.
$ touch /tmp/myfile
$ rm /tmp/myfile
$ mkdir /tmp/mydir
$ rm /tmp/mydir
rm: cannot remove ‘/tmp/mydir/’: Is a directory
You can remove directories by specifying the -d (to delete empty directories) or the -r (to delete directories and content recursively) flag:
$ mkdir /tmp/mydir
$ rm -r /tmp/mydir
$
this is well described in man rm.
apart from that, you seem to ignore quoting:
$ rm $OFILE
might break badly if the value of OFILE contains spaces, use quotes instead:
$ rm "${OFILE}"
and never parse the output of ls:
ls $2 | grep ".doc" > $OFILE
(e.g. if your "$2" is actually "/home/foo/my.doc.files/" it will put all files in this directory into $OFILE).
and then you iterate over the contents of this file?
instead, just use loop with file-globbing:
for o in "${2}"/*.doc
do
## loop code in here
done
or just do the filtering with find (and don't forget to call an executable with -exex):
find "$2" -name "$SEARCHFILE" -mindepth 1 -maxdepth 1 -type f -exec convertfile \{\} \;
I want to move all my files older than 1000 days, which are distributed over various subfolders, from /home/user/documents into /home/user/archive. The command I tried was
find /home/user/documents -type f -mtime +1000 -exec rsync -a --progress --remove-source-files {} /home/user/archive \;
The problem is, that (understandably) all files end up being moved into the single folder /home/user/archive. However, what I want is to re-construct the file tree below /home/user/documents inside /home/user/archive. I figure this should be possible by simply replacing a string with another somehow, but how? What is the command that serves this purpose?
Thank you!
I would take this route instead of rsync:
Change directories so we can deal with relative path names instead of absolute ones:
cd /home/user/documents
Run your find command and feed the output to cpio, requesting it to make hard-links (-l) to the files, creating the leading directories (-d) and preserve attributes (-m). The -print0 and -0 options use nulls as record terminators to correctly handle file names with whitespace in them. The -l option to cpio uses links instead of actually copying the files, so very little additional space is used (just what is needed for the new directories).
find . -type f -mtime +1000 -print0 | cpio -dumpl0 /home/user/archives
Re-run your find command and feed the output to xargs rm to remove the originals:
find . -type f -mtime +1000 -print0 | xargs -0 rm
Here's a script too.
#!/bin/bash
[ -n "$BASH_VERSION" ] && [[ BASH_VERSINFO -ge 4 ]] || {
echo "You need Bash version 4.0 to run this script."
exit 1
}
# SOURCE=/home/user/documents/
# DEST=/home/user/archive/
SOURCE=$1
DEST=$2
declare -i DAYSOLD=10
declare -a DIRS=()
declare -A DIRS_HASH=()
declare -a FILES=()
declare -i E=0
# Check directories.
[[ -n $SOURCE && -d $SOURCE && -n $DEST && -d $DEST ]] || {
echo "Source or destination directory may be invalid."
exit 1
}
# Format source and dest variables properly:
SOURCE=${SOURCE%/}
DEST=${DEST%/}
SOURCE_LENGTH=${#SOURCE}
# Copy directories first.
echo "Creating directories."
while read -r FILE; do
DIR=${FILE%/*}
if [[ -z ${DIRS_HASH[$DIR]} ]]; then
PARTIAL=${DIR:SOURCE_LENGTH}
if [[ -n $PARTIAL ]]; then
TARGET=${DEST}${PARTIAL}
echo "'$TARGET'"
mkdir -p "$TARGET" || (( E += $? ))
chmod --reference="$DIR" "$TARGET" || (( E += $? ))
chown --reference="$DIR" "$TARGET" || (( E += $? ))
touch --reference="$DIR" "$TARGET" || (( E += $? ))
DIRS+=("$DIR")
fi
DIRS_HASH[$DIR]=.
fi
done < <(exec find "$SOURCE" -mindepth 1 -type f -mtime +"$DAYSOLD")
# Copy files.
echo "Copying files."
while read -r FILE; do
PARTIAL=${FILE:SOURCE_LENGTH}
cp -av "$FILE" "${DEST}${PARTIAL}" || (( E += $? ))
FILES+=("$FILE")
done < <(exec find "$SOURCE" -mindepth 1 -type f -mtime +"$DAYSOLD")
# Remove old files.
if [[ E -eq 0 ]]; then
echo "Removing old files."
rm -fr "${DIRS[#]}" "${FILES[#]}"
else
echo "An error occurred during copy. Not removing old files."
exit 1
fi
Is there an easy way to find all files where no part of the path of the file is a symbolic link?
Short:
find myRootDir -type f -print
This would answer the question.
Care to not add a slash at end of specified dir ( not myRootDir/ but myRootDir ).
This won't print other than real files in real path.
No symlinked file nor file in symlinked dir.
But...
If you wanna ensure that a specified dir contain a symlink, there is a litte bash function to could do the job:
isPurePath() {
if [ -d "$1" ];then
while [ ! -L "$1" ] && [ ${#1} -gt 0 ] ;do
set -- "${1%/*}"
if [ "${1%/*}" == "$1" ] ;then
[ ! -L "$1" ] && return
set -- ''
fi
done
fi
false
}
if isPurePath /usr/share/texmf/dvips/xcolor ;then echo yes; else echo no;fi
yes
if isPurePath /usr/share/texmf/doc/pgf ;then echo yes; else echo no;fi
no
So you could Find all files where no part of the path of the file is a symbolic link in running this command:
isPurePath myRootDir && find myRootDir -type f -print
So if something is printed, there are no symlink part !
You can use this script : (copy/paste the whole code in a shell)
cat<<'EOF'>sympath
#!/bin/bash
cur="$1"
while [[ $cur ]]; do
cur="${cur%/*}"
if test -L "$cur"; then
echo >&2 "$cur is a symbolic link"
exit 1
fi
done
EOF
${cur%/*} is a bash parameter expansion
EXAMPLE
chmod +x sympath
./sympath /tmp/foo/bar/base
/tmp/foo/bar is a symbolic link
I don't know any easy way, but here's an answer that fully answers your question, using two methods (that are, in fact, essentially the same):
Using an auxiliary script
Create a file called hasnosymlinkinname (or choose a better name --- I've always sucked at choosing names):
#!/bin/bash
name=$1
if [[ "$1" = /* ]]; then
name="$(pwd)/$1"
else
name=$1
fi
IFS=/ read -r -a namearray <<< "$name"
for ((i=0;i<${#namearray[#]}; ++i)); do
IFS=/ read name <<< "${namearray[*]:0:i+1}"
[[ -L "$name" ]] && exit 1
done
exit 0
Then chmod +x hasnosymlinkinname. Then use with find:
find /path/where/stuff/is -exec ./hasnosymlinkinname {} \; -print
The scripts works like this: using IFS trickery, we decompose the filename into each part of the path (separated by the /) and put each part in an array namearray. Then, we loop through the (cumulative) parts of the array (joined with the / thanks to some IFS trickery) and if this part is a symlink (see the -L test), we exit with a non-success return code (1), otherwise, we exit with a success return code (0).
Then find runs this script to all files in /path/where/stuff/is. If the script exits with a success return code, the name of the file is printed out (but instead of -print you could do whatever else you like).
Using a one(!)-liner (if you have a large screen) to impress your grand-mother (or your dog)
find /path/where/stuff/is -exec bash -c 'if [[ "$0" = /* ]]; then name=$0; else name="$(pwd)/$0"; fi; IFS=/ read -r -a namearray <<< "$name"; for ((i=0;i<${#namearray[#]}; ++i)); do IFS=/ read name <<< "${namearray[*]:0:i+1}"; [[ -L "$name" ]] && exit 1; done; exit 0' {} \; -print
Note
This method is 100% safe regarding spaces or funny symbols that could appear in file names. I don't know how you'll use the output of this command, but please make sure that you'll use a good method that will also be safe regarding spaces and funny symbols that could appear in a file name, i.e., don't parse its output with another script unless you use -print0 or similar smart thing.