I have a bash script in which I need to iterate over each line of the ouput of the find command, but it appears that I am iterating over each Word (space delimited) from the find command. My script looks like this so far:
folders=`find -maxdepth 1 -type d`
for $i in $folders
do
echo $i
done
I would expect this to give output like:
./dir1 and foo
./dir2 and bar
./dir3 and baz
But I am insted getting output like this:
./dir1
and
foo
./dir2
and
bar
./dir3
and
baz
What am I doing wrong here?
folders=`foo`
is always wrong, because it assumes that your directories won't contain spaces, newlines (yes, they're valid!), glob characters, etc. One robust approach (which requires the GNU extension -print0) follows:
while IFS='' read -r -d '' filename; do
: # something with "$filename"
done < <(find . -maxdepth 1 -type d -print0)
Another safe and robust approach is to have find itself directly invoke your desired command:
find . -maxdepth 1 -type d -exec printf '%s\n' '{}' +
See the UsingFind wiki page for a complete treatment of the subject.
Since you aren't using any of the more advanced features of find, you can use a simple pattern to iterate over the subdirectories:
for i in ./*/; do
echo "$i"
done
You can do something like this:
find -maxdepth 1 -type d | while read -r i
do
echo "$i"
done
Related
I'm trying to write a bash script to copy all .desktop files under /nix/store to ~/.local/share/applications/. I'm not particular good with bash, so I would like assistance. I used the find command to find all files. Now I'm trying to create an array from the output with the help of readarray:
files=$(find /nix/store -type f -name \*.desktop)
echo $files
x=$(readarray -d 's' <<<$files)
echo $x
The echo $files will print the result of the find command, however the echo $x prints an empty line.
#!/usr/bin/env bash
files=$(find /nix/store -type f -name \*.desktop)
readarray array <<<$files
for i in ${array[#]}; do
cp $i ~/.loca/share/applications/
done
or
find /nix/store -type f -name \*.desktop -exec cp {} ~/.local/share/applications/ \;
Copying all .desktop files to ~/.local/share/applications/
find /nix/store -type f -name '*.desktop' \
-exec cp -v {} ~/.local/share/applications/ ';'
however the echo $x prints an empty line.
readarray produces no output. Instead, it stores the lines in the argument, which represents the name of the variable, the name of the array.
readarray -d 's' <<<$files
# ^^^ - stores output in array named s
printf "%s\n" "${s[#]}" # you can print the array s
If I'm not wrong and /nix/store/ is a regular directory, this should work:
cp /nix/store/*.desktop ~/.local/share/applications/
I want to write a shell script. I list my jpg files inside nested subdirectories with the following command line:
find . -type f -name "*.jpg"
How can I save the output of this command inside a variable and write a for loop for that? (I want to do some processing steps for each jpg file)
You don't want to store output containing multiple files into a variable/array and then post-process it later. You can just do those actions on the files on-the-run.
Assuming you have bash shell available, you could write a small script as
#!/usr/bin/env bash
# ^^^^ bash shell needed over any POSIX shell because
# of the need to use process-substitution <()
while IFS= read -r -d '' image; do
printf '%s\n' "$image"
# Your other actions can be done here
done < <(find . -type f -name "*.jpg" -print0)
The -print0 option writes filenames with a null byte terminator, which is then subsequently read using the read command. This will ensure the file names containing special characters are handled without choking on them.
Better than storing in a variable, use this :
find . -type f -name "*.jpg" -exec command {} \;
Even, if you want, command can be a full bloated shell script.
A demo is better than an explanation, no ? Copy paste the whole lines in a terminal :
cat<<'EOF' >/tmp/test
#!/bin/bash
echo "I play with $1 and I can replay with $1, even 3 times: $1"
EOF
chmod +x /tmp/test
find . -type f -name "*.jpg" -exec /tmp/test {} \;
Edit: new demo (from new questions from comments)
find . -type f -name "*.jpg" | head -n 10 | xargs -n1 command
(this another solution doesn't take care of filenames with newlines or spaces)
This one take care :
#!/bin/bash
shopt -s globstar
count=0
for file in **/*.jpg; do
if ((++count < 10)); then
echo "process file $file number $count"
else
break
fi
done
Let say I have 200 directories and it have variable hierarchy sub-directories, How can I rename the directory and its sub directories using mv command with find or any sort of combination?
for dir in ./*/; do (i=1; cd "$dir" && for dir in ./*; do printf -v dest %s_%02d "$dir" "$((i++))"; echo mv "$dir" "$dest"; done); done
This is for 2 level sub directory, is there more cleaner way to do it for multiple hierarchy? Any other one line command suggestions/ solutions are welcome.
I had a specific task - to replace non-ASCII symbols and square brackets, in directories and in files as well. It works fine.
First, exactly my case, as a working example:
find . -depth -execdir rename -v 's/([^\x00-\x7F]+)|([\[\]]+)/\_/g' {} \;
or separately non-ascii and brackets:
find . -depth -execdir rename -v 's/[^\x00-\x7F]+/\_/g' {} \;
find . -depth -execdir rename -v 's/[\[\]]+/\_/g' {} \;
If we'd like to work only with directories, add -type d (after the -depth option)
Now, in more generalized view:
find . -depth [-type d] [-type f] -execdir rename [-v] 's/.../.../g' '{}' \;
Here we can control dirs/files and verbosity. Quotes around {} may be needed or not on your machine (backslash before ; serves the same, may be replaced with quotes)
You have two options when you want to do recursive operations in files/directories:
Option 1 : Find
while IFS= read -r -d '' subd;do
#do your stuff here with var $subd
done < <(find . -type d -print0)
In this case we use find to return only dirs using -type d
We can ask find to return only files using -type f or not to specify any type and both directories and files will be returned.
We also use find option -print0 to force null separation of the find results and thus to ensure correct names handling in case names include special chars like spaces, etc.
Testing:
$ while IFS= read -r -d '' s;do echo "$s";done < <(find . -type d -print0)
.
./dir1
./dir1/sub1
./dir1/sub1/subsub1
./dir1/sub1/subsub1/subsubsub1
./dir2
./dir2/sub2
Option 2 : Using Bash globstar option
shopt -s globstar
for subd in **/ ; do
#Do you stuff here with $subd directories
done
In this case , the for loop will match all subdirs under current working directory (operation **/).
You can also ask bash to return both files and folders using
for sub in ** ;do #your commands;done
if [[ -d "$sub" ]];then
#actions for folders
elif [[ -e "$sub" ]];then
#actions for files
else
#do something else
fi
done
Folders Test:
$ shopt -s globstar
$ for i in **/ ;do echo "$i";done
dir1/
dir1/sub1/
dir1/sub1/subsub1/
dir1/sub1/subsub1/subsubsub1/
dir2/
dir2/sub2/
In your small script, just by enabling shopt -s globstar and by changing your for to for dir in **/;do it seems that work as you expect.
In a directory with a bunch of files that look like this:
./test1_November 08, 2014 AM.flv
./test2.flv
./script1.sh
./script2.sh
I want to process only files that have an .flv extension and no underscore. I'm trying to eliminate the files with underscores without much luck.
bash --version
GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)
script:
#!/bin/bash
FILES=$(find . -mtime 0)
for f in "${FILES}"
do
if [[ "$f" != *_* ]]; then
echo "$f"
fi
done
This gives me no files. Changing the != to == gives me all files instead of just those with an underscore. Other answers on SO indicate this should work.
Am I missing something simple here?
You can use this extended glob pattern:
shopt -s extglob
echo +([^_]).flv
+([^_]) will match 1 or more of any non underscore character.
Testing:
ls -l *flv
-rw-r--r-- 1 user1 staff 0 Nov 8 12:44 test1_November 08, 2014 AM.flv
-rw-r--r-- 1 user1 staff 0 Nov 8 12:44 test2.flv
echo +([^_]).flv
test2.flv
To process these files in a loop use:
for f in +([^_]).flv; do
echo "Processing: $f"
done
PS: Not sure you're using -mtime 0 in your find as my answer is for the requiremnt:
I want to process only files that have an .flv extension and no underscore
You can pass multiple patterns to find and include a not
find . -name "*.flv" -not -name "*_*"
You can loop over the results of find by piping it into a while
find -name "*.flv" -not -name "*_*" -print0 | while IFS= read -r -d '' filename; do
echo "$filename"
done
or you can forgo the loop completely and use xargs
find -name "*.flv" -not -name "*_*" -print0 | xargs -0 -n1 echo
The problem is this line:
for f in "${FILES}"
The quotes are preventing word splitting, so entire list of filenames is being processed as a single item. What you want is:
IFS=$'\n'
for f in $FILES
The IFS setting makes it use newlines as the word delimiters, so you can have filenames with spaces in them.
A better way to write loops like this is to avoid using the variable:
find ... | while read -r f
do
...
done
How can I search for files in directories that contain spaces in names, using find?
i use script
#!/bin/bash
for i in `find "/tmp/1/" -iname "*.txt" | sed 's/[0-9A-Za-z]*\.txt//g'`
do
for j in `ls "$i" | grep sh | sed 's/\.txt//g'`
do
find "/tmp/2/" -iname "$j.sh" -exec cp {} "$i" \;
done
done
but the files and directories that contain spaces in names are not processed?
This will grab all the files that have spaces in them
$ls
more space nospace stillnospace this is space
$find -type f -name "* *"
./this is space
./more space
I don't know how to achieve you goal. But given your actual solution, the problem is not really with find but with the for loops since "spaces" are taken as delimiter between items.
find has a useful option for those cases:
from man find:
-print0
True; print the full file name on the standard output, followed by a null character
(instead of the newline character that -print uses). This allows file names
that contain newlines or other types of white space to be correctly interpreted
by programs that process the find output. This option corresponds to the -0
option of xargs.
As the man saids, this will match with the -0 option of xargs. Several other standard tools have the equivalent option. You probably have to rewrite your complex pipeline around those tools in order to process cleanly file names containing spaces.
In addition, see bash "for in" looping on null delimited string variable to learn how to use for loop with 0-terminated arguments.
Do it like this
find . -type f -name "* *"
Instead of . you can specify your path, where you want to find files with your criteria
Your first for loop is:
for i in `find "/tmp/1" -iname "*.txt" | sed 's/[0-9A-Za-z]*\.txt//g'`
If I understand it correctly, it is looking for all text files in the /tmp/1 directory, and then attempting to remove the file name with the sed command right? This would cause a single directory with multiple .txt files to be processed by the inner for loop more than once. Is that what you want?
Instead of using sed to get rid of the filename, you can use dirname instead. Also, later on, you use sed to get rid of the extension. You can use basename for that.
for i in `find "/tmp/1" -iname "*.txt"` ; do
path=$(dirname "$i")
for j in `ls $path | grep POD` ; do
file=$(basename "$j" .txt)
# Do what ever you want with the file
This doesn't solve the problem of having a single directory processed multiple times, but if it is an issue for you, you can use the for loop above to store the file name in an array instead and then remove duplicates with sort and uniq.
Use while read loop with null-delimited pathname output from find:
#!/bin/bash
while IFS= read -rd '' i; do
while IFS= read -rd '' j; do
find "/tmp/2/" -iname "$j.sh" -exec echo cp '{}' "$i" \;
done <(exec find "$i" -maxdepth 1 -mindepth 1 -name '*POD*' -not -name '*.txt' -printf '%f\0')
done <(exec find /tmp/1 -iname '*.txt' -not -iname '[0-9A-Za-z]*.txt' -print0)
Never used for i in $(find...) or similar as it'll fail for file names containing white space as you saw.
Use find ... | while IFS= read -r i instead.
It's hard to say without sample input and expected output but something like this might be what you need:
find "/tmp/1/" -iname "*.txt" |
while IFS= read -r i
do
i="${i%%[0-9A-Za-z]*\.txt}"
for j in "$i"/*sh*
do
j="${j%%\.txt}"
find "/tmp/2/" -iname "$j.sh" -exec cp {} "$i" \;
done
done
The above will still fail for file names that contains newlines. If you have that situation and can't fix the file names then look into the -print0 option for find, and piping it to xargs -0.