Bash find fails to return all matching files when called from a script - linux

Running the same command from the command line and from a bash script produces different results on Ubuntu 16.04.
I have a folder with the following contents:
├── audio
│   └── delete_me.mp3
├── words
│   └── audio
│   └── delete_me.mp3
│   └── images
│   └── delete_me.jpg
└── keep_me.txt
I have a bash script named findKeepers.sh:
#!/usr/bin/env bash
findKeepers () {
local dir=$1
echo "$(find $dir -type f ! -name delete_me*)"
}
findKeepers /path/to/directory
I expect it to output the path to the keep_me.txt file. Instead, I get a blank line.
 
If I run what seems to me to be identical commands from the command line, I get what I expect:
dir=/path/to/directory; echo "$(find $dir -type f ! -name delete_me*)"
/path/to/directory/keep_me.txt
If search instead for all files not called keep_me, the bash script ignores the audio folder. Here's another bash script called findUnwanted.sh:
#!/usr/bin/env bash
findUnwanted () {
local dir=$1
echo "$(find $dir -type f ! -name keep_me*)"
}
findUnwanted /path/to/directory
Here's the result:
$ ./findUnwanted.sh
/path/to/directory/words/audio/delete_me.mp3
/path/to/directory/words/images/delete_me.jpg
If I run the same thing from the command line, I get all three delete_me files:
$ dir=/path/to/directory; echo "$(find $dir -type f ! -name keep_me*)"
/path/to/directory/words/audio/delete_me.mp3
/path/to/directory/words/images/delete_me.jpg
/path/to/directory/audio/delete_me.mp3
It seems to me that the bash script starts by going deep into the words folder, and then does not come out again to search adjacent folders or files. Is there something special about the #!/usr/bin/env bash environment that makes it do this? Or is there some other difference that I'm not seeing?
CODA: I'm guessing it was pilot error, because after more modifications it started working for me again. For anyone who is interested, the final version of my function is shown below.
#!/usr/bin/env bash
# Returns 1 if the given directory contains only placeholder files, or
# 0 if the directory contains something worth keeping
checkForDeletion () {
local _dir=$1
local _temp=$(find "$_dir" -type f ! -regex '.*\(unused.txt\|delete_me.*\)')
if [ -z "$_temp" ]
then
return 1
fi
}
I use it like this:
parent=/path/to/parent/
for dir in $parent*/
do
checkForDeletion $dir
if [ $? = 1 ]
then
echo "DELETE? $dir" # rm -rf $dir
fi
done

I am guessing that your '!' is breaking the whole pipe. Try using '-not' instead, so your first code snippet would look like this:
echo "$(find $dir -type f -not -name delete_me*)"
I am not that good at explaining where you should escape special characters and where not, but the fact that things work differently when using that outside function suggests that escaping may be the issue.

Related

find files based on extension but display name without extension no basename, sed, awk, grep or ; allowed

I need to write a script that lists all the files with a .gif extension in the current directory and all its sub-directories BUT DO NOT use ANY of:
basename
grep
egrep
fgrep
rgrep
&&
||
;
sed
awk
AND still include hidden files.
I tried find . -type f -name '*.gif' -printf '%f\n' which will succesfully display .gif files, but still shows extension. Here's the catch: if I try to use cut -d . -f 1 to remove file extension, I also remove hidden files (which I don't want to) because their names start with ".".
Then I turned to use tr -d '.gif' but some of the files have a 'g' or a '.' in their name.
I also tried to use some of these answers BUT all of them include either basename, sed, awk or use some ";" in their script.
With so many restrictions I really don't know if it's even possible to achieve that but I'm supposed to.
How would you do it?
files/dirs structure:
$ tree -a
.
├── bar
├── bar.gif
├── base
│   └── foo.gif
├── foo
│   └── aaa.gif
└── .qux.gif
3 directories, 4 files
Code
find -type f -name '*.gif' -exec bash -c 'printf "%s\n" "${#%.gif}"' bash {} +
Output
./bar
./.qux
./foo/aaa
./base/foo
Explanations
Parameter Expansion expands parameters: $foo, $1. You can use it to perform string or array operations: "${file%.mp3}", "${0##*/}", "${files[#]: -4}". They should always be quoted. See: http://mywiki.wooledge.org/BashFAQ/073 and "Parameter Expansion" in man bash. Also see http://wiki.bash-hackers.org/syntax/pe.
Something like:
find . -name '*.gif' -type f -execdir bash -c 'printf "%s\n" "${#%.*}"' bash {} +
Using perl:
perl -MFile::Find::Rule -E '
say s/\.gif$//r for File::Find::Rule
->file()
->name(qr/\.gif\z/)
->in(".")
'
Output:
bar
.qux
foo/aaa
base/foo

Give out parent folder name if not containing a certain file

I am looking for a terminal linux command to give out the folder parent name that does not contain a certain file:
By now I use the following command:
find . -type d -exec test -e '{}'/recon-all.done \; -print| wc -l
Which gives me the amount of folders which contain then file.
The file recon-all.done would be in /subject/../../recon-all.done and I would need every single "subject" name which does not contain the recon-all.done file.
Loop through the directories, test for the existence of the file, and print the directory if the test fails.
for subject in */; do
if ! [ -e "${subject}scripts/recon-all.done" ]
then echo "$subject"
fi
done
Your command;
find . -type d -exec test -e '{}'/recon-all.done \; -print| wc -l
Almost does the job, we'll just need to
Remove | wc -l to show the directory path witch does not contain the recon-all file
Now, we can negate the -exec test by adding a ! like so:
find . -type d \! -exec test -e '{}'/recon-all.done \; -print
This way find will show each folder name if it does not contain the recon-all file
Note; Based on your comment on Barmar's answer, I've added a -maxdepth 1 to prevent deeper directorys from being checked.
Small example from my local machine:
$ /tmp/test$ tree
.
├── a
│   └── test.xt
├── b
├── c
│   └── test.xt
└── x
├── a
│   └── test.xt
└── b
6 directories, 3 files
$ /tmp/test$ find . -maxdepth 1 -type d \! -exec test -e '{}/test.xt' \; -print
.
./b
./x
$ /tmp/test$

BEST Linux script; to rename SRT to name of movie file in same folder; multiple sub folders [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
There have been multiple attempts to answer this question, but no correct script can be found.
The problem:
SRT subtitles will not load unless having the same name as the movie, or same name as movie +.en.srt or .es.srt or .fr.srt and so on.
1000's of movie directories within a main movie directory having within their respective movie directory, sometimes 1+ .srt files (1_English.srt, 2_English.srt, *French.srt, etc.).
My media server is using Ubuntu, so the solution should use a BASH script.
Here is a snippet of my file structure:
Test-dir$ tree
.
├── renamer.sh
├── Saga.of.the.Phoenix.1990.1080p
│   ├── 1_French.srt
│   ├── 1_Spanish.srt
│   ├── 2_English.srt
│   ├── 3_English.srt
│   └── Saga.of.the.Phoenix.1990.1080p.BluRay.x265.mp4
├── Salt.and.Pepper.1968.1080p
│   ├── 1_French.srt
│   ├── 1_Spanish.srt
│   ├── 2_English.srt
│   ├── 4_English.srt
│   └── Salt.and.Pepper.1968.1080p.mp4
└── Salyut-7.2017.1080p.BluRay.x265
├── 2_English.srt
└── Salyut-7.2017.1080p.BluRay.x265.mp4
The questions:
In writing a BASH script,
There are multiple srt files with the same language, I usually like to choose the bigger file and remove the smaller file, the first part of script would have to sort same language srt and delete the smaller ones, how to script this?
How to change the name of srt's to have the same name as the movie file (not always mp4, sometimes mkv or avi.), while appending acronyms for language (en, es, fr, ru,..) if English.srt then change name to "MovieName".en.srt?
I have started the script removing srt files from the SUB directories of the movie directory and then deleting the SUB directory.
Also, added a script to delete any unwanted parts in the string of the movie, or delete unwanted files.
#!/bin/bash/
# Using current working DIR of where script is ran from
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
# Moves srt files from SUB folders to their movie folder.
for i in */Subs; do
mv "$i"/* "$i"/..
done
# Removes Subs directory.
find $DIR/* -type d -name "Subs" -exec rm -rf {} +
# Removing the additional rar string from the folders and their movie names.
find . -depth -name '*-rar*' -execdir bash -c 'for f; do mv -i "$f" "${f//-rar/}"; done' bash {} +
# Removing unwanted files from all movie folders.
find . -maxdepth 2 -type f \( -name "RAR.txt" -o -name "RAR.nfo" \) -delete
######## Your helper code starts from here to answer questions 1 and 2 #####################
Many thanks for helping with this conundrum, not only will this help one person, but many, on our quest to free many hours of copying, deleting, pasting, and all with a single script.
Update:
BTB91 gave a brilliant answer and has worked, however to help others learn the many ways to go about solving the same problem above I would like to keep this thread open.
IFS=$'\n' eval "MOVS=(\$( find \"\$DIR\" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' ))" # list of movies
for M in "${MOVS[#]}" ; do
cd "$DIR/$M"
IFS=$'\n' eval "LANGS=(\$( ls | sed -nr 's/.*_([[:alpha:]]+).srt/\1/p' | sort -u ))" # list of languages for movie
for L in "${LANGS[#]}" ; do
IFS=$'\n' eval "FILES=(\$( ls -S *_$L.srt))" # list files for language sorted by size
case "${L,,}" in
en*)
L=en
;;
sp*|es*)
L=es
;;
esac
mv -v "${FILES[0]}" "$M.$L.srt"
FILES[0]=
rm -vf "${FILES[#]}"
done
cd "$OLDPWD"
done
I used "IFS=$'\n' eval ..." because the directory or file names might contain spaces.

How to list directories and files in a Bash by script?

I would like to list directory tree, but I have to write script for it and as parameter script should take path to base directory. Listing should start from this base directory.
The output should look like this:
Directory: ./a
File: ./a/A
Directory: ./a/aa
File: ./a/aa/AA
Directory: ./a/ab
File: ./a/ab/AB
So I need to print path from the base directory for every directory and file in this base directory.
UPDATED
Running the script I should type in the terminal this: ".\test.sh /home/usr/Desktop/myDirectory" or ".\test.sh myDirectory" - since I run the test.sh from the Desktop level.
And right now the script should be run from the level of /home/usr/Dekstop/myDirectory"
I have the following command in my test.sh file:
find . | sed -e "s/[^-][^\/]*\// |/g"
But It is the command, not shell code and prints the output like this:
DIR: dir1
DIR: dir2
fileA
DIR: dir3
fileC
fileB
How to print the path from base directory for every dir or file from the base dir? Could someone help me to work it out?
Not clear what you want maybe,
find . -type d -printf 'Directory: %p\n' -o -type f -printf 'File: %p\n'
However to see the subtree of a directory, I find more useful
find "$dirname" -type f
To answer comment it can also be done in pure bash (builtin without external commands), using a recursive function.
rec_find() {
local f
for f in "$1"/*; do
[[ -d $f ]] && echo "Directory: $f" && rec_find "$f"
[[ -f $f ]] && echo "File: $f"
done
}
rec_find "$1"
You can use tree command. Key -L means max depth. Examples:
tree
.
├── 1
│   └── test
├── 2
│   └── test
└── 3
└── test
3 directories, 3 files
Or
tree -L 1
.
├── 1
├── 2
└── 3
3 directories, 0 files
Create your test.sh with the below codes. Here you are reading command line parameter in system variable $1 and provides parameter to find command.
#!/bin/bash #in which shell you want to execute this script
find $1 | sed -e "s/[^-][^\/]*\// |/g"
Now how will it work:-
./test.sh /home/usr/Dekstop/myDirectory #you execute this command
Here command line parameter will be assign into $1. More than one parameter you can use $1 till $9 and after that you have to use shift command. (You will get more detail information online).
So your command will be now:-
#!/bin/bash #in which shell you want to execute this script
find /home/usr/Dekstop/myDirectory | sed -e "s/[^-][^\/]*\// |/g"
Hope this will help you.

Bash - finding files with spaces and rename with sed [duplicate]

This question already has answers here:
Recursively rename files using find and sed
(20 answers)
Closed 9 years ago.
I have been trying to write a script to rename all files that contain a space and replace the space with a dash.
Example: "Hey Bob.txt" to "Hey-Bob.txt"
When I used a for-loop, it just split up the file name at the space, so "Hey Bob.txt" gave separate argument like "Hey" and "Bob.txt".
I tried the following script but it keeps hanging on me.
#!/bin/bash
find / -name '* *' -exec mv {} $(echo {} | sed 's/ /-g')\;
Building off OP's idea:
find ${PATH_TO_FILES} -name '* *' -exec bash -c 'eval $(echo mv -v \"{}\" $(echo {} | sed "s/ /-/g"))' \;
NOTE: need to specify the PATH_TO_FILES variable
EDIT: BroSlow pointed out need to consider directory structure:
find ${PATH_TO_FILES} -name '* *' -exec bash -c 'DIR=$(dirname "{}" | sed "s/ /-/g" ); BASE=$(basename "{}"); echo mv -v \"$DIR/$BASE\" \"$DIR/$(echo $BASE | sed "s/ /-/g")\"' \; > rename-script.sh ; sh rename-script.sh
Another way:
find . -name "* *" -type f |while read file
do
new=${file// /}
mv "${file}" $new
done
Not one line, but avoids sed and should work just as well if you're going to be using it for a script anyway. (replace the mv with an echo if you want to test)
In bash 4+
#!/bin/bash
shopt -s globstar
for file in **/*; do
filename="${file##*/}"
if [[ -f $file && $filename == *" "* ]]; then
onespace=$(echo $filename)
dir="${file%/*}"
[[ ! -f "$dir/${onespace// /-}" ]] && mv "$file" "$dir/${onespace// /-}" || echo "$dir/${onespace// /-} already exists, so not moving $file" 1>&2
fi
done
Older bash
#!/bin/bash
find . -type f -print0 | while read -r -d '' file; do
filename="${file##*/}"
if [[ -f $file && $filename == *" "* ]]; then
onespace=$(echo $filename)
dir="${file%/*}"
[[ ! -f "$dir/${onespace// /-}" ]] && mv "$file" "$dir/${onespace// /-}" || echo "$dir/${onespace// /-} already exists, so not moving $file" 1>&2
fi
done
Explanation of algorithm
**/* This recursively lists all files in the current directory (** technically does it but /* is added at the end so it doesn't list the directory itself)
${file##*/} Will search for the longest pattern of */ in file and remove it from the string. e.g. /foo/bar/test.txt gets printed as test.txt
$(echo $filename) Without quoting echo will truncate spaces to one, making them easier to replace with one - for any number of spaces
${file%/*} Remove everything after and including the last /, e.g. /foo/bar/test.txt prints /foo/bar
mv "$file" ${onespace// /-} replace every space in our filename with - (we check if the hyphened version exists before hand and if it does echo that it failed to stderr, note && is processed before || in bash)
find . -type f -print0 | while read -r -d '' file This is used to avoid break up strings with spaces in them by setting a delimiter and not processing \
Sample Output
$ tree
.
├── bar
│   ├── some dir
│   │   ├── some-name-without-space1.pdf
│   │   ├── some name with space1.pdf
│   ├── some-name-without-space1.pdf
│   ├── some name with space1.pdf
│   └── some-name-with-space1.pdf
└── space.sh
$ ./space.sh
bar/some-name-with-space1.pdf already exists, so not moving bar/some name with space1.pdf
$ tree
.
├── bar
│   ├── some dir
│   │   ├── some-name-without-space1.pdf
│   │   ├── some-name-with-space1.pdf
│   ├── some-name-without-space1.pdf
│   ├── some name with space1.pdf
│   └── some-name-with-space1.pdf
└── space.sh

Resources