Following command
"find . -type f -regextype posix-extended -regex './ctrf.|./rbc.' -exec basename {} ;"
And executing it.
I am stroring the command in variable in shell script link
Find_Command=$1
For Execution
Files="$(${Find_Command})"
Not working.
Best Practice: Accept An Array, Not A String
First, your shell script should take the command to run as a series of separate arguments, not a single argument.
#!/usr/bin/env bash
readarray -d '' Files < <("$#")
echo "Found ${#Files[#]} files" >&2
printf ' - %q\n' "${Files[#]}"
called as:
./yourscript find . -type f -regextype posix-extended -regex './ctrf.*|./rbc.*' -printf '%f\0'
Note that there's no reason to use the external basename command: find -printf can directly print you only the filename.
Fallback: Parsing A String To An Array Correctly
If you must accept a string, you can use the answers in Reading quoted/escaped arguments correctly from a string to convert that string to an array safely.
Compromising complete shell compatibility to avoid needing nonstandard tools, we can use xargs:
#!/usr/bin/env bash
readarray -d '' Command_Arr < <(xargs printf '%s\0' <<<"$1")
readarray -d '' Files < <("${Command_Arr[#]}")
echo "Found ${#Files[#]} files" >&2
printf ' - %q\n' "${Files[#]}"
...with your script called as:
./yourscript $'find . -type f -regextype posix-extended -regex \'./ctrf.*|./rbc.*\' -printf \'%f\\0\''
If you want to run a command specified in a variable and save the output in another variable, you can use following commands.
command="find something" output=$($command)
Or if you want to store output in array:
typeset -a output=$($command)
However, storing filenames in variables and then attempting to access files with those filenames is a bad idea because it is impossible to set the proper delimiter to separate filenames because filenames can contain any character except NUL (see https://mywiki.wooledge.org/BashPitfalls).
I'm not sure what you're trying to accomplish, but your find command contains an error. The -exec option must end with ; to indicate the end of the -exec parameters. Aside from that, it appears to be 'The xy problem' see https://xyproblem.info/
If you want to get basename of regular files with the extension .ctrf or.rbc, use the bash script below.
for x in **/*.+(ctrf|rbc); do basename $x ; done
Or zsh script
basename **/*.(ctrf|rbc)(#q.)
Make sure you have enabled 'extended glob' option in your shell.
To enable it in bash run following comand.
shopt -s extglob
And for zsh
setopt extendedglob
You should use array instead of string for Find_Command :
#!/usr/bin/env bash
Find_Command=(find . -type f -regextype posix-extended -regex '(./ctrf.|./rbc.)' -exec basename {} \;)
Files=($(“${Find_Command[#]}”))
Second statement assumes you don't have special characters (like spaces) in your file names.
Use eval:
Files=$(eval "${Find_Command}")
Be mindful of keeping the parameter sanitized and secure.
Related
From reading this stackoverflow answer I was able to remove the file extension from the files using find:
find . -name "S4*" -execdir basename {} .fastq.gz ';'
returned:
S9_S34_R1_001
S9_S34_R2_001
I'm making a batch script where I want to extract the filename with the above prefix to pass as arguments into a program. At the moment I'm currently doing this with a loop but am wondering if it can be achieved using find.
for i in $(ls | grep 'S9_S34*' | cut -d '.' -f 1); do echo "$i"_trim.log "$i"_R1_001.fastq.gz "$i"_R2_001.fastq.gz; done; >> trim_script.sh
Is it possible to do something as follows:
find . -name "S4*" -execdir basename {} .fastq.gz ';' | echo {}_trim.log {}_R1_001.fastq.gz {}_R2_001.fastq.gz {}\ ; >> trim_script.sh
You don't need basename at all, or -exec, if all you're doing is generating a series of strings that contain your file's basenames within them; the -printf action included in GNU find can do all that for you, as it provides a %P built-in to insert the basename of your file:
find . -name "S4*" \
-printf '%P_trim.log %P_R1_001.fastq.gz %P_R2_001.fastq.gz %P\n' \
>trim_script.sh
That said, be sure you only do this if you trust your filenames. If you're truly running the result as a script, there are serious security concerns if someone could create a S4$(rm -rf ~).txt file, or something with a similarly malicious name.
What if you don't trust your filenames, or don't have the GNU version of find? Then consider making find pass them into a shell (like bash or ksh) that supports the %q extension, to generate a safely-escaped version of those names (note that you should run the script with the same interpreter you used for this escaping):
find . -name "S4*" -exec bash -c '
for file do # iterates over "$#", so processes each file in turn
file=${file##*/} # get the basename
printf "%q_trim.log %q_R1_001.fastq.gz %q_R2_001.fastq.gz %q\n" \
"$file" "$file" "$file" "$file"
done
' _ {} + >trim_script.sh
Using -exec ... {} + invokes the smallest possible number of subprocesses -- not one per file found, but instead one per batch of filenames (using the largest possible batch that can fit on a command line).
I am trying to write a bash script which does some processing on music files. Here is the script so far:
#!/bin/bash
SAVEIFS=$IFS
IFS=printf"\n\0"
find `pwd` -iname "*.mp3" -o -iname "*.flac" | while read f
do
echo "$f"
$arr=($(f))
exiftool "${arr[#]}"
done
IFS=$SAVEIFS
This fails with:
[johnd:/tmp/tunes] 2 $ ./test.sh
./test.sh: line 9: syntax error near unexpected token `$(f)'
./test.sh: line 9: ` $arr=($(f))'
[johnd:/tmp/tunes] 2 $
I have tried many different incantations, none of which have worked. The bottom line is I'm trying to call a command exiftool, and one of the parameters of that command is a filename which may contain spaces. Above I'm trying to assign the filename $f to an array and pass that array to exiftool, but I'm having trouble with the construction of the array.
Immediate question is, how do I construct this array? But the deeper question is how, from within a bash script, do I call an external command with parameters which may contain spaces?
You actually did have the call-with-possibly-space-containing-arguments syntax right (program "${args[#]}"). There were several problems, though.
Firstly, $(foo) executes a command. If you want a variable's value, use $foo or ${foo}.
Secondly, if you want to append something onto an array, the syntax is array+=(value) (or, if that doesn't work, array=("${array[#]}" value)).
Thirdly, please separate filenames with \0 whenever possible. Newlines are all well and good, but filenames can contain newlines.
Fourthly, read takes the switch -d, which can be used with an empty string '' to specify \0 as the delimiter. This eliminates the need to mess around with IFS.
Fifthly, be careful when piping into while loops - this causes the loop to be executed in a subshell, preventing variable assignments inside it from taking effect outside. There is a way to get around this, however - instead of piping (command | while ... done), use process substitution (while ... done < <(command)).
Sixthly, watch your process substitutions - there's no need to use $(pwd) as an argument to a command when . will do. (Or if you really must have full paths, try quoting the pwd call.)
tl;dr
The script, revised:
while read -r -d '' f; do
echo "$f" # For debugging?
arr+=("$f")
done < <(find . -iname "*.mp3" -o -iname "*.flac" -print0)
exiftool "${arr[#]}"
Another way
Leveraging find's full capabilities:
find . -iname "*.mp3" -o -iname "*.flac" -exec exiftool {} +
# Much shorter!
Edit 1
So you need to save the output of exiftool, manipulate it, then copy stuff? Try this:
while read -r -d '' f; do
echo "$f" # For debugging?
arr+=("$f")
done < <(find . -iname "*.mp3" -o -iname "*.flac" -print0)
# Warning: somewhat misleading syntax highlighting ahead
newfilename="$(exiftool "${arr[#]}")"
newfilename="$(manipulate "$newfilename")"
cp -- "$some_old_filename" "$newfilename"
You probably will need to change that last bit - I've never used exiftool, so I don't know precisely what you're after (or how to do it), but that should be a start.
You can do this just with bash:
shopt -s globstar nullglob
a=( **/*.{mp3,flac} )
exiftool "${a[#]}"
This probably works too: exiftool **/*.{mp3,flac}
Good day,
I've found an easy way to find files that have certain content, but I would like to create a bash script to do it quickier,
The script is:
#!/bin/bash
DIRECTORY=$(cd `dirname .` && pwd)
ARGUMENTS="'$#'"
echo find: $ARGUMENTS on $DIRECTORY
find $DIRECTORY -iname '*' | xargs grep $ARGUMENTS -sl
So if I write:
$ script.sh text
It should find in that directory files that contains 'text'
But when I execute this script it always fails, but the echo command shows exactly what I need, what's wrong with this script?
Thank you!
Luis
References: http://www.liamdelahunty.com/tips/linux_find_string_files.php
There are problems with quoting that will break in this script if either the current directory or the search pattern contains a space. The following is more simply, and fixes both issues:
find . -maxdepth 1 -type f -exec grep "$#" {} +
With the proper quoting of $#, you can even pass options to grep, such as -i.
./script -i "some text"
Try this version, with the following changes:
1.Use $1 instead of $# unless you intend to run multiple find/grep to search for multiple patterns.
2.Use find $DIR -type f to find all files instead of find $DIR -iname '*'
3.Avoid piping by using the -exec command line option of find.
4.Do not single quote the command line arguments to your script, this was the main problem with the version you had. Your grep string had escaped single quotes \'search_string\'
#!/bin/bash
DIRECTORY=$(cd `dirname .` && pwd)
ARGUMENTS="$1"
echo find: $ARGUMENTS on $DIRECTORY
find $DIRECTORY . -type f -exec grep -sl "$ARGUMENTS" {} \;
There is no point extracting all the command line arguments and passing it to grep. If you want to search for a string with spaces, pass the string within single quotes from the command line as follows:
/home/user/bin/test-find.sh 'i need to search this'
Why not just run the following?:
grep -R text .
The following command attempts to enumerate all *.txt files in the current directory and process them one by one:
for line in "find . -iname '*.txt'"; do
echo $line
ls -l $line;
done
Why do I get the following error?:
ls: invalid option -- 'e'
Try `ls --help' for more information.
Here is a better way to loop over files as it handles spaces and newlines in file names:
#!/bin/bash
find . -type f -iname "*.txt" -print0 | while IFS= read -r -d $'\0' line; do
echo "$line"
ls -l "$line"
done
The for-loop will iterate over each (space separated) entry on the provided string.
You do not actually execute the find command, but provide it is as string (which gets iterated by the for-loop).
Instead of the double quotes use either backticks or $():
for line in $(find . -iname '*.txt'); do
echo "$line"
ls -l "$line"
done
Furthermore, if your file paths/names contains spaces this method fails (since the for-loop iterates over space separated entries). Instead it is better to use the method described in dogbanes answer.
To clarify your error:
As said, for line in "find . -iname '*.txt'"; iterates over all space separated entries, which are:
find
.
-iname
'*.txt' (I think...)
The first two do not result in an error (besides the undesired behavior), but the third is problematic as it executes:
ls -l -iname
A lot of (bash) commands can combine single character options, so -iname is the same as -i -n -a -m -e. And voila: your invalid option -- 'e' error!
More compact version working with spaces and newlines in the file name:
find . -iname '*.txt' -exec sh -c 'echo "{}" ; ls -l "{}"' \;
Use command substitution instead of quotes to execute find instead of passing the command as a string:
for line in $(find . -iname '*.txt'); do
echo $line
ls -l $line;
done
I am trying to manipulate the filename from the find command:
find . -name "*.xib" -exec echo '{}' ';'
For example, this might print:
./Views/Help/VCHelp.xib
I would like to make it:
./Views/Help/VCHelp.strings
What I tried:
find . -name "*.xib" -exec echo ${'{}'%.*} ';'
But, the '{}' is not being recognized as a string or something...
I also tried the following:
find . -name "*.xib" -exec filename='{}' ";" -exec echo ${filename%.*} ";"
But it is trying to execute a command called "filename" instead of assigning the variable:
find: filename: No such file or directory
You can't use Parameter Expansion with literal string. Try to store it in a variable first:
find . -name '*.xib' -exec bash -c "f='{}' ; echo \${f%.xib}.strings" \;
-exec sees first argument after it as the command, therefore you can't simply give it filename='{}' because find doesn't use sh to execute what you give it. If you want to run some shell stuff, you need to use sh or bash to wrap up.
Or use sed:
find . -name '*.xib' | sed 's/.xlib$/.strings/'
For such a simple search, you can use a pure bash solution:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
found=( **.xib )
for f in "${found[#]}"; do
echo "${f%xib}strings"
done
Turning the globstar shell option on enables the ** to "match all files and zero or more directories and subdirectories" (as quoted from the bash reference manual). The nullglob option helps if there's no match: in this case, the glob will be expanded to nothing instead of the ugly **.xib. This solution is safe regarding filenames containing funny characters.
find . -name "*.xib" | sed -e 's/\.xib$/.strings/'