I want to find all the .pdf files recursively by using find
So I typed in find . -name *.pdf
And the output was weird ,it only contains all the pdf files in the current directory , other pdf fils in the sub directory is omitted
Then I tried this find . -name '*.pdf'
This time ,every thing is fine .And I got what I want, I mean all the pdf files including those located in the sub directory.
So here comes the deal: what is the difference between find . -name *.pdf and find . -name '*.pdf'
Yes as you've found that quoting makes all the difference there.
Without quoting *.pdf gets expanded by shell glob expansion rule even before find runs and thus find command shows all the pdf files from that list only.
In other words this find command:
find . -name *.pdf
is same as:
printf "%s\n" *.pdf
So right way to use find is:
find . -name '*.pdf'
In the Linux shell (bash, csh, sh, and probably many others I'm not as familiar with), different quotes mean different things.
Fundamentally, quotes do two things for you:
Limiting text substitution: You're experiencing the shell substituting references to all PDF files in your current directory for *.pdf. That's an example of text substitution. Text substitution can also occur with variable names--for example:
bash$ MYVAR=test
bash$ echo $MYVAR
test
bash$ echo '$MYVAR'
$MYVAR
will give you the output test, because $MYVAR is substituted with the value the variable is set to.
Overriding space as an argument separator: Let's pretend you have a directory with these files
bash$ ls -1
file1
file1 file2
file2
If you type ls file1, you'd get file1 as you expect. Similarly, ls file2 gives you file2. The following commands show the significance of quotes overriding space as an argument separator:
bash$ ls -1 file1 file2
file1
file2
bash$ ls -1 "file1 file2"
file1 file2
Notice how the first example displays two files file1 and file2, while the second example displays one file file1 file2. That's because the quotes stop " " (a single space) from being used as an argument separator.
One final note: your original question asks about the difference between something and 'something'. It's worth nothing that there is actually a difference between something, 'something', and "something". Consider this:
bash$ MYVAR=test
bash$ echo $MYVAR
test
bash$ echo '$MYVAR'
$MYVAR
bash$ echo "$MYVAR"
test
Note the difference between '$MYVAR' and "$MYVAR". The ' (single quote) is considered a "strong quote," meaning everything contained inside it is explicit. The " (double quote) is a "weak quote," which does not expand * or ?, but does expand variable names and command substitutions.
The Grymorie provides a tremendous amount of information about quotes. Have fun learning!
Related
I'm using for loop, with arguments i. Each argument contains ".net" at the end and in directory they are in one line, divided by some space. Now I need to get rid of these ".net" using substitution of sed, but it's not working. I went through different options, the most recent one is
sed 's/\.(net)//g' $i;
which is obviously not correct, but I just can't find anything online about this.
To make it clear, lets say I have a directory with 5 files with names
file1.net
file2.net
file3.net
file4.net
file5.net
I would like my output to be
file1
file2
file3
file
file5
...Could somebody give me some advice?
You can use
for f in *.net; do mv "$f" "${f%.*}"; done
Details:
for f in *.net; - iterates over files with net extension
mv "$f" "${f%.*}" - renames the files with the file without net extension (${f%.*} removes all text - as few as possible - from the end of f till the first ., see Parameter expansion).
This is a work for perl's rename :
rename -n 's/\.net//' *.net
The -n is for test purpose. Remove it if the output looks good for you
This way:
sed -i.backup 's/\.net$//g' "$1";
It will create a backup for safeness
How do i print out all the files of the current directory that start with the letter "k" ?Also needs to count this files.
I tried some methods but i only got errors or wrong outputs. Really stuck on this as a newbie in bash.
Try this Shellcheck-clean pure POSIX shell code:
count=0
for file in k*; do
if [ -f "$file" ]; then
printf '%s\n' "$file"
count=$((count+1))
fi
done
printf 'count=%d\n' "$count"
It works correctly (just prints count=0) when run in a directory that contains nothing starting with 'k'.
It doesn't count directories or other non-files (e.g. fifos).
It counts symlinks to files, but not broken symlinks or symlinks to non-files.
It works with 'bash' and 'dash', and should work with any POSIX-compliant shell.
Here is a pure Bash solution.
files=(k*)
printf "%s\n" "${files[#]}"
echo "${#files[#]} files total"
The shell expands the wildcard k* into the array, thus populating it with a list of matching files. We then print out the array's elements, and their count.
The use of an array avoids the various problems with metacharacters in file names (see e.g. https://mywiki.wooledge.org/BashFAQ/020), though the syntax is slightly hard on the eyes.
As remarked by pjh, this will include any matching directories in the count, and fail in odd ways if there are no matches (unless you set nullglob to true). If avoiding directories is important, you basically have to get the directories into a separate array and exclude those.
To repeat what Dominique also said, avoid parsing ls output.
Demo of this and various other candidate solutions:
https://ideone.com/XxwTxB
To start with: never parse the output of the ls command, but use find instead.
As find basically goes through all subdirectories, you might need to limit that, using the -maxdepth switch, use value 1.
In order to count a number of results, you just count the number of lines in your output (in case your output is shown as one piece of output per line, which is the case of the find command). Counting a number of lines is done using the wc -l command.
So, this comes down to the following command:
find ./ -maxdepth 1 -type f -name "k*" | wc -l
Have fun!
This should work as well:
VAR="k"
COUNT=$(ls -p ${VAR}* | grep -v ":" | wc -w)
echo -e "Total number of files: ${COUNT}\n" 1>&2
echo -e "Files,that begin with ${VAR} are:\n$(ls -p ${VAR}* | grep -v ":" )" 1>&2
This question already has answers here:
Rename multiple files based on pattern in Unix
(24 answers)
Closed 5 years ago.
Write a simple script that will automatically rename a number of files. As an example we want the file *001.jpg renamed to user defined string + 001.jpg (ex: MyVacation20110725_001.jpg) The usage for this script is to get the digital camera photos to have file names that make some sense.
I need to write a shell script for this. Can someone suggest how to begin?
An example to help you get off the ground.
for f in *.jpg; do mv "$f" "$(echo "$f" | sed s/IMG/VACATION/)"; done
In this example, I am assuming that all your image files contain the string IMG and you want to replace IMG with VACATION.
The shell automatically evaluates *.jpg to all the matching files.
The second argument of mv (the new name of the file) is the output of the sed command that replaces IMG with VACATION.
If your filenames include whitespace pay careful attention to the "$f" notation. You need the double-quotes to preserve the whitespace.
You can use rename utility to rename multiple files by a pattern. For example following command will prepend string MyVacation2011_ to all the files with jpg extension.
rename 's/^/MyVacation2011_/g' *.jpg
or
rename <pattern> <replacement> <file-list>
this example, I am assuming that all your image files begin with "IMG" and you want to replace "IMG" with "VACATION"
solution : first identified all jpg files and then replace keyword
find . -name '*jpg' -exec bash -c 'echo mv $0 ${0/IMG/VACATION}' {} \;
for file in *.jpg ; do mv $file ${file//IMG/myVacation} ; done
Again assuming that all your image files have the string "IMG" and you want to replace "IMG" with "myVacation".
With bash you can directly convert the string with parameter expansion.
Example: if the file is IMG_327.jpg, the mv command will be executed as if you do mv IMG_327.jpg myVacation_327.jpg. And this will be done for each file found in the directory matching *.jpg.
IMG_001.jpg -> myVacation_001.jpg
IMG_002.jpg -> myVacation_002.jpg
IMG_1023.jpg -> myVacation_1023.jpg
etcetera...
find . -type f |
sed -n "s/\(.*\)factory\.py$/& \1service\.py/p" |
xargs -p -n 2 mv
eg will rename all files in the cwd with names ending in "factory.py" to be replaced with names ending in "service.py"
explanation:
In the sed cmd, the -n flag will suppress normal behavior of echoing input to output after the s/// command is applied, and the p option on s/// will force writing to output if a substitution is made. Since a sub will only be made on match, sed will only have output for files ending in "factory.py"
In the s/// replacement string, we use "& " to interpolate the entire matching string, followed by a space character, into the replacement. Because of this, it's vital that our RE matches the entire filename. after the space char, we use "\1service.py" to interpolate the string we gulped before "factory.py", followed by "service.py", replacing it. So for more complex transformations youll have to change the args to s/// (with an re still matching the entire filename)
Example output:
foo_factory.py foo_service.py
bar_factory.py bar_service.py
We use xargs with -n 2 to consume the output of sed 2 delimited strings at a time, passing these to mv (i also put the -p option in there so you can feel safe when running this). voila.
NOTE: If you are facing more complicated file and folder scenarios, this post explains find (and some alternatives) in greater detail.
Another option is:
for i in *001.jpg
do
echo "mv $i yourstring${i#*001.jpg}"
done
remove echo after you have it right.
Parameter substitution with # will keep only the last part, so you can change its name.
Can't comment on Susam Pal's answer but if you're dealing with spaces, I'd surround with quotes:
for f in *.jpg; do mv "$f" "`echo $f | sed s/\ /\-/g`"; done;
You can try this:
for file in *.jpg;
do
mv $file $somestring_${file:((-7))}
done
You can see "parameter expansion" in man bash to understand the above better.
I have to echo an output of a command substitution concatenated with string. The string to be prepended is in fact the string of pathname. The need for use of absolute path together with filename arises due to filename containing special character,-, at the beginning of it. I've come up with a draft that only works as planned for the first line of output. How do I expand it to other lines as well?
The example scenario is as provided below.
Inside /tmp directory the files are:
-foo 1.txt
-bar 1.txt
Command and the output is:
$ echo "$PWD/$(ls | grep "^-")"
/tmp/-bar 1.txt
-foo 1.txt
While I want it to be like
/tmp/-bar 1.txt
/tmp/-foo 1.txt
I read about this brace expansion feature but I'm not sure if it works for variables, or command substitution for that matter, as stated here. I also want the separate lines for each files and the filename words unsplitted, which is suggested at when the brace expansion is carried out. (Honestly, I don't understand much of the literature about the features such as brace expansion!)
Also, are there other more convenient ways to do this? Any help is appreciated.
To do what you're asking, getting a list of full paths for files starting with -, you can use readlink:
$ readlink -f ./-*
/tmp/-bar 1.pdf
/tmp/-foo 1.pdf
However, to do directly what you mentioned in comments (using filenames starting with - as arguments for pdfgrep), you can take advantage of the common convention that -- marks the end of options, so everything after it is recognized as a filename:
pdfgrep 'pattern' -- -*
See also POSIX Utility Syntax Guidelines (Guideline 10):
The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the - character.
Don't use ls at all. You can use a glob, and your loop can be implicit
$ printf '%s\n' /tmp/-*
/tmp/-foo 1.txt
/tmp/-bar 1.txt
or explicit
$ for f in /tmp/-*; do echo "$f"; done
/tmp/-foo 1.txt
/tmp/-bar 1.txt
Use this
for f in /tmp/-*
do
echo $f
done
I'm pretty sure I've seen this done before but I can't remember the exact syntax.
Suppose you have a couple of files with different file extensions:
foo.txt
bar.rtf
index.html
and instead of doing something with all of them (cat *), you only want to run a command on 2 of the 3 file extensions.
Can't you do something like this?
cat ${*.txt|*.rtf}
I'm sure there's some find trickery to identify the files first and pipe them to a command, but I think bash supports what I'm talking about without having to do that.
The syntax you want is cat *.{txt,rft}. A comma is used instead of a pipe.
$ echo foo > foo.txt
$ echo bar > bar.rft
$ echo "bar txt" > bar.txt
$ echo "test" > index.html
$ cat *.{txt,rft}
bar txt
foo
bar
$ ls *.{txt,rft}
bar.rft bar.txt foo.txt
But as Anthony Geoghegan said in their answer there's a simpler approach you can use.
Shell globbing is much more basic than regular expressions. If you want to cat all the files which have a .txt or .rtf suffix, you'd simply use:
cat *.txt *.rtf
The glob patterns will be expanded to list all the filenames that match the pattern. In your case, the above command would call the cat command with foo.txt and bar.rtf as its arguments.
Here's a simple way i use to do it using command substitution.
cat $(find . -type f \( -name "*.txt" -o -name "*.rtf" \))
But Anthony Geoghegan's answer is much simpler. I learned from it too.