I am quite new to bash scripting. I am trying to write a line which searches for all files with a particular name, and renames them, except for the currently running script.
Here is what I have so far:
find $1 -name thescript.sh -exec /bin/bash -c 'test \'"$(readlink -f "$0")" = {}\' || mv {} \$\(dirname {}\)/thescript0.sh' \; || true
Here is a brief explanation of how that works in my head:
Determine the absolute path to the currently running script using readlink
For all files {} named thescript.sh and in some subdirectory of $1:
Test if the resulting absolute path is equal to the file {}
If the test fails (i.e. they are not equal):
Determine the directory of the file {}
Move the file {} to a file under the same directory but with name thescript0.sh
Always return true
A few notes:
$1 the first argument to the script is the base directory for the renaming
Because I want to use "or" (||) within an exec clause, I followed some answers to this question and started a new shell with a quoted command within which I could use ||, so that the shell didn't think this operator is part of the parent command. The quoted command starts from test and ends at thescript0.sh
readlink is in a subshell because the result of readlink is passed to test
I used escaped quotes for the test because otherwise I thought the shell will think the first quote is actually the terminator of the previous single quote (which started the quoted command)
I also escaped the subshell, not sure if this is correct (I haven't got that far yet) because I want the subshell to be executed as part of the quoted command based on the output of find, rather than be executed at the beginning (when it won't have an output to work on yet)
If the test failed (which is expected for most cases) then I don't want the whole command to return false (not sure if it would) so that's why I put || true at the end
The error I am getting is:
line 15: unexpected EOF while looking for matching `''
I cannot find any mismatching quotes at the moment; there is only a single pair of unescaped quotes.
Single quotes don't accept any escape sequences, not even \'. To include a single quote inside a single quoted string, you have to leave and re-enter:
echo 'What'"'"'s wrong?'
echo 'Oh, it'\''s nothing.'
Additional notes:
$0 won't be expanded inside of single quotes.
find's exit code is unrelated to the results of -exec, so || true isn't needed.
For maximum safety it's best to pass {} to bash as an argument rather than embedding it in the script. That way whitespace and other special characters won't trip bash up.
Always quote variable expansions in case they contain whitespace or wildcards.
My recommended update with all of the above taken into consideration:
find "$1" -name thescript.sh -exec bash -c 'test "$1" == "$2" || mv "$2" "$(dirname "$2")"/thescript0.sh' bash "$(readlink -f "$0")" {} \;
(I haven't tested it so hopefully you get the gist even if I made some typos.)
Related
I need to accommodate spaces in filepaths. Why does "find" not work from the script, but works from the cli?
MyLaptop$ ./my-bash-script.sh
find: '/Sandbox/test folder/testfiles-good/ResidentFile_1.pcapng': No such file or directory
MyLaptop$ find '/Sandbox/test folder/testfiles-good/ResidentFile_1.pcapng'
/Sandbox/test folder/testfiles-good/ResidentFile_1.pcapng
Using echo find -f "'$line'"
output: find -f '/Sandbox/test folder/testfiles-good/ResidentFile_1.pcapng'
But in this case: FOUND="$(find -f "'$line'")"
it does not
In this case ...
FOUND="$(find -f "'$line'")"
... you are asking find for a file whose name contains leading and trailing ' characters. That is unlikely to be what you intended. Though it may look strange, this is probably what you meant:
FOUND="$(find -f "$line")"
The " characters inside the command substitution do not pair with those outside (and if they did then the original command substitution would be wrong a different way). On the other hand, word splitting is not performed on the result of the command substitution when it is used on the right-hand side of a variable assignment, so you could also just use
FOUND=$(find -f "$line")
I would like to find some file types in pictures folder and I have created the following bash-script in /home/user/pictures folder:
for i in *.pdf *.sh *.txt;
do
echo 'all file types with extension' $i;
find /home/user/pictures -type f -iname $i;
done
But when I execute the bash-script, it does not work as expected for files that are located on the base directory /home/user/pictures. Instead of echo 'All File types with Extension *.sh' the command interprets the variable for base directory:
all file types with extension file1.sh
/home/user/pictures/file1.sh
all file types with extension file2.sh
/home/user/pictures/file2.sh
all file types with extension file3.sh
/home/user/pictures/file3.sh
I would like to know why echo - command does not print "All File types with Extension *.sh".
Revised code:
for i in '*.pdf' '*.sh' '*.txt'
do
echo "all file types with extension $i"
find /home/user/pictures -type f -iname "$i"
done
Explanation:
In bash, a string containing *, or a variable which expands to such a string, may be expanded as a glob pattern unless that string is protected from glob expansion by putting it inside quotes (although if the glob pattern does not match any files, then the original glob pattern will remain after attempted expansion).
In this case, it is not wanted for the glob expansion to happen - the string containing the * needs to be passed as a literal to each of the echo and the find commands. So the $i should be enclosed in double quotes - these will allow the variable expansion from $i, but the subsequent wildcard expansion will not occur. (If single quotes, i.e. '$i' were used instead, then a literal $i would be passed to echo and to find, which is not wanted either.)
In addition to this, the initial for line needs to use quotes to protect against wildcard expansion in the event that any files matching any of the glob patterns exist in the current directory. Here, it does not matter whether single or double quotes are used.
Separately, the revised code here also removes some unnecessary semicolons. Semicolons in bash are a command separator and are not needed merely to terminate a statement (as in C etc).
Observed behaviour with original code
What seems to be happening here is that one of the patterns used in the initial for statement is matching files in the current directory (specifically the *.sh is matching file1.sh file2.sh, and file3.sh). It is therefore being replaced by a list of these filenames (file1.sh file2.sh file3.sh) in the expression, and the for statement will iterate over these values. (Note that the current directory might not be the same as either where the script is located or the top level directory used for the find.)
It would also still be expected that the *.pdf and *.txt would be used in the expression -- either substituted or not, depending on whether any matches are found. Therefore the output shown in the question is probably not the whole output of the script.
Such expressions (*.blabla) changes the value of $i in the loop. Here is the trick i would do :
for i in pdf sh txt;
do
echo 'all file types with extension *.'$i;
find /home/user/pictures -type f -iname '*.'$i;
done
Hi… Need a little help here…
I tried to emulate the DOS' dir command in Linux using bash script. Basically it's just a wrapped ls command with some parameters plus summary info. Here's the script:
#!/bin/bash
# default to current folder
if [ -z "$1" ]; then var=.;
else var="$1"; fi
# check file existence
if [ -a "$var" ]; then
# list contents with color, folder first
CMD="ls -lgG $var --color --group-directories-first"; $CMD;
# sum all files size
size=$(ls -lgGp "$var" | grep -v / | awk '{ sum += $3 }; END { print sum }')
if [ "$size" == "" ]; then size="0"; fi
# create summary
if [ -d "$var" ]; then
folder=$(find $var/* -maxdepth 0 -type d | wc -l)
file=$(find $var/* -maxdepth 0 -type f | wc -l)
echo "Found: $folder folders "
echo " $file files $size bytes"
fi
# error message
else
echo "dir: Error \"$var\": No such file or directory"
fi
The problem is when the argument contains an asterisk (*), the ls within the script acts differently compare to the direct ls command given at the prompt. Instead of return the whole files list, the script only returns the first file. See the video below to see the comparation in action. I don't know why it behaves like that.
Anyone knows how to fix it? Thank you.
Video: problem in action
UPDATE:
The problem has been solved. Thank you all for the answers. Now my script works as expected. See the video here: http://i.giphy.com/3o8dp1YLz4fIyCbOAU.gif
The asterisk * is expanded by the shell when it parses the command line. In other words, your script doesn't get a parameter containing an asterisk, it gets a list of files as arguments. Your script only works with $1, the first argument. It should work with "$#" instead.
This is because when you retrieve $1 you assume the shell does NOT expand *.
In fact, when * (or other glob) matches, it is expanded, and broken into segments by $IFS, and then passed as $1, $2, etc.
You're lucky if you simply retrieved the first file. When your first file's path contains spaces, you'll get an error because you only get the first segment before the space.
Seriously, read this and especially this. Really.
And please don't do things like
CMD=whatever you get from user input; $CMD;
You are begging for trouble. Don't execute arbitrary string from the user.
Both above answers already answered your question. So, i'm going a bit more verbose.
In your terminal is running the bash interpreter (probably). This is the program which parses your input line(s) and doing "things" based on your input.
When you enter some line the bash start doing the following workflow:
parsing and lexical analysis
expansion
brace expansion
tidle expansion
variable expansion
artithmetic and other substitutions
command substitution
word splitting
filename generation (globbing)
removing quotes
Only after all above the bash
will execute some external commands, like ls or dir.sh... etc.,
or will do so some "internal" actions for the known keywords and builtins like echo, for, if etc...
As you can see, the second last is the filename generation (globbing). So, in your case - if the test* matches some files, your bash expands the willcard characters (aka does the globbing).
So,
when you enter dir.sh test*,
and the test* matches some files
the bash does the expansion first
and after will execute the command dir.sh with already expanded filenames
e.g. the script get executed (in your case) as: dir.sh test.pas test.swift
BTW, it acts exactly with the same way for your ls example:
the bash expands the ls test* to ls test.pas test.swift
then executes the ls with the above two arguments
and the ls will print the result for the got two arguments.
with other words, the ls don't even see the test* argument - if it is possible - the bash expands the wilcard characters. (* and ?).
Now back to your script: add after the shebang the following line:
echo "the $0 got this arguments: $#"
and you will immediatelly see, the real argumemts how your script got executed.
also, in such cases is a good practice trying to execute the script in debug-mode, e.g.
bash -x dir.sh test*
and you will see, what the script does exactly.
Also, you can do the same for your current interpreter, e.g. just enter into the terminal
set -x
and try run the dir.sh test* = and you will see, how the bash will execute the dir.sh command. (to stop the debug mode, just enter set +x)
Everbody is giving you valuable advice which you should definitely should follow!
But here is the real answer to your question.
To pass unexpanded arguments to any executable you need to single quote them:
./your_script '*'
The best solution I have is to use the eval command, in this way:
#!/bin/bash
cmd="some command \"with_quetes_and_asterisk_in_it*\""
echo "$cmd"
eval $cmd
The eval command takes its arguments and evaluates them into the command as the shell does.
This solves my problem when I need to call a command with asterisk '*' in it from a script.
It has been a long time since I did much bash script writing.
This is a bash script to copy and rename files by deleting all before the first period delimiter:
#!/bin/bash
mkdir fullname
mv *.audio fullname
cd fullname
for x in * ;
do
cp $x ../`echo $x | cut -d "." -f 2-`
done
cd ..
ls
It works well for file names with no embedded spaces but not for those with spaces.
How can I change the code to fix this simple Linux bash script? Any suggestions for improving the code for other reasons would also be welcome.
Example filenames, some with embedded spaces and some not (from link)
http://www.homenetvideo.com/demo/index.php?/Radio%20%28VLC%29
Ambient.A6.SOMA Space Station.audio
Blues.B9.Blues Radio U.K.audio
Classical.K3.Radio Stephansdom - Vienna.audio
College.CI.KDVS U of California, Davis.audio
Country.Q1.K-FROG.audio
Easy.G4.WNYU.audio
Eclectic.M2.XPN.audio
Electronica.E2.Rinse.audio
Folk.F1.Radionomy.audio
Hiphop.H1.NPR.audio
Indie.I4.WAUG.audio
Jazz.J6.KCSM.audio
Latin.L3.Mega.audio
Misc.X7.Gaydio.audio
News.N9.KQED.audio
Oldies.O1.Lonestar.audio
OldTime.Y1.Roswell.audio
Progressive.P1.Aural Moon.audio
Rock.R8.WXRT.audio
Scanner.Z3.Montreal.audio
Soul.S1.181.FM.audio
Talk.T2.TWiT.audio
World.W3.Persian.audio
http://lh5.googleusercontent.com/-QjLEiAtT4cw/U98_UFcWvvI/AAAAAAAABv8/gyPhbg8s7Bw/w681-h373-no/homenet-radio.png
Whenever you deal with file names that might have spaces in them, you must reference them as "$x" rather than just $x. That's what's causing your cp command to fail.
Your echo command is also problematic. Although echo does the right thing for simple spaces - it echoes a file named A B C as A B C - it will still fail if you have more than one consecutive space in the name, or whitespace that isn't a simple space character.
Instead of passing the file names to external programs for processing, which always requires getting them through the whitespace-hostile command line, you should use bash built-in functions for string manipulations wherever possible, e.g. ${x%%foo}, ${x#bar} and similar functions. The man page describes them under "Parameter expansion".
Here's my suggestion:
#!/bin/bash
shopt -s nullglob
mkdir fullname
mv *.audio fullname
(
cd fullname || exit
for x in *; do
cp "$x" "../${x#*.}"
done
)
ls
nullglob prevents * from presenting itself if no file matches it. Just optional.
() summons a subshell and saves you from changing back to another directory.
|| exit terminates the subshell if cd fails to change directory.
${x#*.} removes the <first>. from $x and expands it.
The script goal is simple.
I have many directory which contains some captured traffic files.
I want to run a command for each directory. So I came up with a script. But I don't know why the script is run only with the first match.
#!/bin/bash
# Collect throughput from a group of directory containing capture files
# Group of directory can be specify by pattern
# Usage: ./collectThroughputList [regex]
# [regex] is the name pattern of the group of directory
for DIR in $( ls -d $1 ); do
if test -d "$DIR"; then
echo Collecting throughputs from directory: "$DIR"
( sh collectThroughput.sh $DIR > $DIR.txt )
fi
done
echo Done\!
I try it with:
for DIR in $1; do
or
for DIR in `ls -d $1`; do
or
for DIR in $( ls -d "$1" ); do
or
for DIR in $( ls -d $1 ); do
But the result is the same. The for loop runs only one time.
Finally I found this one and did some tricks for it to work. However, I would like to know why my first script doesn't work.
find *Delay50ms* -type d -exec bash -c "cd '{}' && echo enter '{}' && ../collectThroughput.sh ../'{}' > ../'{}'.txt" \;
"*Delay*" is the directory pattern name that I want to run the command with.
Thanks for pointing out the issues.
Since you want to find all sub-directories under $1, use it like this:
for DIR in $(find $1 -type d)
Problem
Most probably the problem you are encountering is due to the fact that you are trying to use some kind of pattern like * as argument to your script.
Running it with something like:
my_script *
What's happening here is, that the shell will expand * prior to calling your script.
Thus after word splitting has been performed $1 in your script will just reference the first entry returned by ls.
Example
Given the following directory layout:
directory_a
directory_b
directory_c
Calling my_script * will result in:
my_script directory_a directory_b directory_c
being called thus your loop just iterating over $(ls -d directory_a) which in fact is nothing else but directory_a alone.
Solution
To have the program run with $1=* you would have to escape the * prior to calling your script.
Try running:
my_script \*
To see it effectively does what it is intended to do then. This way $1 in your script will contain * instead of directory_a which most probably is the way you wanted your script to work.
as mikyra has pointed out, the shell expands your argument * to all entries in your directory prior to passing it to your script.
if you want shell-expansion of your wildcards (e.g. * matches all but hidden files), you could simply leave the expansion to the shell and use the result, by iterating over all arguments, rather than just the first one:
for DIR in $#; do
# ...
done
if you want to do the expansion yourself (e.g. because the pattern should be applied only to a pre-filtered list or to files in a different directory, or because you want regex-expansion rather than shell globbing), you have to protect the argument from being expanded by the shell, either using backslash notation (like mikyra's \*) or by using quotes (which is often easier to use):
my_script "*"