I was trying to delete some files in a bash script and exclude some I still need afterwards. This is what I came up with:
if [ $var1 -eq 0 ]; then
echo "$var1 = 0"
shopt -s extglob
rm /home/someone/!(file1.txt|file2.png|file3.pdf|file4.csv)
else
echo "$var1 = 1"
shopt -s extglob
rm -rf /home/someone/!(file1.txt|file2.png|file3.pdf|file4.csv)
fi
It is working if I execute it manually on the shell, but in the script it does not even call the if-loop. As soon as I comment out the lines containing the "!" it works.
You cannot turn the extglob shell option from within a block or a function. E.g., this will fail:
shopt -u extglob
f() {
shopt -s extglob
echo *+(a)*
}
f
The reason is that the command shopt -s extglob is not executed when the block is parsed… and the parser will hence complain when it encounters the extended glob +(a). See the relevant section of the glob page on Greg's wiki:
extglob changes the way certain characters are parsed. It is necessary to have a newline (not just a semicolon) between shopt -s extglob and any subsequent commands to use it. You cannot enable extended globs inside a group command that uses them, because the entire block is parsed before the shopt is evaluated. Note that the typical function body is a group command. An unpleasant workaround could be to use a subshell command list as the function body.
If you really want this behavior, though, you can use eval, but that's really not recommended at all! Instead, move the shopt -s extglob out of the block. In fact, it is customary to put the shell options at the beginning of the script and use them throughout the script (when applicable). I don't think you'll run into any problems if you use extglob throughout the script.
As #choroba is hinting towards, you need to run your script either as bash my_script.sh or ./my_script.sh to run it with Bash. If you run sh my_script.sh the shell may not support extglob.
A useful way to tell what's actually happening is to add set -o xtrace to the top of the script (after the shebang line). That should tell you what actually gets executed. I'd also add set -o errexit to make sure the script stops at the first failing command, and quote the reference to $var1.
Related
I have a line of code that works fine in my terminal:
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
Then I put the exact same line of code in a script myscript.sh:
#!/bin/sh
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
However, now I get an error when running it:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
Based on other questions I tried changing the shebang to #!/bin/bash, but I get the exact same error. Why can't I run this script?
TL;DR: Since you are using Bash specific features, your script has to run with Bash and not with sh:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
$ bash myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
See Difference between sh and Bash. To find out which sh you are using: readlink -f $(which sh).
The best way to ensure a bash specific script always runs correctly
The best practices are to both:
Replace #!/bin/sh with #!/bin/bash (or whichever other shell your script depends on).
Run this script (and all others!) with ./myscript.sh or /path/to/myscript.sh, without a leading sh or bash.
Here's an example:
$ cat myscript.sh
#!/bin/bash
for i in *.mp4
do
echo ffmpeg -i "$i" "${i/.mp4/.mp3}"
done
$ chmod +x myscript.sh # Ensure script is executable
$ ./myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
(Related: Why ./ in front of scripts?)
The meaning of #!/bin/sh
The shebang suggests which shell the system should use to run a script. This allows you to specify #!/usr/bin/python or #!/bin/bash so that you don't have to remember which script is written in what language.
People use #!/bin/sh when they only use a limited set of features (defined by the POSIX standard) for maximum portability. #!/bin/bash is perfectly fine for user scripts that take advantage of useful bash extensions.
/bin/sh is usually symlinked to either a minimal POSIX compliant shell or to a standard shell (e.g. bash). Even in the latter case, #!/bin/sh may fail because bash will run in compatibility mode as explained in the man page:
If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well.
The meaning of sh myscript.sh
The shebang is only used when you run ./myscript.sh, /path/to/myscript.sh, or when you drop the extension, put the script in a directory in your $PATH, and just run myscript.
If you explicitly specify an interpreter, that interpreter will be used. sh myscript.sh will force it to run with sh, no matter what the shebang says. This is why changing the shebang is not enough by itself.
You should always run the script with its preferred interpreter, so prefer ./myscript.sh or similar whenever you execute any script.
Other suggested changes to your script:
It is considered good practice to quote variables ("$i" instead of $i). Quoted variables will prevent problems if the stored file name contains white space characters.
I like that you use advanced parameter expansion. I suggest to use "${i%.mp4}.mp3" (instead of "${i/.mp4/.mp3}"), since ${parameter%word} only substitutes at the end (for example a file named foo.mp4.backup).
The ${var/x/y/} construct is not POSIX. In your case, where you just remove a string at the end of a variable and tack on another string, the portable POSIX solution is to use
#!/bin/sh
for i in *.mp4; do
ffmpeg -i "$i" "${i%.mp4}.mp3"
done
or even shorter, ffmpeg -i "$i" "${i%4}3".
The definitive dope for these constructs is the chapter on Parameter Expansion for the POSIX shell.
I have a line of code that works fine in my terminal:
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
Then I put the exact same line of code in a script myscript.sh:
#!/bin/sh
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
However, now I get an error when running it:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
Based on other questions I tried changing the shebang to #!/bin/bash, but I get the exact same error. Why can't I run this script?
TL;DR: Since you are using Bash specific features, your script has to run with Bash and not with sh:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
$ bash myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
See Difference between sh and Bash. To find out which sh you are using: readlink -f $(which sh).
The best way to ensure a bash specific script always runs correctly
The best practices are to both:
Replace #!/bin/sh with #!/bin/bash (or whichever other shell your script depends on).
Run this script (and all others!) with ./myscript.sh or /path/to/myscript.sh, without a leading sh or bash.
Here's an example:
$ cat myscript.sh
#!/bin/bash
for i in *.mp4
do
echo ffmpeg -i "$i" "${i/.mp4/.mp3}"
done
$ chmod +x myscript.sh # Ensure script is executable
$ ./myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
(Related: Why ./ in front of scripts?)
The meaning of #!/bin/sh
The shebang suggests which shell the system should use to run a script. This allows you to specify #!/usr/bin/python or #!/bin/bash so that you don't have to remember which script is written in what language.
People use #!/bin/sh when they only use a limited set of features (defined by the POSIX standard) for maximum portability. #!/bin/bash is perfectly fine for user scripts that take advantage of useful bash extensions.
/bin/sh is usually symlinked to either a minimal POSIX compliant shell or to a standard shell (e.g. bash). Even in the latter case, #!/bin/sh may fail because bash will run in compatibility mode as explained in the man page:
If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well.
The meaning of sh myscript.sh
The shebang is only used when you run ./myscript.sh, /path/to/myscript.sh, or when you drop the extension, put the script in a directory in your $PATH, and just run myscript.
If you explicitly specify an interpreter, that interpreter will be used. sh myscript.sh will force it to run with sh, no matter what the shebang says. This is why changing the shebang is not enough by itself.
You should always run the script with its preferred interpreter, so prefer ./myscript.sh or similar whenever you execute any script.
Other suggested changes to your script:
It is considered good practice to quote variables ("$i" instead of $i). Quoted variables will prevent problems if the stored file name contains white space characters.
I like that you use advanced parameter expansion. I suggest to use "${i%.mp4}.mp3" (instead of "${i/.mp4/.mp3}"), since ${parameter%word} only substitutes at the end (for example a file named foo.mp4.backup).
The ${var/x/y/} construct is not POSIX. In your case, where you just remove a string at the end of a variable and tack on another string, the portable POSIX solution is to use
#!/bin/sh
for i in *.mp4; do
ffmpeg -i "$i" "${i%.mp4}.mp3"
done
or even shorter, ffmpeg -i "$i" "${i%4}3".
The definitive dope for these constructs is the chapter on Parameter Expansion for the POSIX shell.
I have a line of code that works fine in my terminal:
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
Then I put the exact same line of code in a script myscript.sh:
#!/bin/sh
for i in *.mp4; do echo ffmpeg -i "$i" "${i/.mp4/.mp3}"; done
However, now I get an error when running it:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
Based on other questions I tried changing the shebang to #!/bin/bash, but I get the exact same error. Why can't I run this script?
TL;DR: Since you are using Bash specific features, your script has to run with Bash and not with sh:
$ sh myscript.sh
myscript.sh: 2: myscript.sh: Bad substitution
$ bash myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
See Difference between sh and Bash. To find out which sh you are using: readlink -f $(which sh).
The best way to ensure a bash specific script always runs correctly
The best practices are to both:
Replace #!/bin/sh with #!/bin/bash (or whichever other shell your script depends on).
Run this script (and all others!) with ./myscript.sh or /path/to/myscript.sh, without a leading sh or bash.
Here's an example:
$ cat myscript.sh
#!/bin/bash
for i in *.mp4
do
echo ffmpeg -i "$i" "${i/.mp4/.mp3}"
done
$ chmod +x myscript.sh # Ensure script is executable
$ ./myscript.sh
ffmpeg -i bar.mp4 bar.mp3
ffmpeg -i foo.mp4 foo.mp3
(Related: Why ./ in front of scripts?)
The meaning of #!/bin/sh
The shebang suggests which shell the system should use to run a script. This allows you to specify #!/usr/bin/python or #!/bin/bash so that you don't have to remember which script is written in what language.
People use #!/bin/sh when they only use a limited set of features (defined by the POSIX standard) for maximum portability. #!/bin/bash is perfectly fine for user scripts that take advantage of useful bash extensions.
/bin/sh is usually symlinked to either a minimal POSIX compliant shell or to a standard shell (e.g. bash). Even in the latter case, #!/bin/sh may fail because bash will run in compatibility mode as explained in the man page:
If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well.
The meaning of sh myscript.sh
The shebang is only used when you run ./myscript.sh, /path/to/myscript.sh, or when you drop the extension, put the script in a directory in your $PATH, and just run myscript.
If you explicitly specify an interpreter, that interpreter will be used. sh myscript.sh will force it to run with sh, no matter what the shebang says. This is why changing the shebang is not enough by itself.
You should always run the script with its preferred interpreter, so prefer ./myscript.sh or similar whenever you execute any script.
Other suggested changes to your script:
It is considered good practice to quote variables ("$i" instead of $i). Quoted variables will prevent problems if the stored file name contains white space characters.
I like that you use advanced parameter expansion. I suggest to use "${i%.mp4}.mp3" (instead of "${i/.mp4/.mp3}"), since ${parameter%word} only substitutes at the end (for example a file named foo.mp4.backup).
The ${var/x/y/} construct is not POSIX. In your case, where you just remove a string at the end of a variable and tack on another string, the portable POSIX solution is to use
#!/bin/sh
for i in *.mp4; do
ffmpeg -i "$i" "${i%.mp4}.mp3"
done
or even shorter, ffmpeg -i "$i" "${i%4}3".
The definitive dope for these constructs is the chapter on Parameter Expansion for the POSIX shell.
Running this script, bash ./cleanup.bash,
#!/bin/bash
## Going to directory-moving stuff
rm -rf !(composer.json|.git)
Gives the error:
cleanup.bash: line 10: syntax error near unexpected token '('
cleanup.bash: line 10: 'rm -rf !(composer.json|.git)'
But if I run in in the terminal directly, there aren't any problems:
rm -rf !(composer.json|.git)
I tried stripping out all other lines, but I still get the error.
How do I enter this correctly in the Bash script?
I'm on Ubuntu, and this was all done locally, not on a remote.
I guess your problem is due to the shell extended glob option not set when run from the script. When you claim it works in the command line, you have somehow set the extglob flag which allow to !() globs.
Since the Bash script, whenever started with a #!/bin/bash, starts a new sub-shell, the extended options set in the parent shell may not be reflected in the new shell. To make it take effect, set it in the script after the shebang:
#!/bin/bash
shopt -s extglob
## Going to directory-moving stuff
rm -rf !(composer.json|.git)
I'm trying to replace a extension of a filename considering case but without success.
#!/bin/bash
pdf_file="/root/users/test.pdf"
jpg_file="${pdf_file/.pdf/.jpg}"
echo $jpg_file
I tried it, but it doesn't work:
jpg_file="${pdf_file/(.pdf|.PDF)/.jpg}"
You could use a glob pattern like this:
$ echo "${pdf_file/.[Pp][Dd][Ff]/.jpg}"
/root/users/test.jpg
If you use extended glob patterns (shopt -s extglob), you could use this instead:
$ echo "${pdf_file/.#(PDF|pdf)/.jpg}"
/root/users/test.jpg
Or you could use the shell option to ignore case when matching:
$ shopt -s nocasematch
$ pdf_file="/root/users/test.PDF"
$ echo "${pdf_file/.pdf/.jpg}"
/root/users/test.jpg
Remark valid for all three solutions: ${parameter/pattern/string} replaces the pattern wherever it occurs, but the extension is likely at the end – we can make sure we'll only replace it at the end:
echo "${pdf_file%.[Pp][Dd][Ff]}.jpg"
which works in any POSIX shell, or
shopt -s extglob
echo "${pdf_file%.#(PDF|pdf)}.jpg"
or
shopt -s nocasematch
pdf_file="/root/users/test.PDF"
echo "${pdf_file%.pdf}.jpg"