I've written a bash script which install 3 packages from source. The script is fairly simple with the ./configure, make, make install statements written thrice (after cding into the source folder). To make it look a little cleaner, I have redirected the output to another file like so:./configure >> /usr/local/the_packages/install.log 2>&1.
The question is, if any one package fails to compile due to some reason (I'm not even sure of what reason, because it has always run successfully till now - this is just something I want to add), I'd like to terminated the script and rollback.
I figure rolling back would simply be deleting the destination folders specified in prefix=/install/path but how do I terminate the script itself?
Perhaps something like this could work:
./configure && make && make install || rm -rf /install/path
Option 1
You can check the return code of something run from a script with the $? bash variable.
moo#cow:~$ false
moo#cow:~$ echo $?
1
moo#cow:~$ true
moo#cow:~$ echo $?
0
Option 2
You can also check the return code by directly putting the command into a if statement like so.
moo#cow:~$ if echo a < bad_command; then echo "success"; else echo "fail"; fi
fail
Invert the return code
The return code of a command can be inverted with the ! character.
moo#cow:~$ if ! echo a < bad_command; then echo "success"; else echo "fail"; fi
success
Example Script
Just for fun, I decided to write this script based on your question.
#!/bin/bash
_installed=()
do_rollback() {
echo "rolling back..."
for i in "${_installed[#]}"; do
echo "removing $i"
rm -rf "$i"
done
}
install_pkg() {
local _src_dir="$1"
local _install_dir="$2"
local _prev_dir="$PWD"
local _res=0
# Switch to source directory
cd "$_src_dir"
# Try configuring
if ! ./configure --prefix "$_install_dir"; then
echo "error: could not configure pkg in $_src_dir"
do_rollback
exit 1
fi
# Try making
if ! make; then
echo "error: could not make pkg in $_src_dir"
do_rollback
exit 1
fi
# Try installing
if ! make install; then
echo "error: could not install pkg from $_src_dir"
do_rollback
exit 1
fi
# Update installed array
echo "installed pkg from $_src_dir"
_installed=("${_installed[#]}" "$_install_dir")
# Restore previous directory
cd "$_prev_dir"
}
install_pkg /my/source/directory1 /opt/install/dir1
install_pkg /my/source/directory2 /opt/install/dir2
install_pkg /my/source/directory3 /opt/install/dir3
In two parts:
To make the script abort as soon as any command returns an error, you want to use set -e. From the man page (BUILTINS section; description of the set builtin):
-e
Exit immediately if a pipeline (which may consist of a single simple command),
a subshell command enclosed in parentheses, or one of the commands executed as
part of a command list enclosed by braces (see SHELL GRAMMAR above) exits with
a non-zero status. The shell does not exit if the command that fails is part of
the command list immediately following a while or until keyword, part of the
test following the if or elif reserved words, part of any command executed in a
&& or ││ list except the command following the final && or ││, any command in a
pipeline but the last, or if the command's return value is being inverted with
!. A trap on ERR, if set, is executed before the shell exits. This option
applies to the shell environment and each subshell environment separately (see
COMMAND EXECUTION ENVIRONMENT above), and may cause subshells to exit before
executing all the commands in the subshell.
You can set this in three ways: Chang your shebang line to #!/bin/bash -e; call the script as bash -e scriptname; or simply use set -e near the top of your script.
The second part of the question is (to paraphrase) how to catch the exit and clean up before exiting. The answer is referenced above - You want to set a trap on ERR.
To show you how these work together, here's a simple script being run. Note that as soon as we have a non-zero exit code, execution transfers to the signal handler which takes care of doing the cleanup:
james#bodacious:tmp$cat test.sh
#!/bin/bash -e
cleanup() {
echo I\'m cleaning up!
}
trap cleanup ERR
echo Hello
false
echo World
james#bodacious:tmp$./test.sh
Hello
I'm cleaning up!
james#bodacious:tmp$
Related
This question already has answers here:
Aborting a shell script if any command returns a non-zero value
(10 answers)
Closed 3 years ago.
I've been writing some shell script and I would find it useful if there was the ability to halt the execution of said shell script if any of the commands failed. See below for an example:
#!/bin/bash
cd some_dir
./configure --some-flags
make
make install
So in this case, if the script can't change to the indicated directory, then it would certainly not want to do a ./configure afterwards if it fails.
Now I'm well aware that I could have an if check for each command (which I think is a hopeless solution), but is there a global setting to make the script exit if one of the commands fails?
Use the set -e builtin:
#!/bin/bash
set -e
# Any subsequent(*) commands which fail will cause the shell script to exit immediately
Alternatively, you can pass -e on the command line:
bash -e my_script.sh
You can also disable this behavior with set +e.
You may also want to employ all or some of the the -e -u -x and -o pipefail options like so:
set -euxo pipefail
-e exits on error, -u errors on undefined variables, and -o (for option) pipefail exits on command pipe failures. Some gotchas and workarounds are documented well here.
(*) Note:
The shell does not exit if the command that fails is part of the
command list immediately following a while or until keyword,
part of the test following the if or elif reserved words, part
of any command executed in a && or || list except the command
following the final && or ||, any command in a pipeline but
the last, or if the command's return value is being inverted with
!
(from man bash)
To exit the script as soon as one of the commands failed, add this at the beginning:
set -e
This causes the script to exit immediately when some command that is not part of some test (like in a if [ ... ] condition or a && construct) exits with a non-zero exit code.
Use it in conjunction with pipefail.
set -e
set -o pipefail
-e (errexit): Abort the script at the first error, when a command exits with non-zero status (except in until or while loops, if-tests, and list constructs)
-o pipefail: Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value.
Chapter 33. Options
Here is how to do it:
#!/bin/sh
abort()
{
echo >&2 '
***************
*** ABORTED ***
***************
'
echo "An error occurred. Exiting..." >&2
exit 1
}
trap 'abort' 0
set -e
# Add your script below....
# If an error occurs, the abort() function will be called.
#----------------------------------------------------------
# ===> Your script goes here
# Done!
trap : 0
echo >&2 '
************
*** DONE ***
************
'
An alternative to the accepted answer that fits in the first line:
#!/bin/bash -e
cd some_dir
./configure --some-flags
make
make install
One idiom is:
cd some_dir && ./configure --some-flags && make && make install
I realize that can get long, but for larger scripts you could break it into logical functions.
I think that what you are looking for is the trap command:
trap command signal [signal ...]
For more information, see this page.
Another option is to use the set -e command at the top of your script - it will make the script exit if any program / command returns a non true value.
One point missed in the existing answers is show how to inherit the error traps. The bash shell provides one such option for that using set
-E
If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.
Adam Rosenfield's answer recommendation to use set -e is right in certain cases but it has its own potential pitfalls. See GreyCat's BashFAQ - 105 - Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?
According to the manual, set -e exits
if a simple commandexits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in a if statement, part of an && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted via !".
which means, set -e does not work under the following simple cases (detailed explanations can be found on the wiki)
Using the arithmetic operator let or $((..)) ( bash 4.1 onwards) to increment a variable value as
#!/usr/bin/env bash
set -e
i=0
let i++ # or ((i++)) on bash 4.1 or later
echo "i is $i"
If the offending command is not part of the last command executed via && or ||. For e.g. the below trap wouldn't fire when its expected to
#!/usr/bin/env bash
set -e
test -d nosuchdir && echo no dir
echo survived
When used incorrectly in an if statement as, the exit code of the if statement is the exit code of the last executed command. In the example below the last executed command was echo which wouldn't fire the trap, even though the test -d failed
#!/usr/bin/env bash
set -e
f() { if test -d nosuchdir; then echo no dir; fi; }
f
echo survived
When used with command-substitution, they are ignored, unless inherit_errexit is set with bash 4.4
#!/usr/bin/env bash
set -e
foo=$(expr 1-1; true)
echo survived
when you use commands that look like assignments but aren't, such as export, declare, typeset or local. Here the function call to f will not exit as local has swept the error code that was set previously.
set -e
f() { local var=$(somecommand that fails); }
g() { local var; var=$(somecommand that fails); }
When used in a pipeline, and the offending command is not part of the last command. For e.g. the below command would still go through. One options is to enable pipefail by returning the exit code of the first failed process:
set -e
somecommand that fails | cat -
echo survived
The ideal recommendation is to not use set -e and implement an own version of error checking instead. More information on implementing custom error handling on one of my answers to Raise error in a Bash script
I am writing shell script to install my application. I have more number of commands in my script such as copy, unzip, move, if and so on. I want to know the error if any of the commands fails. Also I don't want to send exit codes other than zero.
Order of script installation(root-file.sh):-
./script-to-install-mongodb
./script-to-install-jdk8
./script-to-install-myapplicaiton
Sample script file:-
cp sourceDir destinationDir
unzip filename
if [ true]
// success code
if
I want to know by using variable or any message if any of my scripts command failed in root-file.sh.
I don't want to write code to check every command status. Sometimes cp or mv command may fail due to invalid directory. At the end of script execution, I want to know all commands were executed successfully or error in it?
Is there a way to do it?
Note: I am using shell script not bash
/* the status of your last command stores in special variable $?, you can define variable for $? doing export var=$? */
unzip filename
export unzipStatus=$?
./script1.sh
export script1Status=$?
if [ !["$unzipStatus" || "$script1Status"]]
then
echo "Everything successful!"
else
echo "unsuccessful"
fi
Well as you are using shell script to achieve this there's not much external tooling. So the default $? should be of help. You may want to check for retrieval value in between the script. The code will look like this:
./script_1
retval=$?
if $retval==0; then
echo "script_1 successfully executed ..."
continue
else;
echo "script_1 failed with error exit code !"
break
fi
./script_2
Lemme know if this added any value to your scenario.
Exception handling in linux shell scripting can be done as follows
command || fallback_command
If you have multiple commands then you can do
(command_one && command_two) || fallback_command
Here fallback_command can be an echo or log details in a file etc.
I don't know if you have tried putting set -x on top of your script to see detailed execution.
Want to give my 2 cents here. Run your shell like this
sh root-file.sh 2> errors.txt
grep patterns from errors.txt
grep -e "root-file.sh: line" -e "script-to-install-mongodb.sh: line" -e "script-to-install-jdk8.sh: line" -e "script-to-install-myapplicaiton.sh: line" errors.txt
Output of above grep command will display commands which had errors in it along with line no. Let say output is:-
test.sh: line 8: file3: Permission denied
You can just go and check line no.(here it is 8) which had issue. refer this go to line no. in vi.
or this can also be automated: grep specific line from your shell script. grep line with had issue here it is 8.
head -8 test1.sh |tail -1
hope it helps.
This question already has answers here:
Pass commands as input to another command (su, ssh, sh, etc)
(3 answers)
Closed 6 years ago.
I'm very very new to Linux(coming from windows) and trying to write a script that i can hopefully execute over multiple systems. I tried to use Python for this but fount it hard too. Here is what i have so far:
cd /bin
bash
source compilervars.sh intel64
cd ~
exit #exit bash
file= "~/a.out"
if[! -f "$file"]
then
icc code.c
fi
#run some commands here...
The script hangs in the second line (bash). I'm not sure how to fix that or if I'm doing it wrong. Please advice.
Also, any tips of how to run this script over multiple systems on the same network?
Thanks a lot.
What I believe you'd want to do:
#!/bin/bash
source /bin/compilervars.sh intel64
file="$HOME/a.out"
if [ ! -f "$file" ]; then
icc code.c
fi
You would put this in a file and make it executable with chmod +x myscript. Then you would run it with ./myscript. Alternatively, you could just run it with bash myscript.
Your script makes little sense. The second line will open a new bash session, but it will just sit there until you exit it. Also, changing directories back and forth is very seldom required. To execute a single command in another directory, one usually does
( cd /other/place && mycommand )
The ( ... ) tells the shell that you'd like to do this in a sub-shell. The cd happens within that sub-shell and you don't have to cd back after it's done. If the cd fails, the command will not be run.
For example: You might want to make sure you're in $HOME when you compile the code:
if [ ! -f "$file" ]; then
( cd $HOME && icc code.c )
fi
... or even pick out the directory name from the variable file and use that:
if [ -f "$file" ]; then
( cd $(dirname "$file") && icc code.c )
fi
Assigning to a variable needs to happen as I wrote it, without spaces around the =.
Likewise, there needs to be spaces after if and inside [ ... ] as I wrote it above.
I also tend to use $HOME rather than ~ in scripts as it's more descriptive.
A shell script isn't a record of key strokes which are typed into a terminal. If you write a script like this:
command1
bash
command2
it does not mean that the script will switch to bash, and then execute command2 in the different shell. It means that bash will be run. If there is a controlling terminal, that bash will show you a prompt and wait for a command to be typed in. You will have to type exit to quit that bash. Only then will the original script then continue with command2.
There is no way to switch a script to a different shell halfway through. There are ways to simulate this. A script can re-execute itself using a different shell. In order to do that, the script has to contain logic to detect that it is being re-executed, so that it can prevent re-executing itself again, and to skip some code that shouldn't be run twice.
In this script, I implemented such a re-execution hack. It consists of these lines:
#
# The #!/bin/sh might be some legacy piece of crap,
# not even up to 1990 POSIX.2 spec. So the first step
# is to look for a better shell in some known places
# and re-execute ourselves with that interpreter.
#
if test x$txr_shell = x ; then
for shell in /bin/bash /usr/bin/bash /usr/xpg4/bin/sh ; do
if test -x $shell ; then
txr_shell=$shell
break
fi
done
if test x$txr_shell = x ; then
echo "No known POSIX shell found: falling back on /bin/sh, which may not work"
txr_shell=/bin/sh
fi
export txr_shell
exec $txr_shell $0 ${#+"$#"}
fi
The txr_shell variable (not a standard variable, my invention) is how this logic detects that it's been re-executed. If the variable doesn't exist then this is the original execution. When we re-execute we export txr_shell so the re-executed instance will then have this environment variable.
The variable also holds the path to the shell; that is used later in the script; it is passed through to a Makefile as the SHELL variable, so that make build recipes use that same shell. In the above logic, the contents of txr_shell don't matter; it's used as Boolean: either it exists or it doesn't.
The programming style in the above code snippet is deliberately coded to work on very old shells. That is why test x$txr_shell = x is used instead of the modern syntax [ -z "$txr_shell" ], and why ${#+"$#"} is used instead of just "$#".
This style is no longer used after this point in the script, because the
rest of the script runs in some good, reasonably modern shell thanks to the re-execution trick.
This question already has answers here:
set -e and short tests
(4 answers)
Closed 9 years ago.
Let's say we have a script like this.
#!/bin/bash
set -e
ls notexist && ls notexist
echo still here
won't exit because of set -e
but
#!/bin/bash
set -e
ls notexist || ls notexist
echo still here
will.
Why?
the bash manual says for set -e:
The shell does not exit if the command that fails is [...]
part of any command executed in a && or || list except the
command following the final && or ||
the dash manual says:
If not interactive, exit immediately if any untested command fails.
The exit status of a command is considered to be explicitly tested
if the command is used to control an if, elif, while, or until;
or if the command is the left hand operand of an “&&” or “||” operator.
for the AND test, the shell will stop early, during the test of the "left hand operand".
because there are still tests, it will consider the entire command to be "tested" and thus will not abort.
for the OR test, the shell has to run all (both) tests, and once the last test fails, it will conclude that there has been an unchecked error and thus will abort.
i agree it's a bit counterintuitive.
Because as Bash manual says regarding set -e:
The shell does not exit if the command that fails is part of the command
list immediately following a while or until keyword, part of the test
following the if or elif reserved words, part of any command executed in a
&& or || list except the command following the final && or ||, any command
in a pipeline but the last, or if the command's return value is being
inverted with !.
The ls notexist || ls notexist command terminates the shell because the second (last) ls notexist exits unsuccessfully. The ls notexist && ls notexist doesn't terminate the shell, because execution of "&& list" is stopped after the first ls notexist fails and the second (last) one is never reached.
BTW, it's easier and more reliable to do such tests using true and false instead of specific commands such as ls notexist.
As part of my build process, I am running a git commit as an execute shell step. However, if there are no changes in the workspace, Jenkins is failing the build. This is because git is returning an error code when there are no changes to commit. I'd like to either abort the build, or just mark it as unstable if this is the case. Any ideas?
To stop further execution when command fails:
command || exit 0
To continue execution when command fails:
command || true
Jenkins is executing shell build steps using /bin/sh -xe by default. -x means to print every command executed. -e means to exit with failure if any of the commands in the script failed.
So I think what happened in your case is your git command exit with 1, and because of the default -e param, the shell picks up the non-0 exit code, ignores the rest of the script and marks the step as a failure. We can confirm this if you can post your build step script here.
If that's the case, you can try to put #!/bin/sh so that the script will be executed without option; or do a set +e or anything similar on top of the build step to override this behavior.
Edited: Another thing to note is that, if the last command in your shell script returns non-0 code, the whole build step will still be marked as fail even with this setup. In this case, you can simply put a true command at the end to avoid that.
Another related question
If there is nothing to push git returns exit status 1. Execute shell build step is marked as failed respectively. You can use OR statement || (double pipe).
git commit -m 'some messasge' || echo 'Commit failed. There is probably nothing to commit.'
That means, execute second argument if first failed (returned exit status > 0). Second command always returns 0. When there is nothing to push (exit status 1 -> execute second command) echo will return 0 and build step continues.
To mark build as unstable you can use post-build step Jenkins Text Finder. It can go through console output, match pattern (your echo) and mark build as unstable.
There is another smooth way to tell Jenkins not to fail.
You can isolate your commit in a build step and set the shell to not fail:
set +e
git commit -m "Bla."
set -e
This answer is correct, but it doesn't specify the || exit 0 or || true goes inside the shell command. Here's a more complete example:
sh "adb uninstall com.example.app || true"
The above will work, but the following will fail:
sh "adb uninstall com.example.app" || true
Perhaps it's obvious to others, but I wasted a lot of time before I realized this.
I was able to get this working using the answer found here:
How to git commit nothing without an error?
git diff --quiet --exit-code --cached || git commit -m 'bla'
https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
if you include a returnStatus: true property, then the shell return is ignored.
Jenkins determines the success/failure of a step by the return value of the step. For the case of a shell, it should be the return of the last value. For both Windows CMD and (POSIX) Bash shells, you should be able to set the return value manually by using exit 0 as the last command.
On the (more general) question in title - to prevent Jenkins from failing you can prevent it from seeing exit code 1. Example for ping:
bash -c "ping 1.2.3.9999 -c 1; exit 0"
And now you can e.g. get output of ping:
output=`bash -c "ping 1.2.3.9999 -c 1; exit 0"`
Of course instead of ping ... You can use any command(s) - including git commit.
You can use the Text-finder Plugin. It will allow you to check the output console for an expression of your choice then mark the build as Unstable.
For multiple shell commands, I ignores the failures by adding:
set +e
commands
true
If you put this commands into shell block:
false
true
your build will be marked as fail ( at least 1 non-zero exit code ), so you can add (set +e) to ignore it:
set +e
false
true
will not fail. However, this will fail even with the (set +e) in place:
set +e
false
because the last shell command must exit with 0.
The following works for mercurial by only committing if there are changes. So the build only fails if the commit fails.
hg id | grep "+" || exit 0
hg commit -m "scheduled commit"
Another one answer with some tips, can be helpful for somebody:
remember to separate your commands with the following rule:
command1 && command2 - means, that command2 will be executed, only if command1 success
command1 ; command2 - means, that command 2 will be executed despite on result of command1
for example:
String run_tests = sh(script: "set +e && cd ~/development/tests/ && gmake test ;set -e;echo 0 ", returnStdout: true).trim()
println run_tests
will be executed successfully with set -e and echo 0 commands if gmake test failed (your tests failed), while the following code snipped:
String run_tests = sh(script: "set +e && cd ~/development/tests/ && gmake test && set -e && echo 0 ", returnStdout: true).trim()
println run_tests
a bit wrong and commands set -e and echo 0 in&& gmake test && set -e && echo 0 will be skipped, with the println run_tests statement, because failed gmake test will abort the jenkins build. As workaround you can switch to returnStatus:true, but then you will miss the output from your command.