I have a bash script which executes about 20 commands and for debugging purposes I find myself scrolling through the output a lot.
Unfortunately bash doesn't tell me which part of the output is part of what command.
When I use "set -x" in the script it at least prints some information on what it just executed, but I don't really like the output it generates.
For instance, if I have this script:
#!/bin/bash
set -x
echo "foo"
if [ "$ASD" == "QWE" ] ; then
echo "bar"
fi
I would like the output to be something like this:
echo "foo"
foo
echo "bar"
bar
Or maybe:
echo "foo"
foo
if [ "value_of_ASD" == "QWE" ] ; then
echo "bar"
bar
fi
Instead of printing the commands in bold, highlighting with a color would also be okay. But I don't just want to have "+" characters in front of the commands and I also don't like the if statements showing up like '[' value_of_ASD == QWE ']'.
How can I accomplish that with bash?
At the moment the output looks like this btw:
+ echo foo
foo
+ '[' value_of_ASD == QWE ']'
+ echo bar
bar
Edit:
One idea I had was to write a script that I would source in the very beginning of the main script and then let the sourced script parse the main one. Something like this:
source_me.sh
#!/bin/bash
SCRIPT_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/$(basename $0)"
FORMAT_SET_BOLD='\033[0;1m'
FORMAT_RESET='\033[0m'
cat $SCRIPT_PATH | tail -n "+$START_LINE" | while read line; do
printf "${FORMAT_SET_BOLD}${line}${FORMAT_RESET}\n"
eval "${line}"
done
exit 0;
main.sh
#!/bin/bash
START_LINE=$((LINENO+1)) source ./source_me.sh
echo "Foo"
echo "Bar"
echo "+Hello"
The output in that case is:
echo "Foo"
Foo
echo "Bar"
Bar
echo "+Hello"
+Hello
But this method will fail if I use more complex code that goes over multiple lines (if statements, loops etc):
#!/bin/bash
START_LINE=$((LINENO+1)) source ./source_me.sh
echo "Foo"
if [ "$FOOBAR" == "" ] ; then
echo "Bar"
fi
echo "+Hello"
In this case I get:
echo "Foo"
Foo
if [ "$FOOBAR" == "" ] ; then
./source_me.sh: eval: line 9: syntax error: unexpected end of file
echo "Bar"
Bar
fi
./source_me.sh: eval: line 8: syntax error near unexpected token ´fi'
./source_me.sh: eval: line 8: ´fi'
echo "+Hello"
+Hello
I would like to extend rubo77's answer with a few examples that I think deserve a separate answer:
Plain text prefix
So the basic example is to set PS4 to some plain text, e.g.:
PS4="# "; set -x
Which will result in:
Color & extra line text prefix
But because you can use special characters and ANSI escape codes you can for example add a new line before each new command and print the prefix in a color, e.g.:
PS4="\n\033[1;33m>>>\033[0m "; set -x
Result:
Dynamic color prefix
Finally you can make the command prefix call other programs with each use, which you can use to add a timestamp, e.g.:
# yes, there have to be single quotes below, not double!
PS4='\033[1;34m$(date +%H:%M:%S)\033[0m '; set -x
Result:
You can change the + to a string of your desire by setting PS4, e.g.:
PS4="# "; set -x
Note that set -x has no innate knowledge of your terminal, which would be required to send the correct characters to create bold or coloured text.
If you can capture all output through a single file handle, you can probably do this with a pipe:
$ /path/to/your/script 2>&1 | sed "/^+ /s/.*/$(tput bold)&$(tput sgr0)/"
You might want to man tput for more information about the tool we're using to get bold type.
The viability of this depends on many factors I don't know about your environment. YMMV, may contain nuts.
Note:
You might think it was better to capture stderr separately with a line like this:
$ /path/to/your/script 2> >(sed "/^+ /s/.*/$(tput bold)&$(tput sgr0)/")
Alas, this doesn't behave consistently, because stderr passing through the pipe may not be submitted to the terminal before subsequent stdout from the script itself. Note also that this solution, because it uses process substitution, is a bashism and is not portable to POSIX.
What you are looking for is set -v.
-v prints out the line the interpreter just read.
-x prints out the post-parsing results of the line that the interpreter just read.
file x:
set -vx
foo=1
echo $foo
set +vx
Executing:
$: . x
foo=1
++ foo=1
echo $foo
++ echo 1
1
set +vx
++ set +vx
Related
This code always returns the TRUE answer but the value of $myvar is "null"
if [ $myvar = "true" ]
then
cat << EOF > file1
blabla
EOF
echo $myvar
else
cat << EOF > file1
blublu
EOF
echo $myvar
fi
Where did I make a mistake ?
Many Thanks
Olive
There are multiple obvious errors in the command.
About your if statement:
First of all, square brackets [] are an alternative version of the test command. You want to use no brackets or parentheses ().
When writing oneliner you need to end each part of the command with a semicolon ; or that your shell knows at which point you start a new command.
Also in nearly all cases you want to quote your variables.
This leaves you with the following:
if "$myvar" = "true"; then echo "is true"; echo "some other command"; else echo "is not true"; echo "another command; fi
I cannot say much about the rest of the command, as it is unclear what you are trying to do with all of these EOFs.
never worked with shell scripts before,but i need to in my current task.
So i have to run a command that returns output like this:
awd54a7w6ds54awd47awd refs/heads/SomeInfo1
awdafawe23413f13a3r3r refs/heads/SomeInfo2
a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3
g9reh9wrg69egs7ef987e refs/heads/SomeInfo4
And i need to loop over every line of output get only the "SomeInfo" part and write it to a file in a format like this:
["SomeInfo1","SomeInfo2","SomeInfo3"]
I've tried things like this:
for i in $(some command); do
echo $i | cut -f2 -d"heads/" >> text.txt
done
But i don't know how to format it into an array without using a temporary file.
Sorry if the question is dumb and probably too easy and im sure i can figure it out on my own,but i just don't have the time for it because its just an extra conveniance feature that i personally want to implement.
Try this
# json_encoder.sh
arr=()
while read line; do
arr+=(\"$(basename "$line")\")
done
printf "[%s]" $(IFS=,; echo "${arr[*]}")
And then invoke
./your_command | json_encoder.sh
PS. I personally do this kind of data massaging with Vim.
Using Perl one-liner
$ cat petar.txt
awd54a7w6ds54awd47awd refs/heads/SomeInfo1
awdafawe23413f13a3r3r refs/heads/SomeInfo2
a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3
g9reh9wrg69egs7ef987e refs/heads/SomeInfo4
$ perl -ne ' { /.*\/(.*)/ and push(#res,"\"$1\"") } END { print "[".join(",",#res)."]\n" }' petar.txt
["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]
While you should rarely ever use a script to format json, in your case you are simply parsing output into a comma-separated line with added end-caps of [...]. You can use bash parameter expansion to avoid spawning any additional subshells to obtain the last field of information in each line as follows:
#!/bin/bash
[ -z "$1" -o ! -r "$1" ] && { ## validate file given as argument
printf "error: file doesn't exist or not readable.\n" >&2
exit 1
}
c=0 ## simple flag variable
while read -r line; do ## read each line
if [ "$c" -eq '0' ]; then ## is flag 0?
printf "[\"%s\"" "${line##*/}" ## output ["last"
else ## otherwise
printf ",\"%s\"" "${line##*/}" ## output ,"last"
fi
c=1 ## set flag 1
done < file ## redirect file to loop
echo "]" ## append closing ]
Example Use/Output
Using your given data as the input file, you would get the following:
$ bash script.sh file
["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]
Look things over and let me know if you have any questions.
You can also use awk without any loops I guess:
cat prev_output | awk -v ORS=',' -F'/' '{print "\042"$3"\042"}' | \
sed 's/^/[/g ; s/,$/]\n/g' > new_output
cat new_output
["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]
In the following example
for f in *
do
res=$(ls -la "$f")
echo "$res"
done
Is this the correct way to use quotes? I know I should always quote the variable, but is the subshell output implicitly quoted?
The output from command substitution ($()) is not implicitly quoted:
$ for i in $(echo "foo bar"); do echo $i; done
foo
bar
The loop above splits the unquoted output along words. We can prevent this behavior by quoting the result:
$ for i in "$(echo "foo bar")"; do echo $i; done
foo bar
However, when assigning a variable, as in your example, the result of the subshell is not split, even without quotes:
$ baz=$(echo "foo bar")
$ echo "$baz"
foo bar
Unlike StackOverflow's syntax highlighting, the shell understands quotes inside command substitution, so we don't need to escape nested quotes:
$ baz="foo"
$ echo "$(echo "$baz $(echo "bar")")"
foo bar
I've got two sh files, which are "main.sh" and "sub.sh" i want to return a variable's value inside "sub.sh" and use it inside main.sh .There is so many "echo" command so i can't just return the value from sub.sh file. I need only one variable's value. How can that be possible?
main.sh
echo "start"
//how to get a variable from the sh below?
//dene=$(/root/sub.sh)
echo "finish"
sub.sh
echo "sub function"
a="get me out of there" // i want to return that variable from script
echo "12345"
echo "kdsjfkjs"
To "send" the variable, do this:
echo MAGIC: $a
To "receive" it:
dene=$(./sub.sh | sed -n 's/^MAGIC: //p')
What this does is to discard all lines that don't start with MAGIC: and print the part after that token when a match is found. You can substitute your own special word instead of MAGIC.
Edit: or you could do it by "source"ing the sub-script. That is:
source sub.sh
dene=$a
What that does is to run sub.sh in the context of main.sh, as if the text were just copy-pasted right in. Then you can access the variables and so on.
main.sh
#!/bin/sh
echo "start"
# Optionally > /dev/null to suppress output of script
source /root/sub.sh
# Check if variable a is defined and contains sth. and print it if it does
if [ -n "${a}" ]; then
# Do whatever you want with a at this point
echo $a
fi
echo "finish"
sub.sh
#!/bin/sh
echo "sub function"
a="get me out of there"
echo "12345"
echo -e "kdsjfkjs"
exit 42
You can export variable to shell session in sub.sh and catch it later in main.sh.
sub.sh
#!/usr/bin/sh
export VARIABLE="BLABLABLA"
main.sh
#!/bin/sh
. ./sub.sh
echo $VARIABLE
I have bash script where i have echo before every command showing what is happening.
But i need to disbale echo when setting as cron job and then enable again if do some testing.
i find it very hard to go to each line and then add/remove comment
is there anything which i can include at top something like
enable echo or disable echo
so that i don't have to waste time
The absolute easiest would be to insert the following line after the hashbang line:
echo() { :; }
When you want to re-enable, either delete the line or comment it out:
#echo() { :; }
If you're not using echo but printf, same strategy, i.e.:
printf() { :; }
If you absolutely need to actually echo/printf something, prepend the builtin statement, e.g.:
builtin echo "This 'echo' will not be suppressed."
This means that you can do a conditional output, e.g.:
echo () {
[[ "$SOME_KIND_OF_FLAG" ]] && builtin echo $#
}
Set the SOME_KIND_OF_FLAG variable to something non-null, and the overridden echo function will behave like normal echo.
EDIT: another alternative would be to use echo for instrumenting (debugging), and printf for the outputs (e.g., for piping purposes). That way, no need for any FLAG. Just disable/enable the echo() { :; } line according to whether you want to instrument or not, respectively.
Enable/Disable via CLI Parameter
Put these lines right after the hashbang line:
if [[ debug == "$1" ]]; then
INSTRUMENTING=yes # any non-null will do
shift
fi
echo () {
[[ "$INSTRUMENTING" ]] && builtin echo $#
}
Now, invoking the script like this: script.sh debug will turn on instrumenting. And because there's the shift command, you can still feed parameters. E.g.:
Without instrumenting: script.sh param1 param2
With instrumenting: script.sh debug param1 param2
The above can be simplified to:
if [[ debug != "$1" ]]; then
echo () { :; }
shift
fi
if you need the instrumenting flag (e.g. to record the output of a command to a temp file only if debugging), use an else-block:
if [[ debug != "$1" ]]; then
echo () { :; }
shift
else
INSTRUMENTING=yes
fi
REMEMBER: in non-debug mode, all echo commands are disabled; you have to either use builtin echo or printf. I recommend the latter.
Several things:
Don't use echo at all
Instead use set -xv to set debug mode which will echo each and every command. You can set PS4 to the desired prompt: for example PS4='$LINENO: ' will print out the line number on each line. In BASH, I believe it's the same. Then, you don't have to clean up your script. To shut off, use set +xv.
Example:
foo=7
bar=7
PS4='$LINENO: '
set -xv #Begin debugging
if [ $foo = $bar ]
then
echo "foo certainly does equal bar"
fi
set +xv #Debugging is off
if [ $bar = $foo ]
then
echo "And bar also equals foo"
fi
Results:
$ myprog.sh
if [ $foo = $bar ]
then
echo "foo certainly does equal bar"
fi
5: [ 7 = 7 ]
7: echo 'foo certainly does equal bar'
foo certainly does equal bar
set +xv #Debugging is off
And bar also equals foo
Use a function
Define a function instead of using echo:
Example:
function myecho {
if [ ! -z "$DEBUG" ]
then
echo "$*"
fi
}
DEBUG="TRUE"
my echo "Will print out this line"
unset DEBUG
myecho "But won't print out this line"
Use the nop command
The colon (:) is the nop command in BASH. It doesn't do anything. Use an environment variable and define it as either echo or :. When set to a colon, nothing happens. When set to echo, the line prints.
Example:
echo=":"
$echo "This line won't print"
echo="echo"
$echo "But this line will."
Building on Matthew's answer, how about something like this:
myEcho = "/bin/true"
if [ ! "$CRON" ]: then
myEcho = "/bin/echo"
fi
and then use $myEcho instead of echo in your script?
You can do one better. If you setup your crontab as detailed in another answer, you can then check if you are running in cron and only print if you are not. This way you don't need to modify your script at all between different runs.
You should then be able to use something like this (probably doesn't quite work, I'm not proficient in bash):
if [ ! "$CRON" ]; then
echo "Blah blah"
fi
Try set -v at the top to echo each command. To stop echoing change it to set +v.
Not sure if I miss the below solution to use a variable (e.g. debug) at the start of the bash script.
Once you set the debug=true, any conditional-if will enable or disable multiple “echo statements” in bash script.
typeset debug=false # set to true if need to debug
...
if [ $debug == "true" ]; then
echo
echo "Filter"
read
fi
...
if [ $debug == "true" ]; then
echo
echo "to run awk"
fi
Couldn't post a code block in a comment, so I'll post this as an answer.
If you're a perfectionist (like I am) and don't want the last set +x line to be printed... and instead print Success or FAIL, this works:
(
set -e # Stop at first error
set -x # Print commands
set -v # Print shell input lines as they are read
git pull
// ...other commands...
) && echo Success || echo FAIL
It will create a sub process, though, which may be an overkill solution.
If you're running it in cron, why not just dump the output? Change your crontab entry so that it has > /dev/null at the end of the command, and all output will be ignored.