The following will output to stdout if the script is run in a terminal:
echo "some message"
If the script is called by another script, where does the output go to? Is there any significant overhead involved?
I'm using GNU bash, version 4.3.33.
Many thanks
The output of
echo "some message"
should go to the stdout except (not exclusive) in cases where
You have a o/p redirection, as given below, which affect the echo statement.
exec 1>/dev/null # 1 is the file descriptor for stdout, this should be before the echo
./script >outfile # The whole output is redirected to a file
You have a do-nothing directive(:) before the echo command
: echo "some message" # Does nothing
Related
I'm trying to redirect a program's stdin and stdout. I'm currently experimenting with a bash mockup of this, but I'm getting some odd behavior.
I have the following:
mkfifo in
mkfifo out
I also have the following script, test.sh
#!/bin/bash
while read line; do
echo "I read ${line}"
done < /dev/stdin
In terminal 1, I do the following:
tail -f out
In terminal 2, I do the following:
./test.sh < in > out
In terminal 3, I do the following:
echo "foo" > in
echo "bar > in
However, instead of seeing "I read foo" followed by "I read bar" in terminal 1, I get nothing after the first echo, both lines after the second echo, and then the test.sh program in terminal 2 exits. How can I prevent the exit so I can keep sending test.sh input? Also, instead of buffering and then dumping when the program terminates, how can I get the output from test.sh to flush to the tail -f in terminal 1?
Use the redirection on a single compound command that contains your two echo commands.
{
echo "foo"
echo "bar"
} > in
If, as seems likely on a closer reading, you want in to stay open while you are executing commands interactively, use exec to open in on another file descriptor:
exec 3> in # Open in on file descriptor 3
echo "foo" >&3 # Write to file descriptor 3 instead of standard output
echo "bar" >&3 # "
exec 3>&- # Close file descriptor 3
Note that exec 3> in will block until something (test.sh in your case) opens in for reading, and due to buffering, you may not see any output from tail -f out until you close file descriptor 3.
Recently in my work, I am facing an interesting problem regarding tee and process substitution.
Let's start with examples:
I have three little scripts:
$ head *.sh
File one.sh
#!/bin/bash
echo "one starts"
if [ -p /dev/stdin ]; then
echo "$(cat /dev/stdin) from one"
else
echo "no stdin"
fi
File two.sh
#!/bin/bash
echo "two starts"
if [ -p /dev/stdin ]; then
echo "$(cat /dev/stdin) from two"
else
echo "no stdin"
fi
File three.sh
#!/bin/bash
echo "three starts"
if [ -p /dev/stdin ]; then
sed 's/^/stdin for three: /' /dev/stdin
else
echo "no stdin"
fi
All three scripts read from standard input and print something to standard output.
The one.sh and two.sh are quite similar, but the three.sh is a bit different. It just adds some prefix to show what it reads from the standard input.
Now I am going to execute two commands:
1: echo "hello" | tee >(./one.sh) >(./two.sh) | ./three.sh
2: echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
First in Bash and then in Z shell (zsh).
Bash (GNU bash, version 5.0.17(1))
$ echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh
three starts
stdin for three: hello
stdin for three: one starts
stdin for three: two starts
stdin for three: hello from two
stdin for three: hello from one
Why are the outputs of one.sh and two.sh mixed with the origin "hello" and passed to three.sh? I expected to see the output of one and two in standard output and only the "hello" is going to pass to three.sh.
Now the other command:
$ echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
one starts
two starts
three starts
stdin for three: hello
hello from two
hello from one
<---!!!note here I don't have prompt unless I press Enter or Ctrl-c)
I redirect all standard output to /dev/null. Why do I see all output from all process substitution this time? Does it seem this behavior conflict with the one above?
Why don't I have the prompt after having executed the command?
Why does the command start in order one->two->three, but outputs come in 3->2->1? Even if I added sleep 3 in three.sh, the output is always 3-2-1. I know it should have something to do with standard input blocking, but I'd learn the exact reason.
Zsh (zsh 5.8 (x86_64-pc-linux-gnu))
Both commands,
echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh
Give the expected result:
one starts
three starts
two starts
hello from two
hello from one
stdin for three: hello
It works as expected. But the order of the output is random, it seems that Z shell does something non-blocking here, and the order of the output is dependent on how long each script has been running. What exactly leads to the result?
echo "hello"|tee >(./one.sh) >(./two.sh) |./three.sh
There are two possible order of operations for the tee part of the pipeline
First
Redirect standard output to a pipe that's connected to ./three.sh's standard input.
Set up the pipes and subprocesses for the command substitutions. They inherit the same redirected standard output pipe used by tee.
Execute tee.
Second
Set up the pipes and subprocesses for the the command substitutions. They share the same default standard output - to the terminal.
Redirect tee's standard output to a pipe that's connected to ./three.sh's standard input. This redirection doesn't affect the pipes set up in step 1.
Execute tee.
bash uses the first set of operations, zsh uses the second. In both cases, the order you see output from your shell scripts in is controlled by your OS's process scheduler and might as well be random. In the case where you redirect tee's standard output to /dev/null, they both seem to follow the second scenario and set up the subprocesses before the parent tee's redirection. This inconsistency on bash's part does seem unusual and a potential source of subtle bugs.
I can't replicate the missing prompt issue, but that's with bash 4.4.20 - I don't have 5 installed on this computer.
Script and output is as below :
Script :
#!/bin/bash
#tee_with_read.sh
function tee_test()
{
echo "***This should be printed first but it is not***"
read -r -p "Enter input : "
echo "You entered : $REPLY"
}
tee_test | tee -a logfile
Output :
$ ./tee_with_read.sh
Enter input : ***This should be printed first, but it is not***
"My Input"
You entered : "My Input"
I'm trying to append output to logfile.
But as you can see in ouput, it seems like first read gets excuted and then echo which is not as expected.
I'm using Git Bash Version 3.1.23 on windows 10.
Since named pipe is not available in this version, I cannot use named pipe for logging purpose.
read -p is writing it's output to stderr and echo is writing to stdout. stdout is typically buffered while stderr is not, and so it's not uncommon to see stderr things show up before stdout.
You could have your echo also go to stderr if you like by doing echo "string" >&2 or you could run it in the unbuffer command or similar tools if you have them available
Im wondering why awk print different output when run in background
My script:
#!/bin/bash
echo "Name of shell is $SHELL"
relase=`uname -r`
echo "Release is: $relase"
if [ $SHELL != "/bin/bash" ] || [ $relase != "3.13.0-32-generic" ] ; then
echo "Warning, different configuration"
fi
if [ $# -eq 0 ] ; then
echo "Insert name of shell"
read sname
else
sname=$1
fi
awk -v sname="$sname" 'BEGIN {FS=":"} {if ($7 == sname) print $1 }' </etc/passwd &
When i run awk without ampersand, output is:
petr#PetrLinux-VirtualBox:~/Documents$ ./script1 /bin/bash
Name of shell is /bin/bash
Release is: 3.13.0-32-generic
root
petr
but when i run awk with ampersand - in background, output is folowing:
petr#PetrLinux-VirtualBox:~/Documents$ ./script1 /bin/bash
Name of shell is /bin/bash
Release is: 3.13.0-32-generic
petr#PetrLinux-VirtualBox:~/Documents$ root
petr
First record (root) is not printed on single line. Please tell me why ańd if there is way how to print on single line while running on background. Thanks.
What you see is a mix of two outputs. The first output is of your shell, printing the command prompt (petr#PetrLinux-VirtualBox:~/Documents$). The second output is root from your script.
As your shell script runs in the background, you now have two processes writing to your terminal window: the bash (printing the prompt), and your script, printing the awk-output. This then just mixes up.
The only way to prevent that is to redirect the output of the script to a file or other device, instead of your console. For example:
$ ./script1 /bin/bash &> output.txt &
The output is the same. It just appears to be different because two processes write on the same channel (your terminal) and mix their output. One process is the awk script and the other is your shell which prints a new prompt.
There is no way to determine the precise point in which the output will switch from one process to the other. It can be different on different systems (with the same software), it can also depend on the load of the computer and lots of other things.
The only decent solution is to redirect the output into a different stream, e. g. a file using > outfile.
How to disable stdout or stderr in bash scripts temporarily?
Of course the most common way is to redirect stdout or stderr to /dev/null.
But on some systems /dev/null may be unwritable for normal users.
I am writing some scripts that is aim to be portable, so I do not prefer using /dev/null
Some blogs/posts say that >&- can close stdout, but when I tried echo 123 >&- in a bash terminal, it just failed with the message "bash: echo: write error: Bad file descriptor"
Surely I can do it by redirecting stdout or stderr to a tmp file like this:
some_command > /tmp/null
But what I want is a more "elegant" way
I think perhaps I can achieve this by using pipe like this:
some_command | :
But in this way, it may "pollutes" the exit code of the original command
Here is a possible way to do what you want:
( my_cmd 3>&1 1>&2 2>&3- ) | :
This temporarily send stdout to a new file handle, 3 and redirect stderr to stdout so that the stderr pipes into the command (in this case, :). Then the new file handle is routed back out to stdout. These avoid piping the stdout of my_cmd into :. The - in closes the handle after it's used.
To check the exist status of my_cmd after the above you examine the environment variable $PIPESTATUS[0]. $PIPESTATUS is a bash environment array variable that holds the exit status of each piped command in the last pipe that was done.
I think the really correct answer is to investigate why /dev/null isn't world writable. Having it not so is an off-standard system configuration and may cause system problems. The above work-around is a little messy by comparison.
Based on what I wrote earlier and #nos's comment above, here's an example:
(assuming you have no file called 'zzz' in your current directory, and that '.' is readable)
#!/bin/bash
set -o pipefail
ls . 2>&1 |:
echo $?
ls zzz 2>&1 |:
echo $?
The pipelines succeed and fail silently and maintain the exit code. Note that you can probably still make a pipeline example where this would not produce the desired results. I haven't come up with one in my head yet, but that doesn't mean it's not out there. The best answer, as many have noted already, is to fix the system so that /dev/null is world writable.
EDIT: Changed /bin/sh to /bin/bash, although this probably isn't necessary. But since I haven't tested this against a true Bourne Shell, I decided to err on the side of caution.
EDIT: Another script, showing several different redirections, and using the |& shortcut for 2>&1 |. If you run this, you'll notice that some of the ls failures return a 141 exit status rather than the expected 2. This is a broken pipe exit status, but still represents a failure.
#!/bin/bash
set -o pipefail
# start with commands that should succeed
# redirect everything to ':'
echo "ls . |& :"
ls . |& :
echo $?
# redirect only stdout to ':'
echo "ls . | :"
ls . | :
echo $?
# redirect only stderr to ':'
echo "((ls . 1>&3) |& : ) 3>&1"
((ls . 1>&3) |& : ) 3>&1
echo $?
# now move to failures
# redirect everything to ':'
echo "ls zzz |& :"
ls zzz |& :
echo $?
# redirect only stdout to ':'
echo "ls zzz |:"
ls zzz |:
echo $?
# redirect only stderr to ':'
echo "((ls zzz 1>&3) |& : ) 3>&1"
((ls zzz 1>&3) |& : ) 3>&1
echo $?
I use two subshells when I'm attempting to destroy stdout but keep stderr. You could do it without the outer one. In fact, that might be better. Instead of getting a broken pipe error, you get a 1 exit status.