New to bash programming. I am not sure what is meant by 'output to stdout'. Does it mean print out to the command line?
If I have a simple bash script:
#!/bin/bash
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello'
it outputs a string to the terminal. Does this mean it's 'outputting to stdout' ?
Thanks
Yes, stdout is the terminal (unless it's redirected to a file using the > operator or into the stdin of another process using |)
In your specific example, you're actually redirecting using | grep ... through grep then to the terminal.
Every process on a Linux system (and most others) has at least 3 open file descriptors:
stdin (0)
stdout (1)
stderr (2)
Regualary every of this file descriptors will point to the terminal from where the process was started. Like this:
cat file.txt # all file descriptors are pointing to the terminal where you type the command
However, bash allows to modify this behaviour using input / output redirection:
cat < file.txt # will use file.txt as stdin
cat file.txt > output.txt # redirects stdout to a file (will not appear on terminal anymore)
cat file.txt 2> /dev/null # redirects stderr to /dev/null (will not appear on terminal anymore
The same is happening when you are using the pipe symbol like:
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello'
What is actually happening is that the stdout of the wget process (the process before the | ) is redirected to the stdin of the grep process. So wget's stdout isn't a terminal anymore while grep's output is the current terminal. If you want to redirect grep's output to a file for example, then use this:
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello' > output.txt
Unless redirected, standard output is the text terminal which initiated the program.
Here's a wikipedia article: http://en.wikipedia.org/wiki/Standard_streams#Standard_output_.28stdout.29
Related
I know this much:
$ command 2>> error
$ command 1>> output
Is there any way I can output the stderr to the error file and output stdout to the output file in the same line of bash?
Just add them in one line command 2>> error 1>> output
However, note that >> is for appending if the file already has data. Whereas, > will overwrite any existing data in the file.
So, command 2> error 1> output if you do not want to append.
Just for completion's sake, you can write 1> as just > since the default file descriptor is the output. so 1> and > is the same thing.
So, command 2> error 1> output becomes, command 2> error > output
Try this:
your_command 2>stderr.log 1>stdout.log
More information
The numerals 0 through 9 are file descriptors in bash.
0 stands for standard input, 1 stands for standard output, 2 stands for standard error. 3 through 9 are spare for any other temporary usage.
Any file descriptor can be redirected to a file or to another file descriptor using the operator >. You can instead use the operator >> to appends to a file instead of creating an empty one.
Usage:
file_descriptor > filename
file_descriptor > &file_descriptor
Please refer to Advanced Bash-Scripting Guide: Chapter 20. I/O Redirection.
Like that:
$ command >>output 2>>error
Or if you like to mix outputs (stdout & stderr) in one single file you may want to use:
command > merged-output.txt 2>&1
Multiple commands' output can be redirected. This works for either the command line or most usefully in a bash script. The -s directs the password prompt to the screen.
Hereblock cmds stdout/stderr are sent to seperate files and nothing to display.
sudo -s -u username <<'EOF' 2>err 1>out
ls; pwd;
EOF
Hereblock cmds stdout/stderr are sent to a single file and display.
sudo -s -u username <<'EOF' 2>&1 | tee out
ls; pwd;
EOF
Hereblock cmds stdout/stderr are sent to separate files and stdout to display.
sudo -s -u username <<'EOF' 2>err | tee out
ls; pwd;
EOF
Depending on who you are(whoami) and username a password may or may not be required.
I need to run a script and output the full contents to the terminal.
I then want to get the last 10 lines from the terminal output and put them in a log file.
I have tried using ./script.sh 2>&1 | tail -10 > log.log
but this stops the output to the terminal.
Leverage process substitution of bash with tee:
./script.sh |& tee >(tail -10 >file.txt)
|& is a shortcut for sending both STDOUT and STDERR over the pipe.
tee redirects it's STDIN to both STDOUT and to the file(s) given as argument(s) -- we have used process substitution to get a file descriptor and used tail -10 >file.txt inside process substitution to save the desired content.
For that you have to use the tee command. Then you can pipe to a file and your console.
ls -a | tee output.file
I'm using "strace -f -T -tt -o foo.txt -p 1234" to print the time spent in system calls. This makes the output file huge, is there a way to just print the system calls that took greater than 1second. I can grep it out from the file later, but is there a better way?
If we simply omit the -o foo.txt argument, the output goes to standard output. We can pipe it through grep and redirect to the file:
strace -f -T -tt -p 1234 | grep pattern > foo.txt
To watch the output at the same time:
strace -f -T -tt -p 1234 | grep pattern | tee foo.txt
If a command prints only to a file that is passed as an argument, and we want to filter/redirect its output, the first step is to check whether it implements the dash convention: can you specify standard input or output using - as a filename argument:
some_command - | our_pipe > file.txt
If not, then the recourse is to use Bash process substitution substitution syntax: >(output command) and <(input command):
some_command >(our_pipe > file.txt)
The process substitution syntax expands into a token that is suitable as a filename argument for a command or function. When the program opens that token, it gets a file descriptor to the command's input or output, depending on direction.
With process substitution, we can redirect the input or output of stubborn programs which work only with files passed as by name as arguments, and which do not support any convention for requesting that standard input or output be used in place of a file.
The token used by process substitution is platform-dependent; we can see what it is using echo. For instance on GNU/Linux, Bash takes advantage of the /dev/fd operating system feature:
$ echo <(ls -l)
/dev/fd/63
You can use the following command:
strace -T command 2>&1 >/dev/null | awk '{gsub(/[<>]/,"",$NF)}$NF+.0>1.0'
Explanation:
strace -T adds the time spent in the syscall end the end of the line, enclosed in <...>
2>&1 >/dev/null | awk pipes stderr to awk. (strace writes it's output to stderr!)
The awk command removes the <> from the last field $NF and prints lines where the time spent is higher than a second.
Probably you'll also want to pass the threshold as a variable to the awk command:
strace -T command 2>&1 >/dev/null \
| awk -v thres=0.001 '{gsub(/[<>]/,"",$NF)}$NF+.0>thres+.0'
This question already has answers here:
How to redirect output to a file and stdout
(11 answers)
Closed 4 years ago.
I am trying to throw command output to file plus console also. This is because i want to keep record of output in file. I am doing following and it appending to file but not printing ls output on terminal.
$ls 2>&1 > /tmp/ls.txt
Yes, if you redirect the output, it won't appear on the console. Use tee.
ls 2>&1 | tee /tmp/ls.txt
It is worth mentioning that 2>&1 means that standard error will be redirected too, together with standard output. So
someCommand | tee someFile
gives you just the standard output in the file, but not the standard error: standard error will appear in console only. To get standard error in the file too, you can use
someCommand 2>&1 | tee someFile
(source: In the shell, what is " 2>&1 "? ). Finally, both the above commands will truncate the file and start clear. If you use a sequence of commands, you may want to get output&error of all of them, one after another. In this case you can use -a flag to "tee" command:
someCommand 2>&1 | tee -a someFile
In case somebody needs to append the output and not overriding, it is possible to use "-a" or "--append" option of "tee" command :
ls 2>&1 | tee -a /tmp/ls.txt
ls 2>&1 | tee --append /tmp/ls.txt
I have some scripts where I need to see the output and log the result to a file, with the simplest example being:
$ update-client > my.log
I want to be able to see the output of the command while it's running, but also have it logged to the file. I also log stderr, so I would want to be able to log the error stream while seeing it as well.
update-client 2>&1 | tee my.log
2>&1 redirects standard error to standard output, and tee sends its standard input to standard output and the file.
Just use tail to watch the file as it's updated. Background your original process by adding & after your above command After you execute the command above just use
$ tail -f my.log
It will continuously update. (note it won't tell you when the file has finished running so you can output something to the log to tell you it finished. Ctrl-c to exit tail)
You can use the tee command for that:
command | tee /path/to/logfile
The equivelent without writing to the shell would be:
command > /path/to/logfile
If you want to append (>>) and show the output in the shell, use the -a option:
command | tee -a /path/to/logfile
Please note that the pipe will catch stdout only, errors to stderr are not processed by the pipe with tee. If you want to log errors (from stderr), use:
command 2>&1 | tee /path/to/logfile
This means: run command and redirect the stderr stream (2) to stdout (1). That will be passed to the pipe with the tee application.
Learn about this at askubuntu site
another option is to use block based output capture from within the script (not sure if that is the correct technical term).
Example
#!/bin/bash
{
echo "I will be sent to screen and file"
ls ~
} 2>&1 | tee -a /tmp/logfile.log
echo "I will be sent to just terminal"
I like to have more control and flexibility - so I prefer this way.