I need to monitor stderr for a certain error message, and when it is detected, run a shell script.
Current idea is to pipe stderr to grep, and filter for the message. I would pipe grep's output to some program that would run my script upon receipt of any user input.
Alternatively, if I could make grep output a specific command-line option parameter, I could presumably pipe that directly to my shell script.
Is there a better way of doing this?
I would like to recommend inotifywait.
Monitor stderr-log.txt for changes.
Read file and grep the stderr message.
Call another script if error message is found
while:;do
inotifywait -q -e modify ./stderr-log.txt > /dev/null
if cat stderr-log.txt | grep -q <error message>; then
<do something here..>
fi
done
Related
On a Linux server, I have a script here that will work fine when I start it from the terminal, but fail when started and then detached by another process. So there is probably a difference in the script's environment to fix.
The trouble is, the other process integrating that script does not provide access to its error messages when the script fails. What is an easy (and ideally generic) way to see the output of such a script when it's failing?
Let's assume I have no easy way to change the code of the process calling this script. The failure happens right at the start of the script's run, so there is not enough time to manually attach to it with strace to see its output.
(The specifics should not matter, but for what it's worth: the failing script is the backup script of Discourse, a widespread open source forum software. Discourse and this script are written in Ruby.)
The idea is to substitute original script with wrapper which calls original script and saves its stdin and stderr to files. Wrapper may be like this:
#!/bin/bash
exec /path/to/original/script "$#" 1> >(tee /tmp/out.log) 2> >(tee /tmp/err.log >&2)
1> >(tee /tmp/out.log) redirects stdout to tee /tmp/out.log input in subshell. tee /tmp/out.log passes it to stdout but saves copy to the file.
2> >(tee /tmp/err.log) redirects stderr to tee /tmp/err.log input in subshell. tee /tmp/err.log >&2 passes it to stderr but saves copy to the file.
If script is invoked multiple times you may want to append stdout and stderr to files. Use tee -a in this case.
The problem is how to force caller to execute wrapper script instead of original one.
If caller invokes script in a way that it is searched in PATH you can put wrapper script to a separate directory and provide modified PATH to the caller. For example, script name is script. Put wrapper to /some/dir/script and run caller as
$ PATH="/some/dir:$PATH" caller
/path/to/original/script in wrapper must be absolute.
If caller invokes script from specific path then you can rename original script e.g. to original-script and name wrapper as script. In this case wrapper should call /path/to/original/original-script.
Another problem may rise if script behaves differently depending on name it's called. In this case exec -a ... may be needed.
You can use a bash script that (1) does "busy waiting" until it sees the targeted process, and then (2) immediately attaches to it with strace and prints its output to the terminal.
#!/bin/sh
# Adapt to a regex that matches only your target process' full command.
name_pattern="bin/ruby.*spawn_backup_restore.rb"
# Wait for a process to start, based on its name, and capture its PID.
# Inspiration and details: https://unix.stackexchange.com/a/410075
pid=
while [ -z "$pid" ] ; do
pid="$(pgrep --full "$name_pattern" | head -n 1)"
# Set delay for next check to 1ms to try capturing all output.
# Remove completely if this is not enough to capture from the start.
sleep 0.001
done
echo "target process has started, pid is $pid"
# Print all stdout and stderr output of the process we found.
# Source and explanations: https://unix.stackexchange.com/a/58601
strace -p "$pid" -s 9999 -e write
I have a shell script that I want to automate downloading tasks, i would like to get response of command to a variable, command defines as follow.
var=wget --ftp-user=MyName --ftp-password=MyPassword --directory prefix=/home/pi/Desktop/FTP_File/ ftp://202.xx.xx.xx/VideoFiles/Video_1.mp4 2>&1
echo check "$var"
i have achieved that after adding 2>&1 at the end of line and command is in "``", i would like to know what is meant by 2>&1 and is there any other way to achieve it?
Have a look at this: http://www.learnlinux.org.za/courses/build/shell-scripting/ch01s04.html
Any program that is written must have some error checking and it should output some message if any error occurs.
It is a standard practice to output error messages on stderr, informative messages on stdout.
2>&1 means to display all the prints of stderr and stdout related to a particular command when executed.
Hope I have answered your question.
I have a shell script which writes all output to logfile
and terminal, this part works fine, but if I execute the script
a new shell prompt only appear if I press enter. Why is that and how do I fix it?
#!/bin/bash
exec > >(tee logfile)
echo "output"
First, when I'm testing this, there always is a new shell prompt, it's just that sometimes the string output comes after it, so the prompt isn't last. Did you happen to overlook it? If so, there seems to be a race where the shell prints the prompt before the tee in the background completes.
Unfortunately, that cannot fixed by waiting in the shell for tee, see this question on unix.stackexchange. Fragile workarounds aside, the easiest way to solve this that I see is to put your whole script inside a list:
{
your-code-here
} | tee logfile
If I run the following script (suppressing the newline from the echo), I see the prompt, but not "output". The string is still written to the file.
#!/bin/bash
exec > >(tee logfile)
echo -n "output"
What I suspect is this: you have three different file descriptors trying to write to the same file (that is, the terminal): standard output of the shell, standard error of the shell, and the standard output of tee. The shell writes synchronously: first the echo to standard output, then the prompt to standard error, so the terminal is able to sequence them correctly. However, the third file descriptor is written to asynchronously by tee, so there is a race condition. I don't quite understand how my modification affects the race, but it appears to upset some balance, allowing the prompt to be written at a different time and appear on the screen. (I expect output buffering to play a part in this).
You might also try running your script after running the script command, which will log everything written to the terminal; if you wade through all the control characters in the file, you may notice the prompt in the file just prior to the output written by tee. In support of my race condition theory, I'll note that after running the script a few times, it was no longer displaying "abnormal" behavior; my shell prompt was displayed as expected after the string "output", so there is definitely some non-deterministic element to this situation.
#chepner's answer provides great background information.
Here's a workaround - works on Ubuntu 12.04 (Linux 3.2.0) and on OS X 10.9.1:
#!/bin/bash
exec > >(tee logfile)
echo "output"
# WORKAROUND - place LAST in your script.
# Execute an executable (as opposed to a builtin) that outputs *something*
# to make the prompt reappear normally.
# In this case we use the printf *executable* to output an *empty string*.
# Use of `$ec` is to ensure that the script's actual exit code is passed through.
ec=$?; $(which printf) ''; exit $ec
Alternatives:
#user2719058's answer shows a simple alternative: wrapping the entire script body in a group command ({ ... }) and piping it to tee logfile.
An external solution, as #chepner has already hinted at, is to use the script utility to create a "transcript" of your script's output in addition to displaying it:
script -qc yourScript /dev/null > logfile # Linux syntax
This, however, will also capture stderr output; if you wanted to avoid that, use:
script -qc 'yourScript 2>/dev/null' /dev/null > logfile
Note, however, that this will suppress stderr output altogether.
As others have noted, it's not that there's no prompt printed -- it's that the last of the output written by tee can come after the prompt, making the prompt no longer visible.
If you have bash 4.4 or newer, you can wait for your tee process to exit, like so:
#!/usr/bin/env bash
case $BASH_VERSION in ''|[0-3].*|4.[0-3]) echo "ERROR: Bash 4.4+ needed" >&2; exit 1;; esac
exec {orig_stdout}>&1 {orig_stderr}>&2 # make a backup of original stdout
exec > >(tee -a "_install_log"); tee_pid=$! # track PID of tee after starting it
cleanup() { # define a function we'll call during shutdown
retval=$?
exec >&$orig_stdout # Copy your original stdout back to FD 1, overwriting the pipe to tee
exec 2>&$orig_stderr # If something overwrites stderr to also go through tee, fix that too
wait "$tee_pid" # Now, wait until tee exits
exit "$retval" # and complete exit with our original exit status
}
trap cleanup EXIT # configure the function above to be called during cleanup
echo "Writing something to stdout here"
I want to show program output on screen and also put them into a log file on linux
I tried:
<program> 2>&1 | tee logfile
or
<program> |& tee logfile
indeed, they can do what I want
but the output is not shown on screen and I can't see anything when vim the logfile
until the program finishes
how to deal with this problem?
thanks!
You can run that command:
script -f -c "program" logfile
This will launch your program and logs both its standard output and error streams to logfile in an unbuffered manner.
One drawback will be stdin will also present in the output (unless un-echoed on screen) but that might be a useful feature.
I am running a task on the CLI, which prompts me for a yes/no input.
After selecting a choice, a large amount of info scrolls by on the screen - including several errors. I want to pipe this output to a file so I can see the errors. A simple '>' is not working since the command expects keyboard input.
I am running on Ubuntu 9.1.
command &> output.txt
You can use &> to redirect both stdout and stderr to a file. This is shorthand for command > output.txt 2>&1 where the 2>&1 means "send stderr to the same place as stdout" (stdout is file descriptor 1, stderr is 2).
For interactive commands I usually don't bother saving to a file if I can use less and read the results right away:
command 2>&1 | less
echo yes | command > output.txt
Depending on how the command reads it's input (some programs discard whatever was on stdin before it displays it's prompt, but most don't), this should work on any sane CLI-environment.
Use 2> rather than just >.
If the program was written by a sane person what you probably want is the stderr not the stdout. You would achieve this by using something like
foo 2> errors.txt
you can use 2> option to send errors to the file.
example:
command 2> error.txt
(use of option 2>) --- see if their would be any error while the execution of the command it will send it to the file error.txt.