I am trying to write a script that starts rtmpsrv and waits for some output from it. rtmpsrv gives the desirable output and continues running, but the script is waiting for a termination of rtmpsrv. How do I gain access to the output of rtmpsrv without stopping it?
Well, I'm not familiar with rtmpsrv, but unless necessary you should wait for it to finish. However, you can probably redirect its output to a file, and then grep the file to see if it contains the string you are looking for.
(fictional code... you can expect syntax hell, just want to give you an idea)
nohup rtmpsrv >log.rtmpsrv 2>&1 &
...
while :; do
if ! result=$(grep "your desired line" log.rtmpsrv); then
echo "success: found $result"
break
fi
done
Note: the if constructs should work as per http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html - just to have nicer code, as #Charles Duffy suggested.
The most simple way is this:
rtmpsrv > logfile &
Then you can search logfile for the text that you're looking for. Meanwhile, rtmpsrv will do it's thing, completely unaware of your script.
This question contains examples how to wait in your script for a certain pattern to appear in the logfile (so you don't have to search it again and again): Do a tail -F until matching a pattern
Note: If you start the rtmpsrv process in the script and the script terminates, it will probably kill the rtmpsrv process. To avoid that use nohup.
Just attach to the process using gdb -p <pid> where the pid is the process id of your script.
You can find the pid of your running script by doing smth like this ps ax | grep <My Script's Name>
http://etbe.coker.com.au/2008/02/27/redirecting-output-from-a-running-process/
Related
On a Linux server, I have a script here that will work fine when I start it from the terminal, but fail when started and then detached by another process. So there is probably a difference in the script's environment to fix.
The trouble is, the other process integrating that script does not provide access to its error messages when the script fails. What is an easy (and ideally generic) way to see the output of such a script when it's failing?
Let's assume I have no easy way to change the code of the process calling this script. The failure happens right at the start of the script's run, so there is not enough time to manually attach to it with strace to see its output.
(The specifics should not matter, but for what it's worth: the failing script is the backup script of Discourse, a widespread open source forum software. Discourse and this script are written in Ruby.)
The idea is to substitute original script with wrapper which calls original script and saves its stdin and stderr to files. Wrapper may be like this:
#!/bin/bash
exec /path/to/original/script "$#" 1> >(tee /tmp/out.log) 2> >(tee /tmp/err.log >&2)
1> >(tee /tmp/out.log) redirects stdout to tee /tmp/out.log input in subshell. tee /tmp/out.log passes it to stdout but saves copy to the file.
2> >(tee /tmp/err.log) redirects stderr to tee /tmp/err.log input in subshell. tee /tmp/err.log >&2 passes it to stderr but saves copy to the file.
If script is invoked multiple times you may want to append stdout and stderr to files. Use tee -a in this case.
The problem is how to force caller to execute wrapper script instead of original one.
If caller invokes script in a way that it is searched in PATH you can put wrapper script to a separate directory and provide modified PATH to the caller. For example, script name is script. Put wrapper to /some/dir/script and run caller as
$ PATH="/some/dir:$PATH" caller
/path/to/original/script in wrapper must be absolute.
If caller invokes script from specific path then you can rename original script e.g. to original-script and name wrapper as script. In this case wrapper should call /path/to/original/original-script.
Another problem may rise if script behaves differently depending on name it's called. In this case exec -a ... may be needed.
You can use a bash script that (1) does "busy waiting" until it sees the targeted process, and then (2) immediately attaches to it with strace and prints its output to the terminal.
#!/bin/sh
# Adapt to a regex that matches only your target process' full command.
name_pattern="bin/ruby.*spawn_backup_restore.rb"
# Wait for a process to start, based on its name, and capture its PID.
# Inspiration and details: https://unix.stackexchange.com/a/410075
pid=
while [ -z "$pid" ] ; do
pid="$(pgrep --full "$name_pattern" | head -n 1)"
# Set delay for next check to 1ms to try capturing all output.
# Remove completely if this is not enough to capture from the start.
sleep 0.001
done
echo "target process has started, pid is $pid"
# Print all stdout and stderr output of the process we found.
# Source and explanations: https://unix.stackexchange.com/a/58601
strace -p "$pid" -s 9999 -e write
I'm using this code to write all terminal's output to the file
exec > >(tee -a "${LOG_FILE}" )
exec 2> >(tee -a "${LOG_FILE}" >&2)
How to tell it to stop? If, for example, I don't want some output to get into log..
Thank you!
It's not completely clear what your goal is here, but here is something to try.
Do you know about the script utility? You can run script myTerminalOutputFile and any commands and output after that will be captured to myTerminalOutputFile. The downside is that it captures everything. You'll see funny screen control chars like ^[[5m;27j. So do a quick test to see if that works for you.
To finish capturing output just type exit and you are returned to you parent shell cmd-line.
Warning: check man script for details about the inherent funkyness of your particular OS's version of script. Some are better than others. script -k myTerminalOutputFile may work better in this case.
IHTH
You may pipe through "sed" and then tee to log file
This question already has answers here:
Find and kill a process in one line using bash and regex
(30 answers)
Closed 6 years ago.
I'm looking into writing shells scripts as a prerequisite for a class and would like some help to get started. I'm currently doing a warm up exercise that requires me to write a shell script that, when executed, will kill any currently running process of a command I have given. For this exercise, I'm using the 'less' command (so to test I would input 'man ps | less').
However, since this is the first REAL script I'm writing (besides the traditional "Hello World!" one I've done), I'm a little stuck on how to start. I'm googled a lot and have returned some rather confusing results. I'm aware I need to start with a shebang, but I'm not sure why. I was thinking of using 'if' statement; something along the lines of
if 'less' is running
kill 'less' process
fi
But I'm not sure of how to go about that. Since I'm incredibly new at this, I also want to make sure I'm going about writing a script correctly. I'm using notepad as a text editor, and once I've written my script there, I'll save it to a directory that I access in a terminal and then run from there, correct?
Thank you very much for any advice or resources you could give me. I'm certain I can figure out harder exercises once I get the basics of writing a script down.
Try:
pgrep less && killall less
pgrep less looks process ids of any process named less. If a process is found, it returns true in which case the && clause is triggered. killall less kills any process named less.
See man pgrep and man killall.
Simplification
This may miss the point of your exercise, but there is no real need to test for a less process running. Just run:
killlall less
If no less process is running, then killall does nothing.
Try this simple snippet:
#!/bin/bash
# if one or more processes matching "less" are running
# (ps will return 0 which corresponds to true in that case):
if ps -C less
then
# send all processes matching "less" the TERM signal:
killall -TERM less
fi
For more information on available signals, see the table in the man page available via man 7 signal.
You might try the following code in bash:
#Tell which interpreter will process the code
#!/bin/bash
#Creating a variable to hold program name you want to serach and kill
#mind no-space between variable name value and equals sign
program='less'
#use ps to list all process and grep to search for the specific program name
# redirect the visible text output to /dev/null(linux black hole) since we don't want to see it on screen
ps aux | grep "$program" | grep -v grep > /dev/null
#If the given program is found $? will hold 0, since if successfull grep will return 0
if [ $? -eq 0 ]; then
#program is running kill it with killall
killall -9 "$program"
fi
On running nohup with & on command line, it is returning the process id,
while the same command I am running in perl script within backticks and trying to read output is not returning any output.
Can anyone please guide?
nohup rm -rf ragh &
[1] 10029
The job number and PID are printed by the shell when starting a background process in a terminal. nohup is irrelevant. If you don't start the job from a terminal (i.e. you use backticks in Perl on shell, or you use a plain subshell) the information isn't shown. Why do you need it, anyway? See perlipc - Perl interprocess communication for details.
If you need the process ID of the background job then use the $! variable, for example:
nohup start_long_running_job &
echo $! > jobid.txt
And then if you need to kill the job:
kill $(cat jobid.txt)
It applies equally with or without nohup.
nohup means that you spawn a new process and execute the script in that context.
If your command there takes longer than your starting script it will survive the closing of your shell. If you need the output you should pipe it somewhere else
nohup rm -rf ragh > log.txt &
choroba correctly stated when the PID isn't shown ("If you don't start the job from a terminal").
Richard RP correctly stated that $! can be used. But for running in a Perl script within backticks, in addition we need to close the command's standard output, otherwise the backtick invocation would return only after the process has finished, because perl waits for the output's EOF.
$pid = `nohup rm -rf ragh >&-& echo \$!`
gets us rm's PID in $pid.
For some reasons not relevant to this question, I am running a Java server in a bash script not directly but via command substitution under a separate sub-shell, and in the background. The intent is for the subcommand to return the process id of the Java server as its standard output. The fragement in question is as follows:
launch_daemon()
{
/bin/bash <<EOF
$JAVA_HOME/bin/java $JAVA_OPTS -jar $JAR_FILE daemon $PWD/config/cl.yml <&- &
pid=\$!
echo \${pid} > $PID_FILE
echo \${pid}
EOF
}
daemon_pid=$(launch_daemon)
echo ${daemon_pid} > check.out
The Java daemon in question prints to standard error and quits if there is a problem in initialization, otherwise it closes standard out and standard err and continues on its way. Later in the script (not shown) I do a check to make sure the server process is running. Now on to the problem.
Whenever I check the $PID_FILE above, it contains the correct process id on one line.
But when I check the file check.out, it sometimes contains the correct id, other times it contains the process id repeated twice on the same line separated by a space charcater as in:
34056 34056
I am using the variable $daemon_pid in the script above later on in the script to check if the server is running, so if it contains the pid repeated twice this totally throws off the test and it incorrectly thinks the server is not running. Fiddling with the script on my server box running CentOS Linux by putting in more echo statements etc. seems to flip the behavior back to the correct one of $daemon_pid containing the process id just once, but if I think that has fixed it and check in this script to my source code repo and do a build and deploy again, I start seeing the same bad behavior.
For now I have fixed this by assuming that $daemon_pid could be bad and passing it through awk as follows:
mypid=$(echo ${daemon_pid} | awk '{ gsub(" +.*",""); print $0 }')
Then $mypid always contains the correct process id and things are fine, but needless to say I'd like to understand why it behaves the way it does. And before you ask, I have looked and looked but the Java server in question does NOT print its process id to its standard out before closing standard out.
Would really appreciate expert input.
Following the hint by #WilliamPursell, I tracked this down in the bash source code. I honestly don't know whether it is a bug or not; all I can say is that it seems like an unfortunate interaction with a questionable use case.
TL;DR: You can fix the problem by removing <&- from the script.
Closing stdin is at best questionable, not just for the reason mentioned by #JonathanLeffler ("Programs are entitled to have a standard input that's open.") but more importantly because stdin is being used by the bash process itself and closing it in the background causes a race condition.
In order to see what's going on, consider the following rather odd script, which might be called Duff's Bash Device, except that I'm not sure that even Duff would approve: (also, as presented, it's not that useful. But someone somewhere has used it in some hack. Or, if not, they will now that they see it.)
/bin/bash <<EOF
if (($1<8)); then head -n-$1 > /dev/null; fi
echo eight
echo seven
echo six
echo five
echo four
echo three
echo two
echo one
EOF
For this to work, bash and head both have to be prepared to share stdin, including sharing the file position. That means that bash needs to make sure that it flushes its read buffer (or not buffer), and head needs to make sure that it seeks back to the end of the part of the input which it uses.
(The hack only works because bash handles here-documents by copying them into a temporary file. If it used a pipe, it wouldn't be possible for head to seek backwards.)
Now, what would have happened if head had run in the background? The answer is, "just about anything is possible", because bash and head are racing to read from the same file descriptor. Running head in the background would be a really bad idea, even worse than the original hack which is at least predictable.
Now, let's go back to the actual program at hand, simplified to its essentials:
/bin/bash <<EOF
cmd <&- &
echo \$!
EOF
Line 2 of this program (cmd <&- &) forks off a separate process (to run in the background). In that process, it closes stdin and then invokes cmd.
Meanwhile, the foreground process continues reading commands from stdin (its stdin fd hasn't been closed, so that's fine), which causes it to execute the echo command.
Now here's the rub: bash knows that it needs to share stdin, so it can't just close stdin. It needs to make sure that stdin's file position is pointing to the right place, even though it may have actually read ahead a buffer's worth of input. So just before it closes stdin, it seeks backwards to the end of the current command line. [1]
If that seek happens before the foreground bash executes echo, then there is no problem. And if it happens after the foreground bash is done with the here-document, also no problem. But what if it happens while the echo is working? In that case, after the echo is done, bash will reread the echo command because stdin has been rewound, and the echo will be executed again.
And that's precisely what is happening in the OP. Sometimes, the background seek completes at just the wrong time, and causes echo \${pid} to be executed twice. In fact, it also causes echo \${pid} > $PID_FILE to execute twice, but that line is idempotent; had it been echo \${pid} >> $PID_FILE, the double execution would have been visible.
So the solution is simple: remove <&- from the server start-up line, and optionally replace it with </dev/null if you want to make sure the server can't read from stdin.
Notes:
Note 1: For those more familiar with bash source code and its expected behaviour than I am, I believe that the seek and close takes place at the end of case r_close_this: in function do_redirection_internal in redir.c, at approximately line 1093:
check_bash_input (redirector);
close_buffered_fd (redirector);
The first call does the lseek and the second one does the close. I saw the behaviour using strace -f and then searched the code for a plausible looking lseek, but I didn't go to the trouble of verifying in a debugger.