using bash in linux is it possible to spawn parallel processes in the foreground? For example the following :
top.sh
#!/bin/bash
./myscript1.sh &
./myscript2.sh &
will spawn two processes in parallel as background threads. However is it possible to spawn these as foreground threads? The aim is to automatically kill myscript1.sh and myscript2.sh, when top.sh is killed. Thanks
You can only have one job in the foreground. You can make your script react to any signal that reaches it and forward the signal to other jobs. You need to make sure your script sticks around, if you want to have a central way of killing the subprocesses: call wait so that your script will not exit until all the jobs have died or the script itself is killed.
#!/bin/bash
jobs=
trap 'kill -HUP $jobs' INT TERM HUP
myscript1.sh & jobs="$jobs $!"
myscript2.sh & jobs="$jobs $!"
wait
You can still kill only the wrapper script by sending it a signal that it doesn't catch, such as SIGQUIT (which I purposefully left out) or SIGKILL (which can't be caught).
There's a way to have all the processes in the foreground: connect them through pipes. Ignore SIGPIPE so that the death of a process doesn't kill the previous one. Save and restore stdin and stdout through other file descriptors if you need them. This way the script and the background tasks will be in the same process group, so pressing Ctrl+C will kill both the wrapper script and the subprocesses. If you kill the wrapper script directly, that won't affect the subprocesses; you can kill the process group instead by passing the negative of the PID of the wrapper script (e.g. kill -TERM -1234).
trap '' PIPE
{
myscript1.sh <&3 >&4 |
myscript2.sh <&3 >&4
} 3<&0 4>&1
Using GNU Parallel your script would be:
#!/bin/bash
parallel ::: ./myscript1.sh ./myscript2.sh
Or even:
#!/usr/bin/parallel --shebang -r
./myscript1.sh
./myscript2.sh
Watch the intro videos to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Related
If I run a command, such as grep, at the command line and hit ^C, the command is properly killed (with SIGINT I think). And if I run the grep in background and then run a kill SIGINT on its PID, it similarly gets terminated. But if I'm inside a script and run grep in background from the script, get its PID and then use 'kill -s SIGINT $PID', grep does not get killed. Why? If I use SIGTERM, instead of SIGINT, the kill does work.
#!/bin/bash
grep -rqa shazam /usr &
PID=$!
kill -s SIGINT $PID
Even if I put the grep in a subprocess, preceded by a SIGINT handler (in the subprocess), and hit the subprocess with SIGINT, the handler is not invoked.
#!/bin/bash
( trap 'echo "caught signal"' SIGINT; grep -rqa shazam /usr ) &
PID=$!
kill -s SIGINT $PID
The trap handler is invoked if I use SIGTERM, instead of SIGINT, but does not interrupt grep. If I add '/bin/kill -s SIGTERM 0' to the trap handler, there is an indication that the grep process gets terminated, but grep has already completed its work by then. I realize that Bash may have different default behaviors for the different signals, but I don't understand why my call to kill SIGINT is different than a ^C, why the trap call works for SIGTERM, but not for SIGINT, nor why SIGTERM isn't handled by the subprocess immediately.
Well, with further digging, I figured out 2 of my 3 questions. When I backgrounded grep within the script, the shell told it to ignore SIGINT. And Bash says it will wait to handle the signal until the subcommand is complete in some situations (which I don't fully follow at the moment), but the signal is handled immediately if hit the grep process directly with pkill.
"Actually bash will disable SIGINT (and SIGQUIT) on background processes and they can't be enabled" Background process and signals How SIGINT works
"Further background jobs are not supposed to be tied to the shell that started them. If you exit a shell, they will continue running. As such they shouldn't be interrupted by SIGINT, not by default. When job control is enabled, that is fulfilled automatically, since background jobs are running in separate process groups. When job control is disabled (generally in non-interactive shells), bash makes the asynchronous commands ignore SIGINT." Independent Program
Reason why SIGTERM works
Ok, just like in this thread, How to get PID of background process?, I know how to get the PID of background process. However, what I need to do countains more than one operation.
{
sleep 300;
echo "Still running after 5 min, killing process manualy.";
COMMAND COMMAND COMMAND
echo "Shutdown complete"
}&
PID_CHECK_STOP=$!
some stuff...
kill -9 $PID_CHECK_STOP
But it doesn't work. It seems i get either a bad PID or I just can't kill it. I tried to run ps | grep sleep and the pid it gives is always right next to the one i get in PID_CHECK_STOP. Is there a way to make it work? Can i wrap those commands an other way so i can kill them all when i need to?
Thx guys!
kill -9 kills the process before it can do anything else, including signalling its children to exit. Use a gentler signal (kill by itself, which sends a TERM, should be sufficient). You do need to have the process signal its children to exit (if any) explicitly, though, via a trap command.
I'm assuming sleep is a placeholder for the real command. sleep is tricky, however, as it ignores any signals until it returns (i.e., it is non-interruptible). To make your example work, put sleep itself in the background and immediately wait on it. When you kill the "outer" background process, it will interrupt the wait call, which will allow sleep to be killed as well.
{
trap 'kill $(jobs -p)' EXIT
sleep 300 & wait
echo "Still running after 5 min, killing process manualy.";
COMMAND COMMAND COMMAND
echo "Shutdown complete"
}&
PID_CHECK_STOP=$!
some stuff...
kill $PID_CHECK_STOP
UPDATE: COMMAND COMMAND COMMAND includes a command that runs via sudo. To kill that process, kill must also be run via sudo. Keep in mind that doing so will run the external kill program, not the shell built-in (there is little difference between the two; the built-in exists to allow you to kill a process when your process quota has been reached).
You can have another script containing those commands and kill that script. If you are dynamically generating code for the block, just write out a script, execute it and kill when you are done.
The { ... } surrounding the statements starts a new shell, and you get its PID afterwards. sleep and other commands within the block get separate PIDs.
To illustrate, look for your process in ps afux | less - the parent shell process (above the sleep) has the PID you were just given.
I current have a script that looks like this.
# code
mplayer "$vid"
# more code
The problem is that if this script is killed the mplayer process lives. I wondering how I could make it so that killing the script would kill mplayer as well.
I can't use exec because I need to run commands after mplayer.
exec mplayer "$vid"
The only possible solution I can think of is to spawn it in the background and wait until it finishes manually. That way I can get it's PID and kill it when the script gets killed, not exactly elegant. I was wondering what the "proper" or best way of doing this is.
I was able to test the prctl idea I posted about in a comment and it seems to work. You will need to compile this:
#include "sys/prctl.h"
#include "stdlib.h"
#include "string.h"
#include "unistd.h"
int main(int argc, char ** argv){
prctl(PR_SET_PDEATHSIG, atoi(argv[1]),0,0,0);
char * argv0 = strdup(argv[2]);
char * slashptr = strrchr(argv0, '/');
if(slashptr){
argv0 = slashptr + 1;
}
return execvp(argv0, &(argv[2]));
}
Let's say you have compiled the above to an executable named "prun" and it is in your path. Let's say your script is called "foo.sh" and it is also in your path. Make a wrapper script that calls
prun 15 foo.sh
foo.sh should get SIGTERM when the wrapper script is terminated for any reason, even SIGKILL.
Note: this is a linux only solution and the c source code presented is without detailed checking of arguments
Thanks to Mux for the lead. It appears that there is no way to do this in bash except for manually catching signals. Here is a final working (overly commented) version.
trap : SIGTERM SIGINT # Trap these two (killing) signals. These will cause wait
# to return a value greater than 128 immediately after received.
mplayer "$vid" & # Start in background (PID gets put in `$!`)
pid=$!
wait $pid # Wait for mplayer to finish.
[ $? -gt 128 ] && { kill $pid ; exit 128; } ; # If a signal was recieved
# kill mplayer and exit.
Refrences:
- traps: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_12_02.html
(Updated) I think I understand what you are looking for now:
You can accomplish this by spawning a new terminal to run your script:
gnome-terminal -x /path_to_dir_of_your_script/your_script_name
(or use xterm -e or konsole -e instead of gnome-terminal -x, depending on what system you are on)
So now whenever your script ends / exits (I assume you have exit 0 or exit 1 in certain parts of the script), the newly spawned terminal will also exit since the script is finished - this will in turn also kill any applications spawned under that new terminal.
For example, I just tested the above command with this script:
#!/bin/bash
gedit &
pid=$!
echo "$pid"
sleep 5
exit 0
As you can see, there are no explicit calls to kill the new gedit process, but the application (gedit) closes as soon as the script exits anyway.
(Previous answer: alternatively, if you were simply asking about how to kill a process) Here's a short example of how you can accomplish that with kill.
#!/bin/bash
gedit &
pid=$!
echo "$pid"
sleep 5
kill -s SIGKILL $pid
Unless I misunderstood your question, you can get the PID of the spawned process right away instead of waiting until it finishes.
Well, you can simply kill the process group instead, this way the whole process tree will be killed, first find out the group id
ps x -o "%p %r %c" | grep <name>
And then use kill like so:
kill -TERM -<gid>
Note the dash before the process group id. Or a one-liner:
kill -TERM -$(pgrep <name>)
Perhaps use command substitution to run mplayer "$vid" in a subshell:
$(mplayer "$vid")
I tested it this way:
tesh.sh:
#!/bin/sh
$vid = "..."
$(mplayer "$vid")
% test.sh
In a separate terminal:
% pkill test.sh
In the orginal terminal, mplayer stops, printing to stderr
Terminated
MPlayer interrupted by signal 13 in module: av_sync
I have a bash script that does some parallel processing in a loop. I don't want the parallel process to spike the CPU, so I use a sleep command. Here's a simplified version.
(while true;do sleep 99999;done)&
So I execute the above line from a bash prompt and get something like:
[1] 12345
Where [1] is the job number and 12345 is the process ID (pid) of the while loop. I do a kill 12345 and get:
[1]+ Terminated ( while true; do
sleep 99999;
done )
It looks like the entire script was terminated. However, I do a ps aux|grep sleep and find the sleep command is still going strong but with its own pid! I can kill the sleep and everything seems fine. However, if I were to kill the sleep first, the while loop starts a new sleep pid. This is such a surprise to me since the sleep is not parallel to the while loop. The loop itself is a single path of execution.
So I have two questions:
Why did the sleep command get its own process ID?
How do I easily kill the while loop and the sleep?
Sleep gets its own PID because it is a process running and just waiting. Try which sleep to see where it is.
You can use ps -uf to see the process tree on your system. From there you can determine what the PPID (parent PID) of the shell (the one running the loop) of the sleep is.
Because "sleep" is a process, not a build-in function or similar
You could do the following:
(while true;do sleep 99999;done)&
whilepid=$!
kill -- -$whilepid
The above code kills the process group, because the PID is specified as a negative number (e.g. -123 instead of 123). In addition, it uses the variable $!, which stores the PID of the most recently executed process.
Note:
When you execute any process in background on interactive mode (i.e. using the command line prompt) it creates a new process group, which is what is happening to you. That way, it's relatively easy to "kill 'em all", because you just have to kill the whole process group. However, when the same is done within a script, it doesn't create any new group, because all new processes belong to the script PID, even if they are executed in background (jobs control is disabled by default). To enable jobs control in a script, you just have to put the following at the beginning of the script:
#!/bin/bash
set -m
Have you tried doing kill %1, where 1 is the number you get after launching the command in background?
I did it right now after launching (while true;do sleep 99999;done)& and it correctly terminated it.
"ps --ppid" selects all processes with the specified parent pid, eg:
$ (while true;do sleep 99999;done)&
[1] 12345
$ ppid=12345 ; kill -9 $ppid $(ps --ppid $ppid -o pid --no-heading)
You can kill the process group.
To find the process group of your process run:
ps --no-headers -o "%r" -p 15864
Then kill the process group using:
kill -- -[PGID]
You can do it all in one command. Let's try it out:
$ (while true;do sleep 99999;done)&
[1] 16151
$ kill -- -$(ps --no-headers -o "%r" -p 16151)
[1]+ Terminated ( while true; do
sleep 99999;
done )
To kill the while loop and the sleep using $! you can also use a trap signal handler inside the subshell.
(trap 'kill ${!}; exit' TERM; while true; do sleep 99999 & wait ${!}; done)&
kill -TERM ${!}
I am wrapping a fastcgi app in a bash script like this:
#!/bin/bash
# stuff
./fastcgi_bin
# stuff
As bash only executes traps for signals when the foreground script ends I can't just kill -TERM scriptpid because the fastcgi app will be kept alive.
I've tried sending the binary to the background:
#!/bin/bash
# stuff
./fastcgi_bin &
PID=$!
trap "kill $PID" TERM
# stuff
But if I do it like this, apparently the stdin and stdout aren't properly redirected because it does not connect with lighttpds mod_fastgi, the foreground version does work.
EDIT: I've been looking at the problem and this happens because bash redirects /dev/null to stdin when a program is launched in the background, so any way of avoiding this should solve my problem as well.
Any hint on how to solve this?
There are some options that come to my mind:
When a process is launched from a shell script, both belong to the same process group. Killing the parent process leaves the children alive, so the whole process group should be killed. This can be achieved by passing the negated PGID (Process Group ID) to kill, which is the same as the parent's PID. ej: kill -TERM -$PARENT_PID
Do not execute the binary as
a child, but replacing the script
process with exec. You lose the
ability to execute stuff afterwards
though, because exec completely
replaces the parent process.
Do not kill the shell script process, but the FastCGI binary. Then, in the script, examine the return code and act accordingly. e.g: ./fastcgi_bin || exit -1
Depending on how mod_fastcgi handles worker processes, only the second option might be viable.
I have no idea if this is an option for you or not, but since you have a bounty I am assuming you might go for ideas that are outside the box.
Could you rewrite the bash script in Perl? Perl has several methods of managing child processes. You can read perldoc perlipc and more specifics in the core modules IPC::Open2 and IPC::Open3.
I don't know how this will interface with lighttpd etc or if there is more functionality in this approach, but at least it gives you some more flexibility and some more to read in your hunt.
I'm not sure I fully get your point, but here's what I tried and the process seems to be able to manage the trap (call it trap.sh):
#!/bin/bash
trap "echo trap activated" TERM INT
echo begin
time sleep 60
echo end
Start it:
./trap.sh &
And play with it (only one of those commands at once):
kill -9 %1
kill -15 %1
Or start in foreground:
./trap.sh
And interrupt with control-C.
Seems to work for me.
What exactly does not work for you?
I wrote this script just minutes ago to kill a bash script and all of its children...
#!/bin/bash
# This script will kill all the child process id for a given pid
# based on http://www.unix.com/unix-dummies-questions-answers/5245-script-kill-all-child-process-given-pid.html
ppid=$1
if [ -z $ppid ] ; then
echo "This script kills the process identified by pid, and all of its kids";
echo "Usage: $0 pid";
exit;
fi
for i in `ps j | awk '$3 == '$ppid' { print $2 }'`
do
$0 $i
kill -9 $i
done
Make sure the script is executable, or you will get an error on the $0 $i
You can override the implicit </dev/null for a background process by redirecting stdin yourself, for example:
sh -c 'exec 3<&0; { read x; echo "[$x]"; } <&3 3<&- & exec 3<&-; wait'
Try keeping the original stdin using ./fastcgi_bin 0<&0 &:
#!/bin/bash
# stuff
./fastcgi_bin 0<&0 &
PID=$!./fastcgi_bin 0<&0 &
trap "kill $PID" TERM
# stuff
# test
#sh -c 'sleep 10 & lsof -p ${!}'
#sh -c 'sleep 10 0<&0 & lsof -p ${!}'
You can do that with a coprocess.
Edit: well, coprocesses are background processes that can have stdin and stdout open (because bash prepares fifos for them). But you still need to read/write to those fifos, and the only useful primitive for that is bash's read (possibly with a timeout or a file descriptor); nothing robust enough for a cgi. So on second thought, my advice would be not to do this thing in bash. Doing the extra work in the fastcgi, or in an http wrapper like WSGI, would be more convenient.