In a bash script, I want to launch a process in the foreground, then print a list of all the process names and PIDs that were started as children of that process. For example, suppose I have the following scripts, but I can only modify the first one:
A.sh:
#!/bin/bash
B.sh
B.sh:
#!/bin/bash
C.sh
C.sh:
#!/bin/bash
echo "Running C.sh"
Without modifying B.sh, C.sh or the echo command, and without starting any of the child processes in the background, I would like A.sh to print the following:
B.sh 1208
C.sh 1210
echo 1211
Can A.sh fork a process that records this information while the child processes are running in the foreground of A.sh?
Update: In the comments below my answer it turned out that:
I need something that observes the creation of all child processes during a span of time. Given that, filtering to isolate my subtree will not be difficult.
... was the intention behind the question and it was for debugging purposes.
In that case I'd recommend to use strace like this:
strace -f command
-f will track child processes - recursively. Since forking and exec-ing requires system calls, strace will list any child creation plus the pids.
Original answer:
You can use pgrep for that:
run_process &
pid=${!}
pgrep --parent "${pid}"
wait # wait for run_process to finish
Btw, you may want to use the pstree command, it is nice to use:
run_process &
pid=${!}
pstree -p "${pid}"
wait # wait for run_process to finish
Anyhow, you'll need to install pstree.
You can try doing this with A.sh
#!/usr/bin/env bash
./B.sh &
b_PID=$!
./C.sh &
c_PID=$!
echo "B.sh $b_PID"
echo "C.sh $c_PID"
The output will look something like this
B.sh 22802
C.sh 22803
Running C.sh
Related
Hi all/ I'm trying to make something like parallel tool for shell simply because the functionality of parallel is not enough for my task. The reason is that I need to run different versions of compiler.
Imagine that I need to compile 12 programs with different compilers, but I can run only 4 of them simultaneously (otherwise PC runs out of memory and crashes :). I also want to be able to observe what's going on with each compile, therefore I execute every compile in new window.
Just to make it easier here I'll replace compiler that I run with small script that waits and returns it's process id sleep.sh:
#!/bin/bash
sleep 30
echo $$
So the main script should look like parallel_run.sh :
#!/bin/bash
for i in {0..11}; do
xfce4-terminal -H -e "./sleep.sh" &
pids[$i]=$!
pstree -p $pids
if (( $i % 4 == 0 ))
then
for pid in ${pids[*]}; do
wait $pid
done
fi
done
The problem is that with $! I get pid of xfce4-terminal and not the program it executes. So if I look at ptree of 1st iteration I can see output from main script:
xfce4-terminal(31666)----{xfce4-terminal}(31668)
|--{xfce4-terminal}(31669)
and sleep.sh says that it had pid = 30876 at that time. Thus wait doesn't work at all in this case.
Q: How to get right PID of compiler that runs in subshell?
Maybe there is the other way to solve task like this?
It seems like there is no way to trace PID from parent to child if you invoke process in new xfce4-terminal as terminal process dies right after it executed given command. So I came to the solution which is not perfect, but acceptable in my situation. I run and put compiler's processes in background and redirect output to .log file. Then I run tail on these logfiles and I kill all tails which belongs to current $USER when compilers from current batch are done, then I run the other batch.
#!/bin/bash
for i in {1..8}; do
./sleep.sh > ./process_$i.log &
prcid=$!
xfce4-terminal -e "tail -f ./process_$i.log" &
pids[$i]=$prcid
if (( $i % 4 == 0 ))
then
for pid in ${pids[*]}; do
wait $pid
done
killall -u $USER tail
fi
done
Hopefully there will be no other tails running at that time :)
I was trying to get pid of process I ran with setsid and which ought to run in background like this:
test.sh:
#/bin/bash
setsid nohup ./my_program &
echo $!
if I run ./test.sh it will print a pid of my_program process and it's exactly what I need. But if run this commands one by one in my shell like this:
$ setsid nohup ./my_program &
$ echo $!
It will give me a pid of setsid command (or may be something else, but it almost all times gives me pid of my_program minus one).
What is happening here? Why results of commands I ran in terminal by myself differs from results of test.sh script?
Btw, may be you know some easy way of process which I started with setsid and which I need to run in background?
Repost of comments above as an answer:
This is because setsid only forks the current process if it is the process group leader. A detailed explanation can be found here.
To get the pid of a process executed via setsid, the approaches given here may be tried.
setsid will call fork to ensure that it creates a new process group aswell as a new session, hence the resulting pid will not match the pid of setsid. The most clean work-around would be that my_program stores its pid into a file.
When you later want to send kill to my_program, you should check that the pid actually matches a program named my_program via /proc file system or calling the ps command with some magic code around it. (This is a very common method used by many daemons)
I have a lengthy menu script that relies on a few command outputs for it's variables. These commands take several seconds to run each and I would like to spawn new processes to set these variables. It would look something like this:
VAR1=`somecommand` &
VAR2=`somecommand` &
...
wait
echo $VAR1 $VAR2
The problem is that the processes are spawned and die with those variables they set. I realize that I can do this by sending these to a file and then reading that but I would like to do it without a temp file. Any ideas?
You can get the whole process' output using command substitution, like:
VAR1=$(somecommand &)
VAR2=$(somecommand &)
...
wait
echo $VAR1 $VAR2
This is rather clunky, but works for me. I have three scripts.
cmd.sh is your "somecommand", it is a test script only:
#!/bin/ksh
sleep 10
echo "End of job $1"
Below is wrapper.sh, which runs a single command, captures the output, signals the parent when done, then writes the result to stdout:
#!/bin/ksh
sig=$1
shift
var=$($#)
kill -$sig $PPID
echo $var
and here is the parent script:
#!/bin/ksh
trap "read -u3 out1" SIGUSR1
trap "read -p out2" SIGUSR2
./wrapper.sh SIGUSR1 ./cmd.sh one |&
exec 3<&p
exec 4>&p
./wrapper.sh SIGUSR2 ./cmd.sh two |&
wait
wait
echo "out1: $out1, out2: $out2"
echo "Ended"
2x wait because the first will be interrupted.
In the parent script I am running the wrapper twice, once for each job, passing in the command to be run and any arguments. The |& means "pipe to background" - run as a co-process.
The two exec commands copy the pipe file descriptors to fds 3 and 4. When the jobs are finished, the wrapper signals the main process to read the pipes. The signals are caught using the trap, which read the pipe for the appropriate child process, and gather the resulting data.
Rather convoluted and clunky, but it appears to work.
How to set process group of a shell script ? Also I want all the child process to be in the same process group
I expect something similar to setpgid() in C.
As PSkocik points out, it is possible to run a process in its own process group, in most shells, by activating job control (“monitor mode”).
(set -m; exec process_in_its_own_group)
Linux has a setsid utility, which runs the command passed as argument in its own session (using the eponymous system call). This is stronger than running it in its own process group à la setpgrp, but that may be ok for your purpose.
If you want to place the process in an existing group rather than in its own group (i.e. if you want the full power of setpgid), there's no common shell utility. You have to use C/Perl/…
I'll answer part of what I understand:
How to force current bash shell script to be it self process group:
I put this in the beginning of my bash script:
pgid_from_pid() {
local pid=$1
ps -o pgid= "$pid" 2>/dev/null | egrep -o "[0-9]+"
}
pid="$$"
if [ "$pid" != "$(pgid_from_pid $pid)" ]; then
exec setsid "$(readlink -f "$0")" "$#"
fi
Why do I would need this ?
When launching a program from an interactive bash session, it gets its own new process group. But this is not the case if your program is called from a bash script (non-interactive). If your program relies on being the process group owner in both condition you'll need this.
I don't think Bourne, bash, or zsh will let you do that, but you could do it in perl using the built-in setpgrp (note the slight name difference from POSIX). Pass zero as the PID to modify the group of the perl process itself:
setpgrp(0, 12345) || die "$!"
You might think you could use perl from, say, bash to set the bash process's group (by passing $$ to a perl script, for example), but I don't think the perl process would be able to modify the group of a process that it didn't fork.
Depending on what you're trying to do, the job control features in various shells may give you what you need, in a different way, like if you just want to detach from the terminal.
UPDATE: I think it's strange that this answer has received a couple of down-votes without clear explanation why. My guess is that the downvoters are misunderstanding the question, which is asking how to change the process group of the current shell. Or perhaps they know how to do a setpgrp from the shell but are keeping the secret to themselves.
If you turn set -m on, new processes will be spawned in a new process group, and if they're backgrounded, they won't have SIGINT and SIGQUIT ignored.
if [ $$ = $(ps -o pgid -hp $$) ]; then
echo already a process group leader;
else
set -m
$0 "$#" #optionally with &
set +m
fi
The new processes group of programs run after set -m takes over as the foreground process group of the terminal, unless they're run in the background.
The set -m is apparently semi-standard, required by POSIX if the implementation supports "User Portability Utilities".
In practice it works on bash, dash, ksh, pdksh, sh, yash, and zsh. posh doesn't have it.
Here's a late synthesis, taken from several other good answers here, if your intention is to cleanup any spawned subshell processes (even if the script itself is not directly launched from an interactive shell, but from another process, and therefore doesn't automatically becomes its own process group leader), relaunching the current script as a new process group leader if necessary.
# First, obtain the current PGID, by parsing the output of "ps".
pgid=$(($(ps -o pgid= -p "$$")))
# Check if we're already the process group leader; if not, re-launch ourselves.
# Use setsid instead of set -m (...) to avoid having another subshell in between. This helps that the trap gets executed when the script is killed.
[ $$ -eq $pgid ] || exec setsid --wait "${BASH_SOURCE[0]}" "$#"
# Kill any subshell processes when the script exits.
trap "kill -- -$pgid" EXIT
# Note: If the script only starts background jobs, and that's all you care about, you can replace all of the above with this simple trap:
#trap "jobs -p | xargs kill --" EXIT # Kill remaining jobs when the script exits.
Nested commands
Another complication is introduced when one script that does subshell cleanup is invoked by another such script. The process group leadership does not nest; once a script assumes leadership, its lifetime is not controlled any longer by a parent script, so when the parent script gets interrupted or killed, the nested script will linger on. That's not what the user usually wants.
The following script fragments extend the above implementation with a cooperation model, so that only the toplevel script assumes the process group leadership, indicating this to subshells by exporting $PGID. If a subshell finds an existing leader, it will not assume leadership itself, and limits its own cleanup tasks to remaining jobs. Other subshells will only be killed once the toplevel script exits. (So this cooperation model works best when one script only invokes one or only few other scripts.)
if [ -z "$PGID" ]; then # No parent script has become the process group leader yet.
pgid=$(($(ps -o pgid= -p "$$"))) # By defining this, we'll be killing subshell processes of this process group when we're done or interrupted. Any children with the same ambition will defer to us.
if [ $$ -eq $pgid ]; then
export PGID=$pgid # We are (already / after setsid) in our own process group, announce our leadership to any children, so that they don't become leaders themselves and thereby decouple themselves from our lifetime control.
else
exec setsid --wait "${BASH_SOURCE[0]}" "$#" # Use setsid instead of set -m (...) to avoid having another subshell in between.
fi
fi
if [ -n "$pgid" ]; then
trap "kill -- -$pgid" EXIT # If we're the leader, kill subshell processes when the script exits.
else
trap "jobs -p | xargs kill --" EXIT # Someone else is the leader; killing remaining jobs is all we can do here.
fi
As #Rob Davis pointed out in his answer, setting process group is not what you want for shells.
Instead you want to use their process control mechanisms. This answer covers doing this for sh on linux and borne. In short:
#! /bin/sh
# Kill all opened jobs on exit.
trap 'kill $(jobs -p)' EXIT
This will kill any jobs opened in the backrground (e.g. with &).
I am wrapping a fastcgi app in a bash script like this:
#!/bin/bash
# stuff
./fastcgi_bin
# stuff
As bash only executes traps for signals when the foreground script ends I can't just kill -TERM scriptpid because the fastcgi app will be kept alive.
I've tried sending the binary to the background:
#!/bin/bash
# stuff
./fastcgi_bin &
PID=$!
trap "kill $PID" TERM
# stuff
But if I do it like this, apparently the stdin and stdout aren't properly redirected because it does not connect with lighttpds mod_fastgi, the foreground version does work.
EDIT: I've been looking at the problem and this happens because bash redirects /dev/null to stdin when a program is launched in the background, so any way of avoiding this should solve my problem as well.
Any hint on how to solve this?
There are some options that come to my mind:
When a process is launched from a shell script, both belong to the same process group. Killing the parent process leaves the children alive, so the whole process group should be killed. This can be achieved by passing the negated PGID (Process Group ID) to kill, which is the same as the parent's PID. ej: kill -TERM -$PARENT_PID
Do not execute the binary as
a child, but replacing the script
process with exec. You lose the
ability to execute stuff afterwards
though, because exec completely
replaces the parent process.
Do not kill the shell script process, but the FastCGI binary. Then, in the script, examine the return code and act accordingly. e.g: ./fastcgi_bin || exit -1
Depending on how mod_fastcgi handles worker processes, only the second option might be viable.
I have no idea if this is an option for you or not, but since you have a bounty I am assuming you might go for ideas that are outside the box.
Could you rewrite the bash script in Perl? Perl has several methods of managing child processes. You can read perldoc perlipc and more specifics in the core modules IPC::Open2 and IPC::Open3.
I don't know how this will interface with lighttpd etc or if there is more functionality in this approach, but at least it gives you some more flexibility and some more to read in your hunt.
I'm not sure I fully get your point, but here's what I tried and the process seems to be able to manage the trap (call it trap.sh):
#!/bin/bash
trap "echo trap activated" TERM INT
echo begin
time sleep 60
echo end
Start it:
./trap.sh &
And play with it (only one of those commands at once):
kill -9 %1
kill -15 %1
Or start in foreground:
./trap.sh
And interrupt with control-C.
Seems to work for me.
What exactly does not work for you?
I wrote this script just minutes ago to kill a bash script and all of its children...
#!/bin/bash
# This script will kill all the child process id for a given pid
# based on http://www.unix.com/unix-dummies-questions-answers/5245-script-kill-all-child-process-given-pid.html
ppid=$1
if [ -z $ppid ] ; then
echo "This script kills the process identified by pid, and all of its kids";
echo "Usage: $0 pid";
exit;
fi
for i in `ps j | awk '$3 == '$ppid' { print $2 }'`
do
$0 $i
kill -9 $i
done
Make sure the script is executable, or you will get an error on the $0 $i
You can override the implicit </dev/null for a background process by redirecting stdin yourself, for example:
sh -c 'exec 3<&0; { read x; echo "[$x]"; } <&3 3<&- & exec 3<&-; wait'
Try keeping the original stdin using ./fastcgi_bin 0<&0 &:
#!/bin/bash
# stuff
./fastcgi_bin 0<&0 &
PID=$!./fastcgi_bin 0<&0 &
trap "kill $PID" TERM
# stuff
# test
#sh -c 'sleep 10 & lsof -p ${!}'
#sh -c 'sleep 10 0<&0 & lsof -p ${!}'
You can do that with a coprocess.
Edit: well, coprocesses are background processes that can have stdin and stdout open (because bash prepares fifos for them). But you still need to read/write to those fifos, and the only useful primitive for that is bash's read (possibly with a timeout or a file descriptor); nothing robust enough for a cgi. So on second thought, my advice would be not to do this thing in bash. Doing the extra work in the fastcgi, or in an http wrapper like WSGI, would be more convenient.