As we all know, placing a list of commands between curly braces causes the list to be executed in the current shell context. No subshell is created. But when using "&" after "{}", why two subshells are created? pid 1002 and 1003.
{
./a.out
} &
sleep 19
when using "./a.out &", only a subshell is created. pid 17358.
./a.out &
sleep 19
Why ?
Background execution of a list uses a subshell because something needs to wait for each member of that list and run the next one. After a list is backgrounded, the parent shell needs to be available for new commands; it can't manage the backgrounded list too. bash can't do more than one thing at a time. So, to make the backgrounded list work, it runs a subshell.
Note that you can disown a backgrounded list and it will keep running, showing that the subshell is doing its work:
$ {
> sleep 1; sleep 2; sleep 3; sleep 4; sleep 5
> } &
$ disown
$ ps -f | grep sleep
dave 31845 31842 0 03:50 pts/1 00:00:00 sleep 3
dave 31849 31771 0 03:50 pts/1 00:00:00 grep sleep
You could even log out and the subshell would continue running processes in the list.
When you background a single command, there is no need for a subshell because there is no more work for the shell to do after it has run the command.
In your example, the second additional bash subprocess, PID 1002, appears to be a script which you're executing. That's unrelated (conceptually, at least) to the list-backgrounding mechanism; any script in a separate file has its own bash process.
If a command is terminated by the control operator &, the shell executes the command (or list of commands that are enclosed in {...}) in the background (or asynchronously) in a subshell.
The shell does not wait for the command to finish, and the return status is 0.
In C program it is done by doing a fork() followed by execvp() system calls.
Update: Based on comments below and updated question. Here is what is happening.
When you run:
./a.out &
BASH directly just runs a.out in background as running binary a.out doesn't need a separate shell process.
When you run:
{ ./a.out; } &
BASH must first fork and create a subshell as you can have series of commands inside {...} and then the newly forked subshell runs a.out in a separate process. So it is not that BASH is creating 2 subshells for this. Only one subshell gets created and 2nd pid you're seeing is for a.out.
Related
Let's say I have a silly script:
while true;do
touch ~/test_file
sleep 3
done
And I start the script into the background and leave the terminal:
chmod u+x silly_script.sh
./silly_script.sh &
exit
Is there a way for me to identify and stop that script now? The way I see it is, that every command is started in it's own process and I might be able to catch and kill one command like the 'sleep 3' but not the execution of the entire script, am I mistaken? I expected a process to appear with the scripts name, but it does not. If I start the script with 'source silly_script.sh' I can't find a process by the name of 'source'. Do I need to identify the instance of bash, that is executing the script? How would I do that?
EDIT: There have been a few creative solutions, but so far they require the PID of the script execution to be stored right away, or the bash session to not be left with ^D or exit. I understand, that this way of running scripts should maybe be avoided, but I find it hard to believe, that any low privilege user could, even by accident, start an annoying script into the background, that is for instance filling the drive with garbage files or repeatedly starting new instances of some software and even the admin has no other option, than to restart the server, because a simple script can hide it's identifier without even trying.
With the help of the fine people here I was able to derive the answer I needed:
It is true, that the script runs every command in it's own process, so for instance killing the sleep 3 command won't do anything to the script being run, but through a command like the sleep 3 you can find the bash instance running the script, by looking for the parent process:
So after doing the above, you can run ps axf to show all processes in a tree form. You will then find this section:
18660 ? S 0:00 /bin/bash
18696 ? S 0:00 \_ sleep 3
Now you have found the bash instance, that is running the script and can stop it: kill 18660
(Of course your PID will be different from mine)
The jobs command will show you all running background jobs.
You can kill background jobs by id using kill, e.g.:
$ sleep 9999 &
[1] 58730
$ jobs
[1]+ Running sleep 9999 &
$ kill %1
[1]+ Terminated sleep 9999
$ jobs
$
58730 is the PID of the backgrounded task, and 1 is the task id of it. In this case kill 58730 and kill %1` would have the same effect.
See the JOB CONTROL section of man bash for more info.
When you exit, the backgrounded job will get a kill signal and die (assuming that's how it handles the signal - in your simple example it is), unless you disown it first.
That kill will propogate to the sleep process, which may well ignore it and continue sleeping. If this is the case you'll still see it in ps -e output, but with a parent pid of 1 indicating its original parent no longer exists.
You can use ps -o ppid= <pid> to find the parent of a process, or pstree -ap to visualise the job hierarchy and find the parent visually.
I call this in my local machine
ssh -t anon#192.168.50.81 -p 10086 'echo $SHELL && pstree'
I got /bin/zsh and a normal pstree output without shell process.
Why? And is the first output a fake one?
Some shells, like zsh, do not fork a child process to execute the last command in a command line or script. Since the exit status of the line or script is the same as the exit status of the last command, they call exec() in the shell process without forking. So if you execute
sleep 5 && pstree
it will fork a child for sleep, wait for it to finish, then call exec() to run pstree.
Since the pstree process replaces the shell, you don't see the shell in the process tree. pstree will be the child of sshd.
If you change it to
pstree && sleep 5
then you should see the shell in the pstree output, because pstree is no longer the last command.
I am trying to use the "nohup" command to avoid killing a background process when exiting the terminal on linux MATE.
The process I want to run is a MPIrun process and I use the following command:
nohup mpirun -np 8 solverName -parallel >log 2>&1
when I leave the terminal, the processes running on the different cores are killed.
Also another thing I remarked in the log file, is that if I try to just run the following command
mpirun -np 8 solverName -parallel >log 2>&1
and then to CTRL+Z (stopping the process) the log file indicates :
Forwarding signal 20 to job
and I am unable to actually stop the mpirun command. So I guess there is something I don't understand in what I am doing
The job run in the background is still owned by your login shell (the nohup command doesn't exit until the mpirun command terminates), so it gets signalled when you disconnect. This script (I call it bk) is what I use:
#!/bin/sh
#
# #(#)$Id: bk.sh,v 1.9 2008/06/25 16:43:25 jleffler Exp $"
#
# Run process in background
# Immune from logoffs -- output to file log
(
echo "Date: `date`"
echo "Command: $*"
nice nohup "$#"
echo "Completed: `date`"
echo
) >>${LOGFILE:=log} 2>&1 &
(If you're into curiosities, note the careful use of $* and "$#". The nice runs the job at a lower priority when I'm not there. And version 1.1 was checked into version control — SCCS at the time — on 1987-08-10.)
For your process, you'd run:
$ bk mpirun -np 8 solverName -parallel
$
The prompt returns almost immediately. The key differences between what is in that code and what you do direct from the command line are:
There's a sub-process for the shell script, which terminates promptly.
The script itself runs the command in a sub-shell in background.
Between them, these mean that the process is not interfered with by your login shell; it doesn't know about the grandchild process.
Running direct on the command line, you'd write:
(nohup mpirun -np 8 solverName -parallel >log 2>&1 &)
The parentheses start a subshell; the sub-shell runs nohup in the background with I/O redirection and terminates. The continuing command is a grandchild of your login shell and is not interfered with by your login shell.
I'm not an expert in mpirun, never having used it, so there's a chance it does something I'm not expecting. My impression from the manual page is that it acts more or less like a regular process even though it can run multiple other processes, possibly on multiple nodes. That is, it runs the other processes but monitors and coordinates them and only exits when its children are complete. If that's correct, then what I've outlined is accurate enough.
To kill the process you need the following command.
first:
$ jobs -l
this gives you the PID of the process like this
[1]+ 47274 Running nohup mpirun -np 8 solverName -parallel >log 2>&1
then execute the following command to kill the process.
kill -9 {program PID i.e 47274 }
this will help you with killing the process.
note that ctrl+Z does not kill the process but it suspends it.
for the first part of the question, I recommend to try this command and see if it works or not.
nohup nohup mpirun -n 8 --your_flags ./compited_solver_name > Output.txt &
it worked for me.
tell us if it doesn't work for you.
Bumped into an unexpected bash/sh behavior and I wonder someone can explain the rationale behind it, and provide a solution to the question below.
In an interactive bash shell session, I execute:
$ bash -c 'sleep 10 && echo'
With ps on Linux it looks like this:
\_ -bash
\_ bash -c sleep 10 && echo
\_ sleep 10
The process tree is what I would expect:
My interactive bash shell process ($)
A children shell process (bash -c ...)
a sleep children process
However, if the command portion of my bash -c is a single command, e.g.:
$ bash -c 'sleep 10'
Then the middle sub-shell is swallowed, and my interactive terminal session executes sleep "directly" as children process.
The process tree looks like this:
\_ -bash
\_ sleep 10
So from process tree perspective, these two produce the same result:
$ bash -c 'sleep 10'
$ sleep 10
What is going on here?
Now to my question: is there a way to force the intermediate shell, regardless of the complexity of the expression passed to bash -c ...?
(I could append something like ; echo; to my actual command and that "works", but I'd rather not. Is there a more proper way to force the intermediate process into existence?)
(edit: typo in ps output; removed sh tag as suggested in comments; one more typo)
There's actually a comment in the bash source that describes much of the rationale for this feature:
/* If this is a simple command, tell execute_disk_command that it
might be able to get away without forking and simply exec.
This means things like ( sleep 10 ) will only cause one fork.
If we're timing the command or inverting its return value, however,
we cannot do this optimization. */
if ((user_subshell || user_coproc) && (tcom->type == cm_simple || tcom->type == cm_subshell) &&
((tcom->flags & CMD_TIME_PIPELINE) == 0) &&
((tcom->flags & CMD_INVERT_RETURN) == 0))
{
tcom->flags |= CMD_NO_FORK;
if (tcom->type == cm_simple)
tcom->value.Simple->flags |= CMD_NO_FORK;
}
In the bash -c '...' case, the CMD_NO_FORK flag is set when determined by the should_suppress_fork function in builtins/evalstring.c.
It is always to your benefit to let the shell do this. It only happens when:
Input is from a hardcoded string, and the shell is at the last command in that string.
There are no further commands, traps, hooks, etc. to be run after the command is complete.
The exit status does not need to be inverted or otherwise modified.
No redirections need to be backed out.
This saves memory, causes the startup time of the process to be slightly faster (since it doesn't need to be forked), and ensures that signals delivered to your PID go direct to the process you're running, making it possible for the parent of sh -c 'sleep 10' to determine exactly which signal killed sleep, should it in fact be killed by a signal.
However, if for some reason you want to inhibit it, you need but set a trap -- any trap will do:
# run the noop command (:) at exit
bash -c 'trap : EXIT; sleep 10'
(OSX 10.7) An application we use let us assign scripts to be called when certain activities occur within the application. I have assigned a bash script and it's being called, the problem is that what I need to do is to execute a few commands, wait 30 seconds, and then execute some more commands. If I have my bash script do a "sleep 30" the entire application freezes for that 30 seconds while waiting for my script to finish.
I tried putting the 30 second wait (and the second set of commands) into a separate script and calling "./secondScript &" but the application still sits there for 30 seconds doing nothing. I assume the application is waiting for the script and all child processes to terminate.
I've tried these variations for calling the second script from within the main script, they all have the same problem:
nohup ./secondScript &
( ( ./secondScript & ) & )
( ./secondScript & )
nohup script -q /dev/null secondScript &
I do not have the ability to change the application and tell it to launch my script and not wait for it to complete.
How can I launch a process (I would prefer the process to be in a scripting language) such that the new process is not a child of the current process?
Thanks,
Chris
p.s. I tried the "disown" command and it didn't help either. My main script looks like this:
[initial commands]
echo Launching second script
./secondScript &
echo Looking for jobs
jobs
echo Sleeping for 1 second
sleep 1
echo Calling disown
disown
echo Looking again for jobs
jobs
echo Main script complete
and what I get for output is this:
Launching second script
Looking for jobs
[1]+ Running ./secondScript &
Sleeping for 1 second
Calling disown
Looking again for jobs
Main script complete
and at this point the calling application sits there for 45 seconds, waiting for secondScript to finish.
p.p.s
If, at the top of the main script, I execute "ps" the only thing it returns is the process ID of the interactive bash session I have open in a separate terminal window.
The value of $SHELL is /bin/bash
If I execute "ps -p $$" it correctly tells me
PID TTY TIME CMD
26884 ?? 0:00.00 mainScript
If I execute "lsof -p $$" it gives me all kinds of results (I didn't paste all the columns here assuming they aren't relevant):
FD TYPE NAME
cwd DIR /private/tmp/blahblahblah
txt REG /bin/bash
txt REG /usr/lib/dyld
txt REG /private/var/db/dyld/dyld_shared_cache_x86_64
0 PIPE
1 PIPE -> 0xffff8041ea2d10
2 PIPE -> 0xffff 8017d21cb
3r DIR /private/tmp/blahblah
4r REG /Volumes/DATA/blahblah
255r REG /Volumes/DATA/blahblah
The typical way of doing this in Unix is to double fork. In bash, you can do this with
( sleep 30 & )
(..) creates a child process, and & creates a grandchild process. When the child process dies, the grandchild process is inherited by init.
If this doesn't work, then your application is not waiting for child processes.
Other things it may be waiting for include the session and open lock files:
To create a new session, Linux has a setsid. On OS X, you might be able to do it through script, which incidentally also creates a new session:
# Linux:
setsid sleep 30
# OS X:
nohup script -q -c 'sleep 30' /dev/null &
To find a list of inherited file descriptors, you can use lsof -p yourpid, which will output something like:
sleep 22479 user 0u CHR 136,32 0t0 35 /dev/pts/32
sleep 22479 user 1u CHR 136,32 0t0 35 /dev/pts/32
sleep 22479 user 2u CHR 136,32 0t0 35 /dev/pts/32
sleep 22479 user 5w REG 252,0 0 1048806 /tmp/lockfile
In this case, in addition to the standard FDs 0, 1 and 2, you also have a fd 5 open with a lock file that the parent can be waiting for.
To close fd 5, you can use exec 5>&-. If you think the lock file might be stdin/stdout/stderr themselves, you can use nohup to redirect them to something else.
Another way is to abandon the child
#!/bin/bash
yourprocess &
disown
As far as I understand, the application replaces the normal bash shell because it is still waiting for a process to finish even if init should have taken care of this child process.
It could be that the "application" intercepts the orphan handling which is normally done by init.
In that case, only a parallel process with some IPC can offer a solution (see my other answer)
I think it depends on how your parent process tries to detect if your child process has been finished.
In my case (my parent process was gnu make), I succeed by closing stdout and stderr (slightly based on the answer of that other guy) like this:
sleep 30 >&- 2>&- &
You might also close stdin
sleep 30 <&- >&- 2>&- &
or additionally disown your child process (not for Mac)
sleep 30 <&- >&- 2>&- & disown
Currently tested only in bash on kubuntu 14.04 and Mac OSX.
If all else fails:
Create a named pipe
start the "slow" script independent from the "application", make sure executes it's task in an endless loop, starting with reading from the pipe. It will become read-blocked when it tries to read..
from the application, start your other script. When it needs to invoke the "slow" script, just write some data to the pipe. The slow script will start independently so your script won't wait for the "slow" script to finish.
So, to answer the question:
bash - how can I launch a new process that is NOT a child of the original process?
Simple: don't launch it but let an independent entity launch it during boot...like init or on the fly with the command at or batch
Here I have a shell
└─bash(13882)
Where I start a process like this:
$ (urxvt -e ssh somehost&)
I get a process tree (this output snipped from pstree -p):
├─urxvt(14181)───ssh(14182)
where the process is parented beneath pid 1 (systemd in my case).
However, had I instead done this (note where the & is) :
$ (urxvt -e ssh somehost)&
then the process would be a child of the shell:
└─bash(13882)───urxvt(14181)───ssh(14182)
In both cases the shell prompt is immediately returned and I can exit
without terminating the process tree that I started above.
For the latter case the process tree is reparented beneath pid 1 when
the shell exits, so it ends up the same as the first example.
├─urxvt(14181)───ssh(14182)
Either way, the result is a process tree that outlives the shell. The
only difference is the initial parenting of that process tree.
For reference, you can also use
nohup urxvt -e ssh somehost &
urxvt -e ssh somehost & disown $!
Both give the same process tree as the second example above.
└─bash(13882)───urxvt(14181)───ssh(14182)
When the shell is terminated the process tree is, like before, reparented
to pid 1.
nohup additionally redirects the process' standard output to a file
nohup.out so, if that is a useful trait, it may be a more useful choice.
Otherwise, with the first form above, you immediately have a completely
detached process tree.