I need some help because I don't get something. From what I read from Internet, a subshell is created when we execute a shell script or if we run command in brackets: ( )
I tried to test this with a script which contains only the following command:
ps -f
When I run it I see the following result:
ID PID PPID C STIME TTY TIME CMD
me 2213 2160 0 08:53 pts/14 00:00:00 bash
me 3832 2213 0 18:41 pts/14 00:00:00 bash
me 3833 3832 0 18:41 pts/14 00:00:00 ps -f
Which is good, because I see that my bash process has spawned another bash process for my script.
But when I do:
( ps -f )
it produces:
UID PID PPID C STIME TTY TIME CMD
me 2213 2160 0 08:53 pts/14 00:00:00 bash
me 3840 2213 0 18:46 pts/14 00:00:00 ps -f
So if brackets spawn a subshell why it is not shown in the processes? And why does ps -f is counted as another process? Does every command run as a separate process?
It seems you've caught bash in a little bit of an optimization. if a subshell contains only a single command, why really make it a subshell?
$ ( ps -f )
UID PID PPID C STIME TTY TIME CMD
jovalko 29393 24133 0 12:05 pts/10 00:00:00 bash
jovalko 29555 29393 0 12:07 pts/10 00:00:00 ps -f
However, add a second command, say : (the bash null command, which does nothing) and this is the result:
$ ( ps -f ; : )
UID PID PPID C STIME TTY TIME CMD
jovalko 29393 24133 0 12:05 pts/10 00:00:00 bash
jovalko 29565 29393 0 12:08 pts/10 00:00:00 bash
jovalko 29566 29565 0 12:08 pts/10 00:00:00 ps -f
One of the main reasons to use a subshell is that you can perform operations like I/O redirection on a group of commands instead a single command, but if your subshell contains only a single command there's not much reason to really fork a new bash process first.
As to ps counting as a process, it varies. Many commands you use like ls, grep, awk are all external programs. But, there are builtins like cd, kill, too.
You can determine which a command is in bash using the type command:
$ type ps
ps is hashed (/bin/ps)
$ type cd
cd is a shell builtin
The main part of the question is:
Does every command run as a separate process?
YES!. Every command what isn't built-in into bash (like declare and such), runs as separate process. How it is works?
When you type ps and press enter, the bash analyze what you typed, do usual things as globbing, variable expansionas and such, and finally when it is an external command
the bash forks itself.
The forking mean, than immediatelly after the fork, you will have two identical bash processes (each one with different process ID (PID)) - called as "parent" and "child", and the only difference between those two running bash programs is, than the "parent" gets (return value from the fork) the PID of the child but the child don't know the PID of the parent. (fork for the child returns 0).
after the fork, (bash is written this way) - the child replaces itself with the new program image (such: ps) - using the exec call.
after this, the child bash of course doesn't exist anymore, and running only the newly executed command - e.g. the ps.
Of course, when the bash going to fork itself, do many other things, like I/O redirections, opens-closes filehandles, changes signal handling for the child and many-many other things.
Related
I know that grouping commands(command-list) creates a subshell environment, and each listed command is executed in that subshell. But if I execute a simple command in the grouping command, (use the ps command to output the processes), then no subshell process is output. But if I tried to execute a list of commands (compound command) in the grouping command, then a subshell process is output. Why does it produce such a result?
A test of executing a simple command (only a ps command) in a grouping command:
[root#localhost ~]# (ps -f)
with the following output:
UID PID PPID C STIME TTY TIME CMD
root 1625 1623 0 13:49 pts/0 00:00:00 -bash
root 1670 1625 0 15:05 pts/0 00:00:00 ps -f
Another test of executing a compound command(a list of commands) in a grouping command:
[root#localhost ~]# (ps -f;cd)
with the following output:
UID PID PPID C STIME TTY TIME CMD
root 1625 1623 0 13:49 pts/0 00:00:00 -bash
root 1671 1625 0 15:05 pts/0 00:00:00 -bash
root 1672 1671 0 15:05 pts/0 00:00:00 ps -f
I tested a lot of other commands (compound commands and simple commands), but the results are the same. I guess even if I execute a simple command in a grouping command, bash should fork a subshell process, otherwise it can't execute the command. But why can't I see it?
Bash optimizes the execution. It detects that only one command is inside the ( ) group and calls fork + exec instead of fork + fork + exec. That's why you see one bash process less in the list of processes. It is easier to detect when using command that take more time ( sleep 5 ) to eliminate timing. Also, you may want to read this thread on unix.stackexchange.
I think the optimization is done somewhere inside execute_cmd.c in execute_in_subshell() function (arrows > added by me):
/* If this is a simple command, tell execute_disk_command that it
might be able to get away without forking and simply exec.
>>>> This means things like ( sleep 10 ) will only cause one fork
If we're timing the command or inverting its return value, however,
we cannot do this optimization. */
and in execute_disk_command() function we can also read:
/* If we can get away without forking and there are no pipes to deal with,
don't bother to fork, just directly exec the command. */
It looks like an optimization and dash appears to be doing it too:
Running
bash -c '( sleep 3)' & sleep 0.2 && ps #or with dash
as does, more robustly:
strace -f -e trace=clone dash -c '(/bin/sleep)' 2>&1 |grep clone # 1 clone
shows that the subshell is skipped, but if there's post work to be done in the subshell after the child, the subshell is created:
strace -f -e trace=clone dash -c '(/bin/sleep; echo done)' 2>&1 |grep clone #2 clones
Zsh and ksh are taking it even one step further and for (when they see it's the last command in the script):
strace -f -e trace=clone ksh -c '(/bin/sleep; echo done)' 2>&1 |grep clone # 0 clones
they don't fork (=clone) at all, execing directly in the shell process.
As we all know, placing a list of commands between curly braces causes the list to be executed in the current shell context. No subshell is created. But when using "&" after "{}", why two subshells are created? pid 1002 and 1003.
{
./a.out
} &
sleep 19
when using "./a.out &", only a subshell is created. pid 17358.
./a.out &
sleep 19
Why ?
Background execution of a list uses a subshell because something needs to wait for each member of that list and run the next one. After a list is backgrounded, the parent shell needs to be available for new commands; it can't manage the backgrounded list too. bash can't do more than one thing at a time. So, to make the backgrounded list work, it runs a subshell.
Note that you can disown a backgrounded list and it will keep running, showing that the subshell is doing its work:
$ {
> sleep 1; sleep 2; sleep 3; sleep 4; sleep 5
> } &
$ disown
$ ps -f | grep sleep
dave 31845 31842 0 03:50 pts/1 00:00:00 sleep 3
dave 31849 31771 0 03:50 pts/1 00:00:00 grep sleep
You could even log out and the subshell would continue running processes in the list.
When you background a single command, there is no need for a subshell because there is no more work for the shell to do after it has run the command.
In your example, the second additional bash subprocess, PID 1002, appears to be a script which you're executing. That's unrelated (conceptually, at least) to the list-backgrounding mechanism; any script in a separate file has its own bash process.
If a command is terminated by the control operator &, the shell executes the command (or list of commands that are enclosed in {...}) in the background (or asynchronously) in a subshell.
The shell does not wait for the command to finish, and the return status is 0.
In C program it is done by doing a fork() followed by execvp() system calls.
Update: Based on comments below and updated question. Here is what is happening.
When you run:
./a.out &
BASH directly just runs a.out in background as running binary a.out doesn't need a separate shell process.
When you run:
{ ./a.out; } &
BASH must first fork and create a subshell as you can have series of commands inside {...} and then the newly forked subshell runs a.out in a separate process. So it is not that BASH is creating 2 subshells for this. Only one subshell gets created and 2nd pid you're seeing is for a.out.
My unix production server has test.ksh files, but every day it's running on daily basics using job.
I want to know which crontab job is calling this script. I checked usign below command, but i didn't find exact job name,
crontab -l
--It has been listed 100 job --
I have analysed above mentioned 100 job, but i didn't get test.ksh file
crontab -l | grep "test.ksh"
--file not found
But the file available in one directory, I can't find which job is called test.ksh script.
Finding:
1. Whether it's child job? - If yes, how can i identify the child job?
you could use pstree -p xxxx where xxxx is the pid of crond. You will then get a nice hierarchical overview of all offspring processes of crond.
If it is a child script, use ps -ef and use the ppid of the test.ksh job to identify the calling script.
For example, consider these two scripts, the first just calls the second
parent
#! /bin/sh
# Run child process
./child
child
#! /bin/sh
sleep 60
ps -ef shows (with a lot of other processes removed)
UID PID PPID C STIME TTY TIME CMD
501 5725 5724 0 8:22pm ttys000 0:00.28 -bash
501 6046 5725 0 11:38am ttys000 0:00.01 /bin/sh ./parent
501 6047 6046 0 11:38am ttys000 0:00.00 /bin/sh ./child
501 6048 6047 0 11:38am ttys000 0:00.00 sleep 60
The pid is the process identifier, so child has process id 6047. Its ppid - 6046 - is the process id of its parent as you can see looking at the entry for the parent process.
Im' trying to find a way to execute a block in the same way I would execute a code calling an external script.
Let me exemplify ...
# caller.sh
!#/bin/bash
/soft/executer.sh &
After executing "caller.sh", the "ps" command return will be something like:
PID TTY TIME CMD
19566 pts/7 00:00:00 bash
22689 pts/7 00:00:00 executer.sh
22694 pts/7 00:00:00 ps
But if a change the way to call the script "caller.sh" like this:
# caller.sh
!#/bin/bash
{
/soft/executer.sh
} &
The "ps" command shows up both commands (caller.sh and executer.sh)
PID TTY TIME CMD
19566 pts/7 00:00:00 bash
22689 pts/7 00:00:00 caller.sh
22694 pts/7 00:00:00 ps
22685 pts/7 00:00:00 executer.sh
Both "caller.sh" and "ler.sh" commands are showing up.
I know I could simply use the first option to call this, but this is just a simple example to ask how to unlik the processes "caller.sh" and "execute.sh" in the second example that uses blocks
Thanks
I would try this in caller.sh:
#!/bin/bash
(
exec /soft/executer.sh
)&
The issue is that a block or subshell is merely a copy of the parent meaning the parent may be gone, but the child has the same name therefore shows up in ps. So, if you have:
#!/bin/bash
(
/soft/executer.sh
)&
sleep 60
You will see two copies of caller.sh (the parent and the child). The parent is sleeping and the child is waiting for executer.sh to finish.
There are various tricks to daemonize a linux process, i.e. to make a command running after the terminal is closed.
nohup is used for this purpose, and fork()/setsid() combination can be used in a C program to make itself a daemon process.
The above was my knowledge about linux daemon, but today I noticed that exiting the terminal doesn't really terminate processes started with & at the end of the command.
$ while :; do echo "hi" >> temp.log ; done &
[1] 11108
$ ps -ef | grep 11108
username 11108 11076 83 15:25 pts/0 00:00:05 /bin/sh
username 11116 11076 0 15:25 pts/0 00:00:00 grep 11108
$ exit
(after reconnecting)
$ ps -ef | grep 11108
username 11108 1 91 15:25 pts/0 00:00:17 /bin/sh
username 11130 11540 0 15:25 pts/0 00:00:00 grep 11108
So apparently, the process's PPID changed to 1, meaning that it got daemonized somehow.
This contradicts my knowledge, that & is not enough and one must use nohup or some other tricks to a process 'daemon'.
Does anyone know who is doing this daemonizing?
I'm using a CentOS 6.3 host and putty/cygwin/sshclient produced the same result.
You can daemonize a process if that doesn't respond to SIGHUP signal.
When bash shell is terminated while it is running background tasks, bash shell sends SIGHUP
(hangup signal) to all tasks. However bash won't wait until child processes are completely
terminated. If child process doesn't respond to SIGHUP signal, that process becomes an orphan
process. (its parent pid is changed to 1 - init process - to prevent becoming a useless zombie process)
Subshell executions basically do not responds to SIGHUP signals, thus your command will still be running after logging out from the first shell.