Background processes don't belong to a user and a terminal, nor do daemon processes. What is the main difference between the two? If I were to write a server program, should I run it as a background process or a daemon?
When one says 'background process', it's usually in the context of a shell (like bash), which implements job control.
When a process (or a process group) is put into the background, it's still part of the session created by the shell and will still have an association with the shell's controlling terminal. The standard input/output of a background process will still be linked to the terminal (unless explicitly changed). Also, depending on how the shell exits, it may send a SIGHUP signal to all the background processes (See this answer to know exactly when). Until the shell terminates, it remains the parent of the background process.
A daemon on the other hand does not have a controlling terminal and is usually explicitly made to be a child of the init process. The standard input/output of a dare usually redirected to /dev/null
A Background process usually refers to a process which:
Another process is its parent; eg, a shell;
It has standard streams (input, output, error) connected to that parent
The most common type is when you run a shell program with a trailing &. It generally shares the shell’s output streams, but will get a signal and stop if it tries to read from its input stream.
More significantly (usually), a background process like this is still parented, so signals to that process group will continue to it. If the parent process terminates, the children will receive signals that will most likely terminate them, as well. (This is probably the biggest difference between the two for most users.)
A Daemon process is one that:
Has no parent, ie, its parent process is the system (or container) initial thread, commonly systemd (Linux), init (other Unix), or launchd? (MacOS);
Typically has its output disconnected, or connected to a log file;
Typically has its input disconnected.
Daemons are usually also written to accept the “user hung up” signal (SIGHUP), which would terminate a program if not handled, as a special instruction to re-read their configuration files and continue working.
Most often, these are processes created by some system-level facility that continue to operate completely independently of user activity (logins, logouts, and the like). Things that, themselves, handle logins (getty or gdm and the like), as well as other network-facing services (web servers, mail servers, etc) may be daemons, as well as self-monitoring services like cron, or smartd.
Related
I am writing a Linux daemon . I found two ways to do it.
Daemonize your process by calling fork() and setting sid.
Running your program with &.
Which is the right way to do it?
From http://www.steve.org.uk/Reference/Unix/faq_2.html#SEC16
Here are the steps to become a daemon:
fork() so the parent can exit, this returns control to the command line or shell invoking your program. This step is required so that the new process is guaranteed not to be a process group leader. The next step, setsid(), fails if you're a process group leader.
setsid() to become a process group and session group leader. Since a controlling terminal is associated with a session, and this new session has not yet acquired a controlling terminal our process now has no controlling terminal, which is a Good Thing for daemons.
fork() again so the parent, (the session group leader), can exit. This means that we, as a non-session group leader, can never regain a controlling terminal.
chdir("/") to ensure that our process doesn't keep any directory in use. Failure to do this could make it so that an administrator couldn't unmount a filesystem, because it was our current directory. [Equivalently, we could change to any directory containing files important to the daemon's operation.]
umask(0) so that we have complete control over the permissions of anything we write. We don't know what umask we may have inherited. [This step is optional]
close() fds 0, 1, and 2. This releases the standard in, out, and error we inherited from our parent process. We have no way of knowing where these fds might have been redirected to. Note that many daemons use sysconf() to determine the limit _SC_OPEN_MAX. _SC_OPEN_MAX tells you the maximun open files/process. Then in a loop, the daemon can close all possible file descriptors. You have to decide if you need to do this or not. If you think that there might be file-descriptors open you should close them, since there's a limit on number of concurrent file descriptors.
Establish new open descriptors for stdin, stdout and stderr. Even if you don't plan to use them, it is still a good idea to have them open. The precise handling of these is a matter of taste; if you have a logfile, for example, you might wish to open it as stdout or stderr, and open '/dev/null' as stdin; alternatively, you could open '/dev/console' as stderr and/or stdout, and '/dev/null' as stdin, or any other combination that makes sense for your particular daemon.
Better yet, just call the daemon() function if it's available.
I suggest not writing your program as a daemon at all. Make it run in the foreground with the file descriptors, current directory, process group, etc as given to it.
If you want to then run this program as a daemon, use start-stop-daemon(8), init(8), runsv (from runit), upstart, systemd, or whatever to launch your process as a daemon. That is, let your user decide how to run your program and don't enforce that it must run as a daemon.
Just use daemon(3) (from unistd.h).
The daemon() function is for programs
wishing to detach themselves from the
controlling terminal and run in the
background as system daemons. ...
The first. The second is not daemonizing, but running on the background. Daemonized programs should be on its own session and process group, and should not have a controlling terminal.
Actually to make a daemon you have to double fork.
Running the program with a & makes the shell run the program in the background, which does not make it a daemon. Daemons have init (pid 1) as a parent, that's why the double fork is needed.
So the nice way to do things, if your program is a daemon, would be to take care of this issue yourself (there are more methods, see here too). You could also use the start-stop-daemon program.
What language are you using? Some languages have helper methods that make daemonizing easier. For example, Ruby has the daemons package.
I need to watch for a process with a known PID in Linux. Once it is terminated want to execute a command with reason of termination.
Questions
How to subscribe to process health rather than polling (e.g. watch
command)?
Where to inject the event handler in OS' user space?
How to detect the termination/failure reason inside handler?
Note
The process I intend to keep a tab on is not forked as a child process of some parent through which it can be monitored.
The process type is generic (good number of them are daemons)
The only way to get that kind of control over another process, is to use ptrace(2) to trace the target process. You would use ptrace(PTRACE_ATTACH, pid) to attach to the process, after which you effectively become the target process's parent (and can use wait or more ptrace calls to figure out what the process is doing).
In am writing an SDK in Go that integrators will communicate with via a local socket connection.
From the integrating application I need a way to start the SDK as a process but more importantly, I need to be able to cancel that process when the main application is closing too.
This question is language agnostic (I think) as I think the challenge is linux related. i.e. How to start a program and cancel it at a later stage.
Some possible approaches:
I am thinking that it's a case of starting the program via exec, getting it's PID or some ID then using that to kill later. Sudo may be required to do this, which is not ideal. Also, not good practice as you will be effectively force closing the SDK, offering no time for cleanup.
Start the program via any means. Once ready to close, just send a "shutdown" command via the SDK API which will allow the SDK to cleanup, manage state then exit the application.
What is best practice for this please?
Assuming you're using Linux or similar Unix:
You are on the right track. You won't need sudo. The comments thus far are pointing in the right direction, but not spelling out the details.
See section 2 of the manual pages (man 2 ...) for details on the functions mentioned here. They are documented for calling from C. I don't have experience with Go to help determine how to use them there.
The integrator application will be called the "parent" process. The SDK-as-a-process will be called the "child" process. A process creates a child and becomes its parent by calling fork(). The new process is a duplicate of the parent, running the same code, and having for the most part all the same state (data in memory). But fork() returns different values to parent and child, so each can determine its role in the relationship. This includes informing the parent of the process identifier (pid) of the child. Hang on to this value. Then, the child uses exec() to execute a different program within the existing process, i.e. your SDK binary. An alternative to fork-then-exec is posix_spawn(), which has rather involved parameters (but gives greater control if you need it).
Designing the child to shutdown in response to a signal, rather than a command through the API, will allow processes other than the parent to initiate clean shutdown in standard fashion. For example, this might be useful for the administrator or user; it enables sending the shutdown signal from the shell command-line or script.
The child installs a signal handler function, that will be called when the child process receives a signal, by calling signal() (or the more complex sigaction() recommended for its portability). There are different signals that can be sent/received, identified by different integer values (and also given names like SIGTERM). You indicate which you're interested in receiving when calling signal(). When your signal handler function is invoked, you've received the signal, and can initiate clean shutdown.
When the parent wants the child to shut down cleanly, the parent sends a signal to the child using the unfortunately named kill(). Unfortunately named because signals can be used for other purposes. Anyway, you pass to kill() the pid (returned by fork()) and the specific signal (e.g. SIGTERM) you want to send.
The parent can also determine when the child has completely shut down by calling waitpid(), again passing the pid returned by fork(); or alternately by registering to receive signal SIGCHLD. Register to receive SIGCHLD before fork()/exec() or you might miss the signal.
Actually, it's important that you do call waitpid(), optionally after receiving SIGCHLD, in order to deallocate a resource holding the child process's exit status, so the OS can cleanup that last remnant of the process. Failing to do so keeps the child as a "zombie" process, unable to be fully reclaimed. Too many zombies and the OS will be unable to launch new processes.
If a process refuses to shut down cleanly or as quickly as you require, you may force it to quit (without executing its cleanup code) by sending the signal SIGKILL.
There are variants of exec(), waitpid() and posix_spawn(), with different names and behaviors, mentioned in their man pages.
I am reading about daemonizing a process at https://en.wikipedia.org/wiki/Daemon_%28computing%29#Creation
In a strictly technical sense, a Unix-like system process is a daemon
when its parent process terminates and the daemon is assigned the init
process (process number 1) as its parent process and has no
controlling terminal. However, more commonly, a daemon may be any
background process, whether a child of the init process or not.
On a Unix-like system, the common method for a process to become a
daemon, when the process is started from the command line or from a
startup script such as an init script or a SystemStarter script,
involves:
Dissociating from the controlling tty
Becoming a session leader
Becoming a process group leader
Executing as a background task by forking and exiting (once or twice). This is required sometimes for the process to become a session
leader. It also allows the parent process to continue its normal
execution.
Setting the root directory (/) as the current working directory so that the process does not keep any directory in use that may be on
a mounted file system (allowing it to be unmounted).
Changing the umask to 0 to allow open(), creat(), and other operating system calls to provide their own permission masks and not
to depend on the umask of the caller
Closing all inherited files at the time of execution that are left open by the parent process, including file descriptors 0, 1 and 2
for the standard streams (stdin, stdout and stderr). Required files
will be opened later.
Using a logfile, the console, or /dev/null as stdin, stdout, and stderr
If the process is started by a super-server daemon, such as inetd,
launchd, or systemd, the super-server daemon will perform those
functions for the process[5][6][7] (except for old-style daemons not
converted to run under systemd and specified as Type=forking[7] and
"multi-threaded" datagram servers under inetd[5]).
Is there a step there that changes the parent process of a process
to be daemonized? It seems to me none of the steps does that?
Is changing parent process necessary when daemonize a process?
After changing the parent process of a process (a process not necessarily
to be daemonized), can the process be associated to the controlling
tty of the new parent process? (The purpose of the question is to
see whether "keeping a process disassociated from the the
controlling tty of the new parent process" is a necessary condition
of "changing the parent process of the process".)
See my related question https://unix.stackexchange.com/questions/266565/daemonize-a-process-in-shell
Thanks.
The parent of a Unix process can't be changed by the process itself. The typical method of creating a daemon involves a fork call (which creates the process that will become the daemon). The initial process then exits, and the newly-orphaned child process will be inherited by the init process which becomes it's new parent. That's handled in step 4. The only thing init will do is wait for all it's children to exit. init doesn't have a controlling TTY, so once inherited by init the daemon can't become associated with a controlling TTY anymore. The main reason to become disassociated is to prevent signals generated from the TTY (hangups and control-C's etc.) from getting to the daemon.
There are two ways daemons are usually run:
From a shell script. The script runs the daemon's executable with the & operator at the end of the command to put the daemon into the background, possibly with I/O redirection to set the daemon's stdin, stdout and/or stderr, and then exits leaving the daemon without a parent. Running an executable from the shell involves the shell doing a fork followed by an exec in the child process of the executable to be run.
The daemon program has an option to daemonize itself. When run with that option it does a fork followed in the child process by an exec of itself with an appropriate set of arguments. The parent will normally exit after the fork since the work it's been asked to do is done. If it doesn't, the child process needs an extra fork to give it a parent that can exit. NB: this is why so many programs that normally run as daemons can be run directly without becoming a daemon, the "become a daemon" option causes the child process to close stdin/stdout/stderr and then just exec it's own executable without the "become a daemon" option.
I would suggest to use daemon(3). See also credentials(7)
Your list does not mention explicitly setsid(2).
MUSL libc has a legacy/daemon.c which forks twice and do setsid
I've been studying this link to finish my shell assignment: http://www.gnu.org/software/libc/manual/html_node/Launching-Jobs.html#Launching-Jobs and it's been particularly helpful. My confusion is that, to give the shell control of the stdin file descriptor again, I need to call tcsetpgrp from the shell after the child is terminated.
How do I get tcsetpgrp() to work in C?
I've searched different Stack Overflow questions, but none properly tell me why GNU promotes this approach. Because the shell currently is in the "background", tcsetpgrp() will send SIGTTOU to my process group. The current solution is to ignore it before calling the method and maybe reset it to default afterwards. What should I do?
EDIT: I would like to note that the child is first set in another process group before the shell passes control of stdin to it with tcsetpgrp(). Once the child dies, the shell calls tcsetpgrp() to reclaim stdin. GNU suggests this as a possible implementation, but says it uses a slightly different implementation for simplicity here.
If tcsetpgrp() is called by a member of a background process group in its session, and the calling process is not blocking or ignoring SIGTTOU, a SIGTTOU signal is sent to all members of this background process group.
I'm also towards the end of implementing a shell program, and can take a stab at this question:
TL;DR: even though the shell will be a background process at that point, it needs to reclaim the terminal foreground for itself once the most recent foreground process group exits; otherwise the terminal will hang and no process will consume the user's input.
Here's what the session looks like when a shell launches a job from a line of command:
before the child processes call execve(), they are assigned into a process group, so that it's simpler to manage them as a job;
now that the shell and the child processes are in different process group, and that the terminal can only serve one process group at a time, the shell has to donate the terminal foreground to this child process group, so that the child job could read inputs and receive signals directly from the user;
after the foreground donation, the shell becomes a background process, but since there's no point in running the shell in the background (its main function is to read/write to the terminal and launch jobs), it should not proceed until it reclaims the foreground;
another reason why the shell must immediately reclaim the foreground after the exit of the most recent child foreground process group, is that there will be no running process consuming the terminal input/signal between the time a) when the child foreground process group exit, and b) when the shell reclaims the foreground. The terminal will essentially hang and become unresponsive;
as for why the shell must reclaim the foreground itself, it's because the child processes will be calling execve() and loading entirely new process images. It is extremely difficult to enforce and advocate for a contract where every child process in the foreground returns the foreground back to the shell process;
lastly, since tcsetpgrp() sends the caller background process a SIGTTOU, the shell must register an ignorer handler for SIGTTOU, at least during the time it's reclaiming the terminal foreground;
The GNU C Library manual also has another section (28.5.4 Foreground and Background) that briefly explains why the shell must reclaim the terminal forg