It seems that there is no difference between nohup and at now, but maybe there are subtleties?
The difference is that now runs a command that can respond to HUP signal, where as the nohup runs a command that is immune to HUP signal.
Ed Heal is right. But another difference is that something run by nohup still has a controlling terminal, whereas something run by at now does not.
In addition to that, backgrounding something with nohup causes it to run immediately, whereas at now simply queues something to be run the next time atrun(8) runs. In BSD unix, (FreeBSD/OpenBSD) at jobs are launched by atrun which is launched periodically by cron (or launchd in OSX). In Linux, at jobs are run by at's own daemon, atd, which by default launches jobs every 60 seconds.
Other flavours of unix may have different strategies, but in most cases you'll probably find that jobs launched by at now are less immediate than jobs launched using nohup.
nohup tells the system to continue running even after you log out.
at is used to execute a command or multiple commands once at some future time.
Related
I have an embedded system, on which I do telnet and then I run an application in background:
./app_name &
Now if I close my terminal and do telnet from other terminal and if I check then I can see this process is still running.
To check this I have written a small program:
#include<stdio.h>
main()
{
while(1);
}
I ran this program in my local linux pc in background and I closed the terminal.
Now, when I checked for this process from other terminal then I found that this process was also killed.
My question is:
Why undefined behavior for same type of process?
On which it is dependent?
Is it dependent on version of Linux?
Who should kill jobs?
Normally, foreground and background jobs are killed by SIGHUP sent by kernel or shell in different circumstances.
When does kernel send SIGHUP?
Kernel sends SIGHUP to controlling process:
for real (hardware) terminal: when disconnect is detected in a terminal driver, e.g. on hang-up on modem line;
for pseudoterminal (pty): when last descriptor referencing master side of pty is closed, e.g. when you close terminal window.
Kernel sends SIGHUP to other process groups:
to foreground process group, when controlling process terminates;
to orphaned process group, when it becomes orphaned and it has stopped members.
Controlling process is the session leader that established the connection to the controlling terminal.
Typically, the controlling process is your shell. So, to sum up:
kernel sends SIGHUP to the shell when real or pseudoterminal is disconnected/closed;
kernel sends SIGHUP to foreground process group when the shell terminates;
kernel sends SIGHUP to orphaned process group if it contains stopped processes.
Note that kernel does not send SIGHUP to background process group if it contains no stopped processes.
When does bash send SIGHUP?
Bash sends SIGHUP to all jobs (foreground and background):
when it receives SIGHUP, and it is an interactive shell (and job control support is enabled at compile-time);
when it exits, it is an interactive login shell, and huponexit option is set (and job control support is enabled at compile-time).
See more details here.
Notes:
bash does not send SIGHUP to jobs removed from job list using disown;
processes started using nohup ignore SIGHUP.
More details here.
What about other shells?
Usually, shells propagate SIGHUP. Generating SIGHUP at normal exit is less common.
Telnet or SSH
Under telnet or SSH, the following should happen when connection is closed (e.g. when you close telnet window on PC):
client is killed;
server detects that client connection is closed;
server closes master side of pty;
kernel detects that master pty is closed and sends SIGHUP to bash;
bash receives SIGHUP, sends SIGHUP to all jobs and terminates;
each job receives SIGHUP and terminates.
Problem
I can reproduce your issue using bash and telnetd from busybox or dropbear SSH server: sometimes, background job doesn't receive SIGHUP (and doesn't terminate) when client connection is closed.
It seems that a race condition occurs when server (telnetd or dropbear) closes master side of pty:
normally, bash receives SIGHUP and immediately kills background jobs (as expected) and terminates;
but sometimes, bash detects EOF on slave side of pty before handling SIGHUP.
When bash detects EOF, it by default terminates immediately without sending SIGHUP. And background job remains running!
Solution
It is possible to configure bash to send SIGHUP on normal exit (including EOF) too:
Ensure that bash is started as login shell. The huponexit works only for login shells, AFAIK.
Login shell is enabled by -l option or leading hyphen in argv[0]. You can configure telnetd to run /bin/bash -l or better /bin/login which invokes /bin/sh in login shell mode.
E.g.:
telnetd -l /bin/login
Enable huponexit option.
E.g.:
shopt -s huponexit
Type this in bash session every time or add it to .bashrc or /etc/profile.
Why does the race occur?
bash unblocks signals only when it's safe, and blocks them when some code section can't be safely interrupted by a signal handler.
Such critical sections invoke interruption points from time to time, and if signal is received when a critical section is executed, it's handler is delayed until next interruption point happens or critical section is exited.
You can start digging from quit.h in the source code.
Thus, it seems that in our case bash sometimes receives SIGHUP when it's in a critical section. SIGHUP handler execution is delayed, and bash reads EOF and terminates before exiting critical section or calling next interruption point.
Reference
"Job Control" section in official Glibc manual.
Chapter 34 "Process Groups, Sessions, and Job Control" of "The Linux Programming Interface" book.
When you close the terminal, shell sends SIGHUP to all background processes – and that kills them. This can be suppressed in several ways, most notably:
nohup
When you run program with nohup it catches SIGHUP and redirect program output.
$ nohup app &
disown
disown tells shell not to send SIGHUP
$ app &
$ disown
Is it dependent on version of linux?
It is dependent on your shell. Above applies at least for bash.
AFAIK in both cases the process should be killed. In order to avoid this you have to issue a nohup like the following:
> nohup ./my_app &
This way your process will continue executing. Probably the telnet part is due to a BUG similar to this one:
https://bugzilla.redhat.com/show_bug.cgi?id=89653
In order completely understand whats happening you need to get into unix internals a little bit.
When you are running a command like this
./app_name &
The app_name is sent to background process group. You can check about unix process groups here
When you close bash with normal exit it triggers SIGHUP hangup signal to all its jobs. Some information on unix job control is here.
In order to keep your app running when you exit bash you need to make your app immune to hangup signal with nohup utility.
nohup - run a command immune to hangups, with output to a non-tty
And finally this is how you need to do it.
nohup app_name & 2> /dev/null;
In modern Linux--that is, Linux with systemd--there is an additional reason this might happen which you should be aware of: "linger".
systemd kills processes left running from a login shell, even if the process is properly daemonized and protected from HUP. This is the default behavior in modern configurations of systemd.
If you run
loginctl enable-linger $USER
you can disable this behavior, allowing background processes to keep running. The mechanisms covered by the other answers still apply, however, and you should also protect your process against them.
enable-linger is permanent until it is re-disabled. You can check it with
ls /var/lib/systemd/linger
This may have files, one per username, for users who have enable-linger. Any user listed in the directory has the ability to leave background processes running at logout.
I've searched, googled, sat in IRC for a week and even talked to a friend who is devoutly aligned with linux but I haven't yet received a solid answer.
I have written a shell script that runs as soon as I log into my non-root user and runs basically just does "./myprogram &" without quotation. When I exit shh my program times out and I am unable to connect to it until I log back in. How can I keep my program running after I exit SSH of my non-root user?
I am curious if this has to be done on the program level or what? My apologizes if this does not belong here, I am not sure where it goes to be perfectly honest.
Beside using nohup, you can run your program in terminal multiplexer like screen or tmux. With them, you can reattach to sessions, which is for example quite helpful if you need to run terminal-based interactive programs or long time running scripts over a unstable ssh connections.
boybu is a nice enhancement of screen.
Try nohup: http://linux.die.net/man/1/nohup
Likely your program receives a SIGHUP signal when you exit your ssh session.
There's two signals that can cause your program to die after your ssh session ends: SIGHUP and SIGPIPE.
SIGHUP will be sent to your program because the parent process (ssh) has died. You can get around this either by using the program nohup (i.e. nohup ./myprogram &) or by using the shell builtin disown (./myprogram& disown)
SIGPIPE will be sent to your program if it tries to write to stdout or stderr after the ssh session has been disconnected. To get around this, redirect them to a file or /dev/null, i.e. nohup ./myprogram >/dev/null 2>/dev/null &
You might also want to use the batch (or at) command, in addition to the other answers (nohup, screen, ...). And ssh has a -f option which might interest you.
I want to run a process in a remote linux server and keep that process alive after close the putty terminal,
what is the correct command?
You have two options:
Use GNU screen, which will allow you to run the command and detach it from your terminal, and later re-attach it to a different session. I use it for long-running processes whose output I want to be able to monitor at any time. Screen is a truly powerful tool and I would highly recommend spending some time to learn it.
Run the command as nohup some-command &, which will run the command in the background, detach it from the console, and redirect its output into nohup.out. It will swallow SIGHUPs that are sent to the process. (When you close the terminal or log out, SIGHUP is sent to all processes that were started by the login shell, and the default action the kernel will take is to kill the process off. This is why appending & to put the process in the background is not enough for it to survive a logout.)
don't use that nohup junk, i hate seeing that on servers; screen is a wasting pile of bits and rot -- use tmux.
if you want to background a process, double fork like every other daemon since the beginning of time:
# ((exec sleep 30)&)
# grep PPid /proc/`pgrep sleep`/status
PPid: 1
# jobs
# disown
bash: disown: current: no such job
enjoy.
The modern and easy to use approach that allows managing multiple processes and has a nice terminal UI is hapless utility.
Install with pip install hapless (or python3 -m pip install hapless) and just run
$ hap run my-command # e.g. hap run python my_long_running_script.py
$ hap status # check all the launched processes
See docs for more info.
A command launched enclosed in parenthesis
(command &)
will survive the death of the originating shell.
Somehow this isn't yielded by a google search.
I'm scripting a server in node.js. I start the server by executing its script with the node program:
node myserver.js
But the server staying up is dependent on my ssh session. How can I make this (and all such processes) persistent? Init.d?
Use the nohup command:
From http://en.wikipedia.org/wiki/Nohup
nohup is a POSIX command to ignore the HUP (hangup) signal, enabling the command to keep running after the user who issues the command has logged out. The HUP (hangup) signal is by convention the way a terminal warns depending processes of logout.
Try this:
nohup node myserver.js &
Have you tried GNU screen? Using it, a process can continue to run when you end your ssh session. nohup is a standard *nix command that will allow you to do the same, albeit in a more limited way.
Use screen. Type screen from the terminal and then launch your process. If you disconnect you can reconnect to the ssh session, by typing 'screen -ls' (to see active screens) and 'screen -r' to reconnect.
The program needs to run in a daemonized mode. Here's a good post for doing this in Ubuntu.
nohup is good to run the job under. If the job is already running, you can try disown -h (at least in bash)
I have a PHP script with infinite loop. I need this script running forever. So, I run
php /path/to/script.php > /dev/null &
And it works in background in my current user's security context. But when I close terminal window (log off), of course, CentOS Linux kills my program.
I see two guesses: run from a different user in background or make a daemon. I need help in each situation.
Thanks a lot!
nohup is your friend.
nohup command &
I think the general solution to that is nohup:
nohup is a POSIX command to ignore the HUP (hangup) signal, enabling the command to keep running after the user who issues the command has logged out. The HUP (hangup) signal is by convention the way a terminal warns depending processes of logout.
nohup is most often used to run commands in the background as daemons. Output that would normally go to the terminal goes to a file called nohup.out if it has not already been redirected. This command is very helpful when there is a need to run numerous batch jobs which are inter-dependent.
nohup is your friend.
You could:
Install screen and run the command from there. screen is a persistent terminal session that you can leave running.
Write an init/upstart (whatever you use) script so it loads on boot
Use the pear lib system_daemon
Use cron if batch work fits the scenario better (just remember to check for running instances before you launch another, iff concurrency is an issue)
Edit: or as everybody else and their brother has just said, nohup
Using command
nohup your_command &
For example
nohup phantomjs highcharts-convert.js -host 127.0.0.1 -port 3003 &
here "phantomjs highcharts-convert.js -host 127.0.0.1 -port 3003" was my command