I am running my code from the shell(SSH) in google cloud as
python3 mycode.py
If I close the shell, the computation stops. How can I start a computation and then close the shell(Computation takes a long Time:)).....come back later and see how it is doing.
My code keeps printing results after a certain number of iteration.
Well, in general what you can do is run the code in a way where you can detach from the interactive environment. Using a tool such as screen or tmux. However, Google Cloud Shell is not made for running background tasks, and if i recall correctly, it will terminate after an hour.
You might need to provision a virtual machine to run it on instead. I can recommend using tmux. With tmux, it will be as simple as running tmux and then in the new shell running your script python3 mycode.py. You can then detach using ctrl+b d or simply disconnect. When you reconnect you run tmux attach -dto get back to your script.
I will provide you as well the following possibility:
A Bash Builtin command that you could also use is disown. First, run your process/script in the background (using &, or stopping it with ^Z and then restarting with bg):
$ long_operation_command &
[1] 1156
Note that at this point the process is still linked to the session and in case it is closed it will be killed.
You can the process attached to the session check running jobs in the background:
$ jobs
[1]+ Running long_operation_command
Therefore you can run disown in order to detach the processes from the session:
$ disown
You can confirm this checking the result of your script or command logging in again or checking with top the process still running.
Check also this because it could be interesting, i.e. the difference between nohup foo, foo & and $ foo & disown
Related
I am trying to launch a GUI application (rhythmbox) on a Ubuntu. In the following I try to explain the chain of executed files.
# Window manager executes first
~/i3wm_cmd_wrapper.sh Window_Name ~/mount_enc.sh
This wrapper uses gnome-terminal to execute stuff. This enables opening a terminal at startup where users can enter information.
# mount_enc.sh launches the following command in the end
bash ~/launch_in_bg.sh rhythmbox
mount_enc.sh does exactly what it is supposed to do when starting from a normal terminal. But I'd like to start it automatically at startup and rhythmbox should be kept open after the script is done.
# launch_in_bg.sh is just doing what it's supposed to
($PRGRM > /dev/null 2>&1) &
I can not get the gnome-terminal to open rhythmbox for me. Also I think my approach is wrong if I want rhythmbox to keep running after the gnome-terminal finishes executing the mount_enc.sh script. Can anybody think of a better solution?
If you open a program from the console (even in background), the process of the program is a child process of the console process and will terminate when the console process terminates.
To keep the program's process running it has to be detached from the console process. Detaching can be done in multiple ways. Some examples:
nohup rhythmbox &
or
rhythmbox & disown
To suppress output, use redirection as in your script.
I'd like to start an extensive IDL process which will be run on a linux based server.
Given a normal a procedure like this:
open PuTTy
move to right directory
compile IDL scripts (1 main script executing 2 others)
run main script.
Now I would like to do that in the background. After some Google research I found out that it is adivisable to use the nohup-command. But I don't know what this command should look like including my scripts....
Thanks for your help.
nohup command can be used to run any job in background which you run in foreground.
The syntax for it is :
nohup your_script &
You can use the fg to bring this job to foreground.
To detach your current process you can use two solutions:
nohup
Run:
nohup ./process </dev/null &>log.txt &
disown to detach the process from your pseudo-TTY, so that when you exit, the process doesn't terminate.
screen
With nohup you lose all control over the process and all you can do is to kill it. It might be a better idea to use screen which lets you gain access to the detached "TTY" again:
Run screen; screen will spawn a new shell.
Run ./process normally
Now hit Ctrl+A and then D - you will get back to your parent shell
You can logout now
After you login again, you can view the task again with screen -r
I am a system administrator and I have been asked to run a linux script to clean the system.
The command is this:
perl script.pl > output.log &
so this command is ending with a & sign, is there any special significance of it?
I have basic knowledge of shell but I have never seen this before.
The & makes the command run in the background.
From man bash:
If a command is terminated by the control operator &, the shell
executes the command in the background in a subshell. The shell does
not wait for the command to finish, and
the return status is 0.
When not told otherwise commands take over the foreground. You only have one "foreground" process running in a single shell session. The & symbol instructs commands to run in a background process and immediately returns to the command line for additional commands.
sh my_script.sh &
A background process will not stay alive after the shell session is closed. SIGHUP terminates all running processes. By default anyway. If your command is long-running or runs indefinitely (ie: microservice) you need to pr-pend it with nohup so it remains running after you disconnect from the session:
nohup sh my_script.sh &
EDIT: There does appear to be a gray area regarding the closing of background processes when & is used. Just be aware that the shell may close your process depending on your OS and local configurations (particularly on CENTOS/RHEL):
https://serverfault.com/a/117157.
In addition, you can use the "&" sign to run many processes through one (1) ssh connections in order to to keep minimum number of terminals. For example, I have one process that listens for messages in order to extract files, the second process listens for messages in order to upload files: Using the "&" I can run both services in one terminal, through single ssh connection to my server.
These processes running through the "&" will also "stay alive" after ssh session is closed. Pretty neat and useful if the ssh connection to the server is interrupted and no terminal multiplexer (screen, tmux, byobu) was used.
I don’t know for sure but I’m reading a book right now and what I am getting is that a program need to handle its signal ( as when I press CTRL-C). Now a program can use SIG_IGN to ignore all signals or SIG_DFL to restore the default action.
Now if you do $ command & then this process running as background process simply ignores all signals that will occur. For foreground processes these signals are not ignored.
If you have a command which executes and doesn't return status 0(control of prompt) quickly.
For example:
command gedit launches the default editor gedit UI.
commandeclipse launches eclipse IDE.
Such commands keep throwing the logs of activities in the terminal and don't return the command prompt.
Question is, how to run such commands in background so that, we will get back command terminal and we can use terminal for other tasks.
Answer is: by appending & after such command.
user#mymachine:~$ <command> &
Examples:
user#mymachine:~$ edit &
user#mymachine:~$ eclipse &
I have spawned a process with another application. I can see that it is running with:
ps -ef
How can I switch into that process as if I started it manually myself by entering the command into the console?
If it's started from current shell, use standard job-control e.g.
$ jobs
$ gedit &
[1] 3341
$ jobs
[1]+ Running gedit &
$ fg %1
gedit
Basically, you can only manage processes with job control that are children of your current shell, that is, jobs started by the shell you are working with. If you did start and background the process with your current shell, fg and the other job control options will work. If not, you cannot manage the job with the shell.
The mostly used "workaround" (actually much more powerful than the shell) is GNU screen.
I'm trying to use a shell script to start a command. I don't care if/when/how/why it finishes. I want the process to start and run, but I want to be able to get back to my shell immediately...
You can just run the script in the background:
$ myscript &
Note that this is different from putting the & inside your script, which probably won't do what you want.
Everyone just forgot disown. So here is a summary:
& puts the job in the background.
Makes it block on attempting to read input, and
Makes the shell not wait for its completion.
disown removes the process from the shell's job control, but it still leaves it connected to the terminal.
One of the results is that the shell won't send it a SIGHUP(If the shell receives a SIGHUP, it also sends a SIGHUP to the process, which normally causes the process to terminate).
And obviously, it can only be applied to background jobs(because you cannot enter it when a foreground job is running).
nohup disconnects the process from the terminal, redirects its output to nohup.out and shields it from SIGHUP.
The process won't receive any sent SIGHUP.
Its completely independent from job control and could in principle be used also for foreground jobs(although that's not very useful).
Usually used with &(as a background job).
nohup cmd
doesn't hangup when you close the terminal. output by default goes to nohup.out
You can combine this with backgrounding,
nohup cmd &
and get rid of the output,
nohup cmd > /dev/null 2>&1 &
you can also disown a command. type cmd, Ctrl-Z, bg, disown
Alternatively, after you got the program running, you can hit Ctrl-Z which stops your program and then type
bg
which puts your last stopped program in the background. (Useful if your started something without '&' and still want it in the backgroung without restarting it)
screen -m -d $command$ starts the command in a detached session. You can use screen -r to attach to the started session. It is a wonderful tool, extremely useful also for remote sessions. Read more at man screen.