There is a benchmarking process, which should be run on a system. It takes maybe a day, so I would like to run it nohup. I use this command:
nohup bash ./run_terasort.sh > terasort.out 2>&1 &
After that I can see with PID in jobs -l output, but after closing PuTTy it stops(as I can see, when I login again).
This is a KVM virtualized machine.
You are using nohup right from what I know. But you have an issue detecting the process.
jobs -l only give the processes of current session. Rather try the below to display the process started in your initial session:
ps -eafww|grep run_terasort.sh|grep -v grep
Related
I'm using nohup to submit jobs in background in the machines I got through BSUB and ssh.
My primary machine is on RHEL, from there I am picking up other AIX machine by BSUB(Submits a job to LSF) and also doing a SSH login to another server.
After getting these two machines, executing a script(inner.sh) there through nohup.
I'm capturing the respective PIDs through echo $$ in the script which I am executing(inner.sh).
After submitting the nohup execution in background, I am exiting both of the machines and landing back to primary RHEL machine.
Now, from this RHEL machine, I'm trying to get the status of the nohup execution by ps -p PID with the help of the previously captured two PIDs, but not getting any process listed.
Top level wrapper script wrapper.sh:
#!/bin/bash
#login to a remote server
ssh -k xyz#abc < env_setup.sh
#picking up a AIX machine form LSF
bsub -q night -Is ksh -i env_setup.sh
ps -p process-<AIX_machine>.pid
#got no output
ps -p process-<server_machine>.pid
#got no output
Script passed to Machines picked up by BSUB/SSH to execute nohup env_setup.sh:
#!/bin/bash
nohup sh /path/to/dir/inner.sh > /path/to/dir/log-<hostname>.out &
exit
The actual script which I am trying to execute in machines picked up by BSUB/SSH inner.sh:
#!/bin/bash
echo $$ > /path/to/dir/process-<hostname>.pid
#hope this would give correct us the PID of the remote machine
#execute some other commands
Now, I am getting two process-<hostname>.pid files updated with two PIDs respectively each for both of the machines.
But ps -p at wrapper script is giving us no output.
I am picking up the process IDs from remote machines and doing ps -p at my local RHEL machine.
Is it the reason I am not getting any status update of those two processes?
Can I do anything else to get the status?
ps give the status of local processes. bsub can be used to get the status of processes on each remote machine.
I was using
nohup ./program_name &
to run my program, program_name prints out some values and status of the running process including how much percentage the program has finished, but since I'm running it using nohup so I can't see how close my program to finish is, is there anyway I can still get that information?
We have to Just open nohup.out to see output. Probably you want
tail -f nohup.out
for streaming output
Perhaps adjust your nohup command line to capture all output to a file:
nohup ./program_name > /tmp/programName.log 2>&1 &
Then, you can monitor programName.log using tail:
tail -f /tmp/programName.log
Put the below command in current terminal where the program is running
jobs command used to lists the jobs that you are running in the background and in the foreground
jobs -l
[6]+ 6069 Running nohup perl test1.pl &
[6]+ 6069 Done nohup perl test1.pl
Using jobs I know the process is running.
bash-4.2$ jobs
[1]+ Running test.sh &
I wanted it to be set NOHUP so that it won't be killed when I exit. I used
disown
and
bash-4.2$ jobs
shows nothing. I'm not sure if the process is set NOHUP or not. I'm curious about this because after I read the manual it says
disown -h
should be used to set NOHUP.
Edit
I don't think the link Find the Process run by nohup command helps. The question is different than that one.
I'm gonna restate my problem. I run a program without nohup, and later I wanted it to be set NOHUP so that it won't be killed when I exit the system. So I used disown, but later I found the manual says I should have used disown -h to set NOHUP. I want to check if my process is set NOHUP or not successfully. If not, what can I do to set it to be NOHUP?
UPDATE
I know two ways my be helpful:
1) Whenever a process is running over nohup It writes output on ~/nohup.out . So you can check this file by running command find -cmin 2. It shows you if nohup.out is changing each 2 seconds or not.
If it is changing you would understand that sth is running by nohup command, after that you can check it with lsof and continue your checking...
2) If you logout from specific user andgo to tty then do ps aux | grep <user> or ps aux | grep ? then you can understand that is running with nohup command... because there is no pts then it shows you ? instead...
useful command:
ps aux | grep <program> | awk -F" " '{print $7}'
Hope to be helpful
I made a python script running on background:
nohup python app.py &
then close the terminal, a few days later, I want to see this job, so I run
jobs
there list no jobs, but I'm sure the script app.py is still running.
jobs will only give you a list of process running under the session group of the bash shell. nohup processes run in their own session group. There are a number of simple commands you can run to check if your nohup'd process is still running, the simplest being lsof | grep nohup (this command may take a few seconds to run)
I am accessing a server running CentOS (linux distribution) with an SSH connection.
Since I can't always stay logged in, I use "nohup [command] &" to run my programs.
I couldn't find how to get a list of all the programs I started using nohup.
"jobs" only works out before I log out. After that, if I log back again, the jobs command shows me nothing, but I can see in my log files that my programs are still running.
Is there a way to get a list of all the programs that I started using "nohup" ?
When I started with $ nohup storm dev-zookeper ,
METHOD1 : using jobs,
prayagupd#prayagupd:/home/vmfest# jobs -l
[1]+ 11129 Running nohup ~/bin/storm/bin/storm dev-zookeeper &
NOTE: jobs shows nohup processes only on the same terminal session where nohup was started. If you close the terminal session or try on new session it won't show the nohup processes. Prefer METHOD2
METHOD2 : using ps command.
$ ps xw
PID TTY STAT TIME COMMAND
1031 tty1 Ss+ 0:00 /sbin/getty -8 38400 tty1
10582 ? S 0:01 [kworker/0:0]
10826 ? Sl 0:18 java -server -Dstorm.options= -Dstorm.home=/root/bin/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dsto
10853 ? Ss 0:00 sshd: vmfest [priv]
TTY column with ? => nohup running programs.
Description
TTY column = the terminal associated with the process
STAT column = state of a process
S = interruptible sleep (waiting for an event to complete)
l = is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)
Reference
$ man ps # then search /PROCESS STATE CODES
Instead of nohup, you should use screen. It achieves the same result - your commands are running "detached". However, you can resume screen sessions and get back into their "hidden" terminal and see recent progress inside that terminal.
screen has a lot of options. Most often I use these:
To start first screen session or to take over of most recent detached one:
screen -Rd
To detach from current session: Ctrl+ACtrl+D
You can also start multiple screens - read the docs.
If you have standart output redirect to "nohup.out" just see who use this file
lsof | grep nohup.out
You cannot exactly get a list of commands started with nohup but you can see them along with your other processes by using the command ps x. Commands started with nohup will have a question mark in the TTY column.
You can also just use the top command and your user ID will indicate the jobs running and the their times.
$ top
(this will show all running jobs)
$ top -U [user ID]
(This will show jobs that are specific for the user ID)
sudo lsof | grep nohup.out | awk '{print $2}' | sort -u | while read i; do ps -o args= $i; done
returns all processes that use the nohup.out file