I have a bash script called run.sh which launches two python scripts meter_1.py and meter_2.py
#!/bin/sh
./meter_1.py &
./meter_2.py &
When the scripts are running and I search for the PID of the scripts using the command
ps -aux | grep python
The output is
openhab+ 9328 84.0 1.6 25320 16580 pts/0 R 22:23 0:04 python ./meter_1.py
openhab+ 9329 84.6 1.6 25320 16596 pts/0 R 22:23 0:04 python ./meter_2.py
Using the pgrep command I can get the the PID
>pgrep python
9328
9329
However I could have multiple python scripts running and I want to get the process ID name by the script it is running not based on if it is python or not.
For example:
>pgrep python" "./meter_1.py
9328
Is there a functionality for this in pgrep? The following seems to work however it would be nice to get just the process ID back.
>ps -aux | grep python" "./meter_1.py
openhab+ 9328 84.0 1.6 25320 16580 pts/0 R 22:23 0:04 python ./meter_1.py
In ba(sh) you can get the PID of the last started process with $!
so in your run.sh script you can simply use:
#!/bin/sh
./meter_1.py &
echo PID of process1: $!
./meter_2.py &
echo PID of process2: $!
If you have control of the process, like you're the one starting the process in a script, to expand on what have posted already.
#!/bin/sh
./meter_1.py & meter_1_py_pid=$!
./meter_2.py & meter_2_py_pid=$!
The pids are in the variables $meter__1_py_pid and $meter__2_py_pid
You can do whatever you want with the pids, check if it is running, kill it and so on.
Related
I have a bash script that executes in loop a python script:
while true; do python3 script.py && break; done
launched with nohup:
nohup ./run_script.sh &
Why if I run the command:
tail -f nohup.out
I don't see the output of the python script?
I correctly see the script in my running process:
pi 2757 1.5 3.4 37268 33064 ? S 15:06 0:18 python3 script.py
pi 2819 0.0 0.0 1908 388 ? S 15:07 0:00 /bin/sh ./run_script.sh
pi 2820 1.6 3.5 37268 33096 ? S 15:07 0:18 python3 script.py
If I directly launch the python script with nohup, I see the output but I need to relaunch the script everytime it fails, so I need to lauch it with the bash script
Maybe I'm missing some concept concerning nohup usage.
I resolved modifing the script in this way:
while true; do python3 -u script.py && break; done
The output file nohup.out during execution remains empty until the execution is finished. This happens because of output buffering. Adding the -u flag I can avoid output buffering.
For other details see this blog page https://janakiev.com/blog/python-background/.
I need some help because I don't get something. From what I read from Internet, a subshell is created when we execute a shell script or if we run command in brackets: ( )
I tried to test this with a script which contains only the following command:
ps -f
When I run it I see the following result:
ID PID PPID C STIME TTY TIME CMD
me 2213 2160 0 08:53 pts/14 00:00:00 bash
me 3832 2213 0 18:41 pts/14 00:00:00 bash
me 3833 3832 0 18:41 pts/14 00:00:00 ps -f
Which is good, because I see that my bash process has spawned another bash process for my script.
But when I do:
( ps -f )
it produces:
UID PID PPID C STIME TTY TIME CMD
me 2213 2160 0 08:53 pts/14 00:00:00 bash
me 3840 2213 0 18:46 pts/14 00:00:00 ps -f
So if brackets spawn a subshell why it is not shown in the processes? And why does ps -f is counted as another process? Does every command run as a separate process?
It seems you've caught bash in a little bit of an optimization. if a subshell contains only a single command, why really make it a subshell?
$ ( ps -f )
UID PID PPID C STIME TTY TIME CMD
jovalko 29393 24133 0 12:05 pts/10 00:00:00 bash
jovalko 29555 29393 0 12:07 pts/10 00:00:00 ps -f
However, add a second command, say : (the bash null command, which does nothing) and this is the result:
$ ( ps -f ; : )
UID PID PPID C STIME TTY TIME CMD
jovalko 29393 24133 0 12:05 pts/10 00:00:00 bash
jovalko 29565 29393 0 12:08 pts/10 00:00:00 bash
jovalko 29566 29565 0 12:08 pts/10 00:00:00 ps -f
One of the main reasons to use a subshell is that you can perform operations like I/O redirection on a group of commands instead a single command, but if your subshell contains only a single command there's not much reason to really fork a new bash process first.
As to ps counting as a process, it varies. Many commands you use like ls, grep, awk are all external programs. But, there are builtins like cd, kill, too.
You can determine which a command is in bash using the type command:
$ type ps
ps is hashed (/bin/ps)
$ type cd
cd is a shell builtin
The main part of the question is:
Does every command run as a separate process?
YES!. Every command what isn't built-in into bash (like declare and such), runs as separate process. How it is works?
When you type ps and press enter, the bash analyze what you typed, do usual things as globbing, variable expansionas and such, and finally when it is an external command
the bash forks itself.
The forking mean, than immediatelly after the fork, you will have two identical bash processes (each one with different process ID (PID)) - called as "parent" and "child", and the only difference between those two running bash programs is, than the "parent" gets (return value from the fork) the PID of the child but the child don't know the PID of the parent. (fork for the child returns 0).
after the fork, (bash is written this way) - the child replaces itself with the new program image (such: ps) - using the exec call.
after this, the child bash of course doesn't exist anymore, and running only the newly executed command - e.g. the ps.
Of course, when the bash going to fork itself, do many other things, like I/O redirections, opens-closes filehandles, changes signal handling for the child and many-many other things.
There are two different python programs running in this VM
one is a background job who monitors a folder and then 'does stuff' (with several workers)
10835 ? Sl 0:03 python main.py
10844 ? Sl 34:02 python main.py
10845 ? S 33:43 python main.py
the second one is started via script
20056 pts/1 S+ 0:00 /bin/bash ./exp.sh
20069 pts/1 S+ 0:00 /bin/bash ./exp.sh
20087 pts/1 S+ 0:10 python /home/path/second.py
i have tried numerous things to find a way to kill only the main program (i want to build a cron watchdog), but non succeded
first one i want to find only the hanging 'python main.py' process (accompanied by [defunct]), but i cant even find just this process alone.
the upper ones are from ps -ax (so they both are running currently)
pgrep 'python' returns all PIDs, also from second.py which i dont want: (not usefull, so is pkill therefore)
pgrep 'python'
10835
10844
10845
20087
pgrep 'python main.py' always returns empty, so does pgrep 'main.py;
the only thing which works
ps ax | grep 'python main.py'
but this one also returns its own PID, grepping 'ps' isn't a prefered solution afair. when main.py hangs, it shows "python main.py [defunct]". a
ps ax | grep 'python main.py [defunct]'
is useless as test as it always returns true. pgrep for anything more than 'python' also returns always false. i am a bit clueless.
This works for me. Found it on the pgrep bro pages.
Find the pids of processes with 'test.py' as an argument, like 'python test.py'
pgrep -f test.py
And I use it to check if a python process is running:
searcher="backend/webapi.py"
if pgrep -f "$searcher" > /dev/null
then
echo "$(date +%y-%m-%d-%H-%M-%S) $searcher is alive. Doing nothing."
else
echo "No $searcher. Kickstarting..."
pushd $HOME/there/;
./run_backend
popd;
echo "Pgrepping $searcher:"
pgrep "$searcher" # out to loggers
fi
In your daemon python script you should create PID file:
def writePidFile():
pid = str(os.getpid())
f = open('/tmp/my_pid', 'w')
f.write(pid)
f.close()
Now killing this process is simple:
kill `cat /tmp/my_pid`
Or you can just use grep and filter its own process:
ps ax | grep 'python main.py [defunct]' | grep -v grep
I think I've found a pretty safe way to do find the proper python process PID for a specific file only. The idea is to find all the python processes, then all "test.py" processes and intersect the results to find the correct PID.
I use the subprocess module in a python script (you can interpolate for .sh script):
import subprocess as subp
#Get a list of all processes with a certain name
def get_pid(name):
return list(map(int,subp.check_output(["pidof", "-c", name]).split()))
#Get a list of all pids for python3 processes
python_pids = get_pid('python3')
#Get a list of all pids for processes with "main.py" argument
main_py_pids = list(map(int,subp.check_output(["pgrep", "-f", "main.py"]).split()))
python_main_py_pid = set(python_pids).intersection(main_py_pids)
print(python_pids)
print(main_py_pids)
print(python_main_py_pid)
Result with 3 running python processes (including the one from this script) and one "sudo nano main.py" process:
Python pids: [3032, 2329, 1698]
main.py pids: [1698, 3031]
Interesection: {1698}
There are various tricks to daemonize a linux process, i.e. to make a command running after the terminal is closed.
nohup is used for this purpose, and fork()/setsid() combination can be used in a C program to make itself a daemon process.
The above was my knowledge about linux daemon, but today I noticed that exiting the terminal doesn't really terminate processes started with & at the end of the command.
$ while :; do echo "hi" >> temp.log ; done &
[1] 11108
$ ps -ef | grep 11108
username 11108 11076 83 15:25 pts/0 00:00:05 /bin/sh
username 11116 11076 0 15:25 pts/0 00:00:00 grep 11108
$ exit
(after reconnecting)
$ ps -ef | grep 11108
username 11108 1 91 15:25 pts/0 00:00:17 /bin/sh
username 11130 11540 0 15:25 pts/0 00:00:00 grep 11108
So apparently, the process's PPID changed to 1, meaning that it got daemonized somehow.
This contradicts my knowledge, that & is not enough and one must use nohup or some other tricks to a process 'daemon'.
Does anyone know who is doing this daemonizing?
I'm using a CentOS 6.3 host and putty/cygwin/sshclient produced the same result.
You can daemonize a process if that doesn't respond to SIGHUP signal.
When bash shell is terminated while it is running background tasks, bash shell sends SIGHUP
(hangup signal) to all tasks. However bash won't wait until child processes are completely
terminated. If child process doesn't respond to SIGHUP signal, that process becomes an orphan
process. (its parent pid is changed to 1 - init process - to prevent becoming a useless zombie process)
Subshell executions basically do not responds to SIGHUP signals, thus your command will still be running after logging out from the first shell.
I have a php script which is responsible for sending emails based on a queue contained in a database.
The script works when it is executed from my shell as such:
/usr/bin/php -f /folder/email.php
However, when I execute it to run in the background:
/usr/bin/php -f /folder/email.php > /dev/null &
It never completes, and the process just sits in the process queue:
clickonce: ps T
PID TT STAT TIME COMMAND
1246 s000 Ss 0:00.03 login -pf
1247 s000 S 0:00.03 -bash
1587 s000 T 0:00.05 /usr/bin/php -f /folder/email.php
1589 s000 R+ 0:00.00 ps T
So my question is how can I run this as a background process and have it actually execute? Do I need to configure my OS? Do I need to change the way I execute the command?
"T" in the "STAT" column indicates a stopped process. I would guess that your script is attempting to read input from stdin and is getting stopped because it is not the foreground process and thus is not allowed to read.
You should check if the script does indeed read something while executing.