Following this solution from another post I've made an alias in my .zshrc (I use ohmyzsh) to stop my celery workers:
alias stopcelery="ps auxww | grep 'celery' | awk '{print $2}' | xargs kill -9"
However when I have running celery workers, using this command fails:
➜ stopcelery
kill: invalid argument
Usage:
kill [options] <pid> [...]
Options:
<pid> [...] send signal to every <pid> listed
-<signal>, -s, --signal <signal>
specify the <signal> to be sent
-l, --list=[<signal>] list all signal names, or convert one to a name
-L, --table list all signal names in a nice table
-h, --help display this help and exit
-V, --version output version information and exit
For more details see kill(1).
The workers are still running:
➜ ps auxww | grep 'celery'
myuser 49126 67.0 0.5 189132 92280 ? S 15:59 0:02 /home/myuser/.local/share/virtualenvs/myproject-JLNbaOhA/bin/python -m celery worker -A wsgi.celery --loglevel=INFO --logfile=/tmp/celery.log --pidfile=celeryd.pid
myuser 49304 0.0 0.5 188160 85060 ? S 15:59 0:00 /home/myuser/.local/share/virtualenvs/myproject-JLNbaOhA/bin/python -m celery worker -A wsgi.celery --loglevel=INFO --logfile=/tmp/celery.log --pidfile=celeryd.pid
myuser 49305 0.0 0.5 188164 84844 ? S 15:59 0:00 /home/myuser/.local/share/virtualenvs/myproject-JLNbaOhA/bin/python -m celery worker -A wsgi.celery --loglevel=INFO --logfile=/tmp/celery.log --pidfile=celeryd.pid
myuser 49306 0.0 0.5 188168 84848 ? S 15:59 0:00 /home/myuser/.local/share/virtualenvs/myproject-JLNbaOhA/bin/python -m celery worker -A wsgi.celery --loglevel=INFO --logfile=/tmp/celery.log --pidfile=celeryd.pid
myuser 49307 0.0 0.5 188172 84844 ? S 15:59 0:00 /home/myuser/.local/share/virtualenvs/myproject-JLNbaOhA/bin/python -m celery worker -A wsgi.celery --loglevel=INFO --logfile=/tmp/celery.log --pidfile=celeryd.pid
myuser 49312 0.0 0.0 20556 2924 pts/8 S+ 15:59 0:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox celery
But using the actual command it works:
➜ ps auxww | grep 'celery' | awk '{print $2}' | xargs kill -9
kill: (48078): No such process
Indeed, checking again:
➜ ps auxww | grep 'celery'
myuser 49782 0.0 0.0 20556 664 pts/8 S+ 16:03 0:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox celery
So what is the difference, why using the alias does not having the same effect?
As #Philippe has pointed out, I forgot to escape the $ sign as I was using double quotes.
Related
Now I am using celery 5.1.2 and Python 3.9 to run some task concurrency, this is my start script:
export LC_ALL=en_US.utf-8 \
&& export LANG=en_US.utf-8 \
&& nohup celery -A dolphin.tasks.tasks worker \
--loglevel=INFO -n worker2 -Q non_editor_pick_and_diff_pull --concurrency 2 > /dev/null &
export LC_ALL=en_US.utf-8 \
&& export LANG=en_US.utf-8 \
&& nohup celery -A dolphin.tasks.tasks worker \
--loglevel=INFO -n worker3 -Q non_editor_pick_and_non_diff_pull --concurrency 3 > /dev/null &
to my surprise, the celery worker take 100MB+ for each worker, why the celery take so much RAM memory? Is it possible to make the memory lower,10MB or less? I just run a function in the celery task, really need 100MB? This is the top command output:
24073 root 25 5 677220 139988 4456 S 0.0 7.4 22:14.10 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker2 -Q non_editor_pick_and_diff_pull --concurrency 2
24076 root 25 5 677024 139744 4412 S 0.0 7.4 22:11.04 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker2 -Q non_editor_pick_and_diff_pull --concurrency 2
24078 root 25 5 676848 134760 3848 S 0.0 7.2 93:01.93 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker3 -Q non_editor_pick_and_non_diff_pull --concurrency 3
24072 root 25 5 675332 133964 4560 S 7.3 7.1 173:10.84 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker3 -Q non_editor_pick_and_non_diff_pull --concurrency 3
24075 root 25 5 670876 128804 3860 S 0.0 6.8 129:24.36 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker3 -Q non_editor_pick_and_non_diff_pull --concurrency 3
24071 root 25 5 655128 123692 4132 S 0.0 6.6 13:17.96 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker1 -Q editor_pick_and_diff_pull --concurrency 2
24074 root 25 5 646624 120068 4272 S 0.0 6.4 13:04.01 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker1 -Q editor_pick_and_diff_pull --concurrency 2
24000 root 25 5 624856 98748 2728 S 0.0 5.2 9:41.82 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker1 -Q editor_pick_and_diff_pull --concurrency 2
24001 root 25 5 624868 98724 2716 S 0.0 5.2 12:43.68 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker2 -Q non_editor_pick_and_diff_pull --concurrency 2
24002 root 25 5 624840 98708 2676 S 1.3 5.2 44:47.66 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker3 -Q non_editor_pick_and_non_diff_pull --concurrency 3
24005 root 25 5 624696 98060 2412 S 0.0 5.2 7:59.62 /usr/bin/python3 /usr/local/bin/celery -A dolphin.tasks.tasks worker --loglevel=INFO -n worker4 -Q cert_expire_check --concurrency 1
This question already has answers here:
How to get process ID of background process?
(9 answers)
Closed 1 year ago.
Below is the output of executing two commands ads2 svcd& and ps -aux|grep ads2
nvidia#nvidia-desktop:~$ ads2 svcd&
[1] 4593
nvidia#nvidia-desktop:~$ ps -aux|grep ads2
nvidia 4593 0.5 0.0 39796 23864 pts/0 Sl 08:20 0:00 /opt/ads2/arm-
linux64/bin/ads2svcd
nvidia 4603 0.0 0.0 6092 672 pts/0 S+ 08:20 0:00 grep --color=auto ads2
nvidia#nvidia-desktop:~$
nvidia#nvidia-desktop:~$
the command ads2 svcd& runs a process related to ads2 software. with ps -aux|grep ads2 i displayed the whole processes that contains the name "ads2".
Now What i'm trying to do is to get the process number of the ads2 which in this example is 4593. So i wrote the follwing bash script:
#!/usr/bin/env bash
process="$(ps -aux|grep ads | grep 'nvidia' | cut -d' ' -f 3)"
echo "The current ads2 process is " $process
The bash script outputs the following:
nvidia#nvidia-desktop:~$ ./test.sh
The current ads2 process is
As you see the process number is not filtered. So what i'm i doing wrong?
thanks in advance
List all the processes in the current shell with $$ built-in variable
ps --forest -gp $$
PID TTY STAT TIME COMMAND
3809 pts/1 Ss 0:01 bash
4896 pts/1 T 0:00 \_ vim file.json
22965 pts/1 S+ 0:00 \_ ssh dw
3607 pts/0 Ss 0:01 bash
2500 pts/0 R+ 0:00 \_ ps --forest -gp 3607
3327 tty2 Ssl+ 0:00 /usr/lib/gdm3/gdm-x-session --run-script i3
3329 tty2 Sl+ 8:12 \_ /usr/lib/xorg/Xorg vt2 -displayfd 3 -auth /run/user/1000/gdm/Xauthority -background none -noreset -keeptty -verbose 3
3346 tty2 S+ 0:03 \_ i3
Just see the pid of them:
ps -opid --forest -gp $$
PID
3809
4896
22965
3607
2688
3327
3329
3346
If you need to use grep for any reason, use -opid,cmd with current shell:
ps -opid,cmd -gp $$ | grep vim
3851 grep --color=auto vim
4896 vim file.json
For all, just use -e
ps -e -opid,cmd | grep vim
4141 grep --color=auto vim
4896 vim file.json
The complete one, we have to ignore the grep itself:
ps -e -opid,cmd | grep vim | grep -v grep | cut -d' ' -f 2
4896
Without double grep using comm option for ps
ps -opid,comm -gp $$ | grep vim
4896 vim
of course the simplest one is pgrep
pgrep vim
4896
NOTE for variable assignment there should NOT be any space:
# wrong
ads2ProcessId = $(pgrep ads2)
# right
ads2ProcessId=$(pgrep ads2)
when I give
ps -aux|grep -w 'bash'|grep -v 'grep'|awk '{print $2}'
I get output:
32356
Also when I give
echo $(ps -aux|grep -w 'bash'|grep -v 'grep'|awk '{print $2}')
i get output:
32356 32551
Why there is difference in the outputs?
What is happening is that the $(...) syntax starts a subshell. So the ps command inside those parentheses will find both your current shell (which you can get using $$, as #tink said his their answer), and the subshell that is invoked by the parentheses.
Using ps fux and rediction into a file, you can see exactly what is happening:
ps fux | grep -w bash > a
cat a
me 11603 0.1 0.0 114408 3728 pts/1 Ss 13:37 0:00 \_ -bash
me 11955 0.0 0.0 103348 872 pts/1 S+ 13:41 0:00 \_ grep -w bash
echo $(ps fux | grep -w bash > b)
cat b
me 11603 0.1 0.0 114408 3728 pts/1 Ss+ 13:37 0:00 \_ -bash
me 11981 0.0 0.0 114408 2304 pts/1 S+ 13:41 0:00 \_ -bash
me 11983 0.0 0.0 103348 872 pts/1 S+ 13:41 0:00 \_ grep -w bash
Although #tink gave you a good solution, I hope this answer helps understand what is happening.
It would appear that you're after the pid of the shell you're currently running. That's best achieved with
echo $$ # which is a bash special variable
Problem: We use Zabbix as monitoring system. In addition to using its built in items, we also use something called external scripts feature (this), where custom scripts can be written and called via Zabbix. The problem facing here is its getting timed out. Script is simple expect file which goes inside a device and pulls some data. This works when called via root. But when called via Zabbix user, its complaining
/usr/bin/expect: /usr/bin/expect: cannot execute binary file
Script looks like this,
#!/usr/bin/expect
set host "IP_ADDRESS"
set uname "username"
set pwd "password"
set prompt "#|>|:|\\\$";
set val ""
set domain [lindex $argv 0]
log_user 0
set timeout -1
spawn /usr/bin/ssh "$uname#$host"
expect "$uname#$host's password:"
send "$pwd\n"
sleep 1
#expect -re "$prompt"
expect ">"
sleep 1
send "show wireless rf-domain statistics detail on $domain | grep Signals\r"
sleep 1
expect ">"
set val $expect_out(buffer)
send "exit\n"
puts $val
This is named as rf_signal.exp. Its called via a wrapper shell script named rf_signal.
#!/bin/bash
val=$(/usr/bin/expect '/usr/local/etc/scripts/rf_signal.exp' $1 | grep 'RF Signals' | cut -d':' -f2 | cut -d',' -f1 | cut -d' ' -f3 | sed -e 's/\s//g')
echo "$val"
And if called as root, this works fine for example
[root#zbx-proxy2 externalscripts]# pwd
/usr/local/share/zabbix/externalscripts
[root#zbx-proxy2 externalscripts]# whoami
root
[root#zbx-proxy2 externalscripts]# /usr/local/share/zabbix/externalscripts/rf_signal DOMAIN_NAME
241
[root#zbx-proxy2 externalscripts]#
Where as if I call as zabbix user I am getting, cannot execute binary file error. With expect as a path
[root#zbx-proxy2 externalscripts]# runuser -l zabbix /usr/bin/expect /usr/local/share/zabbix/externalscripts/rf_signal
/usr/bin/expect: /usr/bin/expect: cannot execute binary file
[root#zbx-proxy2 externalscripts]#
Without expect as a path, it waits -
[root#zbx-proxy2 externalscripts]# runuser -l zabbix /usr/local/share/zabbix/externalscripts/rf_signal
PSTree command output shows it calls expect and contents inside the file
[root#zbx-proxy2 ~]# pstree -p 26295
runuser(26295)---bash(26296)---bash(26309)-+-cut(26312)
|-cut(26313)
|-cut(26314)
|-expect(26310)---ssh(26316)
|-grep(26311)
`-sed(26315)
PS details.
[root#zbx-proxy2 ~]# ps aux | grep zabbix | grep -v "proxy\|agent\|fping"
root 26295 0.0 0.0 130700 1388 pts/3 S+ 15:46 0:00 runuser -l zabbix /usr/local/share/zabbix/externalscripts/rf_signal
zabbix 26296 0.0 0.0 108184 1628 pts/3 S+ 15:46 0:00 -bash /usr/local/share/zabbix/externalscripts/rf_signal
zabbix 26309 0.0 0.0 108184 576 pts/3 S+ 15:46 0:00 -bash /usr/local/share/zabbix/externalscripts/rf_signal
zabbix 26310 0.0 0.0 115336 2260 pts/3 S+ 15:46 0:00 /usr/bin/expect /usr/local/etc/scripts/rf_signal.exp
zabbix 26311 0.0 0.0 103260 868 pts/3 S+ 15:46 0:00 grep RF Signals
zabbix 26312 0.0 0.0 100972 676 pts/3 S+ 15:46 0:00 cut -d: -f2
zabbix 26313 0.0 0.0 100972 672 pts/3 S+ 15:46 0:00 cut -d, -f1
zabbix 26314 0.0 0.0 100972 676 pts/3 S+ 15:46 0:00 cut -d -f3
zabbix 26315 0.0 0.0 105268 872 pts/3 S+ 15:46 0:00 sed -e s/\s//g
zabbix 26316 0.0 0.0 59856 3220 pts/14 Ss+ 15:46 0:00 /usr/bin/ssh username#IP_ADDRESS
root 26688 0.0 0.0 105324 912 pts/7 S+ 15:47 0:00 grep zabbix
[root#zbx-proxy2 ~]#
All the scripts have read and execute permissions to all the users. And expect/grep/cut - whatever used inside the scripts are having read/execute permissions. What could be the issue?
I would suggest you use:
su - zabbix -c "/usr/local/share/zabbix/externalscripts/rf_signal"
to run the script instead of runuser.
Usually, the zabbix user has /sbin/nologin set up as a login shell, which means you won't be able to login via ssh onto the respective server. You may check /etc/passwd on the remote server to verify this.
As an additional note, you may use expect -d to enable debug in your expect script and see where it fails. Set expect's timeout to a different value like 180 (3 minutes) as opposed to -1 otherwise it will never exit.
I am trying to find the PID of a process (motion_sensor.py), but pgrep returns nothing. Why does it not return the process id?
pgrep -u www-data motion_sensor.py
ps -ef | grep "motion_sensor" returns
root 7149 1 93 Apr25 ? 15:59:08 python motion_sensor.py
www-data 31872 23531 0 14:09 ? 00:00:00 sh -c sudo python /home/pi/Desktop/PiControl/motion_sensor.py
root 31873 31872 0 14:09 ? 00:00:00 sudo python /home/pi/Desktop/PiControl/motion_sensor.py
root 31874 31873 47 14:09 ? 00:14:30 python /home/pi/Desktop/PiControl/motion_sensor.py
pi 32645 32202 0 14:39 pts/0 00:00:00 grep --color=auto motion_sensor.py
Normally pgrep applies the search pattern to process names. The process name in this case is python and not motion_sensor.py. If you want to grep for the full path rather than just the process name you need to pass -f:
pgrep -u www-data -f motion_sensor.py
Check man pgrep
the requirement is to find out PID of a process,
So you can try :
ps aux | grep www-data motion_sensor.py