I and new to airflow and accidentally started airflow scheduler in daemon mode. Now, I want to kill the scheduler and possibly restart it. I tried doing
sudo kill -9 <list of pids>
pkill <name>
nothing is happening. When I run
ps aux | grep 'airflow scheduler'
I see these entries:
user1 2907 6.0 1.0 329788 62996 ? Sl 17:37 1:26 /users/user1/anaconda2/bin/python /users/user1/anaconda2/bin/airflow scheduler -D
user1 2909 0.0 0.9 327576 58948 ? Sl 17:37 0:00 /users/user1/anaconda2/bin/python /users/user1/anaconda2/bin/airflow scheduler -D
user1 2910 0.0 0.9 327576 58944 ? Sl 17:37 0:00 /users/user1/anaconda2/bin/python /users/user1/anaconda2/bin/airflow scheduler -D
user1 2911 0.0 0.9 327576 58944 ? Sl 17:37 0:00 /users/user1/anaconda2/bin/python /users/user1/anaconda2/bin/airflow scheduler -D
...and so on for 35 lines with different pids.
Any recommendation as to how I can stop/kill airflow scheduler without restarting my machine. I have also checked the pid file for scheduler and tried killing that pid but no effects.
Any help is appreciated. Thanks!
Unfortuntely
kill $(ps -ef | grep "airflow scheduler" | awk '{print $2}')
I was not able to find a clean solution.
Also looking into the code
https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py
Go to the airflow directory where the pid file is and use:
cat airflow-scheduler.pid | xargs kill
Another alternative is:
/usr/bin/rm -f ${AIRFLOW_HOME}/airflow-scheduler.pid
/usr/bin/pkill -f "airflow-scheduler"
Please note better to remove older daemon .pid file; to avoid any issue while restarting on daemon mode.
You can also see: https://github.com/apache/airflow/issues/77
/bin/sh -c 'PATH=/bin:/sbin:/usr/bin:/usr/sbin mPPID=`cat ${AIRFLOW_HOME}/airflow-scheduler.pid`;ps -o pid= --ppid $mPPID | xargs kill -15 && kill -15 $mPPID && rm -f ${AIRFLOW_HOME}/airflow-scheduler.pid'"
But this command relies on parent PID (.pid file) so if parent process was removed and child processes are still running it won't work.
So in my opinion accepted answer is the best. Or if you have installed use pkill
If using any monitoring service i.e monit like in the github link.
Valid stop command would be:
gPPID=`ps -ef | /usr/bin/grep 'airflow scheduler' | /usr/bin/grep -v grep | awk '{print $2}' `; echo $gPPID; | xargs kill -15 && rm -f ${AIRFLOW_HOME}/airflow-scheduler.pid
cd ~/airflow
cat airflow-scheduler.pid | xargs kill
To kill airflow webserver and scheduler, you can use below command
If you have supervisord configured for airflow, then stop it first
supervisorctl stop all
kill -9 `ps aux | grep airflow | awk '{print $2}'`
Related
I have to do the same thing many times a day:
ps aux
look for process running ssh to one of my servers ...
kill -9 <pid>
I'm looking to see if I can alias process into one line. The output of ps aux is usually something like this:
user 6871 0.0 0.0 4351260 8 ?? Ss 3:28PM 0:05.95 ssh -Nf -L 18881:my-server:18881
user 3018 0.0 0.0 4334292 52 ?? S 12:08PM 0:00.15 /usr/bin/ssh-agent -l
user 9687 0.0 0.0 4294392 928 s002 S+ 10:48AM 0:00.00 grep ssh
I always want to kill the process with the my-server in it.
Does anyone know how I could accomplish this?
for pid in $(ps aux | grep "[m]y-server" | awk '{print $2}'); do kill -9 $pid; done
ps aux | grep "[m]y-server" | awk '{print $2}' - this part gives you list of pids processes that include "my-server". And this script will go through this list and kill all this processes.
I use pgrep -- if you're sure you want to kill all the processes that match:
$ kill -9 `pgrep my-server`
A simple solution is to use pkill:
sudo pkill my-server
I am trying to kill a process in linux
ps -aux
root 14074 0.0 0.4 6586120 67452 pts/0 Sl 22:45 0:01 java -cp target/cronscheduler-1.0-SNAPSHOT.jar com.cronscheduler.QuartzMain
Kill the process in the stop script using the below command
ps aux | grep "java -cp target/cronscheduler-1.0-SNAPSHOT.jar com.cronscheduler.QuartzMain" | \
grep -v grep | awk '{print $2}' | xargs kill -9
Issue is this command works fine when cronscheduler.QuartzMain is running. But when this process is already killed then the above command throws error
Usage:
kill [options] <pid|name> [...]
Your inputs on removing the errors are appreciated
pkill can search through the complete command line. Try
pkill -9 -f 'java -cp target/cronscheduler-1.0-SNAPSHOT.jar com.cronscheduler.QuartzMain'
Your command may create errors, because it sends more than the pid (the complete line from ps) to kill.
One of the solution I found was to redirect the error message :
cat /etc/*.conf 2> /dev/null
ps aux | grep httpd
httpd 3486 0.0 0.1 4248 1432 ? S Jul31 0:00 /usr/sbin/httpd -f /etc/httpd/httpd.conf
# kill 3486
OR
$ sudo kill 3486
Please try below this will help in clearing the child process id's as well.
for i in `ps -ef| grep "java -cp target/cronscheduler-1.0-SNAPSHOT.jar com.cronscheduler.QuartzMain" | grep -v grep | awk '{print $2}'`
do
ptree $i|awk '{print $1}' >>pids.txt
done
for i in $(cat pids.txt)
do
kill -9 $i
done
I'm trying to execute the following command:
ps aux | grep com.scheduler.app.workermain | kill -15 [pid]
How can I obtain the [pid] (or list of PID) using ps aux | grep "expression" and pipe that to kill? There may be zero or many processes running the machine. This is part of an automated job, to ensure all the processes spun will be terminated.
A sample line from the command line, when ps aux | grep com.scheduler.app.workermain is executed is:
jenkins 12373 1.1 4.2 2905440 173628 ? Sl 19:28 0:05 java -Xmx600m -Dlog4j.configurationFile=log4j2-trace.xml -Dpid=foobar -Dipaddr=127.0.0.1 -cp build/classes:build/dependency/* com.scheduler.app.workermain testing.properties
pkill is used for exactly this purpose. How about:
pkill -15 -f com.scheduler.app.workermain
Also if you just want to grep for a PID you can use pgrep:
pgrep -f com.scheduler.app.workermain
pkill man page
kill -15 $(ps aux | grep -i com.scheduler.app.workermain | awk -F' ' '{ print $2 }')
One of possible solutions is to use the pidof command:
kill $( pidof com.scheduler.app.workermain )
PS. You don't need to pass -15 (or -TERM) to the kill command, as SIGTERM is the default signal sent.
Say I want to kill every process containing the word amarok. I can print out the commands I want to execute. But how do I actually make the shell execute them. ie.
ps aux | grep -ie amarok | awk '{print "kill -9 " $2}'
Output:
kill -9 3052
kill -9 3071
kill -9 3076
kill -9 3077
kill -9 3079
kill -9 3080
kill -9 3082
kill -9 3083
kill -9 3084
kill -9 3085
kill -9 3086
kill -9 3087
kill -9 3088
kill -9 3089
kill -9 4031
From man 1 pkill
-f The pattern is normally only matched against the process name.
When -f is set, the full command line is used.
Which means, for example, if we see these lines in ps aux:
apache 24268 0.0 2.6 388152 27116 ? S Jun13 0:10 /usr/sbin/httpd
apache 24272 0.0 2.6 387944 27104 ? S Jun13 0:09 /usr/sbin/httpd
apache 24319 0.0 2.6 387884 27316 ? S Jun15 0:04 /usr/sbin/httpd
We can kill them all using the pkill -f option:
pkill -f httpd
ps aux | grep -ie amarok | awk '{print $2}' | xargs kill -9
xargs(1): xargs -- construct argument list(s) and execute utility. Helpful when you want to pipe in arguments to something like kill or ls or so on.
use pgrep
kill -9 $(pgrep amarok)
The safe way to do this is:
pkill -f amarok
I think this command killall is exactly what you need.
The command is described as "kill processes by name".It's easy to use.For example
killall chrome
This command will kill all process of Chrome.Here is a link about killall command
http://linux.about.com/library/cmd/blcmdl1_killall.htm
Hope this command could help you.
pkill -x matches the process name exactly.
pkill -x amarok
pkill -f is similar but allows a regular expression pattern.
Note that pkill with no other parameters (e.g. -x, -f) will allow partial matches on process names. So "pkill amarok" would kill amarok, amarokBanana, bananaamarok, etc.
I wish -x was the default behavior!
try kill -s 9 `ps -ef |grep "Nov 11" |grep -v grep | awk '{print $2}'` To kill processes of November 11
or
kill -s 9 `ps -ef |grep amarok|grep -v grep | awk '{print $2}'`
To kill processes that contain the word amarok
If you want to execute the output of a command, you can put it inside $(...), however for your specific task take a look at the killall and pkill commands.
You can also evaluate your output as a sub-process, by surrounding everything with back ticks or with putting it inside $():
`ps aux | grep -ie amarok | awk '{print "kill -9 " $2}'`
$(ps aux | grep -ie amarok | awk '{print "kill -9 " $2}')
Maybe adding the commands to executable file, setting +x permission and then executing?
ps aux | grep -ie amarok | awk '{print "kill -9 " $2}' > pk;chmod +x pk;./pk;rm pk
If you're using cygwin or some minimal shell that lacks killall you can just use this script:
killall.sh - Kill by process name.
#/bin/bash
ps -W | grep "$1" | awk '{print $1}' | xargs kill --
Usage:
$ killall <process name>
I want linux script to kill java program running on console.
Following is the process running as jar.
[rapp#s1-dlap0 ~]$ ps -ef |grep java
rapp 9473 1 0 15:03 pts/1 00:00:15 java -jar wskInterface-0.0.1-SNAPSHOT-jar-with-dependencies.jar
rapp 10177 8995 0 16:00 pts/1 00:00:00 grep java
[rapp#s1-dlap0 ~]$
You can simply use pkill -f like this:
pkill -f 'java -jar'
EDIT: To kill a particular java process running your specific jar use this regex based pkill command:
pkill -f 'java.*lnwskInterface'
If you just want to kill any/all java processes, then all you need is;
killall java
If, however, you want to kill the wskInterface process in particular, then you're most of the way there, you just need to strip out the process id;
PID=`ps -ef | grep wskInterface | awk '{ print $2 }'`
kill -9 $PID
Should do it, there is probably an easier way though...
if there are multiple java processes and you wish to kill them with one command
try the below command
kill -9 $(ps -ef | pgrep -f "java")
replace "java" with any process string identifier , to kill anything else.
pkill -f for whatever reason does not work for me. Whatever that does, it seems very finicky about actually grepping through what ps aux shows me clearly is there.
After an afternoon of swearing I went for putting the following in my start script:
(ps aux | grep -v -e 'grep ' | grep MainApp | tr -s " " | cut -d " " -f 2 | xargs kill -9 ) || true
Use jps to list running java processes. The command returns the process id along with the main class. You can use kill command to kill the process with the returned id or use following one liner script.
kill $(jps | grep <MainClass> | awk '{print $1}')
MainClass is a class in your running java program which contains the main method.