Which command is correct to check the current job in running status on a specific agent in Autosys
1 ) Autorep -I machine_name | find RU
OR
2 ) Autorep -j ALL - M machine_name | find RU
Below commands returns the list of running job on a particular machine / agent.
autorep -M <machine/agent_name> -d
example:
autorep -M host123#some.com -d
Output Columns:
JobName | Machine | Status | Load | Priority
If executing from linux server, use grep, awk as per need to reformat the report.
Related
I am trying to combine the following commands:
docker ps | grep track
that will give me
6b86b28a27b0 dev/jobservice/worker-jobtracking:3.5.0-SNAPSHOT "/tini -- /startup/s…" 25 seconds ago Up 2 seconds (health: starting)
jobservice_jobTrackingWorker_1
So then, I grab the id and use it in the next request as:
docker logs 6b8 | grep -A 3 'info'
So far, the easiest way I could find was to send those commands separately, but i wonder if there would be a simple way to do it.
I think that the main issue here is that I am trying to find the name of the container based on part of its name.
So, to resume, I would like to find and store the id of a container based on its name then use it to explore its logs.
Thanks!
Perhaps there are cleaner ways to do it, but this works.
To get the ID of a partially matching container name:
$ docker ps --format "{{.ID}} {{.Names}}" | grep "partial" | cut -d " " -f1
Then you can use it in another bash command:
$ docker logs $(docker ps --format "{{.ID}} {{.Names}}" | grep "partial" | cut -d " " -f1)
Or wrap it in a function:
$ function dlog() { docker logs $(docker ps --format "{{.ID}} {{.Names}}" | grep "$1" | cut -d " " -f1); }
Which can then be used as:
$ dlog partial
In a nutshell the pure bash approach to achieve what you want:
With sudo:
sudo docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs sudo docker logs
Without sudo:
docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs docker logs
Now let's break it down...
Let's see what containers I have running in my laptop for the Elixir programming language:
command:
sudo docker ps | grep -i elixir -
output:
0a19c6e305a2 exadra37/phoenix-dev:1.5.3_elixir-1.10.3_erlang-23.0.2_git "iex -S mix phx.serv…" 7 days ago Up 7 days 127.0.0.1:2000-2001->2000-2001/tcp Projects_todo-tasks_app
65ef527065a8 exadra37/st3-3211-elixir:latest "bash" 7 days ago Up 7 days SUBL3_1600981599
232d8cfe04d5 exadra37/phoenix-dev:1.5.3_elixir-1.10.3_erlang-23.0.2_git "mix phx.server" 8 days ago Up 8 days 127.0.0.1:4000-4001->4000-4001/tcp Staging_todo-tasks_app
Now let's find their ids:
command:
sudo docker ps | grep -i elixir - | awk '{print $1}'
output:
0a19c6e305a2
65ef527065a8
232d8cfe04d5
Let's get the first container ID:
command:
sudo docker ps | grep -i elixir - | awk '{print $1}' | head -1
NOTE: replace head -1 with head -2 to get the second line in the output...
output:
0a19c6e305a2
Let's see the logs for the first container in the list
command:
sudo docker ps | grep -i elixir - | awk '{print $1}' | head -1 | xargs sudo docker logs
NOTE: replace head -1 with tail -1 to get the logs for the last container in the list.
output:
[info] | module=WebIt.Live.Calendar.Socket function=mount/1 line=14 | Mount Calendar for date: 2020-09-30 23:29:38.229174Z
[debug] | module=Tzdata.ReleaseUpdater function=poll_for_update/0 line=40 | Tzdata polling for update.
[debug] | module=Tzdata.ReleaseUpdater function=poll_for_update/0 line=44 | Tzdata polling shows the loaded tz database is up to date.
Combining the different replies, I used:
function dlog() { docker ps | grep -i track - | awk '{print $1}' | head -1 | xargs docker logs | grep -i -A 4 "$2";}
to get the best of both worlds. So I can have a function that will have me type 4 letters instead of 2 commands and with no case sensitivity
I can then use dlog keyword to get my logs.
I hardcoded track and -A 4 as I usually use that query but I could have passed them as arguments to add on modularity (my goal here was really simplicity)
Thanks for your help!
I need help with the command where I am trying to grep the PIDs of ecm simulator and kill the same using kubectl :
kubectl exec eric-service-0 -n cicd --kubeconfig /root/admin.conf -- bash -c "ps -ef | grep ecm | grep node | awk '{print $2}' "
Output of the above command:
root 9857 0 0 07:11 ? 00:00:00 bash -c /tmp/simulator/node-v8.11.3-linux-x64/bin/node /tmp/simulator/ecm_mod.js> /tmp/simulatorEcmResponse.txt
root 9863 9857 0 07:11 ? 00:00:00 /tmp/simulator/node-v8.11.3-linux-x64/bin/node /tmp/simulator/ecm_mod.js
Expected output is:
9857
9863
Then further I need to kill the PIDs:
kubectl exec eric-service-0 -n cicd --kubeconfig /root/admin.conf -- bash -c "ps -ef | grep ecm | grep node | awk '{print $2}' | xargs kill -9"
When I am executing the same within the service pod it's working but it's giving issues when I am doing via kubectl from outside.
Could anyone please let me know what I am doing wrong here?
NOTE: There are 2 PIDs which needs to be killed from the below output:
eric-service-0:/ # ps -ef | grep ecm | grep node
root 9857 0 0 07:11 ? 00:00:00 bash -c /tmp/simulator/node-v8.11.3-linux-x64/bin/node /tmp/simulator/ecm_mod.js> /tmp/simulatorEcmResponse.txt
root 9863 9857 0 07:11 ? 00:00:00 /tmp/simulator/node-v8.11.3-linux-x64/bin/node /tmp/simulator/ecm_mod.js
EDIT:
Output of the command as asked by #Cyrus below:
Posting this this as Community Wiki answer for better visibility. Solution has been provided in comments by #Cyrus.
In Short, OP wanted to Kill/interrupt some process using their PID's. OP wanted to do it from cluster level on specific pod/container which included ecm simulator.
To do it, commands below were used:
exec - execute a command in a container
-- bash - run bash inside container
ps -ef - list all process on the system
grep - serch specific pattern
awk - pattern scanning and processing language.
xargs - build and execute command lines from standard input
kill - send a signal to a process
In MANUAL you can find some information about ps flags:
To see every process on the system using standard syntax:
ps -e
ps -ef
ps -eF
ps -ely
however each flag will still give another output, like below:
-e
PID TTY TIME CMD
-ef
UID PID PPID C STIME TTY TIME CMD
Cyrus advised to use following command:
kubectl exec eric-service-0 -n cicd --kubeconfig /root/admin.conf -- bash -c "pgrep -f 'node.*ecm'"
bash -c - If the -c option is present, then commands are read from the first non-option argument command_string.
Also explain in comment:
pgrep looks through the currently running processes and lists the process IDs which match the selection criteria to stdout. From man pgrep. node.*ecm is a regex.
I have a bunch of job arrays that are running right now (SLURM).
For example, 2552376_1, 2552376_10, 2552376_20, 2552376_80, 2552377_1, 2552377_10, 2552377_20, 2552377_80 and so on.
Currently, I am interested in that which end with _1.
Is there any way to hold all others without specifying job ids (because I have several hundreds of them)?
The following command works for holding all the jobs:
squeue -r -t PD -u $USER -o "scontrol hold %i" | tail -n +2 | sh
For releasing the one with needed id I use
squeue -r -u $USER -o "scontrol release %i" | tail -n +2 | grep "_1$" | sh
which picks correct jobs.
Mass update of jobs can be done by abusing the output formatting of squeue:
Hold all your pending jobs:
squeue -r -t PD -u $USER -o "scontrol hold %i" | sh
then release all your jobs ending in _1
squeue -r -t PD -u $USER -o "scontrol release %i" | grep "_1$" | sh
First run the commands without the | sh part to make sure it is working the way intended.
Note the -r option to display one job array element per line.
Is there a simple way to get SLURM to print out, for a given user, the number of jobs of each status (e.g., running, pending, completed, failed, etc.)?
One way to get that information is with:
squeue -u $USER -o%T -ST | uniq -c
The -u argument will filter jobs for the specific user, the -o%T argument will only output the job state, and the -S argument will sort them. Then uniq -c will do the counting.
Example output:
$ squeue -u $USER -o%T -ST | uniq -c
147 PENDING
49 RUNNING
I was looking for the best way to find the number of running processes with the same name via the command line in Linux. For example if I wanted to find the number of bash processes running and get "5". Currently I have a script that does a 'pidof ' and then does a count on the tokenized string. This works fine but I was wondering if there was a better way that can be done entirely via the command line. Thanks in advance for your help.
On systems that have pgrep available, the -c option returns a count of the number of processes that match the given name
pgrep -c command_name
Note that this is a grep-style match, not an exact match, so e.g. pgrep sh will also match bash processes. If you want an exact match, also use the -x option.
If pgrep is not available, you can use ps and wc.
ps -C command_name --no-headers | wc -l
The -C option to ps takes command_name as an argument, and the program prints a table of information about processes whose executable name matches the given command name. This is an exact match, not grep-style. The --no-headers option suppresses the headers of the table, which are normally printed as the first line. With --no-headers, you get one line per process matched. Then wc -l counts and prints the number of lines in its input.
result=`ps -Al | grep command-name | wc -l`
echo $result
ps -Al | grep -c bash
You can try :
ps -ef | grep -cw [p]rocess_name
OR
ps aux | grep -cw [p]rocess_name
For e.g.,:
ps -ef | grep -cw [i]nit
Some of the above didn't work for me, but they helped me on my way to this.
ps aux | grep [j]ava -c
For newbies to Linux:
ps aux prints all the currently running processes, grep searches for all processes that match the word java, the [] brackets remove the process you just ran so it wont include that as a running process and finally the -c option stands for count.
List all process names, sort and count
ps --no-headers -A -o comm | sort | uniq -c
You also can list process attached to a tty
ps --no-headers a -o comm | sort | uniq -c
You may filter with:
ps --no-headers -A -o comm | awk '{ list[$1] ++ } END { for (i in list) { if (list[i] > 10) printf ("%20s: %s\n", i, list[i]) } }'
Following bash script can be run as a cron job and you can possibly get email if any process forks itself too much.
for i in `ps -A -o comm= --sort=+comm | uniq`;
do
if (( `ps -C $i --no-headers | wc -l` > 10 )); then
echo `hostname` $i `ps -C $i --no-headers | wc -l` ;
fi
done
Replace 10 with your number of concern.
TODO: "10" could be passed as command line parameter as well. Also, few system processes can be put into exception list.
You can use ps(will show snapshot of processes) with wc(will count number of words, wc -l option will count lines i.e. newline characters).
Which is very easy and simple to remember.
ps -e | grep processName | wc -l
This simple command will print number of processes running on current server.
If you want to find the number of process running on current server for current user then use -U option of ps.
ps -U root | grep processName | wc -l
change root with username.
But as mentioned in lot of other answers you can also use ps -e | grep -c process_name which is more elegant way.
ps aux | wc -l
This command shows number of processes running on the system by all the users.
For a specific user you can use the following command:
ps -u <username> | wc -l
replace with the actual username before running :)
ps -awef | grep CAP | wc -l
Here "CAP" is the word which is in the my Process_Names.
This command output = Number of Processes + 1
This is why When we are running this command , our system read thats "ps -awef | grep CAP | wc -l " is also a process.
So yes our real answer is (Number of Processes) = Command Output - 1
Note : These processes are only those processes who include the name of "CAP"