I am looking for a way to log and graphically display cpu and RAM usage of linux processes over time. Since I couldn't find a simple tool to so (I tried zabbix and munin but installation failed) I started writing a shell script to do so
The script file parses the output of top command through awk and logs into a csv file. It
Figures out the pid of the processes through ps command
Uses top and awk to log cpu and memory usage.
Here is how the script looks like
#!/bin/sh
#A script to log the cpu and memory usage of linux processes namely - redis, logstash, elasticsearch and kibana
REDIS_PID=$(ps -ef | grep redis | grep -v grep | awk '{print $2}')
LOGSTASH_PID=$(ps -ef | grep logstash | grep -v grep | awk '{print $2}')
ELASTICSEARCH_PID=$(ps -ef | grep elasticsearch | grep -v grep | awk '{print $2}')
KIBANA_PID=$(ps -ef | grep kibana | grep -v grep | awk '{print $2}')
LOG_FILE=/var/log/user/usage.log
echo $LOG_FILE
top -b | awk -v redis="$REDIS_PID" -v logstash="$LOGSTASH_PID" '/redis|logstash/ {print $1","$9","$10","$12}'
How do I
Print the resource usage for multiple processes. Specifying multiple
variables in the awk pattern is not working. It prints the usage for
the first pid (redis in the above script)
Print current timestamp when printing the resource details (through date +"%T")
Print the process name along with the resource usage. Redis, Logstash, ElasticSearch or Kibana in the above case
Redirect the above commands output to a log file. I tried > $LOG_FILE but it didn't work.
Thoughts/Inputs?
Thanks in advance.
To figure out PIDs you can simplify your script greatly using pgrep:
REDIS_PID=$(pgrep -f redis)
LOGSTASH_PID=$(pgrep -f logstash)
ELASTICSEARCH_PID=$(pgrep -f elasticsearch)
KIBANA_PID=$(pgrep -f kibana)
EDIT: Sorry had to leave for some work and couldn't provide the full answer.
In order to capture top's output use following script:
while :; do
top -n 1 -b | awk -v redis="$REDIS_PID" -v logstash="$LOGSTASH_PID"
'$1 == redis || $1 == logstash {print $1","$9","$10","$12}' >> $LOG_FILE
sleep 3
done
Related
if I run the expression
ps -fu $USER| grep 'mount' | grep -v 'grep' | awk '{print $2}'
in the command line, I get - as expected - the PID of the processes containing "mount" in their description.
I want to achieve the following to kill certain background processes programmatically. The following code in the shell script:
#!/usr/bin/env bash
mountcmd="ps -fu $USER| grep 'mount' | grep -v 'grep' | awk '{print $2}' "
mountpid=$(eval "$mountcmd")
echo "Found existing background job PID: " "$mountpid"
does not provide the PID, but the output of echo is:
Found existing background job PID: wgeithne 6284 1 0 17:09 pts/3 00:00:00 minikube mount /u/wgeithne/bin/grafana/config:/grafana
How do I get the only the PID as output of my script?
The stupid eval trick requires additional escaping of the dollar sign in the Awk script. But really, a massively superior solution is to avoid stupid eval tricks.
Perhaps see also https://mywiki.wooledge.org/BashFAQ/050
If you really need to reinvent pidof, probably get rid of the antipatterns.
mountpids=$(ps -fu "$USER" | awk '/[m]ount/ { print $2 }')
I have a System V init script I've developed that starts a Java program. For some reason whenever the PID file gets created, it contains multiple PIDs instead of one.
Here's the relevant code that starts the service and writes to the PID file:
daemon --pidfile=$pidfile "$JAVA_CMD &" >> $logfile 2>&1
RETVAL=$?
usleep 500000
if [ $RETVAL -eq 0 ]; then
touch "$lock"
PID=$(ps aux | grep -vE 'grep|runuser|bash' | grep <myservice> | awk '{print $2}')
echo $PID > $pidfile
When I test the ps aux... command manually, a single line returns. When running as a script, it appears that this call is returning multiple PIDs.
Example contents in the PID file: 16601 16602 16609 16619 16690. 16619 is the actual process ID found when manually running the ps aux... command mentioned above.
Try reversing your greps. The first one (-vE) may run BEFORE the myservice one starts up. Grep for your service FIRST, then filter out the unwanted lines:
PID=$(ps aux | grep <myservice> | grep -vE 'grep|runuser|bash' | awk '{print $2}')
I encounted the same issue but not the same statement, it was like this:
PID="$(ps -ef|grep command|grep options|grep -v grep|awk '{print $2}')"
in which I used the same grep order as #Marc said in first answer, but did not filter all the unwanted lines.
So I tried the below one and it worked:
PID="$(ps -ef|grep command|grep options|grep -vE 'grep|runuser|bash'|awk '{print $2}')"
I want to check if a process is running or not. I've been trying by
ps -C /path/file
and get this response:
PID TTY TIME CMD
If I do
pgrep php
I get a list of php processes running, but only the PID.
Is there a possibility to
determine the PID of a file I specify (I want to type the file and get the PID)
get the filename if I type the PID
get all the running processes PIDs in a file to work with that in a later script
OS: Ubuntu 14.04 LTS
I've been looking for this since quite some time, tried all the possibilities I found on SO and else but just can't figure out how to do this best.
"Determine the PID of a file I specify."
lsof | grep <file> | awk '{print $2}'
"Get the filename if I type the PID."
lsof | grep <PID>
lsof | grep <PID> | awk '{print $NF}'
"Get all the running processes PIDs in a file to work with that in a later script."
ps x | awk '{print $1}' > pid-list.txt # PIDs of all processes run by current user
ps ax | awk '{print $1}' > pid-list.txt # PIDs of all processes run by all users
What about ps aux | grep <the-name-of-the-process>.
I'm running a script at the end of a Jenkins build to restart Tomcat. Tomcat's shutdown.sh script is widely known not to work all in many instances and so my script is supposed to capture the PID of the Tomcat process and then attempt to manually shut it down. Here is the command I'm using to capture the PID:
ps -ef | grep Bootstrap | grep -v grep | awk '{print $2}' > tomcat.pid
The output when manually runs retrieves the PID perfectly. During the Jenkins build I have to switch users to run the command. I'm using "su user -c 'commands'" like this:
su user -c "ps -ef | grep Bootstrap | grep -v grep | awk '{print $2}' > tomcat.pid"
Whenever I do this however, the "awk" portion doesn't seem to be working. Instead of just retrieving the PID, it's capturing the entire process information. Why is this? How can I fix the command?
The issue is that $2 is being processed by the original shell before being sent to the new user. Since the value of $2 in the shell is blank, the awk command at the target shell essentially becomes awk {print }. To fix it, you just escape the $2:
su user -c "pushd $TOMCAT_HOME;ps -ef | grep Bootstrap | grep -v grep | awk '{print \$2}' > $TOMCAT_HOME/bin/tomcat.pid"
Note that you want the $TOMCAT_HOME to be processed by the original shell so that it's value is set properly.
You don't need the pushd command as you can replace the awk command with:
cut -d\ -f2
Note: two 2 spaces between -d\ and -f2
I am writing one shell script and I want to get PID of one process with name as "ABCD". What i did was :
process_id=`/bin/ps -fu $USER|grep "ABCD"|awk '{print $2}'`
This gets PID of two processes i.e. of process ABCD and the GREP command itself what if I don't want to get PID of GREP executed and I want PID only of ABCD process?
Please suggest.
Just grep away grep itself!
process_id=`/bin/ps -fu $USER| grep "ABCD" | grep -v "grep" | awk '{print $2}'`
Have you tried to use pidof ABCD ?
It's very straight forward. ABCD should be replaced by your process name.
#!/bin/bash
processId=$(ps -ef | grep 'ABCD' | grep -v 'grep' | awk '{ printf $2 }')
echo $processId
Sometimes you need to replace ABCD by software name. Example - if you run a java program like java -jar TestJar.jar & then you need to replace ABCD by TestJar.jar.
ps has an option for that:
process_id=`/bin/ps -C ABCD -o pid=`
You can also do away with grep and use only awk.
Use awk's expression matching to match the process name but not itself.
/bin/ps -fu $USER | awk '/ABCD/ && !/awk/ {print $2}'
You can use this command to grep the pid of a particular process & echo $b to print pid of any running process:
b=`ps -ef | grep [A]BCD | awk '{ printf $2 }'`
echo $b
ps | pgrep ABCD
You can try the above command to return the process id of the ABCD process.
I found a better way to do this.
top -n 1 | grep "##" | grep -Eo '^[^ ]+'