So I am trying to check the internal CPU and RAM usage from JVM's.
The set up is that we have a single server which hosts 39 JVM's each running its own service and has it's own unique CPU & RAM allocation. So for example it could be set up like the following:
CPU(Cores) RAM(MB)
JVM1 2 300
JVM2 1 50
JVM3 5 1024
These are fictional as I don't have the actual values to hand.
I know the PID's for each of the JVM's but I am wondering how would I see the CPU and RAM usage of each JVM on it's own dissregarding the Host systems usage. Also is it possible to pass multiple PID's into a jstat -gc as I will be looking to script this?.
I know that if I use:
ps -p <PID> -o %CPU %RAM
That will give me the CPU and RAM of that process on the host machine.
Any Help is greatly appreciated.
After playing around for a while I came up with the following script which pulls the memory usage and total memory in use to 2DP:
Script
#!/bin/sh
service=$(mktemp)
jstatop=$(mktemp)
ServiceStatsPre=$(mktemp)
datet=$(date +%Y-%m-%d)
hourt=$(date +%H)
ps -ef | grep java | grep Service | awk '{key=substr($9,31,match($9,"#")-31); serv[key]++}END{for(name in serv){if($9 ~ /Service/){print name}}}'>$service
printf "%-8sService\t\t\t\t%7s PID\t%7s Used\t%7s Total\n" > $ServiceStatsPre
filename=$service
while read -r line; do
if [ ! -z $line ]; then
pid=$(ps -ef | grep java | grep Service |awk -v svc="$line" '{if (match($9,svc)){print $2}}')
rncnt=0
rnag=1
while [ $rnag -eq 1 ]; do
jstat -gc $pid > $jstatop
if [ $? -ne 0 ]; then
sleep 5;
rncnt++;
else
rnag=0
fi
if [ $rncnt -eq 5 ]; then
rnag=0
fi
done
cat $jstatop | awk '{if (NR !=1) print}'|awk -v pid="$pid" -v svc="$line" -v d="$datet" -v h="$hourt" '{printf("%-40s %7d %6.2fMB %6.2fMB %11s %3s\n",svc,pid,($1+$5+$7)/1024,($3+$4+$6+$8)/1024,d,h)}' >> $ServiceStatsPre
fi
done < $filename
#printf "Date,Hour,Service,Used,Total\n" > Service_Stats.csv #Uncomment this line on initial Run to create file with headders
cat $ServiceStatsPre | awk '{if (NR !=1) print}' |
awk '{
printf("%-1s,%1s,%1s,%4.2f,%4.2f\n",$5,$6,$1,$3,$4);
}' >> Service_Stats.csv
rm $service;
rm $jstatop;
rm $ServiceStatsPre;
Breakdown
ps -ef | grep java | grep Service | awk '{key=substr($9,31,match($9,"#")-31); serv[key]++}END{for(name in serv){if($9 ~ /Service/){print name}}}'>$service
This is returning the name of the Service and putting it to a tempary file $service
printf "%-8sService\t\t\t\t%7s PID\t%7s Used\t%7s Total\n" > $ServiceStatsPre
This is just creating Headings in a Temp file $ServiceStatsPre (I know this isn't needed but I was originally using it to debug to a file not a temp file)
Outer While loop
filename=$service
if [ ! -z $line ]; then
while read -r line; do
pid=$(ps -ef | grep java | grep Service |awk -v svc="$line" '{if (match($9,svc)){print $2}}')
rncnt=0
rnag=1
...
cat $jstatop | awk '{if (NR !=1) print}'|awk -v pid="$pid" -v svc="$line" -v d="$datet" -v h="$hourt" '{printf("%-40s %7d %6.2fMB %6.2fMB %11s %3s\n",svc,pid,($1+$5+$7)/1024,($3+$4+$6+$8)/1024,d,h)}' >> $ServiceStatsPre
fi
done < $filename
This is first checking to see if the line that it is passing in is not empty (if [ ! -z $line ]; then) then it is extracting the PID for each of the services in the temp file $service and is setting a check (rnag) + retry counter (rncnt). The final part of this outer while loop is calculating the stats, memory usage in MB (($1+$5+$7)/1024), Total Memory used (($3+$4+$6+$8)/1024) as well as retuning the PID,Service Name,Date and Hour.
Inner While Loop
...
while [ $rnag -eq 1 ]; do
jstat -gc $pid > $jstatop
if [ $? -ne 0 ]; then
sleep 5;
rncnt++;
else
rnag=0
fi
if [ $rncnt -eq 5 ]; then
rnag=0
fi
done
...
This is to basically ensure that the jstat is running correctly and isn't returning an error.
The final part of this script:
#printf "Date,Hour,Service,Used,Total\n" > Service_Stats.csv #Uncomment this line on initial Run to create file with headders
cat $ServiceStatsPre | awk '{if (NR !=1) print}' |
awk '{
printf("%-1s,%1s,%1s,%4.2f,%4.2f\n",$5,$6,$1,$3,$4);
}' >> Service_Stats.csv
is just to resolve the formatting and put it into a format so that it can be used in a CSV.
Output
This script out puts like the following:
Date,Hour,Service,Used,Total
2016-07-03,07,undMFSAdapterServiceVM,331.00,188.87
2016-07-03,07,entSCSServiceVM,332.50,278.19
2016-07-03,07,kServiceVM,457.50,132.91
2016-07-03,07,ServiceTuTrackVm,432.00,282.66
2016-07-03,07,ter1WMSAdapterServiceVm,233.00,77.02
2016-07-03,07,kingMFSAdapterServiceVM,451.50,126.69
2016-07-03,07,erBuilderServiceVM,261.50,211.27
2016-07-03,07,ter3MFSAdapterServiceVM,449.50,210.23
2016-07-03,07,rServiceVM1,1187.00,529.26
2016-07-03,07,rServiceVM2,597.50,398.43
2016-07-03,07,rServiceVM3,2786.00,819.30
2016-07-03,07,rServiceVM4,451.50,163.13
2016-07-03,07,MessagingServiceVm,457.50,357.11
2016-07-03,07,viceVM,444.50,263.59
2016-07-03,07,ServiceVM,1910.50,909.19
2016-07-03,07,undPackingMFSAdapterServiceVM,208.00,113.51
2016-07-03,07,gisticLockServiceVM,245.00,173.05
2016-07-03,07,kingMFSAdapterServiceVM,781.50,327.13
2016-07-03,07,ferWMSAdapterServiceVm,196.00,84.02
2016-07-03,07,geMFSAdapterServiceVM,499.50,256.91
2016-07-03,07,ferMFSAdapterServiceVM,456.50,246.89
2016-07-03,07,kingWMSAdapterServiceVm,195.00,73.70
2016-07-03,07,AdapterServiceVm,149.50,72.62
2016-07-03,07,ter2MFSAdapterServiceVM,455.00,136.02
2016-07-03,07,ionServiceVM,484.00,240.46
2016-07-03,07,ter3WMSAdapterServiceVm,266.00,138.70
2016-07-03,07,iceVm,135.50,106.07
2016-07-03,07,viceVm,3317.00,1882.15
2016-07-03,07,entBCSServiceVM,356.50,143.93
2016-07-03,07,ServiceVm,951.00,227.12
2016-07-03,07,lingServiceVM,145.50,76.61
2016-07-03,07,entDBHServiceVM,182.50,4.63
2016-07-03,07,kingWMSAdapterServiceVm,208.00,103.13
2016-07-03,07,gingServiceVM,1529.50,235.84
2016-07-03,07,ServiceVM,249.50,131.78
2016-07-03,07,ter1MFSAdapterServiceVM,453.00,394.11
2016-07-03,07,AdapterServiceVM,461.00,208.41
2016-07-03,07,ter2WMSAdapterServiceVm,178.50,79.93
2016-07-03,07,AdapterServiceVm,395.00,131.08
2016-07-03,07,ingServiceVM,184.50,126.28
The $3 has been trimmed just to hide service names etc..
Helpful Links
A nice explanation of jstat -gc output
I am trying to figure out how to extract the cpu core utilization for a SINGLE process in linux and parse it. I know that I can get the overall core utilization via top and then press "1". I am already able to parse that. However, now I want to do the same thing for a single process. I tried it with ps and calculating the core utilization myself but I am not so sure if my script is accurate enough, something seems off. (Note that this version calculates overall core utilization since it is WIP) I get errors like this after a while in my terminal:
test.sh: line 31: +: syntax error: operand expected (error token is
"+")
I cannot figure out why this error just randomly occurs.
#!/bin/env bash
read -p "Enter PID to observe:" pid
run=false
ps -p $pid -L -o cputime,etime,psr,pcpu
for (( u = 1 ; u <= 100 ; u++ ))
do
lines=$(ps -p $pid -L -o psr,pcpu | awk 'END{print NR}')
for (( i=1; i<=$lines; i++))
do
core=$(ps -p $pid -L -o psr,pcpu | awk 'NR=='$i'{print $1}')
if [ $run == false ]
then cpuTimeS=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $1}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
elapsedTimeS=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $2}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
cpuTimeSi=${cpuTimeS%.*}
elapsedTimeSi=${elapsedTimeS%.*}
cpuTimeSiResult=$(( cpuTimeSi + cpuTimeSiResult ))
elapsedTimeSiResult=$(( elapsedTimeSi + elapsedTimeSiResult ))
else
cpuTimeE=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $1}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
elapsedTimeE=+$(ps -p $pid -L -o cputime,etime | awk 'NR=='$i'{print $2}' | awk -F : '{ printf("%.2f\n", $1*60+$2*60+$3); }')
cpuTimeEi=${cpuTimeE%.*}
elapsedTimeEi=${elapsedTimeE%.*}
cpuTimeEiResult=$(( cpuTimeEi + cpuTimeEiResult ))
elapsedTimeEiResult=$(( elapsedTimeEi + elapsedTimeEiResult ))
fi
done
if [ "$run" = true ]
then result=$( echo "scale=2; ($cpuTimeEiResult - $cpuTimeSiResult) / ($elapsedTimeEiResult - $elapsedTimeSiResult) * 100.0" | bc)
echo "RESULT:" $result
echo "cpuTimeSTART:" $cpuTimeSiResult
echo "elapsedTimeSTART:" $elapsedTimeSiResult
echo "cpuTimeEND:" $cpuTimeEiResult
echo "elapsedTimeEND:" $elapsedTimeEiResult
fi
sleep 1
if [ "$run" = false ]
then run=true
else
run=false
fi
done
Are there any ideas on how I could solve this better?
I am glad for any advice
Looking at your actual request to get the CPU utilization from a particular process; this could be done in less lines; e.g. below is the CPU usage after giving a seq command.
#using the process name
$ ps -C seq -o %cpu
%CPU
10.4
#using the process id
$ ps -p 5710 -o %cpu
%CPU
10.4
There is a perl script that needs to run as root but we must make sure the user who runs the script did not log-in originally as user 'foo' as it will be removed during the script.
So how can I find out if the user, who might have su-ed several times since she logged in has not impersonated 'foo' at any time in that chain?
I found an interesting perl script that was calling the following two shell scripts, but I think that would only work on Solaris.
my $shell_parent =
`ps -ef | grep -v grep | awk \'{print \$2\" \"\$3}\' | egrep \"^#_\" | awk \'{print \$2}'`;
my $parent_owner =
`ps -ef | grep -v grep | awk \'{print \$1\" \"\$2}\' | grep #_ | awk \'{print \$1}\'`;
This needs to work on both Linux and Solaris and I'd rather eliminate the repeated calls to he the shell and keep the whole thing in Perl.
Quick and dirty and (UNIX only):
my $user = (split /\s/,`who am i`)[0];
The who am i command returns the owner of the TTY - i.e. who you were when you logged in.
If you want to do this in pure perl:
use POSIX;
my $tty = POSIX::ttyname(1); # The tty we are running in
my $uid = (stat $tty)[4]; # The owner uid of that tty
my $user = getpwuid($uid); # The user with that uid
This will return the correct user, even after multiple su's. This usually freaks out your (less experienced) sysadmins.
Here's a Perl program that checks for direct setuid change:
#! /usr/bin/perl
sub callingUser() {
my ($login, $pass, $uid, $gid) = getpwuid($<);
return $login;
}
sub effectiveUser() {
my ($login, $pass, $uid, $gid) = getpwuid($>);
return $login;
}
printf("Real user name: %s\n", effectiveUser());
printf("Calling user name: %s\n", callingUser());
But since you mentioned that the setuid change may have occured anytime before, you probably have to parse the output of ps: I would do it using the following command. This command only uses features defined in POSIX, so I hope it is portable to all kinds of systems:
ps -e -o pid,ppid,user,ruser
Maybe the following is what you want. The function hasBeenUser reads the process table and then follows the process chain from the current process down the parent process. If any of the processes on the way has a user or real user field equal to the username in question, the function returns a nonzero value.
#! /usr/bin/perl
sub hasBeenUser($) {
my ($username) = #_;
my $procs = {};
open(PS, "ps -e -o pid,ppid,user,ruser |") or die;
while (defined(my $line = <PS>)) {
next unless $line =~ m"^(\d+)\s+(\d+)\s+(\S+)\s+(\S+)\s+$";
my ($pid, $ppid, $user, $ruser) = (int($1), int($2), $3, $4);
$procs->{$pid} = [$pid, $ppid, $user, $ruser];
}
close(PS) or die;
my $pid = $$;
while (exists($procs->{$pid})) {
my $proc = $procs->{$pid};
delete $procs->{$pid}; # don't risk ending in an endless loop.
warn "D: checking process $pid\n";
if ($proc->[2] eq $username || $proc[3] eq $username) {
warn "E: process $pid was called by $username.\n";
return 1;
}
last if $pid < 2;
$pid = $proc->[1];
}
return 0;
}
hasBeenUser("del"); # should return 0
hasBeenUser("root"); # should return nonzero
I recognized a corner case when calling scripts from mc (at least in our RHEL's), which results that the who am i does not output anything. To circumvent that, I produced the following one-liner in bash:
REALUSERNAME=$(ps uhp `ps -AjH | grep \`ps -u $USER fh | awk '{ print $0; if(index($0, "ps -u $USER fh")) exit 0;}' | tac | awk '{if(!index($0, "\\\\\_")){print $1; exit 0;}}'\` | awk '{print $3}'` | awk '{print $1}')
Essentially, this walks backwards on the tree output of ps -u $USER fh and then crops on the topmost username column.
Thoughts, better solutions are welcome :-)