This question already has answers here:
How do I get the total CPU usage of an application from /proc/pid/stat?
(6 answers)
Closed 7 years ago.
I need to write script which returns file with information about cpu and memory usage of specified process through given time period.
When I use ps -p pid I only get usage of one cpu core, and when I use top I get binary file as output. I tried with next :
while :;
top -n 1 -p pid | awk '{ print $9" "$10 }'
sleep 10;
done
Information the kernel offers for your process is in the /proc filesystem. Primarily you would need to parse these two files to get the pertinent data for your script
/proc/(pid)/status
/proc/(pid)/stat
This thread describes getting this CPU data in detail so I wont here.
The problem I think you'll find is CPU usage for a process is not broken down into the various cores available on your system, but rather summarized into a number that approaches 100% * (number of cores). The closest to this is the "last processor used" column of top (option f, J), though this hardly addresses your problem. A profiling tool like the one in this thread is likely the final answer.
I don't know your environment or requrements; however a solution could be running only the process isolated on a machine, then collecting each cores CPU usage at the system level loosely representing the process demand.
Try this:
$PID=24748; while true; do CPU=$(ps H -q $PID -eo "pid %cpu %mem" | grep $PID | cut -d " " -f 3 | sed -e 's/$/\+/' | tr -d "\n" | sed -e 's/+$/\n/' | bc) && MEM=$(ps H -q $PID -eo "pid %cpu %mem" | grep $PID | cut -d " " -f 4 | head -n 1) && echo $PID $CPU $MEM && sleep 3; done
It basically just adds up the CPU % usage of each thread with bc, takes the memory usage (as a whole), and prints them, of a specified PID.
Related
My intent is to log into several servers and print out their memory & cpu usage one by one. I wrote the follow scripts
START=1
END=5
for i in {$START..$END}
do
echo "myserver$i"
ssh myserver$i
free -m | awk 'NR==2{printf "Memory Usage: %s/%sMB (%.2f%%)\n", $3,$2,$3*100/$2 }'
top -bn1 | grep load | awk '{printf "CPU Load: %.2f\n", $(NF-2)}'
logout
done
But it doesn't work. Who can give a solution to this? Thanks a lot!
Look carefully at your code.
After the SSH command, you are on the remote server, in an SSH shell. And obviously your script now wants you to talk (via keyboard) to the remote server. When it is finished, e.g. if you hit ctrl-c or ctrl-d, then the next commands like "free" and "top" are running on your local machine.
You have to tell ssh with a kind of "-exec" argument that it should execute free and top on the remote server :D
I'm sure you figure it out yourself how to do that, have fun.
There is one useful command for CPU/mem usage - top.
To get the result, run this command.
CPU Usage - top -b -n 1 | grep Cpu
Mem Usage - top -b -n 1 | grep 'KiB Mem'
After searching online and combining a few answers from other questions on stackflow. I get the following solution.
Solution
On your local computer, you might want to have the following bash script, named, say, usage_ssh
START=1
END=3
date
for i in $(seq $START $END)
do
printf '=%.0s' {1..50};
printf '\n'
echo myservery$i
ssh myserver$i -o LogLevel=QUIET -t "~/bin/usage"
done
printf '=%.0s' {1..50};
printf '\n'
printf 'CPU Load: \n'
printf 'First Field\tprocesses per processor\n'
printf 'Second Filed\tidling percentage in last 5 minutes\n'
printf '\n'
printf '\n'
On your remote server, you should have the following bash script named usage. This script should be located in ~/bin.
free -m | awk 'NR==2{printf "Memory Usage\t%s/%sMB\t\t%.2f%\n", $3, $2, $3/$2*100}';
top -n 1 | grep load | awk '{printf "CPU Load\t%.2f\t\t\t%.2f\n", $(NF-2), $(NF-1)}';
Explanation
The idea is that You will call the use ssh -t <your command> to run executable on your remote file and get the output on the screen of your local computer.
Output
Sat Mar 28 10:32:34 CDT 2020
==================================================
myserver1
Memory Usage 47418/48254MB 98.27%
CPU Load 0.01 0.02
==================================================
myserver2
Memory Usage 47421/48254MB 98.27%
CPU Load 0.01 0.02
==================================================
myserver3
Memory Usage 4300/84541MB 5.09%
CPU Load 0.02 0.02
==================================================
CPU Load:
First Field processes per processor
Second Filed idling percentage in last 5 minutes
I am using a bash script that calls multiple processes which have to start up in a particular order, and certain actions have to be completed (they then print out certain messages to the logs) before the next one can be started. The bash script has the following code which works really well for most cases:
tail -Fn +1 "$log_file" | while read line; do
if echo "$line" | grep -qEi "$search_text"; then
echo "[INFO] $process_name process started up successfully"
pkill -9 -P $$ tail
return 0
elif echo "$line" | grep -qEi '^error\b'; then
echo "[INFO] ERROR or Exception is thrown listed below. $process_name process startup aborted"
echo " ($line) "
echo "[INFO] Please check $process_name process log file=$log_file for problems"
pkill -9 -P $$ tail
return 1
fi
done
However, when we set the processes to print logging in DEBUG mode, they print so much logging that this script cannot keep up, and it takes about 15 minutes after the process is complete for the bash script to catch up. Is there a way of optimizing this, like changing 'while read line' to 'while read 100 lines', or something like that?
How about not forking up to two grep processes per log line?
tail -Fn +1 "$log_file" | grep -Ei "$search_text|^error\b" | while read line; do
So one long running grep process shall do preprocessing if you will.
Edit: As noted in the comments, it is safer to add --line-buffered to the grep invocation.
Some tips relevant for this script:
Checking that the service is doing its job is a much better check for daemon startup than looking at the log output
You can use grep ... <<<"$line" to execute fewer echos.
You can use tail -f | grep -q ... to avoid the while loop by stopping as soon as there's a matching line.
If you can avoid -i on grep it might be significantly faster to process the input.
Thou shalt not kill -9.
This question already has answers here:
How to get overall CPU usage (e.g. 57%) on Linux [closed]
(6 answers)
Closed 9 years ago.
I can't obtain CPU% usage of all the pid, without know any program names.
I feel I am close to the solution, this is what I've done so far:
for line in $(pgrep -f chrome); \
do echo -n $line" - "; \
ps -p $line -o %cpu | sed -n 2p | sed 's/ //'; done
In this example I obtain only all chrome pid.. in next step I want all executing pid.
You can do this easily with the top command alone.
To order by CPU percentage (descending), you could use top -o -cpu
If you don't want to use top for some reason, couple of other ways I can think of doing this.
> ps -e -o "%p-%C"
Or if you wanted to do it in a script, something like (alternatively could just parse ps again or check /proc/pid/stat for cpu usage)
#!/bin/bash
shopt -s extglob
for line in /proc/+([0-9]); do
echo -n "${line##*/}- "
ps -p "${line##*/}" -o %cpu | sed -n 2p | sed 's/ //'
done
Where
shopt -s extglob Turns on extended file globing in bash
+([0-9]) Matches any files containing 1 or more digits
${line##*/} Strips everything before and including the last / character
I'm using a 12 core 24 thread Linux machine to performing the following task. Each task is independent.
while read parameter
do
./program_a $parameter $parameter.log 2>&1 &
done < parameter_file
However this code will dispatch all the tasks at one time, which could read to serious context switch lack of idle cpu power and/or lack of memory.
I want to exploit system information tools such as free, top, and ps to determine if the task should be dispatched.
Using free.
while read parameter
do
#for instance using free
free_mem=`free -g | grep Mem | awk '{print $4}'`
if [ $free_mem -gt 10]; then
./program_a $parameter $parameter.log 2>&1 &
fi
done < parameter_file
But this won't work because this won't wait until the condition meet the criteria. How should I do this?
Besides how should I use top and ps to determine if the system is busy or not. I want to dispatch new task when the system is too busy.
Maybe I can use
ps aux | grep "program_a " | grep -v "grep" | wc -l
to limit the number of dispatched tasks. But it is a implicit way to determine if the system is busy or not. Any other thought?
while read parameter
do
#for instance using free
while 1; do
free_mem=`free -g | awk '/Mem/{print $4}'`
if (( $free_mem > 10 )); then
break
fi
sleep 1 # wait so that some tasks might finish
done
./program_a $parameter $parameter.log 2>&1 &
done < parameter_file
I've been tinkering with a regex answer by yukondude with little success. I'm trying to kill processes that are older than 10 minutes. I already know what the process IDs are. I'm looping over an array every 10 min to see if any lingering procs are around and need to be killed. Anybody have any quick thoughts on this?
ps -eo uid,pid,etime 3233332 | egrep ' ([0-9]+-)?([0-9]{2}:?){3}' | awk '{print $2}' | xargs -I{} kill {}
Just like real Zombies, Zombie processes can't be killed - they're already dead.
They will go away when their parent process calls wait() to get their exit code, or when their parent process exits.
Oh, you're not really talking about Zombie processes at all. This bash script should be along the lines of what you're after:
ps -eo uid,pid,lstart |
tail -n+2 |
while read PROC_UID PROC_PID PROC_LSTART; do
SECONDS=$[$(date +%s) - $(date -d"$PROC_LSTART" +%s)]
if [ $PROC_UID -eq 1000 -a $SECONDS -gt 600 ]; then
echo $PROC_PID
fi
done |
xargs kill
That will kill all processes owned by UID 1000 that have been running longer than 10 minutes (600 seconds). You probably want to filter it out to just the PIDs you're interested in - perhaps by parent process ID or similar? Anyway, that should be something to go on.