How to create a CPU spike with a bash command - linux

I want to create a near 100% load on a Linux machine. It's quad core system and I want all cores going full speed. Ideally, the CPU load would last a designated amount of time and then stop. I'm hoping there's some trick in bash. I'm thinking some sort of infinite loop.

I use stress for this kind of thing, you can tell it how many cores to max out.. it allows for stressing memory and disk as well.
Example to stress 2 cores for 60 seconds
stress --cpu 2 --timeout 60

You can also do
dd if=/dev/zero of=/dev/null
To run more of those to put load on more cores, try to fork it:
fulload() { dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null & }; fulload; read; killall dd
Repeat the command in the curly brackets as many times as the number of threads you want to produce (here 4 threads).
Simple enter hit will stop it (just make sure no other dd is running on this user or you kill it too).

I think this one is simpler. Open Terminal and type the following and press Enter.
yes > /dev/null &
To fully utilize modern CPUs, one line is not enough, you may need to repeat the command to exhaust all the CPU power.
To end all of this, simply put
killall yes
The idea was originally found here, although it was intended for Mac users, but this should work for *nix as well.

Although I'm late to the party, this post is among the top results in the google search "generate load in linux".
The result marked as solution could be used to generate a system load, i'm preferring to use sha1sum /dev/zero to impose a load on a cpu-core.
The idea is to calculate a hash sum from an infinite datastream (eg. /dev/zero, /dev/urandom, ...) this process will try to max out a cpu-core until the process is aborted.
To generate a load for more cores, multiple commands can be piped together.
eg. generate a 2 core load:
sha1sum /dev/zero | sha1sum /dev/zero

To load 3 cores for 5 seconds:
seq 3 | xargs -P0 -n1 timeout 5 yes > /dev/null
This results in high kernel (sys) load from the many write() system calls.
If you prefer mostly userland cpu load:
seq 3 | xargs -P0 -n1 timeout 5 md5sum /dev/zero
If you just want the load to continue until you press Ctrl-C:
seq 3 | xargs -P0 -n1 md5sum /dev/zero

One core (doesn't invoke external process):
while true; do true; done
Two cores:
while true; do /bin/true; done
The latter only makes both of mine go to ~50% though...
This one will make both go to 100%:
while true; do echo; done

Here is a program that you can download Here
Install easily on your Linux system
./configure
make
make install
and launch it in a simple command line
stress -c 40
to stress all your CPUs (however you have) with 40 threads each running a complex sqrt computation on a ramdomly generated numbers.
You can even define the timeout of the program
stress -c 40 -timeout 10s
unlike the proposed solution with the dd command, which deals essentially with IO and therefore doesn't really overload your system because working with data.
The stress program really overloads the system because dealing with computation.

An infinite loop is the idea I also had. A freaky-looking one is:
while :; do :; done
(: is the same as true, does nothing and exits with zero)
You can call that in a subshell and run in the background. Doing that $num_cores times should be enough. After sleeping the desired time you can kill them all, you get the PIDs with jobs -p (hint: xargs)

:(){ :|:& };:
This fork bomb will cause havoc to the CPU and will likely crash your computer.

I would split the thing in 2 scripts :
infinite_loop.bash :
#!/bin/bash
while [ 1 ] ; do
# Force some computation even if it is useless to actually work the CPU
echo $((13**99)) 1>/dev/null 2>&1
done
cpu_spike.bash :
#!/bin/bash
# Either use environment variables for NUM_CPU and DURATION, or define them here
for i in `seq ${NUM_CPU}` : do
# Put an infinite loop on each CPU
infinite_loop.bash &
done
# Wait DURATION seconds then stop the loops and quit
sleep ${DURATION}
killall infinite_loop.bash

to increase load or consume CPU 100% or X%
sha1sum /dev/zero &
on some system this will increase the load in slots of X%, in that case you have to run the same command multiple time.
then you can see CPU uses by typing command
top
to release the load
killall sha1sum

cat /dev/urandom > /dev/null

#!/bin/bash
duration=120 # seconds
instances=4 # cpus
endtime=$(($(date +%s) + $duration))
for ((i=0; i<instances; i++))
do
while (($(date +%s) < $endtime)); do :; done &
done

I've used bc (binary calculator), asking them for PI with a big lot of decimals.
$ for ((i=0;i<$NUMCPU;i++));do
echo 'scale=100000;pi=4*a(1);0' | bc -l &
done ;\
sleep 4; \
killall bc
with NUMCPU (under Linux):
$ NUMCPU=$(grep $'^processor\t*:' /proc/cpuinfo |wc -l)
This method is strong but seem system friendly, as I've never crashed a system using this.

#!/bin/bash
while [ 1 ]
do
#Your code goes here
done

I went through the Internet to find something like it and found this very handy cpu hammer script.
#!/bin/sh
# unixfoo.blogspot.com
if [ $1 ]; then
NUM_PROC=$1
else
NUM_PROC=10
fi
for i in `seq 0 $((NUM_PROC-1))`; do
awk 'BEGIN {for(i=0;i<10000;i++)for(j=0;j<10000;j++);}' &
done

Using examples mentioned here, but also help from IRC, I developed my own CPU stress testing script. It uses a subshell per thread and the endless loop technique. You can also specify the number of threads and the amount of time interactively.
#!/bin/bash
# Simple CPU stress test script
# Read the user's input
echo -n "Number of CPU threads to test: "
read cpu_threads
echo -n "Duration of the test (in seconds): "
read cpu_time
# Run an endless loop on each thread to generate 100% CPU
echo -e "\E[32mStressing ${cpu_threads} threads for ${cpu_time} seconds...\E[37m"
for i in $(seq ${cpu_threads}); do
let thread=${i}-1
(taskset -cp ${thread} $BASHPID; while true; do true; done) &
done
# Once the time runs out, kill all of the loops
sleep ${cpu_time}
echo -e "\E[32mStressing complete.\E[37m"
kill 0

Utilizing ideas here, created code which exits automatically after a set duration, don't have to kill processes --
#!/bin/bash
echo "Usage : ./killproc_ds.sh 6 60 (6 threads for 60 secs)"
# Define variables
NUM_PROCS=${1:-6} #How much scaling you want to do
duration=${2:-20} # seconds
function infinite_loop {
endtime=$(($(date +%s) + $duration))
while (($(date +%s) < $endtime)); do
#echo $(date +%s)
echo $((13**99)) 1>/dev/null 2>&1
$(dd if=/dev/urandom count=10000 status=none| bzip2 -9 >> /dev/null) 2>&1 >&/dev/null
done
echo "Done Stressing the system - for thread $1"
}
echo Running for duration $duration secs, spawning $NUM_PROCS threads in background
for i in `seq ${NUM_PROCS}` ;
do
# Put an infinite loop
infinite_loop $i &
done

You can try to test the performance of cryptographic algorithms.
openssl speed -multi 4

If you do not want to install additional software, you may use a compression utility which utilizes all CPU cores automatically. For example, xz:
cat /dev/zero | xz -T0 > /dev/null
This takes infinite stream of dummy data from /dev/zero and compresses it using all cores available in the system.

This does a trick for me:
bash -c 'for (( I=100000000000000000000 ; I>=0 ; I++ )) ; do echo $(( I+I*I )) & echo $(( I*I-I )) & echo $(( I-I*I*I )) & echo $(( I+I*I*I )) ; done' &>/dev/null
and it uses nothing except bash.

To enhance dimba's answer and provide something more pluggable (because i needed something similar). I have written the following using the dd load-up concept :D
It will check current cores, and create that many dd threads.
Start and End core load with Enter
#!/bin/bash
load_dd() {
dd if=/dev/zero of=/dev/null
}
fulload() {
unset LOAD_ME_UP_SCOTTY
export cores="$(grep proc /proc/cpuinfo -c)"
for i in $( seq 1 $( expr $cores - 1 ) )
do
export LOAD_ME_UP_SCOTTY="${LOAD_ME_UP_SCOTTY}$(echo 'load_dd | ')"
done
export LOAD_ME_UP_SCOTTY="${LOAD_ME_UP_SCOTTY}$(echo 'load_dd &')"
eval ${LOAD_ME_UP_SCOTTY}
}
echo press return to begin and stop fullload of cores
read
fulload
read
killall -9 dd

Dimba's dd if=/dev/zero of=/dev/null is definitely correct, but also worth mentioning is verifying maxing the cpu to 100% usage. You can do this with
ps -axro pcpu | awk '{sum+=$1} END {print sum}'
This asks for ps output of a 1-minute average of the cpu usage by each process, then sums them with awk. While it's a 1 minute average, ps is smart enough to know if a process has only been around a few seconds and adjusts the time-window accordingly. Thus you can use this command to immediately see the result.

awk is a good way to write a long-running loop that's CPU bound without generating a lot of memory traffic or system calls, or using any significant amount of memory or polluting caches so it slows down other cores a minimal amount. (stress or stress-ng can also do that if you either installed, if you use a simple CPU-stress method.)
awk 'BEGIN{for(i=0;i<100000000;i++){}}' # about 3 seconds on 4GHz Skylake
It's a counted loop so you can make it exit on its own after a finite amount of time. (Awk uses FP numbers, so a limit like 2^54 might not be reachable with i++ due to rounding, but that's way larger than needed for a few seconds to minutes.)
To run it in parallel, use a shell loop to start it in the background n times
for i in {1..6};do awk 'BEGIN{for(i=0;i<100000000;i++){}}' & done
###### 6 threads each running about 3 seconds
$ for i in {1..6};do awk 'BEGIN{for(i=0;i<100000000;i++){}}' & done
[1] 3047561
[2] 3047562
[3] 3047563
[4] 3047564
[5] 3047565
[6] 3047566
$ # this shell is usable.
(wait a while before pressing return)
[1] Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
[2] Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
[3] Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
[4] Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
[5]- Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
[6]+ Done awk 'BEGIN{for(i=0;i<100000000;i++){}}'
$
I used perf to see what kind of load it put on the CPU: it runs 2.6 instructions per clock cycle, so it's not the most friendly to a hyperthread sharing the same physical core. But it has a very small cache footprint, getting negligible cache misses even in L1d cache. And strace will show it makes no system calls until exit.
$ perf stat -r5 -d awk 'BEGIN{for(i=0;i<100000000;i++){}}'
Performance counter stats for 'awk BEGIN{for(i=0;i<100000000;i++){}}' (5 runs):
3,277.56 msec task-clock # 0.997 CPUs utilized ( +- 0.24% )
7 context-switches # 2.130 /sec ( +- 12.29% )
1 cpu-migrations # 0.304 /sec ( +- 40.00% )
180 page-faults # 54.765 /sec ( +- 0.18% )
13,708,412,234 cycles # 4.171 GHz ( +- 0.18% ) (62.29%)
35,786,486,833 instructions # 2.61 insn per cycle ( +- 0.03% ) (74.92%)
9,696,339,695 branches # 2.950 G/sec ( +- 0.02% ) (74.99%)
340,155 branch-misses # 0.00% of all branches ( +-122.42% ) (75.08%)
12,108,293,527 L1-dcache-loads # 3.684 G/sec ( +- 0.04% ) (75.10%)
217,064 L1-dcache-load-misses # 0.00% of all L1-dcache accesses ( +- 17.23% ) (75.10%)
48,695 LLC-loads # 14.816 K/sec ( +- 31.69% ) (49.90%)
5,966 LLC-load-misses # 13.45% of all LL-cache accesses ( +- 31.45% ) (49.81%)
3.28711 +- 0.00772 seconds time elapsed ( +- 0.23% )
The most "friendly" to the other hyperthread on an x86 CPU would be a C program like this, which just runs a pause instruction in a loop. (Or portably, a Rust program that runs std::hint::spin_loop.) As far as the OS's process scheduler, it stays in user-space (nothing like a yield() system call), but in hardware it doesn't take up many resources, letting the other logical core have the front-end for multiple cycles.
#include <immintrin.h>
int main(){ // use atoi(argv[1])*10000ULL as a loop count if you want.
while(1) _mm_pause();
}

I combined some of the answers and added a way to scale the stress to all available cpus:
#!/bin/bash
function infinite_loop {
while [ 1 ] ; do
# Force some computation even if it is useless to actually work the CPU
echo $((13**99)) 1>/dev/null 2>&1
done
}
# Either use environment variables for DURATION, or define them here
NUM_CPU=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu)
PIDS=()
for i in `seq ${NUM_CPU}` ;
do
# Put an infinite loop on each CPU
infinite_loop &
PIDS+=("$!")
done
# Wait DURATION seconds then stop the loops and quit
sleep ${DURATION}
# Parent kills its children
for pid in "${PIDS[#]}"
do
kill $pid
done

Just paste this bad boy into the SSH or console of any server running linux. You can kill the processes manually, but I just shutdown the server when I'm done, quicker.
Edit: I have updated this script to now have a timer feature so that there is no need to kill the processes.
read -p "Please enter the number of minutes for test >" MINTEST && [[ $MINTEST == ?(-)+([0-9]) ]]; NCPU="$(grep -c ^processor /proc/cpuinfo)"; ((endtime=$(date +%s) + ($MINTEST*60))); NCPU=$((NCPU-1)); for ((i=1; i<=$NCPU; i++)); do while (($(date +%s) < $endtime)); do : ; done & done

Related

Bash script: CPU stress test while watching clock speed

I am totally new to this forum and also new to bash, so please bear with me :).
I would like to write a bash script to conduct a CPU stress test while observing the clock speed. Therefore, I have done the following:
1.) For the CPU stress test, I have created a script named "bernoulli" with the following code:
#!/bin/bash
# argument 1: n
function bernoulli()
{
if (( $1 < 3 ))
then
echo 1
else
echo $(( $(bernoulli $(( $1 - 1 ))) + $(bernoulli $(( $1 - 2 ))) ))
fi
}
bernoulli $1
2.) I have figured out that by using the "timeout" command I can kill a task after a specified time. For example,
timeout 30s ./bernoulli 35
starts a task calculating the 35th bernoulli number and the task is killed after 30 seconds.
3.) I also found out that by typing
timeout 30s watch grep \"cpu MHz\" /proc/cpuinfo
I can watch the clock speed of my cores (updated every 2 seconds) for 30 seconds (at which point "timeout 30s" kills this task).
What I want: I would like to do the above stress test and simultaneously observe the clock speed. In other words, I would somehow run the two commands
timeout 30s ./bernoulli 35
timeout 30s watch grep \"cpu MHz\" /proc/cpuinfo
"at the same time". I hope I could make it clear what I would like to achieve. Can anyone help with my issue? Thanks a lot for every comment!
How about
timeout 30s ./bernoulli 35 &
timeout 30s watch grep \"cpu MHz\" /proc/cpuinfo
& at the end will make command to run at background, so that second timeout will be executed almost instantly after the first one.
PS: this is rather poor way to test modern CPU. You will be exercising only single core and most likely only limited part of your CPU (no sse, etc). It is not trivial to write CPU benchmark, so you might want to use one of already available. For example, you can take a look at sysbench with something like sysbench --test=cpu --cpu-max-prime=20000 run.
You can run them in a dedicated shell:
timeout 30s bash -c './bernoulli 35 & watch grep \"cpu MHz\" /proc/cpuinfo'
Note that the single & is not a typo. It is not a logical and, it runs the bernoulli script in background.

How do I determine the slowest component of my shell pipeline?

I have an extremely long and complicated shell pipeline set up to grab 2.2Gb of data and process it. It currently takes 45 minutes to process. The pipeline is a number of cut, grep, sort, uniq, grep and awk commands tied together. I have my suspicion that it's the grep portion that is causing it to take so much time but I have no way of confirming it.
Is there anyway to "profile" the entire pipeline from end to end to determine which component is the slowest and if it is CPU or IO bound so it can be optimised?
I cannot post the entire command here unfortunately as it would require posting proprietary information but I suspect it is the following bit checking it out with htop:
grep -v ^[0-9]
One way to do this is to gradually build up the pipeline, timing each addition, and taking as much out of the equation as possible (such as outputting to a terminal or file). A very simple example is shown below:
pax:~$ time ( cat bigfile >/dev/null )
real 0m4.364s
user 0m0.004s
sys 0m0.300s
pax:~$ time ( cat bigfile | tr 'a' 'b' >/dev/null )
real 0m0.446s
user 0m0.312s
sys 0m0.428s
pax:~$ time ( cat bigfile | tr 'a' 'b' | tail -1000l >/dev/null )
real 0m0.796s
user 0m0.516s
sys 0m0.688s
pax:~$ time ( cat bigfile | tr 'a' 'b' | tail -1000l | sort -u >/dev/null )
real 0m0.892s
user 0m0.556s
sys 0m0.756s
If you add up the user and system times above, you'll see that the incremental increases are:
0.304 (0.004 + 0.300) seconds for the cat;
0.436 (0.312 + 0.428 - 0.304) seconds for the tr;
0.464 (0.516 + 0.688 - 0.436 - 0.304) seconds for the tail; and
0.108 (0.556 + 0.756 - 0.464 - 0.436 - 0.304) seconds for the sort.
This tells me that the main things to look into are the tail and the tr.
Now obviously, that's for CPU only, and I probably should have done multiple runs at each stage for averaging purposes, but that's the basic first approach I would take.
If it turns out it really is your grep, there are a few other options available to you. There are numerous other commands that can strip lines not starting with a digit but you may find that a custom-built command for doing this may be faster still, pseudo-code like (untested, but you should get the idea):
state = echo
lastchar = newline
while not end of file:
read big chunk from file
for every char in chunk:
if lastchar is newline:
if state is echo and char is non-digit:
state = skip
else if state is skip and and char is digit:
state = echo
if state is echo:
output char
lastchar = char
Custom, targeted code like this can sometimes be made more efficient than a general-purpose regex processing engine, simply because it can be optimised to the specific case. Whether that's true is this case, or any case for that matter, is something you should test. My number one optimisation mantra is measure, don't guess!
I found the problem myself after some further experimentation. It appears to be due to the encoding support in grep. Using the following hung the pipeline:
grep -v ^[0-9]
I replaced it with sed as follows and it finished in under 45 seconds!
sed '/^[0-9]/d'
This is straightforward with zsh:
zsh-4.3.12[sysadmin]% time sleep 3 | sleep 5 | sleep 2
sleep 3 0.01s user 0.03s system 1% cpu 3.182 total
sleep 5 0.01s user 0.01s system 0% cpu 5.105 total
sleep 2 0.00s user 0.05s system 2% cpu 2.121 total

Scons command with time limit

I want to limit the execution time of a program I am running under Linux. I put in my scons script a line like:
Command("com​","",​"ulimit -t 1; myprogram")
and tested it with an infinite loop program: it did not work and the program ran forever.
Am I missing something?
-- tsf
ulimit -t 1 means that the limit is set to 1 second of CPU time. If your infinite loop program uses any sort of sleep in its inner loop then it will use practically no CPU time. This means it will not get killed in 1 second of real, on the clock time. In fact it may take minutes or hours to use up its 1 second allocation.
What happens if you run the command outside of SCons? Perhaps you don't have permission to change the limit at all...
ulimit -t 1; ./myprogram
For example, it may say the following if the limit is already set to 0:
bash: ulimit: cpu time: cannot modify limit: Operation not permitted
Edit: it seems that the -t option is broken on Ubuntu 9.04. A fix has been committed 05 June 2009, but it may take a while to trickle into the updates - it may not be fixed until 9.10.
As an historical note, this problem no longer exists in Ubuntu 10.04.
You can also use this script:
(taken from http://newsgroups.derkeiler.com/Archive/Comp/comp.sys.mac.system/2005-12/msg00247.html)
#!/bin/sh
# timeout script
#
usage()
{
echo "usage: timeout seconds command args ..."
exit 1
}
[[ $# -lt 2 ]] && usage
seconds=$1; shift
timeout()
{
sleep $seconds
kill -9 $pid >/dev/null 2>/dev/null
}
eval "$#" &
pid=$!
timeout &
wait $pid
.

Get program execution time in the shell

I want to execute something in a linux shell under a few different conditions, and be able to output the execution time of each execution.
I know I could write a perl or python script that would do this, but is there a way I can do it in the shell? (which happens to be bash)
Use the built-in time keyword:
$ help time
time: time [-p] PIPELINE
Execute PIPELINE and print a summary of the real time, user CPU time,
and system CPU time spent executing PIPELINE when it terminates.
The return status is the return status of PIPELINE. The `-p' option
prints the timing summary in a slightly different format. This uses
the value of the TIMEFORMAT variable as the output format.
Example:
$ time sleep 2
real 0m2.009s
user 0m0.000s
sys 0m0.004s
You can get much more detailed information than the bash built-in time (i.e time(1), which Robert Gamble mentions). Normally this is /usr/bin/time.
Editor's note:
To ensure that you're invoking the external utility time rather than your shell's time keyword, invoke it as /usr/bin/time.
time is a POSIX-mandated utility, but the only option it is required to support is -p.
Specific platforms implement specific, nonstandard extensions: -v works with GNU's time utility, as demonstrated below (the question is tagged linux); the BSD/macOS implementation uses -l to produce similar output - see man 1 time.
Example of verbose output:
$ /usr/bin/time -v sleep 1
Command being timed: "sleep 1"
User time (seconds): 0.00
System time (seconds): 0.00
Percent of CPU this job got: 1%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 210
Voluntary context switches: 2
Involuntary context switches: 1
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
#!/bin/bash
START=$(date +%s)
# do something
# start your script work here
ls -R /etc > /tmp/x
rm -f /tmp/x
# your logic ends here
END=$(date +%s)
DIFF=$(( $END - $START ))
echo "It took $DIFF seconds"
For a line-by-line delta measurement, try gnomon.
$ npm install -g gnomon
$ <your command> | gnomon --medium=1.0 --high=4.0 --ignore-blank --real-time=100
A command line utility, a bit like moreutils's ts, to prepend timestamp information to the standard output of another command. Useful for long-running processes where you'd like a historical record of what's taking so long.
You can also use the --high and/or --medium options to specify a length threshold in seconds, over which gnomon will highlight the timestamp in red or yellow. And you can do a few other things, too.
Should you want more precision, use %N with date (and use bc for the diff, because $(()) only handles integers).
Here's how to do it:
start=$(date +%s.%N)
# do some stuff here
dur=$(echo "$(date +%s.%N) - $start" | bc)
printf "Execution time: %.6f seconds" $dur
Example:
start=$(date +%s.%N); \
sleep 0.1s; \
dur=$(echo "$(date +%s.%N) - $start" | bc); \
printf "Execution time: %.6f seconds\n" $dur
Result:
Execution time: 0.104623 seconds
If you intend to use the times later to compute with, learn how to use the -f option of /usr/bin/time to output code that saves times. Here's some code I used recently to get and sort the execution times of a whole classful of students' programs:
fmt="run { date = '$(date)', user = '$who', test = '$test', host = '$(hostname)', times = { user = %U, system = %S, elapsed = %e } }"
/usr/bin/time -f "$fmt" -o $timefile command args...
I later concatenated all the $timefile files and pipe the output into a Lua interpreter. You can do the same with Python or bash or whatever your favorite syntax is. I love this technique.
If you only need precision to the second, you can use the builtin $SECONDS variable, which counts the number of seconds that the shell has been running.
while true; do
start=$SECONDS
some_long_running_command
duration=$(( SECONDS - start ))
echo "This run took $duration seconds"
if some_condition; then break; fi
done
You can use time and subshell ():
time (
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
)
Or in same shell {}:
time {
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
}
The way is
$ > g++ -lpthread perform.c -o per
$ > time ./per
output is >>
real 0m0.014s
user 0m0.010s
sys 0m0.002s
one possibly simple method ( that may not meet different users needs ) is the use of shell PROMPT.it is a simple solution that can be useful in some cases. You can use the bash prompting feature as in the example below:
export PS1='[\t \u#\h]\$'
The above command will result in changing the shell prompt to :
[HH:MM:SS username#hostname]$
Each time you run a command (or hit enter) returning back to the shell prompt, the prompt will display current time.
notes:
1) beware that if you waited for sometime before you type your next command, then this time need to be considered, i.e the time displayed in the shell prompt is the timestamp when the shell prompt was displayed, not when you enter command. some users choose to hit Enter key to get a new prompt with a new timestamp before they are ready for the next command.
2) There are other available options and modifiers that can be used to change the bash prompt, refer to ( man bash ) for more details.
perf stat Linux CLI utility
This tool is overkill for just getting time. But it can do so much more for you to help profile and fix slowness that it is worth knowing about. Ubuntu 22.04 setup:
sudo apt install linux-tools-common linux-tools-generic
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
Usage:
perf stat <mycmd>
Sample run with stress-ng:
perf stat stress-ng --cpu 1 --cpu-method matrixprod -t 5
Sample output:
Performance counter stats for 'stress-ng --cpu 1 --cpu-method matrixprod -t 5':
5,005.46 msec task-clock # 0.999 CPUs utilized
88 context-switches # 17.581 /sec
1 cpu-migrations # 0.200 /sec
1,188 page-faults # 237.341 /sec
18,847,667,167 cycles # 3.765 GHz
26,544,261,897 instructions # 1.41 insn per cycle
3,239,655,001 branches # 647.225 M/sec
25,393,369 branch-misses # 0.78% of all branches
5.012218939 seconds time elapsed
4.998051000 seconds user
0.009122000 seconds sys
perf can also do a bunch more advanced things, e.g. here I show how to use it to profile code: How do I profile C++ code running on Linux?

Limiting certain processes to CPU % - Linux

I have the following problem: some processes, generated dynamically, have a tendency to eat 100% of CPU. I would like to limit all the process matching some criterion (e.g. process name) to a certain amount of CPU percentage.
The specific problem I'm trying to solve is harnessing folding#home worker processes. The best solution I could think of is a perl script that's executed periodically and uses the cpulimit utility to limit the processes (if you're interested in more details, check this blog post). It works, but it's a hack :/
Any ideas? I would like to leave the handling of processes to the OS :)
Thanks again for the suggestions, but we're still missing the point :)
The "slowDown" solution is essentially what the "cpulimit" utility does. I still have to take care about what processes to slow down, kill the "slowDown" process once the worker process is finished and start new ones for new worker processes. It's precisely what I did with the Perl script and a cron job.
The main problem is that I don't know beforehand what processes to limit. They are generated dynamically.
Maybe there's a way to limit all the processes of one user to a certain amount of CPU percentage? I already set up a user for executing the folding#home jobs, hoping that i could limit him with the /etc/security/limits.conf file. But the nearest I could get there is the total CPU time per user...
It would be cool if to have something that enables you to say:
"The sum of all CPU % usage of this user's processes cannot exceed 50%". And then let the processes fight for that 50% of CPU regarding to their priorities...
Guys, thanks for your suggestions, but it's not about priorities - I want to limit the CPU % even when there's plenty of CPU time available. The processes are already low priority, so they don't cause any performance issues.
I would just like to prevent the CPU from running on 100% for extended periods...
I had a slightly similar issue with gzip.
Assuming we want to decrease the CPU of a gzip process:
gzip backup.tar & sleep 2 & cpulimit --limit 10 -e gzip -z
Options:
I found sleep useful as the cpulimit sometimes didn't pick up the new gzip process immediately
--limit 10 limits gzip CPU usage to 10%
-z automatically closes cpulimit when gzip process finishes
Another option is to run the cpulimit daemon.
I don't remember and dont think there was something like this in the unix scheduler. You need a little program which controls the other process and does the following:
loop
wait for some time tR
send SIGSTOP to the process you want to be scheduled
wait for some time tP
send SIGCONT to the process.
loopEnd
the ratio tR/tP controls the cpu load.
Here is a little proof of concept. "busy" is the program which uses up your cpu time and which you want to be slowed-down by "slowDown":
> cat > busy.c:
main() { while (1) {}; }
> cc -o busy busy.c
> busy &
> top
Tasks: 192 total, 3 running, 189 sleeping, 0 stopped, 0 zombie
Cpu(s): 76.9% us, 6.6% sy, 0.0% ni, 11.9% id, 4.5% wa, 0.0% hi, 0.0% si
Mem: 6139696k total, 6114488k used, 25208k free, 115760k buffers
Swap: 9765368k total, 1606096k used, 8159272k free, 2620712k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26539 cg 25 0 2416 292 220 R 90.0 0.0 3:25.79 busy
...
> cat > slowDown
while true; do
kill -s SIGSTOP $1
sleep 0.1
kill -s SIGCONT $1
sleep 0.1
done
> chmod +x slowDown
> slowDown 26539 &
> top
Tasks: 200 total, 4 running, 192 sleeping, 4 stopped, 0 zombie
Cpu(s): 48.5% us, 19.4% sy, 0.0% ni, 20.2% id, 9.8% wa, 0.2% hi, 2.0% si
Mem: 6139696k total, 6115376k used, 24320k free, 96676k buffers
Swap: 9765368k total, 1606096k used, 8159272k free, 2639796k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26539 cg 16 0 2416 292 220 T 49.7 0.0 6:00.98 busy
...
ok, that script needs some more work (for example, to care for being INTR-upted and let the controlled process continue in case it was stopped at that moment), but you get the point. I would also write that little script in C or similar and compute the cpu ratio from a comand line argument....
regards
I think in Linux there is no solution to cap the cpu usage, but there is an acceptable way to limit any process to a certain CPU usage: http://ubuntuforums.org/showthread.php?t=992706
In case they remove the info, here is again
INSTALL PACKAGES
Install cpulimit package.
Code:
sudo apt-get install cpulimit
Install gawk package.
Code:
sudo apt-get install gawk
CREATE CPULIMIT DAEMON FILE
Open text editor with root privileges and save bellow daemon script text to new file /usr/bin/cpulimit_daemon.sh
Code:
#!/bin/bash
# ==============================================================
# CPU limit daemon - set PID's max. percentage CPU consumptions
# ==============================================================
# Variables
CPU_LIMIT=20 # Maximum percentage CPU consumption by each PID
DAEMON_INTERVAL=3 # Daemon check interval in seconds
BLACK_PROCESSES_LIST= # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
WHITE_PROCESSES_LIST= # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.
# Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
if [[ -n "$BLACK_PROCESSES_LIST" && -n "$WHITE_PROCESSES_LIST" ]] ; then # If both variables are defined then error is produced.
echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty."
exit 1
elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
else
NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
fi
# Search and limit violating PIDs
while sleep $DAEMON_INTERVAL
do
NEW_PIDS=$(eval "$NEW_PIDS_COMMAND") # Violating PIDs
LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}') # Already limited PIDs
QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$') # PIDs in queue
for i in $QUEUE_PIDS
do
cpulimit -p $i -l $CPU_LIMIT -z & # Limit new violating processes
done
done
CHANGE VARIABLES TO YOUR ENVIRONMENT NEEDS
CPU_LIMIT
Change this variable in above script if you would like to omit CPU consumption for every process to any other percentage then 20%. Please read "If using SMP computer" chapter bellow if you have SMP computer (more then 1 CPU or CPU with more then 1 core).
DAEMON_INTERVAL
Change this variable in above script if you would like to have more/less regular checking. Interval is in seconds and default is set to 3 seconds.
BLACK_PROCESS_LIST and WHITE_PROCESSES_LIST
Variable BLACK_PROCESSES_LIST limits only specified processes. If variable is empty (default) all violating processes are limited.
Variable WHITE_PROCESSES_LIST limits all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.
One or both of the variables BLACK_PROCESSES_LIST and WHITE_PROCESSES_LIST has to be empty - it is not logical that both variables are defined.
You can specify multiple processes in one of this two variables using delimiter characters "|" (without double quotes). Sample: if you would like to cpulimit all processes except mysql, firefox and gedit processes set variable: WHITE_PROCESSES_LIST="mysql|firefox|gedit"
PROCEDURE TO AUTOMATICALLY START DAEMON AT BOOT TIME
Set file permissions for root user:
Code:
sudo chmod 755 /usr/bin/cpulimit_daemon.sh
Open text editor with root privileges and save bellow script to new file /etc/init.d/cpulimit
Code:
#!/bin/sh
#
# Script to start CPU limit daemon
#
set -e
case "$1" in
start)
if [ $(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l) -eq 0 ]; then
nohup /usr/bin/cpulimit_daemon.sh >/dev/null 2>&1 &
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon started successfully"; else print " * cpulimit daemon can not be started" }'
else
echo " * cpulimit daemon can't be started, because it is already running"
fi
;;
stop)
CPULIMIT_DAEMON=$(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l)
CPULIMIT_INSTANCE=$(ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | wc -l)
CPULIMIT_ALL=$((CPULIMIT_DAEMON + CPULIMIT_INSTANCE))
if [ $CPULIMIT_ALL -gt 0 ]; then
if [ $CPULIMIT_DAEMON -gt 0 ]; then
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | xargs kill -9 # kill cpulimit daemon
fi
if [ $CPULIMIT_INSTANCE -gt 0 ]; then
ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | xargs kill -9 # release cpulimited process to normal priority
fi
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon can not be stopped"; else print " * cpulimit daemon stopped successfully" }'
else
echo " * cpulimit daemon can't be stopped, because it is not running"
fi
;;
restart)
$0 stop
sleep 3
$0 start
;;
status)
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon is running"; else print " * cpulimit daemon is not running" }'
;;
esac
exit 0
Change file's owner to root:
Code:
sudo chown root:root /etc/init.d/cpulimit
Change permissions:
Code:
sudo chmod 755 /etc/init.d/cpulimit
Add script to boot-up procedure directories:
Code:
sudo update-rc.d cpulimit defaults
Reboot to check if script starts cpulimit daemon at boot time:
Code:
sudo reboot
MANUALLY CHECK, STOP, START AND RESTART DAEMON
Note: Daemon and service in this tutorial has equal meaning.
Note: For users using prior to Ubuntu 8.10 (like Ubuntu 8.04 LTS) instead of service command use "sudo /etc/init.d/cpulimit status/start/stop/restart" syntax or install sysvconfig package using command: sudo apt-get install sysvconfig
Check if cpulimit service is running
Check command returns: "cpulimit daemon is running" if service is running, or "cpulimit daemon is not running" if service is not running.
Code:
sudo service cpulimit status
Start cpulimit service
You can manually start cpulimit daemon which will start to omit CPU consumption.
Code:
sudo service cpulimit start
Stop cpulimit service
Stop command stops cpulimit daemon (so no new process will be limited) and also sets to all existing limited processes to have full access to CPU, just like it was before cpulimit was not running.
Code:
sudo service cpulimit stop
Restart cpulimit service
If you change some variables settings in /usr/bin/cpulimit_daemon.sh like CPU_LIMIT, DAEMON_INTERVAL, BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST, then after changing settings you must restart service.
Code:
sudo service cpulimit restart
CHECK CPU CONSUMPTION WITH OR WITHOUT CPULIMIT DAEMON
Without daemon
1. stop cpulimit daemon (sudo service cpulimit stop)
2. execute CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU is probably more then 20% for each process.
With daemon turned on
1. start cpulimit daemon (sudo service cpulimit start)
2. execute the same CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU should be maximum 20% for each process.
Note: Don't forget at beginning %CPU can be more then 20%, because daemon has to catch violating process in interval of 3 seconds (set in script by default)
IF USING SMP COMPUTER
I have tested this code on Intel dual-core CPU computer - that behaves like SMP computer. Don't forget that top command and also cpulimit by default behaves in Irix mode, where 20% means 20% of one CPU. If there are two CPUs (or dual-core) then total %CPU can be 200%. In top command Irix mode can be turned off with command I (pressing +i when top command is running) and Solaris mode is turned on, where total amount of CPU is divided by number of CPUs, so %CPU can be no more then 100% on any number of CPU computer. Please read more info about top command in top man page (search for I command). Please also read more about how cpulimit is operating on SMP computer in cpulimit official page.
But how does cpulimit daemon operates on SMP computer? Always in Irix mode. So if you would like to spend 20% of CPU power on 2-CPU computer then 40% should be used for CPU_LIMIT variable in cpulimit daemon script.
UNINSTALL CPULIMIT DAEMON AND CPULIMIT PROGRAM
If you would like to get rid of cpulimit daemon you can clean up your system by removing cpulimit daemon and uninstalling cpulimit program.
Stop cpulimit daemon
Code:
sudo service cpulimit stop # Stop cpulimit daemon and all cpulimited processes
Remove daemon from boot-up procedure
Code:
sudo update-rc.d -f cpulimit remove # Remove symbolic links
Delete boot-up procedure
Code:
sudo rm /etc/init.d/cpulimit # Delete cpulimit boot-up script
Delete cpulimit daemon
Code:
sudo rm /usr/bin/cpulimit_daemon.sh # Delete cpulimit daemon script
Uninstall cpulimit program
Code:
sudo apt-get remove cpulimit
Uninstall gawk program
If you don't need this program for any other script, you can remote it.
Code:
sudo apt-get remove gawk
NOTE ABOUT AUTHORS
I have just written daemon for cpulimit (bash scripts above). I am not the author of cpulimit project. If you need more info about cpulimit program, please read official cpulimit web page: http://cpulimit.sourceforge.net/.
Regards,
Abcuser
I had a similar problem, and the other solutions presented in the thread don't address it at all. My solution works for me right now, but it is suboptimal, particularly for the cases where the process is owned by root.
My workaround for now is to try very hard to make sure that I don't have any long-running processes owned by root (like have backup be done only as a user)
I just installed the hardware sensors applet for gnome, and set up alarms for high and low temperatures on the CPU, and then set up the following commands for each alarm:
low:
mv /tmp/hogs.txt /tmp/hogs.txt.$$ && cat /tmp/hogs.txt.$$ | xargs -n1 kill -CONT
high:
touch /tmp/hogs.txt && ps -eo pcpu,pid | sort -n -r | head -1 | gawk '{ print $2 }' >> /tmp/hogs.txt && xargs -n1 kill -STOP < /tmp/hogs.txt
The good news is that my computer no longer overheats and crashes. The downside is that terminal processes get disconnected from the terminal when they get stopped, and don't get reconnected when they get the CONT signal. The other thing is that if it was an interactive program that caused the overheating (like a certain web browser plugin!) then it will freeze in the middle of what I'm doing while it waits for the CPU to cool off. It would be nicer to have CPU scaling take care of this at a global level, but the problem is that I only have two selectable settings on my CPU and the slow setting isn't slow enough to prevent overheating.
Just to re-iterate here, this has nothing at all to do with process priority, re-nicing,and obviously nothing to do with stopping jobs that run for a long time. This has to do with preventing CPU utilization from staying at 100% for too long, because the hardware is unable to dissipate the heat quickly enough when running at full capacity (idle CPU generates less heat than a fully loaded CPU).
Some other obvious possibilities that might help are:
Lower the CPU speed overall in the BIOS
Replace the heatsink or re-apply the thermal gel to see if that helps
Clean the heatsink with some compressed air
Replace the CPU fan
[edit]
Note: no more overheating at 100% CPU when I disable variable fan speed in the bios (asus p5q pro turbo). With the CPU fully loaded, each core tops out at 49 celcius.
Using cgroups' cpu.shares does nothing that a nice value wouldn't do. It sounds like you want to actually throttle the processes, which can definitely be done.
You will need to use a script or two, and/or edit /etc/cgconfig.conf to define the parameters you want.
Specifically, you want to edit the values cpu.cfs_period_us and cpu.cfs_quota_us. The process will then be allowed to run for cpu.cfs_quota_us microseconds per cpu.cfs_period_us microseconds.
For example:
If cpu.cfs_period_us = 50000 and cpu.cfs_quota_us = 10000 then the process will receive 20% of the CPU time maximum, no matter what else is going on.
In this screenshot I have given the process 2% of CPU time:
As far as the process is concerned it is running at 100%.
Settings cpu.shares on the other hand can and will still use 100% of the idle CPU time.
In this similar example I have given the process cpu.shares = 100 (of 1024):
As you can see the process is still consuming all the idle CPU time.
References:
http://manpages.ubuntu.com/manpages/precise/man5/cgconfig.conf.5.html
http://kennystechtalk.blogspot.co.uk/2015/04/throttling-cpu-usage-with-linux-cgroups.html
In a system that uses systemd, you can run a single process with a maximum CPU usage using a command like the following:
sudo systemd-run --scope --uid=1000 -p CPUQuota=20% my_heavy_computation.sh
This will use up to 20% of a single CPU core (or CPU thread if using hyper-threading).
I also had the need to limit CPU time for certain processes. Cpulimit is a good tool, but I always had to manually find out the PID and manually start cpulimit, so I wanted something more convenient.
I came up with this bash script:
#!/bin/bash
function lp
{
ps aux | grep $1 | termsql "select COL1 from tbl" > /tmp/tmpfile
while read line
do
TEST=`ps aux | grep "cpulimit -p $line" | wc -l`
[[ $TEST -eq "2" ]] && continue #cpulimit is already running on this process
cpulimit -p $line -l $2
done < /tmp/tmpfile
}
while true
do
lp gnome-terminal 5
lp system-journal 5
sleep 10
done
This example limits the cpu time of each gnome-terminal instance to 5% and the cpu time of each system-journal instance to 5% as well.
I used another script I created named termsql in this example to extract the PID . You can get it here:
https://gitorious.org/termsql/termsql/source/master:
You can limit the amount of cpu time with cgroups. The OS will handle resource management just like you ask for in you question.
Here is a wiki with examples: https://wiki.archlinux.org/index.php/Cgroups
Since the processes already run as a separate user limiting all process from that user should be the easiest solution.
I see at least two options:
Use "ulimit -t" in the shell that creates your process
Use "nice" at process creation or "renice" during runtime
PS + GREP + NICE
The nice command will probably help.
This can be done using setrlimit(2) (specifically by setting RLIMIT_CPU parameter).
Throwing some sleep calls in there should force the process off the CPU for a certain time. If you sleep 30 seconds once a minute, your process shouldn't average more than 50% CPU usage during that minute.
I dont really see why you want to limit the CPU time... you should limit the total load on that machine, and the load is determined by IO operations mostly .
Ex: if i create a while(1){} loop, it will get the total load to 1.0, but if this loop does some disk writes the load jumps to 2.0... 4.0. And that's what killing your machine, not the CPU usage. The CPU usage can be easily handled by nice/renice.
Anyways, you could make a script that does a 'kill -SIGSTOP PID' for a specific PID, when the load gets too high, and kill -SIGCONT when everything gets back to normal... The PID's can be determined by using the 'px aux' command, because i see that it displays the CPU usage, and you should be able to sort the list using that column. I think this the whole thing could be done in bash...
You could scale down the CPU frequency. Then you don't have to worry about the individual processes. When you need more cpu's, scale the frequency back up.

Resources