I have a bash script where I would like to get know how much percentage of CPU time a process uses until I kill it in the last line of the script.
I know that I could normally do this via the time -v command, however my process is Erlang/OTP based and by using the time command it only measures the startup process statistics.
Therefore I'd like to use the process PID I can get easily and use that one to get the CPU time percentage until the end of the script.
Currently I'm using pidstat but it is only giving me statistics for linear time intervals.
I want to measure the exact timeinterval from when the process started until it gets killed.
Peak RAM statistics woul also be nice.
Could you recommend me any command I could use in this case?
This is my bash script:
sudo emqx start
sleep 10
mypid=$(sudo emqx pid)
echo $mypid
sudo pidstat -h -r -u -v -p "$mypid" 5 > $local/server_results/test1/emqxstats_$b.txt &
# process for load testing
# jthreads = amount of publishing users
sleep 5
until sudo ~/Downloads/apache-jmeter-5.2.1/bin/jmeter -n -t $local/testplans/csv.jmx -Jport=$a | grep -m 1 "... end of run"; do : ; done
sudo emqx stop
kill %!
So I want to measure the CPU percentage from the interval between starting mosquitto and until the Apache Jmeter test finished when it reaches the last 2 lines.
Kind regards
I found the command I was looking after some more research.
ps -o pcpu -p $pid
this was exactly the command I needed, because it calculates the percentage over the lifetime of the process.
How can I use a linux command to get the wall time in seconds spent for executing a program. In the example below,I expected to get "0.005".
$ time ls >/dev/null
real 0m0.005s
user 0m0.001s
sys 0m0.003s
Depending on your path:
/usr/bin/time -f "%e"
The normal time is given by bash's (if you happen to use bash) intern time command
type time
while you need the one,
which time
will find.
So in context of your command:
/usr/bin/time -f "%e" ls > /dev/null
But to store it in a variable, you can't use
a=$(/usr/bin/time -f "%e" ls > /dev/null)
because the output of time is written to the error stream, to not inflict with the programs output (in this example ls). See the manpage of time for further details.
I tried to measure execution time and format it with this command:
time -f "%e" ./1 1000 1
-f: command not found
real 0m0.066s
user 0m0.044s
sys 0m0.023s
But such command works:
/usr/bin/time -f "%e" ./1 1000 1
31245 212 443
0.00
I tried to determine where another time is located, but all showes to /usr/bin/time
which time
/usr/bin/time
or
whereis time
time: /usr/bin/time /usr/bin/X11/time /usr/include/time.h /usr/share/man/man7/time.7.gz /usr/share/man/man2/time.2.gz /usr/share/man/man1/time.1.gz
or
type -a time
time is a shell keyword
time is /usr/bin/time
How to define where another time is located?
Users of the bash shell need to use an explicit path in order to run
the external time command and not the shell builtin variant. On system
where time is installed in /usr/bin, the first example would become
/usr/bin/time wc /etc/hosts
OR
Note: some shells (e.g., bash(1)) have a built-in time command that
provides less functionality than the command described here. To
access the real command, you may need to specify its pathname
(something like /usr/bin/time).
http://man7.org/linux/man-pages/man1/time.1.html
guys. I am confused about the result below:
1). time xxxxx
real 0m28.942s
user 0m28.702s
sys 0m0.328s
2). /usr/bin/time -p xxxxx
real 28.48
user 0.00
sys 0.13
so. I have some question(user: 0m28.702s != 0, sys: 0m0.328s != 0.13):
what's different between time and /usr/bin/time ?
what's different in differnt cpu platform, one core or multicore ?
any suggestion？
It's quite easy to find out the answer to your first question using type:
$ type time
time is a shell keyword
$ type /usr/bin/time
/usr/bin/time is /usr/bin/time
So the first command uses a bash built-in, while the latter defers to an external program. However, not knowing what system you are using, I have no idea where that program comes from. On Gentoo Linux, there's no /usr/bin/time by default, and the only implementation available is GNU time that has different output.
That said, I have tried a command similar to yours (assuming it's working on a 1G file), and got the following results:
$ time sed -e 's/0//g' big-file > big-file2
real 0m40.600s
user 0m31.295s
sys 0m4.174s
$ /usr/bin/time sed -e 's/0//g' big-file > big-file2
35.06user 3.31system 0:40.58elapsed 94%CPU (0avgtext+0avgdata 3488maxresident)k
8inputs+2179176outputs (0major+276minor)pagefaults 0swaps
As you can see, the numbers are similar.
Then, given your results (0 userspace time is quite impossible) I'd say that your /usr/bin/time is simply broken. This might be worth reporting a bug to its author.
I want to execute something in a linux shell under a few different conditions, and be able to output the execution time of each execution.
I know I could write a perl or python script that would do this, but is there a way I can do it in the shell? (which happens to be bash)
Use the built-in time keyword:
$ help time
time: time [-p] PIPELINE
Execute PIPELINE and print a summary of the real time, user CPU time,
and system CPU time spent executing PIPELINE when it terminates.
The return status is the return status of PIPELINE. The `-p' option
prints the timing summary in a slightly different format. This uses
the value of the TIMEFORMAT variable as the output format.
Example:
$ time sleep 2
real 0m2.009s
user 0m0.000s
sys 0m0.004s
You can get much more detailed information than the bash built-in time (i.e time(1), which Robert Gamble mentions). Normally this is /usr/bin/time.
Editor's note:
To ensure that you're invoking the external utility time rather than your shell's time keyword, invoke it as /usr/bin/time.
time is a POSIX-mandated utility, but the only option it is required to support is -p.
Specific platforms implement specific, nonstandard extensions: -v works with GNU's time utility, as demonstrated below (the question is tagged linux); the BSD/macOS implementation uses -l to produce similar output - see man 1 time.
Example of verbose output:
$ /usr/bin/time -v sleep 1
Command being timed: "sleep 1"
User time (seconds): 0.00
System time (seconds): 0.00
Percent of CPU this job got: 1%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 210
Voluntary context switches: 2
Involuntary context switches: 1
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
#!/bin/bash
START=$(date +%s)
# do something
# start your script work here
ls -R /etc > /tmp/x
rm -f /tmp/x
# your logic ends here
END=$(date +%s)
DIFF=$(( $END - $START ))
echo "It took $DIFF seconds"
For a line-by-line delta measurement, try gnomon.
$ npm install -g gnomon
$ <your command> | gnomon --medium=1.0 --high=4.0 --ignore-blank --real-time=100
A command line utility, a bit like moreutils's ts, to prepend timestamp information to the standard output of another command. Useful for long-running processes where you'd like a historical record of what's taking so long.
You can also use the --high and/or --medium options to specify a length threshold in seconds, over which gnomon will highlight the timestamp in red or yellow. And you can do a few other things, too.
Should you want more precision, use %N with date (and use bc for the diff, because $(()) only handles integers).
Here's how to do it:
start=$(date +%s.%N)
# do some stuff here
dur=$(echo "$(date +%s.%N) - $start" | bc)
printf "Execution time: %.6f seconds" $dur
Example:
start=$(date +%s.%N); \
sleep 0.1s; \
dur=$(echo "$(date +%s.%N) - $start" | bc); \
printf "Execution time: %.6f seconds\n" $dur
Result:
Execution time: 0.104623 seconds
If you intend to use the times later to compute with, learn how to use the -f option of /usr/bin/time to output code that saves times. Here's some code I used recently to get and sort the execution times of a whole classful of students' programs:
fmt="run { date = '$(date)', user = '$who', test = '$test', host = '$(hostname)', times = { user = %U, system = %S, elapsed = %e } }"
/usr/bin/time -f "$fmt" -o $timefile command args...
I later concatenated all the $timefile files and pipe the output into a Lua interpreter. You can do the same with Python or bash or whatever your favorite syntax is. I love this technique.
If you only need precision to the second, you can use the builtin $SECONDS variable, which counts the number of seconds that the shell has been running.
while true; do
start=$SECONDS
some_long_running_command
duration=$(( SECONDS - start ))
echo "This run took $duration seconds"
if some_condition; then break; fi
done
You can use time and subshell ():
time (
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
)
Or in same shell {}:
time {
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
}
The way is
$ > g++ -lpthread perform.c -o per
$ > time ./per
output is >>
real 0m0.014s
user 0m0.010s
sys 0m0.002s
one possibly simple method ( that may not meet different users needs ) is the use of shell PROMPT.it is a simple solution that can be useful in some cases. You can use the bash prompting feature as in the example below:
export PS1='[\t \u#\h]\$'
The above command will result in changing the shell prompt to :
[HH:MM:SS username#hostname]$
Each time you run a command (or hit enter) returning back to the shell prompt, the prompt will display current time.
notes:
1) beware that if you waited for sometime before you type your next command, then this time need to be considered, i.e the time displayed in the shell prompt is the timestamp when the shell prompt was displayed, not when you enter command. some users choose to hit Enter key to get a new prompt with a new timestamp before they are ready for the next command.
2) There are other available options and modifiers that can be used to change the bash prompt, refer to ( man bash ) for more details.
perf stat Linux CLI utility
This tool is overkill for just getting time. But it can do so much more for you to help profile and fix slowness that it is worth knowing about. Ubuntu 22.04 setup:
sudo apt install linux-tools-common linux-tools-generic
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
Usage:
perf stat <mycmd>
Sample run with stress-ng:
perf stat stress-ng --cpu 1 --cpu-method matrixprod -t 5
Sample output:
Performance counter stats for 'stress-ng --cpu 1 --cpu-method matrixprod -t 5':
5,005.46 msec task-clock # 0.999 CPUs utilized
88 context-switches # 17.581 /sec
1 cpu-migrations # 0.200 /sec
1,188 page-faults # 237.341 /sec
18,847,667,167 cycles # 3.765 GHz
26,544,261,897 instructions # 1.41 insn per cycle
3,239,655,001 branches # 647.225 M/sec
25,393,369 branch-misses # 0.78% of all branches
5.012218939 seconds time elapsed
4.998051000 seconds user
0.009122000 seconds sys
perf can also do a bunch more advanced things, e.g. here I show how to use it to profile code: How do I profile C++ code running on Linux?