get average of times that take job finish in for loop shell script - linux

Here is my first shell script
#!/bin/bash
COUNTER=0
while [ $COUNTER -lt 10000 ]; do
date;
time wget -q 'http://exmple.com/' > /dev/null | grep real;
sleep 3;
let COUNTER=COUNTER+1
done
echo ${COUNTER} Request Sent\n
Average Response Time is :
this script download page content and calculate time of page response
i need to store Real parameter and calculate average of it
the output of script is something like that
Tue Oct 25 22:43:36
real 0m13.275s
user 0m0.004s
sys 0m0.008s
So my problems are :
How can i add some jobs (like echo ${COUNTER} ) after pressing crtl+c (stopping script)
How can i store value of "real" in seconds // sometimes page response goes to minutes , so basically need some time convert function (?)
i tried to solve the problems but as i mentioned i am training shell script

#!/bin/bash
echo "How many runs would you like to perform?"
read limit
total_time=0
counter=1
while [ ${counter} -le ${limit} ]
do
date
run_time=`(time -p wget -qO- http://exmple.com/) 2>&1 > /dev/null | grep real | awk '{print $2}'`
echo "Run ${counter} completed in ${run_time} seconds"
total_time=$(bc<<<"${total_time}+${run_time}")
if [ ${counter} -ne ${limit} ]
then
sleep 3
let counter=counter+1
fi
done
avg=$(printf "%.3f" "$(bc -l <<<"${total_time}/${limit}")")
echo "${counter} requests were sent taking ${total_time} seconds"
echo "Average response time was : ${avg} seconds"
Above is a modified script to perform the operations you requested, with a couple modifications
Added a question then input to ask how many runs to make each time, instead of a static number
Print out total time for runs and then average per run in 3 decimal places
Skip sleeping if it's the last run
You'll notice we use bc instead of let for a couple math lines. The bc command allows the use of decimal places.
https://www.gnu.org/software/bc/manual/html_mono/bc.html
To get the value in seconds I had to use awk to separate the value from the title 'real' in the line. Then you can do the math on the number value.
One other thing I noticed in your initial script. You used both ways of displaying a variable:
$COUNTER and ${COUNTER}
While both are legitimate, I got into the habit of always encapsulating my variables, it makes things easier when you want to add to the end of it.
For example let's say I have a number defined (NUM=7), but want to display it times 10. If I don't encapsulate echo $NUM0 then I get an error because the variable $NUM0 wasn't defined. But if I encapsulate echo ${NUM}0 will display an output of 70.
All of this results in a final output as below
How many runs would you like to perform?
5
Tue Oct 25 16:02:58 MST 2016
Run 1 completed in 0.09 seconds
Tue Oct 25 16:03:01 MST 2016
Run 2 completed in 0.08 seconds
Tue Oct 25 16:03:05 MST 2016
Run 3 completed in 0.07 seconds
Tue Oct 25 16:03:08 MST 2016
Run 4 completed in 0.09 seconds
Tue Oct 25 16:03:11 MST 2016
Run 5 completed in 0.08 seconds
5 requests sent taking .41 seconds
Average response time was : 0.08 seconds
Please let me know if you have any other questions about the script.

You can record a reasonably precise time with e.g.
t1=$(date +%s.%N)
wget ...
t2=$(date +%s.%N)
diff=$(($t2-$t1))
For doing stuff after Ctrl+C, which is really a SIGINT signal, read up about the trap builtin.

Related

Calculating the time until the start of next hour in Bash

Hi I have a bash script called by a systemd service that needs to run until the start of the next hour. Currently I have been using:
currentTime=$(date +"%s")
nextHour=$(date -d "$(date -d 'next hour' '+%H:00:00')" '+%s')
duration=$(((nextHour-currentTime)*1000))
Which works except for trying to calculate the difference between 11pm and midnight were as far as I can tell it gets the current days midnight from 23 hours previous.
Oct 13 23:00:05 host bash[2019]: 1665698405
Oct 13 23:00:05 host bash[2019]: 1665615600
Oct 13 23:00:05 host bash[2019]: -82805000
I figure I could put a conditional check with a different calculation if needed or perhaps look at a systemd timer for triggering the service but as the service needs to always activate on boot/reboot as well as running hour to hour this setup seemed more appropriate.
Would appreciate any advice on why this is happening and advice on most streamlined steps to avoid it.
This isn't really a question about bash, but seems to be more about date. Given that date has multiple different implementations, it seems wiser to choose a different tool to do the calculation. I suspect perl is more standardized and (almost) as available as date, so you might try to get the difference with:
perl -MTime::Seconds -MTime::Piece -E '
my $t = localtime; my $m = ($t + ONE_HOUR)->truncate(to => "hour"); say $m - $t'

How to get the time when I last used the computer

In GNU/Linux on xorg session, what I want to do is to get how many seconds have passed since I stopped working with the computer (i.e. no keys pressed and/or cursor moved).
Running in the background, the script below will display secs in the status bar.
But the question is what THE_COMMAND will be.
While true; do
last_touched="$(THE_COMMAND)"
now="$(date +%s)"
secs=$((now - last_touched))
echo "${secs} seconds ago"
sleep 3
done
I remember asking the same question a while back.
Here is what I found,
last -aiF -n 1 userName
command can give you the current session.
When combined with awk you can get the result as follows
$ last -aiF -n2 username
username :1 Wed Apr 21 13:09:00 2021 still logged in 0.0.0.0
username :1 Wed Apr 21 07:28:47 2021 - down (05:39) 0.0.0.0
$ last -aiF -n 2 ogulcan | awk '{print $10}'
in
(05:39)
the lines here are the session times.
These times are counted as now - first boot login
But I believe these does not work best for you.
So here is the 8 year old question that may be helpful to you.
User Idle time in Linux
Using python you can calculate the idle time passed. Maybe this way, you can simply get what you want with python.

Shell Script time control

I am running a shell script which runs for ONCE or twice a week..
and this program runs for few hours (depends on how many files) a day..
But problem is, I have to pause this program during the working hours..
for example, if the working hour is 1 am to 3 am, then all the script work should pause and wait till 3:01 am to start again..
i don't have to KILL the current running process if it is running after 1 am..
but whenever it is done with that specific FILE, I have to make it pause for next files..
currently this is what i figure out
while true
do
curr_time= date +"%H%M%S"
if [ $curr_time -ge 005000 -a $curr_time -le 030000 ]
then
echo "Pause for 12000 seconds"
sleep 12000
else
break;
fi
done
So start pausing at 12:50 instead of hoping that everything can phase before 1am
and sleep for 12000 sec which is about 3 hours and restart..
but the problem is if for some reason, when it wakes up from 12000second sleep and it is not 3 am yet, it will sleep for another 12000 seconds..
how can I go around with this ??
i want it to pause around 1 am (each file process takes about 1 minute but few hundreds and thousands files are there to process) and starts RIGHT BACK AT 3 am..
I don't have to kill it if it passes 1 am.. just pause after that file is done processing.
As your question is tagged with bash, you might use arithmetic expansion to just sleep the needed number of seconds.
sleep $(($(date -d 3:01 +%s) - $(date +%s)))
date -d 3:01 +%s should tell the number of seconds since the epoch until 3:01, date +%s the number of seconds since the epoch until now.
The difference should be the number of seconds you have to wait until 3:01.

Bash: Program next execution of current script using 'at'

I want to execute a script and make it schedule the next execution. A sample would be:
#!/bin/bash
TMP=/tmp/text.txt
SCRIPT=$(readlink -f $0)
date >>$TMP
at -f $SCRIPT now + 1 minutes >>$TMP 2>&1
echo -e "\n" >>$TMP
A sample execution would do as follows:
First execution OK. Schedules to next minute
Second execution writes OK but doesn't schedule
Resulting output would be:
tue mar 5 14:34:01 CET 2013
job 15 at 2013-03-05 14:35
tue mar 5 14:35:00 CET 2013
job 16 at 2013-03-05 14:36
[now at 2013-03-05 14:38]
atq outputs nothing and I don't see any /var/at/jobs (In fact, ls /var/at* outputs nothing. There is no message in any user in /var/mail/. I'm trying on a CentOS release 5.6 x86_64
Anyone has any hint as to what may be happening?
suspectus, you have hit the point... echo $SCRIPT gives '/bin/bash'... I've manually written the full path and now it works

Get program execution time in the shell

I want to execute something in a linux shell under a few different conditions, and be able to output the execution time of each execution.
I know I could write a perl or python script that would do this, but is there a way I can do it in the shell? (which happens to be bash)
Use the built-in time keyword:
$ help time
time: time [-p] PIPELINE
Execute PIPELINE and print a summary of the real time, user CPU time,
and system CPU time spent executing PIPELINE when it terminates.
The return status is the return status of PIPELINE. The `-p' option
prints the timing summary in a slightly different format. This uses
the value of the TIMEFORMAT variable as the output format.
Example:
$ time sleep 2
real 0m2.009s
user 0m0.000s
sys 0m0.004s
You can get much more detailed information than the bash built-in time (i.e time(1), which Robert Gamble mentions). Normally this is /usr/bin/time.
Editor's note:
To ensure that you're invoking the external utility time rather than your shell's time keyword, invoke it as /usr/bin/time.
time is a POSIX-mandated utility, but the only option it is required to support is -p.
Specific platforms implement specific, nonstandard extensions: -v works with GNU's time utility, as demonstrated below (the question is tagged linux); the BSD/macOS implementation uses -l to produce similar output - see man 1 time.
Example of verbose output:
$ /usr/bin/time -v sleep 1
Command being timed: "sleep 1"
User time (seconds): 0.00
System time (seconds): 0.00
Percent of CPU this job got: 1%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 210
Voluntary context switches: 2
Involuntary context switches: 1
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
#!/bin/bash
START=$(date +%s)
# do something
# start your script work here
ls -R /etc > /tmp/x
rm -f /tmp/x
# your logic ends here
END=$(date +%s)
DIFF=$(( $END - $START ))
echo "It took $DIFF seconds"
For a line-by-line delta measurement, try gnomon.
$ npm install -g gnomon
$ <your command> | gnomon --medium=1.0 --high=4.0 --ignore-blank --real-time=100
A command line utility, a bit like moreutils's ts, to prepend timestamp information to the standard output of another command. Useful for long-running processes where you'd like a historical record of what's taking so long.
You can also use the --high and/or --medium options to specify a length threshold in seconds, over which gnomon will highlight the timestamp in red or yellow. And you can do a few other things, too.
Should you want more precision, use %N with date (and use bc for the diff, because $(()) only handles integers).
Here's how to do it:
start=$(date +%s.%N)
# do some stuff here
dur=$(echo "$(date +%s.%N) - $start" | bc)
printf "Execution time: %.6f seconds" $dur
Example:
start=$(date +%s.%N); \
sleep 0.1s; \
dur=$(echo "$(date +%s.%N) - $start" | bc); \
printf "Execution time: %.6f seconds\n" $dur
Result:
Execution time: 0.104623 seconds
If you intend to use the times later to compute with, learn how to use the -f option of /usr/bin/time to output code that saves times. Here's some code I used recently to get and sort the execution times of a whole classful of students' programs:
fmt="run { date = '$(date)', user = '$who', test = '$test', host = '$(hostname)', times = { user = %U, system = %S, elapsed = %e } }"
/usr/bin/time -f "$fmt" -o $timefile command args...
I later concatenated all the $timefile files and pipe the output into a Lua interpreter. You can do the same with Python or bash or whatever your favorite syntax is. I love this technique.
If you only need precision to the second, you can use the builtin $SECONDS variable, which counts the number of seconds that the shell has been running.
while true; do
start=$SECONDS
some_long_running_command
duration=$(( SECONDS - start ))
echo "This run took $duration seconds"
if some_condition; then break; fi
done
You can use time and subshell ():
time (
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
)
Or in same shell {}:
time {
for (( i=1; i<10000; i++ )); do
echo 1 >/dev/null
done
}
The way is
$ > g++ -lpthread perform.c -o per
$ > time ./per
output is >>
real 0m0.014s
user 0m0.010s
sys 0m0.002s
one possibly simple method ( that may not meet different users needs ) is the use of shell PROMPT.it is a simple solution that can be useful in some cases. You can use the bash prompting feature as in the example below:
export PS1='[\t \u#\h]\$'
The above command will result in changing the shell prompt to :
[HH:MM:SS username#hostname]$
Each time you run a command (or hit enter) returning back to the shell prompt, the prompt will display current time.
notes:
1) beware that if you waited for sometime before you type your next command, then this time need to be considered, i.e the time displayed in the shell prompt is the timestamp when the shell prompt was displayed, not when you enter command. some users choose to hit Enter key to get a new prompt with a new timestamp before they are ready for the next command.
2) There are other available options and modifiers that can be used to change the bash prompt, refer to ( man bash ) for more details.
perf stat Linux CLI utility
This tool is overkill for just getting time. But it can do so much more for you to help profile and fix slowness that it is worth knowing about. Ubuntu 22.04 setup:
sudo apt install linux-tools-common linux-tools-generic
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
Usage:
perf stat <mycmd>
Sample run with stress-ng:
perf stat stress-ng --cpu 1 --cpu-method matrixprod -t 5
Sample output:
Performance counter stats for 'stress-ng --cpu 1 --cpu-method matrixprod -t 5':
5,005.46 msec task-clock # 0.999 CPUs utilized
88 context-switches # 17.581 /sec
1 cpu-migrations # 0.200 /sec
1,188 page-faults # 237.341 /sec
18,847,667,167 cycles # 3.765 GHz
26,544,261,897 instructions # 1.41 insn per cycle
3,239,655,001 branches # 647.225 M/sec
25,393,369 branch-misses # 0.78% of all branches
5.012218939 seconds time elapsed
4.998051000 seconds user
0.009122000 seconds sys
perf can also do a bunch more advanced things, e.g. here I show how to use it to profile code: How do I profile C++ code running on Linux?

Resources