formatting bash/shell script time to days, hours, minutes and second - linux

I have a backup script which is written in bash/shell scripting language. I calculate the total runtime/execution time by doing this:
#!/bin/bash
# start time
res1=$(date +%s.%N)
### do some work here
# end time & calculate
res2=$(date +%s.%N)
dt=$(echo "$res2 - $res1" | bc)
dd=$(echo "$dt/86400" | bc)
dt2=$(echo "$dt-86400*$dd" | bc)
dh=$(echo "$dt2/3600" | bc)
dt3=$(echo "$dt2-3600*$dh" | bc)
dm=$(echo "$dt3/60" | bc)
ds=$(echo "$dt3-60*$dm" | bc)
# finished
printf " >>> Process Completed - Total Runtime (d:h:m:s) : %d:%02d:%02d:%02.4f\n" $dd $dh $dm $ds
echo " "
exit 0
This outputs something like this:
How do you format the result, so it looks something like this:
0 Days, 0 Hours, 0 Minutes and 0.0968 Seconds
If it can intelligently show only values > 0, like these examples - it would be abonus:
7 Minutes and 5.126 Seconds
or
2 hours, 4 Minutes and 1.106 Seconds
or
7.215 Seconds etc...

You can use your last printf like this:
printf " >>> Process Completed - Total Runtime (d:h:m:s) : %d Days, %02d Hours, %02d Minutes, %02.4f Seconds\n" $dd $dh $dm $ds
However I would suggest you to use awk and do all calculations and formatting in awk itself so that you can avoid many invocations of bc.
Suggested awk script:
awk -v res1="$res1" -v res2="$res2" 'BEGIN {dt=res2-res1; dd=dt/86400; dt2=dt-86400*dd;
dh=dt2/3600; dt3=dt2-3600*dh; dm=dt3/60; ds=dt3-60*dm;
printf " >>> Process Completed - Total Runtime (d:h:m:s) : %d Days, %02d Hours, %02d Minutes, %02.4f Seconds\n",
dt/86400, dd, dh, dm, ds}'

Related

Time difference in shell (hour)

I'm trying to calculate time difference stored inside of two variables inside of a shell script, I'm observing the following pattern:
hhmm -> 0950
so:
time1=1333
time2=0950
Now I need to calculate the difference in time between time1 and time2, as for now I have tried:
deltaTime=$(($time1-$time2))
but I'm facing the following error message
1333-0950: value too great for base (error token is "0950")
I'm expecting as a result: $deltaTime=0343
Unfortunately, I am strictly bound to use this time pattern. I have already researched for a solution online, some of them propose to use date -d... but I couldn't get it to work :(
Your approach has two issues.
First issue: bash recognizes numbers with leading zeroes as octal. You can force base10 by adding 10# prefix.
Second issue: it is incorrect to consider strings in hhmm format as numbers and substract them. e.g. 1333-950=383 but difference between 09:50 and 13:33 is 3 hours and 43 minutes. You should convert string values to common units, e.g. to minutes, substract them and convert back to hhmm format.
time1=1333
time2=0950
str2min()
{
printf "%u" $((10#${1%??} * 60 + 10#${1#??}))
}
min2str()
{
printf "%02u%02u" $(($1 / 60)) $(($1 % 60))
}
time1m=$(str2min $time1)
time2m=$(str2min $time2)
timediff=$(($time1m - $time2m))
deltaTime=$(min2str $timediff)
You could use this implementation maybe?
#!/usr/bin/env bash
diff_hhmm() {
local -r from=$1
local -i from_hh=10#${from:0:2} # skip 0 chars, read 2 chars (`${from:0:2}`) using base 10 (`10#`)
local -ri from_mm=10#${from:2:2} # skip 2 chars, read 2 chars (`${from:0:2}`) using base 10 (`10#`)
local -r upto=$2
local -ri upto_hh=10#${upto:0:2}
local -ri upto_mm=10#${upto:2:2}
local -i diff_hh
local -i diff_mm
# Compute difference in minutes
(( diff_mm = from_mm - upto_mm ))
# If it's negative, we've "breached" into the previous hour, so adjust
# the `diff_mm` value to be modulo 60 and compensate the `from_hh` var
# to reflect that we've already subtracted some of the minutes there.
if (( diff_mm < 0 )); then
(( diff_mm += 60 ))
(( from_hh -= 1 ))
fi
# Compute difference in hours
(( diff_hh = from_hh - upto_hh ))
# Ensure the result is modulo 24, the number of hours in a day.
if (( diff_hh < 0 )); then
(( diff_hh += 24 ))
fi
# Print the values with 0-padding if necessary.
printf '%02d%02d\n' "$diff_hh" "$diff_mm"
}
$ diff_hhmm 1333 0950
0343
$ diff_hhmm 0733 0950
2143
$ diff_hhmm 0733 0930
2203
Or an even shorter implementation using a big arithmetic compound command ((( ... )) ) and inlining some variables:
diff_hhmm_terse() {
local -i diff_hh diff_mm
((
diff_mm = 10#${1:2:2} - 10#${2:2:2},
diff_hh = 10#${1:0:2} - 10#${2:0:2},
diff_hh -= diff_mm < 0 ? 1 : 0,
diff_mm += diff_mm < 0 ? 60 : 0,
diff_hh += diff_hh < 0 ? 24 : 0
))
printf '%02d%02d\n' "$diff_hh" "$diff_mm"
}
Do you have the possibility to drop the leading zero?
As you can see from my prompt:
Prompt> echo $((1333-0950))
-bash: 1333-0950: value too great for base (error token is "0950")
Prompt> echo $((1333-950))
383
Other proposal:
date '+%s'
Let me give you some examples:
date '+%s'
1662357975
... (after some time)
date '+%s'
1662458180
=>
echo $((1662458180-1662357975))
100205 (amount of seconds)
=>
echo $(((1662458180-1662357975)/3600))
27 (amount of hours)
This bash one-liner may be used if time difference is not negative (that is, time1 >= time2):
printf '%04d\n' $(( 10#$time1 - 10#$time2 - (10#${time1: -2} < 10#${time2: -2} ? 40 : 0) ))

Calculate the 5 minutes ceiling

Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
I want to round the time to the nearest 5 minutes, only up, not down, for example:
08:09:15 should be 08:10:00
08:11:26 should be 08:15:00
08:17:58 should be 08:20:00
I have been trying with:
(date -d #$(( (($(date +%s) + 150) / 300) * 300)) "+%H:%M:%S")
This will round the time but also down (08:11:18 will result in 08:10:00 and not 08:15:00)
Any idea how i can achieve this?
You may use this utility function for your rounding up:
roundDt() {
local n=300
local str="$1"
date -d #$(( ($(date -d "$str" '+%s') + $n)/$n * $n)) '+%H:%M:%S'
}
Then invoke this function as:
roundDt '08:09:15'
08:10:00
roundDt '08:11:26'
08:15:00
roundDt '08:17:58'
08:20:00
To trace how this function is computing use -x (trace mode) after exporting:
export -f roundDt
bash -cx "roundDt '08:11:26'"
+ roundDt 08:11:26
+ typeset n=300
+ typeset str=08:11:26
++ date -d 08:11:26 +%s
+ date -d #1535631300 +%H:%M:%S
08:15:00
GNU date can calculate already. It is explained in the manual in the chapter "Relative items in date strings". So you need just one date call.
d=$(date +%T) # get the current time
IFS=: read h m s <<< "$d" # parse it in hours, minutes and seconds
inc=$(( 300 - (m * 60 + s) % 300 )) # calculate the seconds to increment
date -d "$d $inc sec" +%T # output the new time with the offset
Btw: +%T is the same as +%H:%M:%S.

Write a bash script that accepts a time duration as arguments?

I'm looking for a bash script that can parse a time duration.
If three arguments are given, they represent hours, minutes, and seconds. If two arguments are given, they represent minutes and seconds, with the hours zero.
What about the following:
#!/bin/bash
h=0
if [ "$#" -ge 3 ]
then
h=$1
shift
fi
sec=$((3600*$h+60*$1+$2))
echo "The total number of seconds is $sec"
Since the question does not specify what you aim to do with the given time, the program calculates the total number of seconds. Furthermore perhaps it is useful to do a check if at least two arguments are given.
The script uses the shift operation, the shift makes makes $1 := $2; $2 := $3, etc. In other words, the first argument is processed, and then you "pretend" it never existed.
By default you set h to zero, and only if the number of arguments is greater than or equal to 3, it will set h.
This is a more or less general solution for that type of task. Sorry, if it is a monkeycode, but I think it is sufficient:
gettime() {
params=(
years months weeks days hours minutes seconds
)
for i in `seq ${#params}`; do
param_i=$((${#params} - i + 1)) # reversed params index
[ $i -le $# ] && {
eval "local ${params[$param_i]}=\$$(($# - i + 1))"
} || {
eval "local ${params[$param_i]}=0"
}
eval "echo ${params[$param_i]} '==' \$${params[$param_i]}" # debug output
done
}
Here's the sample output:
$ gettime 3 4 5 6 7
seconds == 7
minutes == 6
hours == 5
days == 4
weeks == 3
months == 0
years == 0
Note, that the shell you are using must be not only support POSIX standards, but also arrays.
First Argument: $1
Second Argument: $2
Third Argument: $3
and so on...
Example:
bash-2.05a$ ./parseDuration.sh 13 25 25
13 hours and 25 minutes and 25 seconds
bash-2.05a$ cat ./parseDuration.sh
#!/bin/bash
echo "$1 hours and $2 minutes and $3 seconds"

How can you can calculate the time span between two time entries in a file using a shell script?

In a Linux script: I have a file that has two time entries for each message within the file. A 'received time' and a 'source time'. there are hundreds of messages within the file.
I want to calculate the elapsed time between the two times.
2014-07-16T18:40:48Z (received time)
2014-07-16T18:38:27Z (source time)
The source time is 3 lines after the received time, not that it matters.
info on the input data:
The input has a lines are as follows:
TimeStamp: 2014-07-16T18:40:48Z
2 lines later: a bunch of messages in one line and within each line, multiple times is:
sourceTimeStamp="2014-07-16T18:38:27Z"
If you have GNU's date (not busybox's), you can give difference in seconds with:
#!/bin/bash
A=$(date -d '2014-07-16T18:40:48Z' '+%s')
B=$(date -d '2014-07-16T18:38:27Z' '+%s')
echo "$(( A - B )) seconds"
For busybox's date and ash (modern probably / BusyBox v1.21.0):
#!/bin/ash
A=$(busybox date -d '2014-07-16 18:40:48' '+%s')
B=$(busybox date -d '2014-07-16 18:38:27' '+%s')
echo "$(( A - B )) seconds"
you should be able to use date like this (e.g.)
date +%s --date="2014-07-16T18:40:48Z"
to convert both timestamps into a unix timestamp. Getting the time difference between them is then reduced to a simple subtraction.
Does this help?
I would use awk. The following script searches for the lines of interest, converts the time value into a UNIX timestamp and saves them in the start, end variables. At the end of the script the difference will get calculated and printed:
timediff.awk:
/received time/ {
"date -d "$1" +%s" | getline end
}
/source time/ {
"date -d "$1" +%s" | getline start
exit
}
END {
printf "%s seconds in between", end - start
}
Execute it like this:
awk -f timediff.awk log.file
Output:
141 seconds in between

Get/extract the data from log file of last 3 minutes? [duplicate]

This question already has answers here:
Filter log file entries based on date range
(5 answers)
Closed 6 years ago.
I have agent.log file. This file is updating as regular interval.
Entries are as follows 2014-01-07 03:43:35,223 INFO ...some data
I want to extract data of last 3 minutes, Is there any way so that I will get this data using bash script?
Try the solution below:
awk \
-v start="$(date +"%F %R" --date=#$(expr `date +%s` - 180))" \
-v end="$(date "+%F %R")" \
'$0 ~ start, $0 ~ end' \
agent.log
In the start variable there is the time stamp 3 minutes (180 seconds) before the current time.
In the end there is the current time.
$0 ~ start, $0 ~ end selects the lines between start and end
date +"%F %R" gives you the current time down to the minute.
grep '^'"$(date +"%F %R")" agent.log will select the last minute from the file
Now for the previous two minutes it's more tricky... I have developed some scripts that can do complete time manipulation in relative or absolute, and it may be simpler than fiddling with date...
2 minutes ago in the right format: date --date="#$(($(date +"%s") - 2*60))" +"%F %R"
Merge all 3:
NOW=$(date +"%F %R")
M1=$(date --date="#$(($(date +"%s") - 1*60))" +"%F %R")
M2=$(date --date="#$(($(date +"%s") - 2*60))" +"%F %R")
grep '^'"$NOW\|$M1\|$M2" agent.log
my answer considers the followings:
using bash and UNIX/Linux commands
the last log line is the start time not the actual server time
there is no expectation about the lines' date (minutes, days, years, etc.)
the whole script should be expandable to the inverse, or a specified from-to interval
#!/bin/bash
# this script expects descending dates in a log file (reverse as real life examples)!!!
FILE=$1
INTV=180 # sec
while read LINE
do
if [ -z $LAST_LOG_LINE ]
then
# interval stat line
LAST_LOG_LINE=$(date --date="$( echo "$LINE" | sed -e 's/INFO.*//')" +%s)
# mod
#continue
fi
ACT_LOG_LINE=$(date --date="$( echo "$LINE" | sed -e 's/INFO.*//')" +%s)
# print line if not greater than $INTV (180s)
# else break the reading and exit
if [ $(($LAST_LOG_LINE-$ACT_LOG_LINE)) -gt $INTV ]
then
break
fi
# actual print
echo "$LINE"
done < $FILE
Testing:
2014-01-07 03:43:35,223 INFO ...some data
2014-01-07 03:42:35,223 INFO ...some data
2014-01-07 03:41:35,223 INFO ...some data
2014-01-07 03:40:35,223 INFO ...some data
2014-01-07 02:43:35,223 INFO ...some data
2014-01-07 01:43:35,223 INFO ...some data
2014-01-06 03:43:35,223 INFO ...some data
$ /tmp/stack.sh /tmp/log
2014-01-07 03:42:35,223 INFO ...some data
2014-01-07 03:41:35,223 INFO ...some data
2014-01-07 03:40:35,223 INFO ...some data
$
I think you may be somewhat better off using Python in this case. Even if this script doesn't find a date exactly 3 minutes ago, it will still get any log entries in between the time the script was called and 3 minutes ago. This is both concise and more robust than some of the previous solutions offered.
#!/usr/bin/env python
from datetime import datetime, timedelta
with open('agent.log') as f:
for line in f:
logdate = datetime.strptime(line.split(',')[0], '%Y-%m-%d %H:%M:%S')
if logdate >= datetime.now() - timedelta(minutes=3):
print(line)
A Ruby solution (tested on ruby 1.9.3)
You can pass days, hours, minutes or seconds as a parameter and it will search for the expression and on the file specified (or directory, in which case it will append '/*' to the name):
In your case just call the script like so: $0 -m 3 "expression" log_file
Note: Also if you know the location of 'ruby' change the shebang (first line of the script),
for security reasons.
#! /usr/bin/env ruby
require 'date'
require 'pathname'
if ARGV.length != 4
$stderr.print "usage: #{$0} -d|-h|-m|-s time expression log_file\n"
exit 1
end
begin
total_amount = Integer ARGV[1]
rescue ArgumentError
$stderr.print "error: parameter 'time' must be an Integer\n"
$stderr.print "usage: #{$0} -d|-h|-m|-s time expression log_file\n"
end
if ARGV[0] == "-m"
gap = Rational(60, 86400)
time_str = "%Y-%m-%d %H:%M"
elsif ARGV[0] == "-s"
gap = Rational(1, 86400)
time_str = "%Y-%m-%d %H:%M:%S"
elsif ARGV[0] == "-h"
gap = Rational(3600, 86400)
time_str = "%Y-%m-%d %H"
elsif ARGV[0] == "-d"
time_str = "%Y-%m-%d"
gap = 1
else
$stderr.print "usage: #{$0} -d|-h|-m|-s time expression log_file\n"
exit 1
end
pn = Pathname.new(ARGV[3])
if pn.exist?
log = (pn.directory?) ? ARGV[3] + "/*" : ARGV[3]
else
$stderr.print "error: file '" << ARGV[3] << "' does not exist\n"
$stderr.print "usage: #{$0} -d|-h|-m|-s time expression log_file\n"
end
search_str = ARGV[2]
now = DateTime.now
total_amount.times do
now -= gap
system "cat " << log << " | grep '" << now.strftime(time_str) << ".*" << search_str << "'"
end

Resources