I need to run many times an aplication and get some statistcs using some commands.
A simple exemple of what I'm traying to do is:
/usr/bin/time --format="%U" date
This line will print:
Seg Nov 2 22:15:28 BRST 2015
0.00
I need to format this output, and I want to do this using bash, like this:
echo "$usertime & $date"
where $usertime is the result from /usr/bin/time --format="%U" and $date the result of date
I know that the below works, but I don't know how to separate the values into two variables:
OUTPUT=`/usr/bin/time --format="%U" date`
echo "$OUTPUT"
the output from above is:
0.00
Seg Nov 2 22:23:50 BRST 2015
(in reverse order)
Using awk
Here, awk is used to reverse the line order and format the output:
$ /usr/bin/time --format="%U" date 2>&1 | awk 'NR==1{d=$0;next} {print $0 " & " d}'
0.00 & Mon Nov 2 16:58:40 PST 2015
Using sed
$ /usr/bin/time --format="%U" date 2>&1 | sed -E 'N; s/([^\n]*)\n(.*)/\2 \& \1/'
0.00 & Mon Nov 2 17:01:15 PST 2015
Using shell
$ /usr/bin/time --format="%U" date 2>&1 | ( read d; read t; echo "$t & $d" )
0.00 & Mon Nov 2 17:04:40 PST 2015
/usr/bin/time is writing the time to standard error, whereas the output of date is going to standard output. If you want them in separate variables, you can write the output of time to a file with the -o flag, then read it back in to another variable:
OUTPUT=`/usr/bin/time -o timefile --format="%U" date`
TIME=$(<timefile)
Related
The command i run is as follows:
rpm -qi setup | grep Install
The output of the command:
Install Date: Do 30 Jul 2020 15:55:28 CEST
I would like to edit this output further more in order to remain with just:
30 Jul 2020
And the rest of the output not to be displayed.
What best editing way in bash can i possibly simply get this end result?
Use grep -Po like so (-P = use Perl regex engine, and -o = print just the match, not the entire line):
echo '**Install Date: Do 30 Jul 2020 15:55:28 CEST**' | grep -Po '\d{1,2}\s+\w{3}\s+\d{4}'
You can also use cut like so (-d' ' = split on blanks, -f4-6 =
print fields 4 through 6):
echo '**Install Date: Do 30 Jul 2020 15:55:28 CEST**' | cut -d' ' -f4-6
Output:
30 Jul 2020
You can do it using just rpmqueryformat and bashprintf:
$ printf '%(%d %b %Y)T\n' $(rpm -q --queryformat '%{INSTALLTIME}\n' setup)
29 Apr 2020
I want to split date command output and extract time zone difference and represent that time difference in terms of 30 min.
I tried this:
date -R | awk 'NF>1{print $NF}'
I got the output -0530
This means 5 hours and 30 min difference from GMT. Now I want to convert this -0530 in to -11. So what logic should I use in shell command?
date -R | awk '{tz=$NF;ew=substr(tz,1,1);h=substr(tz,1,3)*2;m=ew substr(tz,4,2)/30;print h + m}'
https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC
Edited to work with omitted sign
date -R | awk '{tz=$NF;l=length(tz);m=substr(tz,l-1,2)/30;h=substr(tz,l-3,2)*2;ew=l>4?ew=substr(tz,1,1):"+";print (ew h)+(ew m)}'
No need to complicate things.
tz=$(date '+%z'); echo "${tz:0:1}$(( 2 * ${tz:1:2} + ${tz:3} ))"
I wouldn't, and instead use a different tool. In the perl module Time::Piece you have tzoffset which returns the current timezone skew in seconds.
Thus:
#!/usr/bin/env perl
use strict;
use warnings;
use Time::Piece;
$ENV{'TZ'}='GMT-6';
my $now = localtime;
#find number of 30m differences;
print $now -> tzoffset / 60 / 30;
You can also use variants of strftime to print time formatted to your liking:
print $now -> strftime("%c");
This can be cut down into a one liner if so desired:
perl -MTime::Piece 'print localtime -> tzoffset / 60 / 30'
With GNU date, you could perform math on time/date values (in UTC):
$ date -uR -d "now"; date -uR -d "now -60 min"
Sun, 27 Dec 2015 20:34:17 +0000
Sun, 27 Dec 2015 19:34:17 +0000
A time of -5:30 is -330 min:
$ date -uR -d "now"; date -uR -d "now -330 min"
Sun, 27 Dec 2015 20:34:55 +0000
Sun, 27 Dec 2015 15:04:55 +0000
That could be further moved by the time zone:
$ TZ=America/New_York date -R -d "now"; TZ=America/New_York date -R -d "now -330 min"
Sun, 27 Dec 2015 15:34:26 -0500
Sun, 27 Dec 2015 10:04:26 -0500
That is the local time (now) subtracting 5 hours and 30 minutes.
However, what is a time zone of -5:30?
I know that India is +5:30 and there are some (very few) places that are +11:00.
But I fail to find any -5:30 and -11:00 seems just odd.
I have a date in YYYY.MM.DD HH:SS format (e.g. 2014.02.14 13:30). I'd like to convert it in seconds since epoch using the date command.
The command
date -d"2014.02.14 13:30" +%s
won't work, because of the dots separation.
Any Ideas?
Why don't you make the date format acceptable? Just replace dots with dashes:
$ date --date="`echo '2014.02.14 13:30' | sed 's/\./-/g'`" +%s
1392370200
Here I first change the format:
$ echo '2014.02.14 13:30' | sed 's/\./-/g'
2014-02-14 13:30
and then use the result as a parameter for date.
Note that the result depends on your timezone.
You can use:
s='2014.02.14 13:30'
date -d "${s//./}"
Fri Feb 14 13:30:00 EST 2014
To get EPOCH value:
date -d "${s//./}" '+%s'
1392402600
using awk :
s=`echo "2014.02.14 13:30" | awk '{gsub(/\./,"-",$0);print $0}'`
echo -d "$s"
date -d "$s" +%s
output:
Fri Feb 14 13:30:00 IST 2014
1392364800
Perl: does not require you to munge the string
d="2014.02.14 13:30"
epoch_time=$(perl -MTime::Piece -E 'say Time::Piece->strptime(shift, "%Y.%m.%d %H:%M")->epoch' "$d")
echo $epoch_time
1392384600
Timezone: Canada/Eastern
I Finally solved it using
awk 'BEGIN{FS=","}{ gsub(/./," ",$1);gsub(/:/," ",$2); var=sprintf("%s %s 00",$1,$2); print mktime(var), $3,$4,$5,$6,$7 }' myfile | less
so myfile:
2014.09.24,15:15,1.27921,1.27933,1.279,1.27924,234
became
1411582500 1.27921 1.27933 1.279 1.27924 234
:)
So I have this script file that automatically saves the date and time to a file when the terminal is open. But i'm having a hard time putting number lines in it. I tried cat -n, grep -n, ls -l, but I either get errors or doesn't increment. As you can see:
My Script
echo $(date) >> .test
I would like to see something like this:
0 : Tue Feb 15 13:10 EST 2014
1 : Tue Feb 18 12:10 EST 2014
2 : Tue Feb 18 10:10 EST 2014
3 : Tue Feb 19 13:22 EST 2014
If you pipe the result into cat, you can use the -n option to number each line like so:
ls | grep "whatever" | cat -n
Here's a solution:
echo $(wc -l < filename) : $(date) >> filename
Try saving the last number in a file. Then, in your script, read in the number, increment it, and write it back to the file. If you are using bash, something like this should work.
num=$(cat number.txt)
num=$(($num+1))
echo $num > number.txt
I'm looking for a way to replace strings of a file, matched by a regular expression, with another string that will be generated/evaluated out of the matched string.
For example, I want to replace the timestamps (timestamp + duration) in this file
1357222500 3600 ...
Maybe intermediate strings...
1357226100 3600 ...
Maybe intermediate strings...
...
By human readable date representations (date range).
Until now, I always used shell scripts like Bash to iterate over each line, matching for the line X, getting the matched group string and printing the line after processing, for example this way (from memory):
IFS="
"
for L in `cat file.txt`; do
if [[ "${L}" =~ ^([0-9]{1,10})\ ([0-9]{1,4})\ .*$ ]]; then
# Written as three lines for better readability/recognition
echo -n "`date --date=#${BASH_REMATCH[1]}` - "
echo -n "`date --date=#$(( ${BASH_REMATCH[1]} + ${BASH_REMATCH[2]} ))`"
echo ""
else
echo "$L"
fi
done
I wonder if there's something like this with a fictional(?) "sed-2.0":
cat file.txt | sed-2.0 's+/^\([0-9]\{1,10\}\) \([0-9]\{1,4\}\) .*$+`date --date="#\1"` - `date --date="#$(( \1 + \2 ))`'
Whereas the backticks in the sed-2.0 replacement will be evaluated as shell command passing the matched groups \1 and \2.
I know that this does not work as expected, but I'd like to write someting like this.
Edit 1
Edit of question above: added missing echo "" in if of Bash script example.
This should be the expected output:
Do 3. Jan 15:15:00 CET 2013 - Do 3. Jan 16:15:00 CET 2013
Maybe intermediate strings...
Do 3. Jan 16:15:00 CET 2013 - Do 3. Jan 17:15:00 CET 2013
Maybe intermediate strings...
...
Note, that the timestamp depends on the timezone.
Edit 2
Edit of question above: fixed syntax error of Bash script example, added comment.
Edit 3
Edit of question above: fixed syntax error of Bash script example. Changed the phrase "old-school example" to "Bash script example".
Summary of Kent's and glenn jackman's answer
There's a huge difference in both approaches: the execution time. I've compared all four methods, here are the results:
gawk using strftime()
/usr/bin/time gawk '/^[0-9]+ [0-9]+ / {t1=$1; $1=strftime("%c -",t1); $2=strftime("%c",t1+$2)} 1' /tmp/test
...
0.06user 0.12system 0:00.30elapsed 60%CPU (0avgtext+0avgdata 1148maxresident)k
0inputs+0outputs (0major+327minor)pagefaults 0swaps
gawk using execution through getline (Gnu AWK Manual)
/usr/bin/time gawk '/^[0-9]{1,10} [0-9]{1,4}/{l=$1+$2; "date --date=#"$1|getline d1; "date --date=#"l|getline d2;print d1" - "d2;next;}1' /tmp/test
...
1.89user 7.59system 0:10.34elapsed 91%CPU (0avgtext+0avgdata 5376maxresident)k
0inputs+0outputs (0major+557419minor)pagefaults 0swaps
Custom Bash script
./sed-2.0.sh /tmp/test
...
3.98user 10.33system 0:15.41elapsed 92%CPU (0avgtext+0avgdata 1536maxresident)k
0inputs+0outputs (0major+759829minor)pagefaults 0swaps
sed using e option
/usr/bin/time sed -r 's#^([0-9]{1,10}) ([0-9]{1,4})(.*$)#echo $(date --date=#\1 )" - "$(date --date=#$((\1+\2)))#ge' /tmp/test
...
3.88user 16.76system 0:21.89elapsed 94%CPU (0avgtext+0avgdata 1272maxresident)k
0inputs+0outputs (0major+1253409minor)pagefaults 0swaps
Input data
for N in `seq 1 1000`; do echo -e "$(( 1357226100 + ( $N * 3600 ) )) 3600 ...\nSomething else ..." >> /tmp/test ; done
We can see that AWK using the strffime() method is the fastest. But even the Bash script is faster than sed with shell execution.
Kent showed us a more generic, universal way to accomplish what I've asked for. My question actually was not only limited to my timestamp example. In this case I had to do exactly this (replacing timestamp + duration by human readable date representation), but I had situations where I had to execute other code.
glenn jackman showed us a specific solution which is suitable for situations were you can do string operations and calculation directly in AWK.
So, it depends on the time you have (or time your script may run), the amount of the data and use case which method should be preferred.
based on your sample input:
gawk '/^[0-9]+ [0-9]+ / {t1=$1; $1=strftime("%c -",t1); $2=strftime("%c",t1+$2)} 1'
outputs
Thu 03 Jan 2013 09:15:00 AM EST - Thu 03 Jan 2013 10:15:00 AM EST ...
Maybe intermediate strings...
Thu 03 Jan 2013 10:15:00 AM EST - Thu 03 Jan 2013 11:15:00 AM EST ...
Maybe intermediate strings...
...
awk oneliner: (the datetime format could be different from your output)
awk '/^[0-9]{1,10} [0-9]{1,4}/{l=$1+$2; "date --date=#"$1|getline d1; "date --date=#"l|getline d2;print d1" - "d2;next;}1' file
test:
kent$ echo "1357222500 3600 ...
Maybe intermediate strings...
1357226100 3600 ...
Maybe intermediate strings...
..."|awk '/^[0-9]{1,10} [0-9]{1,4}/{l=$1+$2; "date --date=#"$1|getline d1; "date --date=#"l|getline d2;print d1" - "d2;next;}1'
Thu Jan 3 15:15:00 CET 2013 - Thu Jan 3 16:15:00 CET 2013
Maybe intermediate strings...
Thu Jan 3 15:15:00 CET 2013 - Thu Jan 3 17:15:00 CET 2013
Maybe intermediate strings...
...
Gnu sed
if you have gnu sed, the idea from your "not working" sed line could work in real world by applying gnu sed's s/foo/shell cmds/ge see below:
sed -r 's#^([0-9]{1,10}) ([0-9]{1,4})(.*$)#echo $(date --date=#\1 )" - "$(date --date=#$((\1+\2)))#ge' file
test
kent$ echo "1357222500 3600 ...
Maybe intermediate strings...
1357226100 3600 ...
Maybe intermediate strings...
..."|sed -r 's#^([0-9]{1,10}) ([0-9]{1,4})(.*$)#echo $(date --date=#\1 )" - "$(date --date=#$((\1+\2)))#ge'
Thu Jan 3 15:15:00 CET 2013 - Thu Jan 3 16:15:00 CET 2013
Maybe intermediate strings...
Thu Jan 3 16:15:00 CET 2013 - Thu Jan 3 17:15:00 CET 2013
Maybe intermediate strings...
...
if I would work on this, personally I would go with awk. because it is straightforward and easy to write.
at the end I paste my sed/awk version info :
kent$ sed --version|head -1
sed (GNU sed) 4.2.2
kent$ awk -V|head -1
GNU Awk 4.0.1