After executing this
ta=`zcat abc.log.2019071814.gz |grep "R_MT"|grep "A:1234"|grep "ID:413"|awk -F"|" '{print $20}'|sort|uniq -c|awk '{$1=$1};1'`
Here $20 indicates the "S:" entry in each row (I am taking the unique count of all s values),I am getting result as
93070 S:1 11666 S:8 230 S:9
so what I need is the sum of all occurrence of s values .i.e 93070+11666+230
so result be total=104966
$ echo 93070 S:1 11666 S:8 230 S:9 | sed -E 's,S:[0-9]+,,g' | sed 's, ,+,g' | bc -
104966
Append to your last awk:
| awk '{sum+=$1} END {print sum}'
or use this (awk ignores columns with S:1, S:8 and S:9):
echo $ta | awk '{for(i=1;i<=NF;i++) t+=$i; print t; t=0}'
or use every second column:
echo $ta | awk '{for(i=1;i<=NF;i=i+2) t+=$i; print t; t=0}'
I won't help you all the way, but know that you can use bc to perform arithmetic.
echo "93070 + 11666 + 230" | bc
would give you:
104966
Related
I have the following script:
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
PorcUsado='scale=2;UsadoMem/TotalMem'|bc -l
echo $PorcUsado
The variable PorcUsado returns empty. I search for the use of bc, but something is wrong...
You're assigning PorcUsado to scale=2;UsadoMem/TotalMem and then piping the output of that assignment (nothing) into bc. You probably want the pipe inside a command substitution, e.g. (using a here string instead of a pipe):
PorcUsado=$(bc -l <<<'scale=2;UsadoMem/TotalMem')
But you'll also need to evaluate those shell variables - bc can't do it for you:
PorcUsado=$(bc -l <<<"scale=2;$UsadoMem/$TotalMem")
Notice the use of " instead of ' and the $ prefix to allow Bash to evaluate the variables.
Also, if this is the whole script, you can just skip the PorcUsado variable at all and let bc write directly to stdout.
#!/bin/bash
TotalMem=$(top -n 1 | grep Mem | awk 'NR==1{print $4}') #integer
UsadoMem=$(top -n 1 | grep Mem | awk 'NR==1{print $8}') #integer
bc -l <<<"scale=2;$UsadoMem/$TotalMem"
Why pipe top output at all? Seems too costly.
$ read used buffers < <(
awk -F':? +' '
{a[$1]=$2}
END {printf "%d %d", a["MemTotal"]-a["MemFree"], a["Buffers"]}
' /proc/meminfo
)
Of course, it can easily be a one-liner if you value brevity over readability.
I think the pipe is the problem try something like this:
PorcUsado=$(echo "scale=2;$UsadoMem/$TotalMem" | bc -l)
i haven't tested it yet but you have to echo the string and pipe the result from echo to bc.
EDIT: Correcting the variable names
You don't need grep or bc, since awk can search and do math all by itself:
top -n 1 -l 1 | awk '/Mem/ {printf "%0.2f\n",$8/$4;exit}'
I have a line in a bash script that calculates the sum of unique IP requests to a certain page.
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END { print " ", sum, "total"}'
I am trying to get the value of sum to a variable outside the awk statement so I can compare pages to each other. So far I have tried various combinations of something like this:
unique_sum=0
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print ; $unique_sum=sum} END { print " ", sum, "total"}'
echo "${unique_sum}"
This results in an echo of "0". I've tried placing __$unique_sum=sum__ in the END, various combinations of initializing the variable (awk -v unique_sum=0 ...) and placing the variable assignment outside of the quoted sections.
So far, my Google-fu is failing horribly as most people just send the whole of the output to a variable. In this example, many lines are printed (one for each IP) in addition to the total. Failing a way to capture the 'sum' variable, is there a way to capture that last line of output?
This is probably one of the most sophisticated things I've tried in awk so my confidence that I've done anything useful is pretty low. Any help will be greatly appreciated!
You can't assign a shell variable inside an awk program. In general, no child process can alter the environment of its parent. You have to have the awk program print out the calculated value, and then shell can grab that value and assign it to a variable:
output=$( grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}' | sort | uniq -c | awk '{sum += 1; print } END {print sum}' )
unique_sum=$( sed -n '$p' <<< "$output" ) # grab the last line of the output
sed '$d' <<< "$output" # print the output except for the last line
echo " $unique_sum total"
That pipeline can be simplified quite a lot: awk can do what grep can do, so first
grep $YESTERDAY $ACCESSLOG | grep "$1" | awk -F" - " '{print $1}'
is (longer, but only one process)
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '$0 ~ date && $0 ~ patt {print $1}' "$ACCESSLOG"
And the last awk program just counts how many lines and can be replaced with wc -l
All together:
unique_output=$(
awk -F" - " -v date="$YESTERDAY" -v patt="$1" '
$0 ~ date && $0 ~ patt {print $1}
' "$ACCESSLOG" | sort | uniq -c
)
echo "$unique_output"
unique_sum=$( wc -l <<< "$unique_output" )
echo " $unique_sum total"
I am new in bash
I wrote a bash script and it gives me an output like this:
3387 /test/file1
23688 /test/file2
5813 /test/file3
10415 /test/file4
1304 /test/file5
46 /test/file6
8 /test/file7
138 /test/file8
I can delimit them by
wc -l /path/to/$dir/test | cut -d" " -f1
how can I add numbers to eachother and caculate them?
can I do:
output=`wc -l /path/to/$dir/test | cut -d" " -f1`
Is it possible to use "while" or "for" loop and add those numbers?
how?
thank you in advance
You want awk here to avoid explicit loops. If your output was in the file data.txt you could use:
$ awk '{sum += $1} END {print sum}' data.txt
44799
In your case, pipe the output of your script to awk:
$ your_script.sh | awk '{sum += $1} END {print sum}'
Since the output you gave in your question was the output of wc -l, try:
$ wc -l /path/to/$dir/test | awk '{sum += $1} END {print sum}'
(Aside for anyone else landing on this page: wc -l, when given wildcards, will also give you a total, but it's great to use awk in this case because you can deal directly with the total line count and pipe just that to another process.)
I am using awk to get all of the 7th field separated by a comma.
1,root,init,20,0.0,0.1,0:01.78
1008,root,migration/0,1,2.0,1.8,7:04.32
26542,peter.ca,top,20,1.9,0.1,5:42.78
18776,john.sel,awk,15,0.2,0.0,0:01.00
49841,murray.s,topsum,16,1.3,0.1,0:02.11
29806,peter.ca,bash,20,0.1,0.0,0:00.03
9845,murray.s,awk,18,0.5,0.2,0:14.01
6549,peter.ca,topsum,16,1.4,0.2,0:04.81
18746,john.sel,bash,20,0.1,0.0,0:00.50
24987,john.bel,who,20,3.0,1.1,0:00.01
5,root,watchdog/0,1,0.0,0.0,0:00.00
29874,murray.s,timeon,20,0.1,0.2,0:02.15
cat data.top | grep root | awk 'BEGIN { FS = "," } ; { print $7 }'
Output:
0:01.78
7:04.32
0:00.00
I am trying to add these together (I am also assuming that the total will be less than 24 hours so that is not an issue).
Thanks in advance.
$ awk -F, '/root/{split($7, t, ":"); sum+=60*t[1]+t[2]}; END{print sum}' input.txt
426.1
$ date -ud #426.1 +%T.%N | sed 's/0*$//'
00:07:06.1
I am working with the zone.tab under /usr/share/zoneinfo/zonetab and I am having trouble displaying the data in a certain format.
the command I run:
cat zone.tab | awk '!/#/ {print $3}' | sort
this returns a list of regions and contries:
America/Washington
Pacific/Enderbury
What I need is for the above to return everything after the last / on each line.
There are some cases such as Pacific/Somewhere/A. I have a regex ([^/]+$) that should work but it doesn't. Any ideas?
You can also do it all in a single awk command:
awk '!/^#/ { sub(".*/", "", $3); print $3 }' /usr/share/zoneinfo/zone.tab
---- ----------------- --------
| | |
for non-comment lines | |
| |
modify 3rd col |
leaving only text |
after last slash |
|
Then print modified 3rd col
Pipe the output to sed -e 's;^.*/;;'. For example,
echo -e "America/Washington\nPacific/Enderbury" | sed 's;^.*/;;'
sed s:.*/:: /usr/share/zoneinfo/zone.tab
awk '!/^#/ { print $3;} ' < /usr/share/zoneinfo/zone.tab | awk -F/ ' { print $NF; }'
This regex might work:
# echo -e "a\n\a/b\na/b/c\na/b/c/d\n" |sed 's#^\(\([^/]*/\)*\)\(.*\)#\3#'
a
b
c
d
Perhaps sed -r 's#^(([^/]*/)*)(.*)#\3#' which removes the tangle of backslashes is clearer.