Calculate min, max, count in linux file [closed] - linux

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have a file on linux server that has data like :
a 22
a 10
a 17
a 51
a 33
b 51
b 47
c 33
I want a shell script or linux commands to find min, avg, 90%, max and count for each value in column 1.
Example:
for a min = 10, avg = 26, 90% = 33, max = 51, and count = 5.

Answer 1

Here a version with even the 90% percentile using gawk.
The definition of percentile is that one given by
Wikipedia and called Nearest rank.
The function round can be found here.
#!/bin/bash
gawk '
function round(x, ival, aval, fraction)
{
ival = int(x) # integer part, int() truncates
# see if fractional part
if (ival == x) # no fraction
return ival # ensure no decimals
if (x < 0) {
aval = -x # absolute value
ival = int(aval)
fraction = aval - ival
if (fraction >= .5)
return int(x) - 1 # -2.5 --> -3
else
return int(x) # -2.3 --> -2
} else {
fraction = x - ival
if (fraction >= .5)
return ival + 1
else
return ival
}
}
# the following block processes all the lines
# and populates counters and values
{
if($1 in counters) {
counters[$1]++;
} else {
counters[$1] = 1;
}
i = counters[$1];
values[$1, i] = $2;
} END {
for (c in counters) {
delete tmp;
min = values[c, 1];
max = values[c, 1];
sum = values[c, 1];
tmp[1] = values[c, 1];
for (i = 2; i <= counters[c]; i++) {
if (values[c, i] < min) min = values[c, i];
if (values[c, i] > max) max = values[c, i];
sum += values[c, i];
tmp[i] = values[c, i];
}
# The following 3 lines compute the percentile.
n = asort(tmp, tmp_sorted);
idx = round(0.9 * n + 0.5); # Nearest rank definition
percentile = tmp_sorted[idx];
# Output of the statistics for this group.
printf "for %s min = %d, avg = %f, 90 = %d,max = %d, count = %d\n", c, min, (sum / counters[c]), percentile, max, counters[c];
}
}'
To run execute:
./stats.sh < input.txt
I am assuming that the above script is named stats.sh and your input is saved in input.txt.
The output is:
for a min = 10, avg = 26.600000, 90 = 51,max = 51, count = 5
for b min = 47, avg = 49.000000, 90 = 51,max = 51, count = 2
for c min = 33, avg = 33.000000, 90 = 33,max = 33, count = 1
Here the explanation:
counters is an associative array, the key is the value in column 1
and the value is the number of values found in the input for each
value in column 1.
values is a two dimensional (value_in_column_one, counter_per_value)
array that keeps all the values grouped by value in column one.
At the end of the script the outermost loop goes trough all the values
found in column 1. The innermost for loop analyses all the values belonging
to a particular value in column 1 and it computes all the statics.

Answer 2

For lines starting with a, here's an awk script.
$ echo 'a 22
a 10
a 17
a 51
a 33
b 51
b 47
c 33' | awk 'BEGIN{n=0;s=0;};/^a/{n=n+1;s=s+$2;};END{print n;print s;print s/n;}'
5
133
26.6

Answer 3

Using awk:
awk 'NR==1{min=$1} {sum+=$2; if(min>=$2) min=$2; if(max<$2) max=$2}
END{printf("max=%d,min=%d,count=%d,avg=%.2f\n", max, min, NR, (sum/NR))}' file
max=51,min=10,count=8,avg=33.00
EDIT:
awk '$1 != v {
if (NR>1)
printf("For %s max=%d,min=%d,count=%d,avg=%.2f\n", v, max, min, k, (sum/k));
v=$1;
min=$2;
k=sum=max=0
}
{
k++;
sum+=$2;
if (min > $2)
min=$2;
if (max < $2)
max=$2
}
END {
printf("For %s max=%d,min=%d,count=%d,avg=%.2f\n", v, max, min, k, (sum/k))
}' < <(sort -n -k1,2 f)
OUTPUT:
For a max=51,min=10,count=5,avg=26.60
For b max=51,min=47,count=2,avg=49.00
For c max=33,min=33,count=1,avg=33.00