Compare numerical output of command bash - linux

I am running a script that bumps a parameter value in a file:
Before the script executes:
$cat file
revision 1.2 date: 2018/11/15 09:28:12; author: root; state: Exp;
lines: +1 -1
After running the script:
$cat file
revision 1.3 date: 2018/11/15 09:28:12; author: root; state: Exp;
lines: +1 -1 revision 1.2 date: 2018/11/15 09:28:12; author: root;
state: Exp; lines: +1 -1
I need a script that could checks if the version is updated/bumped as one version greater than the previous version, if yes it should echo the message as 'Script ran successfully'.
$grep -iw 'revision' file | head -1 | cut -d' ' -f2
1.2
$sh <script>
$grep -iw 'revision' file | head -1 | cut -d' ' -f2
1.3
$echo -e "Script ran successfully!!! \n"

Here's a function bumped that will compare to version strings and determine if they are in succession. I made a couple of assumptions in the comparison. The number of "decimal points" has to be the same. I also have left the door wide open for big errors , see my last test case. Consider this quick and dirty. It would probably be better to write logic to compare each decimal value and make sure that the last decimal place was one more than the previous. A more correct version would be able to compare a.b.c with d.e.f by first comparing a == d , then b == c , and then f == c + 1. However, depending on what you are trying to do, quick and dirty may be the easiest to maintain.
Without any additional discussion, here is the quick and dirty version:
function bumped() {
prev=$1
next=$2
PREV=$( sed 's/[^0-9]//g' <<<"$prev" )
NEXT=$( sed 's/[^0-9]//g' <<<"$next" )
[ $(( NEXT - PREV )) -eq 1 ]
}
function testcases() {
cat <<EOF
1.2 1.3 0
1.3 1.2 1
1.2.1 1.2.2 0
1.3 1.3.1 1
EOF
}
testcases | while
read prev next expected extra
do
bumped $prev $next
observed=$?
result="Success"
[ $observed -ne $expected ] && result="Failed"
echo "$result: bumped( $prev , $next ) = $observed ( expected: $expected )"
done
The above program yields these results:
Success: bumped( 1.2 , 1.3 ) = 0 ( expected: 0 )
Success: bumped( 1.3 , 1.2 ) = 1 ( expected: 1 )
Success: bumped( 1.2.1 , 1.2.2 ) = 0 ( expected: 0 )
Success: bumped( 1.3 , 1.3.1 ) = 1 ( expected: 1 )
Failed: bumped( 1.3.1.1 , 13.1.2 ) = 0 ( expected: 1 )
So clearly this is not an optimal solution, but it may get you where you need to go. If you are looking for something that is correct all the time, consider the logic discussed above.

Related

How can I detect a sequence of "hollows" (holes, lines not matching a pattern) bigger than n in a text file?

Case scenario:
$ cat Status.txt
1,connected
2,connected
3,connected
4,connected
5,connected
6,connected
7,disconnected
8,disconnected
9,disconnected
10,disconnected
11,disconnected
12,disconnected
13,disconnected
14,connected
15,connected
16,connected
17,disconnected
18,connected
19,connected
20,connected
21,disconnected
22,disconnected
23,disconnected
24,disconnected
25,disconnected
26,disconnected
27,disconnected
28,disconnected
29,disconnected
30,connected
As can be seen, there are "hollows", understanding them as lines with the "disconnected" value inside the sequence file.
I want, in fact, to detect these "holes", but it would be useful if I could set a minimum n of missing numbers in the sequence.
I.e: for ' n=5' a detectable hole would be the 7... 13 part, as there are at least 5 "disconnected" in a row on the sequence. However, the missing 17 should not be considered as detectable in this case. Again, at line 21 whe get a valid disconnection.
Something like:
$ detector Status.txt -n 5 --pattern connected
7
21
... that could be interpreted like:
- Missing more than 5 "connected" starting at 7.
- Missing more than 5 "connected" starting at 21.
I need to script this on Linux shell, so I was thinking about programing some loop, parsing strings and so on, but I feel like if this could be done by using linux shell tools and maybe some simpler programming. Is there a way?
Even when small programs like csvtool are a valid solution, some more common Linux commands (like grep, cut, awk, sed, wc... etc) could be worth for me when working with embedded devices.
#!/usr/bin/env bash
last_connected=0
min_hole_size=${1:-5} # default to 5, or take an argument from the command line
while IFS=, read -r num state; do
if [[ $state = connected ]]; then
if (( (num-last_connected) > (min_hole_size+1) )); then
echo "Found a hole running from $((last_connected + 1)) to $((num - 1))"
fi
last_connected=$num
fi
done
# Special case: Need to also handle a hole that's still open at EOF.
if [[ $state != connected ]] && (( num - last_connected > min_hole_size )); then
echo "Found a hole running from $((last_connected + 1)) to $num"
fi
...emits, given your file on stdin (./detect-holes <in.txt):
Found a hole running from 7 to 13
Found a hole running from 21 to 29
See:
BashFAQ #1 - How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
The conditional expression -- the [[ ]] syntax used to make it safe to do string comparisons without quoting expansions.
Arithmetic comparison syntax -- valid in $(( )) in all POSIX-compliant shells; also available without the expansion side effects as (( )) as a bash extension.
This is the perfect use case for awk, since the machinery of line reading, column splitting, and matching is all built in. The only tricky bit is getting the command line argument to your script, but it's not too bad:
#!/usr/bin/env bash
awk -v window="$1" -F, '
BEGIN { if (window=="") {window = 1} }
$2=="disconnected"{if (consecutive==0){start=NR}; consecutive++}
$2!="disconnected"{if (consecutive>window){print start}; consecutive=0}
END {if (consecutive>window){print start}}'
The window value is supplied as the first command line argument; left out, it defaults to 1, which means "display the start of gaps with at least two consecutive disconnections". Probably could have a better name. You can give it 0 to include single disconnections. Sample output below. (Note that I added series of 2 disconnections at the end to test the failure that Charles metions).
njv#organon:~/tmp$ ./tst.sh 0 < status.txt # any number of disconnections
7
17
21
31
njv#organon:~/tmp$ ./tst.sh < status.txt # at least 2 disconnections
7
21
31
njv#organon:~/tmp$ ./tst.sh 8 < status.txt # at least 9 disconnections
21
Awk solution:
detector.awk script:
#!/bin/awk -f
BEGIN { FS="," }
$2 == "disconnected"{
if (f && NR-c==nr) c++;
else { f=1; c++; nr=NR }
}
$2 == "connected"{
if (f) {
if (c > n) {
printf "- Missing more than 5 \042connected\042 starting at %d.\n", nr
}
f=c=0
}
}
Usage:
awk -f detector.awk -v n=5 status.txt
The output:
- Missing more than 5 "connected" starting at 7.
- Missing more than 5 "connected" starting at 21.

How to print something to the right-most of the console in Linux shell script

Say I want to search for "ERROR" within a bunch of log files.
I want to print one line for every file that contains "ERROR".
In each line, I want to print the log file path on the left-most edge while the number of "ERROR" on the right-most edge.
I tried using:
printf "%-50s %d" $filePath $errorNumber
...but it's not perfect, since the black console can vary greatly, and the file path sometimes can be quite long.
Just for the pleasure of the eyes, but I am simply incapable of doing so.
Can anyone help me to solve this problem?
Using bash and printf:
printf "%-$(( COLUMNS - ${#errorNumber} ))s%s" \
"$filePath" "$errorNumber"
How it works:
$COLUMNS is the shell's terminal width.
printf does left alignment by putting a - after the %. So printf "%-25s%s\n" foo bar prints "foo", then 22 spaces, then "bar".
bash uses the # as a parameter length variable prefix, so if x=foo, then ${#x} is 3.
Fancy version, suppose the two variables are longer than will fit in one column; if so print them on as many lines as are needed:
printf "%-$(( COLUMNS * ( 1 + ( ${#filePath} + ${#errorNumber} ) / COLUMNS ) \
- ${#errorNumber} ))s%s" "$filePath" "$errorNumber"
Generalized to a function. Syntax is printfLR foo bar, or printfLR < file:
printfLR() { if [ "$1" ] ; then echo "$#" ; else cat ; fi |
while read l r ; do
printf "%-$(( ( 1 + ( ${#l} + ${#r} ) / COLUMNS ) \
* COLUMNS - ${#r} ))s%s" "$l" "$r"
done ; }
Test with:
# command line args
printfLR foo bar
# stdin
fortune | tr -s ' \t' '\n\n' | paste - - | printfLR

bc truncate floating point number

How do I truncate a floating point number using bc
e.g if I do
echo '4.2-1.3' | bc
which outputs 2.9 how I get it to truncate/use floor to get 2
Use / operator.
echo '(4.2-1.3) / 1' | bc
Dividing by 1 works ok if scale is 0 (eg, if you start bc with bc and don't change scale) but fails if scale is positive (eg, if you start bc with bc -l or increase scale). (See transcript below.) For a general solution, use a trunc function like the following:
define trunc(x) { auto s; s=scale; scale=0; x=x/1; scale=s; return x }
Transcript that illustrates how divide by 1 by itself fails in the bc -l case, but how trunc function works ok at truncating toward zero:
> bc -l
bc 1.06.95
[etc...]
for (x=-4; x<4; x+=l(2)) { print x,"\t",x/1,"\n"}
-4 -4.00000000000000000000
-3.30685281944005469059 -3.30685281944005469059
-2.61370563888010938118 -2.61370563888010938118
-1.92055845832016407177 -1.92055845832016407177
-1.22741127776021876236 -1.22741127776021876236
-.53426409720027345295 -.53426409720027345295
.15888308335967185646 .15888308335967185646
.85203026391961716587 .85203026391961716587
1.54517744447956247528 1.54517744447956247528
2.23832462503950778469 2.23832462503950778469
2.93147180559945309410 2.93147180559945309410
3.62461898615939840351 3.62461898615939840351
define trunc(x) { auto s; s=scale; scale=0; x=x/1; scale=s; return x }
for (x=-4; x<4; x+=l(2)) { print x,"\t",trunc(x),"\n"}
-4 -4
-3.30685281944005469059 -3
-2.61370563888010938118 -2
-1.92055845832016407177 -1
-1.22741127776021876236 -1
-.53426409720027345295 0
.15888308335967185646 0
.85203026391961716587 0
1.54517744447956247528 1
2.23832462503950778469 2
2.93147180559945309410 2
3.62461898615939840351 3
Try the following solution. It will truncate anything after the decimal point without a problem:
echo 'x = 4.2 - 1.3; scale = 0; x / 1' | bc -l
echo 'x = l(101) / l(10); scale = 0; x / 1' | bc -l
You can make the code a tad shorter by performing calculations directly on the numbers:
echo 'scale = 0; (4.2 - 1.3) / 1' | bc -l
echo 'scale = 0; (l(101) / l(10)) / 1' | bc -l
In general, you can use this function to get only the integer part of a number:
define int(x) {
auto s;
s = scale;
scale = 0;
x /= 1; /* This will have the effect of truncating x to its integer value */
scale = s;
return (x);
}
Save that code into a file (let's call it int.bc) and run the following command:
echo 'int(4.2 - 1.3);' | bc -l int.bc
The variable governing the amount of decimals on division is scale.
So, if scale is 0 (the default), dividing by 1 would truncate to 0 decimals:
$ echo '(4.2-1.3) / 1 ' | bc
2
In other operations, the number of decimals is calculated from the scale (number of decimals) of each operand. In add, subtract and multiplication, for example, the resulting scale is the biggest of both:
$ echo ' 4.2 - 1.33333333 ' | bc
2.86666667
$ echo ' 4.2 - 1.333333333333333333 ' | bc
2.866666666666666667
$ echo ' 4.2000 * 1.33 ' | bc
5.5860
Instead, in division, the number of decimals is strictly equal to th evalue of the variable scale:
$ echo 'scale=0;4/3;scale=3;4/3;scale=10;4/3' | bc
1
1.333
1.3333333333
As the value of scale has to be restored, it is better to define a function (GNU syntax):
$ echo ' define int(x){ os=scale;scale=0;x=x/1;scale=os;return(x) }
int( 4.2-1.3 )' | bc
2
Or in older POSIX language:
$ echo ' define i(x){
o=scale;scale=0;x=x/1;scale=o;return(x)
}
i( 4.2-1.3 )' | bc
2
You say:
truncate/use floor
And those are not the same thing in all cases. The other answers so far only show you how to truncate (i.e. "truncate towards zero" i.e. "discard the part after the decimal").
For negative numbers, the behavior is different.
To wit:
truncate(-2.5) = -2
floor(-2.5) = -3
So, here is a floor function for bc:
# Note: trunc(x) is defined as noted elsewhere in the other answers
define floor(x) {
auto t
t=trunc(x)
if (t>x) {
return t-1
} else {
return t
}
}
Aside:
You can put this, and other helper functions, in a file. For instance, I have this alias in my shell:
alias bc='bc -l ~/.bcinit'
And so whenever I run bc, I get all my utility functions from ~/.bcinit available by default.
Also, there is a good list of bc functions here: http://phodd.net/gnu-bc/code/funcs.bc
You may do something like this:
$ printf "%.2f\n" $(echo "(4530 / 4116 - 1) * 100" | bc -l)
10.06
Here I am trying to find the % change. Not purely bc though.

SED command inside a loop

Hello: I have a lot of files called test-MR3000-1.txt to test-MR4000-1.nt, where the number in the name changes by 100 (i.e. I have 11 files),
$ ls test-MR*
test-MR3000-1.nt test-MR3300-1.nt test-MR3600-1.nt test-MR3900-1.nt
test-MR3100-1.nt test-MR3400-1.nt test-MR3700-1.nt test-MR4000-1.nt
test-MR3200-1.nt test-MR3500-1.nt test-MR3800-1.nt
and also a file called resonancia.kumac which in a couple on lines contains the string XXXX.
$ head resonancia.kumac
close 0
hist/delete 0
vect/delete *
h/file 1 test-MRXXXX-1.nt
sigma MR=XXXX
I want to execute a bash file which substitutes the strig XXXX in a file by a set of numbers obtained from the command ls *MR* | cut -b 8-11.
I found a post in which there are some suggestions. I try my own code
for i in `ls *MR* | cut -b 8-11`; do
sed -e "s/XXXX/$i/" resonancia.kumac >> proof.kumac
done
however, in the substitution the numbers are surrounded by sigle qoutes (e.g. '3000').
Q: What should I do to avoid the single quote in the set of numbers? Thank you.
This is a reproducer for the environment described:
for ((i=3000; i<=4000; i+=100)); do
touch test-MR${i}-1.nt
done
cat >resonancia.kumac <<'EOF'
close 0
hist/delete 0
vect/delete *
h/file 1 test-MRXXXX-1.nt
sigma MR=XXXX
EOF
This is a script which will run inside that environment:
content="$(<resonancia.kumac)"
for f in *MR*; do
substring=${f:7:3}
echo "${content//XXXX/$substring}"
done >proof.kumac
...and the output looks like so:
close 0
hist/delete 0
vect/delete *
h/file 1 test-MR300-1.nt
sigma MR=300
There are no quotes anywhere in this output; the problem described is not reproduced.
or if it could be perl:
#!/usr/bin/perl
#ls = glob('*MR*');
open (FILE, 'resonancia.kumac') || die("not good\n");
#cont = <FILE>;
$f = shift(#ls);
$f =~ /test-MR([0-9]*)-1\.nt/;
$nr = $1;
#out = ();
foreach $l (#cont){
if($l =~ s/XXXX/$nr/){
$f = shift(#ls);
$f =~ /test-MR([0-9]*)-1\.nt/;
$nr = $1;
}
push #out, $l;
}
close FILE;
open FILE, '>resonancia.kumac' || die("not good\n");
print FILE #out;
That would replace the first XXXX with the first filename, what seemed to be the question before change.

Finding minimum and maximum values of the first column - grouped by the second column

I have lots of unsorted data in text file in the following form:
1.0 10
1.8 10
1.1 10
1.9 20
2.8 20
2.1 20
2.9 20
...
For each value in the second column, I want to get the interval of values in the first column. So for the example above, the result should be
1.0 1.8 10
1.9 2.9 20
How can I do this with c/c++, awk or other linux shell tools?
You can use this awk:
awk '{
if (!($2 in nmin) || $1<nmin[$2])
nmin[$2]=$1;
else if ($1>=nmax[$2])
nmax[$2]=$1
}
END {
for (a in nmin)
print nmin[a], nmax[a], a
}
' inFile
this one-liner should work for you:
awk '!($2 in i){i[$2]=$1}{a[$2]=$1}END{for(x in i)print i[x],a[x],x}' file
output:
1.0 1.8 10
1.9 2.9 20
I think this should work:
{ read vStart int &&
while read vNext nextInt; do
if [ $int -ne $nextInt ]; then
echo "$vStart $v $int";
vStart=$vNext;
fi
v=$vNext;
int=$nextInt;
done &&
echo "$vStart $v $int"; }
To add another alternative, you could do this in R as well:
d.in <- read.table(file = commandArgs(trailingOnly = T)[1]);
write.table(
aggregate(V1 ~ V2, d.in, function (x) c(min(x),max(x)))[,c(2,1)]
, row.names = F
, col.names = F
, sep = "\t");
Then just call this script with Rscript:
$ Rscript script.R data.txt
1 1.8 10
1.9 2.9 20

Resources