compare float numbers with integers to get output in shell programing? [duplicate] - linux

I have this Bash script and I had a problem in line 16.
How can I take the previous result of line 15 and add
it to the variable in line 16?
#!/bin/bash
num=0
metab=0
for ((i=1; i<=2; i++)); do
for j in `ls output-$i-*`; do
echo "$j"
metab=$(cat $j|grep EndBuffer|awk '{sum+=$2} END { print sum/120}') (line15)
num= $num + $metab (line16)
done
echo "$num"
done

For integers:
Use arithmetic expansion: $((EXPR))
num=$((num1 + num2))
num=$(($num1 + $num2)) # Also works
num=$((num1 + 2 + 3)) # ...
num=$[num1+num2] # Old, deprecated arithmetic expression syntax
Using the external expr utility. Note that this is only needed for really old systems.
num=`expr $num1 + $num2` # Whitespace for expr is important
For floating point:
Bash doesn't directly support this, but there are a couple of external tools you can use:
num=$(awk "BEGIN {print $num1+$num2; exit}")
num=$(python -c "print $num1+$num2")
num=$(perl -e "print $num1+$num2")
num=$(echo $num1 + $num2 | bc) # Whitespace for echo is important
You can also use scientific notation (for example, 2.5e+2).
Common pitfalls:
When setting a variable, you cannot have whitespace on either side of =, otherwise it will force the shell to interpret the first word as the name of the application to run (for example, num= or num)
num= 1 num =2
bc and expr expect each number and operator as a separate argument, so whitespace is important. They cannot process arguments like 3+ +4.
num=`expr $num1+ $num2`

Use the $(( )) arithmetic expansion.
num=$(( $num + $metab ))
See Chapter 13. Arithmetic Expansion for more information.

There are a thousand and one ways to do it. Here's one using dc (a reverse Polish desk calculator which supports unlimited precision arithmetic):
dc <<<"$num1 $num2 + p"
But if that's too bash-y for you (or portability matters) you could say
echo $num1 $num2 + p | dc
But maybe you're one of those people who thinks RPN is icky and weird; don't worry! bc is here for you:
bc <<< "$num1 + $num2"
echo $num1 + $num2 | bc
That said, there are some unrelated improvements you could be making to your script:
#!/bin/bash
num=0
metab=0
for ((i=1; i<=2; i++)); do
for j in output-$i-* ; do # 'for' can glob directly, no need to ls
echo "$j"
# 'grep' can read files, no need to use 'cat'
metab=$(grep EndBuffer "$j" | awk '{sum+=$2} END { print sum/120}')
num=$(( $num + $metab ))
done
echo "$num"
done
As described in Bash FAQ 022, Bash does not natively support floating point numbers. If you need to sum floating point numbers the use of an external tool (like bc or dc) is required.
In this case the solution would be
num=$(dc <<<"$num $metab + p")
To add accumulate possibly-floating-point numbers into num.

In Bash,
num=5
x=6
(( num += x ))
echo $num # ==> 11
Note that Bash can only handle integer arithmetic, so if your AWK command returns a fraction, then you'll want to redesign: here's your code rewritten a bit to do all math in AWK.
num=0
for ((i=1; i<=2; i++)); do
for j in output-$i-*; do
echo "$j"
num=$(
awk -v n="$num" '
/EndBuffer/ {sum += $2}
END {print n + (sum/120)}
' "$j"
)
done
echo "$num"
done

I always forget the syntax so I come to Google Search, but then I never find the one I'm familiar with :P. This is the cleanest to me and more true to what I'd expect in other languages.
i=0
((i++))
echo $i;

I really like this method as well. There is less clutter:
count=$[count+1]

#!/bin/bash
read X
read Y
echo "$(($X+$Y))"

You should declare metab as integer and then use arithmetic evaluation
declare -i metab num
...
num+=metab
...
For more information, see 6.5 Shell Arithmetic.

Use the shell built-in let. It is similar to (( expr )):
A=1
B=1
let "C = $A + $B"
echo $C # C == 2
Source: Bash let builtin command

Another portable POSIX compliant way to do in Bash, which can be defined as a function in .bashrc for all the arithmetic operators of convenience.
addNumbers () {
local IFS='+'
printf "%s\n" "$(( $* ))"
}
and just call it in command-line as,
addNumbers 1 2 3 4 5 100
115
The idea is to use the Input-Field-Separator(IFS), a special variable in Bash used for word splitting after expansion and to split lines into words. The function changes the value locally to use word-splitting character as the sum operator +.
Remember the IFS is changed locally and does not take effect on the default IFS behaviour outside the function scope. An excerpt from the man bash page,
The shell treats each character of IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly , the default, then sequences of , , and at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words.
The "$(( $* ))" represents the list of arguments passed to be split by + and later the sum value is output using the printf function. The function can be extended to add scope for other arithmetic operations also.

#!/usr/bin/bash
#integer numbers
#===============#
num1=30
num2=5
echo $(( num1 + num2 ))
echo $(( num1-num2 ))
echo $(( num1*num2 ))
echo $(( num1/num2 ))
echo $(( num1%num2 ))
read -p "Enter first number : " a
read -p "Enter second number : " b
# we can store the result
result=$(( a+b ))
echo sum of $a \& $b is $result # \ is used to espace &
#decimal numbers
#bash only support integers so we have to delegate to a tool such as bc
#==============#
num2=3.4
num1=534.3
echo $num1+$num2 | bc
echo $num1-$num2 | bc
echo $num1*$num2 |bc
echo "scale=20;$num1/$num2" | bc
echo $num1%$num2 | bc
# we can store the result
#result=$( ( echo $num1+$num2 ) | bc )
result=$( echo $num1+$num2 | bc )
echo result is $result
##Bonus##
#Calling built in methods of bc
num=27
echo "scale=2;sqrt($num)" | bc -l # bc provides support for calculating square root
echo "scale=2;$num^3" | bc -l # calculate power

#!/bin/bash
num=0
metab=0
for ((i=1; i<=2; i++)); do
for j in `ls output-$i-*`; do
echo "$j"
metab=$(cat $j|grep EndBuffer|awk '{sum+=$2} END { print sum/120}') (line15)
let num=num+metab (line 16)
done
echo "$num"
done

Works on MacOS. bc is a command line calculator
#!/bin/bash
sum=0
for (( i=1; i<=5; i++ )); do
sum=$(echo "$sum + 1.1" | bc) # bc: if you want to use decimal
done
echo "Total: $sum"

Related

Operand Error in For loop in BASH script? [duplicate]

How do I iterate over a range of numbers in Bash when the range is given by a variable?
I know I can do this (called "sequence expression" in the Bash documentation):
for i in {1..5}; do echo $i; done
Which gives:
1
2
3
4
5
Yet, how can I replace either of the range endpoints with a variable? This doesn't work:
END=5
for i in {1..$END}; do echo $i; done
Which prints:
{1..5}
for i in $(seq 1 $END); do echo $i; done
edit: I prefer seq over the other methods because I can actually remember it ;)
The seq method is the simplest, but Bash has built-in arithmetic evaluation.
END=5
for ((i=1;i<=END;i++)); do
echo $i
done
# ==> outputs 1 2 3 4 5 on separate lines
The for ((expr1;expr2;expr3)); construct works just like for (expr1;expr2;expr3) in C and similar languages, and like other ((expr)) cases, Bash treats them as arithmetic.
discussion
Using seq is fine, as Jiaaro suggested. Pax Diablo suggested a Bash loop to avoid calling a subprocess, with the additional advantage of being more memory friendly if $END is too large. Zathrus spotted a typical bug in the loop implementation, and also hinted that since i is a text variable, continuous conversions to-and-fro numbers are performed with an associated slow-down.
integer arithmetic
This is an improved version of the Bash loop:
typeset -i i END
let END=5 i=1
while ((i<=END)); do
echo $i
…
let i++
done
If the only thing that we want is the echo, then we could write echo $((i++)).
ephemient taught me something: Bash allows for ((expr;expr;expr)) constructs. Since I've never read the whole man page for Bash (like I've done with the Korn shell (ksh) man page, and that was a long time ago), I missed that.
So,
typeset -i i END # Let's be explicit
for ((i=1;i<=END;++i)); do echo $i; done
seems to be the most memory-efficient way (it won't be necessary to allocate memory to consume seq's output, which could be a problem if END is very large), although probably not the “fastest”.
the initial question
eschercycle noted that the {a..b} Bash notation works only with literals; true, accordingly to the Bash manual. One can overcome this obstacle with a single (internal) fork() without an exec() (as is the case with calling seq, which being another image requires a fork+exec):
for i in $(eval echo "{1..$END}"); do
Both eval and echo are Bash builtins, but a fork() is required for the command substitution (the $(…) construct).
Here is why the original expression didn't work.
From man bash:
Brace expansion is performed before
any other expansions, and any
characters special to other
expansions are preserved in the
result. It is strictly textual. Bash
does not apply any syntactic
interpretation to the context of
the expansion or the text between the
braces.
So, brace expansion is something done early as a purely textual macro operation, before parameter expansion.
Shells are highly optimized hybrids between macro processors and more formal programming languages. In order to optimize the typical use cases, the language is made rather more complex and some limitations are accepted.
Recommendation
I would suggest sticking with Posix1 features. This means using for i in <list>; do, if the list is already known, otherwise, use while or seq, as in:
#!/bin/sh
limit=4
i=1; while [ $i -le $limit ]; do
echo $i
i=$(($i + 1))
done
# Or -----------------------
for i in $(seq 1 $limit); do
echo $i
done
1. Bash is a great shell and I use it interactively, but I don't put bash-isms into my scripts. Scripts might need a faster shell, a more secure one, a more embedded-style one. They might need to run on whatever is installed as /bin/sh, and then there are all the usual pro-standards arguments. Remember shellshock, aka bashdoor?
The POSIX way
If you care about portability, use the example from the POSIX standard:
i=2
end=5
while [ $i -le $end ]; do
echo $i
i=$(($i+1))
done
Output:
2
3
4
5
Things which are not POSIX:
(( )) without dollar, although it is a common extension as mentioned by POSIX itself.
[[. [ is enough here. See also: What is the difference between single and double square brackets in Bash?
for ((;;))
seq (GNU Coreutils)
{start..end}, and that cannot work with variables as mentioned by the Bash manual.
let i=i+1: POSIX 7 2. Shell Command Language does not contain the word let, and it fails on bash --posix 4.3.42
the dollar at i=$i+1 might be required, but I'm not sure. POSIX 7 2.6.4 Arithmetic Expansion says:
If the shell variable x contains a value that forms a valid integer constant, optionally including a leading plus or minus sign, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.
but reading it literally that does not imply that $((x+1)) expands since x+1 is not a variable.
You can use
for i in $(seq $END); do echo $i; done
Another layer of indirection:
for i in $(eval echo {1..$END}); do
∶
I've combined a few of the ideas here and measured performance.
TL;DR Takeaways:
seq and {..} are really fast
for and while loops are slow
$( ) is slow
for (( ; ; )) loops are slower
$(( )) is even slower
Worrying about N numbers in memory (seq or {..}) is silly (at least up to 1 million.)
These are not conclusions. You would have to look at the C code behind each of these to draw conclusions. This is more about how we tend to use each of these mechanisms for looping over code. Most single operations are close enough to being the same speed that it's not going to matter in most cases. But a mechanism like for (( i=1; i<=1000000; i++ )) is many operations as you can visually see. It is also many more operations per loop than you get from for i in $(seq 1 1000000). And that may not be obvious to you, which is why doing tests like this is valuable.
Demos
# show that seq is fast
$ time (seq 1 1000000 | wc)
1000000 1000000 6888894
real 0m0.227s
user 0m0.239s
sys 0m0.008s
# show that {..} is fast
$ time (echo {1..1000000} | wc)
1 1000000 6888896
real 0m1.778s
user 0m1.735s
sys 0m0.072s
# Show that for loops (even with a : noop) are slow
$ time (for i in {1..1000000} ; do :; done | wc)
0 0 0
real 0m3.642s
user 0m3.582s
sys 0m0.057s
# show that echo is slow
$ time (for i in {1..1000000} ; do echo $i; done | wc)
1000000 1000000 6888896
real 0m7.480s
user 0m6.803s
sys 0m2.580s
$ time (for i in $(seq 1 1000000) ; do echo $i; done | wc)
1000000 1000000 6888894
real 0m7.029s
user 0m6.335s
sys 0m2.666s
# show that C-style for loops are slower
$ time (for (( i=1; i<=1000000; i++ )) ; do echo $i; done | wc)
1000000 1000000 6888896
real 0m12.391s
user 0m11.069s
sys 0m3.437s
# show that arithmetic expansion is even slower
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; i=$(($i+1)); done | wc)
1000000 1000000 6888896
real 0m19.696s
user 0m18.017s
sys 0m3.806s
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; ((i=i+1)); done | wc)
1000000 1000000 6888896
real 0m18.629s
user 0m16.843s
sys 0m3.936s
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $((i++)); done | wc)
1000000 1000000 6888896
real 0m17.012s
user 0m15.319s
sys 0m3.906s
# even a noop is slow
$ time (i=1; e=1000000; while [ $((i++)) -le $e ]; do :; done | wc)
0 0 0
real 0m12.679s
user 0m11.658s
sys 0m1.004s
If you need it prefix than you might like this
for ((i=7;i<=12;i++)); do echo `printf "%2.0d\n" $i |sed "s/ /0/"`;done
that will yield
07
08
09
10
11
12
If you're on BSD / OS X you can use jot instead of seq:
for i in $(jot $END); do echo $i; done
This works fine in bash:
END=5
i=1 ; while [[ $i -le $END ]] ; do
echo $i
((i = i + 1))
done
There are many ways to do this, however the ones I prefer is given below
Using seq
Synopsis from man seq
$ seq [-w] [-f format] [-s string] [-t string] [first [incr]] last
Syntax
Full command
seq first incr last
first is starting number in the sequence [is optional, by default:1]
incr is increment [is optional, by default:1]
last is the last number in the sequence
Example:
$ seq 1 2 10
1 3 5 7 9
Only with first and last:
$ seq 1 5
1 2 3 4 5
Only with last:
$ seq 5
1 2 3 4 5
Using {first..last..incr}
Here first and last are mandatory and incr is optional
Using just first and last
$ echo {1..5}
1 2 3 4 5
Using incr
$ echo {1..10..2}
1 3 5 7 9
You can use this even for characters like below
$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
I know this question is about bash, but - just for the record - ksh93 is smarter and implements it as expected:
$ ksh -c 'i=5; for x in {1..$i}; do echo "$x"; done'
1
2
3
4
5
$ ksh -c 'echo $KSH_VERSION'
Version JM 93u+ 2012-02-29
$ bash -c 'i=5; for x in {1..$i}; do echo "$x"; done'
{1..5}
This is another way:
end=5
for i in $(bash -c "echo {1..${end}}"); do echo $i; done
If you want to stay as close as possible to the brace-expression syntax, try out the range function from bash-tricks' range.bash.
For example, all of the following will do the exact same thing as echo {1..10}:
source range.bash
one=1
ten=10
range {$one..$ten}
range $one $ten
range {1..$ten}
range {1..10}
It tries to support the native bash syntax with as few "gotchas" as possible: not only are variables supported, but the often-undesirable behavior of invalid ranges being supplied as strings (e.g. for i in {1..a}; do echo $i; done) is prevented as well.
The other answers will work in most cases, but they all have at least one of the following drawbacks:
Many of them use subshells, which can harm performance and may not be possible on some systems.
Many of them rely on external programs. Even seq is a binary which must be installed to be used, must be loaded by bash, and must contain the program you expect, for it to work in this case. Ubiquitous or not, that's a lot more to rely on than just the Bash language itself.
Solutions that do use only native Bash functionality, like #ephemient's, will not work on alphabetic ranges, like {a..z}; brace expansion will. The question was about ranges of numbers, though, so this is a quibble.
Most of them aren't visually similar to the {1..10} brace-expanded range syntax, so programs that use both may be a tiny bit harder to read.
#bobbogo's answer uses some of the familiar syntax, but does something unexpected if the $END variable is not a valid range "bookend" for the other side of the range. If END=a, for example, an error will not occur and the verbatim value {1..a} will be echoed. This is the default behavior of Bash, as well--it is just often unexpected.
Disclaimer: I am the author of the linked code.
These are all nice but seq is supposedly deprecated and most only work with numeric ranges.
If you enclose your for loop in double quotes, the start and end variables will be dereferenced when you echo the string, and you can ship the string right back to BASH for execution. $i needs to be escaped with \'s so it is NOT evaluated before being sent to the subshell.
RANGE_START=a
RANGE_END=z
echo -e "for i in {$RANGE_START..$RANGE_END}; do echo \\${i}; done" | bash
This output can also be assigned to a variable:
VAR=`echo -e "for i in {$RANGE_START..$RANGE_END}; do echo \\${i}; done" | bash`
The only "overhead" this should generate should be the second instance of bash so it should be suitable for intensive operations.
Replace {} with (( )):
tmpstart=0;
tmpend=4;
for (( i=$tmpstart; i<=$tmpend; i++ )) ; do
echo $i ;
done
Yields:
0
1
2
3
4
If you're doing shell commands and you (like I) have a fetish for pipelining, this one is good:
seq 1 $END | xargs -I {} echo {}
if you don't wanna use 'seq' or 'eval' or jot or arithmetic expansion format eg. for ((i=1;i<=END;i++)), or other loops eg. while, and you don't wanna 'printf' and happy to 'echo' only, then this simple workaround might fit your budget:
a=1; b=5; d='for i in {'$a'..'$b'}; do echo -n "$i"; done;' echo "$d" | bash
PS: My bash doesn't have 'seq' command anyway.
Tested on Mac OSX 10.6.8, Bash 3.2.48
This works in Bash and Korn, also can go from higher to lower numbers. Probably not fastest or prettiest but works well enough. Handles negatives too.
function num_range {
# Return a range of whole numbers from beginning value to ending value.
# >>> num_range start end
# start: Whole number to start with.
# end: Whole number to end with.
typeset s e v
s=${1}
e=${2}
if (( ${e} >= ${s} )); then
v=${s}
while (( ${v} <= ${e} )); do
echo ${v}
((v=v+1))
done
elif (( ${e} < ${s} )); then
v=${s}
while (( ${v} >= ${e} )); do
echo ${v}
((v=v-1))
done
fi
}
function test_num_range {
num_range 1 3 | egrep "1|2|3" | assert_lc 3
num_range 1 3 | head -1 | assert_eq 1
num_range -1 1 | head -1 | assert_eq "-1"
num_range 3 1 | egrep "1|2|3" | assert_lc 3
num_range 3 1 | head -1 | assert_eq 3
num_range 1 -1 | tail -1 | assert_eq "-1"
}

How to write script for arithmetic mean of even numbers

I have to write a script in bash where I have to provide 10 numbers to the table. Then script have to write the content out and arithmetic mean of even numbers. I did like 90% of the script, but I can't find out how to extract information about quantity of even numbers that is needed for arithmetic mean.
Here is my code:
echo "Provide data:"
i=0
for (( i = 0 ; i < 10; i++ ))
do
echo "Provide $[$i+1] number:"
read x
if [ "$x" = "" ]
then
break
else
table[$i]=$x
fi
done
echo "Provided data: ${table[*]}"
result=0
for (( i = 0 ; i < 10; i++))
do
res=$[${table[i]}%2]
if [ $res -eq 0 ]
then
echo "Number ${table[i]} is even"
result=$[$result+${table[$i]}]
fi
done
echo "SUM:$[$result]"
ignoring data input adding only odd inputs can look like :
$ cat c.sh
#!/bin/bash
declare -A xDarray
sum=0
xDarray[0 1]=1
xDarray[0 2]=3
xDarray[1 0]=2
xDarray[2 0]=4
for var in ${xDarray[#]}
do
if [ $(( $var & 1 )) == 0 ] ; then
echo $var is even
i+=1
tab[$i]=$var
sum=$(( $sum + $var))
fi
done
var=$(echo ${tab[#]} | sed 's/ / + /g' )
echo $var = $sum
result in
$ ./c.sh
2 is even
4 is even
2 + 4 = 6
$
whatever the number of data is used it would work
I let you work around your data input
Here are some suggested modifications for your script, syntax is simple.
#!/bin/bash
arr=()
for (( i=1;i<=10;i++ )); do
number=''
while [[ ! $number =~ ^[0-9]+$ ]]; do
printf "Please enter number $i:\n"
read number
done
arr+=($number)
done
printf "\nProvided numbers:"
printf " %d" "${arr[#]}"
printf "\nEven numbers:"
s=0
n=0
for x in "${arr[#]}"; do
if ! (( x % 2 )); then
printf " %d" "$x"
s=$(( s + x ))
(( n++ ))
fi
done
m=$(( s / n ))
printf "\nMean of the %d even numbers: %d / %d = %d\n" "$n" "$s" "$n" "$m"
Use an array arr to hold the input numbers, declare with arr=(), append numbers with arr+=($x), we refer only ${arr[#]} for all the items and we avoid any other complex array references, indices etc.
Every input number is tested against regular expression ^[0-9]+$ which means one or more digits (and only digits) with the =~ operator, and if this is not true, we prompt again for the same i-th input number.
Also we prefer printf for printing.
The last loop is the standard array loop, where we use again the arithmetic expansion syntax to find the even numbers, to add them to the sum and get the mean of them (result is an integer).
If you want to print a decimal result, e.g. with 2 floating points, you could use bc and printf "%f" like this:
m=$( bc <<< "scale=2; $s/$n" )
printf "%.2f" "$m"

Fibonacci sequence calculator

I have an issue with some homework, so basically we were asked to create a bash script that takes a variable greater than 2 and gives back the same number of Fibonacci sequence numbers, i.o. if i were to give 5 it would print:
0 1 1 2 3
I have done some research and have come up with a unique idea i havent really seen anywhere online, (keep in mind i have some experience in python) but i have run into a problem, so basically i use a for expression in bash and 2 variables to calculate the sequence, i first manually set them to be 0 and 1 (the first and second numbers in the sequence) and then i add them together while changing their values,
So here is the code :
#!/bin/bash
a=0
b=1
for i in $(seq 1 $1);
do
if ["$a" -gt "$b"]
then
b=($a +$b)
echo "$b"
else
a=($a +$b)
echo $a
fi
done
I have many issues and expect to be completely out of context but i hope you get the general idea and can help guide me through the problem :/
It seems like i want something to be treated as a number but its treated as text... Not sure though, any help is highly appreciated
You were very close, some small syntactic changes and new initial values make it work.
Notes:
whitespace is important in bash, especially with special characters [ ] ;
also, calculation with integer number can be forced using let
Nice idea to use a comparison to do the calculation with just two variables :)
#!/bin/bash -
a=1
b=0
echo $a
for i in $(seq 1 $1);
do
if [ $a -gt $b ] ; then
let b=($a +$b)
echo $b
else
let a=($a +$b)
echo $a
fi
done
there is an arithmetic context in bash.
slight re-write can be
$ a=0; b=1;
for i in {1..9};
do c=$((a+b));
echo $c;
if ((a>b)); then b=$c; else a=$c; fi;
done
1
2
3
5
8
13
21
34
55
You need a space
if [ "$a" -gt "$b" ]
and you can do arithmetic evaluation using
b=$((a+b))
once the script runs, verify the result is what you are expecting (i.e. does it print 0?)
Use double parens to evaluate arithmetic expressions. Instead of:
if [ "$a" -gt "$b" ]
write:
if ((a > b))
and instead of:
($a + $b)
write:
$((a + b))
You can also write the for header this way:
for ((i = 0; i < $1; i++))
But this is not required in this case. Your loop then becomes:
for ((i = 0; i < $1; i++)); do
if ((a > b)); then
b=$((a + b))
echo "$b"
else
a=$((a + b))
echo "$a"
fi
done
#!/bin/bash
a=0
b=1
echo $a
c=$1
for i in $(seq 1 $((c-1)));
do
if [ $a -gt $b ]; then
b=$((a+b))
echo $b
else
a=$((a+b))
echo $a
fi
done

Running pi.sh using for loops for N times [duplicate]

How do I iterate over a range of numbers in Bash when the range is given by a variable?
I know I can do this (called "sequence expression" in the Bash documentation):
for i in {1..5}; do echo $i; done
Which gives:
1
2
3
4
5
Yet, how can I replace either of the range endpoints with a variable? This doesn't work:
END=5
for i in {1..$END}; do echo $i; done
Which prints:
{1..5}
for i in $(seq 1 $END); do echo $i; done
edit: I prefer seq over the other methods because I can actually remember it ;)
The seq method is the simplest, but Bash has built-in arithmetic evaluation.
END=5
for ((i=1;i<=END;i++)); do
echo $i
done
# ==> outputs 1 2 3 4 5 on separate lines
The for ((expr1;expr2;expr3)); construct works just like for (expr1;expr2;expr3) in C and similar languages, and like other ((expr)) cases, Bash treats them as arithmetic.
discussion
Using seq is fine, as Jiaaro suggested. Pax Diablo suggested a Bash loop to avoid calling a subprocess, with the additional advantage of being more memory friendly if $END is too large. Zathrus spotted a typical bug in the loop implementation, and also hinted that since i is a text variable, continuous conversions to-and-fro numbers are performed with an associated slow-down.
integer arithmetic
This is an improved version of the Bash loop:
typeset -i i END
let END=5 i=1
while ((i<=END)); do
echo $i
…
let i++
done
If the only thing that we want is the echo, then we could write echo $((i++)).
ephemient taught me something: Bash allows for ((expr;expr;expr)) constructs. Since I've never read the whole man page for Bash (like I've done with the Korn shell (ksh) man page, and that was a long time ago), I missed that.
So,
typeset -i i END # Let's be explicit
for ((i=1;i<=END;++i)); do echo $i; done
seems to be the most memory-efficient way (it won't be necessary to allocate memory to consume seq's output, which could be a problem if END is very large), although probably not the “fastest”.
the initial question
eschercycle noted that the {a..b} Bash notation works only with literals; true, accordingly to the Bash manual. One can overcome this obstacle with a single (internal) fork() without an exec() (as is the case with calling seq, which being another image requires a fork+exec):
for i in $(eval echo "{1..$END}"); do
Both eval and echo are Bash builtins, but a fork() is required for the command substitution (the $(…) construct).
Here is why the original expression didn't work.
From man bash:
Brace expansion is performed before
any other expansions, and any
characters special to other
expansions are preserved in the
result. It is strictly textual. Bash
does not apply any syntactic
interpretation to the context of
the expansion or the text between the
braces.
So, brace expansion is something done early as a purely textual macro operation, before parameter expansion.
Shells are highly optimized hybrids between macro processors and more formal programming languages. In order to optimize the typical use cases, the language is made rather more complex and some limitations are accepted.
Recommendation
I would suggest sticking with Posix1 features. This means using for i in <list>; do, if the list is already known, otherwise, use while or seq, as in:
#!/bin/sh
limit=4
i=1; while [ $i -le $limit ]; do
echo $i
i=$(($i + 1))
done
# Or -----------------------
for i in $(seq 1 $limit); do
echo $i
done
1. Bash is a great shell and I use it interactively, but I don't put bash-isms into my scripts. Scripts might need a faster shell, a more secure one, a more embedded-style one. They might need to run on whatever is installed as /bin/sh, and then there are all the usual pro-standards arguments. Remember shellshock, aka bashdoor?
The POSIX way
If you care about portability, use the example from the POSIX standard:
i=2
end=5
while [ $i -le $end ]; do
echo $i
i=$(($i+1))
done
Output:
2
3
4
5
Things which are not POSIX:
(( )) without dollar, although it is a common extension as mentioned by POSIX itself.
[[. [ is enough here. See also: What is the difference between single and double square brackets in Bash?
for ((;;))
seq (GNU Coreutils)
{start..end}, and that cannot work with variables as mentioned by the Bash manual.
let i=i+1: POSIX 7 2. Shell Command Language does not contain the word let, and it fails on bash --posix 4.3.42
the dollar at i=$i+1 might be required, but I'm not sure. POSIX 7 2.6.4 Arithmetic Expansion says:
If the shell variable x contains a value that forms a valid integer constant, optionally including a leading plus or minus sign, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.
but reading it literally that does not imply that $((x+1)) expands since x+1 is not a variable.
You can use
for i in $(seq $END); do echo $i; done
Another layer of indirection:
for i in $(eval echo {1..$END}); do
∶
I've combined a few of the ideas here and measured performance.
TL;DR Takeaways:
seq and {..} are really fast
for and while loops are slow
$( ) is slow
for (( ; ; )) loops are slower
$(( )) is even slower
Worrying about N numbers in memory (seq or {..}) is silly (at least up to 1 million.)
These are not conclusions. You would have to look at the C code behind each of these to draw conclusions. This is more about how we tend to use each of these mechanisms for looping over code. Most single operations are close enough to being the same speed that it's not going to matter in most cases. But a mechanism like for (( i=1; i<=1000000; i++ )) is many operations as you can visually see. It is also many more operations per loop than you get from for i in $(seq 1 1000000). And that may not be obvious to you, which is why doing tests like this is valuable.
Demos
# show that seq is fast
$ time (seq 1 1000000 | wc)
1000000 1000000 6888894
real 0m0.227s
user 0m0.239s
sys 0m0.008s
# show that {..} is fast
$ time (echo {1..1000000} | wc)
1 1000000 6888896
real 0m1.778s
user 0m1.735s
sys 0m0.072s
# Show that for loops (even with a : noop) are slow
$ time (for i in {1..1000000} ; do :; done | wc)
0 0 0
real 0m3.642s
user 0m3.582s
sys 0m0.057s
# show that echo is slow
$ time (for i in {1..1000000} ; do echo $i; done | wc)
1000000 1000000 6888896
real 0m7.480s
user 0m6.803s
sys 0m2.580s
$ time (for i in $(seq 1 1000000) ; do echo $i; done | wc)
1000000 1000000 6888894
real 0m7.029s
user 0m6.335s
sys 0m2.666s
# show that C-style for loops are slower
$ time (for (( i=1; i<=1000000; i++ )) ; do echo $i; done | wc)
1000000 1000000 6888896
real 0m12.391s
user 0m11.069s
sys 0m3.437s
# show that arithmetic expansion is even slower
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; i=$(($i+1)); done | wc)
1000000 1000000 6888896
real 0m19.696s
user 0m18.017s
sys 0m3.806s
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $i; ((i=i+1)); done | wc)
1000000 1000000 6888896
real 0m18.629s
user 0m16.843s
sys 0m3.936s
$ time (i=1; e=1000000; while [ $i -le $e ]; do echo $((i++)); done | wc)
1000000 1000000 6888896
real 0m17.012s
user 0m15.319s
sys 0m3.906s
# even a noop is slow
$ time (i=1; e=1000000; while [ $((i++)) -le $e ]; do :; done | wc)
0 0 0
real 0m12.679s
user 0m11.658s
sys 0m1.004s
If you need it prefix than you might like this
for ((i=7;i<=12;i++)); do echo `printf "%2.0d\n" $i |sed "s/ /0/"`;done
that will yield
07
08
09
10
11
12
If you're on BSD / OS X you can use jot instead of seq:
for i in $(jot $END); do echo $i; done
This works fine in bash:
END=5
i=1 ; while [[ $i -le $END ]] ; do
echo $i
((i = i + 1))
done
There are many ways to do this, however the ones I prefer is given below
Using seq
Synopsis from man seq
$ seq [-w] [-f format] [-s string] [-t string] [first [incr]] last
Syntax
Full command
seq first incr last
first is starting number in the sequence [is optional, by default:1]
incr is increment [is optional, by default:1]
last is the last number in the sequence
Example:
$ seq 1 2 10
1 3 5 7 9
Only with first and last:
$ seq 1 5
1 2 3 4 5
Only with last:
$ seq 5
1 2 3 4 5
Using {first..last..incr}
Here first and last are mandatory and incr is optional
Using just first and last
$ echo {1..5}
1 2 3 4 5
Using incr
$ echo {1..10..2}
1 3 5 7 9
You can use this even for characters like below
$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
I know this question is about bash, but - just for the record - ksh93 is smarter and implements it as expected:
$ ksh -c 'i=5; for x in {1..$i}; do echo "$x"; done'
1
2
3
4
5
$ ksh -c 'echo $KSH_VERSION'
Version JM 93u+ 2012-02-29
$ bash -c 'i=5; for x in {1..$i}; do echo "$x"; done'
{1..5}
This is another way:
end=5
for i in $(bash -c "echo {1..${end}}"); do echo $i; done
If you want to stay as close as possible to the brace-expression syntax, try out the range function from bash-tricks' range.bash.
For example, all of the following will do the exact same thing as echo {1..10}:
source range.bash
one=1
ten=10
range {$one..$ten}
range $one $ten
range {1..$ten}
range {1..10}
It tries to support the native bash syntax with as few "gotchas" as possible: not only are variables supported, but the often-undesirable behavior of invalid ranges being supplied as strings (e.g. for i in {1..a}; do echo $i; done) is prevented as well.
The other answers will work in most cases, but they all have at least one of the following drawbacks:
Many of them use subshells, which can harm performance and may not be possible on some systems.
Many of them rely on external programs. Even seq is a binary which must be installed to be used, must be loaded by bash, and must contain the program you expect, for it to work in this case. Ubiquitous or not, that's a lot more to rely on than just the Bash language itself.
Solutions that do use only native Bash functionality, like #ephemient's, will not work on alphabetic ranges, like {a..z}; brace expansion will. The question was about ranges of numbers, though, so this is a quibble.
Most of them aren't visually similar to the {1..10} brace-expanded range syntax, so programs that use both may be a tiny bit harder to read.
#bobbogo's answer uses some of the familiar syntax, but does something unexpected if the $END variable is not a valid range "bookend" for the other side of the range. If END=a, for example, an error will not occur and the verbatim value {1..a} will be echoed. This is the default behavior of Bash, as well--it is just often unexpected.
Disclaimer: I am the author of the linked code.
These are all nice but seq is supposedly deprecated and most only work with numeric ranges.
If you enclose your for loop in double quotes, the start and end variables will be dereferenced when you echo the string, and you can ship the string right back to BASH for execution. $i needs to be escaped with \'s so it is NOT evaluated before being sent to the subshell.
RANGE_START=a
RANGE_END=z
echo -e "for i in {$RANGE_START..$RANGE_END}; do echo \\${i}; done" | bash
This output can also be assigned to a variable:
VAR=`echo -e "for i in {$RANGE_START..$RANGE_END}; do echo \\${i}; done" | bash`
The only "overhead" this should generate should be the second instance of bash so it should be suitable for intensive operations.
Replace {} with (( )):
tmpstart=0;
tmpend=4;
for (( i=$tmpstart; i<=$tmpend; i++ )) ; do
echo $i ;
done
Yields:
0
1
2
3
4
If you're doing shell commands and you (like I) have a fetish for pipelining, this one is good:
seq 1 $END | xargs -I {} echo {}
if you don't wanna use 'seq' or 'eval' or jot or arithmetic expansion format eg. for ((i=1;i<=END;i++)), or other loops eg. while, and you don't wanna 'printf' and happy to 'echo' only, then this simple workaround might fit your budget:
a=1; b=5; d='for i in {'$a'..'$b'}; do echo -n "$i"; done;' echo "$d" | bash
PS: My bash doesn't have 'seq' command anyway.
Tested on Mac OSX 10.6.8, Bash 3.2.48
This works in Bash and Korn, also can go from higher to lower numbers. Probably not fastest or prettiest but works well enough. Handles negatives too.
function num_range {
# Return a range of whole numbers from beginning value to ending value.
# >>> num_range start end
# start: Whole number to start with.
# end: Whole number to end with.
typeset s e v
s=${1}
e=${2}
if (( ${e} >= ${s} )); then
v=${s}
while (( ${v} <= ${e} )); do
echo ${v}
((v=v+1))
done
elif (( ${e} < ${s} )); then
v=${s}
while (( ${v} >= ${e} )); do
echo ${v}
((v=v-1))
done
fi
}
function test_num_range {
num_range 1 3 | egrep "1|2|3" | assert_lc 3
num_range 1 3 | head -1 | assert_eq 1
num_range -1 1 | head -1 | assert_eq "-1"
num_range 3 1 | egrep "1|2|3" | assert_lc 3
num_range 3 1 | head -1 | assert_eq 3
num_range 1 -1 | tail -1 | assert_eq "-1"
}

Keep zero after Hexdecimal conversion in bash

I'm writing a very simple script in bash:
for i in {000..1024} ; do
echo $i ;
echo "obase=16; $i" | bc
done
I want to print all numbers with minimum three digits as 000, 001, 002 .. until 099. With integer numbers it works good, but after obase = 16; $ i '| bcs number return with one or two digits.
How can I solve it in the easiest way?
You might want to use printf to format the number, for example:
for i in {0..1024} ; do
echo $i
printf '%03X\n' $i
done
Or j=$(printf '%03X' $i) and then echo $j.
For more on formatting check the Format strings subsection of Syntax section here.
Not sure it's "easiest", but generally printf is good for formatting issues. eg
printf "%03s\n" "$(echo "obase=16; $i" | bc)"

Resources