This question already has answers here:
Brace expansion with a Bash variable - {0..$foo}
(5 answers)
Closed 8 years ago.
I have the following shell script
cat test.sh
j=00000001;
k=00000005;
l=$(echo {00000001..00000005}.jpg);
m=$(echo {$j..$k}.jpg);
ls $l
ls $m
Here is the output
./test.sh
00000001.jpg 00000002.jpg 00000003.jpg 00000004.jpg 00000005.jpg
ls: cannot access {00000001..00000005}.jpg: No such file or directory
My doubt is "Why is the ls $m not working".
and How to make that work?
Thanks in advance.
lin
Sequence expansion only happens for literal numbers. Variable expansion occurs after sequence expansion:
A sequence expression takes the form {x..y}, where x and y are either
integers or single characters. When integers are supplied, the expression
expands to each number between x and y, inclusive. When characters are
supplied, the expression expands to each character lexicographically
between x and y, inclusive. Note that both x and y must be of the same type.
Brace expansion is performed before any other expansions, and any characters
special to other expansions are preserved in the result. It is strictly textual.
Bash does not apply any syntactic interpretation to the context of the expansion
or the text between the braces.
For your case, you can use eval:
m=`eval echo {$j..$k}.jpg`
Related
This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 4 years ago.
I have four files:
1.txt 2.txt 3.txt 4.txt
in linux shell, I could use :
ls {1..4}.txt to list all the four files
but if I set two variables : var1=1 and var2=4, how to list the four files?
that is:
var1=1
var2=4
ls {$var1..$var2}.txt # error
what is the correct code?
Using variables with the sequence-expression form ({<numFrom>..<numTo>}) of brace expansion only works in ksh and zsh, but, unfortunately, not in bash (and (mostly) strictly POSIX-features-only shells such as dash do not support brace expansion at all, so brace expansion should be avoided with /bin/sh altogether).
Given your symptoms, I assume you're using bash, where you can only use literals in sequence expressions (e.g., {1..3}); from the manual (emphasis mine):
Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result.
In other words: at the time a brace expression is evaluated, variable references have not been expanded (resolved) yet; interpreting literals such as $var1 and $var2 as numbers in the context of a sequence expression therefore fails, so the brace expression is considered invalid and as not expanded.
Note, however, that the variable references are expanded, namely at a later stage of overall expansion; in the case at hand the literal result is the single word '{1..4}' - an unexpanded brace expression with variable values expanded.
While the list form of brace expansion (e.g., {foo,bar)) is expanded the same way, later variable expansion is not an issue there, because no interpretation of the list elements is needed up front; e.g. {$var1,$var2} correctly results in the 2 words 1 and 4.
As for why variables cannot be used in sequence expressions: historically, the list form of brace expansion came first, and when the sequence-expression form was later introduced, the order of expansions was already fixed.
For a general overview of brace expansion, see this answer.
Workarounds
Note: The workarounds focus on numerical sequence expressions, as in the question; the eval-based workaround also demonstrates use of variables with the less common character sequence expressions, which produce ranges of English letters (e.g., {a..c} to produce a b c).
A seq-based workaround is possible, as demonstrated in Jameson's answer.
A small caveat is that seq is not a POSIX utility, but most modern Unix-like platforms have it.
To refine it a little, using seq's -f option to supply a printf-style format string, and demonstrating two-digit zero-padding:
seq -f '%02.f.txt' $var1 $var2 | xargs ls # '%02.f'==zero-pad to 2 digits, no decimal places
Note that to make it fully robust - in case the resulting words contain spaces or tabs - you'd need to employ embedded quoting:
seq -f '"%02.f a.txt"' $var1 $var2 | xargs ls
ls then sees 01 a.txt, 02 a.txt, ... with the argument boundaries correctly preserved.
If you want to robustly collect the resulting words in a Bash array first, e.g., ${words[#]}:
IFS=$'\n' read -d '' -ra words < <(seq -f '%02.f.txt' $var1 $var2)
ls "${words[#]}"
The following are pure Bash workarounds:
A limited workaround using Bash features only is to use eval:
var1=1 var2=4
# Safety check
(( 10#$var1 + 10#$var2 || 1 )) 2>/dev/null || { echo "Need decimal integers." >&2; exit 1; }
ls $(eval printf '%s\ ' "{$var1..$var2}.txt") # -> ls 1.txt 2.txt 3.txt 4.txt
You can apply a similar technique to a character sequence expression;
var1=a var2=c
# Safety check
[[ $var1 == [a-zA-Z] && $var2 == [a-zA-Z] ]] || { echo "Need single letters."; exit 1; }
ls $(eval printf '%s\ ' "{$var1..$var2}.txt") # -> ls a.txt b.txt c.txt
Note:
A check is performed up front to ensure that $var1 and $var2 contain decimal integers or single English letters, which then makes it safe to use eval. Generally, using eval with unchecked input is a security risk and use of eval is therefore best avoided.
Given that output from eval must be passed unquoted to ls here, so that the shell splits the output into individual arguments through words-splitting, this only works if the resulting filenames contain no embedded spaces or other shell metacharacters.
A more robust, but more cumbersome pure Bash workaround to use an array to create the equivalent words:
var1=1 var2=4
# Emulate brace sequence expression using an array.
args=()
for (( i = var1; i <= var2; i++ )); do
args+=( "$i.txt" )
done
ls "${args[#]}"
This approach bears no security risk and also works with resulting filenames with embedded shell metacharacters, such as spaces.
Custom increments can be implemented by replacing i++ with, e.g., i+=2 to step in increments of 2.
Implementing zero-padding would require use of printf; e.g., as follows:
args+=( "$(printf '%02d.txt' "$i")" ) # -> '01.txt', '02.txt', ...
For that particular piece of syntax (a "sequence expression") you're out of luck, see Bash man page:
A sequence expression takes the form {x..y[..incr]}, where x and y are
either integers or single characters, and incr, an optional increment,
is an integer.
However, you could instead use the seq utility, which would have a similar effect -- and the approach would allow for the use of variables:
var1=1
var2=4
for i in `seq $var1 $var2`; do
ls ${i}.txt
done
Or, if calling ls four times instead of once bothers you, and/or you want it all on one line, something like:
for i in `seq $var1 $var2`; do echo ${i}.txt; done | xargs ls
From seq(1) man page:
seq [OPTION]... LAST
seq [OPTION]... FIRST LAST
seq [OPTION]... FIRST INCREMENT LAST
If you run the following bash script on Ubuntu 12.04:
t="t"
echo "1st Output this $15"
echo "2nd this $test"
The output is:
1st Output this 5
2nd this
How can the first echo takes $1 as a variable name (non-greedy) interpreting it as ${1}5 while the second echo takes $test as a variable name (greedy) instead of interpreting it as ${t}est?
There are two parts to your question:
$15 would always be interpreted as $1, i.e. the first positional parameter1, concatenated with 5. In order to use the fifteenth positional parameter, you'd need to say ${15}.
$test would be interpreted as variable test. So if you want it to be interpreted as $t concatenated with est, then you need to say ${t}est
1Quoting from info bash:
When a positional parameter consisting of more than a single digit is
expanded, it must be enclosed in braces (see EXPANSION below).
...
EXPANSION
Expansion is performed on the command line after it has been split into
words. There are seven kinds of expansion performed: brace expansion,
tilde expansion, parameter and variable expansion, command substitu‐
tion, arithmetic expansion, word splitting, and pathname expansion.
You may also want to refer to:
What's the difference between ${varname} and $varname in a shell scripts
This question already has answers here:
How do I iterate over a range of numbers defined by variables in Bash?
(20 answers)
Closed 8 years ago.
How can I use variable in for loop digits?
for example:
num="12"
for i in {0..$num}; do
...
done
Brace expansion with variables doesn't work as one would expect (see Appendix B for juicy details), i.e. {0..$num} would only return {0..12} literally instead of a list of number.
Try seq instead like this:
num="12"
for i in $(seq 0 $num); do
echo $i
done
Appendix B: Juicy details
The bash manual saith,
The order of expansions is: brace expansion, tilde expansion, parameter, variable, and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and filename expansion.
At the time shell expands {0..$num} (brace expansion), $num isn't expanded (variable expansion) yet. The sequence expression a..b needs both a and b to be numbers to generate a sequence, but here we have one number and one non-number (the literal string $num). Failing this, the shell falls back to interpreting {0..$num} literally. Then variable expansion takes over, and finally we get {0..12}
Bash does brace expansion before variable expansion, so you will get output like {1..12}. With use of eval you can get it to work.
Test:
$ num=5
$ for i in {1..$num}; do echo "$i"; done
{1..5}
$ for i in $(eval echo {1..$num}); do echo "$i"; done
1
2
3
4
5
Please note: eval is evil in disguise.
This question already has answers here:
Command substitution: backticks or dollar sign / paren enclosed? [duplicate]
(3 answers)
Closed 9 years ago.
What is the difference between executing a command like this:
var=$(ls -alh /dir)
And doing it like this:
var=`ls -alh /dir`
Is one method able to be used in more interpreters than the other?
They are both command substitution
Section 3.5.4 http://www.gnu.org/software/bash/manual/bashref.html
Bash performs the expansion by executing command and replacing the
command substitution with the standard output of the command, with any
trailing newlines deleted. Embedded newlines are not deleted, but they
may be removed during word splitting. The command substitution $(cat
file) can be replaced by the equivalent but faster $(< file).
When the old-style backquote form of substitution is used, backslash
retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’.
The first backquote not preceded by a backslash terminates the command
substitution. When using the $(command) form, all characters between
the parentheses make up the command; none are treated specially.
Command substitutions may be nested. To nest when using the backquoted
form, escape the inner backquotes with backslashes.
If the substitution appears within double quotes, word splitting and
filename expansion are not performed on the results.
It is very interesting that if you intend to display 0_1 with Bash using the code
x=0
y=1
echo "$x_$y"
then it will only display
1
I tried echo "$x\_$y" and it doesn't work.
How can I echo the form $x_$y? I'm going to use it on a file name string.
Because variable names are allowed to have underscores in them, the command:
echo "$x_$y"
is trying to echo ${x_} (which is probably empty in your case) followed by ${y}. The reason for this is because parameter expansion is a greedy operation - it will take as many legal characters as possible after the $ to form a variable name.
The relevant part of the bash manpage states:
The $ character introduces parameter expansion, command substitution, or arithmetic expansion.
The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.
When braces are used, the matching ending brace is the first } not escaped by a backslash or within a quoted string, and not within an embedded arithmetic expansion, command substitution, or parameter expansion.
Hence, the solution is to ensure that the _ is not treated as part of the first variable, which can be done with:
echo "${x}_${y}"
I tend to do all my bash variables like this, even standalone ones like:
echo "${x}"
since it's more explicit, and I've been bitten so many times in the past :-)
This way:
$ echo "${x}_${y}"
0_1
wrap it in curly braces:
echo "${x}_${y}"
Just to buck the trend, you can also do this:
echo $x'_'$y
You can have quoted and unquoted parts next to each other with no space between. And since ' isn't a legal character for a variable name, bash will substitute only $x. :)