In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.
Related
For debugging my scripts, I would like to add the internal variables $FUNCNAME and $LINENO at the beginning of each of my outputs, so I know what function and line number the output occurs on.
foo(){
local bar="something"
echo "$FUNCNAME $LINENO: I just set bar to $bar"
}
But since there will be many debugging outputs, it would be cleaner if I could do something like the following:
foo(){
local trace='$FUNCNAME $LINENO'
local bar="something"
echo "$trace: I just set bar to $bar"
}
But the above literally outputs:
"$FUNCNAME $LINENO: I just set bar to something"
I think it does this because double quotes only expands variables inside once.
Is there a syntactically clean way to expand variables twice in the same line?
You cannot safely evaluate expansions twice when handling runtime data.
There are means to do re-evaluation, but they require trusting your data -- in the NSA system design sense of the word: "A trusted component is one that can break your system when it fails".
See BashFAQ #48 for a detailed discussion. Keep in mind that if you could be logging filenames, that any character except NUL can be present in a UNIX filename. $(rm -rf ~)'$(rm -rf ~)'.txt is a legal name. * is a legal name.
Consider a different approach:
#!/usr/bin/env bash
trace() { echo "${FUNCNAME[1]}:${BASH_LINENO[0]}: $*" >&2; }
foo() {
bar=baz
trace "I just set bar to $bar"
}
foo
...which, when run with bash 4.4.19(1)-release, emits:
foo:7: I just set bar to baz
Note the use of ${BASH_LINENO[0]} and ${FUNCNAME[1]}; this is because BASH_LINENO is defined as follows:
An array variable whose members are the line numbers in source files where each corresponding member of FUNCNAME was invoked.
Thus, FUNCNAME[0] is trace, whereas FUNCNAME[1] is foo; whereas BASH_LINENO[0] is the line from which trace was called -- a line which is inside the function foo.
Although eval has its dangers, getting a second expansion is what it does:
foo(){
local trace='$FUNCNAME $LINENO'
local bar="something"
eval echo "$trace: I just set bar to $bar"
}
foo
Gives:
foo 6: I just set bar to something
Just be careful not to eval anything that has come from external sources, since you could get a command injected into the string.
Yes to double expansion; but no, it won't do what you are hoping for.
Yes, bash offers a way to do "double expansion" of a variable, aka, a way to first interpret a variable, then take that as the name of some other variable, where the other variable is what's to actually be expanded. This is called "indirection". With "indirection", bash allows a shell variable to reference another shell variable, with the final value coming from the referenced variable. So, a bash variable can be passed by reference.
The syntax is just the normal braces style expansion, but with an exclamation mark prepended to the name.
${!VARNAME}
It is used like this:
BAR="my final value";
FOO=BAR
echo ${!FOO};
...which produces this output...
my final value
No, you can't use this mechanism to do the same as $( eval "echo $VAR1 $VAR2" ). The result of the first interpretation must be exactly the name of a shell variable. It does not accept a string, and does not understand the dollar sign. So this won't work:
BAR="my final value";
FOO='$BAR'; # The dollar sign confuses things
echo ${!FOO}; # Fails because there is no variable named '$BAR'
So, it does not solve your ultimate quest. None-the-less, indirection can be a powerful tool.
I am fairly new to unix bash scripting and need to know if this is possible. I want to ask user for their input multiple times and then store that input in to one variable.
userinputs= #nothing at the start
read string
<code to add $string to $userinputs>
read string
<code to add $string to $userinputs> #this should add this input along with the other input
so if the user enters "abc" when asked first time, it add's "abc" in $userinputs
then when asked again for the input and the user enters "123" the script should store it in the same $userinputs
this would make the $userinput=abc123
The usual way to concat two strings in Bash is:
new_string="$string1$string2"
{} are needed around the variable name only if we have a literal string that can obstruct the variable expansion:
new_string="${string1}literal$string2"
rather than
new_string="$string1literal$string2"
You can also use the += operator:
userinputs=
read string
userinputs+="$string"
read string
userinputs+="$string"
Double quoting $string is optional in this case.
See also:
How to concatenate string variables in Bash?
You can concatentate variables and store multiple strings in the same one like so:
foo=abc
echo $foo # prints 'abc'
bar=123
foo="${foo}${bar}"
echo $foo # prints 'abc123'
You can use the other variables, or the same variable, when assigning to a variable, e.g. a="${a}123${b}". See this question for more info.
You don't have to quote the strings you're assigning to, or do the ${var} syntax, but learning when to quote and not to quote is a surprisingly nuanced art, so it's often better to be safe than sorry, and the "${var}" syntax in double quotes is usually the safest approach (see any of these links for more than you ever wanted to know: 1 2 3).
Anyway, you should read into a temporary variable (read, by default, reads into $REPLY) and concatentate that onto your main variable, like so:
allinput=
read # captures user input into $REPLY
allinput="${REPLY}"
read
allinput="${allinput}${REPLY}"
Beware that the read command behaves very differently depending on supplied switches and the value of the IFS global variable, especially in the face of unusual input with special characters. A common "just do what I mean" choice is to empty out IFS via IFS= and use read -r to capture input. See the read builtin documentation for more info.
I want to create a bash script, that gets 3 parameters. But the second needs to be $*, because i need later these lines. The other two parameters (first and third) doesn't need this.
for x in $* do
The first and second parameter aren't the problem, this one works:
parameter1="$1"
shift
parameter2="$*"
But i need the third parameter at the end and something like this
parameter1="$1"
parameter3="$3"
shift
parameter2="$*"
won't work. My command at the end should look like this:
bash myscript parameter1 parameter2 parameter3
For specifically three parameters, you can use substring parameter expansion in a simple way:
parameter1=$1
parameter2="${#:2:1}" # One parameter, starting with #2
parameter3=$3
Or course, that's unnecessary, since you can just use $2 instead of ${#:2:1}, but I point it out as a simple introduction to the syntax (and not at all because I overlooked the fact you would use $2, really....)
(You can also use it as a substitute for indirect parameter expansion; "${#:n:1}" and "${!n}" are basically equivalent when n is a variable with an integer value.)
For the more general case, where you want an arbitrary number of arguments between the first and last, it gets a little more complicated, although the principle is the same:
parameter1=$1
middleParameters=( "${#:2:$#-2}" ) # n - 2 parameters, starting with #2, i.e., all but $1 and ${!n} for n=$#
lastParameter="${#:$#}"
shift removes an argument from the left. If you want to remove an argument from the right, you can do that with:
set -- "${#:1:$# - 1}"
Thus:
parameter1=$1 # capture leftmost argument
shift # remove leftmost argument
parameter3=${*:$#:1} # capture rightmost argument
set -- "${#:1:$# - 1}" # remove rightmost argument
parameter2=$* # concatenate remaining arguments and store in a string
Note that $* is almost certainly the Wrong Thing. If you want to keep your arguments separate, respecting their quoting, instead use an array:
parameter2=( "$#" )
for item in "${parameter2[#]}"; do
echo "Processing item: $item"
done
If your script is run as yourscript arg1 "item A" "item B" arg3, then the above will ensure that item A and item B are treated as individual arguments, rather than treating item as an argument, A as another, etc.
So I'm trying to do something, not sure if it's possible. I have the following code:
for i in {0..5}; do
if [[ -f ./user$i ]]; then
group$i=$(grep -w "group" ./user0|awk '{print $2}'|perl -lape 's/\s+//sg')
What I want to do is assign a unique variable for each instance of the {0..5} so group1 group2 group3 group4 for each variable name. Then I would change ./user0 to ./user$i and create a dynamic list of variables based on my sequence.
Is this possible? I get the following error when trying to execute this and I'm unsure of what I have actually done that bash doesn't like.
test.sh: line 16: group0=j: command not found
Kurt Stutsman provides the right pointer in a comment on the question: use Bash arrays to solve your problem.
Here's a simplified example:
groups=() # declare an empty array; same as: declare -a groups
for i in {0..5}; do
groups[i]="group $i" # dynamically create element with index $i
done
# Print the resulting array's elements.
printf '%s\n' "${groups[#]}"
See the bottom of this answer for other ways to enumerate the elements of array ${groups[#]}.
bash arrays can be dynamically expanded (and can even be sparse - element indices need not be contiguous)
Hence, simply assigning to element $i works, without prior sizing of the array.
Note how $i need not be prefixed with $ in the array subscript, because array subscripts are evaluated in an arithmetic context (the same context in which $(( ... )) expressions are evaluated).
As for what you did wrong:
group$i=...
is not recognized as a variable assignment by Bash, because - taken literally - group$i is not a valid identifier (variable name).
Because it isn't, Bash continues to parse until the next shell metacharacter is found, and then interprets the resulting word as a command to execute, which in your case resulted in error message group0=j: command not found.
If, for some reason, you don't want to use arrays to avoid this problem entirely, you can work around the problem:
By involving a variable-declaring builtin [command] such as declare, local, or export, you force Bash to perform expansions first, which expands group$i to a valid variable name before passing it to the builtin.
user2683246's answer demonstrates the next best approach by using declare (or, if local variables inside a function are desired, local) to create the variables.
Soren's answer uses export, but that is only advisable if you want to create environment variables visible to child processes rather than mere shell variables.
Caveat: With this technique, be sure to double-quote the RHS in order to capture the full value; to illustrate:
i=0; declare v$i=$(echo 'hi, there'); echo "$v0" # !! WRONG -> 'hi,': only UP TO 1ST SPACE
i=0; declare v$i="$(echo 'hi, there')"; echo "$v0" # OK -> 'hi, there'
Other ways to enumerate the groups array created above:
# Enumerate array elements directly.
for element in "${groups[#]}"; do
echo "$element"
done
# Enumerate array elements by index.
for (( i = 0; i < ${#groups[#]}; i++ )); do
echo "#$i: ${groups[i]}"
done
Use declare group$i=... instead of just group$i=...
Try to use the export or declare function like this
for i in {0..5}; do
if [[ -f ./user$i ]]; then
export group$i=$(grep -w "group" ......
with declare
for i in {0..5}; do
if [[ -f ./user$i ]]; then
declare group$i=$(grep -w "group" ......
where export makes the value available to sub-processes, and declare just available within the same script.
I don't know exactly how to ask this in English, but I want to have the value of a variable as a new variable...
The script also has a loop with increasing numbers, and in the end I want to have the variables VAR1, VAR2 etc.
I'm trying this:
COUNT=$(echo 1)
DEFINE=$(echo VAR$COUNT)
$DEFINE=$(echo gotcha!)
When I try this way, I have this error message:
~/script.sh: line n: VAR1=gotcha!: command not found
I played a bit around with brackets and quotation marks, but it didn't work... any solutions?
The problem is that bash expects a command as a result of expansions, not an assignment. VAR1=gotcha! is not a command, hence the error.
It would be better to use an array:
COUNT=$(echo 1)
VAR[COUNT]='gotcha!'
echo ${VAR[COUNT]}
I guess $(echo 1) stands for a more complex command, otherwise you can just use COUNT=1.
You can use declare to create such a "dynamic" variable, but using an array is probably a better choice.
COUNT=1
DEFINE="VAR$COUNT"
declare "$DEFINE=gotcha"