In a zsh script,
echo ${X:-4711}
outputs the value of the variable X, or 4711 if there is none.
echo ${X:u}
outputs the value of the variable X, converted to upper case.
I wonder, whether there is a way to combine the two, i.e. to have the effect of
tmp=${X:-4711}
echo $X:u
without introducing an auxiliary variable.
$ echo ${${X:-4711}:u}
4711
$ X=hello
$ echo ${${X:-4711}:u}
HELLO
From man zshexpn:
If a `${...}` type parameter expression or a `$(...)` type command
substitution is used in place of name above, it is expanded first and
the result is used as if it were the value of name. Thus it is possible
to perform nested operations: `${${foo#head}%tail}` substitutes the value
of `$foo` with both 'head' and 'tail' deleted.
Related
Made up example (perl):
my $x = read_input_from_file();
# $x now contains string $ENV{SOMETHING}/dir/$ENV{SOMETHING_ELSE}
my $y = eval($x); # doesnt work
How can I get value of string contained in $x in the script?
So far I have tried using eval which doesn't generate any output. I am hoping that something already exists in perl and these string expressions do not need to be parsed and evaluated.
The "string" eval is a little specific:
eval in all its forms is used to execute a little Perl program.
...
In a string eval, the value of the expression (which is itself determined within scalar context) is first parsed, and if there were no errors, executed as a block within the lexical context of the current Perl program.
So this evaluates code, and with a variable to be evaluated containing a string literal we have a "bareword" ("Unquoted string") which is generally no good. In your case, those / in $x cause additional trouble.
If the content of the variable to evaluate is a string literal (not code) it need be quoted
my $y = eval q(") . $x . q("); # double-quote so that it interpolates
I use the operator form of a single quote, q(). Quoted under it is a double-quote since $x itself seems to contain variables that need be evaluated (interpolated).
Keep in mind that running code from external sources can be a serious security problem.
For debugging my scripts, I would like to add the internal variables $FUNCNAME and $LINENO at the beginning of each of my outputs, so I know what function and line number the output occurs on.
foo(){
local bar="something"
echo "$FUNCNAME $LINENO: I just set bar to $bar"
}
But since there will be many debugging outputs, it would be cleaner if I could do something like the following:
foo(){
local trace='$FUNCNAME $LINENO'
local bar="something"
echo "$trace: I just set bar to $bar"
}
But the above literally outputs:
"$FUNCNAME $LINENO: I just set bar to something"
I think it does this because double quotes only expands variables inside once.
Is there a syntactically clean way to expand variables twice in the same line?
You cannot safely evaluate expansions twice when handling runtime data.
There are means to do re-evaluation, but they require trusting your data -- in the NSA system design sense of the word: "A trusted component is one that can break your system when it fails".
See BashFAQ #48 for a detailed discussion. Keep in mind that if you could be logging filenames, that any character except NUL can be present in a UNIX filename. $(rm -rf ~)'$(rm -rf ~)'.txt is a legal name. * is a legal name.
Consider a different approach:
#!/usr/bin/env bash
trace() { echo "${FUNCNAME[1]}:${BASH_LINENO[0]}: $*" >&2; }
foo() {
bar=baz
trace "I just set bar to $bar"
}
foo
...which, when run with bash 4.4.19(1)-release, emits:
foo:7: I just set bar to baz
Note the use of ${BASH_LINENO[0]} and ${FUNCNAME[1]}; this is because BASH_LINENO is defined as follows:
An array variable whose members are the line numbers in source files where each corresponding member of FUNCNAME was invoked.
Thus, FUNCNAME[0] is trace, whereas FUNCNAME[1] is foo; whereas BASH_LINENO[0] is the line from which trace was called -- a line which is inside the function foo.
Although eval has its dangers, getting a second expansion is what it does:
foo(){
local trace='$FUNCNAME $LINENO'
local bar="something"
eval echo "$trace: I just set bar to $bar"
}
foo
Gives:
foo 6: I just set bar to something
Just be careful not to eval anything that has come from external sources, since you could get a command injected into the string.
Yes to double expansion; but no, it won't do what you are hoping for.
Yes, bash offers a way to do "double expansion" of a variable, aka, a way to first interpret a variable, then take that as the name of some other variable, where the other variable is what's to actually be expanded. This is called "indirection". With "indirection", bash allows a shell variable to reference another shell variable, with the final value coming from the referenced variable. So, a bash variable can be passed by reference.
The syntax is just the normal braces style expansion, but with an exclamation mark prepended to the name.
${!VARNAME}
It is used like this:
BAR="my final value";
FOO=BAR
echo ${!FOO};
...which produces this output...
my final value
No, you can't use this mechanism to do the same as $( eval "echo $VAR1 $VAR2" ). The result of the first interpretation must be exactly the name of a shell variable. It does not accept a string, and does not understand the dollar sign. So this won't work:
BAR="my final value";
FOO='$BAR'; # The dollar sign confuses things
echo ${!FOO}; # Fails because there is no variable named '$BAR'
So, it does not solve your ultimate quest. None-the-less, indirection can be a powerful tool.
I want to create a bash script, that gets 3 parameters. But the second needs to be $*, because i need later these lines. The other two parameters (first and third) doesn't need this.
for x in $* do
The first and second parameter aren't the problem, this one works:
parameter1="$1"
shift
parameter2="$*"
But i need the third parameter at the end and something like this
parameter1="$1"
parameter3="$3"
shift
parameter2="$*"
won't work. My command at the end should look like this:
bash myscript parameter1 parameter2 parameter3
For specifically three parameters, you can use substring parameter expansion in a simple way:
parameter1=$1
parameter2="${#:2:1}" # One parameter, starting with #2
parameter3=$3
Or course, that's unnecessary, since you can just use $2 instead of ${#:2:1}, but I point it out as a simple introduction to the syntax (and not at all because I overlooked the fact you would use $2, really....)
(You can also use it as a substitute for indirect parameter expansion; "${#:n:1}" and "${!n}" are basically equivalent when n is a variable with an integer value.)
For the more general case, where you want an arbitrary number of arguments between the first and last, it gets a little more complicated, although the principle is the same:
parameter1=$1
middleParameters=( "${#:2:$#-2}" ) # n - 2 parameters, starting with #2, i.e., all but $1 and ${!n} for n=$#
lastParameter="${#:$#}"
shift removes an argument from the left. If you want to remove an argument from the right, you can do that with:
set -- "${#:1:$# - 1}"
Thus:
parameter1=$1 # capture leftmost argument
shift # remove leftmost argument
parameter3=${*:$#:1} # capture rightmost argument
set -- "${#:1:$# - 1}" # remove rightmost argument
parameter2=$* # concatenate remaining arguments and store in a string
Note that $* is almost certainly the Wrong Thing. If you want to keep your arguments separate, respecting their quoting, instead use an array:
parameter2=( "$#" )
for item in "${parameter2[#]}"; do
echo "Processing item: $item"
done
If your script is run as yourscript arg1 "item A" "item B" arg3, then the above will ensure that item A and item B are treated as individual arguments, rather than treating item as an argument, A as another, etc.
So I'm trying to do something, not sure if it's possible. I have the following code:
for i in {0..5}; do
if [[ -f ./user$i ]]; then
group$i=$(grep -w "group" ./user0|awk '{print $2}'|perl -lape 's/\s+//sg')
What I want to do is assign a unique variable for each instance of the {0..5} so group1 group2 group3 group4 for each variable name. Then I would change ./user0 to ./user$i and create a dynamic list of variables based on my sequence.
Is this possible? I get the following error when trying to execute this and I'm unsure of what I have actually done that bash doesn't like.
test.sh: line 16: group0=j: command not found
Kurt Stutsman provides the right pointer in a comment on the question: use Bash arrays to solve your problem.
Here's a simplified example:
groups=() # declare an empty array; same as: declare -a groups
for i in {0..5}; do
groups[i]="group $i" # dynamically create element with index $i
done
# Print the resulting array's elements.
printf '%s\n' "${groups[#]}"
See the bottom of this answer for other ways to enumerate the elements of array ${groups[#]}.
bash arrays can be dynamically expanded (and can even be sparse - element indices need not be contiguous)
Hence, simply assigning to element $i works, without prior sizing of the array.
Note how $i need not be prefixed with $ in the array subscript, because array subscripts are evaluated in an arithmetic context (the same context in which $(( ... )) expressions are evaluated).
As for what you did wrong:
group$i=...
is not recognized as a variable assignment by Bash, because - taken literally - group$i is not a valid identifier (variable name).
Because it isn't, Bash continues to parse until the next shell metacharacter is found, and then interprets the resulting word as a command to execute, which in your case resulted in error message group0=j: command not found.
If, for some reason, you don't want to use arrays to avoid this problem entirely, you can work around the problem:
By involving a variable-declaring builtin [command] such as declare, local, or export, you force Bash to perform expansions first, which expands group$i to a valid variable name before passing it to the builtin.
user2683246's answer demonstrates the next best approach by using declare (or, if local variables inside a function are desired, local) to create the variables.
Soren's answer uses export, but that is only advisable if you want to create environment variables visible to child processes rather than mere shell variables.
Caveat: With this technique, be sure to double-quote the RHS in order to capture the full value; to illustrate:
i=0; declare v$i=$(echo 'hi, there'); echo "$v0" # !! WRONG -> 'hi,': only UP TO 1ST SPACE
i=0; declare v$i="$(echo 'hi, there')"; echo "$v0" # OK -> 'hi, there'
Other ways to enumerate the groups array created above:
# Enumerate array elements directly.
for element in "${groups[#]}"; do
echo "$element"
done
# Enumerate array elements by index.
for (( i = 0; i < ${#groups[#]}; i++ )); do
echo "#$i: ${groups[i]}"
done
Use declare group$i=... instead of just group$i=...
Try to use the export or declare function like this
for i in {0..5}; do
if [[ -f ./user$i ]]; then
export group$i=$(grep -w "group" ......
with declare
for i in {0..5}; do
if [[ -f ./user$i ]]; then
declare group$i=$(grep -w "group" ......
where export makes the value available to sub-processes, and declare just available within the same script.
In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.