This question already has answers here:
Can parameter expansion be nested in Bash? [duplicate]
(2 answers)
Can ${var} parameter expansion expressions be nested in bash?
(15 answers)
Closed 4 years ago.
Is it possible to perform more than one bash variable manipulation in one shot?
For example:
# for this variable
foo="string_that_is_lower"
# can I do this in one shot?
$foo="${foo:0:4}"
echo "${foo^^}"
Is there a way to combine these? I realize this is a trivial problem because this works just fine, or one could simply use other built in tools like echo ${foo:0:4} | tr [[:lower:]] [[:upper:]], but that is lame.
I've tried every logical combination I can think might work:
${${foo^^}:0:4}
${{foo^^}:0:4}
${foo^^,:0:4}
${foo^^;:0:4}
${foo^^ :0:4}
All produce syntax errors.
I find no instances of "(combine|multiple) bash variable manipulations" in the Advanced Bash Scripting Manual, or the manpage, or Google, or here, so maybe you just can't do it, but probably I'm just not searching for the right terms.
It is a very good question, but not something that parameter expansion allows. man bash provides that expansion operates on a parameter name or symbol, e.g.
Parameter Expansion
The `$' character introduces parameter expansion, command
substitution, or arithmetic expansion. The parameter name or
symbol to be expanded may be enclosed in braces, which are optional
but serve to protect the variable to be expanded from characters
immediately following it which could be interpreted as partof the
name.
When you seek to chain expansions together, you are attempting to use the text that results from the first expansion as a parameter name or symbol (which it isn't), so the command fails with a syntax error.
Good question!
(note: to be complete -- there are some instances where you can embed a parameter expansion within another parameter expansion, but those are limited to where the outer expansion is looking for a single number as part of its expansion which can be provided by an included expansion such as the length of a string, e.g. ${#var} in v=foo; echo ${v:$((${#v}-1))})
Related
This question already has answers here:
What is the meaning of the ${0##...} syntax with variable, braces and hash character in bash?
(4 answers)
What does "##" in a shell script mean? [duplicate]
(1 answer)
Closed 7 months ago.
I came across a shell script like the following:
for FILE_PATH in `ls some/directory`
do
export FILE=${FILE_PATH##*/}
done
What exactly is the "##*/" doing? When I echo ${FILE} and ${FILE_PATH}, I don't see any difference. Is this to handle unusually named files?
More generally, how would I go about figuring out this type of question for myself in the future? Google was completely useless.
It's removing everything up to the last / in the value of $FILE. From the Bash Manual:
${parameter#word}
${parameter##word}
The word is expanded to produce a pattern and matched according to the rules described below (see Pattern Matching). If the pattern matches the beginning of the expanded value of parameter, then the result of the expansion is the value of parameter with the shortest matching pattern (the ‘#’ case) or the longest matching pattern (the ‘##’ case) deleted.
You're not seeing any difference in this case because when you list a directory it just outputs the filenames, it doesn't include the directory portion, so there's nothing to remove. You would see the difference if you did:
for FILE in some/directory/*
This question already has answers here:
Backticks vs braces in Bash
(3 answers)
Brackets ${}, $(), $[] difference and usage in bash
(1 answer)
Closed 4 years ago.
I have two questions and could use some help understanding them.
What is the difference between ${} and $()? I understand that ()
means running command in separate shell and placing $ means passing
the value to variable. Can someone help me in understanding
this? Please correct me if I am wrong.
If we can use for ((i=0;i<10;i++)); do echo $i; done and it works fine then why can't I use it as while ((i=0;i<10;i++)); do echo $i; done? What is the difference in execution cycle for both?
The syntax is token-level, so the meaning of the dollar sign depends on the token it's in. The expression $(command) is a modern synonym for `command` which stands for command substitution; it means run command and put its output here. So
echo "Today is $(date). A fine day."
will run the date command and include its output in the argument to echo. The parentheses are unrelated to the syntax for running a command in a subshell, although they have something in common (the command substitution also runs in a separate subshell).
By contrast, ${variable} is just a disambiguation mechanism, so you can say ${var}text when you mean the contents of the variable var, followed by text (as opposed to $vartext which means the contents of the variable vartext).
The while loop expects a single argument which should evaluate to true or false (or actually multiple, where the last one's truth value is examined -- thanks Jonathan Leffler for pointing this out); when it's false, the loop is no longer executed. The for loop iterates over a list of items and binds each to a loop variable in turn; the syntax you refer to is one (rather generalized) way to express a loop over a range of arithmetic values.
A for loop like that can be rephrased as a while loop. The expression
for ((init; check; step)); do
body
done
is equivalent to
init
while check; do
body
step
done
It makes sense to keep all the loop control in one place for legibility; but as you can see when it's expressed like this, the for loop does quite a bit more than the while loop.
Of course, this syntax is Bash-specific; classic Bourne shell only has
for variable in token1 token2 ...; do
(Somewhat more elegantly, you could avoid the echo in the first example as long as you are sure that your argument string doesn't contain any % format codes:
date +'Today is %c. A fine day.'
Avoiding a process where you can is an important consideration, even though it doesn't make a lot of difference in this isolated example.)
$() means: "first evaluate this, and then evaluate the rest of the line".
Ex :
echo $(pwd)/myFile.txt
will be interpreted as
echo /my/path/myFile.txt
On the other hand ${} expands a variable.
Ex:
MY_VAR=toto
echo ${MY_VAR}/myFile.txt
will be interpreted as
echo toto/myFile.txt
Why can't I use it as bash$ while ((i=0;i<10;i++)); do echo $i; done
I'm afraid the answer is just that the bash syntax for while just isn't the same as the syntax for for.
your understanding is right. For detailed info on {} see bash ref - parameter expansion
'for' and 'while' have different syntax and offer different styles of programmer control for an iteration. Most non-asm languages offer a similar syntax.
With while, you would probably write i=0; while [ $i -lt 10 ]; do echo $i; i=$(( i + 1 )); done in essence manage everything about the iteration yourself
I have a theoretical question about the syntax of Bash.
I am running Bash 4.3.11(1) in Linux Ubuntu 14.04.
In the official GNU's website: Bash official web (GNU)
in Subection 9.3.1. it says:
!string
Refer to the most recent command preceding the current position
in the history list starting with string.
In general it's understood that string is, syntactically speaking, a sequence of characters ending before the first blank or newline.
However, when describing quoting in subsection 3.1.2., we can read in paragraph 3.1.2.2. what follows:
Enclosing characters in single quotes (‘'’) preserves the literal
value of each character within the quotes.
In particular, the blanks inside single quotes are not broking the strings in separated words.
So, a expression like !'some text' would have to search in the history list of Bash for the most recent command starting by 'some text'.
However, the blank between some and text is broken when I write it in my terminal, since the following error message is shown:
bash: !'some: event not found
Is this behaviour a bug in the implementation of the shell, or well I am not understanding the expansion rules of Bash for this example?
I wouldn't call the observed behaviour a bug, because there is no specification for history expansion other than the observed behaviour of the bash shell itself. But it is certainly the case that the precise mechanics of parsing a history expansion expression is not well documented and has a lot of possibly surprising corner cases.
The bash manpage does state that history expansion "is performed immediately after a complete line is read, before the shell breaks it into words" (emphasis added), while the bash manual mentions that history expansion is provided by the History library. This is the root cause of most of the history expansion parsing oddities: history expansion works on raw unparsed input without any assistance from the bash tokenizer, and is mostly done with an external library which is not bash-specific. Since tokenizing bash input is non-trivial, it is not really surprising that the relatively simple parsing rules used during history expansion are only a rough approximation to a real bash parse.
For example, the bash manual does indicated that you can prevent a history expansion character (!) from being recognized as such by backslash-quoting it. But it is not explicitly documented that any \ which immediately precedes an ! will inhibit recognition of the history expansion, even if the backslash was itself quoted with a backslash. So the ! in \\!word does not cause the previous command starting with word to be substituted. (\\word is a common way to execute the command word instead of the alias word, so the example is not entirely contrived.)
A longer discussion of some of the corner cases of the recognition of the history expansion character can be found in this answer.
The issue raised by this question is slightly different, since it is about the next phase of the history expansion parse. Once it has been established that a particular character is a history expansion character, it is then necessary to parse the "event" which follows; as indicated by the bash manual, the event can take several forms, one of which is !string, representing the most recent command which starts with "string".
It is implied that this form will only be used if no other form applies, which means that string may not start with a digit or -, !, # or ?. It also may not start with whitespace or = (since those would inhibit history expansion) and in some circumstances ( or " (which may inhibit history expansion). And finally, it may not start with ^, $, % or *, which would be interpreted as a word designator (from the default event, which is the previous command).
The bash manual does not specify what terminates the string. It is semi-documented in the history library manual, which mentions that a history search string (or "event" as it is called in the bash manual) is terminated by whitespace, :, or any of the characters in the history configuration variable history_search_delimiter_chars. (For the record, bash currently (v4.3) sets that variable to ";&()|<>".)
As indicated earlier, quoting is taken into account when deciding whether or not to recognize a history expansion character; as it turns out, if the history expansion occurs inside a double-quoted string, then the closing double-quote is also considered a history search delimiter character. And that, as far as I know, is the entire list of characters which will delimit !string.
Nowhere in either the bash nor the history documentation does it state that a history search delimiter character can be made non-special by quoting, and indeed this does not happen. An open quote, whether double or single, or even a backslash following the ! will be treated as just part of the string to be searched for, without any special processing.
Parsing of the substring-match history expansion -- !?string? -- is completely different. That string can only be terminated by a ? or by a newline. (As the bash manual says, the trailing ? is optional if terminated by a newline.)
Once the history expansion character has been recognized and the history search string has been identified, it may then be necessary to split the retrieved history entry into words. Again, the bash manual is slightly cavalier about corner cases, when it says that "the line is broken into words in the same fashion that Bash does, so that several words surrounded by quotes are considered one word."
A pedant would observe that "in the same fashion that Bash does" is not quite the same as saying "exactly as Bash would do", and in fact the second part of the sentence is literall true: several words surrounded by quotes are considered one word even if the quotes are not really matching quotes. For example, the line:
command "$(echo " foo bar ")"
is considered by the history library to consist of the following five words:
0. command
1. "$(echo "
2. foo
3. bar
4. ")"
although the bash parse would be quite different. By contrast, bash and the history library agree on the parsing of
command "$(echo ' foo bar ')"
as two words.
This question already has answers here:
Backticks vs braces in Bash
(3 answers)
Brackets ${}, $(), $[] difference and usage in bash
(1 answer)
Closed 4 years ago.
I have two questions and could use some help understanding them.
What is the difference between ${} and $()? I understand that ()
means running command in separate shell and placing $ means passing
the value to variable. Can someone help me in understanding
this? Please correct me if I am wrong.
If we can use for ((i=0;i<10;i++)); do echo $i; done and it works fine then why can't I use it as while ((i=0;i<10;i++)); do echo $i; done? What is the difference in execution cycle for both?
The syntax is token-level, so the meaning of the dollar sign depends on the token it's in. The expression $(command) is a modern synonym for `command` which stands for command substitution; it means run command and put its output here. So
echo "Today is $(date). A fine day."
will run the date command and include its output in the argument to echo. The parentheses are unrelated to the syntax for running a command in a subshell, although they have something in common (the command substitution also runs in a separate subshell).
By contrast, ${variable} is just a disambiguation mechanism, so you can say ${var}text when you mean the contents of the variable var, followed by text (as opposed to $vartext which means the contents of the variable vartext).
The while loop expects a single argument which should evaluate to true or false (or actually multiple, where the last one's truth value is examined -- thanks Jonathan Leffler for pointing this out); when it's false, the loop is no longer executed. The for loop iterates over a list of items and binds each to a loop variable in turn; the syntax you refer to is one (rather generalized) way to express a loop over a range of arithmetic values.
A for loop like that can be rephrased as a while loop. The expression
for ((init; check; step)); do
body
done
is equivalent to
init
while check; do
body
step
done
It makes sense to keep all the loop control in one place for legibility; but as you can see when it's expressed like this, the for loop does quite a bit more than the while loop.
Of course, this syntax is Bash-specific; classic Bourne shell only has
for variable in token1 token2 ...; do
(Somewhat more elegantly, you could avoid the echo in the first example as long as you are sure that your argument string doesn't contain any % format codes:
date +'Today is %c. A fine day.'
Avoiding a process where you can is an important consideration, even though it doesn't make a lot of difference in this isolated example.)
$() means: "first evaluate this, and then evaluate the rest of the line".
Ex :
echo $(pwd)/myFile.txt
will be interpreted as
echo /my/path/myFile.txt
On the other hand ${} expands a variable.
Ex:
MY_VAR=toto
echo ${MY_VAR}/myFile.txt
will be interpreted as
echo toto/myFile.txt
Why can't I use it as bash$ while ((i=0;i<10;i++)); do echo $i; done
I'm afraid the answer is just that the bash syntax for while just isn't the same as the syntax for for.
your understanding is right. For detailed info on {} see bash ref - parameter expansion
'for' and 'while' have different syntax and offer different styles of programmer control for an iteration. Most non-asm languages offer a similar syntax.
With while, you would probably write i=0; while [ $i -lt 10 ]; do echo $i; i=$(( i + 1 )); done in essence manage everything about the iteration yourself
I want to write a wrapper bash script, and to pass all arguments to a called program. I was very sure, that this works correctly:
#!/bin/sh
someProgam $#
But when passing exotic arguments (empty, unescaped, in quotes, ...) this fails.
For example: without the wrapper script, someProgram "1 2" 3 results in the arguments
[1 2] and [3].
But called from the script, I get [1], [2], [3].
Braces are just for visualization.
NOTE: It's a Java program, which is called. But I think it doesn't matter.
#!/bin/sh
someProgram "$#"
See also the bash docs on special parameters.
BTW1, "$#" is not specific to bash. You can probably rely on "$#" in cross-platform sh scripts to be run just about anywhere.
BTW2, in case this happens to be the last line in that script, you can save your operating system a few bytes and an entry in the process table by changing the line to something like
exec someProgram "$#"
to augment ndim's answer: the behavior of "$#" is not specific to bash. it's prescribed by the Single Unix Specification:
2.2.3 Double-Quotes
Enclosing characters in double-quotes ( "" ) shall preserve the literal value of all characters within the double-quotes, with the exception of the characters dollar sign, backquote, and backslash, as follows:
The parameter '#' has special meaning inside double-quotes and is described in Special Parameters.
2.5.2 Special Parameters
Listed below are the special parameters and the values to which they shall expand. Only the values of the special parameters are listed; see Word Expansions for a detailed summary of all the stages involved in expanding words.
#
Expands to the positional parameters, starting from one. When the expansion occurs within double-quotes, and where field splitting (see Field Splitting) is performed, each positional parameter shall expand as a separate field, with the provision that the expansion of the first parameter shall still be joined with the beginning part of the original word (assuming that the expanded parameter was embedded within a word), and the expansion of the last parameter shall still be joined with the last part of the original word. If there are no positional parameters, the expansion of '#' shall generate zero fields, even when '#' is double-quoted.