This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Command substitution: backticks or dollar sign / paren enclosed?
I want know the difference between use
var="$(command)"
and
var=`command`
in bash scripting, aparently are two ways to get the same result, but is possible that exist some diferences.
The first is better, as you can nest the command substitutions and it doesn't become awkward.
Further Reading.
From bash man page:
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.
(more interesting things there; do read the whole thing :) )
Related
I want to run a command from a bash script which has single quotes and some other commands inside the single quotes and a variable.
e.g. repo forall -c '....$variable'
In this format, $ is escaped and the variable is not expanded.
I tried the following variations but they were rejected:
repo forall -c '...."$variable" '
repo forall -c " '....$variable' "
" repo forall -c '....$variable' "
repo forall -c "'" ....$variable "'"
If I substitute the value in place of the variable the command is executed just fine.
Please tell me where am I going wrong.
Inside single quotes everything is preserved literally, without exception.
That means you have to close the quotes, insert something, and then re-enter again.
'before'"$variable"'after'
'before'"'"'after'
'before'\''after'
Word concatenation is simply done by juxtaposition. As you can verify, each of the above lines is a single word to the shell. Quotes (single or double quotes, depending on the situation) don't isolate words. They are only used to disable interpretation of various special characters, like whitespace, $, ;... For a good tutorial on quoting see Mark Reed's answer. Also relevant: Which characters need to be escaped in bash?
Do not concatenate strings interpreted by a shell
You should absolutely avoid building shell commands by concatenating variables. This is a bad idea similar to concatenation of SQL fragments (SQL injection!).
Usually it is possible to have placeholders in the command, and to supply the command together with variables so that the callee can receive them from the invocation arguments list.
For example, the following is very unsafe. DON'T DO THIS
script="echo \"Argument 1 is: $myvar\""
/bin/sh -c "$script"
If the contents of $myvar is untrusted, here is an exploit:
myvar='foo"; echo "you were hacked'
Instead of the above invocation, use positional arguments. The following invocation is better -- it's not exploitable:
script='echo "arg 1 is: $1"'
/bin/sh -c "$script" -- "$myvar"
Note the use of single ticks in the assignment to script, which means that it's taken literally, without variable expansion or any other form of interpretation.
The repo command can't care what kind of quotes it gets. If you need parameter expansion, use double quotes. If that means you wind up having to backslash a lot of stuff, use single quotes for most of it, and then break out of them and go into doubles for the part where you need the expansion to happen.
repo forall -c 'literal stuff goes here; '"stuff with $parameters here"' more literal stuff'
Explanation follows, if you're interested.
When you run a command from the shell, what that command receives as arguments is an array of null-terminated strings. Those strings may contain absolutely any non-null character.
But when the shell is building that array of strings from a command line, it interprets some characters specially; this is designed to make commands easier (indeed, possible) to type. For instance, spaces normally indicate the boundary between strings in the array; for that reason, the individual arguments are sometimes called "words". But an argument may nonetheless have spaces in it; you just need some way to tell the shell that's what you want.
You can use a backslash in front of any character (including space, or another backslash) to tell the shell to treat that character literally. But while you can do something like this:
reply=\”That\'ll\ be\ \$4.96,\ please,\"\ said\ the\ cashier
...it can get tiresome. So the shell offers an alternative: quotation marks. These come in two main varieties.
Double-quotation marks are called "grouping quotes". They prevent wildcards and aliases from being expanded, but mostly they're for including spaces in a word. Other things like parameter and command expansion (the sorts of thing signaled by a $) still happen. And of course if you want a literal double-quote inside double-quotes, you have to backslash it:
reply="\"That'll be \$4.96, please,\" said the cashier"
Single-quotation marks are more draconian. Everything between them is taken completely literally, including backslashes. There is absolutely no way to get a literal single quote inside single quotes.
Fortunately, quotation marks in the shell are not word delimiters; by themselves, they don't terminate a word. You can go in and out of quotes, including between different types of quotes, within the same word to get the desired result:
reply='"That'\''ll be $4.96, please," said the cashier'
So that's easier - a lot fewer backslashes, although the close-single-quote, backslashed-literal-single-quote, open-single-quote sequence takes some getting used to.
Modern shells have added another quoting style not specified by the POSIX standard, in which the leading single quotation mark is prefixed with a dollar sign. Strings so quoted follow similar conventions to string literals in the ANSI standard version of the C programming language, and are therefore sometimes called "ANSI strings" and the $'...' pair "ANSI quotes". Within such strings, the above advice about backslashes being taken literally no longer applies. Instead, they become special again - not only can you include a literal single quotation mark or backslash by prepending a backslash to it, but the shell also expands the ANSI C character escapes (like \n for a newline, \t for tab, and \xHH for the character with hexadecimal code HH). Otherwise, however, they behave as single-quoted strings: no parameter or command substitution takes place:
reply=$'"That\'ll be $4.96, please," said the cashier'
The important thing to note is that the single string that gets stored in the reply variable is exactly the same in all of these examples. Similarly, after the shell is done parsing a command line, there is no way for the command being run to tell exactly how each argument string was actually typed – or even if it was typed, rather than being created programmatically somehow.
Below is what worked for me -
QUOTE="'"
hive -e "alter table TBL_NAME set location $QUOTE$TBL_HDFS_DIR_PATH$QUOTE"
EDIT: (As per the comments in question:)
I've been looking into this since then. I was lucky enough that I had repo laying around. Still it's not clear to me whether you need to enclose your commands between single quotes by force. I looked into the repo syntax and I don't think you need to. You could used double quotes around your command, and then use whatever single and double quotes you need inside provided you escape double ones.
just use printf
instead of
repo forall -c '....$variable'
use printf to replace the variable token with the expanded variable.
For example:
template='.... %s'
repo forall -c $(printf "${template}" "${variable}")
Variables can contain single quotes.
myvar=\'....$variable\'
repo forall -c $myvar
I was wondering why I could never get my awk statement to print from an ssh session so I found this forum. Nothing here helped me directly but if anyone is having an issue similar to below, then give me an up vote. It seems any sort of single or double quotes were just not helping, but then I didn't try everything.
check_var="df -h / | awk 'FNR==2{print $3}'"
getckvar=$(ssh user#host "$check_var")
echo $getckvar
What do you get? A load of nothing.
Fix: escape \$3 in your print function.
Does this work for you?
eval repo forall -c '....$variable'
This question already has answers here:
How does one escape backslashes and forward slashes in VIM find/search?
(6 answers)
Closed 2 years ago.
for example, I want to find and replace, say my original string is "//hello" to "hello".
Actually I'm trying to uncomment lines which I've commented before.
substitution
if your pattern contains slashes /, e.g. //hello you can of course escape them in s/pat/repl/ command. However, better to pick some another delimiter for your s command:
:s#//hello#whatever#g
In this way, the command is easier to read and understand.
Search
Say If you want to search //hello, you can try a backward search with ?, then you don't have to escape the slashes.
Did you try this:
%s/\/\/hello/hello
This question already has answers here:
Can parameter expansion be nested in Bash? [duplicate]
(2 answers)
Can ${var} parameter expansion expressions be nested in bash?
(15 answers)
Closed 4 years ago.
Is it possible to perform more than one bash variable manipulation in one shot?
For example:
# for this variable
foo="string_that_is_lower"
# can I do this in one shot?
$foo="${foo:0:4}"
echo "${foo^^}"
Is there a way to combine these? I realize this is a trivial problem because this works just fine, or one could simply use other built in tools like echo ${foo:0:4} | tr [[:lower:]] [[:upper:]], but that is lame.
I've tried every logical combination I can think might work:
${${foo^^}:0:4}
${{foo^^}:0:4}
${foo^^,:0:4}
${foo^^;:0:4}
${foo^^ :0:4}
All produce syntax errors.
I find no instances of "(combine|multiple) bash variable manipulations" in the Advanced Bash Scripting Manual, or the manpage, or Google, or here, so maybe you just can't do it, but probably I'm just not searching for the right terms.
It is a very good question, but not something that parameter expansion allows. man bash provides that expansion operates on a parameter name or symbol, e.g.
Parameter Expansion
The `$' character introduces parameter expansion, command
substitution, or arithmetic expansion. The parameter name or
symbol to be expanded may be enclosed in braces, which are optional
but serve to protect the variable to be expanded from characters
immediately following it which could be interpreted as partof the
name.
When you seek to chain expansions together, you are attempting to use the text that results from the first expansion as a parameter name or symbol (which it isn't), so the command fails with a syntax error.
Good question!
(note: to be complete -- there are some instances where you can embed a parameter expansion within another parameter expansion, but those are limited to where the outer expansion is looking for a single number as part of its expansion which can be provided by an included expansion such as the length of a string, e.g. ${#var} in v=foo; echo ${v:$((${#v}-1))})
More precisely, why does
"`command "$variable"`"
treat the outer quotes as enclosing the inner quotes, instead of expanding the variable outside any quotes?
The exact command I used to test this is similar to an example brought up in another stackoverflow question about the correct method of quoting when using command substitution:
fileName="some path with/spaces"
echo "`dirname "$fileName"`"
which correctly echoes "some path with", instead of complaining because of an invalid number of arguments.
I read Bash's man page, where it states in chapter "EXPANSION", section "Commmand Substitution" that the new-style $() substitution preserves the meaning of any character between the parentheses, however, regarding backquotes, it only mentions that backslashes work in a limited way:
When the old-style backquote form of substitution is used, backslash retains
its literal meaning except when followed by $, `, or \. The first backquote
not preceded by a backslash terminates the command substitution.
My first thought was that backticks do the same, aside from the mentioned exception, thus "quoting" the inner double quotes, however, I got told that is not the case.
The second observation that pointed me to this direction was that
a=\$b
b=hello
echo `echo $a`
prints "$b". Had the backticks let the dollar sign get interpreted, the first variable substitution should have occurred before the subshell was invoked, with the subshell expanding the string "$b", resulting in "hello."
According to the above excerpt from the man page, I can even make sure the dollar sign is actually quoted, by using
echo `echo \$a`
and the results would still be the same.
A third observation gives me some doubts though:
echo `echo \\a`
Result: "\a"
echo \a
Result: a
Here it seems like both backslashes were retained until the subshell came into play, even though the man page states that backslashes within backquotes do not have their literal meaning when followed by another backslash.
EDIT: ^ Everything works as expected in this regard, I must have used the wrong shell (tcsh in my other terminal, and with a different character from "a").
Although I have not been able to find out what actually happens, while I was searching for the answer, I came across some people mentioning the term "quoting context" with regards to command substitution, but without any explanation as to what it means or where it is described.
I have not found any real reference to "quoting contexts" in either Bash references (gnu.org, tldp, man bash) or via DuckDuckGo.
Additionally to knowing what is going on, I'd preferably like to have some reference or guidance as to how this behavior can be discerned from it, because I think I might have failed to put some pieces together from which this naturally comes. Otherwise I'll just forget the answer.
To those recommending people to use the new-style dollar sign and parentheses substitution: on ca. 50 years old Unix machines with tens or hundreds of different proprietary environments (can't throw out a shell for a newer one), when one has to write scripts compatible between most shells that anyone might be using, it is not an option.
Thanks to anyone who can help me.
POSIX has this to say in 2.2.3 (emphasis mine):
` (backquote)
The backquote shall retain its special meaning introducing the other form of command substitution (see Command Substitution). The portion of the quoted string from the initial backquote and the characters up to the next backquote that is not preceded by a <backslash>, having escape characters removed, defines that command whose output replaces "`...`" when the word is expanded. Either of the following cases produces undefined results:
A single-quoted or double-quoted string that begins, but does not end, within the "`...`" sequence
A "`...`" sequence that begins, but does not end, within the same double-quoted string
This, to me, pretty much defines what other people might (informally?) call a quoting context comprising everything within two consecutive backquotes.
In a way, the backquote is the fourth quote in addition to single quote, double quote and backslash. Note that within double quotes, single quotes lose their quoting capability as well, so it should be no surprise that backquotes change the function of double quotes within them.
I tried your example with other shells, like the Almquist shell on FreeBSD and zsh. As expected, they output some path with.
This question already has answers here:
How do I learn how to get quoting right in bash?
(3 answers)
Closed 8 years ago.
When BASH scripting, I am often confused when to put variables in quotes or just have them called without quotes.
"$i" vs. $i
echo hello vs echo "hello"
-eq vs ==
$((i%2))
Could someone point me to a resource that explains this well or give me some basic tips? Thanks.
Here's the basic tip: always double-quote your variable usages in Bash, unless you know for sure you need to not quote them. Quoting them is safer, as it prevents word splitting; you will know during development when you need word splitting and therefore should not quote a variable.