To store the output of a command as a variable in sh/ksh/bash, you can do either
var=$(command)
or
var=`command`
What's the difference if any between the two methods?
The backticks/gravemarks have been deprecated in favor of $() for command substitution because $() can easily nest within itself as in $(echo foo$(echo bar)). There are other differences such as how backslashes are parsed in the backtick/gravemark version, etc.
See BashFAQ/082 for several reasons to always prefer the $(...) syntax.
Also see the POSIX spec for detailed information on the various differences.
They behave the same. The difference is syntactical: it's easier to nest $() than ``:
listing=$(ls -l $(cat filenames.txt))
vs.
listing=`ls -l \`cat filenames.txt\``
July 2014: The commit f25f5e6 (by Elia Pinto (devzero2000), April 2014, Git 2.0) adds to the nesting issue:
The backquoted form is the traditional method for command substitution, and is supported by POSIX.
However, all but the simplest uses become complicated quickly.
In particular, embedded command substitutions and/or the use of double quotes require
careful escaping with the backslash character.
That is why the git/Documentation/CodingGuidelines mentions:
We prefer $( ... ) for command substitution; unlike ``, it properly nests.
It should have been the way Bourne spelled it from day one, but unfortunately isn't.
thiton commented:
That is why `echo `foo`` won't work in general because of the inherent ambiguity because each ``can be opening or closing.
It might work for special cases due to luck or special features.
Update January 2016: Git 2.8 (March 2016) gets rid of backticks entirely.
See commit ec1b763, commit 9c10377, commit c7b793a, commit 80a6b3f, commit 9375dcf, commit e74ef60, commit 27fe43e, commit 2525c51, commit becd67f, commit a5c98ac, commit 8c311f9, commit 57da049, commit 1d9e86f, commit 78ba28d, commit efa639f, commit 1be2fa0, commit 38e9476, commit 8823d2f, commit 32858a0, commit cd914d8 (12 Jan 2016) by Elia Pinto (devzero2000).
(Merged by Junio C Hamano -- gitster -- in commit e572fef, 22 Jan 2016)
From Git 2.8 onwards, it is all $(...), no more `...`.
When the older back-tick form is used, backslash retains its literal meaning except when followed by $, `, or \. The first back-tick not preceded by a backslash terminates the command substitution.
When using the newer $(command) form, all characters between the parentheses make up the command; none are treated specially.
Both forms can be nested, but the back-tick variety requires the following form.
`echo \`foo\``
As opposed to:
$(echo $(foo))
There is little difference, except for what unescaped characters you can use inside of the command. You can even put `...` commands inside $(...) ones (and vice versa) for a more complicated two-level-deep command substitution.
There is a slightly different interpretation of the backslash character/operator. Among other things, when nesting `...` substitution commands, you must escape the inner ` characters with \, whereas with $() substition it understands the nesting automatically.
"What's the difference if any between the two methods?"
Make attention to this behaviour:
A="A_VARIABLE"
echo "$(echo "\$A")"
echo "`echo "\$A"`"
You will get these results:
$A
A_VARIABLE
Related
I'm executing git ls-remote ssh://git#git_repo:port * in two different computers under same network, one Linux another Windows, and on Windows I'm getting the list but on Linux not. No error at all just and empty list on Linux.
Both has the SSH key added to the remote repository and both are able to clone the repository.
Update 1:
Windows Git version: 2.19.2.windows.1
Linux Git version: 2.7.4
Update 2:
The repository is in Gerrit.
Update 3:
I'm facing this problem using the Jenkins plugin Extended Choice Parameter plugin. It has no change since 2016. Any workaround for this would be also an answer.
Any idea?
You probably should use:
git ls-remote ssh://git#git_repo:port
without any suffix, as it defaults to listing everything.
You can use:
git ls-remote ssh://git#git_repo:port '*'
(or the same with double quotes—one or both of these may work on Windows as well). In a Unix/Linux-style command shell, the shell will replace * with a list of all the files in the current directory before running the command, unless you protect the asterisk from the shell.
You can also use a single backlash:
git ls-remote ssh://git#git_repo:port \*
as there are a lot of ways to protect individual characters from shells. The rules get a little complicated, but in general, single quotes are the "most powerful" quotes, while double quotes quote glob characters1 but not other expansions.2 Backslashes quote the immediate next character if you're not already inside quotes (the behavior of backslash within double quotes varies in some shells).
1The glob characters are *, [, and ?. After [, characters inside the glob run to the closing ]. So echo foo[abc] looks for files named fooa, foob, and fooc. Note that . is generally not special: foo.* matches only files whose names start with foo., i.e., including the period: a file named foo does not start with foo., only with foo, and is not matched.
Globs are very different from regular expressions: in regular expressions, . matches any character (like ? does in glob) and asterisk means "repeat previous match zero or more times", so that glob * and regular-expression .* are similar. (In regular expression matches, we also need to consider whether the expression is anchored. Globs are always anchored so that the question does not arise.)
2Most expansions occur with dollar sign $, as in $var or ${var} or $(subcommand), but backquotes also invoke command substitution, as in echo `echo bar`.
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
Below given sed command is working fine on online BASH & KSH shell, but getting an error "Illegal operation --r" while trying to run it on linux server.
I'm trying to make a regex to parse MFBBMYKLAXXX from first line.
echo "{1:F01MFBBMYKLAXXX2474811384}{2:O3001434181108BKKBTHBKBXXX12203020241811081534N}{3:{108:241C182AFFD4403C}}{4:
:15A:
:20:10168957
:22A:NEWT
:94A:BILA
:22C:BKKBBK8308MFBBKL
:82A:BKKBTHBK
:87A:MFBBMYKL
:15B:
:30T:20181108
:30V:20181109
:36:32,8308
:32B:THB2500000,
:53A:/610165
BKKBTHBK
:57A:BKKBTHBK
:33B:USD76148,01
:53A:CHASUS33
:57A:/04058664
BKTRUS33
:58A:MFBBMYKL
:15C:
:24D:ELEC/REUTERS D-3000
-}{5:{CHK:4117CD0206B7}}{S:{COP:S}}
" | sed -rn 's/.*\{1:F01([A-Z]{12}).*/\1/p'
The use of sed -r (or in some dialects sed -E) is nonstandard and optional.
It selects a regex dialect called extended regular expressions, which allows you to express some things more succinctly.
POSIX basic regular expressions support pretty much the same facilities, but with an oddball syntax where you have to backslash some characters to obtain their special meaning (which in other words does exactly the opposite of what backslash escaping originally did).
So if you have an extended regular expression like a+(b{2})c then if your sed does not support either -r or -E, try a\+\(b\{2\}\}c without any special option, and hope that your sed is at least roughly on par with what POSIX specifies. (If you're serious about retrocomputing, this is unlikely, though.)
The original regular expression implementation by Ken Thompson only supported the regex metacharacters [...] and . and *, and for a long time, that's all sed supported, too.
Of course, you could always install a more modern sed. I know SunOS used to have some goodies in their xpg4 directory but I have no idea if this was is the case in Solaris; if so, maybe you just need to add /usr/xpg4/bin to your PATH. (According to this it was true at one point in time at least.)
I'm experimenting with bash scripting and noticed the following behavior:
file="test.jar"
echo "${file: -4}" #prints .jar
echo "${file:-4}" #prints test.jar
Very confusing behavior actually. Can someone explain why the second case prints the whole test.jar?
This is due to inconsistent syntax. {"string":-} means default (all the string), whatever follows -. So you need either a space or parenthesis:
{"string": -4}
{"string":(-4)}
Read bash string manipulation.
This is a compromise due to the timeline of features being added to Bash.
The ${parameter:-word} or ${parameter-word} syntax for "replace parameter with word if parameter is null or unset (:-) / null (-)" was around for pretty much always; the - version was already in the Version 7 Bourne Shell.
The ${parameter:offset:length} and ${parameter:offset} syntax for "substring of parameter starting at offset (with optional length length)" was introduced in Bash 2.0 (no conflict so far).
Negative offsets and length specifications for the substring construct were introduced in Bash 4.2. This leads to a problem:
$ string=01234567890abcdefgh
$ echo ${string:7:-2} # Not ambiguous
7890abcdef
$ echo ${string:-7} # Interpreted as "undefined/null, or..."
01234567890abcdefgh
$ echo ${string: -7} # Interpreted as offset from the end
bcdefgh
$ echo ${string:(-7)} # Interpreted as offset from the end
bcdefgh
The space before - or the parentheses around the negative offset are there to tell the expansion apart from the :- (default value) expansion.
If the space is not there, the expansion ${file:-4} is interpreted as "print 4 if the parameter file is null or unset, and the expansion of file otherwise".
References:
BashFAQ/061: Is there a list of which features were added to specific releases (versions) of Bash?
Bash hackers wiki: Bash changes
Shell parameter expansion in the bash manual
Bash NEWS file describing feature added per version
I am new to Linux.
I was debugging some code. I encountered the following command:
PROGRAM_ID=$(echo $PROGRAM_ID|sed 's/-/,/g')
Can anybody explain what the g represents here?
I understand hyphen is being replaced with comma.
The /g flag means, perform the substitution globally on a line. Without that flag, only the first hyphen on every line would get substituted.
A better way with Bash would be
PROGRAM_ID=${PROGRAM_ID//-/,}
but if you have to be portable to Bourne shell in general, this replacement facility is not available.
(In which case you should take care to keep "$PROGRAM_ID" in double quotes in the echo.)
Its easy to see how g (global) works with these two example:
echo "test-one-two-three" | sed 's/-/,/g'
test,one,two,three
echo "test-one-two-three" | sed 's/-/,/'
test,one-two-three
Without the g it only replace the first hit.