Expand a bash variable, but not the variables it contains - linux

I need to be able to expand a variable into exactly what is it declaration string is rather than bash expanding other variables in its declaration like it normally does.
variable=word
var="this variable contains a $variable"
echo ????
I need a command that results in: "this variable contains a $variable"
...rather than: "this variable contains a word"
The variable declaration must remain the same because it needs to expand normally in all other situations. So I can't use single quotes in the declaration.

What you're asking for is effectively impossible, because inside double quotes, $variable is substituted by the contents of that variable at the time of definition.
At the time you're printing it, the expansion has long happened (as can be seen using set).
If you cannot change the definition at all, the only thing you can try is to revert the substitution:
echo ${var/"$variable"/'$variable'}
This will replace every occurrence of what $variable currently holds by the string literal $variable, so in your example it would replace all occurrences of word with $variable, regardless of whether it was defined that way or not.
Also, as Jasper correctly pointed out, this relies on the fact that $variable does not change between the definition and the printing of $var.
Now, if you actually can change the definition, but only want the expansion to still happen, there are a number of options:
Use single quotes in the definition, and eval when printing.
var='this variable contains a $variable'
# later
eval "echo "'"'"$var"'"' # -> this variable contains a word
echo $var # -> this variable contains a $variable
Admittedly, this is rather cumbersome.
Also note that variable expansion happens at the time of printing rather than at the time of definition (actually, just at whatever time eval run).
Use two different variables.
Instead of writing eval "echo "'"'"$var"'"' every time, you could just store its result in a variable and use that, right?
var='this variable contains a $variable'
varX="$(eval "echo "'"'"$var"'"')"
# later
echo $varX # -> this variable contains a word
echo $var # -> this variable contains a $variable
Or, of course just copy the definition of $var and replace ' by "... but that's boring. :P
Here the expansion happens at the time of defining $varX.
Use an array instead of two different variables.
This is really interesting, because for arrays $var is equal to ${var[0]}, so you can do
tmp='this variable contains a $variable'
var=("$(eval "echo "'"'"$tmp"'"')" "$tmp")
# later
echo $var # -> this variable contains a word
echo ${var[1]} # -> this variable contains a $variable
This is probably as close as it gets to what you want, but you will still have to change the definition.

you can use single quote:
variable=word
var='this variable contains a $variable'
echo var

Also you can use sed
variable=word
var="this variable contains a $variable"
var=$(echo "${var}" | sed "s/\${variable}/${variable}/g")

Related

Why this bash function prints only first word of whole string?

I'm trying to create function that will print message bound to variable in certain color. The message variable is passed as argument of this function. The problem is that I'm getting only text up to first space (only first word of message). My script looks like this:
#!/usr/bash
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$1${NC}
}
echo_message $normalMessage
My output is:
Everything
As Inian pointed out in comments, your problem is unquoted variable expansion
echo_message $normalMessage
becomes
echo_message Everything fine
once the variable expands, meaning that each word of your input string is getting read in as a separate argument. When this happens $1=Everything and $2=fine.
This is fixed by double-quoting your variable, which allows expansion, but will mean the result of the expansion will still be read as one argument.
echo_message "$normalMessage"
becomes
echo_message "Everything fine"
Like this $1=Everything fine
In the future, I recommend using https://www.shellcheck.net/, or the CLI version of shellcheck, it will highlight all kinds of common bash gotcha's, included unquoted expansion.
For me, I had to change the header for "#!/bin/bash", but apparently that is not the problem for you.
In your echo you are printing only the first word with the $1, if you change it to $2 you will print the second word (parameter) and so on.
You can pass the name inside quotes or print all the parameters with $#
Solution 1 (with $#):
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$#${NC}
}
echo_message $normalMessage
Solution 2 (with quotes):
lbGREEN='\e[1;92m'
NC='\e[0m'
normalMessage="Everything fine"
echo_message() {
echo -e ${lbGREEN}$1${NC}
}
echo_message "$normalMessage"
You should get a look to https://stackoverflow.com/a/6212408/1428602
IMHO, $1 only return the 1st word, so you have to use a loop or try with $*
You've got the quoting wrong.
If you want to simulate the behaviour of echo, your function should accept multiple parameters, and print them all. Currently it's only evaluating the first parameter, so I suggest using $* instead. You also need to enclose the argument in double quotes to protect any special characters:
echo_message() {
echo -e "${lbGREEN}$*${NC}"
}
The special variable $* expands to all the arguments, separated by spaces (or more accurately, the first character of $IFS, which is usually a space character). Note that you almost always want "$#" instead of "$*", and this is one of the rare occasions where the latter is also correct, though with slightly different semantics if IFS is set to a non-standard value.
Now the function supports multiple arguments, and prints them all in green, separated by spaces. However, I would recommend that you also quote the argument when calling the function:
echo_message "$normalMessage"
While spaces in $normalMessage will now be treated correctly, other special characters like ! will still require the quotes.

Combining case changing and substring variable expansion

If you append ^ to a variable, Bash capitalises the first letter of its contents. (Similarly, , sends it to lowercase and doubling-up either of these applies the transformation to the whole string, rather than just the first letter.)
foo="hello world"
echo ${foo^} # Hello world
You can also do ${variable:position:length} to extract a substring:
echo ${foo:0:1} # h
So far, I haven't found a way to combine these without, obviously, creating a temporary variable. Is there a form where I can get just the capitalised first letter out of an arbitrary string?
It does not change the basic limitation you are seeing in terms of not being able to "chain" expansions, but you can assign the result of an expansion to the same variable and do away with the temporary variable.
For instance:
A=text
A="${A^}"
A="${A//x/s}"
echo "$A"
echoes "Test".
No. Parameter expansion operators do not compose, so if you want more than one side effect, you need a temporary variable (which can include overwriting the original value as shown by #fred) or an external tool to process the result of the expansion (as shown by #anubhava).
Your other alternative is to use a different shell that does support more complicated operations, like zsh:
% foo="hello world"
% % print ${(U)${foo:0:1}}
H
You can use tr with substring:
tr [[:lower:]] [[:upper:]] <<< "${foo:0:1}"
H

What does whitespace actually mean in bash?

I have something like this:
projectName= echo $tempPBXProjFilePath | sed "s/.*\/\(.*\)\.xcodeproj.*$/\1/g";
I want to extract substring from $tempPBXProjFilePath. And this is correct. However, if I write it like this:
projectName=echo $tempPBXProjFilePath | sed "s/.*\/\(.*\)\.xcodeproj.*$/\1/g";
It is wrong. The difference is the whitespace after the variable.
I know there is no whitespace after variable directly. But what's the meaning of the whitespace after equal-sign. Is there any place whitespace has special meaning?
Variable Assignment
The syntax for variable assignment is:
name=value
Note, there are no spaces around the = sign. If the value has spaces, or special characters, it should be quoted with single quotes:
name='value with spaces or special characters'
or with double quotes for variable expansion:
name="stringA $variable stringB"
If quotes are missing, the second word in the value part is interpreted as a command. Actually, this is a way to pass environment variables to a command (see below).
If the value is missing, a variable with an empty value is created.
Environment Variables
There is another syntax that allows to assign environment variables for a command:
nameA=valueA nameB=valueB nameC=valueC command arguments
The name-value pairs are separated with space characters.
Example
LD_PRELOAD=/path/to/my/malloc.so /bin/ls
The command assigns LD_PRELOAD environment variable to /path/to/my/malloc.so before invoking /bin/ls.
Your Commands
Thus, your command:
projectName= echo $tempPBXProjFilePath
actually means that you call echo command with arguments expanded from $tempPBXProjFilePath, and set projectName environment variable to an empty value.
And this command:
projectName=echo $tempPBXProjFilePath
sets projectName environment variable to echo string, and calls a command expanded from $tempPBXProjFilePath variable.
Note, if a variable is not enclosed in double quotes, the special characters that present in its value are interpreted by the shell. In order to prevent reinterpretation of the special characters, you should use weak quoting: "$variable". And if you want to prevent even variable expansion in a string value, use single quotes: 'some value'.
Bash divides each line into words at each whitespace (spaces or tabs).
The first word it finds is the name of the command to be executed and the remaining words become arguments to that command.
so when you pass
projectName=echo
bash understand projectName=echo as a variable assignment, and
$tempPBXProjFilePath | sed "s/.*\/\(.*\)\.xcodeproj.*$/\1/g";
as a command! (as pointed by Chris Dodd)
Whitespace
Putting spaces on either or both sides of the equal-sign (=) when assigning a value to a variable will fail.
INCORRECT 1
example = Hello
INCORRECT 2
example= Hello
INCORRECT 3
example =Hello
The only valid form is no spaces between the variable name and assigned value:
CORRECT 1
example=Hello
CORRECT 2
example=" Hello"
You can see more at:
http://wiki.bash-hackers.org/scripting/newbie_traps

How to store output from printf with formatting in a variable? [duplicate]

This question already has answers here:
I just assigned a variable, but echo $variable shows something else
(7 answers)
Closed 7 years ago.
I would like to store the output of printf with formatting in a variable, but it strips off the formatting for some reason.
This is the correct output
$ printf "%-40s %8s %9s %7s" "File system" "Free" "Refquota" "Free %"
File system Free Refquota Free
And now the formatting is gone
$ A=$(printf "%-40s %8s %9s %7s" "File system" "Free" "Refquota" "Free %")
$ echo $A
File system Free Refquota Free %
echo will print each of it's arguments in order, separated by one space. You are passing a bunch of different arguments to echo.
The simple solution is to quote $A:
A=$(printf "%-40s %8s %9s %7s" "File system" "Free" "Refquota" "Free %")
echo "$A"
This is because you are not quoting the variable. If you do, the format will show perfectly:
echo "$A" #although $a would be best, uppercase vars are not good practise
That is, your var=$(printf ) approach is completely fine, you just fail to echo properly.
You may want to know why. Find it in Why does my shell script choke on whitespace or other special characters?
Why do I need to write "$foo"? What happens without the quotes?
$foo does not mean “take the value of the variable foo”. It means
something much more complex:
First, take the value of the variable. * Field splitting: treat
that value as a whitespace-separated list of fields, and build the
resulting list. For example, if the variable contains foo * bar ​
then the result of this step is the 3-element list foo, *, bar.
Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containing foo,
following by the list of files in the current directory, and finally
bar. If the current directory is empty, the result is foo, *,
bar.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$#" to expand to the list of positional parameters, e.g. "$# is
equivalent to "$1" "$2" "$3" if there are three positional
parameters. See What is the difference between $* and $#?)
The same happens to command substitution with $(foo) or with
`foo`. On a side note, don't use `foo`: its quoting rules are
weird and non-portable, and all modern shells support $(foo) which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assuming IFS doesn't contain digits or
-).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.

Expanding known env vers in a file and leaving others

I'm sure there's a really easy way of doing this. I'm trying to take a file which contains some environment variables and expand it so that those which are known are expanded to their values whereas those which are not are left alone.
For example, if my file contained the following:
${I_EXIST}
${I_ALSO_EXIST}
${I_DONT_EXIST}
this would be expanded to:
existValue
alsoExistValue
${I_DONT_EXIST}
I ideally want to do this as simply as possible so I don't want a complex substitution using sed, awk or perl. I'm thinking of something similar to a "Here" file, but apart from the fact that I can't get the syntax right, it also blanks out anything which does not expand. E.g:
cat <<EOF
> ${I_EXIST}
> ${I_ALSO_EXIST}
> ${I_DONT_EXIST}
EOF
existValue
alsoExistValue
(i.e. the last value expands to nothing)
Update
Should really have made clear that I was thinking about potentially more than one substitution per line. One way I did find to do this, if we're not fussed about the variables appearing in the file as ${MYVAR} but maybe MYVAR will do:
m4 $( env | sed 's/\([A-Za-z0-9]*\)=\([\/A-Za-z_0-9:|%*. -#]*\)/-D\1=\2' ) myfile
This uses the M4 preprocessor to substitute all the pairs in your environment. A couple of caviats here:
Sorry about the reg exp stuff. It looks pretty nasty and I'm sure there are nicer ways of expressing this. I found problems if my env vars had spaces in them or any unusual characters that weren't in the set.
Of course this is a blunt substitution tool (which I was trying to avoid) so variable might get substituted when you didn't want it to happen.
#!/bin/bash
while read a;
do
n=$(eval echo $a)
if [[ "$n" == "" ]]
then
echo $a
else
echo $n
fi
done < input
Using this as input
${HOME}
${nonexistent}
Gives
/home/myuser
${nonexistent}
Easy to read? Maybe not. It is short and works though :-)
while read r; do
echo $(eval echo ${r%\}}:-'$r'\})
done < input
Magic used:
http://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion
Edit: Further explanation, I hope it makes some sense.
We use two techniques; from the above docs:
${parameter:−word} If parameter is
unset or null, the expansion of word
is substituted. Otherwise, the value
of parameter is substituted.
And
${parameter%word}
The word is expanded to produce a
pattern just as in filename expansion.
If the pattern matches a trailing
portion of the expanded value of
parameter, then the result of the
expansion is the value of parameter
with the shortest matching pattern
(the ‘%’ case) or the longest matching
pattern (the ‘%%’ case) deleted. ...
We use the fact that the input is just what we can use in the shell, we have ${FOOBAR} but need ${FOOBAR:-'${FOOBAR}'} (Single quotes to avoid expansion).
# echo ${doesntexist:-Hello}
Hello
# doesexist=World
# echo ${doesexist:-Will not be printed}
World
So what we need to inject is :-'${FOOBAR}'
To achieve this we trim the } at the end, add the string, then put another } back afterwards.
# echo $r
${FOOBAR}
# echo ${r%\}}
${FOOBAR
The final \} isn't really necessary, since it's got no beginning in this case, but it's better to be explicit and escape it. (Much like you would escape echo \* even if echo * without any matching files gives you a literal *).
Edit2: This of course doesn't take into account that you wanted to support multiple variables in a single row; or any rows with other stuff in them.
while read name; do echo "$name = " $(eval echo $name); done < file_with_vars.txt
will echo all variables what know.
e.g.
in my file called vv
${PATH}
${HAVENOT}
${LOCALE}
will print
${PATH} = /usr/local/narwhal/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:~/bin
${HAVENOT} =
${LOCALE} = UTF-8
modify the output format as you wish :)

Resources