This question already has answers here:
Bash inserting quotes into string before execution
(2 answers)
Closed 5 years ago.
The following is a script where the empty variable is replaced by single quotes. If the variable is not empty then we have the correct substitution.
$ set -x; aa=""; bb=`curl "$aa" -vvv`; set +x
+ aa=
++ curl '' -vvv
The thing to notice is the single quotes in place of the empty variable.
When variable is not empty, everything works fine as in:
$ set -x; aa="google.com"; bb=`curl "$aa" -vvv`; set +x
+ aa=google.com
++ curl google.com -vvv
Q1: Why is an empty variable or a variable with space resulting in single quotes being introduced?
Q2: How do I prevent the single quotes in lieu of the empty variable?
Now, I can remove the double quotes and everything works fine but I need to preserve spaces if there are any.
Thanks.
Single quotes are not being introduced, they are just being used to show you that you are passing an empty argument. The double quotes you use are causing the empty argument to be passed. If you want no argument but want to allow whitespace in the variable, you'll have to do something like:
curl ${aa:+"$aa"} -vvv
This is different than curl ${aa+"$aa"} -vvv when aa is the empty string, but from your description it appears you want the former.
You are seeing the single quotes because you used double quotes around $aa, so curl sees the empty string as your parameter. You won't get this effect if you drop the double quotes like this:
set -x; aa=""; bb=`curl $aa -vvv`; set +x
This is because you enclosed variable substitution into double quotes. $aa expands to empty string, which is placed into quotes, therefore producing quotes with nothing. In this case curl will see 3 arguments in that order: curl, <empty string>, -vvv.
If you remove double quotes round $aa, the second argument will not be passed to curl, and it will see only curl and -vvv in argv parameter in it's main function:
$ set -x; aa=""; bb=`curl $aa -vvv`; set +x
+ aa=
++ curl -vvv
Related
This question already has answers here:
How do I set a variable to the output of a command in Bash?
(15 answers)
Bash variable from command with pipes, quotes, etc
(2 answers)
Variable variable assignment error -"command not found"
(1 answer)
Closed 1 year ago.
i have the following two lines in a batch script
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
$iperf_options=$iperf_options $proto
and
$streams = 2
$proto = -u
but when i run this i get the following error.
./bandwidth: line 116: -O: command not found
I am simply trying to wrote a string and then append it to a variable so why does it throw the error on the -O?
I have looked about the web but i jsut seem to find stuff about spaces around the "="
any help greatfully recived.
Thankyou
code block to show error
proto=-u
streams=2
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
$iperf_options=$iperf_options $proto
running this will give this out put
./test
./test: line 3: 2: command not found
./test: line 4: =: command not found
There are two main mistakes here, in a variety of combinations.
Use $ to get the value of a variable, never when setting the variable (or changing its properties):
$var=value # Bad
var=value # Good
var=$othervar # Also good
Spaces are critical delimiters in shell syntax; adding (or removing) them can change the meaning of a command in unexpected ways:
var = value # Runs `var` as a command, passing "=" and "value" as arguments
var=val1 val2 # Runs `val2` as a command, with var=val1 set in its environment
var="val1 val2" # Sets `var1` to `val1 val2`
So, in this command:
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
The space between iperf_options="..." and $streams means that it'll expand $streams and try to run it as a command (with iperf_options set in its environment). You want something like:
iperf_options=" -O 10 -V -i 10 --get-server-output -P $streams"
Here, since $streams is part of the double-quoted string, it'll be expanded (variable expand inside double-quotes, but not in single-quoted), and its value included in the value assigned to iperf_options.
There's actually a third mistake (or at least dubious scripting practice): building lists of options as simple string variables. This works in simple cases, but fails when things get complex. If you're using a shell that supports arrays (e.g. bash, ksh, zsh, etc, but not dash), it's better to use those instead, and store each option/argument as a separate array element, and then expand the array with "${arrayname[#]}" to get all of the elements out intact (yes, all those quotes, braces, brackets, etc are actually needed).
proto="-u" # If this'll always have exactly one value, plain string is ok
streams=2 # Same here
iperf_options=(-O 10 -V -i 10 --get-server-output -P "$streams")
iperf_options=("${iperf_options[#]}" "$proto")
# ...
iperf "${iperf_options[#]}"
Finally, I recommend shellcheck.net to sanity-check your scripts for common mistakes. A warning, though: it won't catch all errors, since it doesn't know your intent. For instance, if it sees var=val1 val2 it'll assume you meant to run val2 as a command and won't flag it as a mistake.
I am writing a bash script and trying to do error handling and sanitizing the supplied parameters of the script. The supplied parameters have the form of key/value pairs and are separated by the '&' for the purpose of API compatibility:
cluster=xyz&tenant=abcd1234&key1=value1&key2=value2
In the simplest form, just to print out the supplied parameter, this script is just two lines:
#!/bin/bash
echo "The supplied parameter"
echo "$1"
When calling the script with the parameters in single or double quotes, everything works as expected:
$./script.sh 'cluster=xyz&tenant=abcd1234&key1=value1&key2=value2'
The supplied parameters
cluster=xyz&tenant=abcd1234&key1=value1&key2=value2
$./script.sh "cluster=xyz&tenant=abcd1234&key1=value1&key2=value2"
The supplied parameters
cluster=xyz&tenant=abcd1234&key1=value1&key2=value2
However, if I don't single/double quote the string, it causes the script to hang:
$ ./script.sh cluster=xyz&tenant=abcd1234&key1=value1&key2=value2
[1] 1080
[2] 1081
[3] 1082
[2] Done tenant=abcd1234
[3]+ Done key1=value1
$ The supplied parameters
cluster=xyz
And above stays until I press ctrl+c.
My question - how to properly sanitize the string when it is NOT enclosed in single/double quotes and prevent the above from occurring?
Bash version - GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Thanks.
Seems there are only two ways to deal with the problem
Option 1 - one parameter as a string, containing '&' signs
One needs to ensure that the supplied parameters string is enclosed in single or double-quotes.
Option 2 - get rid of '&' char entirely and interpret multiple supplied parameters
The script should be called with multiple key/value parameters separated by space:
./script.sh cluster=xyz tenant=abcd1234 key1=value1 key2=value2
I'm trying to write a bash script that will invoke ansible playbook with extra-vars. Some of this vars is an array of strings with spaces. So i am very confused in how to pass them correctly. I already tried many combinations of quotes and slashes. So I came here for help.
ansible-playbook -i inventory.yml playbook.yml --extra-vars \
'username="${login}" fullname="${username}" password="${password}" groups="['Users','Remote Desktop Users']"';
This is what you need to know about bash variables and quoting:
For the following examples, the variable ${text} is set to Hello:
Variables are expanded inside double quotes. e.g. "${text}" => Hello
Variables are not expanded inside single quotes. e.g. '${text}' => ${text}
Single quotes have no special meaning inside double quotes and vice-versa. e.g. "'${text}'" => 'Hello' and '"${text}"' => "${text}"
If you need to place a double-quote inside a double-quoted string, or a single quote inside a single-quoted string, then it must be escaped. e.g. "\"${text}\"" => "Hello" and '\'${text}\'' => '${text}'
With all that said, in your case, you want the variables to be expanded, so you should enclose the entire --extra-vars value in double quotes. According to the Ansible website, the value of each of these extra variables does not need to be quoted, unless it contains spaces. To be safe, you can quote the variables as you might not be able to control their values.
Try this. I have added extra line breaks to make the code easier to understand:
ansible-playbook -i inventory.yml playbook.yml --extra-vars \
"username='${login}' \
fullname='${username}' \
password='${password}' \
groups=['Users','Remote Desktop Users'] \
"
It seems that variable name 'groups' is reserved by ansible.
I changed name, and script starts working.
The answer of Andrew Vickers is also correct.
I have a Bash script which generates, stores and modifies values in an array. These values are later used as arguments for a command.
For a MCVE I thought of an arbitrary command bash -c 'echo 0="$0" ; echo 1="$1"' which explains my problem. I will call my command with two arguments -option1=withoutspace and -option2="with space". So it would look like this
> bash -c 'echo 0="$0" ; echo 1="$1"' -option1=withoutspace -option2="with space"
if the call to the command would be typed directly into the shell. It prints
0=-option1=withoutspace
1=-option2=with space
In my Bash script, the arguments are part of an array. However
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2="with space"')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
prints
0=-option1=withoutspace
1=-option2="with space"
which still shows the double quotes (because they are interpreted literally?). What works is
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
which prints again
0=-option1=withoutspace
1=-option2=with space
What do I have to change to make ARGUMENTS+=('-option2="with space"') work as well as ARGUMENTS+=('-option2=with space')?
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
Get rid of the single quotes. Write the options exactly as you would on the command line.
ARGUMENTS+=(-option1=withoutspace)
ARGUMENTS+=(-option2="with space")
Note that this is exactly equivalent to your second option:
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
-option2="with space" and '-option2=with space' both evaluate to the same string. They're two ways of writing the same thing.
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
It's the exact right thing to do. Arrays are perfect for this. Using a flat string would be a mistake.
I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.