Bashscript throws command error when populating variable [duplicate] - linux

This question already has answers here:
How do I set a variable to the output of a command in Bash?
(15 answers)
Bash variable from command with pipes, quotes, etc
(2 answers)
Variable variable assignment error -"command not found"
(1 answer)
Closed 1 year ago.
i have the following two lines in a batch script
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
$iperf_options=$iperf_options $proto
and
$streams = 2
$proto = -u
but when i run this i get the following error.
./bandwidth: line 116: -O: command not found
I am simply trying to wrote a string and then append it to a variable so why does it throw the error on the -O?
I have looked about the web but i jsut seem to find stuff about spaces around the "="
any help greatfully recived.
Thankyou
code block to show error
proto=-u
streams=2
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
$iperf_options=$iperf_options $proto
running this will give this out put
./test
./test: line 3: 2: command not found
./test: line 4: =: command not found

There are two main mistakes here, in a variety of combinations.
Use $ to get the value of a variable, never when setting the variable (or changing its properties):
$var=value # Bad
var=value # Good
var=$othervar # Also good
Spaces are critical delimiters in shell syntax; adding (or removing) them can change the meaning of a command in unexpected ways:
var = value # Runs `var` as a command, passing "=" and "value" as arguments
var=val1 val2 # Runs `val2` as a command, with var=val1 set in its environment
var="val1 val2" # Sets `var1` to `val1 val2`
So, in this command:
iperf_options=" -O 10 -V -i 10 --get-server-output -P " $streams
The space between iperf_options="..." and $streams means that it'll expand $streams and try to run it as a command (with iperf_options set in its environment). You want something like:
iperf_options=" -O 10 -V -i 10 --get-server-output -P $streams"
Here, since $streams is part of the double-quoted string, it'll be expanded (variable expand inside double-quotes, but not in single-quoted), and its value included in the value assigned to iperf_options.
There's actually a third mistake (or at least dubious scripting practice): building lists of options as simple string variables. This works in simple cases, but fails when things get complex. If you're using a shell that supports arrays (e.g. bash, ksh, zsh, etc, but not dash), it's better to use those instead, and store each option/argument as a separate array element, and then expand the array with "${arrayname[#]}" to get all of the elements out intact (yes, all those quotes, braces, brackets, etc are actually needed).
proto="-u" # If this'll always have exactly one value, plain string is ok
streams=2 # Same here
iperf_options=(-O 10 -V -i 10 --get-server-output -P "$streams")
iperf_options=("${iperf_options[#]}" "$proto")
# ...
iperf "${iperf_options[#]}"
Finally, I recommend shellcheck.net to sanity-check your scripts for common mistakes. A warning, though: it won't catch all errors, since it doesn't know your intent. For instance, if it sees var=val1 val2 it'll assume you meant to run val2 as a command and won't flag it as a mistake.

Related

How do I pass ">>" or "<<" to my script without the terminal trying to interpret it as me either appending to something or getting stdin?

My python script can take a series of bitwise operators as one of its arguments. They all work fine except for "=<<" which is roll left, and "=>>" which is roll right. I run my script like ./script.py -b +4,-4,=>>10,=<<1, where anything after -b can be any combination of similar operations. As soon as the terminal sees "<<" though, it just drops the cursor to a new line after the command and asks for more input instead of running the script. When it sees ">>", my script doesn't process the arguments correctly. I know it's because bash uses these characters for a specific purpose, but I'd like to get around it while still using "=>>" and "=<<" in my arguments for my script. Is there any way to do it without enclosing the argument in quotation marks?
Thank you for your help.
You should enclose the parameters that contain special symbols into single quotation marks (here, echo represents your script):
> echo '+4,-4,=>>10,=<<1'
+4,-4,=>>10,=<<1
Alternatively, save the parameters to a file (say, params.txt) and read them from the file onto the command line using the backticks:
> echo `cat params.txt`
+4,-4,=>>10,=<<1
Lastly, you can escape some offending symbols:
> echo +4,-4,=\>\>10,=\<\<1
+4,-4,=>>10,=<<1

Python3 run a complex shell command with multiple intended quotations

When i run from bash:
# su -c "psql -d mapping -c \"INSERT INTO mapping (ip,username) VALUES('1.2.3.4','test');\"" postgres
It works fine, but when i try from python:
subprocess.run("su -c \"psql -d mapping -c \"INSERT INTO mapping (ip,username) VALUES('1.2.3.4','test');\"\" postgres")
It fails, i have tried different quotation marks and all failing. Could you please help?
There are two solutions, depending on whether or not you use the shell from Python. The trivial but inefficient solution is to pass the string with shell=True and basically just add Python quoting around it.
subprocess.run(r'''su -c "psql -d mapping -c \"INSERT INTO mapping (ip,username) VALUES('1.2.3.4','test');\"" postgres''', shell=True,
# For good measure, you should check its status
check=True)
More efficiently and perhaps in fact more readably, you can remove the shell from the equation, and split the command into strings yourself.
subprocess.run([
'su', '-c',
# The argument to "su -c" should be one string
# Because we escaped the shell, the double quotes don't need escaping
'''psql -d mapping -c "INSERT INTO mapping (ip,username) VALUES('1.2.3.4','test');"''',
'postgres'],
check=True)
Notice how with shell=True the first argument is a string which gets passed to the shell, whereas without it, you pass a list of tokens directly to the OS-level exec() or (somewhat less straightforwardly on Windows) CreateProcess(). Notice also how in the first instance I used an r'...' string to avoid having Python meddle with the backslashes in the string.

How to store command arguments which contain double quotes in an array?

I have a Bash script which generates, stores and modifies values in an array. These values are later used as arguments for a command.
For a MCVE I thought of an arbitrary command bash -c 'echo 0="$0" ; echo 1="$1"' which explains my problem. I will call my command with two arguments -option1=withoutspace and -option2="with space". So it would look like this
> bash -c 'echo 0="$0" ; echo 1="$1"' -option1=withoutspace -option2="with space"
if the call to the command would be typed directly into the shell. It prints
0=-option1=withoutspace
1=-option2=with space
In my Bash script, the arguments are part of an array. However
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2="with space"')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
prints
0=-option1=withoutspace
1=-option2="with space"
which still shows the double quotes (because they are interpreted literally?). What works is
#!/bin/bash
ARGUMENTS=()
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
bash -c 'echo 0="$0" ; echo 1="$1"' "${ARGUMENTS[#]}"
which prints again
0=-option1=withoutspace
1=-option2=with space
What do I have to change to make ARGUMENTS+=('-option2="with space"') work as well as ARGUMENTS+=('-option2=with space')?
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
Get rid of the single quotes. Write the options exactly as you would on the command line.
ARGUMENTS+=(-option1=withoutspace)
ARGUMENTS+=(-option2="with space")
Note that this is exactly equivalent to your second option:
ARGUMENTS+=('-option1=withoutspace')
ARGUMENTS+=('-option2=with space')
-option2="with space" and '-option2=with space' both evaluate to the same string. They're two ways of writing the same thing.
(Maybe it's even entirely wrong to store arguments for a command in an array? I'm open for suggestions.)
It's the exact right thing to do. Arrays are perfect for this. Using a flat string would be a mistake.

How to get the complete calling command of a BASH script from inside the script (not just the arguments)

I have a BASH script that has a long set of arguments and two ways of calling it:
my_script --option1 value --option2 value ... etc
or
my_script val1 val2 val3 ..... valn
This script in turn compiles and runs a large FORTRAN code suite that eventually produces a netcdf file as output. I already have all the metadata in the netcdf output global attributes, but it would be really nice to also include the full run command one used to create that experiment. Thus another user who receives the netcdf file could simply reenter the run command to rerun the experiment, without having to piece together all the options.
So that is a long way of saying, in my BASH script, how do I get the last command entered from the parent shell and put it in a variable? i.e. the script is asking "how was I called?"
I could try to piece it together from the option list, but the very long option list and two interface methods would make this long and arduous, and I am sure there is a simple way.
I found this helpful page:
BASH: echoing the last command run
but this only seems to work to get the last command executed within the script itself. The asker also refers to use of history, but the answers seem to imply that the history will only contain the command after the programme has completed.
Many thanks if any of you have any idea.
You can try the following:
myInvocation="$(printf %q "$BASH_SOURCE")$((($#)) && printf ' %q' "$#")"
$BASH_SOURCE refers to the running script (as invoked), and $# is the array of arguments; (($#)) && ensures that the following printf command is only executed if at least 1 argument was passed; printf %q is explained below.
While this won't always be a verbatim copy of your command line, it'll be equivalent - the string you get is reusable as a shell command.
chepner points out in a comment that this approach will only capture what the original arguments were ultimately expanded to:
For instance, if the original command was my_script $USER "$(date +%s)", $myInvocation will not reflect these arguments as-is, but will rather contain what the shell expanded them to; e.g., my_script jdoe 1460644812
chepner also points that out that getting the actual raw command line as received by the parent process will be (next to) impossible. Do tell me if you know of a way.
However, if you're prepared to ask users to do extra work when invoking your script or you can get them to invoke your script through an alias you define - which is obviously tricky - there is a solution; see bottom.
Note that use of printf %q is crucial to preserving the boundaries between arguments - if your original arguments had embedded spaces, something like $0 $* would result in a different command.
printf %q also protects against other shell metacharacters (e.g., |) embedded in arguments.
printf %q quotes the given argument for reuse as a single argument in a shell command, applying the necessary quoting; e.g.:
$ printf %q 'a |b'
a\ \|b
a\ \|b is equivalent to single-quoted string 'a |b' from the shell's perspective, but this example shows how the resulting representation is not necessarily the same as the input representation.
Incidentally, ksh and zsh also support printf %q, and ksh actually outputs 'a |b' in this case.
If you're prepared to modify how your script is invoked, you can pass $BASH_COMMANDas an extra argument: $BASH_COMMAND contains the raw[1]
command line of the currently executing command.
For simplicity of processing inside the script, pass it as the first argument (note that the double quotes are required to preserve the value as a single argument):
my_script "$BASH_COMMAND" --option1 value --option2
Inside your script:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
myInvocation=$1 # Save the command line in a variable...
shift # ... and remove it from "$#".
# Now process "$#", as you normally would.
Unfortunately, there are only two options when it comes to ensuring that your script is invoked this way, and they're both suboptimal:
The end user has to invoke the script this way - which is obviously tricky and fragile (you could however, check in your script whether the first argument contains the script name and error out, if not).
Alternatively, provide an alias that wraps the passing of $BASH_COMMAND as follows:
alias my_script='/path/to/my_script "$BASH_COMMAND"'
The tricky part is that this alias must be defined in all end users' shell initialization files to ensure that it's available.
Also, inside your script, you'd have to do extra work to re-transform the alias-expanded version of the command line into its aliased form:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
# Here we also re-transform the alias-expanded command line to
# its original aliased form, by replacing everything up to and including
# "$BASH_COMMMAND" with the alias name.
myInvocation=$(sed 's/^.* "\$BASH_COMMAND"/my_script/' <<<"$1")
shift # Remove the first argument from "$#".
# Now process "$#", as you normally would.
Sadly, wrapping the invocation via a script or function is not an option, because the $BASH_COMMAND truly only ever reports the current command's command line, which in the case of a script or function wrapper would be the line inside that wrapper.
[1] The only thing that gets expanded are aliases, so if you invoked your script via an alias, you'll still see the underlying script in $BASH_COMMAND, but that's generally desirable, given that aliases are user-specific.
All other arguments and even input/output redirections, including process substitutiions <(...) are reflected as-is.
"$0" contains the script's name, "$#" contains the parameters.
Do you mean something like echo $0 $*?

Multiword string as a curl option using Bash

I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.

Resources