I am trying to run an SSH command that will invoke a script on a remote machine that writes some Lua code to a file.
I have this script command that executes under bash:
ssh bob writelua.sh '{version=1,{["foo"]=17}}'
And writelua.sh looks like this:
echo "return $1" > bar.lua
The end result, however, is that bar.lua has the content:
return version=1
I had thought that single quotes prevented all interpretation. How can I edit the scripts and escaping to pass the raw Lua code through unharmed?
The single quotes prevent interpretation on the local host. The remote host sees the command line
writelua.sh {version=1,{["foo"]=17}}
which is subject to brace expansion. You need a second set of quotes so that the first set of single quotes is passed through to the remote host.
ssh bob writelua.sh "'{version=1,{[\"foo\"]=17}}'"
As you can see, the quotes start to get unwieldy. A better solution is to simply copy a script containing
writelua.sh '{version=1,{["foo"]=17}}'
to the remote host and execute that remotely.
An example using the $'...' quotes:
ssh bob writelua.sh $'{version=1,{[\'foo\']=17}}'
Use heredoc and avoid all the excessive quoting:
ssh -T bob << \EOF
writelua.sh '{version=1,{["foo"]=17}}'
EOF
This will send raw script to remote host and it will get interpreted on the remote host itself.
When it gets too complex, particularly with lots of escaping, I prefer generating the command on a temporary script and execute it locally or remotely via SSH as required.
But there's an alternative: using echo to store the command in a variable and taking advantage of three things:
Single quotes don't do variable expansion and allow double quotes, so you can include something like "$myvar" without escaping $ or "
Double quotes allow variable expansion and single quotes, which means you can include something like animals='all'; echo love $animals to have $animals replaced by its value, and without escaping the '
Strings of both types, i.e. enclosed by single quotes or double quotes, can be concatenated simply by putting them together.
As an example, if I want something like this executed on a remote machine:
source /my-env.sh; perl -MMYLIB::DB -e 'my $t=db_list("name", 1553786458); print "#$t"'
But instead of 1553786458 I want to pass the value from a local variable:
now=`date +%s`
We could have this:
get_list=`echo 'source /my-env.sh; perl -MMYLIB::DB -e' "'my " '$t=db_list("name", ' "$now" '); print "#$t"' "'"`
You can see that single and double quotes are alternated, so we din't have to do any escaping! They don't need to be separated by spaces, but it improves readability and won't affect the result in this case.
And now we can execute:
ssh user#host $get_list
There's still no guarantee that this approach will always work, so once you've built your command, the safest bet would be to copy it over in a file.
If you can use Perl...
use Net::OpenSSH;
my $ssh = Net::OpenSSH->new("bob");
$ssh->system('writelua.sh', '{version=1,{["foo"]=17}}')
or die $ssh->error;
Net::OpenSSH takes care of quoting everything for you.
Related
This question already has answers here:
How can I store a command in a variable in a shell script?
(12 answers)
Closed 4 years ago.
These work as advertised:
grep -ir 'hello world' .
grep -ir hello\ world .
These don't:
argumentString1="-ir 'hello world'"
argumentString2="-ir hello\\ world"
grep $argumentString1 .
grep $argumentString2 .
Despite 'hello world' being enclosed by quotes in the second example, grep interprets 'hello (and hello\) as one argument and world' (and world) as another, which means that, in this case, 'hello will be the search pattern and world' will be the search path.
Again, this only happens when the arguments are expanded from the argumentString variables. grep properly interprets 'hello world' (and hello\ world) as a single argument in the first example.
Can anyone explain why this is? Is there a proper way to expand a string variable that will preserve the syntax of each character such that it is correctly interpreted by shell commands?
Why
When the string is expanded, it is split into words, but it is not re-evaluated to find special characters such as quotes or dollar signs or ... This is the way the shell has 'always' behaved, since the Bourne shell back in 1978 or thereabouts.
Fix
In bash, use an array to hold the arguments:
argumentArray=(-ir 'hello world')
grep "${argumentArray[#]}" .
Or, if brave/foolhardy, use eval:
argumentString="-ir 'hello world'"
eval "grep $argumentString ."
On the other hand, discretion is often the better part of valour, and working with eval is a place where discretion is better than bravery. If you are not completely in control of the string that is eval'd (if there's any user input in the command string that has not been rigorously validated), then you are opening yourself to potentially serious problems.
Note that the sequence of expansions for Bash is described in Shell Expansions in the GNU Bash manual. Note in particular sections 3.5.3 Shell Parameter Expansion, 3.5.7 Word Splitting, and 3.5.9 Quote Removal.
When you put quote characters into variables, they just become plain literals (see http://mywiki.wooledge.org/BashFAQ/050; thanks #tripleee for pointing out this link)
Instead, try using an array to pass your arguments:
argumentString=(-ir 'hello world')
grep "${argumentString[#]}" .
In looking at this and related questions, I'm surprised that no one brought up using an explicit subshell. For bash, and other modern shells, you can execute a command line explicitly. In bash, it requires the -c option.
argumentString="-ir 'hello world'"
bash -c "grep $argumentString ."
Works exactly as original questioner desired. There are two restrictions to this technique:
You can only use single quotes within the command or argument strings.
Only exported environment variables will be available to the command
Also, this technique handles redirection and piping, and other shellisms work as well. You also can use bash internal commands as well as any other command that works at the command line, because you are essentially asking a subshell bash to interpret it directly as a command line. Here's a more complex example, a somewhat gratuitously complex ls -l variant.
cmd="prefix=`pwd` && ls | xargs -n 1 echo \'In $prefix:\'"
bash -c "$cmd"
I have built command processors both this way and with parameter arrays. Generally, this way is much easier to write and debug, and it's trivial to echo the command you are executing. OTOH, param arrays work nicely when you really do have abstract arrays of parameters, as opposed to just wanting a simple command variant.
I'm having trouble sending multiple variables to a remote bash script without gobbling occurring.
For the sake of this question the variable $timestamp contains 12-12-15 19:45:21
ssh user#serverip "/usr/path/to/script.sh http://www.web.com/$1 http://web.com/$2 $timestamp";
I am sending 3 variables to script.sh
Two URLs with an amended file name in the form of a variable on the end and then my $timestamp variable
But on myscript.sh, when I try to insert $timestamp into a mysql database it only see's the first part of the date before the white space :
12-12-15
So my quotes around the command aren't preventing gobbling. Do I need to quote each variable separately?
ssh user#serverip "/usr/path/to/script.sh http://www.web.com/$1 http://web.com/$2 $timestamp";
This is equivalent to this locally calling
/usr/path/to/script.sh http://www.web.com/$1 http://web.com/$2 $timestamp
Try to quote each individual argument passed
ssh user#serverip "/usr/path/to/script.sh 'http://www.web.com/$1' 'http://web.com/$2' '$timestamp'";
You can also print each argument in the script to see what's being passed... e.g. echo $1, etc.
You can try something like
ssh localhost "printf \"%s %s %s\n\" a b \"last parameter\""
You need to escape the values for the remote host. The correct way of doing this is with printf %q:
ssh user#serverip "/usr/path/to/script.sh \
$(printf "%q " "http://www.web.com/$1" "http://web.com/$2" "$timestamp")"
This works for all variable values. Wrapping them in single quotes would instead result in syntax error and command injection when the variables themselves contain single quotes.
I want to get some data from a HTTP server. What it sends me depends on what I put in a POST request.
What I put in the INPUT_TEXT field is a sequence of words. When I run the following command, I get good looking output.
$ curl http://localhost:59125/process -d INPUT_TEXT="here are some words"
I want a bash script to take some string as a command line argument, and pass it appropriately to curl. The first thing I tried was to put the following in a script:
sentence=$1
command="curl http://localhost:59125/process -d INPUT_TEXT=\"${sentence}\""
$command
I then run the script like so:
$ ./script "here are some words"
But then I get a curl Couldn't resolve host error for each of "are", "some", and "words". It would seem that "here" got correctly treated as the INPUT_TEXT, but the rest of the words were then considered to be hosts, and not part of the option.
So I tried:
command=("curl" "http://localhost:59125/process" "-d" "INPUT_TEXT='$sentence'")
${command[#]}
I got the same output as the first script. I finally got what I wanted with:
result=$(curl http://localhost:59125/process -d INPUT_TEXT="${sentence}")
echo $result
I'm still unsure as to what the distinction is. In the first two cases, when I echoed out the contents of command, I get exactly what I input from the interactive Bash prompt, which had worked fine. What caused the difference?
The following will work:
command=("curl" "http://localhost:59125/process"
"-d" "INPUT_TEXT=$sentence")
"${command[#]}"
That has two changes from yours:
I removed the incorrect quotes around $sentence since you don't want to send quotes to the server (as far as I can see).
I put double-quotes around the use of "${command[#]}". Without the double quotes, the array's elements are concatenated with spaces between them and then the result is word-split. With double quotes, the individual array elements are used as individual words.
The second point is well-explained in the bash FAQ and a bunch of SO answers dealing with quotes.
The important thing to understand is that quotes only quote when a command is parsed. A quote which is a character in a variable is just a character; it is not reinterpreted when the value of the variable expanded. Whitespace in the variable is used for word-splitting if the variable expansion is unquoted; the fact that the whitespace was quoted in the the command which defined the variable is completely irrelevant. In this sense, bash is just the same as any other programming language.
I am trying to use a FOR loop to iterate over IP addresses (in a bash array), logs in, runs a script and then exits. The array is called ${INSTANCE_IPS[#]}. The following code doesn't work though, as expect doesn't seem to be able to accept the variable $instance.
for instance in ${INSTANCE_IPS[#]}
do
echo $instance
/usr/bin/expect -c '
spawn ssh root#$instance;
expect "?assword: ";
send "<password>\r";
expect "# ";
send ". /usr/local/bin/bootstrap.sh\r";
expect "# ";
send "exit\r" '
done
However, expect complains with:
can't read "instance": no such variable
while executing
"spawn ssh root#$instance"
There is another question on stackoverflow located here, that uses environmental variables to achieve this, however it doesn't allow me to iterate through different IP addresses like I can in an array.
Any help is appreciated.
Cheers
The problem is with quoting. Single quotes surrounding the whole block don't let Bash expand variables ($instance).
You need to switch to double quotes. But then, double quotes inside double quotes are not allowed (unless you escape them), so we are better off using single quotes with expect strings.
Try instead:
for instance in ${INSTANCE_IPS[#]}
do
echo $instance
/usr/bin/expect -c "
spawn ssh root#$instance;
expect '?assword: ';
send '<password>\r';
expect '# ';
send '. /usr/local/bin/bootstrap.sh\r';
expect '# ';
send 'exit\r' "
done
for instance in ${INSTANCE_IPS[&]} ; do
echo $instance
/usr/bin/expect -c '
spawn ssh root#'$instance' "/usr/local/bin/bootstrap.sh"
expect "password:"
send "<password>\r"
expect eof'
done
From the ssh man page:
If command is specified, it is executed on the remote host instead of a login shell.
Specifying a command means expect doesn't have to wait for # to execute your program, then wait for another # just to send the command exit. Instead, when you specify a command to ssh, it executes that command; it exits when done; and then ssh automatically closes the connection.
Alternately, put the value in the environment and expect can find it there
for instance in ${INSTANCE_IPS[&]} ; do
echo $instance
the_host=$instance /usr/bin/expect -c '
spawn ssh root#$env(the_host) ...
Old thread, and one of many, but I've been working on expect for several days. For anyone who comes across this, I belive I've found a doable solution to the problem of passing bash variables inside an expect -c script:
#!/usr/bin/env bash
password="TopSecret"
read -d '' exp << EOF
set user "John Doe"
puts "\$user"
puts "$password"
EOF
expect -c "$exp"
Please note that escaping quotations are typically a cited issue (as #Roberto Reale stated above), which I've solved using a heredoc EOF method, before passing the bash-variable-evaluated string to expect -c. In contrast to escaping quotes, all native expect variables will need to be escaped with \$ (I'm not here to solve all first-world problems--my afternoon schedule is slightly crammed), but this should greatly simplify the problem with little effort. Let me know if you find any issues with this proof of concept.
tl;tr: Been creating an [expect] daemon script with user authentication and just figured this out after I spent a whole day creating separated bash/expect scripts, encrypting my prompted password (via bash) with a different /dev/random salt each iteration, saving the encrypted password to a temp file and passing the salt to the expect script (highly discouraging anyone from easily discovering the password via ps, but not preventative since the expect script could be replaced). Now I should be able to effectively keep it in memory instead.
I am running a bash script that calls mysql. The password ist not correctly transmitted, I guess I have to escape some special chars, like the hash or the dollar sign?
#!/bin/bash
USER=myuser
PASS="#mypass$"
# ... call mysql
Using "..." is already the correct thing to do, but the $ needs to be escaped (\$) if it isn't followed by an "invalid" character. However you also need to make sure to always have the variable in quotation marks as well, as in:
somecommand -p "$PASS"
Try to use "\" before the character that you are trying to escape.
#!/bin/bash
USER=myuser
PASS="#mypass\$"
# ... call mysql