remote ssh command: first echo output is lost - linux

I'm trying to run several commands on a remote box via ssh 1-liner call by specifying them as semicolon-separated string passed to "bash -c". It works for some cases, but does not for others. Check this out:
# Note: the "echo 1" output is lost:
bash-3.2$ ssh sandbox bash -c "echo 1; echo 2; echo 3"
2
3
# Note: first echo is ignored again
bash-3.2$ ssh sandbox bash -c "echo 0; echo 1; echo 2; echo 3"
1
2
3
# But when we run other commands (for example "date") then nothing is lost
bash-3.2$ ssh sandbox bash -c "date; date;"
Wed Nov 7 20:27:55 UTC 3018
Wed Nov 7 20:27:55 UTC 3018
What am I missing?
Remote OS: Ubuntu 16.04.5 LTS
Remote ssh: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
Local OS: macOS High Sierra Versoin 10.13.3
Local ssh: OpenSSH_7.6p1, LibreSSL 2.6.2
Update:
The above example is heavily simplified picture of what I'm trying to do.
The practical application is actually to generate few files on remote box by echo'ing into remote filesystem:
#!/bin/bash
A=a
B=b
C=c
ssh -i ~/.ssh/${REMOTE_FQDN}.pem ${REMOTE_FQDN} sudo bash -c \
"echo $A > /tmp/_a; echo $B > /tmp/_b; echo $C > /tmp/_c;"
After I run the above script and go to remote box to check results I see the following:
root#sandbox:/tmp# for i in `find ./ -name '_*'|sort`; do echo "----- ${i} ----"; cat $i; done
----- ./_a ----
----- ./_b ----
b
----- ./_c ----
c
As you can see the 1st "echo" command generated blank file!

To be clear, there's 3 shells at work here - the one that interprets ssh, your local shell that is; the one that ssh will be automatically running for you, and the bash you're invoking explicitly.
The reason the 1 is "disappearing" is that the shell that interprets the ssh command "eats" the quotes around the -c arguments, and then the shell on the other side of ssh splits the arguments at whitespace. So it ends up looking like bash -c echo 1 ; echo 2; echo 3. In turn, -c just gets echo, which echos an empty line; 1 becomes the value of that shell's $1, which isn't used. Then the inner bash returns, and the direct ssh shell runs the echo 2; echo 3 normally.
Consider this:
$ ssh xxx bash -c "'echo 1'; echo 2; echo 3"
1
2
3
where echo 1 is protected within the ssh arguments, so the 2nd level ssh shell is passed bash -c 'echo 1'; echo 2; echo 3. The innermost 3rd level shell echos 1, and then the 2nd level ssh shell echos 2 and 3.
Here is yet another interesting permutation:
$ ssh xxx bash -c "'echo 1; echo 2; echo 3'"
1
2
3
here, the inner shell gets all the echos as they're kept grouped within the first shell by " and within the second shell by '.
In general, shell scripts to pass arguments to shell scripts that run shell scripts can be pretty difficult to build. I'd recommend you change your technique a bit to save yourself a lot of effort. Instead of passing the shell commands as command line parameters to the ssh argument, instead provide it through the standard input to the shell. Consider using a pipeline like this, which avoids recursive shell interpretation:
$ echo "echo 1; echo 2; echo 3" | ssh -T xxx
1
2
3
( Here, the -T is just to supress ssh complaining of lack of pseudoterminal).

All the arguments to ssh are combined into a single whitespace-separate string passed to sh -c on the remote end. This means that
ssh sandbox bash -c "echo 1; echo 2; echo 3"
results in the execution of
sh -c 'bash -c echo 1; echo 2; echo 3'
Note the loss of quotes; ssh got the three arguments bash, -c, and echo 1; echo 2; echo 3 after quote removal. On the remote end, bash -c echo 1 just executes echo, with $0 in the shell set to 1.
The command
ssh sandbox bash -c "date; date;"
is treated the same way, but now the first command contains no whitespace. The result on the remote end is
sh -c 'bash -c date; date;'
which means first a new instance of bash runs the date command, followed by the date command being executed directly by sh.
In general, it's a bad idea to use ssh's implicit concatenation. Always pass the command you want executed as a properly escaped single argument:
ssh sandbox 'bash -c "echo 1; echo 2; echo 3"'

Related

Bash shell: command not found

I am using the shell module to execute the following command
tasks:
- name: Command
shell: "sshpass -p 123 ssh -o 'StrictHostKeyChecking no' root#10.67.13.50 shell << EOF \n whoami\nEOF | cat"
I am getting the following error
"stderr_lines": [
"/bin/sh: line 2: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')",
"Warning: Permanently added '10.67.13.50' (ECDSA) to the list of known hosts.",
"bash: shell: command not found"
]
What is wrong with my command?
tl;dr you can either a) replace shell with sh or bash, or b) replace shell with whoami and drop the heredoc.
Let's decompose the shell command:
sshpass -p 123 ssh -o 'StrictHostKeyChecking no' root#10.67.13.50 shell << EOF \n whoami\nEOF | cat
There are several processes happening here.
A sh pipeline with two subprocesses:
sshpass which then runs as a subprocess…
ssh which connects to 10.67.13.50 and runs…
shell with \n whoami\n as its standard input
… and cat, which takes the output from the sshpass process hierarchy
There are a couple potential bugs:
You can safely remove cat from the pipeline.
As #KamilCuk mentioned, cat reads from its input and writes it out. It isn't doing anything here; it's neither useful nor harmless.
shell is not a command on the remote server (10.67.13.50). If you want to run a shell, typically sh or bash is used.
Moreover, you can replace the entire shell … EOF sequence with whoami.
The << EOF \n whoami \nEOF is a heredoc to tell the shell on the remote server what commands to execute. However, there is only one command executed.
In summary, the shell: line could be rewritten as:
sshpass -p 123 ssh -o 'StrictHostKeyChecking no' root#10.67.13.50 whoami
… an odd command, since we know the remote user root.

Unable to use local and remote variables within a heredoc or command over SSH

Below is an example of a ssh script using a heredoc (the actual script is more complex). Is it possible to use both local and remote variables within an SSH heredoc or command?
FILE_NAME is set on the local server to be used on the remote server. REMOTE_PID is set when running on the remote server to be used on local server. FILE_NAME is recognised in script. REMOTE_PID is not set.
If EOF is changed to 'EOF', then REMOTE_PID is set and `FILE_NAME is not. I don't understand why this is?
Is there a way in which both REMOTE_PID and FILE_NAME can be recognised?
Version 2 of bash being used. The default remote login is cshell, local script is to be bash.
FILE_NAME=/example/pdi.dat
ssh user#host bash << EOF
# run script with output...
REMOTE_PID=$(cat $FILE_NAME)
echo $REMOTE_PID
EOF
echo $REMOTE_PID
You need to escape the $ sign if you don't want the variable to be expanded:
$ x=abc
$ bash <<EOF
> x=def
> echo $x # This expands x before sending it to bash. Bash will see only "echo abc"
> echo \$x # This lets bash perform the expansion. Bash will see "echo $x"
> EOF
abc
def
So in your case:
ssh user#host bash << EOF
# run script with output...
REMOTE_PID=$(cat $FILE_NAME)
echo \$REMOTE_PID
EOF
Or alternatively you can just use a herestring with single quotes:
$ x=abc
$ bash <<< '
> x=def
> echo $x # This will not expand, because we are inside single quotes
> '
def
remote_user_name=user
instance_ip=127.0.0.1
external=$(ls /home/)
ssh -T -i ${private_key} -l ${remote_user_name} ${instance_ip} << END
internal=\$(ls /home/)
echo "\${internal}"
echo "${external}"
END

Bash script runs one command before previous. I want them one after the other

So part of my script is as follows:
ssh user#$remoteServer "
cd ~/a/b/c/;
echo -e 'blah blah'
sleep 1 # Added this just to make sure it waits.
foo=`grep something xyz.log |sed 's/something//g' |sed 's/something-else//g'`
echo $foo > ~/xyz.list
exit "
In my output I see:
grep: xyz.log: No such file or directory
blah blah
Whereas when I ssh to the server, xyz.log does exist within ~/a/b/c/
Why is the grep statement getting executed before the echo statement?
Can someone please help?
The problem here is that your command in backticks is being run locally, not on the remote end of the SSH connection. Thus, it runs before you've even connected to the remote system at all! (This is true for all expansions that run in double-quotes, so the $foo in echo $foo as well).
Use a quoted heredoc to protect your code against local evaluation:
ssh user#$remoteServer bash -s <<'EOF'
cd ~/a/b/c/;
echo -e 'blah blah'
sleep 1 # Added this just to make sure it waits.
foo=`grep something xyz.log |sed 's/something//g' |sed 's/something-else//g'`
echo $foo > ~/xyz.list
exit
EOF
If you want to pass through a variable from the local side, the easy way is with positional parameters:
printf -v varsStr '%q ' "$varOne" "$varTwo"
ssh "user#$remoteServer" "bash -s $varsStr" <<'EOF'
varOne=$1; varTwo=$2 # set as remote variables
echo "Remote value of varOne is $varOne"
echo "Remote value of varTwo is $varTwo"
EOF
[command server] ------> [remote server]
The better way is to create shell script in the "remote server" , and run the command in the "command server" such as :
ssh ${remoteserver} "/bin/bash /foo/foo.sh"
It will solve many problem , the aim is to make things simple but not complex .

Shell scripting shell inside shell

I would like to connect to different shells (csh, ksh etc.,) and execute command inside each switched shell.
Following is the sample program which reflects my intention:
#!/bin/bash
echo $SHELL
csh
echo $SHELL
exit
ksh
echo $SHELL
exit
Since, i am not well versed with Shell scripting need a pointer on how to achieve this. Any help would be much appreciated.
If you want to execute only one single command, you can use the -c option
csh -c 'echo $SHELL'
ksh -c 'echo $SHELL'
If you want to execute several commands, or even a whole script in a child-shell, you can use the here-document feature of bash and use the -s (read commands from stdin) on the child shells:
#!/bin/bash
echo "this is bash"
csh -s <<- EOF
echo "here go the commands for csh"
echo "and another one..."
EOF
echo "this is bash again"
ksh -s <<- EOF
echo "and now, we're in ksh"
EOF
Note that you can't easily check the shell you are in by echo $SHELL, because the parent shell expands this variable to the text /././bash. If you want to be sure that the child shell works, you should check if a shell-specific syntax is working or not.
It is possible to use the command line options provided by each shell to run a snippet of code.
For example, for bash use the -c option:
bash -c $code
bash -c 'echo hello'
zsh and fish also use the -c option.
Other shells will state the options they use in their man pages.
You need to use the -c command line option if you want to pass commands on bash startup:
#!/bin/bash
# We are in bash already ...
echo $SHELL
csh -c 'echo $SHELL'
ksh -c 'echo $SHELL'
You can pass arbitrary complex scripts to a shell, using the -c option, as in
sh -c 'echo This is the Bourne shell.'
You will save you a lot of headaches related to quotes and variable expansion if you wrap the call in a function reading the script on stdin as:
execute_with_ksh()
{
local script
script=$(cat)
ksh -c "${script}"
}
prepare_complicated_script()
{
# Write shell script on stdout,
# for instance by cat-ting a here-document.
cat <<'EOF'
echo ${SHELL}
EOF
}
prepare_complicated_script | execute_with_ksh
The advantage of this method is that it easy to insert a tee in the pipe or to break the pipe to control the script being passed to the shell.
If you want to execute the script on a remote host through ssh you should consider encode your script in base 64 to transmit it safely to the remote shell.

Bash: how to run a script remotely

I have a script (say run.py) and I want to scp that to a remote machine (say 10.1.100.100), cd into a directory in that remote machine, and execute run.py in that directory.
How do I wrap the above procedure in one single bash script? I don't know how to let bash execute commands remotely in another machine.
Hopefully I can see that stdout of run.py in my terminal. But if I can only redirect it, that's fine as well.
chmod +x ./run.py
scp -pq ./run.py 10.1.100.100:'/home/myremotedirectory/run.py'
ssh 10.1.100.100 'cd /somedirectory && /home/myremotedirectory/run.py'
See if that helps
How to run a local script over SSH
Synopsis:
Script execution over SSH without copying script file.
You need a simple SSH connexion and a local script.
Code:
#!/bin/sh
print_usage() {
echo -e "`basename $0` ssh_connexion local_script"
echo -e "Remote executes local_script on ssh server"
echo -e "For convinient use, use ssh public key for remote connexion"
exit 0
}
[ $# -eq "2" ] && [ $1 != "-h" ] && [ $1 != "--help" ] || print_usage
INTERPRETER=$(head -n 1 $2 | sed -e 's/#!//')
cat $2 | grep -v "#" | ssh -t $1 $INTERPRETER
Examples:
- ssh-remote-exec root#server1 myLocalScript.sh #for Bash
- ssh-remote-exec root#server1 myLocalScript.py #for Python
- ssh-remote-exec root#server1 myLocalScript.pl #for Perl
- ssh-remote-exec root#server1 myLocalScript.rb #for Ruby
Step by step explanations
This script performs this operations:
1° catches first line #! to get interpreter (i.e: Perl, Python, Ruby, Bash interpreter),
2° starts remote interpeter over SSH,
3° send all the script body over SSH.
Local Script:
Local script must start with #!/path/to/interpreter
- #!/bin/sh for Bash script
- #!/usr/bin/perl for Perl script
- #!/usr/bin/python for Python script
- #!/usr/bin/ruby for Ruby script
This script is not based on local script extension but on #! information.
You can do it like this:
ssh -l yourid 10.1.100.100 << DONE
cd /your/dir/
./run.py
DONE
Above has been edited, I don't remember what it was like originally, if I want to do it in one single connection, I will do it this way.
ssh -l yourid 10.1.100.100 python < <(
echo "import os"
echo "os.chdir('/yourdir')"
echo "print(os.getcwd())"
cat yourscript.py
)
Remember, that this is not a rule, that you HAVE TO cd to the requested directory.
Once you get access to the remote machine, just type a relative path to this file, without using cd:
/some_folder/./run.py

Resources