Execute Bash script remotely via cURL - linux

I have a simple Bash script that takes in inputs and prints a few lines out with that inputs
fortinetTest.sh
read -p "Enter SSC IP: $ip " ip && ip=${ip:-1.1.1.1}
printf "\n"
#check IP validation
if [[ $ip =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "SSC IP: $ip"
printf "\n"
else
echo "Enter a valid SSC IP address. Ex. 1.1.1.1"
exit
fi
I tried to upload them into my server, then try to run it via curl
I am not sure why the input prompt never kick in when I use cURL/wget.
Am I missing anything?

With the curl ... | bash form, bash's stdin is reading the script, so stdin is not available for the read command.
Try using a Process Substitution to invoke the remote script like a local file:
bash <( curl -s ... )

Your issue can be simply be reproduced by run the script like below
$ cat test.sh | bash
Enter a valid SSC IP address. Ex. 1.1.1.1
This is because the bash you launch with a pipe is not getting a TTY, when you do a read -p it is read from stdin which is content of the test.sh in this case. So the issue is not with curl. The issue is not reading from the tty
So the fix is to make sure you ready it from tty
read < /dev/tty -p "Enter SSC IP: $ip " ip && ip=${ip:-1.1.1.1}
printf "\n"
#check IP validation
if [[ $ip =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "SSC IP: $ip"
printf "\n"
else
echo "Enter a valid SSC IP address. Ex. 1.1.1.1"
exit
fi
Once you do that even curl will start working
vagrant#vagrant:/var/www/html$ curl -s localhost/test.sh | bash
Enter SSC IP: 2.2.2.2
SSC IP: 2.2.2.2

I personally prefer source <(curl -s localhost/test.sh) option. While it is similar to bash ..., the one significant difference is how processes handled.
bash will result in a new process being spun up, and that process will evoke commands from the script.
source on the other hand will use current process to evoke commands from the script.
In some cases that can play a key role. I admit that is not very often though.
To demonstrate do the following:
### Open Two Terminals
# In the first terminal run:
echo "sleep 5" > ./myTest.sh
bash ./myTest.sh
# Switch to the second terminal and run:
ps -efjh
## Repeat the same with _source_ command
# In the first terminal run:
source ./myTest.sh
# Switch to the second terminal and run:
ps -efjh
Results should look similar to this:
Before execution:
Running bash (main + two subprocesses):
Running source (main + one subprocess):
UPDATE:
Difference in use variable usage by bash and source:
source command will use your current environment. Meaning that upon execution all changes and variable declarations, made by the script, will be available in your prompt.
bash on the other hand will be running in as a different process; therefore, all variables will be discarded when process exits.
I think everyone will agree that there are benefits and drawbacks to each method. You just have to decide which one is better for your use case.
## Test for variables declared by the script:
echo "test_var3='Some Other Value'" > ./myTest3.sh
bash ./myTest3.sh
echo $test_var3
source ./myTest3.sh
echo $test_var3
## Test for usability of current environment variables:
test_var="Some Value" # Setting a variable
echo "echo $test_var" > myTest2.sh # Creating a test script
chmod +x ./myTest2.sh # Adding execute permission
## Executing:
. myTest2.sh
bash ./myTest2.sh
source ./myTest2.sh
./myTest2.sh
## All of the above results should print the variable.
I hope this helps.

Related

How to connect input/output to SSH session

What is a good way to be able to directly send to STDIN and receive from STDOUT of a process? I'm specifically interested in SSH, as I want to do the following:
[ssh into a remote server]
[run remote commands]
[run local commands]
[run remote commands]
etc...
For example, let's say I have a local script "localScript" that will output the next command I want to run remotely, depending on the output of "remoteScript". I could do something like:
output=$(ssh myServer "./remoteScript")
nextCommand=$(./localScript $output)
ssh myServer "$nextCommand"
But it would be nice to do this without closing/reopening the SSH connection at every step.
You can redirect SSH input and output to FIFO-s and then use these for two-way communication.
For example local.sh:
#!/bin/sh
SSH_SERVER="myServer"
# Redirect SSH input and output to temporary named pipes (FIFOs)
SSH_IN=$(mktemp -u)
SSH_OUT=$(mktemp -u)
mkfifo "$SSH_IN" "$SSH_OUT"
ssh "$SSH_SERVER" "./remote.sh" < "$SSH_IN" > "$SSH_OUT" &
# Open the FIFO-s and clean up the files
exec 3>"$SSH_IN"
exec 4<"$SSH_OUT"
rm -f "$SSH_IN" "$SSH_OUT"
# Read and write
counter=0
echo "PING${counter}" >&3
cat <&4 | while read line; do
echo "Remote responded: $line"
sleep 1
counter=$((counter+1))
echo "PING${counter}" >&3
done
And simple remote.sh:
#!/bin/sh
while read line; do
echo "$line PONG"
done
The method you are using works, but I don't think you can reuse the same connection everytime. You can, however, do this using screen, tmux or nohup, but that would greatly increase the complexity of your script because you will now have to emulate keypresses/shortcuts. I'm not even sure if you can if you do directly in bash. If you want to emulate keypresses, you will have to run the script in a new x-terminal and use xdotool to emulate the keypresses.
Another method is to delegate the whole script to the SSH server by just running the script on the remote server itself:
ssh root#MachineB 'bash -s' < local_script.sh

Launching a bash shell from a sudo-ed environment

Apologies for the confusing Question title. I am trying to launch an interactive bash shell from a shell script ( say shel2.sh) which has been launched by a parent script (shel1.sh) in a sudo-ed environment. ( I am creating a guided deployment
script for my software which needs to be installed as super-user , hence the sudo, but may need the user to access the shell. )
Here's shel1.sh
#!/bin/bash
set -x
sudo bash << EOF
echo $?
./shel2.sh
EOF
echo shel1 done
And here's shel2.sh
#!/bin/bash
set -x
bash --norc --verbose --noprofile -i
echo $?
echo done
I expected this to launch an interactive bash shell which waits for my input before returning to shel1.sh. This is what I see:
+ ./shel1.sh
+ sudo bash
0
+ bash --norc --verbose --noprofile -i
bash-4.3# exit
+ echo 0
0
+ echo done
done
+ echo shel1 done
shel1 done
The bash-4.3# calls an exit automatically and quits. Interestingly if I invoke the bash shell with -l (or --login) the automatic entry is logout !
Can someone explain what is happening here ?
When you use a here document, you are tying up the shell's -- and its spawned child processes' -- standard input to the here document input.
You can avoid using a here document in many situations. For example, replace the here document with a single-quoted string.
#!/bin/bash
set -x
sudo bash -c '
# Aside: How is this actually useful?
echo $?
# Spawned script inherits the stdin of "sudo bash"
./shel2.sh'
echo shel1 done
Without more details, it's hard to see where exactly you want to go with this, but most modern Linux platforms have package managers which allow all kinds of hooks for installation, so that you would typically not need to do this sort of thing. Have you looked into that?

Why this Debian-Linux autostart netcat script won't autostart?

I placed a link to my scripts in the rc.local to autostart it on linux debian boot. It starts and then stops at the while loop. It's a netcat script that listens permantently on port 4001.
echo "Start"
while read -r line
do
#some stuff to do
done < <(nc -l -p 4001)
When I start this script as root with command ./myscript it works 100% correctly. Need nc (netcat) root level access or something else?
EDIT:
rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
/etc/samba/SQLScripts
exit 0
rc.local starts my script "SQLScripts"
SQLScripts
#! /bin/sh
# The following part always gets executed.
echo "Starting SQL Scripts" >> /var/log/SQLScriptsStart
/etc/samba/PLCCheck >> /var/log/PLCCheck &
"SQLScripts" starts "PLCCheck" (for example only one)
PLCCheck
#!/bin/bash
echo "before SLEEP" >> /var/log/PLCCheck
sleep 5
echo "after SLEEP" >> /var/log/PLCCheck
echo "vor While" >> /var/log/PLCCheck
while read -r line
do
echo "in While" >> /var/log/PLCCheck
done < <(netcat -u -l -p 6001)
In an rc script you have root level access by default. What does "it stops at the while loop" mean? It quits after a while, or so? I guess you need to run your loop in the background in order to achieve functionality usual in autostart scripts:
echo "Starting"
( while read -r line
do
#some stuff to do
done << (nc -l -p 4001) ) &
echo "Started with pid $( jobs -p )"
I have tested yersterday approximatly the same things, and I have discover that you can bypass the system and execute your netcat script with the following crontask. :
(every minute, but you can ajust that as you want.)
* * * * * /home/kali/script-netcat.sh // working for me
#reboot /home/kali/script-netcat.sh // this is blocked by the system.
According to me, I think that by default debian (and maybe others linux distrib) block every script that try to execute a netcat command.

Read command in bash script not waiting for user input when piped to bash?

Here is what I'm entering in Terminal:
curl --silent https://raw.githubusercontent.com/githubUser/repoName/master/installer.sh | bash
The WordPress installing bash script contains a "read password" command that is supposed to wait for users to input their MySQL password. But, for some reason, that doesn't happen when I run it with the "curl githubURL | bash" command. When I download the script via wget and run it via "sh installer.sh", it works fine.
What could be the cause of this? Any help is appreciated!
If you want to run a script on a remote server without saving it locally, you can try this.
#!/bin/bash
RunThis=$(lynx -dump http://127.0.0.1/example.sh)
if [ $? = 0 ] ; then
bash -c "$RunThis"
else
echo "There was a problem downloading the script"
exit 1
fi
In order to test it, I wrote an example.sh:
#!/bin/bash
# File /var/www/example.sh
echo "Example read:"
read line
echo "You typed: $line"
When I run Script.sh, the output looks like this.
$ ./Script.sh
Example read:
Hello World!
You typed: Hello World!
Unless you absolutely trust the remote scripts, I would avoid doing this without examining it before executing.
It wouldn't stop for read:
As when you are piping in a way you are forking a child which has been given input from parent shell.
You cannot give the values back to parent(modify parent's env) from child.
and through out this process you are always in parent process.

Pseudo-terminal will not be allocated because stdin is not a terminal

I am trying to write a shell script that creates some directories on a remote server and then uses scp to copy files from my local machine onto the remote. Here's what I have so far:
ssh -t user#server<<EOT
DEP_ROOT='/home/matthewr/releases'
datestamp=$(date +%Y%m%d%H%M%S)
REL_DIR=$DEP_ROOT"/"$datestamp
if [ ! -d "$DEP_ROOT" ]; then
echo "creating the root directory"
mkdir $DEP_ROOT
fi
mkdir $REL_DIR
exit
EOT
scp ./dir1 user#server:$REL_DIR
scp ./dir2 user#server:$REL_DIR
Whenever I run it I get this message:
Pseudo-terminal will not be allocated because stdin is not a terminal.
And the script just hangs forever.
My public key is trusted on the server and I can run all the commands outside of the script just fine. Any ideas?
Try ssh -t -t(or ssh -tt for short) to force pseudo-tty allocation even if stdin isn't a terminal.
See also: Terminating SSH session executed by bash script
From ssh manpage:
-T Disable pseudo-tty allocation.
-t Force pseudo-tty allocation. This can be used to execute arbitrary
screen-based programs on a remote machine, which can be very useful,
e.g. when implementing menu services. Multiple -t options force tty
allocation, even if ssh has no local tty.
Also with option -T from manual
Disable pseudo-tty allocation
Per zanco's answer, you're not providing a remote command to ssh, given how the shell parses the command line. To solve this problem, change the syntax of your ssh command invocation so that the remote command is comprised of a syntactically correct, multi-line string.
There are a variety of syntaxes that can be used. For example, since commands can be piped into bash and sh, and probably other shells too, the simplest solution is to just combine ssh shell invocation with heredocs:
ssh user#server /bin/bash <<'EOT'
echo "These commands will be run on: $( uname -a )"
echo "They are executed by: $( whoami )"
EOT
Note that executing the above without /bin/bash will result in the warning Pseudo-terminal will not be allocated because stdin is not a terminal. Also note that EOT is surrounded by single-quotes, so that bash recognizes the heredoc as a nowdoc, turning off local variable interpolation so that the command text will be passed as-is to ssh.
If you are a fan of pipes, you can rewrite the above as follows:
cat <<'EOT' | ssh user#server /bin/bash
echo "These commands will be run on: $( uname -a )"
echo "They are executed by: $( whoami )"
EOT
The same caveat about /bin/bash applies to the above.
Another valid approach is to pass the multi-line remote command as a single string, using multiple layers of bash variable interpolation as follows:
ssh user#server "$( cat <<'EOT'
echo "These commands will be run on: $( uname -a )"
echo "They are executed by: $( whoami )"
EOT
)"
The solution above fixes this problem in the following manner:
ssh user#server is parsed by bash, and is interpreted to be the ssh command, followed by an argument user#server to be passed to the ssh command
" begins an interpolated string, which when completed, will comprise an argument to be passed to the ssh command, which in this case will be interpreted by ssh to be the remote command to execute as user#server
$( begins a command to be executed, with the output being captured by the surrounding interpolated string
cat is a command to output the contents of whatever file follows. The output of cat will be passed back into the capturing interpolated string
<< begins a bash heredoc
'EOT' specifies that the name of the heredoc is EOT. The single quotes ' surrounding EOT specifies that the heredoc should be parsed as a nowdoc, which is a special form of heredoc in which the contents do not get interpolated by bash, but rather passed on in literal format
Any content that is encountered between <<'EOT' and <newline>EOT<newline> will be appended to the nowdoc output
EOT terminates the nowdoc, resulting in a nowdoc temporary file being created and passed back to the calling cat command. cat outputs the nowdoc and passes the output back to the capturing interpolated string
) concludes the command to be executed
" concludes the capturing interpolated string. The contents of the interpolated string will be passed back to ssh as a single command line argument, which ssh will interpret as the remote command to execute as user#server
If you need to avoid using external tools like cat, and don't mind having two statements instead of one, use the read built-in with a heredoc to generate the SSH command:
IFS='' read -r -d '' SSH_COMMAND <<'EOT'
echo "These commands will be run on: $( uname -a )"
echo "They are executed by: $( whoami )"
EOT
ssh user#server "${SSH_COMMAND}"
I'm adding this answer because it solved a related problem that I was having with the same error message.
Problem: I had installed cygwin under Windows and was getting this error: Pseudo-terminal will not be allocated because stdin is not a terminal
Resolution: It turns out that I had not installed the openssh client program and utilities. Because of that cygwin was using the Windows implementation of ssh, not the cygwin version. The solution was to install the openssh cygwin package.
All relevant information is in the existing answers, but let me attempt a pragmatic summary:
tl;dr:
DO pass the commands to run using a command-line argument:
ssh jdoe#server '...'
'...' strings can span multiple lines, so you can keep your code readable even without the use of a here-document:
ssh jdoe#server ' ... '
Do NOT pass the commands via stdin, as is the case when you use a here-document:
ssh jdoe#server <<'EOF' # Do NOT do this ... EOF
Passing the commands as an argument works as-is, and:
the problem with the pseudo-terminal will not even arise.
you won't need an exit statement at the end of your commands, because the session will automatically exit after the commands have been processed.
In short: passing commands via stdin is a mechanism that is at odds with ssh's design and causes problems that must then be worked around.
Read on, if you want to know more.
Optional background information:
ssh's mechanism for accepting commands to execute on the target server is a command-line argument: the final operand (non-option argument) accepts a string containing one or more shell commands.
By default, these commands run unattended, in a non-interactive shell, without the use of a (pseudo) terminal (option -T is implied), and the session automatically ends when the last command finishes processing.
In the event that your commands require user interaction, such as responding to an interactive prompt, you can explicitly request the creation of a pty (pseudo-tty), a pseudo terminal, that enables interacting with the remote session, using the -t option; e.g.:
ssh -t jdoe#server 'read -p "Enter something: "; echo "Entered: [$REPLY]"'
Note that the interactive read prompt only works correctly with a pty, so the -t option is needed.
Using a pty has a notable side effect: stdout and stderr are combined and both reported via stdout; in other words: you lose the distinction between regular and error output; e.g.:
ssh jdoe#server 'echo out; echo err >&2' # OK - stdout and stderr separate
ssh -t jdoe#server 'echo out; echo err >&2' # !! stdout + stderr -> stdout
In the absence of this argument, ssh creates an interactive shell - including when you send commands via stdin, which is where the trouble begins:
For an interactive shell, ssh normally allocates a pty (pseudo-terminal) by default, except if its stdin is not connected to a (real) terminal.
Sending commands via stdin means that ssh's stdin is no longer connected to a terminal, so no pty is created, and ssh warns you accordingly:
Pseudo-terminal will not be allocated because stdin is not a terminal.
Even the -t option, whose express purpose is to request creation of a pty, is not enough in this case: you'll get the same warning.
Somewhat curiously, you must then double the -t option to force creation of a pty: ssh -t -t ... or ssh -tt ... shows that you really, really mean it.
Perhaps the rationale for requiring this very deliberate step is that things may not work as expected. For instance, on macOS 10.12, the apparent equivalent of the above command, providing the commands via stdin and using -tt, does not work properly; the session gets stuck after responding to the read prompt:
ssh -tt jdoe#server <<<'read -p "Enter something: "; echo "Entered: [$REPLY]"'
In the unlikely event that the commands you want to pass as an argument make the command line too long for your system (if its length approaches getconf ARG_MAX - see this article), consider copying the code to the remote system in the form of a script first (using, e.g., scp), and then send a command to execute that script.
In a pinch, use -T, and provide the commands via stdin, with a trailing exit command, but note that if you also need interactive features, using -tt in lieu of -T may not work.
The warning message Pseudo-terminal will not be allocated because stdin is not a terminal. is due to the fact that no command is specified for ssh while stdin is redirected from a here document.
Due to the lack of a specified command as an argument ssh first expects an interactive login session (which would require the allocation of a pty on the remote host) but then has to realize that its local stdin is no tty/pty. Redirecting ssh's stdin from a here document normally requires a command (such as /bin/sh) to be specified as an argument to ssh - and in such a case no pty will be allocated on the remote host by default.
Since there are no commands to be executed via ssh that require the presence of a tty/pty (such as vim or top) the -t switch to ssh is superfluous.
Just use ssh -T user#server <<EOT ... or ssh user#server /bin/bash <<EOT ... and the warning will go away.
If <<EOF is not escaped or single-quoted (i. e. <<\EOT or <<'EOT') variables inside the here document will be expanded by the local shell before it is executing ssh .... The effect is that the variables inside the here document will remain empty because they are defined only in the remote shell.
So, if $REL_DIR should be both accessible by the local shell and defined in the remote shell, $REL_DIR has to be defined outside the here document before the ssh command (version 1 below); or, if <<\EOT or <<'EOT' is used, the output of the ssh command can be assigned to REL_DIR if the only output of the ssh command to stdout is genererated by echo "$REL_DIR" inside the escaped/single-quoted here document (version 2 below).
A third option would be to store the here document in a variable and then pass this variable as a command argument to ssh -t user#server "$heredoc" (version 3 below).
And, last but not least, it would be no bad idea to check if the directories on the remote host were created successfully (see: check if file exists on remote host with ssh).
# version 1
unset DEP_ROOT REL_DIR
DEP_ROOT='/tmp'
datestamp=$(date +%Y%m%d%H%M%S)
REL_DIR="${DEP_ROOT}/${datestamp}"
ssh localhost /bin/bash <<EOF
if [ ! -d "$DEP_ROOT" ] && [ ! -e "$DEP_ROOT" ]; then
echo "creating the root directory" 1>&2
mkdir "$DEP_ROOT"
fi
mkdir "$REL_DIR"
#echo "$REL_DIR"
exit
EOF
scp -r ./dir1 user#server:"$REL_DIR"
scp -r ./dir2 user#server:"$REL_DIR"
# version 2
REL_DIR="$(
ssh localhost /bin/bash <<\EOF
DEP_ROOT='/tmp'
datestamp=$(date +%Y%m%d%H%M%S)
REL_DIR="${DEP_ROOT}/${datestamp}"
if [ ! -d "$DEP_ROOT" ] && [ ! -e "$DEP_ROOT" ]; then
echo "creating the root directory" 1>&2
mkdir "$DEP_ROOT"
fi
mkdir "$REL_DIR"
echo "$REL_DIR"
exit
EOF
)"
scp -r ./dir1 user#server:"$REL_DIR"
scp -r ./dir2 user#server:"$REL_DIR"
# version 3
heredoc="$(cat <<'EOF'
# -onlcr: prevent the terminal from converting bare line feeds to carriage return/line feed pairs
stty -echo -onlcr
DEP_ROOT='/tmp'
datestamp="$(date +%Y%m%d%H%M%S)"
REL_DIR="${DEP_ROOT}/${datestamp}"
if [ ! -d "$DEP_ROOT" ] && [ ! -e "$DEP_ROOT" ]; then
echo "creating the root directory" 1>&2
mkdir "$DEP_ROOT"
fi
mkdir "$REL_DIR"
echo "$REL_DIR"
stty echo onlcr
exit
EOF
)"
REL_DIR="$(ssh -t localhost "$heredoc")"
scp -r ./dir1 user#server:"$REL_DIR"
scp -r ./dir2 user#server:"$REL_DIR"
I don't know where the hang comes from, but redirecting (or piping) commands into an interactive ssh is in general a recipe for problems. It is more robust to use the command-to-run-as-a-last-argument style and pass the script on the ssh command line:
ssh user#server 'DEP_ROOT="/home/matthewr/releases"
datestamp=$(date +%Y%m%d%H%M%S)
REL_DIR=$DEP_ROOT"/"$datestamp
if [ ! -d "$DEP_ROOT" ]; then
echo "creating the root directory"
mkdir $DEP_ROOT
fi
mkdir $REL_DIR'
(All in one giant '-delimited multiline command-line argument).
The pseudo-terminal message is because of your -t which asks ssh to try to make the environment it runs on the remote machine look like an actual terminal to the programs that run there. Your ssh client is refusing to do that because its own standard input is not a terminal, so it has no way to pass the special terminal APIs onwards from the remote machine to your actual terminal at the local end.
What were you trying to achieve with -t anyway?
After reading a lot of these answers I thought I would share my resulting solution. All I added is /bin/bash before the heredoc and it doesn't give the error anymore.
Use this:
ssh user#machine /bin/bash <<'ENDSSH'
hostname
ENDSSH
Instead of this (gives error):
ssh user#machine <<'ENDSSH'
hostname
ENDSSH
Or use this:
ssh user#machine /bin/bash < run-command.sh
Instead of this (gives error):
ssh user#machine < run-command.sh
EXTRA:
If you still want a remote interactive prompt e.g. if the script you're running remotely prompts you for a password or other information, because the previous solutions won't allow you to type into the prompts.
ssh -t user#machine "$(<run-command.sh)"
And if you also want to log the entire session in a file logfile.log:
ssh -t user#machine "$(<run-command.sh)" | tee -a logfile.log
I was having the same error under Windows using emacs 24.5.1 to connect to some company servers through /ssh:user#host. What solved my problem was setting the "tramp-default-method" variable to "plink" and whenever I connect to a server I ommit the ssh protocol. You need to have PuTTY's plink.exe installed for this to work.
Solution
M-x customize-variable (and then hit Enter)
tramp-default-method (and then hit Enter again)
On the text field put plink and then Apply and Save the buffer
Whenever I try to access a remote server I now use C-x-f /user#host: and then input the password. The connection is now correctly made under Emacs on Windows to my remote server.

Resources