Bash Script Here Documents FTP - linux

I recently discovered "here" statements while working with a bash script for automating an ftp process.
Reference for Here Documents: http://tldp.org/LDP/abs/html/here-docs.html
The ftp process takes fairly long in the bash script, and I wanted specifically to run it in the background and have the next line of the bash script continue after the ftp process. How would I do this for "here" documents?
FTP snippet:
USER="test"
PASSWD="test"
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
quit
END_SCRIPT
For example:
I want to be able to do this:
run ftp snippet &
run other shell commands
but I'm not quite sure where to put the &
I have tried so far:
Attempt 1: (I believe that this is syntactically incorrect and does not work):
function do_ftp() {
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
quit
END_SCRIPT
}
do_ftp &
//additional commands
Attempt 2:
USER="test"
PASSWD="test"
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
quit
END_SCRIPT &
Attempt 3 :
USER="test"
PASSWD="test"
ftp -n $HOST & <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
quit
END_SCRIPT

I haven't tested it on ftp BUT:
I know when you want to put variables on HEREDOC you should do something else
For example I tried the following for ssh command:
while read pass port user ip fileinput fileoutput filetemp; do
sshpass -p$pass ssh -o 'StrictHostKeyChecking no' -p $port $user#$ip fileinput=$fileinput fileoutput=$fileoutput filetemp=$filetemp 'bash -s'<<ENDSSH1
python /path/to/f.py $fileinput $fileoutput $filetemp
ENDSSH1
done <<____HERE1
PASS PORT USER IP FILE-INPUT FILE-OUTPUT FILE-TEMP
____HERE1
May be something like this is needed by you...

Related

Can not enter password in Shell Script

I can not enter my pass word.
My shell script is here.
#!/bin/bash
expect -c "
set timeout 2
spawn ftp ${HOST}
expect \"Name\(${HOST}:root\):\"
send ${USER}\n
expect \"Password:\"
send ${PASS}\n
interact
"
I can enter user-name, but password is not work....
like this return.
spawn ftp ******
Connected to ****** (******).
220 (vsFTPd 2.2.2)
Name (******:root): ${USER}
331 Please specify the password.
Password:[root#........ bin]#
I think I send password... Could you please help me?
Thank you.
SOLVE MY QUESTION
I think it is easy for me to use this.
it is easy for me to use this.
#!/bin/sh
SERVER=$1
USER=$2
PASS=$3
FILE=$4
ftp -n <<END
open $SERVER
user $USER $PASS
cd /tmp
binary
prompt
mput $FILE
END
you have to prompt password and send it to variable
echo -n "Enter Password:"
read PASS

Is there a way to automatically answer for user prompt when doing ssh in a shell script without using expect or spawn?

I'm trying to test ssh trust between a linux box against 12 other linux boxes using a shell script and I'm trying to pass user input as 'yes' for the question below automatically.
Are you sure you want to continue connecting (yes/no)?
but the script is failing with error 'Host key verification failed'. I manually executed the ssh command with << EOT on one of the server but the I still get user prompt question. Is there any-other way to pass input value for user prompts automatically while running ssh command?
Note: I cannot use spawn or except do you some system limitation and I cannot install them due to organisations access restrictions.
I tried with the following options but none of them worked for me
[command] << [EOT, EOL, EOF]
echo 'yes'
[EOT, EOL, EOF]
yes | ./script.sh
printf "yes" | ./script.sh
echo "yes" | ./script.sh
./script.sh 'read -p "Are you sure you want to continue connecting (yes/no)?";echo "yes"'
sh```
for server in `cat server_list` ; do
UPPER_MACHINE_NAME=`echo $server | cut -d '.' -f 1`
UPPER_MACHINE_NAME=${UPPER_MACHINE_NAME^^}
ssh -tt user#$UPPER_MACHINE_NAME << EOT
echo 'yes'
touch /usr/Finastra/sshtest.txt
EOT
done
```

Having an issue passing variables to subshell

So here is my problem, I have this script I wrote where I'm exporting two variables however they're not making it into the subshell.
The point of this script is to change a users password and clear out their pam_tally for CentOS and Ubuntu hosts.
A little background is that this environment's users are managed by puppet but the passwords are all local, ssh keys are not allowed either (this is set in stone and can't be changed so I have to work with what I got) and the reason is that every log in has to be manual (even number of sessions are limited to two so you can't even user csshX effectively).
Here is my script
#!/bin/bash
echo "Please enter user whose password you want to change"
read NEWUSER
echo "Please enter new password for user"
read -s -p "Temp Password:" TEMPPASSWORD
PASSWORD=$TEMPPASSWORD
export PASSWORD
NEWUSER2=$NEWUSER
export NEWUSER2
for i in HOST{cluster1,cluster2,cluster3}0{1..9}
do
ping -c 2 $i && (echo $i ; ssh -t $i '
sudo pam_tally2 --user=$NEWUSER2 --reset
echo -e "$PASSWORD\n$PASSWORD" | sudo passwd $NEWUSER2
sudo chage -d 0 $NEWUSER2
')
done
You are using ssh to connect to a remote host and run a script on that host. ssh does not export the local environment to the remote session but instead performs a login on the remote host which sets the environment according to the remote user's configuration on the remote host.
I suggest you pass all needed values via the command you want to execute. This could be done like this:
ssh -t $i '
sudo pam_tally2 --user='"$NEWUSER2"' --reset
echo -e "'"$PASSWORD"'\n'"$PASSWORD"'" | sudo passwd '"$NEWUSER2"'
sudo chage -d 0 '"$NEWUSER2"
Watch closely how this uses quotes. At each occasion where you used a variable, I terminate the single-quoted string (using '), then add a double-quoted use of the variable (e. g. "$PASSWORD") and then start the single-quoted string again (using ' again). This way, the shell executing the ssh command will expand the variables already, so you have no need to pass them into the remote shell.
But be aware that special characters in the password (like " or ' or or maybe a bunch of other characters) can mean trouble using this simple mechanism. To be safe against this as well, you would need to use the %q format specifier of the printf command to quote your values before passing them:
ssh -t "$i" "$(printf '
sudo pam_tally2 --user=%q --reset
{ echo %q; echo %q; } | sudo passwd %q
sudo chage -d 0 %q' \
"$NEWUSER2" "$PASSWORD" "$PASSWORD" "$NEWUSER2" "$NEWUSER2")"

ssh executes remote command in the background [duplicate]

This is a follow-on question to the How do you use ssh in a shell script? question. If I want to execute a command on the remote machine that runs in the background on that machine, how do I get the ssh command to return? When I try to just include the ampersand (&) at the end of the command it just hangs. The exact form of the command looks like this:
ssh user#target "cd /some/directory; program-to-execute &"
Any ideas? One thing to note is that logins to the target machine always produce a text banner and I have SSH keys set up so no password is required.
I had this problem in a program I wrote a year ago -- turns out the answer is rather complicated. You'll need to use nohup as well as output redirection, as explained in the wikipedia artcle on nohup, copied here for your convenience.
Nohuping backgrounded jobs is for
example useful when logged in via SSH,
since backgrounded jobs can cause the
shell to hang on logout due to a race
condition [2]. This problem can also
be overcome by redirecting all three
I/O streams:
nohup myprogram > foo.out 2> foo.err < /dev/null &
This has been the cleanest way to do it for me:-
ssh -n -f user#host "sh -c 'cd /whereever; nohup ./whatever > /dev/null 2>&1 &'"
The only thing running after this is the actual command on the remote machine
Redirect fd's
Output needs to be redirected with &>/dev/null which redirects both stderr and stdout to /dev/null and is a synonym of >/dev/null 2>/dev/null or >/dev/null 2>&1.
Parantheses
The best way is to use sh -c '( ( command ) & )' where command is anything.
ssh askapache 'sh -c "( ( nohup chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'
Nohup Shell
You can also use nohup directly to launch the shell:
ssh askapache 'nohup sh -c "( ( chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'
Nice Launch
Another trick is to use nice to launch the command/shell:
ssh askapache 'nice -n 19 sh -c "( ( nohup chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'
If you don't/can't keep the connection open you could use screen, if you have the rights to install it.
user#localhost $ screen -t remote-command
user#localhost $ ssh user#target # now inside of a screen session
user#remotehost $ cd /some/directory; program-to-execute &
To detach the screen session: ctrl-a d
To list screen sessions:
screen -ls
To reattach a session:
screen -d -r remote-command
Note that screen can also create multiple shells within each session. A similar effect can be achieved with tmux.
user#localhost $ tmux
user#localhost $ ssh user#target # now inside of a tmux session
user#remotehost $ cd /some/directory; program-to-execute &
To detach the tmux session: ctrl-b d
To list screen sessions:
tmux list-sessions
To reattach a session:
tmux attach <session number>
The default tmux control key, 'ctrl-b', is somewhat difficult to use but there are several example tmux configs that ship with tmux that you can try.
I just wanted to show a working example that you can cut and paste:
ssh REMOTE "sh -c \"(nohup sleep 30; touch nohup-exit) > /dev/null &\""
You can do this without nohup:
ssh user#host 'myprogram >out.log 2>err.log &'
Quickest and easiest way is to use the 'at' command:
ssh user#target "at now -f /home/foo.sh"
I think you'll have to combine a couple of these answers to get what you want. If you use nohup in conjunction with the semicolon, and wrap the whole thing in quotes, then you get:
ssh user#target "cd /some/directory; nohup myprogram > foo.out 2> foo.err < /dev/null"
which seems to work for me. With nohup, you don't need to append the & to the command to be run. Also, if you don't need to read any of the output of the command, you can use
ssh user#target "cd /some/directory; nohup myprogram > /dev/null 2>&1"
to redirect all output to /dev/null.
This worked for me may times:
ssh -x remoteServer "cd yourRemoteDir; ./yourRemoteScript.sh </dev/null >/dev/null 2>&1 & "
You can do it like this...
sudo /home/script.sh -opt1 > /tmp/script.out &
It appeared quite convenient for me to have a remote tmux session using the tmux new -d <shell cmd> syntax like this:
ssh someone#elsewhere 'tmux new -d sleep 600'
This will launch new session on elsewhere host and ssh command on local machine will return to shell almost instantly. You can then ssh to the remote host and tmux attach to that session. Note that there's nothing about local tmux running, only remote!
Also, if you want your session to persist after the job is done, simply add a shell launcher after your command, but don't forget to enclose in quotes:
ssh someone#elsewhere 'tmux new -d "~/myscript.sh; bash"'
Actually, whenever I need to run a command on a remote machine that's complicated, I like to put the command in a script on the destination machine, and just run that script using ssh.
For example:
# simple_script.sh (located on remote server)
#!/bin/bash
cat /var/log/messages | grep <some value> | awk -F " " '{print $8}'
And then I just run this command on the source machine:
ssh user#ip "/path/to/simple_script.sh"
If you run remote command without allocating tty, redirect stdout/stderr works, nohup is not necessary.
ssh user#host 'background command &>/dev/null &'
If you use -t to allocate tty to run interactive command along with background command, and background command is the last command, like this:
ssh -t user#host 'bash -c "interactive command; nohup backgroud command &>/dev/null &"'
It's possible that background command doesn't actually start. There's race here:
bash exits after nohup starts. As a session leader, bash exit results in HUP signal sent to nohup process.
nohup ignores HUP signal.
If 1 completes before 2, the nohup process will exit and won't start the background command at all. We need to wait nohup start the background command. A simple workaroung is to just add a sleep:
ssh -t user#host 'bash -c "interactive command; nohup backgroud command &>/dev/null & sleep 1"'
The question was asked and answered years ago, I don't know if openssh behavior changed since then. I was testing on:
OpenSSH_8.6p1, OpenSSL 1.1.1g FIPS 21 Apr 2020
I was trying to do the same thing, but with the added complexity that I was trying to do it from Java. So on one machine running java, I was trying to run a script on another machine, in the background (with nohup).
From the command line, here is what worked: (you may not need the "-i keyFile" if you don't need it to ssh to the host)
ssh -i keyFile user#host bash -c "\"nohup ./script arg1 arg2 > output.txt 2>&1 &\""
Note that to my command line, there is one argument after the "-c", which is all in quotes. But for it to work on the other end, it still needs the quotes, so I had to put escaped quotes within it.
From java, here is what worked:
ProcessBuilder b = new ProcessBuilder("ssh", "-i", "keyFile", "bash", "-c",
"\"nohup ./script arg1 arg2 > output.txt 2>&1 &\"");
Process process = b.start();
// then read from process.getInputStream() and close it.
It took a bit of trial & error to get this working, but it seems to work well now.
YOUR-COMMAND &> YOUR-LOG.log &
This should run the command and assign a process id you can simply tail -f YOUR-LOG.log to see results written to it as they happen. you can log out anytime and the process will carry on
If you are using zsh then use program-to-execute &! is a zsh-specific shortcut to both background and disown the process, such that exiting the shell will leave it running.
A follow-on to #cmcginty's concise working example which also shows how to alternatively wrap the outer command in double quotes. This is how the template would look if invoked from within a PowerShell script (which can only interpolate variables from within double-quotes and ignores any variable expansion when wrapped in single quotes):
ssh user#server "sh -c `"($cmd) &>/dev/null </dev/null &`""
Inner double-quotes are escaped with back-tick instead of backslash. This allows $cmd to be composed by the PowerShell script, e.g. for deployment scripts and automation and the like. $cmd can even contain a multi-line heredoc if composed with unix LF.
First follow this procedure:
Log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:
a#A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a#A
Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):
a#A:~> ssh b#B mkdir -p .ssh
b#B's password:
Finally append a's new public key to b#B:.ssh/authorized_keys and enter b's password one last time:
a#A:~> cat .ssh/id_rsa.pub | ssh b#B 'cat >> .ssh/authorized_keys'
b#B's password:
From now on you can log into B as b from A as a without password:
a#A:~> ssh b#B
then this will work without entering a password
ssh b#B "cd /some/directory; program-to-execute &"
I think this is what you need:
At first you need to install sshpass on your machine.
then you can write your own script:
while read pass port user ip; do
sshpass -p$pass ssh -p $port $user#$ip <<ENDSSH1
COMMAND 1
.
.
.
COMMAND n
ENDSSH1
done <<____HERE
PASS PORT USER IP
. . . .
. . . .
. . . .
PASS PORT USER IP
____HERE

How to pass a value from bash into expect?

I've got a really simple bash script which requires expect.
I need to pass a value from bash into expect and I'm not trying to ssh into another server or anything (cause I only seem to find questions regarding logging into another server via ssh).
The idea is simply something like this:
#!/usr/bin/env bash
echo "Please enter your password: "
read PASSWD
x=$(expect -c '
spawn su -c 'whoami'
expect "Password:"
send "$PASSWD\r"
interact
')
So this doesn't work. The expect shell doesn't recognize the $PASSWD variable.
How may this be accomplished?
Thank you.
Another option would be to store the PASSWD in the environment and let expect pick it up there:
read -p "Your password: " passwd
export passwd
expect -c '... ; send "$env(passwd)\r"; ...'
Probably the best choice security-wise is have expect prompt for the password: then, the password will not show up on the command line nor in the environment.
expect -c <<'END'
stty -echo
send_user "Your password: "
expect_user -re "(.*)\n"
send_user "\n"
set passwd $expect_out(1,string)
stty echo
# your script starts here
...
send "$passwd\r"
...
END
Variables within single quotes are not expanded by the shell, that's why in this case $PASSWD remains the literal string $PASSWD.
Try changing the quotes:
#!/usr/bin/env bash
echo "Please enter your password: "
read PASSWD
x=$(expect -c "
spawn su -c 'whoami'
expect 'Password:'
send '$PASSWD\r'
interact
")
One more note: you should be aware that this could pose a security risk, as the password will be visible in plaintext in the process list while the command is running.

Resources