Getting files using expect command in a shell script - linux

I am trying to automate SFTP command using a UNIX shell expect script with the help of expect command. My code is as below. When I tried executing the script it throws me an exception. Can somebody help me if you have faced the similar issue. Any help would be appreciated. Thank you.
--Shell Script
#This ftp script will copy all the files from Buxton directory into the local directory.
#!/usr/local/bin/expect
spawn sftp -b cmdFile XYZ#sftp.abc.com
expect "password:"
send "6n8W6u0J"
interact
Command File
SRC_DIR='From_Bux'
TRGT_DIR='/work/informatica/release81x/DEV2/DEP_EXT_ARCH/SrcFiles/LANDING'
FILE='buxt_summary_20140702.csv'
cd $SRC_DIR
lcd $TRGT_DIR
get $FILE
Execution command ./get_bxtn_src_files.ksh
Error Message
$ ./get_bxtn_src_files.ksh
./get_bxtn_src_files.ksh[3]: spawn: not found
couldn't read file "password:": no such file or directory
send: unable to stat draft file /home/vvutukuri/Mail/6n8W6u0J: No such file or directory
./get_bxtn_src_files.ksh[6]: interact: not found

This looks like an interpreter issue.
Check again if you script has #! as the to absolute first characters in your script.
head -1 get_bxtn_src_files.ksh | cut -c 1-2
# The result should be: #!
If not, the script will be interpreted with the shell you are logged in with.
The errors you are receiving indicates that the expect commands are interpreted by a regular shell (ksh/sh/bash/...) instead of expect.
See the following examples:
$ ksh
$ cd test
$ cat expect1-test.exp # the hash-bang is in my first line
#!/usr/bin/expect
spawn sftp -b cmdFile XYZ#sftp.abc.com
expect "password:"
send "6n8W6u0J"
interact
$ ./expect1-test.exp # Notice the error messages, they come from Expect
spawn sftp -b cmdFile XYZ#sftp.abc.com
No such file or directory (cmdFile).
send: spawn id exp4 not open
while executing
"send "6n8W6u0J""
(file "./expect1-test.exp" line 4)
$ cat expect2-test.exp # Notice the blank line
#!/usr/bin/expect
spawn sftp -b cmdFile XYZ#sftp.abc.com
expect "password:"
send "6n8W6u0J"
interact
$ ./expect2-test.exp # Notice that i.e. spawn is not found.
./expect2-test.exp[3]: spawn: not found [No such file or directory]
couldn't read file "password:": no such file or directory
./expect2-test.exp[5]: send: not found [No such file or directory]
./expect2-test.exp[6]: interact: not found [No such file or directory]
You should also look into autoexpect which you can use to create an expect script you can use and edit.
Finally: You should consider switching to using ssh private/public keys instead of expect, unless - of course - you are prevented from doing this / need to use a password.
SSH private/public keys
On the remote machine
mkdir -p $HOME/.ssh/
chmod 700 $HOME/.ssh/
chmod go-w $HOME # NB! If sshd is set to strict your $HOME - and parent directories must _not_ be world writable
On the local machine
mkdir -p $HOME/.ssh # Create the .ssh directory if necessary
chmod 700 $HOME/.ssh # Low perms to protect the directory
ssh-keygen -t dsa -f $HOME/.ssh/id_dsa -N "" # Create the private/pulic key pair
# with no pw since you'll use it in a script
scp $HOME/.ssh/id_dsa.pub remote_machine:.ssh/authorized_keys # Provided it's not allready there, in which case you should add your public key to the bottom of the file
Verify that you are able to log in withouth a password:
ssh remote_machine # or user#remote_machine
Your script can now be:
#!/bin/ksh
# Or /usr/bin/ksh..?
sftp -b cmdFileXYZ#sftp.abc.com
On the command line you can also use
scp XYZ#sftp.abc.com:$SRC_DIR/$FILE $TRGT_DIR/

Related

How to remotely SCP latest file in one server to another?

I have 3 Linux machines, namely client, server1 and server2.
I am trying to achieve something like this. I would like to copy the latest file in a particular directory of server1 to server2. I will not be doing this by logging into server1 directly, but I always log on to client machine first. Let me list down the step by step approach which is happening now:
Log on to client machine using ssh
SSH into server1
Go to directory /home/user1 in server1 using the command ls /home/user1 -Art | tail -n 1
SCP the latest file to /home/user2 directory of server2
This manual operation is happening just fine. I have automated this using a one line script like below:
ssh user1#server1 scp -o StrictHostKeyChecking=no /home/user1/test.txt user2#server2:/home/user2
But as you can see, I am uploading the file /home/user1/test.txt. How can I modify this script to always upload the latest file in the directory /home/user1?
If zsh is available on server1, you could use its advanced globbing features to find the most recent file to copy:
ssh user1#server1 \
"zsh -c 'scp -o StrictHostKeyChecking=no /home/user1/*(om[1]) user2#server2:/home/user2'"
The quoting is important in case your remote shell on server1 is not zsh.
You can use SSH to list the last file and after user scp to copy the file as follow:
FILE=$(ssh user1#server1 "ls -tp $REMOTE_DIR |grep -v / | grep -m1 \"\""); scp user1#server1:$FILE user2#server2:/home/user2
Before launch these command you need to set the remote directory where to search for the last modified file as:
REMOTE_DIR=/home/user1

How to get root access after ssh in a script? [duplicate]

I have a script which runs another script via SSH on a remote server using sudo. However, when I type the password, it shows up on the terminal. (Otherwise it works fine)
ssh user#server "sudo script"
What's the proper way to do this so I can type the password for sudo over SSH without the password appearing as I type?
Another way is to use the -t switch to ssh:
ssh -t user#server "sudo script"
See man ssh:
-t Force pseudo-tty allocation. This can be used to execute arbi-
trary screen-based programs on a remote machine, which can be
very useful, e.g., when implementing menu services. Multiple -t
options force tty allocation, even if ssh has no local tty.
I was able to fully automate it with the following command:
echo pass | ssh -tt user#server "sudo script"
Advantages:
no password prompt
won't show password in remote machine bash history
Regarding security: as Kurt said, running this command will show your password on your local bash history, and it's better to save the password in a different file or save the all command in a .sh file and execute it. NOTE: The file need to have the correct permissions so that only the allowed users can access it.
Sudo over SSH passing a password, no tty required:
You can use sudo over ssh without forcing ssh to have a pseudo-tty (without the use of the ssh "-t" switch) by telling sudo not to require an interactive password and to just grab the password off stdin. You do this by using the "-S" switch on sudo. This makes sudo listen for the password on stdin, and stop listening when it sees a newline.
Example 1 - Simple Remote Command
In this example, we send a simple whoami command:
$ ssh user#server cat \| sudo --prompt="" -S -- whoami << EOF
> <remote_sudo_password>
root
We're telling sudo not to issue a prompt, and to take its input from stdin. This makes the sudo password passing completely silent so the only response you get back is the output from whoami.
This technique has the benefit of allowing you to run programs through sudo over ssh that themselves require stdin input. This is because sudo is consuming the password over the first line of stdin, then letting whatever program it runs continue to grab stdin.
Example 2 - Remote Command That Requires Its Own stdin
In the following example, the remote command "cat" is executed through sudo, and we are providing some extra lines through stdin for the remote cat to display.
$ ssh user#server cat \| sudo --prompt="" -S -- "cat" << EOF
> <remote_sudo_password>
> Extra line1
> Extra line2
> EOF
Extra line1
Extra line2
The output demonstrates that the <remote_sudo_password> line is being consumed by sudo, and that the remotely executed cat is then displaying the extra lines.
An example of where this would be beneficial is if you want to use ssh to pass a password to a privileged command without using the command line. Say, if you want to mount a remote encrypted container over ssh.
Example 3 - Mounting a Remote VeraCrypt Container
In this example script, we are remotely mounting a VeraCrypt container through sudo without any extra prompting text:
#!/bin/sh
ssh user#server cat \| sudo --prompt="" -S -- "veracrypt --non-interactive --stdin --keyfiles=/path/to/test.key /path/to/test.img /mnt/mountpoint" << EOF
SudoPassword
VeraCryptContainerPassword
EOF
It should be noted that in all the command-line examples above (everything except the script) the << EOF construct on the command line will cause the everything typed, including the password, to be recorded in the local machine's .bash_history. It is therefore highly recommended that for real-world use you either use do it entirely through a script, like the veracrypt example above, or, if on the command line then put the password in a file and redirect that file through ssh.
Example 1a - Example 1 Without Local Command-Line Password
The first example would thus become:
$ cat text_file_with_sudo_password | ssh user#server cat \| sudo --prompt="" -S -- whoami
root
Example 2a - Example 2 Without Local Command-Line Password
and the second example would become:
$ cat text_file_with_sudo_password - << EOF | ssh va1der.net cat \| sudo --prompt="" -S -- cat
> Extra line1
> Extra line2
> EOF
Extra line1
Extra line2
Putting the password in a separate file is unnecessary if you are putting the whole thing in a script, since the contents of scripts do not end up in your history. It still may be useful, though, in case you want to allow users who should not see the password to execute the script.
Assuming you want no password prompt:
ssh $HOST 'echo $PASSWORD | sudo -S $COMMMAND'
Example
ssh me#localhost 'echo secret | sudo -S echo hi' # outputs 'hi'
The best way is ssh -t user#server "sudo <scriptname>", for example ssh -t user#server "sudo reboot".
It will prompt for password for user first and then root(since we are running the script or command with root privilege.
I hope it helped and cleared your doubt.
NOPASS in the configuration on your target machine is the solution. Continue reading at http://maestric.com/doc/unix/ubuntu_sudo_without_password
echo $VAR_REMOTEROOTPASS | ssh -tt -i $PATH_TO_KEY/id_mykey $VAR_REMOTEUSER#$varRemoteHost
echo \"$varCommand\" | sudo bash
confirming that the answer of #ofirule is working like a charm.
I try ot even with sshpass & it works. This is how to use it with sshpass:
echo $pass | sshpass -p $pass ssh -tt cloud_user#$ip "sudo su -"
you will find yourself in the root shell directly

FTP script not working

I have an FTP script running on Linux and it is failing. Here is the script:
/usr/bin/ftp -v -i -n my.ftp.server << cmd
user ftpuser password
binary
ls
<some other commands here>
quit cmd
It returns an error:
421 Service not available, remote server has closed connection
Not connected.
The weird thing here is that if I just typed this in the command line:
/usr/bin/ftp my.ftp.server
It asks for a username and password and after I supplied them, I was able to connect!
In ftp> I type ls and I can see the files from the FTP server.
What is wrong with my script?
And also, I don't have putty access to the FTP server so I can't see the logs from there. Any ideas?
Thanks!
Here is an example of a correct ftp linux script.
#!/bin/sh
HOST='ftp.server.com'
USER='user'
PASSWD='pw'
FILE='file.txt' #sample file to upload
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
put $FILE #sample command
#some other commands here
quit
END_SCRIPT
exit 0

Assign contents of file to variable over remote ssh from a script running in Jenkins

I have opened a remote ssh session from a script and on remote server there is a file containing version information.
I am trying to assign that version to variable and move current version contents to folder name same as version.
The main script is running in jenkins
I am doing something like this
ssh -i /home/user/.ssh/id_rsa -t -t remoteServer<<EOF
cd $WEB_DIR
VERSION=$(cat $WEB_DIR/version.info)
mv -f $WEB_DIR $BACKUP_DIR/$VERSION
exit
EOF
My VERSION variable is always empty. When I run same locally on that server it gives me version value. Something is different over remote ssh session within a script
Actually I found the way to do it in 2 steps.
$WEB_DIR is set as local variable set in main script.
$WEB_DIR="/usr/local/tomcat/webapps/ROOT"
OLD_VERSION=$(ssh -i /home/user/.ssh/id_rsa -tt user#remoteServer "cat $WEB_DIR/version.info")
ssh -i /home/user/.ssh/id_rsa -t -t user#remoteServer<<EOF
cd $WEB_DIR
mv -f $WEB_DIR $BACKUP_DIR/$OLD_VERSION
# I am executing more commands in here
exit
EOF
Use of double quotes "" in first command is must if want to use local variable.

logging on a remote machine

Is it possible to login to a particular folder of a remote machine, through shell script:
instead of ssh and then cd. Can I do it in one command?
Is it possible to copy a file on two different folders of a remote machine through scp in one go?
You can pass the cp command to ssh as follows:
ssh user#host "cp /path/to/folder1/file /path/to/folder2"
or combine with cd:
ssh user#host "cd /path/to/folder1; cp file /path/to/folder2"
I think there is no real solution to this. You can try ssh user#host bash -c "cd /tmp; bash -i" if your shell is bash. That will look a little like it could work, but you will quickly discover it's not really working. (Try to invoke an editor such as vi and you will see.)
You can also put cd /tmp into the ~/.bashrc on the remote end. This way, ssh user#host does what you ask for. But I guess you want to be able to go to different directories, so this won't fly either.
I think the best approach is to make it convenient to enter the cd command, e.g. using aliases. Then it's less typing.
The cleanest way for your first question is Expect:
#!/usr/local/bin/expect
spawn ssh user#host
expect -re {\$ $} ;# adjust to suit your prompt
send "cd some/dir\r"
interact
I'm unclear about your 2nd question: do you have 2 remote files that you want to copy to your local machine, or do you want to copy a file from one remote dir to another remote dir?
scp user#host:/path/to/file1 user#host:/path/to/file2 .
ssh user#host "cp /path/to/file1 /path/to/folder2/"

Resources