Provide commands automatically to ftp in bash script - linux

I am trying to create a bash script with ftp.
If I use terminal and put the commands below one bye one, it works like a charm.
$ ftp 192.168.1.4 2121
Connected to 192.168.1.4.
220 SwiFTP 1.7.11 ready
Name (192.168.1.4:user):
331 Send password
Password:
230 Access granted
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd Study/Math
ftp> put ~/Documents/Math/lesson.pdf lesson.pdf
I am trying to automate this command with a bash script:
#!/bin/bash
ftp 192.168.1.4 2121
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
It is not working. I know here ftp is a independent tool and I have to put those command while the ftp program is running. I searched internet tried various techniques (like printf, expect, etc...) but it did not worked. I also tried to use some scripts from internet to automate this process, but nothing helped. I am a newbie in bash scripting and these stuffs. Can you guys help me to solve this Problem?
Thanks in advance...

First, don't do this if you have any other option. It's a fairly standard idiom, but it's really broken in a lot of ways. If you're very sure that it won't ever do anything unpredictable, and that when it does it will be ok anyway, then sure, but in general...
use something besides ftp. For example, scp works quite well and has a checkable return code that is actually useful.
use a more granular programming language with modules. Don't get me wrong, I love bash and will always use it first when I can, but pumping a stream of commands into an ftp like a fire-and-forget UDP torpedo without any easy way of checking that each worked is just bad habit. Try Perl, or Python, or any other damned thing that lets you check a return code on each command and react accordingly. :)
if you MUST use bash (and yes, I have done it), and if it's important enough to check (what isn't?), then think about how you're going to do that. Maybe you can just pull lesson.pdf back to a local testme.pdf and cmp them to make sure it's good, which seems pretty easy. For any more complex script, you might need to run the ftp as a coprocesses and feed it commands one at a time, then read back it's output and parse for return codes, because ftp generally only reports errors there...and watch out for "500 bytes sent", which isn't a 500 error.
Either way, good luck. In many ways, simple is still best.

You might be interested in what is known as a heredoc
#!/usr/bin/env bash
ftp -n 92.168.1.4 2121 <<EOF
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
EOF
Since the ftp program reads its input from /dev/stdin, you can use a heredoc to define what should be passed from /dev/stdin

Okay, after User123 gave me a link, I finally solved my problem. So, I am giving my working script.
#!/bin/sh
HOST='192.168.1.4 2121'
USER='username_of_ftp'
PASSWD='password_of_ftp'
/usr/bin/ftp -n $HOST <<END_SCRIPT
user ${USER} ${PASSWD}
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
quit
END_SCRIPT
exit 0

While the ftp approach you're working with might be OK for your purposes, it's not going to behave properly if the server doesn't respond as expected.
As has already been suggested, use something other than ftp if at all possible. Scp or rsync using key authentication will work better, be more convenient since it won't break every time you change your password, and be more secure.

Related

Can PSFTP execute loops?

I've searched a lot on the internet but haven't been able to find any useful info this yet. Does PFTP not allow you to run loops like 'IF' and 'WHILE' at all?
If it does, please let me know the syntax, I'm tried of banging my head against it. Annoyingly, PuTTY allows these commands but psftp doesn't seem to even though both are from the same family. I really hope there is a solution to this!
PSFTP isn't a language. It's just an SFTP client. SFTP itself is just a protocol for moving files between computers. If you have SFTP set up on the remote computer then it suggests that you have SSH running (since SFTP generally comes bundled with the SSH server install).
You can do a test in a bash shell script, for instance, to see if the file exists on the remote server, then execute your psftp command based on the result. Something like:
#!/bin/bash
# test if file exists on remote system
fileExists=$(ssh user#yourothercomputer "test -f /tmp/foo && echo 'true' || echo 'false'")
if $fileExists; then
psftp <whatever>
fi
You can stick that whole mess in a loop or whatevs. What's happening here is that we are sending a command test -f /tmp/foo && echo 'true' || echo 'false' to the remote computer to execute. The stdout of the command is returned and stored in the variable fileExists. Then we just test it.
If you are in windows you could convert this to a batch script and use plink.exe to send the command kind of like they do here. Or maybe just plop cygwin on your computer with an SSH and SFTP client and use what's above.
The big take-away here is that you will need a separate scripting environment to do the loop and run psftp based on a test.

Pass a password to ssh in pure bash

I want to pass a password to ssh using a bash script (Yes, I know that there are ssh keys that I could use, but this is not what I intend).
I found some solutions that were using expect but since it is not a standard bash tool I am wondering if I can do this using pipes.
Can someone explain to me, why exactly something like this:
echo "password\n" | ssh somehost.com
or
ssh somehost.com <(echo "password\n")
doesn't work? Is there any possibility to make it work? Maybe executing ssh as a different process, obtaining its PID and then sending a string directly to it?
You can not specify the password from the command line but you can do either using ssh keys or using sshpass as suggested by John C. or using a expect script.
To use sshpass, you need to install it first. Then
sshpass -f <(printf '%s\n' your_password) ssh user#hostname
instead of using sshpass -p your_password. As mentioned by Charles Duffy in the comments, it is safer to supply the password from a file or from a variable instead of from command line.
BTW, a little explanation for the <(command) syntax. The shell executes the command inside the parentheses and replaces the whole thing with a file descriptor, which is connected to the command's stdout. You can find more from this answer https://unix.stackexchange.com/questions/156084/why-does-process-substitution-result-in-a-file-called-dev-fd-63-which-is-a-pipe
Since there were no exact answers to my question, I made some investigation why my code doesn't work when there are other solutions that works, and decided to post what I found to complete the subject.
As it turns out:
"ssh uses direct TTY access to make sure that the password is indeed
issued by an interactive keyboard user." sshpass manpage
which answers the question, why the pipes don't work in this case. The obvious solution was to create conditions so that ssh "thought" that it is run in the regular terminal and since it may be accomplished by simple posix functions, it is beyond what simple bash offers.

In bash, how do I input a password for a background process command?

I'm trying to do an rsync backup from dreamhost to another host, here's the command i'd like to use:
nohup rsync -e "/usr/bin/ssh" --bwlimit=2000 -av username#server.dreamhost.com:remote_directory local_directory&
I'd like the process to keep running in the background and even when I disconnect. Problem is, I don't know how to put in the password when it's a background process. How do I do this?
Usually this is done by not requiring a password at all. Instead, consider configuring SSH to use a public key. There are several resources online (such as this one from dreamhost) that can help you do that.
I would use a key. If you need to protect the key with a password or you cannot use a key for whatever reason, then use expect to pass the password:
rsync_auto.sh:
#/bin/bash
expect <<<EOF
spawn nohup rsync -e "/usr/bin/ssh" --bwlimit=2000 -av username#server.dreamhost.com:remote_directory
expect "password:"
send "your_password\r"
expect eof
EOF
!!!Make sure that nobody except you can access the file!!!:
chmod 500 rsync_auto.sh
A little bit more elaborated way might be to store the password in a keyring application, like gnome-keyring instead of storing them in a plain file. I've found this article if you are interested.

to transfer files in linux fedora 12 by giving password at command prompt

I am fully aware that this question has been asked many times but I cant able to find any solution which satisy my requirement.
Task -> I need to transfer files from machine A to machineB and remotely execute scripts on Machine B. Due to my limitation I cant able to use keygen, expect utility or any other utility which requires to install packages. To Transfer the file I need to give password and I want to give password in Url. as this will run inside bash script and requires no user interference .
My investigation- I thought of using scp but realise, its not possible to give password at command prompt. So i wondering , if there is any other alternative from rsync .
below is the small attempt
#!/bin/bash
PATH=/usr/bin:/usr/sbin:/bin:/sbin
USER="bob"
RSYNC_PASSWORD="blue"
MACHINE_B="192.168.200.2"
if ping -c 1 -W 1 $MACHINE_B
then
echo "There is machine b as well"
echo " cheking to transfer file to machine b"
rsync lol.sh 192.168.200.2:/home/bob/
fi
Thanks and regards,
Sam
I have tried various option mentioned above ,, but unfortunately none of them works in my case. But I would like to thanks everyone for helping me and surely I have learned few new things specially rsync.
In my case, I have to rely on ssh keys to make it work.
From the rsync man page:
Some modules on the remote daemon may require authentication. If so, you will receive a password prompt when you connect. You can avoid the password
prompt by setting the environment variable RSYNC_PASSWORD to the password you want to use or using the --password-file option. This may be useful
when scripting rsync.

Webapp update shell script

I feel silly asking this...
I am not an expert on shell scripting, but I am finally in enough of a sysadmin role that I want to do this correctly.
I have a production server that hosts a webapp. Here is my routine.
1 - ssh to server
2 - cd django_src/django_apps/team_proj
3 - svn update
4 - sudo /etc/init.d/apache2 restart
5 - logout
I want to create a shell script for steps 2,3,4.
I can do this, but it will be a very plain and simple bash script simply containing the actual commands I type at the command line.
My question: What is the best way to script this kind of repetitive procedure in bash (Linux, Ubuntu) for a remote server?
Thanks!
The best way is simply as you suggest. Some things you should do for your script would be:
put set -e at the top of the script (after the shebang). This will cause your script to stop if any of the commands fail. So if it cannot cd to the directory, it will not run svn update or restart apache. You can do this programmatically by putting || exit 0 after each command, but if that's all you're doing, you may as well use set -e
Use full paths in your script. Do not assume the directory that the script is run from. In this specific case, the cd command has a relative path. Use a full (absolute) path, or use an environment variable like $HOME.
You may want to set up sudo so that it can run the command without asking for a password. This makes your script non-interactive which means it can be run in the background and from cron jobs and such.
As time goes by, you may add features and take command line arguments to parameterise the script. But don't bother doing this up front. Just evolve your scripts as you need.
There is nothing wrong with a simple bash script simply containing the actual commands you type at the command line. Don't make it more complicated than necessary.
I'd setup a cron job doing that automatically.
Since you're using python, check out fabric - you can use it to automate these kind of tasks. First install fabric:
$ sudo easy_install fabric
then write your fabric script:
from __future__ import with_statement
from fabric.api import *
def svnupdate():
with cd('django_src/django_apps/team_proj'):
run('svn update')
sudo('/etc/init.d/apache2 restart')
Save as fabfile.py, then run using the fab command:
$ fab -H hostname svnupdate
Tell me that's not cool! :-)
you can do this with the shell (bash,ksh,zsh + ssh + tools), or programming languages such as Python,Perl(Ruby or PHP or Java) etc, basically a language that supports SSH protocol and operating system functions. The "best" one is the one that you are more comfortable and have knowledge in. If you are doing sysadmin, the shell is the closest thing you can use. Then after you have done your script, you can use the crontab (cron) , or the at command to schedule your task. check their man page for more information
You can easily do the above using bash/Bourne etc.
However I would take the time and effort to learn Perl (or some similarly powerful scripting language). Why ?
the language constructs are much more powerful
there are no end of libraries to interface to the systems/features you want to script
because of the library support, you won't have to spawn off different commands to achieve what you want (possibly valuable on a loaded system)
you can decompose frequently-used scripts into your own libraries for later use
I choose Perl particularly because it's been designed (perhaps designed is too strong a word for Perl) for these sort of tasks. However you may want to check out Ruby/Python or other suggestions from SO contributers.
For the basic steps look at camh's answer. If you plan to run the script via cron, then implement some simple logging, e.g. by appending start time of each command with exit code to a textfile which you can later analyze for failures of the script.
Expect -- scripting interactive applications
Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc.... Expect can make easy all sorts of tasks that are prohibitively difficult with anything else. You will find that Expect is an absolutely invaluable tool - using it, you will be able to automate tasks that you've never even thought of before - and you'll be able to do this automation quickly and easily.
http://expect.nist.gov
bonus: Your tax dollars at work!
I would probably do something like this...
project_update.sh
#!/bin/bash
#
# $1 - user#host
# $2 - project directory
[[ -z $1 || -z $2 ]] && { echo "usage: $(basename $0) user#host project_dir"; exit 1; }
declare host=$1 proj_dir=$2
ssh $host "cd $proj_dir;svn update;sudo /etc/init.d/apache2 restart" && echo "Success"
Just to add another tip - you should not give users access to some application in an unknown state. svn up might break during the update, users might see a page that's half-new half-old, etc. If you're deploying the whole application at once, I'd suggest doing svn export instead to a new directory and then either mv current old ; mv new current, or even keeping current as a link to the directory you're using now. Still not perfect and not blocking every possible race condition, but it definitely takes less time than svn up on the live copy.

Resources