I've searched a lot on the internet but haven't been able to find any useful info this yet. Does PFTP not allow you to run loops like 'IF' and 'WHILE' at all?
If it does, please let me know the syntax, I'm tried of banging my head against it. Annoyingly, PuTTY allows these commands but psftp doesn't seem to even though both are from the same family. I really hope there is a solution to this!
PSFTP isn't a language. It's just an SFTP client. SFTP itself is just a protocol for moving files between computers. If you have SFTP set up on the remote computer then it suggests that you have SSH running (since SFTP generally comes bundled with the SSH server install).
You can do a test in a bash shell script, for instance, to see if the file exists on the remote server, then execute your psftp command based on the result. Something like:
#!/bin/bash
# test if file exists on remote system
fileExists=$(ssh user#yourothercomputer "test -f /tmp/foo && echo 'true' || echo 'false'")
if $fileExists; then
psftp <whatever>
fi
You can stick that whole mess in a loop or whatevs. What's happening here is that we are sending a command test -f /tmp/foo && echo 'true' || echo 'false' to the remote computer to execute. The stdout of the command is returned and stored in the variable fileExists. Then we just test it.
If you are in windows you could convert this to a batch script and use plink.exe to send the command kind of like they do here. Or maybe just plop cygwin on your computer with an SSH and SFTP client and use what's above.
The big take-away here is that you will need a separate scripting environment to do the loop and run psftp based on a test.
I want to pass a password to ssh using a bash script (Yes, I know that there are ssh keys that I could use, but this is not what I intend).
I found some solutions that were using expect but since it is not a standard bash tool I am wondering if I can do this using pipes.
Can someone explain to me, why exactly something like this:
echo "password\n" | ssh somehost.com
or
ssh somehost.com <(echo "password\n")
doesn't work? Is there any possibility to make it work? Maybe executing ssh as a different process, obtaining its PID and then sending a string directly to it?
You can not specify the password from the command line but you can do either using ssh keys or using sshpass as suggested by John C. or using a expect script.
To use sshpass, you need to install it first. Then
sshpass -f <(printf '%s\n' your_password) ssh user#hostname
instead of using sshpass -p your_password. As mentioned by Charles Duffy in the comments, it is safer to supply the password from a file or from a variable instead of from command line.
BTW, a little explanation for the <(command) syntax. The shell executes the command inside the parentheses and replaces the whole thing with a file descriptor, which is connected to the command's stdout. You can find more from this answer https://unix.stackexchange.com/questions/156084/why-does-process-substitution-result-in-a-file-called-dev-fd-63-which-is-a-pipe
Since there were no exact answers to my question, I made some investigation why my code doesn't work when there are other solutions that works, and decided to post what I found to complete the subject.
As it turns out:
"ssh uses direct TTY access to make sure that the password is indeed
issued by an interactive keyboard user." sshpass manpage
which answers the question, why the pipes don't work in this case. The obvious solution was to create conditions so that ssh "thought" that it is run in the regular terminal and since it may be accomplished by simple posix functions, it is beyond what simple bash offers.
I'm trying to do an rsync backup from dreamhost to another host, here's the command i'd like to use:
nohup rsync -e "/usr/bin/ssh" --bwlimit=2000 -av username#server.dreamhost.com:remote_directory local_directory&
I'd like the process to keep running in the background and even when I disconnect. Problem is, I don't know how to put in the password when it's a background process. How do I do this?
Usually this is done by not requiring a password at all. Instead, consider configuring SSH to use a public key. There are several resources online (such as this one from dreamhost) that can help you do that.
I would use a key. If you need to protect the key with a password or you cannot use a key for whatever reason, then use expect to pass the password:
rsync_auto.sh:
#/bin/bash
expect <<<EOF
spawn nohup rsync -e "/usr/bin/ssh" --bwlimit=2000 -av username#server.dreamhost.com:remote_directory
expect "password:"
send "your_password\r"
expect eof
EOF
!!!Make sure that nobody except you can access the file!!!:
chmod 500 rsync_auto.sh
A little bit more elaborated way might be to store the password in a keyring application, like gnome-keyring instead of storing them in a plain file. I've found this article if you are interested.
I am fully aware that this question has been asked many times but I cant able to find any solution which satisy my requirement.
Task -> I need to transfer files from machine A to machineB and remotely execute scripts on Machine B. Due to my limitation I cant able to use keygen, expect utility or any other utility which requires to install packages. To Transfer the file I need to give password and I want to give password in Url. as this will run inside bash script and requires no user interference .
My investigation- I thought of using scp but realise, its not possible to give password at command prompt. So i wondering , if there is any other alternative from rsync .
below is the small attempt
#!/bin/bash
PATH=/usr/bin:/usr/sbin:/bin:/sbin
USER="bob"
RSYNC_PASSWORD="blue"
MACHINE_B="192.168.200.2"
if ping -c 1 -W 1 $MACHINE_B
then
echo "There is machine b as well"
echo " cheking to transfer file to machine b"
rsync lol.sh 192.168.200.2:/home/bob/
fi
Thanks and regards,
Sam
I have tried various option mentioned above ,, but unfortunately none of them works in my case. But I would like to thanks everyone for helping me and surely I have learned few new things specially rsync.
In my case, I have to rely on ssh keys to make it work.
From the rsync man page:
Some modules on the remote daemon may require authentication. If so, you will receive a password prompt when you connect. You can avoid the password
prompt by setting the environment variable RSYNC_PASSWORD to the password you want to use or using the --password-file option. This may be useful
when scripting rsync.
I feel silly asking this...
I am not an expert on shell scripting, but I am finally in enough of a sysadmin role that I want to do this correctly.
I have a production server that hosts a webapp. Here is my routine.
1 - ssh to server
2 - cd django_src/django_apps/team_proj
3 - svn update
4 - sudo /etc/init.d/apache2 restart
5 - logout
I want to create a shell script for steps 2,3,4.
I can do this, but it will be a very plain and simple bash script simply containing the actual commands I type at the command line.
My question: What is the best way to script this kind of repetitive procedure in bash (Linux, Ubuntu) for a remote server?
Thanks!
The best way is simply as you suggest. Some things you should do for your script would be:
put set -e at the top of the script (after the shebang). This will cause your script to stop if any of the commands fail. So if it cannot cd to the directory, it will not run svn update or restart apache. You can do this programmatically by putting || exit 0 after each command, but if that's all you're doing, you may as well use set -e
Use full paths in your script. Do not assume the directory that the script is run from. In this specific case, the cd command has a relative path. Use a full (absolute) path, or use an environment variable like $HOME.
You may want to set up sudo so that it can run the command without asking for a password. This makes your script non-interactive which means it can be run in the background and from cron jobs and such.
As time goes by, you may add features and take command line arguments to parameterise the script. But don't bother doing this up front. Just evolve your scripts as you need.
There is nothing wrong with a simple bash script simply containing the actual commands you type at the command line. Don't make it more complicated than necessary.
I'd setup a cron job doing that automatically.
Since you're using python, check out fabric - you can use it to automate these kind of tasks. First install fabric:
$ sudo easy_install fabric
then write your fabric script:
from __future__ import with_statement
from fabric.api import *
def svnupdate():
with cd('django_src/django_apps/team_proj'):
run('svn update')
sudo('/etc/init.d/apache2 restart')
Save as fabfile.py, then run using the fab command:
$ fab -H hostname svnupdate
Tell me that's not cool! :-)
you can do this with the shell (bash,ksh,zsh + ssh + tools), or programming languages such as Python,Perl(Ruby or PHP or Java) etc, basically a language that supports SSH protocol and operating system functions. The "best" one is the one that you are more comfortable and have knowledge in. If you are doing sysadmin, the shell is the closest thing you can use. Then after you have done your script, you can use the crontab (cron) , or the at command to schedule your task. check their man page for more information
You can easily do the above using bash/Bourne etc.
However I would take the time and effort to learn Perl (or some similarly powerful scripting language). Why ?
the language constructs are much more powerful
there are no end of libraries to interface to the systems/features you want to script
because of the library support, you won't have to spawn off different commands to achieve what you want (possibly valuable on a loaded system)
you can decompose frequently-used scripts into your own libraries for later use
I choose Perl particularly because it's been designed (perhaps designed is too strong a word for Perl) for these sort of tasks. However you may want to check out Ruby/Python or other suggestions from SO contributers.
For the basic steps look at camh's answer. If you plan to run the script via cron, then implement some simple logging, e.g. by appending start time of each command with exit code to a textfile which you can later analyze for failures of the script.
Expect -- scripting interactive applications
Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc.... Expect can make easy all sorts of tasks that are prohibitively difficult with anything else. You will find that Expect is an absolutely invaluable tool - using it, you will be able to automate tasks that you've never even thought of before - and you'll be able to do this automation quickly and easily.
http://expect.nist.gov
bonus: Your tax dollars at work!
I would probably do something like this...
project_update.sh
#!/bin/bash
#
# $1 - user#host
# $2 - project directory
[[ -z $1 || -z $2 ]] && { echo "usage: $(basename $0) user#host project_dir"; exit 1; }
declare host=$1 proj_dir=$2
ssh $host "cd $proj_dir;svn update;sudo /etc/init.d/apache2 restart" && echo "Success"
Just to add another tip - you should not give users access to some application in an unknown state. svn up might break during the update, users might see a page that's half-new half-old, etc. If you're deploying the whole application at once, I'd suggest doing svn export instead to a new directory and then either mv current old ; mv new current, or even keeping current as a link to the directory you're using now. Still not perfect and not blocking every possible race condition, but it definitely takes less time than svn up on the live copy.