I have a FTP Server that uses Cron to automate tasks and I would like to use it to access another ftp server, get a file that starts with 26 and have an extension .csv, transfer to my FTP I am running the cron and delete the file on the origin FTP server, every friday of the week. Can somebody help me with the script code?
What I have right now is this:
#!/bin/bash -x
filename="dir/*.csv"
hostname="files.test"
username="testuser"
password="testpassword"
ftp -in $hostname <<EOF
quote USER $username
quote PASS $password
binary
get $filename
quit
EOF
Please, help
#!/bin/bash
USER=user
PASS=password
URL=myIP
PLACE=tmp
#
ftp -v -n > /tmp/xftpb.log <<EOF
open $URL
user $USER $PASS
binary
cd $PLACE
mget 26*.csv
mdel 26*.csv
quit
EOF
To run every friday, 8:00h, use at crontab:
0 8 * * 5 /path/mybash.sh
Related
I am trying to write shell script that implements FTP command reading here documents.
How do I redirect ftp command line AND results FTP server returns?
My current script is below
#!/bin/bash
ftp -ivn <<EOF |& tee ftplog.text
open <myFtpserver>
user <username> <password>
pwd
ls -l MYFILE_*
bye
EOF
This code outputs ftp server's response but ftp command to ftplog.txt.
How can I redirect both?
I appreciate your help
You should create your script on this way:
#!/bin/bash
ftp -ivn >ftplog.text <<EOF
open <myFtpserver>
user <username> <password>
pwd
ls -l MYFILE_*
bye
EOF
The redirection is to the command, not inline block
I'm using the following command
mysqldump -hHOST -uUSERNAME -pPASSWORD DATABASE > file.sql
Would like a separate job created each time when cron job is executed and not overwrite existing file.
i'm using shell scrip for solve this problem on my server. This is the code if you wan to try. Create file .sh, Ex: backupdb.sh
#!/bin/bash
NOW=$(date +"%m%d%Y")
FILE="backupdb.$NOW.tgz"
SQL="backupdb.$NOW.sql"
DBNAME='live' #name of database
DUMP='/usr/bin/mysqldump'
USER='root' #user
HOST=10.0.0.8 #Ip database
$DUMP -h $HOST -u $USER -pYOURPASSWORD --routines --events $DBNAME > /home/backupdb/$SQL
gzip -9 /home/backupdb/$SQL
echo "BACKUP DATABASES FROM MYSQL MASTER SUCCESS $SQL"|mail -s "BACKUP DATABASES" your#email.com #you can chang this email with your personal email
and save.
Don't forget to change permission,
chmod +x backupdb.sh
Add this code in cron job
00 04 * * * sh /home/backupdb/backupdb.sh
Save
That's it.
Sorry if my English is not good
I have a bash script that creates backups incrementally(daily) and full(on Mondays). Every 7 days the script combines the week of backups(full and Incremental) and sends them off to an FTP server, the problem i am having is i want to delete the files from my backup directory after the FTP upload is finished, but i cant do that until i know the file was successfully uploaded. I need to figure out how to capture the '226 transfer complete' so i can use that in an 'IF' statement to delete the backup files. Any help is greatly appreciated. also he is my FTP portion of the script
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv $HOST << EOF
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
I could use whatever mean i needed i suppose, FTP was something already setup for another backup function for something else, thanks
2nd EDIT Ahmed the rsync works great in test from command line, its a lot faster than FTP, the server is on the local network so SSH not that big of a deal but nice to have for added security, i will finish implementing in my script tomorrow, thanks again
FTP OPTION
The simple solution would be to do something like this:
ftp -inv $HOST >ftp-results-$YEAR-$MONTH-$DAY.out 2>&1 <<-EOF
user $USER $PASS
cd /baks
bin
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
Also there is an issue with your here-document syntax; there is no space between << and the delimiter word (in your case EOF) and I added a - because you are putting white-spaces before the ACTUAL delimeter (it's tabbed in for the if / fi block) so the [-] is required
Now when you do this; you can parse the output file to look for the successful put of the file. For example:
if grep -q '226 transfer complete' ftp-results-$YEAR-$MONTH-$DAY.out; then
echo "It seems that FTP transfer completed fine, we can schedule a delete"
echo "rm -f $PWD/weekending-$YEAR-$MONTH-$DAY.tar.gz" >> scheduled_cleanup.sh
fi
and just run scheduled_cleanup.sh using cron at a given time; this way you will have some margin before the files are cleaned up
If your remote FTP server has good SITE or PROXY options you may be able to get the remote FTP to run a checksum on the uploaded file after successful upload and return the result.
SCP / RSYNC OPTION
Using FTP is clunky and dangerous, you should really try and see if you can have scp or ssh access to the remote system.
If you can then generate an ssh key if you don't have one using ssh-keygen:
ssh-keygen -N "" -t rsa -f ftp-rsa
put the ftp-rsa.pub file into the $HOST/home/$USER/.ssh/authorized_keys and you have a much nicer method for uploading files:
if scp -B -C weekending-$YEAR-$MONTH-$DAY.tar.gz $USER#$HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
Or better yet using rsync:
if rsync --progress -a weekending-$YEAR-$MONTH-$DAY.tar.gz $HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
et voilà you are done since rsync works over ssh you are happy and secure
Try the next
#!/bin/bash
runifok() { echo "will run this when the transfer is OK"; }
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv <<EOF | grep -q '226 transfer complete' && runifok
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
test it and when will run ok - replace the echo in the runifok function for your commands what want execute after the upload is succesful.
I am trying to move txt/ log file from server1 to server2.
I am trying to connect to a server from server1 to server2 using SFTP but some how it is asking for password in the prompt.Can anyone let me know how to give password as input through script and execute this functionality using a Script.
Please let me know asap......
MY CODE:
test.sh is script and 1.txt file has password details.....
code: test.sh
sftp mwctrl#sacsun11 < 1.txt <> out.log 2>&1
cd /usr/ftadapters/logs/adapters/rivaadp
lcd /export/home/eisape
put *.txt
exit
EOF
1.txt:
password m33tzn3
Actually you need add ssh keys to remote machine. Check article below:
Using sftp without password (http://says-story.blogspot.nl/2008/01/using-ssh-scp-sftp-without-password.html)
Setting up ssh keys is relatively simple. Takes <1 min following the instructions in the link posted above by fxzuz.
But generally passing passwords as parameters / storing in config files is considered a security risk.
However, if you still want to go ahead, here is a link -- http://nixcraft.com/shell-scripting/4489-ssh-passing-unix-login-passwords-through-shell-scripts.html
try using this
/usr/bin/expect <<EOF
spawn sftp -oStrictHostKeyChecking=no -oCheckHostIP=no mwctrl#sacsun11 \
"cd /usr/ftadapters/logs/adapters/rivaadp \
lcd /export/home/eisape \
put *.txt \"
expect "*?assword:*"
send "m33tzn3"
send "\r"
set timeout -1
send "\r"
expect EOF
Lets assume the script i want to write ssh to 1.2.3.4 and then invokes
ls.
The problem is that when the line "ssh 1.2.3.4" is invoked, a password is
Required, hence, the flow is stopped, even when i fill the password,
The script wont continue.
How can i make the script continue after the password is given?
Thx!
You want to do public key authentication. Here are some resources which should get you going.
http://magicmonster.com/kb/net/ssh/auto_login.html
http://www.cs.rpi.edu/research/groups/vision/doc/auto/ssh/ssh_public_key_authentication.html
I would post a couple more links, but I don't have enough reputation points. ;) Just google on "SSH automated login" or "SSH public key authentication" if you need more help.
Actually you're trying to run ls locally but you have an ssh session opened. So it won't run ls until the session is opened. If you want to run ls remotely, you should use
ssh username#host COMMAND
Where command is the command you want to run. Ssh session will finish as soon as the command is invoked and you can capture its output normally.
I would suggest you to use RSA authentication method for script that needs ssh.
I just tried this script:
#!/bin/sh
ssh vps1 ls
mkdir temp
cd temp
echo test > file.txt
And it works. I can connect to my server and list my home. Then, locally, it creates temp dir, cd into it and then creates file.txt with 'test' inside.
write simple login bash script named login_to and give exec permissions (chmod 744 login_to)
#!/bin/bash
if [ $1 = 'srv1' ]; then
echo 'srv1-pass' | pbcopy
ssh root#11.11.11.11
fi
if [ $1 = 'foo' ]; then
echo 'barbaz' | pbcopy
ssh -t dux#22.22.22.22 'cd ~/somedir/someotherdir; bash'
fi
now use it like this
login_to srv1
login_to foo
When asked for password, just pate (ctrl+v or command+v) and you will be logged in.