Looping inside ftp command - linux

I'm wondering if I can in some case add 0-10 files, lets say I have a shell script with video1.jpg, and want it to go up to video10.jpgand start over. The script runs and give video1.jpg, I want it to keep adding +1 up to 10.
#!/bin/bash
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
get video1.jpg
bye
EOT

you can try this
#!/bin/bash
for((i=1;i<=10;i++))
do
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
get video${i}.jpg
bye
EOT
done

I have no chance to try it right now, but this should do:
#!/bin/bash
for i in {1..10}; do
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
get video$i.jpg
bye
EOT
done
EDIT #2: I was wrong, my answer is almost identical to Shravan Yadav's as you can't loop inside an ftp transaction. You can pick either one as the correct answer, but you should prefer his as he came first I guess :)

As other answers have already pointed out, the here document you pass to the ftp program is just text, and does not have a facility for looping. However, generating ten ftp sessions to get ten files is wasteful -- you can generate a single here document which instructs ftp to fetch them all in one session.
#!/bin/bash
(
printf '%s\n' ascii "user $USER $PASSWD" prompt
printf 'get video%i.jpg' {1..10}
printf 'bye\n'
) | ftp -n -v "$HOST"
There will be no facility for proper error handling, though. The real solution is to use a properly scriptable ftp program (lftp, ncftp, what have you) or write your own simple client.

Here is an approach to add ftp inside the loop
#!/bin/bash
HOST='hostname'
USER='username'
PASSWD='password'
# Local directory where the files are stored.
cd "/local/directory/from where to upload files/"
# To get all the files added today only.
TODAYSFILES=`find -maxdepth 1 -type f -mtime -1`
# remote server directory to upload backup
REMOTEDIR="/directory on remote ftp computer/"
for FILENAME in ${TODAYSFILES[#]}; do
ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
cd $REMOTEDIR
put $FILENAME
bye
EOT
done

Related

How do I get the files from SFTP server and move them to another folder in bash script?

How do I get the one by one files from SFTP server and move them do another folder in Ubuntu bash script?
#!bin/sh
FOLDER=/home/SFTP/Folder1/
sftp SFTP#ip_address
cd /home/FSTP/Folder1/
for file in "$FOLDER"*
<<EOF
cd /home/local/Folder1
get $file
EOF
mv $file /home/SFTP/Done
done
I know it's not right, but i've tried my best and if anyone can help me, i will appreciate it. Thanks in advance.
OpenSSH sftp is not very powerful client for such tasks. You would have to run it twice. First to collect list of files, use the list to generate list of commands, and execute those in a second run.
Something like this:
# Collect list of files
files=`sftp -b - user#example.com <<EOF
cd /source/folder
ls
EOF`
files=`echo $files|sed "s/.*sftp> ls//"`
# Use the list to generate list of commands for the second run
(
echo cd /source/folder
for file in $files; do
echo get $file
echo rename $file /backup/folder/$file
done
) | sftp -b - user#example.com
Before you run the script on production files, I suggest, you first output the generated command list to a file to check, if the results are as expected.
Just replace the last line with:
) > commands.txt
Maybe use SFTP internal command.
sftp get -r $remote_path $local_path
OR with the -f option to flush files to disk
sftp get -rf $remote_path $local_path

How to download files from my list with wget and ftp

I need to download only defined files with wget and ftp.
For example:
1.I retrieve all files using:
echo ls -R | ftp ftp://user:password#host > ./list.txt
2.Then I will parse the result and get a list with absolute paths for each file:
/path/to-the/file-1
/path/to-the/file-2
etc.
3.And now I need to download all files from the result list using wget and ftp.
And I don't want to create a separate FTP session for each file download process.
Please give your advice. Thank you.
Update:
For recursive download I'm using it: wget -r ftp://user:password#host:/ -nH -P /download/path. It works great, but I need to pass a file with a list of remote files for downloading via FTP with one FTP session.
Sorry, I missed the "single session" part when I commented. I think you need to have your script generate a second script to run a single FTP session.
So, your script will not do any FTP itself, it will just write another script that does the transfers. So, it will write a script that does this
ftp -n <SOMEADDRESS> <<EOS
quote USER <USERNAME>
quote PASS <PASSWORD>
bin
get file1 localname1
get file2 localname2
...
get fileN localnameN
quit
EOS
Then it will execute that script, by doing:
bash < thatScript
So your script will look like this:
#!/bin/bash
ScriptName=funkyFTPer
cat - <<END > $ScriptName
ftp -n 192.168.0.1 <<EOS
quote USER freddy
quote PASS frog
END
# Your selection code goes here ***PHNQZ***
echo get file1 localname1 >> $ScriptName
echo get file2 localname2 >> $ScriptName
echo get fileN localnameN >> $ScriptName
echo quit >> $ScriptName
echo EOS >> $ScriptName
echo "Now run bash < $ScriptName"
Then delete the script as it contains your password. Or you can put the password in your .netrc file.
As regards creating directories locally, you can do that in the first script using mkdir -p. The -p has the advantage that it creates all directories in between in one go and doesn't get upset if they already exist.
So, just looking at the area of code where it says ***PHNQZ*** above, let's say your code decides you need file freddy/frog/c.txt, you could do:
remotename="freddy/frog/c.txt"
localdir=${remotename%/*} # Get just directory part using "bash Parameter Substitution"
mkdir -p "$localdir" # make directory and all parts in between

Get output of FOR with EOF in bash

I've created a bash script to temporarily help me send some files to a FTP server based on the id of the commit, i get the last commit, track the files and send as listed below.
#!/bin/bash
commit_hash=$(git log --format="%H" -n 1)
[[ -z "$1" ]] || commit_hash=$1
files=$(git diff-tree --no-commit-id --name-only -r $commit_hash)
echo -e $(git log -1 $commit_hash --pretty=format:"%h - %an, %ar : %s");
printf "\n"
HOST=
USER=
PASS=
for file in $files; do
ftp -nv $HOST << EOF
user $USER $PASS
cd /www/example
passive
put $file
bye
EOF
done;
of course it isn't the best approach to do that, but i automated some things that i am currently working on.
it is possible to catch the ftp output of the heredoc and apply some filters? with pipelines for example, i only want to know if the transfer was completed successfully.
it is possible to catch the ftp output of the heredoc and apply some filters? with pipelines for example, i only want to know if the transfer was completed successfully.
I presume you mean you want to catch the output of the ftp command whose input is redirected from the heredoc; the heredoc itself does not have output in a sense that anything other than the associated command can see.
But you can redirect the command's output. The thing to remember is that the heredoc begins on the next line, not immediately after the associated redirection operator. Thus, you can add a pipeline to another command after the heredoc operator. For example:
$ cat << EOF | grep flag
flag this line
not this line
or this line
flag this
last flag
EOF
Output:
flag this line
flag this
last flag
Do not use a for loop for this. See Bash FAQ 001.
commit_hash=${1:-$(git log --format="%H" -n 1)}
while IFS= read -r file; do
ftp -nv "$HOST" << EOF
user $USER $PASS
cd /www/example
passive
put $file
bye
EOF
done < <(git diff-tree --no-commit-id --name-only -r "$commit_hash")

FTP File upload - script STUCK

I have a basic bash script that I'm using to copy a file and upload it to my FTP:
cp -i /var/mobile/file.db /var
cd /var
HOST=MYFTPHOST
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd websitefolder
put sms.db
bye
EOF
rm -f file.db
When I run the script, it saves the file to my FTP perfectly. But I'm running the script from different computer's so somehow, I'd like the script to upload the file.db to my FTP like this everytime it uploads it:
file1.db
file2.db
file3.db
file4.db
Your question is a little unclear, but if I understand correctly, you're trying to name the database files in sequential order without overwriting any old files. You'll have to get the list of files from the FTP server in order to find out what files have already been uploaded.
This code will get the list of files from the server that begin with "file" and end with ".db", count them, then change the name of your "file.db" to "fileXX.db", where "XX" is the next number in the naming sequence (i.e. file1.db, file2.db, file3.db, etc).
I'm not sure where "sms.db" came from. I've changed it to "file.db" in the script.
cp -i /var/mobile/file.db /var
cd /var
HOST=MYFTPHOST
USER=USERNAME
PASS=PASSWORD
ftp -inv $HOST << EOF
user $USER $PASS
cd websitefolder
LIST=$(ls | grep file*.db)
bye
EOF
FILECOUNT=0
for FILE in $LIST
do
if [ -f $FILE ];
then
FILECOUNT+=1
done
FILECOUNT+=1
NEXTDB="file$FILECOUNT.db"
mv file.db $NEXTDB
ftp -inv $HOST << EOF
put $NEXTDB
bye
EOF

getting user input while SSH'ed into another box

I have a bash script that basically should work like below:
get build number from user and put it in buildNum var
prep the build on local machine by calling a local script with buildNum as it's argument
sftp the prepped zip file to remote server1
Do this:
ssh -v $server1 <<EOF
rm -rvf path1-on-remote-server1/*
cd path2-on-remote-server1
unzip ../prepped-zip-file-$buildNum.zip
exit
EOF
sftp the prepped zip file to remote server2
The problem i am having is that on the forth step of number 4, $buildNum is not known to the remote server and it fails.
I tried the following two solutions and both failed:
use double quotes "unzip ../prepped-zip-file-$buildNum.zip" which resulted in "unzip ../prepped-zip-file-11.6.zip: Command not found.
tried to get the build number again from the user during the SSH session which failed again by not even waiting for my input and looking for a zip file without the build number at the end of the name, as the var was empty,
i did :
ssh -v $server1 <<EOF
rm -rvf path1-on-remote-server1/*
cd path2-on-remote-server1
echo "enter build num once more: "
read bNum
unzip ../prepped-zip-file-$bNum.zip
exit
EOF
Any suggestions on how to achieve what i am after.
Thanks in advance
Are you sure this doesn't work?
ssh -v $server1 <<EOF
rm -rvf path1-on-remote-server1/*
cd path2-on-remote-server1
unzip ../prepped-zip-file-$buildNum.zip
exit
EOF
When I try it on my machine, with cat instead of ssh -v $server1 for testing, the variable does get substituted into the here-document, just as if the entire document had been on a command line. The remove shell never needs to know there was a variable in the first place.
Actually, though, you may want to give the remote command on the ssh command line rather than redirecting the standard input. This would be more robust in case some of the parts of it unexpectedly decide to try reading from stdin:
ssh -v $server1 "rm -rvf path1-on-remote-server1/*
cd path2-on-remote-server1
unzip ../prepped-zip-file-$buildNum.zip"
(Note that multi-line double-quoted strings are okay with bash).
#!/bin/sh
printf "Enter Build Number: "
read BUILD_NUM
cat << EOF | ssh $server1
hostname
echo "${BUILD_NUM}"
uptime
EOF
That works for me.

Resources