I need script that can download files from ftp via my sh code
I have use expect with ftp, but if I do for loop inside code, I got
wrong # args: should be "for start test next command"
while executing "for v in "a b c""
My code
/usr/bin/expect << EXCEPT_SCRIPT
set timeout 1
spawn ftp -n ${HOST}
send "user ${USER} ${PASSWD}\n"
expect "ftp>"
send "bin\n"
expect "ftp>"
send "dir\n"
for v in "${files_to_download[#]}"
do
ftp_file="${v}.bz2"
#download file
echo ${ftp_file}
#put code to dl here
done
expect "ftp>"
send "bye\n"
EXCEPT_SCRIPT
wget
expect can be tricky to work with so I'd prefer to use GNU Wget as an alternative. The following should work as long as you don’t have any spaces in any of the arguments.
for v in "${files_to_download[#]}"
do
ftp_file="${v}.bz2"
wget --user=${USER} --password=${PASSWD} ${HOST}/${ftp_file}
done
Request multiple resources using only one FTP connection
If it’s an issue, you can avoid having to make multiple FTP connections to the server by using FTP globbing (wildcard characters), e.g.
wget --user=${USER} --password=${PASSWD} "${HOST}/*.bz2"
Make sure the URL is quoted so the shell doesn’t expand the wildcard.
You can also use the -nc, --no-clobber option to avoid re-downloading files that have already been downloaded.
On the other hand, requesting multiple resources (wget url2 url2 ...) does result in multiple FTP logins.
Note: I checked both of the above operations by monitoring Port 21 on my own FTP server with tcpdump.
ncftp
You may also be interested in ncftp. While I haven’t used it in years, I found it – and its ncftpget and ncftpput utilities – much easier to use for command line scripting than the standard UNIX ftp program.
Related
I have log files, based on Linux servers, and I'm working on Windows OS.
I'm using Filezilla to log in the Linux server and searching specific text or strings by open the log file.
I want to automate this process using batch in Windows, I tried using below;
#echo off
cls
set /p string="Enter the string: "
echo open xx.xx.xx.xx 21> ftpc.dat
echo xxxxxxxx>> ftpc.dat
echo xxxxxxxx>> ftpc.dat
echo bin >> ftpc.dat
echo grep '%string%' /PATH IS HERE/log.log >> ftpc.dat
ftp -s:ftpc.dat
I'm just new to that, I want ideas on that, how I automate this search process? where I can make a search tool for any text, that this tool goes and find specific file in linux server and shows the results in lines (before/after 15 lines) of thatsearch results.
Do I need to write bash scripts, or I can do this basic script in batch file as above to show or output the results?
If you have a number of Linux servers to watch, it might be worth installing something like rsyslog or logstash. It's a big topic, but those might be good starting points in your research.
Other things to google: elasticsearch, kibana ... and their alternatives.
You cannot run grep using FTP.
So either:
Use FTP to download whole file and grep/search it locally.
Or (as you seem to have an SSH access too) use a command-line SSH client to execute grep on the server. On Windows, you can use Plink (which comes with PuTTY):
plink -pw password user#example.com grep '%string%' /remote/path/log.log >
I am trying to create a bash script with ftp.
If I use terminal and put the commands below one bye one, it works like a charm.
$ ftp 192.168.1.4 2121
Connected to 192.168.1.4.
220 SwiFTP 1.7.11 ready
Name (192.168.1.4:user):
331 Send password
Password:
230 Access granted
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd Study/Math
ftp> put ~/Documents/Math/lesson.pdf lesson.pdf
I am trying to automate this command with a bash script:
#!/bin/bash
ftp 192.168.1.4 2121
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
It is not working. I know here ftp is a independent tool and I have to put those command while the ftp program is running. I searched internet tried various techniques (like printf, expect, etc...) but it did not worked. I also tried to use some scripts from internet to automate this process, but nothing helped. I am a newbie in bash scripting and these stuffs. Can you guys help me to solve this Problem?
Thanks in advance...
First, don't do this if you have any other option. It's a fairly standard idiom, but it's really broken in a lot of ways. If you're very sure that it won't ever do anything unpredictable, and that when it does it will be ok anyway, then sure, but in general...
use something besides ftp. For example, scp works quite well and has a checkable return code that is actually useful.
use a more granular programming language with modules. Don't get me wrong, I love bash and will always use it first when I can, but pumping a stream of commands into an ftp like a fire-and-forget UDP torpedo without any easy way of checking that each worked is just bad habit. Try Perl, or Python, or any other damned thing that lets you check a return code on each command and react accordingly. :)
if you MUST use bash (and yes, I have done it), and if it's important enough to check (what isn't?), then think about how you're going to do that. Maybe you can just pull lesson.pdf back to a local testme.pdf and cmp them to make sure it's good, which seems pretty easy. For any more complex script, you might need to run the ftp as a coprocesses and feed it commands one at a time, then read back it's output and parse for return codes, because ftp generally only reports errors there...and watch out for "500 bytes sent", which isn't a 500 error.
Either way, good luck. In many ways, simple is still best.
You might be interested in what is known as a heredoc
#!/usr/bin/env bash
ftp -n 92.168.1.4 2121 <<EOF
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
EOF
Since the ftp program reads its input from /dev/stdin, you can use a heredoc to define what should be passed from /dev/stdin
Okay, after User123 gave me a link, I finally solved my problem. So, I am giving my working script.
#!/bin/sh
HOST='192.168.1.4 2121'
USER='username_of_ftp'
PASSWD='password_of_ftp'
/usr/bin/ftp -n $HOST <<END_SCRIPT
user ${USER} ${PASSWD}
cd Study/Math
put ~/Documents/Math/lesson.pdf lesson.pdf
quit
END_SCRIPT
exit 0
While the ftp approach you're working with might be OK for your purposes, it's not going to behave properly if the server doesn't respond as expected.
As has already been suggested, use something other than ftp if at all possible. Scp or rsync using key authentication will work better, be more convenient since it won't break every time you change your password, and be more secure.
I've searched a lot on the internet but haven't been able to find any useful info this yet. Does PFTP not allow you to run loops like 'IF' and 'WHILE' at all?
If it does, please let me know the syntax, I'm tried of banging my head against it. Annoyingly, PuTTY allows these commands but psftp doesn't seem to even though both are from the same family. I really hope there is a solution to this!
PSFTP isn't a language. It's just an SFTP client. SFTP itself is just a protocol for moving files between computers. If you have SFTP set up on the remote computer then it suggests that you have SSH running (since SFTP generally comes bundled with the SSH server install).
You can do a test in a bash shell script, for instance, to see if the file exists on the remote server, then execute your psftp command based on the result. Something like:
#!/bin/bash
# test if file exists on remote system
fileExists=$(ssh user#yourothercomputer "test -f /tmp/foo && echo 'true' || echo 'false'")
if $fileExists; then
psftp <whatever>
fi
You can stick that whole mess in a loop or whatevs. What's happening here is that we are sending a command test -f /tmp/foo && echo 'true' || echo 'false' to the remote computer to execute. The stdout of the command is returned and stored in the variable fileExists. Then we just test it.
If you are in windows you could convert this to a batch script and use plink.exe to send the command kind of like they do here. Or maybe just plop cygwin on your computer with an SSH and SFTP client and use what's above.
The big take-away here is that you will need a separate scripting environment to do the loop and run psftp based on a test.
I need to transfer a file from my linux server to a FTP server.
My shell script is :
#! /bin/ksh
HOST='my_ip'
USER='userid'
PASSWD='password'
FILE='file.txt'
DIREC='/eir_log'
ftp -in $HOST << EOMYF
user $USER $PASSWD
binary
mkdir $DIREC
cd $DIREC
pwd
quit
EOMYF
pretty simple code. but the problem is though I am logging in the FTP server fine, but its not allowing me to create a new directory in the FTP server. At first i thought some error with my script, but even individually if i run a mkdir in the ftp server its showing create directory failed.
Can somebody let me know the possible error, or if any eror in my code that i am missing out on.The pwd is working fine though, which means there is no problem loging in the ftp site through script.
Thanks in advance for your help
Have a look at expect
Something to get you started
#!/usr/bin/expect
set timeout 120
spawn ftp 192.168.0.210
expect "Name"
send "root\r"
expect "Password:"
send "pass\r"
expect "ftp> "
send "bye\r"
Probably lftp ( ftp scripting client ) will be something you need ( look among your distro's packages ). Error creating directories is probably related to permissions of the dir inside which you try to create it.
http://lftp.yar.ru/desc.html
Have you tried using SCP (Secure Copy)?
scp -r /dir/to/"file.txt" username#hostname
*if you're in the directory of the file/folder you wish to send then you don't need to define the complete filepath
Have a look at this article if you want to scp without having to enter your password. http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/
If you are root or have admin privileges then you shouldn't need to sudo your way into making a directory. It will be best to run remote commands without sudo just in case a malicious code piggybacks your script
is there a way to run multiple commands in HPs integrated Lights-Out 3 system via SSH? I can login to iLO and run a command line by line, but I need to create a small shell-script, to connect to iLO and to run some commands one by one.
This is the line I use, to get information about the iLO-version:
/usr/bin/ssh -i dsa_key administrator#<iLO-IP> "version"
Now, how can I do something like this?
/usr/bin/ssh -i dsa_key administrator#<iLO-IP> "version" "show /map1 license" "start /system1"
This doesn't work, because iLO thinks it's all one command. But I need something to login into iLO, run these commands and then exit from iLO. It takes too much time to run them one after the other because every login into iLO-SSH takes ~5-6 seconds (5 commands = 5*5 seconds...).
I've also tried to seperate the commands directly in iLO after manual login but there is no way to use multiple commands in one line. Seems like one command is finished by pressing return.
iLO-SSH Version is: SM-CLP Version 1.0
The following solutions did NOT work:
/usr/bin/ssh -i dsa_key administrator#<iLO-IP> "version; show /map1 license; start /system1"
/usr/bin/ssh -i dsa_key administrator#<iLO-IP> "version && show /map1 license && start /system1"
This Python module is for HP iLO Management. check it out
http://pypi.python.org/pypi/python-hpilo/
Try putting your commands in a file (named theFile in this example):
version
show /map1 license
start /system1
Then:
ssh -i dsa_key administrator#iLO-IP < theFile
Semicolons and such won't work because you're using the iLO shell on the other side, not a normal *nix shell. So above I redirect the file, with newlines intact, as if you were typing all that into the session by hand. I hope it works.
You are trying to treat iLO like it's a normal shell, but its really HP's dopy interface.
That being said, the easiest way is to put all the commands in a file and then pipe it to ssh (sending all of the newline characters):
echo -e "version\nshow /map1 license\nstart /system1" | /usr/bin/ssh -i dsa_key administrator#<iLO-IP>
That's a messy workaround, but would you might fancy using expect? Your script in expect would look something like that:
# Make an ssh connection
spawn ssh -i dsa_key administrator#<iLO-IP>
# Wait for command prompt to appear
expect "$"
# Send your first command
send "version\r"
# Wait for command prompt to appear
expect "$"
# Send your second command
send "show /map1 license\r"
# Etc...
On the bright side, it's guaranteed to work. On the darker side, it's a pretty clumsy workaround, very prone to breaking if something goes not the way it should (for example, command prompt character would appear in version output, or something like that).
I'm on the same case and wish to avoid to run a lot of plink commands. So I've seen you can add a file with the -m option but apparently it executes just one command at time :-(
plink -ssh Administrator#AddressIP -pw password -m test.txt
What's the purpose of the file ? Is there a special format for this file ?
My current text file looks like below:
set /map1/oemhp_dircfg1 oemhp_usercntxt1=CN=TEST
set /map1/oemhp_dircfg1 oemhp_usercntxt2=CN=TEST2
...
Is there a solution to execute these two commands ?
I had similar issues and ended up using the "RIBCL over HTTPS" interface to the iLO. This has advantages in that it is much more responsive than logging in/out over ssh.
Using curl or another command-line HTTP client try:
USERNAME=<YOUR_ILO_USERNAME>
PASSWORD=<YOUR_ILO_PASSWORD>
ILO_URL=https://<YOUR_ILO_IP>/ribcl
curl -k -X POST -d "<RIBCL VERSION=\"2.0\">
<LOGIN USER_LOGIN=\"${USERNAME}\" PASSWORD=\"${PASSWORD}\">
<RIB_INFO MODE="READ">
<GET_FW_VERSION/>
<GET_ALL_LICENSES/>
</RIB_INFO>
<SERVER_INFO MODE=\"write\">
<SET_HOST_POWER HOST_POWER=\"Yes\">
</SERVER_INFO>
</LOGIN>
</RIBCL>" ${ILO_URL}
The formatting isn't exactly the same, but if you have the ability to access the iLO via HTTPS instead of only ssh, this may give you some flexibility.
More details on the various RIBCL commands and options may be found at HP iLO 3 Scripting Guide (PDF).