I run an automated backup shell script, it works great, but for some reason the FTP blocks me for a few minutes. I would like to add a retry and wait feature. below is sample of my code.
echo "Moving to external server"
cd /root/backup/
/usr/bin/ftp -n -i $FTP_SERVER <<END_SCRIPT
user $FTP_USERNAME $FTP_PASSWORD
mput $FILE
bye
END_SCRIPT
after a failed login i get the message below
Authentication failed. Blocked.
Login failed.
Incorrect sequence of commands: PASS required after USER
i need to capture such output and make the code atempt to sleep for few minutes before trying again.
ideas?
If it's possible for you to install additional programs onto the system of interest i encourage you to take a look at lftp.
With lftp it is possible to set paramters like the time between reconnects etc. manually.
To achieve your aim with lftp you have to invoke the following
lftp -u user,password ${FTP_SERVER} <<END
set ftp:retry-530 "Authentication failed"
set net:reconnect-interval-base 60
set net:reconnect-interval-multiplier 10
set net:max-retries 10
<some more custom commands>
END
If the pattern after ftp:retry-530 matches the 530 reply of the server lftp tries to reconnect every 60*10 seconds.
The message below is probably going to stderr instead of stdout so you will need to capture the stderr output first:
while true
do
if ( script 2>&1 |grep -q 'Authentication failed' )
then
echo "authentication failed, sleeping for a while before trying again"
sleep 60
else
#everything worked, break out of the while loop
break
fi
done
Related
Im trying to send a notification upon login via PAM, but i cant figure out how to send it to the user that is logging in.
I'm configuring my PAM to execute a script every time a user logs in. The problem is i need to send a notification if there have been any login attempts (its part of a bigger security thing im trying to add, where my laptop takes a picture with the webcam upon failed logins, and notifies me when i log in again, since my classmates like to try and guess my password for some reason).
The problem is that the line in my .sh file, which sends a user notification, sends it to root since thats the 'user' that executes the script, i want my script to send the notification to my current user (called "andreas"), but im having problems figuring this out.
Here is the line i added to the end of the PAM file system-login:
auth [default=ignore] pam_exec.so /etc/lockCam/call.sh
And here is the call.sh file:
#!/bin/sh
/etc/lockCam/notifier.sh &
The reason im calling another file is because i want it to run in the background WHILE the login process continues, that way the process doesnt slow down logging in.
Here is the script that is then executed:
#!/bin/sh
#sleep 10s
echo -e "foo" > "/etc/lockCam/test"
#This line is simply to make sure that i know that my script was executed
newLogins=`sed -n '3 p' /etc/lockCam/lockdata`
if [ $newLogins -gt 0 ]
then
su andreas -c ' notify-send --urgency=critical --expire-time=6000 "Someone tried to log in!" "$newLogins new lockCam images!" && exit'
callsInRow=`sed -n '2 p' /etc/lockCam/lockdata`
crntS=$(date "+%S")
crntS=${crntS#0}
crntM=$(date "+%M")
crntM=${crntM#0}
crntH=$(date "+%H")
crntH=${crntH#0}
((crntTime = $crntH \* 60 \* 60 + $crntM \* 60 + $crntS ))
#This whole process is absolutely stupid but i cant figure out a better way to make sure none of the integers are called "01" or something like that, which would trigger an error
echo -e "$crntTime\n$callsInRow\n0" > "/etc/lockCam/lockdata"
fi
exit 0
And this is where i THINK my error is, the line "su andreas -c...." is most likely formatted wrong or im doing something else wrong, everythin is executed upon login EXCEPT the notification doesnt show up. If i execute the script from a terminal when im already logged in there is no notification either, unless i remove the "su andreas -c" part and simply do "notify-send...", but that doesnt send out a notification when i log in, and i think thats because the notification is sent to the root user, and not "andreas".
I think your su needs to be passed the desktop users DBUS session bus address. The bus address can be easily obtained and used for X11 user sessions, but Wayland has tighter security, for Wayland the user session actually has to run up proxy to receive the messages. (Had you considered it might be easier to send an email?)
I have notify-desktop gist on github that works for X11 and should also work on Wayland (provided the proxy is running). For completeness I've appended the source code of the script to this post, it's extensively commented, I think it contains the pieces necessary to get you own code working.
#!/bin/bash
# Provides a way for a root process to perform a notify send for each
# of the local desktop users on this machine.
#
# Intended for use by cron and timer jobs. Arguments are passed straight
# to notify send. Falls back to using wall. Care must be taken to
# avoid using this script in any potential fast loops.
#
# X11 users should already have a dbus address socket at /run/user/<userid>/bus
# and this script should work without requiring any initialisation. Should
# this not be the case, X11 users could initilise a proxy as per the wayland
# instructions below.
#
# Due to stricter security requirments Wayland lacks an dbus socket
# accessable to root. Wayland users will need to run a proxy to
# provide root with the necessary socket. Each user can must add
# the following to a Wayland session startup script:
#
# notify-desktop --create-dbus-proxy
#
# That will start xdg-dbus-proxy process and make a socket available under:
# /run/user/<userid>/proxy_dbus_<desktop_sessionid>
#
# Once there is a listening socket, any root script or job can pass
# messages using the syntax of notify-send (man notify-send).
#
# Example messages
# notify-desktop -a Daily-backup -t 0 -i dialog-information.png "Backup completed without error"
# notify-desktop -a Remote-rsync -t 6000 -i dialog-warning.png "Remote host not currently on the network"
# notify-desktop -a Daily-backup -t 0 -i dialog-error.png "Error running backup, please consult journalctl"
# notify-desktop -a OS-Upgrade -t 0 -i dialog-warning.png "Update in progress, do not shutdown until further completion notice."
#
# Warnings:
# 1) There has only been limited testing on wayland
# 2) There has only been no testing for multiple GUI sessions on one desktop
#
if [ $1 == "--create-dbus-proxy" ]
then
if [ -n "$DBUS_SESSION_BUS_ADDRESS" ]
then
sessionid=$(cat /proc/self/sessionid)
xdg-dbus-proxy $DBUS_SESSION_BUS_ADDRESS /run/user/$(id -u)/proxy_dbus_$sessionid &
exit 0
else
echo "ERROR: no value for DBUS_SESSION_BUS_ADDRESS environment variable - not a wayland/X11 session?"
exit 1
fi
fi
function find_desktop_session {
for sessionid in $(loginctl list-sessions --no-legend | awk '{ print $1 }')
do
loginctl show-session -p Id -p Name -p User -p State -p Type -p Remote -p Display $sessionid |
awk -F= '
/[A-Za-z]+/ { val[$1] = $2; }
END {
if (val["Remote"] == "no" &&
val["State"] == "active" &&
(val["Type"] == "x11" || val["Type"] == "wayland")) {
print val["Name"], val["User"], val["Id"];
}
}'
done
}
count=0
while read -r -a desktop_info
do
if [ ${#desktop_info[#]} -eq 3 ]
then
desktop_user=${desktop_info[0]}
desktop_id=${desktop_info[1]}
desktop_sessionid=${desktop_info[2]}
proxy_bus_socket="/run/user/$desktop_id/proxy_dbus_$desktop_sessionid"
if [ -S $proxy_bus_socket ]
then
bus_address="$proxy_bus_socket"
else
bus_address="/run/user/$desktop_id/bus"
fi
sudo -u $desktop_user DBUS_SESSION_BUS_ADDRESS="unix:path=$bus_address" notify-send "$#"
count=$[count + 1]
fi
done <<<$(find_desktop_session)
# If no one has been notified fall back to wall
if [ $count -eq 0 ]
then
echo "$#" | wall
fi
# Don't want this to cause a job to stop
exit 0
So, I have a script which it's intended purpose is to:
Ask for the DC number and Time clock number
log in to the Time clock server for the DC stated above
After log in, it is intended to run a seperate script inside my script which updates the time clock number also stated above.
My issue is that once I trigger the script, it logs into the server as intended, prompts me for my user ID, and then I have to press "enter" when "xterm" comes up after that. After this, the update script is supposed to run, however, it doesn't, and sits at the command line.
After I exit the server, THEN it runs the update script, but fails, because the update script doesn't exist in the jump box.
My question is, after the script logs in to the server, how can I get it to trigger the script inside the Time clock server, as I am wanting it to? Thanks.
Script is below:
#!/bin/bash -x
export LANG="C"
####
####
## This script is intended to speed up the process to setup timeclocks from DC tickets
## Created by Blake Smreker | b0s00dg | bsmreker#walmart.com
####
####
#Asks for DC number
echo "What is the four digit DC number?"
read DC #User input
#Asks for Timeclock number
echo "What is the two digit Timeclock number?"
read TMC #User input
#Defines naming convention of tna server
tnaserver="cs-tna.s0${DC}.us.wal-mart.com"
#creating variable to define the update script
tcupd="/u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED"
#Logging in to the cs-tna package at the specified DC
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
$tcupd #Runs update script
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
You must run the command inside ssh session, not after it:
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
###### $tcupd #Runs update script
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver /bin/bash -c /u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
From man ssh you see ssh [-46AaCfGgKkMNnqsTtVvXxYy] ....... destination [command]. If [command] is not given ssh runs remote login command scripts, for example xterm. You read more here or here or just browse google.
You need to think how and which environment variable you want to pass to the remote machine and remember about properly enclosing the variables, so they get expanded on your or the remote machine.
I am having some issues trying to connect through telnet to a mail server.The main problem that I am having is that I need to create a script that logs me to the destination and I can't find a way to echo the password.
What I tried:
telnet host -l username ; echo 'password'
And still it asks for my password.Is there any way to fix this or I am doing something wrong?
First of all, you can use eval:
eval "{ echo user_name; sleep 1; echo pass; sleep 1; echo '?'; sleep 5; }" | telnet host_address
Make sure to replace user_name, pass, ? which is the command you want to run and host_address where your telnet host is listening; for me it is a local IP.
It’s surprisingly easy to script a set of command and pipe them into the telnet application. All you need to do is something like this:
(echo commandname;echo anothercommand) | telnet host_address
The only problem is the nagging login that you have to get through… it doesn’t show up right away. So if you pipe in an “echo admin” and then “echo password,” it will happen too quickly and won’t be sent to the server. The solution? Use the sleep command!
Adding in a couple of sleep 3 commands, to wait three seconds, solves the problem. First we’ll echo the username and password, and then we’ll echo the reboot command, and each time we’ll wait three seconds between. The final command will reboot the server immediately:
(sleep 3;echo admin;sleep 3;echo mypassword;sleep 3;echo system reboot;sleep 3;) | telnet host_address
You can put this into a shell script and run it whenever you want. Or you can add it to your cron like this (on OS X or Linux):
crontab -e
Add this line somewhere:
1 7 * * * (sleep 3;echo admin;sleep 3;echo mypassword;sleep 3;echo system reboot;sleep 3;) | telnet host_address
This will reboot your router at 7:01 AM each morning.
AFAIK, you won't be able to automate telnet that way. But it is still possible - even if it is a very bad idea (I'll elaborate on that later).
First why does your try fail :
you launched a telnet command reading from stdin (I suppose terminal) and writing to stdout and stderr (I suppose also a terminal)
if your telnet is reasonably recent, it tries to protect your authentication and asks your password from /dev/tty (for security reasons)
when that command has ended you write password on your own terminal
What should you do instead :
launch telnet with automatic authentication disable (on mine it is telnet -X SRA)
feed its input with the commands you want to pass
wait some delay before entering input, at least for login and password, because if you don't telnet clear input before reading and discards your inputs
Here is an example that allowed me to telnet to my own machine :
sh << EOF | telnet -X SRA localhost
sleep 2
echo my_user_name
sleep 1
echo my_password
# sleep 1 # looks like it can be removed
echo echo foo and bar
sleep 1
EOF
It correctly logs me into my box, executes echo foo and bar (essential command :-) ) and disconnects
Now why you should never do that :
you write a password in clear text in a script file which is poor security practice
you use telnet to do batch processing when it is not intended to be used that way : the script may not be portable to another telnet version
If you really want to pass command in a batch way to a remote server, you should instead try to use ssh which :
has options to process authentication securely (no password in script, nothing in clear text)
is intended to be used in batch mode as well as interactively
If you cannot use ssh (some sysadmin do not like to have uncontrolled input ssh connections) you could also try to use rsh. It is older, far less secure, but at least was designed for batch usage.
Thanks to Harvix answer, I got knew that there is also expect alternative native for shell, called sexpect. Get it from here. Then create this script (I call it telnetpass):
#!/bin/bash
# This script is used for automatically pass authentication by username and password in telnet prompt
# Its goal is similar as sshpass, but for telnet, so I call it telnetpass
. ~/.private/cisco_pw # should contain PASSWORD variable
export SEXPECT_SOCKFILE=/tmp/sexpect-telnetpass-$$.sock
sexpect spawn telnet $1
sexpect expect -cstring 'Username:'
sexpect send -enter $USER
sexpect expect -cstring 'Password:'
sexpect send -enter $PASSWORD
sexpect interact
Then you can run: telnetpass Host125 and got pass the authentication automatically
Trying 198.51.100.78 ...
Connected to Host125.
Escape character is '^]'.
User Access Verification
Username: ashark
Password:
host-125>
I like this solution more than using sleep commands as suggested in another answers, because sleep solutions sometimes fail.
Have you tried using the expect command ?? You will have to create a script where you identify the 'expected' response from the server e.g. 'Password:' and then supply the password in the script. The following will explain: https://en.wikipedia.org/wiki/Expect - A good example is also shown here: http://en.kioskea.net/faq/4736-shell-script-for-telnet-and-run-commands
Try eval:
eval "{ echo;
sleep 3;
echo $user;
sleep 1;
echo $pass;
sleep 1;
echo '?';
sleep 1; }" | telnet your_host
In this example, my remote command is '?' (help).
The sleeps (maybe not all of them nor these times; trial-error...) are needed to avoid telnet misses some inputs.
The user and password are passed as variables ($user and $pass). Take into account security recommendations to store the password if you are scripting.
I have a bash script that creates backups incrementally(daily) and full(on Mondays). Every 7 days the script combines the week of backups(full and Incremental) and sends them off to an FTP server, the problem i am having is i want to delete the files from my backup directory after the FTP upload is finished, but i cant do that until i know the file was successfully uploaded. I need to figure out how to capture the '226 transfer complete' so i can use that in an 'IF' statement to delete the backup files. Any help is greatly appreciated. also he is my FTP portion of the script
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv $HOST << EOF
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
I could use whatever mean i needed i suppose, FTP was something already setup for another backup function for something else, thanks
2nd EDIT Ahmed the rsync works great in test from command line, its a lot faster than FTP, the server is on the local network so SSH not that big of a deal but nice to have for added security, i will finish implementing in my script tomorrow, thanks again
FTP OPTION
The simple solution would be to do something like this:
ftp -inv $HOST >ftp-results-$YEAR-$MONTH-$DAY.out 2>&1 <<-EOF
user $USER $PASS
cd /baks
bin
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
Also there is an issue with your here-document syntax; there is no space between << and the delimiter word (in your case EOF) and I added a - because you are putting white-spaces before the ACTUAL delimeter (it's tabbed in for the if / fi block) so the [-] is required
Now when you do this; you can parse the output file to look for the successful put of the file. For example:
if grep -q '226 transfer complete' ftp-results-$YEAR-$MONTH-$DAY.out; then
echo "It seems that FTP transfer completed fine, we can schedule a delete"
echo "rm -f $PWD/weekending-$YEAR-$MONTH-$DAY.tar.gz" >> scheduled_cleanup.sh
fi
and just run scheduled_cleanup.sh using cron at a given time; this way you will have some margin before the files are cleaned up
If your remote FTP server has good SITE or PROXY options you may be able to get the remote FTP to run a checksum on the uploaded file after successful upload and return the result.
SCP / RSYNC OPTION
Using FTP is clunky and dangerous, you should really try and see if you can have scp or ssh access to the remote system.
If you can then generate an ssh key if you don't have one using ssh-keygen:
ssh-keygen -N "" -t rsa -f ftp-rsa
put the ftp-rsa.pub file into the $HOST/home/$USER/.ssh/authorized_keys and you have a much nicer method for uploading files:
if scp -B -C weekending-$YEAR-$MONTH-$DAY.tar.gz $USER#$HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
Or better yet using rsync:
if rsync --progress -a weekending-$YEAR-$MONTH-$DAY.tar.gz $HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
et voilà you are done since rsync works over ssh you are happy and secure
Try the next
#!/bin/bash
runifok() { echo "will run this when the transfer is OK"; }
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv <<EOF | grep -q '226 transfer complete' && runifok
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
test it and when will run ok - replace the echo in the runifok function for your commands what want execute after the upload is succesful.
Sometimes when I execute a bash script with the curl command to upload some files to my ftp server, it will return some error like:
56 response reading failed
and I have to find the wrong line and re-run them manually and it will be OK.
I'm wondering if that could be re-run automatically when the error occurs.
My scripts is like this:
#there are some files(A,B,C,D,E) in my to_upload directory,
# which I'm trying to upload to my ftp server with curl command
for files in `ls` ;
do curl -T $files ftp.myserver.com --user ID:pw ;
done
But sometimes A,B,C, would be uploaded successfully, only D were left with an "error 56", so I have to rerun curl command manually. Besides, as Will Bickford said, I prefer that no confirmation will be required, because I'm always asleep at the time the script is running. :)
Here's a bash snippet I use to perform exponential back-off:
# Retries a command a configurable number of times with backoff.
#
# The retry count is given by ATTEMPTS (default 5), the initial backoff
# timeout is given by TIMEOUT in seconds (default 1.)
#
# Successive backoffs double the timeout.
function with_backoff {
local max_attempts=${ATTEMPTS-5}
local timeout=${TIMEOUT-1}
local attempt=1
local exitCode=0
while (( $attempt < $max_attempts ))
do
if "$#"
then
return 0
else
exitCode=$?
fi
echo "Failure! Retrying in $timeout.." 1>&2
sleep $timeout
attempt=$(( attempt + 1 ))
timeout=$(( timeout * 2 ))
done
if [[ $exitCode != 0 ]]
then
echo "You've failed me for the last time! ($#)" 1>&2
fi
return $exitCode
}
Then use it in conjunction with any command that properly sets a failing exit code:
with_backoff curl 'http://monkeyfeathers.example.com/'
Perhaps this will help. It will try the command, and if it fails, it will tell you and pause, giving you a chance to fix run-my-script.
COMMAND=./run-my-script.sh
until $COMMAND; do
read -p "command failed, fix and hit enter to try again."
done
I have faced a similar problem where I need to make contact with servers using curl that are in the process of starting up and haven't started up yet, or services that are temporarily unavailable for whatever reason. The scripting was getting out of hand, so I made a dedicated retry tool that will retry a command until it succeeds:
#there are some files(A,B,C,D,E) in my to_upload directory,
# which I'm trying to upload to my ftp server with curl command
for files in `ls` ;
do retry curl -f -T $files ftp.myserver.com --user ID:pw ;
done
The curl command has the -f option, which returns code 22 if the curl fails for whatever reason.
The retry tool will by default run the curl command over and over forever until the command returns status zero, backing off for 10 seconds between retries. In addition retry will read from stdin once and once only, and writes to stdout once and once only, and writes all stdout to stderr if the command fails.
Retry is available from here: https://github.com/minfrin/retry