We are using linux Servers (CentOS) and one backup server
We only login to prod server if any there is any issue, else we check connection from backup server
We do no have any kind of monitoring tool
I have created a simple bash script on backup server as below
#!/bin/bash
date
cat /tmp/servers.txt | while read output
do
ping -c 1 "$output" > /dev/null
if [ $? -eq 0 ] ; then
echo "Server $output is UP"
else
echo "Server $output is Down"
fi
done
How do we get output of this script after log into this server automatically.
~/.bash_profile runs every time you log in, so that seems like the place you would want to put this code.
Related
I have a Node.JS server running on PM2 that's crashing every once in a while because of a database limit, which I'm working on.
In the meantime, I thought I'd try just setting up a cron job in cpanel to restart the server every hour if it's down.
So I wrote a bash script like the following:
#!/bin/bash
status_code=$(curl --write-out %{http_code} --silent --output /dev/null https://website.com/)
date >> cronlog.txt
if [[ "$status_code" -ne 200 ]] ; then
pkill node
nohup pm2 start bin/www &
echo "Site status $status_code" >> cronlog.txt
echo "Restarting Server" >> cronlog.txt
exit
else
echo "Site fine" >> cronlog.txt
exit 0
fi
Running this from an SSH terminal works perfectly; if the site is down, it'll restart it.
However, once I set up the cron job in cpanel, like so: 0 * * * * /home/acc123/fix.sh, looking at the output of cronlog.txt, I see that the script is definitely running every hour, trying to restart the server - it's just that the server doesn't restart.
A preliminary Google suggested that maybe pm2 wasn't on the path that the cron job runs from, so I modified the script to look like this:
#!/bin/bash
status_code=$(curl --write-out %{http_code} --silent --output /dev/null https://website.com/)
date >> cronlog.txt
if [[ "$status_code" -ne 200 ]] ; then
pkill node
nohup /home/acc123/bin/pm2 start /home/acc123/bin/www &
echo "Site status $status_code" >> cronlog.txt
echo "Restarting Server" >> cronlog.txt
exit
else
echo "Site fine" >> cronlog.txt
exit 0
fi
But nothing changes. Looking at the text file I write to, the script is definitely running every hour, and it's definitely picking up that the site is down, but while the words "Restarting Server" get written to the text file, the server doesn't actually start.
Checking nohup.out confirms that nothing has been written to it, suggesting that somehow the command nohup /home/acc123/bin/pm2 start /home/acc123/bin/www & isn't running correctly.
I'm stumped. Has anyone seen something similar before?
Found it. Looks like node itself also wasn't on the path variable for the cron job. Explicitly specifying where node was fixed the problem.
I have written the below script to check whether my server running fine or not.But it is not properly working.It always showing Not runing even if it is running fine.Also the telnet in the script is not running working properly .Can any one help?
#!/bin/sh
export smtp=smtprelay.intra.xxx.com:25
Connect_redmine(){
telnet redmine.intra.xxx.com 443 <<EOF
exit 1;
EOF
}
Connect_redmine>/home/ssx00001/log_connect.txt
grep "Connected" /home/ssx00001/log_connect.txt
status=$?
if [ $status == 0 ]; then
echo `date` "Redmine PROD server is running fine"|mailx -r Redmine#xxx -s "Redmine PROD server is running" 777.p#xxx.com
else
echo "Redmine PROD server is not Running"|mailx -r redmine#xxx.com -s "Redmine PROD server is not running" 777.p#xxx.com
fi
A couple of questions first:
1] What does redmine do? Is it just a HTTPS server?
2] If [1] is true, can you do a wget of the index page, and use the result of that? It should be a lot easier to parse.
3] Telnetting into a HTTPS server, as far as I know, won't work, because it's not doing any of the handshaking that would be necessary for an SSL connection (which needs to occur before any content will be sent).
Using wget, you can do something like this:
wget https://redmine.intra.xxx.com/index.html
if [[-f "index.html" ] && [ -s "index.html" ]]
then
# The service is live
else
# Something is wrong
fi
I have a bash script that creates backups incrementally(daily) and full(on Mondays). Every 7 days the script combines the week of backups(full and Incremental) and sends them off to an FTP server, the problem i am having is i want to delete the files from my backup directory after the FTP upload is finished, but i cant do that until i know the file was successfully uploaded. I need to figure out how to capture the '226 transfer complete' so i can use that in an 'IF' statement to delete the backup files. Any help is greatly appreciated. also he is my FTP portion of the script
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv $HOST << EOF
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
I could use whatever mean i needed i suppose, FTP was something already setup for another backup function for something else, thanks
2nd EDIT Ahmed the rsync works great in test from command line, its a lot faster than FTP, the server is on the local network so SSH not that big of a deal but nice to have for added security, i will finish implementing in my script tomorrow, thanks again
FTP OPTION
The simple solution would be to do something like this:
ftp -inv $HOST >ftp-results-$YEAR-$MONTH-$DAY.out 2>&1 <<-EOF
user $USER $PASS
cd /baks
bin
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
Also there is an issue with your here-document syntax; there is no space between << and the delimiter word (in your case EOF) and I added a - because you are putting white-spaces before the ACTUAL delimeter (it's tabbed in for the if / fi block) so the [-] is required
Now when you do this; you can parse the output file to look for the successful put of the file. For example:
if grep -q '226 transfer complete' ftp-results-$YEAR-$MONTH-$DAY.out; then
echo "It seems that FTP transfer completed fine, we can schedule a delete"
echo "rm -f $PWD/weekending-$YEAR-$MONTH-$DAY.tar.gz" >> scheduled_cleanup.sh
fi
and just run scheduled_cleanup.sh using cron at a given time; this way you will have some margin before the files are cleaned up
If your remote FTP server has good SITE or PROXY options you may be able to get the remote FTP to run a checksum on the uploaded file after successful upload and return the result.
SCP / RSYNC OPTION
Using FTP is clunky and dangerous, you should really try and see if you can have scp or ssh access to the remote system.
If you can then generate an ssh key if you don't have one using ssh-keygen:
ssh-keygen -N "" -t rsa -f ftp-rsa
put the ftp-rsa.pub file into the $HOST/home/$USER/.ssh/authorized_keys and you have a much nicer method for uploading files:
if scp -B -C weekending-$YEAR-$MONTH-$DAY.tar.gz $USER#$HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
Or better yet using rsync:
if rsync --progress -a weekending-$YEAR-$MONTH-$DAY.tar.gz $HOST:/baks/ ; then
echo Upload successful 1>&2
else
echo Upload failed 1>&2
fi
et voilà you are done since rsync works over ssh you are happy and secure
Try the next
#!/bin/bash
runifok() { echo "will run this when the transfer is OK"; }
if [ -a "$WKBKDIR/weekending-$YEAR-$MONTH-$DAY.tar.gz" ]; then
HOST=192.168.40.30 #This is the FTP servers host or IP address.
USER=username #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -inv <<EOF | grep -q '226 transfer complete' && runifok
user $USER $PASS
cd /baks
lcd $WKBKDIR
put weekending-$YEAR-$MONTH-$DAY.tar.gz
bye
EOF
fi
test it and when will run ok - replace the echo in the runifok function for your commands what want execute after the upload is succesful.
I want to use ping to check to see if a server is up. How would I do the following:
ping $URL
if [$? -eq 0]; then
echo "server live"
else
echo "server down"
fi
How would I accomplish the above? Also, how would I make it such that it returns 0 upon the first ping response, or returns an error if the first ten pings fail? Or, would there be a better way to accomplish what I am trying to do above?
I'ld recommend not to use only ping. It can check if a server is online in general but you can not check a specific service on that server.
Better use these alternatives:
curl
man curl
You can use curl and check the http_response for a webservice like this
check=$(curl -s -w "%{http_code}\n" -L "${HOST}${PORT}/" -o /dev/null)
if [[ $check == 200 || $check == 403 ]]
then
# Service is online
echo "Service is online"
exit 0
else
# Service is offline or not working correctly
echo "Service is offline or not working correctly"
exit 1
fi
where
HOST = [ip or dns-name of your host]
(optional )PORT = [optional a port; don't forget to start with :]
200 is the normal success http_response
403 is a redirect e.g. maybe to a login page so also accetable and most probably means the service runs correctly
-s Silent or quiet mode.
-L Defines the Location
-w In which format you want to display the response
-> %{http_code}\n we only want the http_code
-o the output file
-> /dev/null redirect any output to /dev/null so it isn't written to stdout or the check variable. Usually you would get the complete html source code before the http_response so you have to silence this, too.
nc
man nc
While curl to me seems the best option for Webservices since it is really checking if the service's webpage works correctly,
nc can be used to rapidly check only if a specific port on the target is reachable (and assume this also applies to the service).
Advantage here is the settable timeout of e.g. 1 second while curl might take a bit longer to fail, and of course you can check also services which are not a webpage like port 22 for SSH.
nc -4 -d -z -w 1 ${HOST} ${PORT} &> /dev/null
if [[ $? == 0 ]]
then
# Port is reached
echo "Service is online!"
exit 0
else
# Port is unreachable
echo "Service is offline!"
exit 1
fi
where
HOST = [ip or dns-name of your host]
PORT = [NOT optional the port]
-4 force IPv4 (or -6 for IPv6)
-d Do not attempt to read from stdin
-z Only listen, don't send data
-w timeout
If a connection and stdin are idle for more than timeout seconds, then the connection is silently closed. (In this case nc will exit 1 -> failure.)
(optional) -n If you only use an IP: Do not do any DNS or service lookups on any specified addresses, hostnames or ports.
&> /dev/null Don't print out any output of the command
You can use something like this -
serverResponse=`wget --server-response --max-redirect=0 ${URL} 2>&1`
if [[ $serverResponse == *"Connection refused"* ]]
then
echo "Unable to reach given URL"
exit 1
fi
Use the -c option with ping, it'll ping the URL only given number of times or until timeout
if ping -c 10 $URL; then
echo "server live"
else
echo "server down"
fi
Short form:
ping -c5 $SERVER || echo 'Server down'
Do you need it for some other script? Or are trying to hack some simple monitoring tool? In this case, you may want to take a look at Pingdom: https://www.pingdom.com/.
I using the following script function to check servers are online or not. It's useful when you want to check multiple servers. The function hide the ping output, and you can handle separately the server live or server down case.
#!/bin/bash
#retry count of ping request
RETRYCOUNT=1;
#pingServer: implement ping server functionality.
#Param1: server hostname to ping
function pingServer {
#echo Checking server: $1
ping -c $RETRYCOUNT $1 > /dev/null 2>&1
if [ $? -ne 0 ]
then
echo $1 down
else
echo $1 live
fi
}
#usage example, pinging some host
pingServer google.com
pingServer server1
One good solution is to use MRTG (a simple graphing tool for *NIX) with ping-probe script. look it up on Google.
read this for start.
Sample Graph:
I have a simple CGI page (running on linux, apache) that grabs some responses from remote servers. When I manually run the script (from a terminal) it echos the complete web-page correctly, including all remote responses. But when I open the browser, the responses are not there!
Here's my script for reference.
#!/bin/bash
echo "Content-type: text/html"
echo ""
echo "<html class="background"><head><title>My Page"
echo "</title></head><body>"
echo ""
echo "<h2>Local Uptime :</h2>"
echo `uptime` #Local commands work normally
echo "<h2>Remote Uptime: </h2>"
echo `/usr/bin/ssh root#remote-server "uptime"`
echo "</body></html>"
Of course, I previously set keys for password-less logins.
Check to make sure selinux is turned off (or set the correct options for the web browser to execute ssh).
cat /selinux/enforce
If it's 1 set it to 0 as root
echo 0 > /selinux/enforce