Continue to grep for traceroute result with bash - linux

Every night I go through the same process of checking failover systems for our T1's. I essentially go through the following process:
Start the failover process.
traceroute $server;
Once I see it's failed over, I verify that connections work by SSHing into a server.
ssh $server;
Then once I see it works, I take it off of failover.
So what I want to do is to continually run a traceroute until I get a certain result, then run a SSH command.

Put your list of successful messages in a file (omit the variable lines and fractions of the line, and use a ^ to identify the start of the line, as such:)
patterns.list:
^ 7 4.68.63.165
^ 8 4.68.17.133
^ 9 4.79.168.210
^10 216.239.48.108
^11 66.249.94.46
^12 72.14.204.99
Then a simple while loop:
while ! traceroute -n ${TARGET} | grep -f patterns.list
do
sleep 5 # 5 second delay between traceroutes, for niceness.
done
ssh ${DESTINATION}
Use traceroute -n to generate the output so you don't get an IP address that resolves one time, but and a name the next, resulting in a false positive.

I think you could be better off using ping command to verify server's accessability than traceroute.
It is easy to check for return status of ping command without using any grep at all:
if [ ping -c 4 -n -q 10.10.10.10 >/dev/null 2>& ]; then
echo "Server is ok"
else
echo "Server is down"
fi
If you want to do it continually in a loop, try this:
function check_ssh {
# do your ssh stuff here
echo "performing ssh test"
}
while : ; do
if [ ping -c 4 -n -q 10.10.10.10 >/dev/null 2>& ]; then
echo "Server is ok"
check_ssh
else
echo "Server is down"
fi
sleep 60
done

Related

Linux shell (sh) CLI test if ping successful

How to wire linux shell (sh) script to test with ping if host is reachable?
I guess there could be solution that uses grep but maybe ping provides that option by itself?
I am more into getting a whitelisting a successful ping operation that reached the host then checking if there was any error. I don't care about the reason of ping not succeeding in reaching a host.
I would like to limit ping attempts count and maximum amount of time to reach the host so the script does not waits too long for ping trying to reach a host.
dt=$(date +%d)
cksize=50
echo "Start $(date)"
while IFS= read -r sn
do
echo "*************************************************"
echo "Begin checking NODES client: $sn"
if ping -c 1 "$sn" -i 5 > /dev/null
then
echo "$sn node up"
else
echo "$sn node down"
fi
done < server_list
parallel -j0 --timeout 15 'ping -c 5 -i 0.2 {} >/dev/null 2>&1 && echo {} up || echo {} down' ::: freenetproject.org debian.org no-such.domain {1..254}.2.3.4
You can do it like this. It will do it in parallel for all hosts.
#!/bin/bash
for server in 'google.com' 'github.com' 'fakeserver.com'
do
{ ping -o "$server" &>/dev/null && echo "$server is UP" || echo "$server is DOWN" ; } &
done
wait
Regards!

Detecting when Mongod's port is open inside a script

I'm trying to write a bash script that starts a mongod process, waits for it to start (i.e. have it's default port open) and then pipe some commands into it through the mongo shell. I'd like some way to wait for the mongod process to be completely up that's more deterministic than just sleep 5.
This is the script so far:
set_up_authorization() {
echo "Setting up access control"
/path/to/mongo < configure_access_controls.js
}
wait_for_mongod_to_start() {
RETRIES=1000
CONNECTED="false"
echo "Waiting for mongod to start"
while [[ $RETRIES -ge 0 && $CONNECTED == "false" ]] ; do
RESPONSE=$(exec 6<>/dev/tcp/127.0.0.1/27017 || echo "1")
if [[ $RESPONSE == "" ]] # which should happen if the exec is successful
CONNECTED="true"
fi
RETRIES=$((RETRIES - 1))
done
if [[ $RETRIES -eq 0 ]] ; then
echo "Max retries reached waiting for mongod to start. Exiting."
exit 1
fi
echo "Mongod started"
}
./start_mongod_instance.sh
wait_for_mongod_to_start
set_up_authorization
While this script works, it produces a ton of output on the terminal while the exec is failing:
./initialize_cluster.sh: connect: Connection refused
./initialize_cluster.sh: line xx: /dev/tcp/127.0.0.1/27017: Connection refused
...which repeats for all ~900 failed attempts.
Neither of the following seems to get rid of the terminal logging either:
exec 6<>/dev/tcp/127.0.0.1/27017 >/dev/null
OR
exec 6<>/dev/tcp/127.0.0.1/27017 2>/dev/null
I've also tried using the following:
ps -aux | grep "mongod" | wc -l
but the process having a pid that ps lists isn't equivalent to it's port being open or it accepting connections.
Any ideas on either front would be appreciated - a more elegant way to wait for the process to start completely or a way to get rid of the excessive logging to the terminal.
Note: I don't have access to nmap or nc to check the port (this is on a client's machine).
exec is a bit special. It affects the output of the current shell. Meaning you need to redirect stderr of the current shell before running the port check:
host="localhost"
port="9000"
exec 2>/dev/null # redirect error here
while ! exec 3<>"/dev/tcp/${host}/${port}" ; do
echo "Waiting ..."
sleep 1
done
Furthermore you might have noticed that I check the exit status of exec rather than some output to decide whether the port is open or not.
If you want to reset it afterwards:
host="localhost"
port="9000"
# Copy fd 2 into fd 3 and redirect fd 2 to /dev/null
exec 3<&2 2>/dev/null
while ! exec 3<>"/dev/tcp/${host}/${port}" ; do
echo "Waiting ..."
sleep 1
done
# Copy back fd 3 into fd 2
exec 2<&3
echo "EE oops!" >&2

why ECHO does not come up?

I have this shell :
===
#!/bin/sh -e
LogFile=/home/pi/logs/prova.log # log file
test -e $LogFile || touch $LogFile # create it if non existent
echo "(1) ======== ======== ======== Inici de PROVA.SH" >> $LogFile
echo "(2) ping 1.2.3.4" >> $LogFile
# ping 1.2.3.4 -W 3 -c 2 >> $LogFile
echo "(3) start APP" >> $LogFile
echo "LOG file is" $LogFile
exit 0
===
The output is
1) one line to screen
2) three lines to file
But if the 8th line (ping 1.2.3.4) is un-commented,
the "echo's" after the 8th line do not get written,
neither to the screen, neither to the file.
I need to understand why, and how to solve it.
I guess is something related to the fact that "ping" runs in another shell,
so the "echo's" write there.
But I don't know how to fix it.
Any pointer or URL to documentation is welcome.
Sebastian.
ping -W 3 -c 2 1.2.3.4 >> $LogFile
Put the IP/Hostname after the ping options.
Most likely:
ping fails because of wrong order of arguments: destination should be last.
Your script runs with -e so it exits at first error, so it stops after ping fails.
You don't redirect standard error for ping : the error message is lost
If you remove -e, ping still fails, but the script continues, executes the last 2 lines and you get their output (but you do not get from ping because that goes to stderr)
Solution, 2 changes:
ping -W 3 -c 2 1.2.3.4 2>&1 >> $LogFile
^^^^^^^ ^^^^
I.P as last argument & Redirect stderr to stdout before redirecting to file

Linux/Unix check if VPN connection is Active/Up

I have a code which detects if OpenVPN connection is up or down:
if echo 'ifconfig tun0' | grep -q "00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00"
then
echo "VPN up"
else
echo "VPN down"
fi
exit 0
now I'm trying to re-write the code to work with PPTP or IPSEC connection. I've tried to do:
if echo 'ifconfig ppp0' | grep -q "00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00"
or the same with ipsec but does not work. Is there any other way to detect PPTP or IPSEC connection?
That echo statement is erroneous. As #unwind says, the single quotes (') should be backtics (`). Your current code is sending the literal value ifconfig ppp0 to grep, which doesn't do anything useful.
But you don't actually need the backtics, either. You can just send the output of ifconfig to grep directory; using echo doesn't get you anything:
if ifconfig ppp0 | grep -q "00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00"; then
echo ppp connection is up
fi
The following script will:
Run the ISPConnectivity.sh script every 5 minutes. This will mean that the VPN tunnel will not be down for more than 5 minutes.
Check if the tun interface is down, and start the vpn script if it is.
Check connectivity if the tun0 interface is up. It does ping tests on 2 Public IPs (if I get even a single response from 1 of the IPs tested, I consider this a success ), and all have to fail to run the vpn script. I ran ping tests on multiple hosts to prevent the vpn script from starting in case the ping test failed on 1 IP.
Send all failure output to a file in my home directory. I do not need to see if any test succeeded.
Contents of sudo crontab:
*/5 * * * * /home/userXXX/ISPConnectivity.sh >> /home/userXXX/ISPConnectivity.log 2>&1
Contents of ISPConnectivity.sh script:
#!/bin/bash
# add ip / hostname separated by white space
#HOSTS="1.2.3.4"
HOSTS="8.8.8.8 4.2.2.4"
# no ping request
totalcount=0
COUNT=4
DATE=`date +%Y-%m-%d:%H:%M:%S`
if ! /sbin/ifconfig tun0 | grep -q "00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00"
then
echo $DATE tun0 down
sudo /home/userXXX/startVPN.sh start
else
for myHost in $HOSTS;
do
count=`ping -c $COUNT $myHost | grep 'received' | awk -F',' '{ print $2 }' | awk '{ print $1 }'`
totalcount=$(($totalcount + $count))
done
if [ $totalcount -eq 0 ]
then
echo $DATE $totalcount "fail"
sudo /home/userXXX/startVPN.sh start
#else
# echo $DATE $totalcount "pass"
fi
fi
You can also check with the nmcli command, to check if VPN is running or not.
nmcli c show --active | grep vpn
I'm actually looking into more flexible solution eg:
MyIP=$(curl http://api.ipify.org/?format=text)
if [ "$MyIP" != "MYORYGINALIP" ]
then
echo "IPSEC VPN is Running - " $MyIP
else
echo "IPSEC VPN is Not Running - " $MyIP
fi
exit 0
what about that? can I improve it any way?
ip route list table 220 if Ip address shown -> VPN connection established, none -> no VPN
or
if [ "0" == ifconfig | grep wlan0 | wc -l ]; then echo "NO wlan0 has no VPN"; else echo "YES wlan0 has VPN"; fi

How to set up an automatic (re)start of a background ssh tunnel

I am a beginner user of linux, and also quite newbie at ssh and tunnels.
Anyway, my goal is to maintain a ssh tunnel open in background.
In order to do that, I wrote the following batch that I then added into crontab (the batch is automatically processed every 5 minutes during workdays and from 8am to 9pm).
I read in some other thread in stackoverflow that one should use autossh that will ensure the ssh will always be ok through a recurrent check. So did I....
#!/bin/bash
LOGFILE="/root/Tunnel/logBatchRestart.log"
NOW="$(date +%d/%m/%Y' - '%H:%M)" # date & time of log
if ! ps ax | grep ssh | grep tunnelToto &> /dev/null
then
echo "[$NOW] ssh tunnel not running : restarting it" >> $LOGFILE
autossh -f -N -L pppp:tunnelToto:nnnnn nom-prenom#193.xxx.yyy.zzz -p qqqq
if ! ps ax | grep ssh | grep toto &> /dev/null
then
echo "[$NOW] failed starting tunnel" >> $LOGFILE
else
echo "[$NOW] restart successfull" >> $LOGFILE
fi
fi
My problem is that sometimes the tunnel stops working, although every thing looks ok (ps ax | grep ssh > the result shows the two expected tasks : autossh main task and the ssh tunnel itself). I actually know about the problem cause the tunnel is used by a third party software that triggers an error as soon as the tunnel is no more responding.
SO I am wondering how I should improve my batch in order It will be able to check the tunnel and restart it if it happens to be dead. I saw some ideas in there, but it was concluded by the "autossh" hint... which I already use. Thus, I am out of ideas... If any of you have, I'd gladly have a look at them!
Thanks for taking interest in my question, and for your (maybe) suggestions!
Instead of checking the ssh process with ps you can do the following trick
create script, that does the following and add it to your crontab via crontab -e
#!/bin/sh
REMOTEUSER=username
REMOTEHOST=remotehost
SSH_REMOTEPORT=22
SSH_LOCALPORT=10022
TUNNEL_REMOTEPORT=8080
TUNNEL_LOCALPORT=8080
createTunnel() {
/usr/bin/ssh -f -N -L$SSH_LOCALPORT:$REMOTEHOST:SSH_REMOTEPORT -L$TUNNEL_LOCALPORT:$REMOTEHOST:TUNNEL_REMOTEPORT $REMOTEUSER#$REMOTEHOST
if [[ $? -eq 0 ]]; then
echo Tunnel to $REMOTEHOST created successfully
else
echo An error occurred creating a tunnel to $REMOTEHOST RC was $?
fi
}
## Run the 'ls' command remotely. If it returns non-zero, then create a new connection
/usr/bin/ssh -p $SSH_LOCALPORT $REMOTEUSER#localhost ls >/dev/null 2>&1
if [[ $? -ne 0 ]]; then
echo Creating new tunnel connection
createTunnel
fi
In fact, this script will open two ports
port 22 which will be used to check if the tunnel is still alive
port 8080 which is the port you might want to use
Please check and send me further questions via comments
(I add this as an answer since there is not enough room for it un a comment)
Ok, I managed to make the batch run to launch the ssh tunnel (I had to specify my hostname instead of localhost in order it could be triggered) :
#!/bin/bash
LOGFILE="/root/Tunnel/logBatchRedemarrage.log"
NOW="$(date +%d/%m/%Y' - '%H:%M)" # date et heure du log
REMOTEUSER=username
REMOTEHOST=remoteHost
SSH_REMOTEPORT=22
SSH_LOCALPORT=10022
TUNNEL_REMOTEPORT=12081
TUNNEL_SPECIFIC_REMOTE_PORT=22223
TUNNEL_LOCALPORT=8082
createTunnel() {
/usr/bin/ssh -f -N -L$SSH_LOCALPORT:$REMOTEHOST:$SSH_REMOTEPORT -L$TUNNEL_LOCALPORT:$REMOTEHOST:$TUNNEL_REMOTEPORT $REMOTEUSER#193.abc.def.ghi -p $TUNNEL_SPECIFIC_REMOTE_PORT
if [[ $? -eq 0 ]]; then
echo [$NOW] Tunnel to $REMOTEHOST created successfully >> $LOGFILE
else
echo [$NOW] An error occurred creating a tunnel to $REMOTEHOST RC was $? >> $LOGFILE
fi
}
## Run the 'ls' command remotely. If it returns non-zero, then create a new connection
/usr/bin/ssh -p $SSH_LOCALPORT $REMOTEUSER#193.abc.def.ghi ls >/dev/null 2>&1
if [[ $? -ne 0 ]]; then
echo [$NOW] Creating new tunnel connection >> $LOGFILE
createTunnel
fi
However, I got some immediate message (below) when the tunnel is running and when cron tries to lauch the batch again... sounds like it cannot listen to it. Also since I need some time to get a proof , I can't say yet it will successfully restart if the tunnel is out.
Here's the response to the second start of the batch.
bind: Address already in use channel_setup_fwd_listener: cannot listen
to port: 10022 bind: Address already in use
channel_setup_fwd_listener: cannot listen to port: 8082 Could not
request local forwarding.

Resources