Check for specific string within wget result set and update the log based on that - string

I have a permanently VPN connection to a server in Germany. I have intermittent outages of this connection where the VPN connection drops and it falls back to default broadband ISP based internet connection path. I am trying to track this outages by using the way google works depending where your connection originating from. If I connect from my default connection in US, I get the standard google.com server, but if I connect over the VPN server in germany google.com connection attempt resolves to google.de site instead of google.com. This is a suitable criteria to see if connection is down.
So, if I issue a wget against "www.google.com" the resulting set will include either google.de in indicating that google detects that the connection is from germany or google.com which will indicate the connection is coming from U.S which means for my purposes that the VPN connection is down. I can't figure out the proper syntax for the wget and the grep to follow to make this specific determination in the script.
The script I came up with doesn't seem to work consistently. When it is executed by cru (cron) it gets US when I execute interactively I get Germany.
Any suggestions?
rm index*
wget -nv google.com 2 > /jffs/user/google.txt
cat google.txt | grep google.de
if [[ $? -eq 0 ]]; then
echo "$(date) Germany" >> /jffs/user/google.log
else
echo "$(date) U.S." >> /jffs/user/google.log
fi

Related

Programatically set DNS servers (Windows, MacOS)

I need to programmatically set DNS servers of the host on their active network interfaces (Wi-fi, ethernet, etc) on both Windows, MacOS and as a bonus Linux.
I want to avoid having to manually update/pollute /etc/hosts for my Kubernetes services I am running on my ingress.
Currently, my process is to manually set the DNS server for each person in my team running our app
The problem with this is that it's a manual process, and I am having trouble trying to automate it because the outputs are weirdly formatted and hard to parse. This means I am unable to know which is the proper network interface to use.
Essentially, what needs to be done is the following (on both platforms)
Get the active networks name
Set the DNS servers for the active network to 127.0.0.1 & 8.8.4.4
What is being done manually currently
MacOS:
networksetup -setdnsservers Wi-Fi 127.0.0.1 8.8.8.8
sudo killall -HUP mDNSResponder
127.0.0.1 is the local DNS server running on node that serves the A record for the service
8.8.8.8 is Google's Public DNS Server
Currently, I am assuming the user on MacOS is using the "Wi-Fi" network, but i'd like to determine this programatically
Windows
As administrator:
netsh interface show interface
Locate the network connection for which you want the DNS server changed (eg: WiFi).
netsh interface ipv4 add dns "WiFi" 127.0.0.1 index=1
netsh interface ipv4 add dns "WiFi" 8.8.8.8 index=2
ipconfig /flushdns
On macOS, I don't think this will do what you want. When you configure multiple DNS servers on macOS, the system resolver doesn't try them in order, it just fires off requests semi-randomly between the available servers. This means it'll sometimes send off requests for your private servers to the public (Google) server, get told there's no such domain, and stop there. Or it'll send requests for pubic sites to the localhost DNS, and if that doesn't respond properly decide that site doesn't work. Basically, the macOS resolver doesn't do failover.
Are your private servers under a non-standard TLD or something like that? If so, you might be able to do the job by adding a file under /etc/resolver/ to redirect queries for that TLD to the private DNS server.
Anyway, in case it is useful, here's a way to detect the primary (active) network interface and set its DNS servers in macOS:
#!/bin/bash
interfaceDevice=$(netstat -rn | awk '($1 == "default") {print $6; exit}')
if [[ -z "$interfaceDevice" ]]; then
echo "Unable to get primary network interface device" >&2
exit 1
fi
interfaceName=$(networksetup -listallhardwareports | grep -B1 "Device: $interfaceDevice\$" | sed -n 's/^Hardware Port: //p')
if [[ -z "$interfaceName" ]]; then
echo "Unable to get primary network interface name" >&2
exit 1
fi
networksetup -setdnsservers "$interfaceName" 127.0.0.1 8.8.8.8

Transfer variables in ssh an get answer back

I want to write script in bash to connect to server, ping on it another server, get ip from ping command, and send that info back to pc who ran the script, and at the end connect to server who is pinged before.
My script:
#!/bin/bash
echo "Script to connect to server"
#ip_p used to connect to first server, on that server i want to use ping command'
ip_p=XYz.XYZ.XYZ.XYZ
user='username to the servers'
#ip_d - in that variable i want to save ip of the pinged server
ssh -t $user#$ip_p ip_d="ip_d=ping -c1 $1.domain | sed -nE 's/^PING[^(]+\(([^)]+)\).*/\1/p' && exit "
echo "start"
echo $ip_d
echo "stop"
ssh -t $user#$ip_d
How i wish to work:
domain i want to check test.nyslay.pl
connect to server which ip, and username was defined in script
ping server(part ".nyslay.pl", is always the same, but "test" i want to read from first argument of script run
get ip of domain from previous point
transfer ip from point: 2 to local machine, on which script is run
connect to the server which ip we get from point: 2
Use command substitution:
ip_d=$(ssh -t $user#$ip_p "ping -c1 $1.domain | sed -nE 's/^PING[^(]+\(([^)]+)\).*/\1/p'")
The output of ping (through sed) comes to the local machine via ssh, whose output is captured in the local variable ip_d.

wget connection timed out on same server

I've got a very strange problem.
There's a cron job on the server to run a script daily:
wget -O /dev/null --timeout=300 --tries=1 "http://website.com/script"
It was all working well since about two weeks ago, I started receiving errors:
--2016-07-13 09:45:01-- http://website.com/script
Resolving website.com (website.com)... 11.22.33.44
Connecting to website.com (website.com)|11.22.33.44|:80... failed: Connection timed out.
Giving up.
These are some information for this question:
The cron job is on the same server of http://website.com hosted.
I can access the script (http://website.com/script) correctly from browser on my desktop.
The server is CentOS 7, with WHM and cPanel installed.
Anyone know what could be the issue? or how do I suppose to identify the issue?
Thanks
If the issue still is unresolved..
You could try running wget in debug mode to see if you get some more info.
wget -dv -O /dev/null --timeout=300 --tries=1 "http://website.com/script"
Also, confirm if the resolved IP "11.22.33.44" belongs to one of the servers NIC's.
ip a s (ip address show) or
ifconfig -a
If the IP is not listed, It could be that the ip "11.22.33.44" is a public facing address of the company's firewall. And that the FW is directing requests on port 80 from the outside/internet (where you're browser is) to that specific server. And the Firewall/Nat/Proxy, could be configured to not allow requests coming from inside the network, reaching the external IP of the firewall and getting back in.
If this is the case, you could try changing you're wget using the internal ip address, something like: (still using -dv for debugging, remove after)
wget -dv -O /dev/null --timeout=300 --tries=1 --header="Host: website.com" http://127.0.0.1/script
Note1: the --header="Host: website.com" will tell you're webserver what site you wanna reach
Note2: maybe you'll have to change the IP: 127.0.0.1 (localhost address) to one of the server's NIC addresses.
If the website is up try a different command.
*/10 * * * * /usr/bin/wget -q -O temp.txt http://website.com/script
Try adding -H
wget -H -O /dev/null --timeout=300 --tries=1 "http://website.com/script"

Domain or IP exists check in Linux

I want to know if a given website or IP address is online or offline. I researched a lot, but all I can find is to install some software or using the ping command.
I did this test:
ping -c 5 -n example.com
It outputs the expected result, but when I do the following where a website does not ext, the result is almost the same as if website existed, with 0% packet loss. Please see the screenshot attached.
ping -c 5 -n examplesurenotexists.com
I am confused by this. Is there a better way to do this task?
If you want to know if a website is online of offline, simply check the website:
if curl -s http://www.alfe.de >/dev/null
then
echo "online"
else
echo "offline"
fi
Using ping instead would not test the HTTP protocol (which is for websites) but the ICMP protocol; one is merely independent from the other (but of course, if the host is down, both won't work). There are sites which still react on ICMP while the HTTP server is down (this is rather typical) and there are sites which won't react on ICMP although the HTTP server is up and running functioning perfectly well.

Bash script to (more or less) reliably check if the Internet is up

I need a Bash (or a plain shell) script to put in a cronjob that every minute checks if the Internet is up.
This is how I did it:
#! /bin/sh
host1=google.com
host2=wikipedia.org
curr_date=`date +"%Y%m%d%H%M"`
echo -n "${curr_date};"
((ping -w5 -c3 $host1 || ping -w5 -c3 $host2) > /dev/null 2>&1) &&
echo "up" || (echo "down" && exit 1)
How would you do it? Which hosts would you ping?
Clarifications:
By "internet is up", I mean my internet connection.
By "up", I mean to have usable connection (doesn't really matter if we are talking about the DNS being down or the connection is really really slow [mind the -w for timeout]). That is also why I didn't include any IP but only hosts.
Should I also ping Stack Overflow? I mean, if I can't access Google, Wikipedia or Stack Overflow, I don't want Internet :p
That one seems like a good solution. Just add a few more hosts, and maybe some pure IP hosts so you don't rely on DNS functioning (which in itself depends on your definition of "up").
Thanks for your code, it works great, I've left only one line actually:
((ping -w5 -c3 8.8.8.8 || ping -w5 -c3 4.2.2.1) > /dev/null 2>&1) && echo "up" || (echo "down" && exit 1)
What portion of Internet connectivity are you looking to check? DHCP? DNS? Physically being plugged into a jack? Kernel recognizing the presence of the NIC?
You can manually query your ISP's DNS server(s) by using the host(1) command. This is generally a good indication of whether your router has lost its connection to the ISP.
You can query what interfaces your kernel has by using netstat(8) or ifconfig(8).
You can get detailed statistics about the interface using ifstat.

Resources