False reporting on bash script - linux

Context
I have a script that pulls down data from a web server using wget, checks the filesize of the index.html page using ls -la, if it is a certain size using grep -o, it is assumed the page is non-functional to customer requirements.
Code
BADSERVER=()
FILE=SERVER
echo '' > $FILE.txt
<pull a list of hosts down>
if ! ssh -n -f $device -i <sshkey> "yes | rm index.* ; wget localhost:8080 ; ls -la index.html | grep -o 12282" > /tmp/$FILE 2> /dev/null; then
echo "Unable to connect to $device"
BADSERVER+=($device)
echo '' > /tmp/$FILE
elif cat /tmp/$FILE | grep '12282' ; then
echo 'Page is non functional to customer requirements on $device'
BADSERVER+=($device)
echo '' > /tmp/$device
else
echo 'all is fine on $ip'
echo '' > /tmp/$FILE
fi
done
Problem
The script reports that some webservers have the filesize, when manually connecting I can see that the file is not the same size.
Script traceback
+ ssh -n -f webserver.1014 -i <omitted> 'yes | rm index.* ; wget localhost:8080 ; ls -la index.html | grep -o 12282'
+ grep 12282
+ cat /tmp/SERVER.txt
+ echo 'All OK on webserver.1014'
+ echo ''
+ cat /tmp/SERVER.txt
+ cat /tmp/SERVER.txt
+ ssh -n -f webserver.1015 -i <omitted> 'yes | rm index.* ; wget localhost:8080 ; ls -la index.html | grep -o 12282'
+ cat /tmp/SERVER.txt
+ grep 12282
12282
+ echo 'Page is non functional to customer requirements on $device'
Page is non functional to customer requirements on webserver.1015
+ BADSERVER+=($device)
+ echo ''
+ cat /tmp/SERVER.txt
+ cat /tmp/SERVER.txt
+ ssh -n -f webserver.1016 -i <omitted> 'yes | rm index.* ; wget localhost:8080 ; ls -la index.html | grep -o 12282'
+ cat /tmp/SERVER.txt
+ grep 12282
+ echo 'All OK on webserver.1017'
+ echo ''
+ cat /tmp/SERVER.txt
+ cat /tmp/SERVER.txt
Actual content of webserver.1015
-r--r--r--. 1 --- --- 25799 Mar 30 23:38 index.html

Is the server up & running at webserver.1015 ?
Why are you using ; instead of && ?
If previous command failed with ; the second one will run.
But with && the second one will not run.
Don't parse the output of ls command.
Extracting only file size with wget
wget http://localhost:8080/index.html --spider --server-response -O - 2>&1 | sed -ne '/Content-Length/{s/.*: //;p}'
Or with curl
size=$(curl -sI "http://localhost:8080/index.html" | awk '/Content-Length/{gsub("\\r", ""); print $2}')

Related

Store output from my bash script in a .txt file

I want store the output of my script in a file, but can't.
This is:
#!/bin/bash
if [ "$1" == "" ]
then
echo "Te falta especificar una dirección ip"
echo "Syntax esperado: ./hostDiscovery.sh <ip>"
else
for ip in `seq 1 254`; do
ping -c 1 $1.$ip | grep "64 bytes" | cut -d " " -f 4 | tr -d ":" | tee ip.txt &
done
fi
The problem is that the file that was created has no data. It is blank.
To quickly answer your question, simply replace your last pipe ... -d ":" | tee ip.txt & with ... -d ":" >> ip.txt &
The >> syntax will append to ip.txt. You can also use tee -a to append but that will also print to stdout.
If you want to make sure that file is cleared each time you run the script you will want to: > ip.txt at the top of your script to clear ip.txt.
Here is what your new script might look like:
#!/bin/bash
if [ "$1" == "" ]; then
echo "Te falta especificar una dirección ip"
echo "Syntax esperado: ./hostDiscovery.sh <ip>"
else
> ip.txt
for ip in `seq 1 254`; do
ping -c 1 $1.$ip | grep "64 bytes" | cut -d " " -f 4 | tr -d ":" | tee -a ip.txt &
done
fi
I have a lot to say about this script though. It looks like you are trying to ping all ip addresses within a range to test to see if they are up/alive. You may want to look into nmap
e.g. nmap -sP 192.168.1.1-254
You are overwriting ip.txt on each call to tee.
Try calling tee -a to append.
man tee for details

How to get pipe string length?

This is a code that shows my all user names.
-q user | grep -A 0 -B 2 -e uid:\ 5'[0-9][0-9]' | grep ^name | cut -d " " -f2-
For example, the output is like...
usernameone
hello
whoami
Then, I hope that I want to check a length of all user names.
Like this output...
11 //usernameone
5 //hello
6 //whoami
How can I get a length of pipeline code?
Given some command cmd that produces the list of users, you can do this pretty easily with xargs:
$ cat x
usernameone
hello
whoami
$ cat x | xargs -L 1 sh -c 'printf "%s //%s\n" "$(echo -n "$1" | wc -c)" "$1"' '{}'
11 //usernameone
5 //hello
6 //whoami
To get a piped command might not be possible, so here's a one liner that uses a split and a while loop to accomplish this:
-q user | grep -A 0 -B 2 -e uid:\ 5'[0-9][0-9]' | grep ^name | cut -d " " -f2-|tr " " "\n"|while read user; do echo $(echo $user|wc -c) '//'$user;done|tr "\n" " ";echo
This should give you an output in the desired format. I used user as a file hence the cat
i=0;for token in $(cat user); do echo -n "${#token} //$token";echo;i=$((i+1));done;echo;

producing a bash command with awk giving runaway string constant - awk

I'm using awk to parse /etc/hosts and produce a command which will format MapR for me. It's being done in a bash utility in Chef:
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
sleep 60
The command above should execute the following command:
/opt/mapr/server/configure.sh -C 10.32.237.251 -Z 10.32.237.251 -N mycluster --create-user -D /dev/xvdb
However, looking into my chef output I see:
==> namenode: Executing awk utility
==> namenode: awk: line 1: runaway string constant "/opt/mapr/ ...
The command never got executed in the MapR node... However when i execute it directly on the terminal it works nicely in the way it's supposed to be. What am I doing wrong?
I'm updating the question to show the complete bash script that executes that utility:
DISK_CONFIG=/home/ubuntu/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility\n"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
sleep 60
fi
Assuming you're using HEREDOC syntax in your bash resource:
bash "whatever" do
code <<-EOH
DISK_CONFIG=/tmp/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility\n"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb\n", ips, nn}' \
| bash
fi
EOH
end
this one leads to your error:
Executing awk utility
awk: line 4: runaway string constant "/opt/mapr/ ...
This is due to the \n in your comand (the one into the awk command is likely to be problematic too)
This resource should do (warning I did replace the DISK_CONFIG path for my tests):
bash "whatever" do
code <<-EOH
DISK_CONFIG=/tmp/disk_config
if [ -f $DISK_CONFIG ];
then
echo "File already exists"
else
echo "Executing awk utility"
touch $DISK_CONFIG
egrep '^[0-9]' /etc/hosts | grep -v 127.0.0.1 \
| awk 'NR==1{ips=$1}
NR>1{ips=ips ", " $1}
$2=="namenode"{nn=$1}
END{ printf "/opt/mapr/server/configure.sh -C %s -Z %s -N mycluster --create-user -D /dev/xvdb", ips, nn}' \
| bash
sleep 60
fi
EOH
end
The reason is that Chef already interpret the \n in the code and so awk see a string never ending (runaway).
As you pipe to bash you can omit the \n as the pipe will end the line.

Scripting HELP needed for finding LIVE HOSTS using bash in linux

How do I create a script that will identify all live hosts (responding to ping) using bin/bash in linux? My thoughts are to first have a
fping -A "some URL" // to get the IP address
then to set that to a var. then run a
fping -g "var" // having the ip address inserted by using a var.
Is there an easier way? If so, what would that script look like?
Not sure what you want to do and why you use fping, but if you just need the IP for a host you should use somthing like this:
getent ahostsv4 www.google.de | grep STREAM | head -n 1 | cut -d ' ' -f 1
getent ahostsv6 www.google.de | grep STREAM | head -n 1 | cut -d ' ' -f 1
getent hosts google.de | head -n 1 | cut -d ' ' -f 1
All commands will resolve an IP address if host still exist. If host points to CNAME it will also get the IP in that case.
Lets say, you have a text file containing all your hosts to test:
like: test-hosts.txt
www.server1.dom
www.server2.dom
server3.dom
1.3.5.7
then the command line (contains bash-script) will do it:
cat test-hosts.txt | xargs -i\{\} bash -c 'HOST="{}"; IP=$(getent ahostsv4 "{}" | grep STREAM | head -n 1 | cut -d " " -f 1); ping -c 1 -w 5 "$IP" >/dev/null 2>&1; RESULT="$?"; echo -e "Host: $HOST ($IP) \c"; case "$RESULT" in 0) echo "is online";; 1) echo "not responding after 5 secs";; *) if [[ "$IP" == "" ]]; then echo "has no resolveable address"; else echo "not availabe due to error [$RESULT]"; fi;; esac;'
and provide outputs like:
is online
not responding
no resolveable address
not available due error [code]

Bash formatting for nice ascii logs

I've written a function to do some logging on different aspects of the files, folder, etc and I 'm going to have this automatically email members of my staff. All this is fine but I wouldn't mind some pointers and help to make the formation nicer - this will get sent out via mailutils as part of the script.
Just looking for some nice spacing tips,etc
function report(){
lsb_release -a
echo "OS: $(uname -s)"
echo "ARCH: $(uname -m)"
echo "VER: $(uname -r)"
echo "Apache running "
ps aux|grep -i apache2
echo "Showing if SSL is open"
apache2ctl -M | grep ssl
echo "Space on local disk"
df -h
echo "Showing permissions for the web folders"
echo $'*** /var/www// ***'
ls -l /var/www | awk '{print $1 " " $3 " " $4 " " $9}'
ls -l /var/www/user1 | awk '{print $1 " " $3 " " $4 " " $9}'
ls -l /var/www/user2 | awk '{print $1 " " $3 " " $4 " " $9}'
echo "Showing network status"
ifconfig eth0
echo " DNS "
tail /etc/resolv.conf
echo "Current workspaces set up on the local server "
grep user2 /var/www/temp/text.txt | grep -E -o '[0-9]+[0-9]'
}
Try piping the "column" command instead of or after using awk or cut.
ls -l | awk '{print $1" "$3" "$4" "$9}' | tail -n +2 | column -t
Check out the difference between "mount" and "mount | column -t" or "df -PH and df -PH | column -t"
If you want to concatenate columns of info with the same number or relational fields you can use process redirection and "paste."
paste -d ' ' <(cat /sys/class/scsi_host/host*/device/fc_host:host*/port_name) \
<(cat /sys/class/scsi_host/host*/device/fc_host:host*/speed) \
<(cat /sys/class/scsi_host/host*/device/fc_host:host*/port_state)
0x218000e05a0001aa 4 Gbit Online
0x218000e05a2001aa 4 Gbit Online

Resources