Multithreading ping inside BASH script - multithreading

Question

I've had a look around here and there only seems to be one question raised in regards to the multithreading of ping inside a BASH environment.
Code
data=$(<~/Desktop/hosts.txt)
for line in $data
do
ping -c1 $line &> /dev/null &
echo $line is reachable ||
echo $line is down
done
The issue is that it cycles through the IP address range inside the file and reports all of the IP addresses as reachable when in fact only 3 out of /24 are reachable...
I've attempted to slow the scan down by adding -W 1 to the ping but this results in substandard performance and necessitates the use of multithreading

Answer 1

As coded you've got 2 separate, unrelated commands:
# run `ping` and place in background
ping -c1 $line &
and
# regardless of what the `ping` command does ... always
# print 'reachable` and if that fails print `down`
echo $line is reachable || echo $line is down
I'm assuming what you really want is to use the results of the ping (success or failure) to determine which echo command to issue; in this case you need to replace the single & with a double `&&' like such:
ping -c1 $line &&
echo $line is reachable ||
echo $line is down
While this should provide the conditional output, your pings are now performed one-at-a-time in the foreground.
If you want to run your pings in the background (ie, 'multithreaded') and also perform the conditional printing, you could place the ping/echo/echo code inside a function, then in your for loop make a call to the function ... making sure to place the function call in the background, eg:
Sample list of sites to ping:
$ cat pinglist
www.yahoo.com
www.google.com
www.peanut_butter.and_jelly
www.bbc.co.uk
www.not_today_your.not
Proposed script:
$ cat pinger
#!/bin/bash
function pingit () {
target=${1}
# send stdout/stderr to /dev/null since all we need is the return code
ping -c1 ${target} >/dev/null 2>&1 &&
echo ${target} is reachable ||
echo ${target} is down
}
data=$(<pinglist)
for line in ${data}
do
# call our function and place the call in the background
pingit ${line} &
done
# wait for all outstanding background jobs to complete before continuing
wait
# [optional] let operator know we're done.
echo "Completed."
Some sample runs, keeping in mind that the order of the output messages will vary based on how quickly each ping command completes:
$ pinger
www.yahoo.com is reachable
www.google.com is reachable
www.peanut_butter.and_jelly is down
www.bbc.co.uk is reachable
www.not_today_your.not is down
Completed.
$ pinger
www.peanut_butter.and_jelly is down
www.not_today_your.not is down
www.google.com is reachable
www.bbc.co.uk is reachable
www.yahoo.com is reachable
Completed.
We'll run the test again and sort the output, ignoring the 'Completed.' message:
$ pinger | egrep -v "Completed" | sort
www.bbc.co.uk is reachable
www.google.com is reachable
www.not_today_your.not is down
www.peanut_butter.and_jelly is down
www.yahoo.com is reachable
!!! Keep in mind that if you've got a long list of hosts to ping and you're running this script on a smallish box, your sysadmin might not be too happy if you kick off 10's of thousands of simultaneous ping calls ... and heaven forbid you make a coding mistake that spawns an infinite loop of ping commands.
This type of misbehavior can be coded for (eg, limiting the number of outstanding ping calls, or sleeping for a couple seconds after every X number of ping calls, etc), but I'll leave that coding as homework ;-)
If you don't want to bother with a function (eg, you're doing a quick command line loop/ping), you could wrap the ping/echo/echo in parentheses and run this construct in the background, though you'll want to decide how to handle the job-related messages, eg:
$ data=$(<pinglist)
$ for line in ${data}
do
(ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) &
done
[1] 12212
[2] 12213
[3] 12214
[4] 12215
[5] 12217
www.google.com is reachable
www.peanut_butter.and_jelly is down
www.not_today_your.not is down
www.bbc.co.uk is reachable
www.yahoo.com is reachable
[5] + Done for line in ${data};do; (ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & 2>/dev/null;done
[4] - Done for line in ${data};do; (ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & 2>/dev/null;done
[3] Done for line in ${data};do; (ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & 2>/dev/null;done
[2] Done for line in ${data};do; (ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & 2>/dev/null;done
[1] Done for line in ${data};do; (ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & 2>/dev/null;done
One option (kludge?) for masking the job related messages requires wrapping our ping/echo/echo/& in a set of braces and directing stderr to /dev/null ...
$ data=$(<pinglist)
$ for line in ${data}
do
{(ping -c1 ${line} >/dev/null 2>&1 && echo ${line} is reachable || echo ${line} is down) & } 2>/dev/null
done
www.peanut_butter.and_jelly is down
www.not_today_your.not is down
www.google.com is reachable
www.yahoo.com is reachable
www.bbc.co.uk is reachable
... though this could mask some error messages that you may be interested in, ymmv.

Answer 2

I would let GNU Parallel do them all in parallel for me, probably 32 at a time:
parallel -j 32 -a hosts.txt 'ping -c1 {} > /dev/null 2>&1 && echo {} is available || echo {} is down'