Bash - Grepping nc output in a for loop - linux

I'm currently trying to grep the output for a nc in a bash single line loop, to only show me lines with openstring within. I've already tried --line-buffered with no success. Can anybody give me a light about what I'm doing wrong? Here goes the command:
root#kali:~# for host in $(seq 200 254); do nc -nvv -w 1 -z 192.168.15.$host 80 | grep --line-buffered "open"; done

Redirect stderr to stdout so you can grep it.
You can do that by putting 2>&1 anywhere between do and |.

Related

Why is the shell output txt file empty?

Kindly assist. I am working on a script that will perform a telnet test to a specific ip address on a specific TCP port and below is my script.
#! /bin/sh
nc -z -v -w5 192.168.88.55 3389 | tee results.txt
During execution, a "results.txt" file is created but it is empty. I want it to have the output of the script after execution.
I have managed to resolve it by making the below modifications to the script
#! /bin/sh
nc -z -v -w5 192.168.88.55 3389 2>&1 | tee results.txt
sleep 5
exit
It is now able to write the output to the results.txt file.
Thank you.

run a command conditionally with netcat and grep

I need netcat to listen on incomming HTTP requests, and depending on the request, I need to run a script.
So far I have this;
netcat -lk 12345 | grep "Keep-Alive"
So every time netcat recieves a package that contains a "keep-alive", I need to fire a script.
It needs to run in the crontab...
Thanks for your help!
How about this?
#!/bin/bash
netcat -lk -p 12345 | while read line
do
match=$(echo $line | grep -c 'Keep-Alive')
if [ $match -eq 1 ]; then
echo "Here run whatever you want..."
fi
done
Replace the "echo" command with the script you want to execute.
How about:
#!/bin/bash
netcat -lk -p 12345 | grep 'Keep-Alive' | while read unused; do
echo "Here run whatever you want..."
done
Or
#!/bin/bash
netcat -lk -p 12345 | sed -n '/Keep-Alive/s/.*/found/p' | xargs -n 1 -I {} do_something
# replace do_something by your command.
Depending on what you have on the client side, it might sit there waiting for netcat/the server to respond.
I did a similar thing to the above but used
while true do
netcat -l 1234 < 404file.txt
Where 404file.txt has HTTP/1.1 404 FILE NOT FOUND
This disconnects the client and since netcat has no 'k' it terminates and restarts because of the while true, all ready to receive and send the 404 again.

pipe or send strings with socat/netcat with a pause

i capture unbuffered codes from ttyUSB0 as an string though hexdump'ing
/usr/bin/stdbuf -o0 /bin/busybox hexdump -v -e '12/1 "%02x" "\n"' /dev/ttyUSB0
with example-output like this (without quotes): "20303830303e353a30373220"
Now i am helpless:
I want a loop to netcat/socat the first capture (maybe with read ?) to another host, than a slight pause e.g 10 sec before the next capture is send.
i searched for an elegant line with socat, but all my piping-examples failed. Seems some problems of understanding the subshell-processes.. and lack of experience. anybody a hint?
The following works for me with my mouse as input:
sudo stdbuf -o0 hexdump -v -e '12/1 "%02x" "\n"' /dev/input/by-id/usb-Logitech_USB_Receiver-if02-mouse | (while true; do read buf; echo $buf; sleep 1; done) | nc localhost 8008
I believe there could be a better implementation of what I do in the while-loop...
I'm able to get the input from my mouse with a previously-started
nc -l localhost 8008

What is meant by 'output to stdout'

New to bash programming. I am not sure what is meant by 'output to stdout'. Does it mean print out to the command line?
If I have a simple bash script:
#!/bin/bash
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello'
it outputs a string to the terminal. Does this mean it's 'outputting to stdout' ?
Thanks
Yes, stdout is the terminal (unless it's redirected to a file using the > operator or into the stdin of another process using |)
In your specific example, you're actually redirecting using | grep ... through grep then to the terminal.
Every process on a Linux system (and most others) has at least 3 open file descriptors:
stdin (0)
stdout (1)
stderr (2)
Regualary every of this file descriptors will point to the terminal from where the process was started. Like this:
cat file.txt # all file descriptors are pointing to the terminal where you type the command
However, bash allows to modify this behaviour using input / output redirection:
cat < file.txt # will use file.txt as stdin
cat file.txt > output.txt # redirects stdout to a file (will not appear on terminal anymore)
cat file.txt 2> /dev/null # redirects stderr to /dev/null (will not appear on terminal anymore
The same is happening when you are using the pipe symbol like:
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello'
What is actually happening is that the stdout of the wget process (the process before the | ) is redirected to the stdin of the grep process. So wget's stdout isn't a terminal anymore while grep's output is the current terminal. If you want to redirect grep's output to a file for example, then use this:
wget -q http://192.168.0.1/test -O - | grep -m 1 'Hello' > output.txt
Unless redirected, standard output is the text terminal which initiated the program.
Here's a wikipedia article: http://en.wikipedia.org/wiki/Standard_streams#Standard_output_.28stdout.29

How can I write a Linux bash script that tells me which computers are ON in my LAN?

How can I write a Linux Bash script that tells me which computers are ON in my LAN?
It would help if I could give it a range of IP addresses as input.
I would suggest using nmap's ping-scan flag,
$ nmap -sn 192.168.1.60-70
Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2009-04-09 20:13 BST
Host machine1.home (192.168.1.64) appears to be up.
Host machine2.home (192.168.1.65) appears to be up.
Nmap finished: 11 IP addresses (2 hosts up) scanned in 0.235 seconds
That said, if you want to write it yourself (which is fair enough), this is how I would do it:
for ip in 192.168.1.{1..10}; do ping -c 1 -t 1 $ip > /dev/null && echo "${ip} is up"; done
..and an explanation of each bit of the above command:
Generating list of IP addresses
You can use the {1..10} syntax to generate a list of numbers, for example..
$ echo {1..10}
1 2 3 4 5 6 7 8 9 10
(it's also useful for things like mkdir {dir1,dir2}/{sub1,sub2} - which makes dir1 and dir2, each containing sub1 and sub2)
So, to generate a list of IP's, we'd do something like
$ echo 192.168.1.{1..10}
192.168.1.1 192.168.1.2 [...] 192.168.1.10
Loops
To loop over something in bash, you use for:
$ for thingy in 1 2 3; do echo $thingy; done
1
2
3
Pinging
Next, to ping.. The ping command varies a bit with different operating-systems, different distributions/versions (I'm using OS X currently)
By default (again, on the OS X version of ping) it will ping until interrupted, which isn't going to work for this, so ping -c 1 will only try sending one packet, which should be enough to determine if a machine is up.
Another problem is the timeout value, which seems to be 11 seconds on this version of ping.. It's changed using the -t flag. One second should be enough to see if a machine on the local network is alive or not.
So, the ping command we'll use is..
$ ping -c 1 -t 1 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
--- 192.168.1.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
Checking ping result
Next, we need to know if the machine replied or not..
We can use the && operator to run a command if the first succeeds, for example:
$ echo && echo "It works"
It works
$ nonexistantcommand && echo "This should not echo"
-bash: nonexistantcommand: command not found
Good, so we can do..
ping -c 1 -t 1 192.168.1.1 && echo "192.168.1.1 is up!"
The other way would be to use the exit code from ping.. The ping command will exit with exit-code 0 (success) if it worked, and a non-zero code if it failed. In bash you get the last commands exit code with the variable $?
So, to check if the command worked, we'd do..
ping -c 1 -t 1 192.168.1.1;
if [ $? -eq 0 ]; then
echo "192.168.1.1 is up";
else
echo "ip is down";
fi
Hiding ping output
Last thing, we don't need to see the ping output, so we can redirect stdout to /dev/null with the > redirection, for example:
$ ping -c 1 -t 1 192.168.1.1 > /dev/null && echo "IP is up"
IP is up
And to redirect stderr (to discard the ping: sendto: Host is down messages), you use 2> - for example:
$ errorcausingcommand
-bash: errorcausingcommand: command not found
$ errorcausingcommand 2> /dev/null
$
The script
So, to combine all that..
for ip in 192.168.1.{1..10}; do # for loop and the {} operator
ping -c 1 -t 1 192.168.1.1 > /dev/null 2> /dev/null # ping and discard output
if [ $? -eq 0 ]; then # check the exit code
echo "${ip} is up" # display the output
# you could send this to a log file by using the >>pinglog.txt redirect
else
echo "${ip} is down"
fi
done
Or, using the && method, in a one-liner:
for ip in 192.168.1.{1..10}; do ping -c 1 -t 1 $ip > /dev/null && echo "${ip} is up"; done
Problem
It's slow.. Each ping command takes about 1 second (since we set the -t timeout flag to 1 second). It can only run one ping command at a time.. The obvious way around this is to use threads, so you can run concurrent commands, but that's beyond what you should use bash for..
"Python threads - a first example" explains how to use the Python threading module to write a multi-threaded ping'er.. Although at that point, I would once again suggest using nmap -sn..
In the real world, you could use nmap to get what you want.
nmap -sn 10.1.1.1-255
This will ping all the addresses in the range 10.1.1.1 to 10.1.1.255 and let you know which ones answer.
Of course, if you in fact want to do this as a bash exercise, you could run ping for each address and parse the output, but that's a whole other story.
Assuming my network is 10.10.0.0/24, if i run a ping on the broadcast address like
ping -b 10.10.0.255
I'll get an answer from all computers on this network that did not block their ICMP ping port.
64 bytes from 10.10.0.6: icmp_seq=1 ttl=64 time=0.000 ms
64 bytes from 10.10.0.12: icmp_seq=1 ttl=64 time=0.000 ms
64 bytes from 10.10.0.71: icmp_seq=1 ttl=255 time=0.000 ms
So you just have to extract the 4th column, with awk for example:
ping -b 10.10.0.255 | grep 'bytes from' | awk '{ print $4 }'
10.10.0.12:
10.10.0.6:
10.10.0.71:
10.10.0.95:
Well, you will get duplicate, and you may need to remove the ':'.
EDIT from comments :
the -c option limits the number of pings
since the script will end, we can also limit ourself on unique IPs
ping -c 5 -b 10.10.0.255 | grep 'bytes from' | awk '{ print $4 }' | sort | uniq
There is also fping:
fping -g 192.168.1.0/24
or:
fping -g 192.168.1.0 192.168.1.255
or show only hosts that are alive:
fping -ag 192.168.1.0/24
It pings hosts in parallel so the scan is very fast. I don't know a distribution which includes fping in its default installation but in most distributions you can get it through the package manager.
Also using the "ping the broadcast address" method pointed out by chburd, this pipe should do the trick for you:
ping -c 5 -b 10.11.255.255 | sed -n 's/.* \([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\).*/\1/p' | sort | uniq
Of course, you'd have to change the broadcast address to that of your network.
Just for fun, here's an alternate
#!/bin/bash
nmap -sP 192.168.1.0/24 > /dev/null 2>&1 && arp -an | grep -v incomplete | awk '{print$2}' | sed -e s,\(,, | sed -e s,\),,
If you're limiting yourself to only having the last octet changing, this script should do it. It should be fairly obvious how to extend it from one to multiple octets.
#! /bin/bash
BASE=$1
START=$2
END=$3
counter=$START
while [ $counter -le $END ]
do
ip=$BASE.$counter
if ping -qc 2 $ip
then
echo "$ip responds"
fi
counter=$(( $counter + 1 ))
done
ip neighbor
arp -a
Arpwatch
As other posters pointed out, nmap is the way to go, but here's how to do the equivalent of a ping scan in bash. I wouldn't use the broadcast ping, as a lot of systems are configured not to respond to broadcast ICMP nowadays.
for i in $(seq 1 254); do
host="192.168.100.$i"
ping -c 1 -W 1 $host &> /dev/null
echo -n "Host $host is "
test $? -eq 0 && echo "up" || echo "down"
done
#!/bin/bash
#Get the ip address for the range
ip=$(/sbin/ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}' | cut -d"." -f1,2,3)
# ping test and list the hosts and echo the info
for range in $ip ; do [ $? -eq 0 ] && ping -c 1 -w 1 $range > /dev/null 2> /dev/null && echo "Node $range is up"
done
Although an old question, it still seems to be important (at least important enough for me to deal with this). My script relies on nmap too, so nothing special here except that ou can define which interface you want to scan and the IP Range is created automagically (at least kind of).
This is what I came up with
#!/bin/bash
#Script for scanning the (local) network for other computers
command -v nmap >/dev/null 2>&1 || { echo "I require nmap but it's not installed. Aborting." >&2; exit 1; }
if [ -n ""$#"" ]; then
ip=$(/sbin/ifconfig $1 | grep 'inet ' | awk '{ print $2}' | cut -d"." -f1,2,3 )
nmap -sP $ip.1-255
else
echo -e "\nThis is a script for scanning the (local) network for other computers.\n"
echo "Enter Interface as parameter like this:"
echo -e "\t./scannetwork.sh $(ifconfig -lu | awk '{print $2}')\n"
echo "Possible interfaces which are up are: "
for i in $(ifconfig -lu)
do
echo -e "\033[32m \t $i \033[39;49m"
done
echo "Interfaces which could be used but are down at the moment: "
for i in $(ifconfig -ld)
do
echo -e "\033[31m \t $i \033[39;49m"
done
echo
fi
One remark: This script is created on OSX, so there might be some changes to linux environments.
If you want to provide a list of hosts it can be done with nmap, grep and awk.
Install nmap:
$ sudo apt-get install nmap
Create file hostcheck.sh like this:
hostcheck.sh
#!/bin/bash
nmap -sP -iL hostlist -oG pingscan > /dev/null
grep Up pingscan | awk '{print $2}' > uplist
grep Down pingscan | awk '{print $2}' > downlist
-sP: Ping Scan - go no further than determining if host is online
-iL : Input from list of hosts/networks
-oG : Output scan results in Grepable format, to the given filename.
/dev/null : Discards output
Change the access permission:
$ chmod 775 hostcheck.sh
Create file hostlist with the list of hosts to be checked (hostname or IP):
hostlist (Example)
192.168.1.1-5
192.168.1.101
192.168.1.123
192.168.1.1-5 is a range of IPs
Run the script:
./hostcheck.sh hostfile
Will be generated files pingscan with all the information, uplist with the hosts online (Up) and downlist with the hosts offline (Down).
uplist (Example)
192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4
192.168.1.101
downlist (Example)
192.168.1.5
192.168.1.123
Some machines don't answer pings (e.g. firewalls).
If you only want the local network you can use this command:
(for n in $(seq 1 254);do sudo arping -c1 10.0.0.$n & done ; wait) | grep reply | grep --color -E '([0-9]+\.){3}[0-9]+'
Explanations part !
arping is a command that sends ARP requests. It is present on most of linux.
Example:
sudo arping -c1 10.0.0.14
the sudo is not necessary if you are root ofc.
10.0.0.14 : the ip you want to test
-c1 : send only one request.
&: the 'I-don't-want-to-wait' character
This is a really useful character that give you the possibility to launch a command in a sub-process without waiting him to finish (like a thread)
the for loop is here to arping all 255 ip addresses. It uses the seq command to list all numbers.
wait: after we launched our requests we want to see if there are some replies. To do so we just put wait after the loop.
wait looks like the function join() in other languages.
(): parenthesis are here to interpret all outputs as text so we can give it to grep
grep: we only want to see replies. the second grep is just here to highlight IPs.
hth
Edit 20150417: Maxi Update !
The bad part of my solution is that it print all results at the end. It is because grep have a big enough buffer to put some lines inside.
the solution is to add --line-buffered to the first grep.
like so:
(for n in $(seq 1 254);do sudo arping -c1 10.0.0.$n & done ; wait) | grep --line-buffered reply | grep --color -E '([0-9]+\.){3}[0-9]+'
#!/bin/bash
for ((n=0 ; n < 30 ; n+=1))
do
ip=10.1.1.$n
if ping -c 1 -w 1 $ip > /dev/null 2> /dev/null >> /etc/logping.txt; then
echo "${ip} is up" # output up
# sintax >> /etc/logping.txt log with .txt format
else
echo "${ip} is down" # output down
fi
done
The following (evil) code runs more than TWICE as fast as the nmap method
for i in {1..254} ;do (ping 192.168.1.$i -c 1 -w 5 >/dev/null && echo "192.168.1.$i" &) ;done
takes around 10 seconds, where the standard nmap
nmap -sP 192.168.1.1-254
takes 25 seconds...
Well, this is part of a script of mine.
ship.sh 🚢 A simple, handy network addressing 🔎 multitool with plenty of features 🌊
Pings network, displays online hosts on that network with their local IP and MAC address
It doesn't require any edit. Needs root permission to run.
GOOGLE_DNS="8.8.8.8"
ONLINE_INTERFACE=$(ip route get "${GOOGLE_DNS}" | awk -F 'dev ' 'NR == 1 {split($2, a, " "); print a[1]}')
NETWORK_IP=$(ip route | awk "/${ONLINE_INTERFACE}/ && /src/ {print \$1}" | cut --fields=1 --delimiter="/")
NETWORK_IP_CIDR=$(ip route | awk "/${ONLINE_INTERFACE}/ && /src/ {print \$1}")
FILTERED_IP=$(echo "${NETWORK_IP}" | awk 'BEGIN{FS=OFS="."} NF--')
ip -statistics neighbour flush all &>/dev/null
echo -ne "Pinging ${NETWORK_IP_CIDR}, please wait ..."
for HOST in {1..254}; do
ping "${FILTERED_IP}.${HOST}" -c 1 -w 10 &>/dev/null &
done
for JOB in $(jobs -p); do wait "${JOB}"; done
ip neighbour | \
awk 'tolower($0) ~ /reachable|stale|delay|probe/{printf ("%5s\t%s\n", $1, $5)}' | \
sort --version-sort --unique

Resources