Is there a way using bash /sed/awk to extract IP addresses from each line of log file to show IP host conversations or connection attempts?
Example of log file:
*Teardown TCP connection -1948864210 for Node14:110.98.8.41 to Net_N:10.98.35.28 duration 0:02:01 bytes 0 SYN Timeout
Built outbound TCP connection -1948863670 for Net11:10.10.2.5 (10.10.2.5 to Net01:10.9.15.2 (10.9.15.2)
Deny tcp src Node22:10.128.4.201/2254 dst outside:10.198.2.1/5560 by access-group "111"*
Required output, listing IP conversation/connection attempt:
110.98.8.41 10.98.35.28
10.10.2.5 10.9.15.2
10.128.4.201 10.198.2.1
I have tried using grep to strip out the IPs:
cat log.file | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | sort | uniq
But the output just lists single IP addresses and not line-by-line IP conversations
Any help is appreciated..
To print IPs next to each other, try the below command:
cat first | grep -o '[0-9]\{0,3\}\.[0-9]\{0,3\}\.[0-9]\{0,3\}\.[0-9]\{0,3\}' | awk 'NR%2{printf $0"\t";next;}1'
You could do this:
octet='\<(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]?)\>'
ip="$octet\\.$octet\\.$octet\\.$octet"
grep -Eo "$ip" file | paste - -
Or, use a wheel that's already been invented
perl -MRegexp::Common -lne '$,=" "; print /$RE{net}{IPv4}/g' file
Related
I'm new to security and was working on a problem where I need to figure out the external DNS used to resolve names to IP. I can filter how to look for dns traffic but how do I figure out the external DNS used to resolve addresses?
There can be several DNS resolvers used. If you know they all listen on standard port UDP/53, you can simply retrieve the destination IP addresses:
$ tshark -r tmp.pcap -T fields -e ip.dst "udp.dstport eq 53" | sort | uniq -c
31 127.0.0.1
3 192.168.1.3
The above will give you the list of destination IP addresses for UDP/53 packets. In my case, I have a local resolver (127.0.0.1) which only calls the above resolver (192.168.1.13) for records that are not cached. Thus, most requests only go to the local resolver (31 out of 34).
It's also fairly common for DNS resolvers to listen on TCP/53. You can use the following command to select these requests as well:
tshark -r capture.pcap -T fields -e ip.dst "udp.dstport eq 53 or tcp.dstport eq 53" | sort | uniq -c
You can also apply filter packets while capturing, to avoid saving unnecessary packets:
tshark -i any -T fields -e ip.dst "dst port 53" > capture.txt
cat capture.txt | sort | uniq -c
I'm very new to linux and bash. I'm trying to find a domain name for an ip address. When I use nslookup I have a bunch of lines like this
nslookup 204.228.150.3
Output
Server: 198.60.22.2
Address: 198.60.22.2#53
Non-authoritative answer:
3.150.228.204.in-addr.arpa name = www.computerhope.com.
Authoritative answers can be found from:
150.228.204.in-addr.arpa nameserver = ns.xmission.com.
150.228.204.in-addr.arpa nameserver = ns1.xmission.com.
150.228.204.in-addr.arpa nameserver = ns2.xmission.com.
ns.xmission.com Internet address = 166.70.254.2
ns1.xmission.com Internet address = 204.228.159.2
ns2.xmission.com Internet address = 207.135.133.2
I only want to print www.computerhope.com in the second line. How do I do this?
Also I tried to use host command as well. It looks cleaner.
206.153.126.75.in-addr.arpa domain name pointer www.cyberciti.biz.
How do I only print www.cyberciti.biz when I use host command ?
PLease help
$ host 75.126.153.202 |sed -e 's/.* //'
www.cyberciti.biz.
nslookup 204.228.150.3 | \
grep "in-addr.arpa" | \
cut -d '=' -f 2 | \
tr -d '[:blank:]'
With grep you find the line containing the address. With the cut command you split the line by the "=" sign. With tr you remove any remaining spaces.
The command does not have any error handling in case the address is unknown.
If you would provide more information on what you want to achieve maybe other solutions would come up. :-)
Another way to do this is to pipe the host command to awk:
$ host 75.126.153.202 | awk '{print $NF}'
NF is a built-in awk variable which prints the last column of the output of the host command, which in this case is the domain name.
You can also pipe nslookup to awk:
$ nslookup 75.126.153.202 | awk '$2 ~ /^name$/{print $NF}'
awk returns the last column of the line containing the name in its 2nd column.
I have many configuration files ,
the line that start with LINE word have IP address
My target to read the line that start with LINE word from the file and print only the IP address
The problem is that IP address can be in any field in the line so I can’t capture the IP according to field number
example
grep LINE file1.txt
LINE /home/Ariate/run.pl "Voda STS 4 Test - " "102841" && ssh 17.77.170.130 -p 2022
grep LINE file2.txt
LINE /home/Ariate/run.pl 137.77.170.30 "Voda STS 4 Test - " "102841" && ssh ACTIVE
please advice how to capture the IP address from the line ( solution can be also with perl one liner )
expected results
echo $IP_FROM_LINE
17.77.170.130
echo $IP_FROM_LINE
137.77.170.30
perl -MRegexp::Common=net -lne 'print $1 if /^LINE.*\b($RE{net}{IPv4})/'
Using this grep -oE:
grep -oE '\d+\.\d+\.\d+\.\d+' file
17.77.170.130
137.77.170.30
OR else:
grep -oP '\d+\.\d+\.\d+\.\d+' file
The following will get you the desired IP addresses:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file
To place the result in a variable as request, you'll need to iterate of the results as follows:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file |
while read IP_FROM_LINE ; do
echo $IP_FROM_LINE
done
grep -oE '[0-9]{2,3}(\.[0-9]{2,3}){3}'
matches
17.77.170.130
137.77.170.30
or
grep -oP '\d{2}(\.\d{2}){3}'
if your grep supports -P option.
both of them works with the data you have given.
But if you want really worried of what to be matched, use
grep -Eo '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)'
which would match excat ip addresses.
I'm maintaining a bash script that runs on RHEL 5. Given an domain name, I want to ensure that there is an interface on the current machine listening to the expected IP address.
# Check that the IP is bound to this host, to avoid running on the wrong machine.
IP=$(dig +short ${EXPECTED_SUBDOMAIN}.example.com | tail -n 1)
if ! $(/sbin/ifconfig 2>/dev/null | /bin/grep -q $IP) ; then
echo "IP for ${EXPECTED_SUBDOMAIN} ($IP) doesn't appear to be on this host"
exit 1
fi
This breaks when IP addresses are substrings of each other. If my expected IP address is 1.2.3.4, it doesn't exit if the machine has an IP address (or even subnet mask!) of 11.2.3.4 or 1.2.3.40.
The only solution I can see is grep "inet addr:${IP} " but it seems dirty to search for other text.
Is there are more robust way to find out the IP addresses of interfaces (without getting confused by things like subnet masks) and compare them? Is there a better approach?
use word boundary in your grep:
if ! $(/sbin/ifconfig 2>/dev/null | /bin/grep -qE "\<$IP\>" ; then
...
Try this:
/sbin/ip addr | grep -Po 'inet \K.*?(?=(/| ))'| /bin/grep -q "^$IP$"
Matching the beginning and end of string should solve your problem if the input is also filtered.
PS. I find ip addr better than ifconfig for vitual ips.
I have a bash script that runs on a variety of different Ubuntu Linux machines. Its job is to find out the LAN IPv4 address of the localhost.
The script is using
ip addr show eth0 | sed -n '/inet /{s/^.*inet \([0-9.]\+\).*$/\1/;p}'
which is fine, but some machines for some reason use eth1 instead of eth0. I would like to be able to discover the LAN iface name, so I can substitute it in here instead of eth0.
Of course, if you can come up with a different oneliner that does the same thing, all good.
The main NIC will usually have a default route. So:
ip -o -4 route show to default
The NIC:
ip -o -4 route show to default | awk '{print $5}'
The gateway:
ip -o -4 route show to default | awk '{print $3}'
Unlike ifconfig, ip has a consistent & parsable output. It only works on Linux; it won't work on other Unixen.
Not sure if this helps, but it seems that ip route get will show which interface it uses to connect to a remote host.
ubuntu#ip-10-40-24-21:/nail/srv/elasticsearch$ ip route get 8.8.8.8
8.8.8.8 via <gateway address> dev eth0 src <eth0 IP Address>
of course you could automate that in shell script with something like,
ip route get 8.8.8.8 | awk '{ print $NF; exit }'
Most recenttly systemd/udev has automatically started to assign interface names for all local Ethernet, WLAN and WWAN interfaces to something that we're all accustomed to . This is a departure from the traditional interface naming scheme ("eth0", "eth1", "wlan0", ...) .. now we have to check first what the local interface name is before we can use it while previously we it was a pretty accurate guess that "eth0" was the right name. What you're asking for is the network NAME .. Here's a small script to solve the problem
Use "ip route get 8.8.8.8 " to figure out which ACTIVE interface has the route to internet ( or currently being used )
Output should look like :
8.8.4.4 via 10.10.1.1 dev enp0s3 src 10.10.1.118
cache
Use awk to print the 5th text block for Interface NAME
]# ip route get 8.8.8.8 | awk -- '{print $5}'
Output : enp0s3
Use awk to print the 7th text block for Interface Address
]# ip route get 8.8.8.8 | awk -- '{print $7}'
Output : 10.10.1.118
How about searching for the string inet and brd (for broadcast)? That would give you:
ip addr show|egrep '^ *inet'|grep brd|awk -- '{ print $2; }'|sed -e 's:/[0-9]*$::'
Note that I'm using more commands than necessary; you can probably achieve the same thing with sed and a more complex regexp but I prefer a command that makes it obvious by which steps I arrive at the result.
If you want to run it in a single command, I suggest to try awk:
ip addr show|awk -- '$1 == "inet" && $3 == "brd" { split($2,a,"/"); print a[1]; }'
which isn't much longer than the sed version but more readable.
+1 Slightly more readable:
ip addr show | awk '$1 == "inet" && $3 == "brd" { sub (/\/.*/,""); print $2 }'
Believe it or not, there is no standard, easy way to get this information. There is no standard give me the current IP and Interface Name command. There isn't even a standard format for the information returned by ifconfig.
I was going to recommend forgoing pure shell and go with a scripting language like Python where you can do this:
import socket
socket.gethostbyname(socket.gethostname())
Except it doesn't work on most Linux systems because there's usually an entry in the /etc/host file pointing to 127.0.0.1, the loopback address. Perl has the same issues.
You have a firm grasp on the scripting involved, and you've seen the issues. The only thing I can recommend is to test this on each machine you're going to run it on, and see what pops out. There isn't going to be a general purpose one liner that works with all operating systems, or even with different systems on the same operating system because of the way the network is setup and the way interfaces may be named by each location.
I'd still like to know if there was an easier way to do this, but this is my workaround: I know the LAN subnet, so...
ip addr show | grep "inet 10.67.5." \
| sed -n '/inet /{s/^.*inet \([0-9.]\+\).*$/\1/;p}'
1) This one print only interface names (I needed that for handing upcoming and downcoming PPP links):
for i in $( ifconfig | grep 'ppp' | awk '{print $1}' );
do
printf "$i "; ## Or echo
done
Result:
ppp0 ppp1 ppp2
2) This one prints interface names and IP
declare -a IPADDR
index=0
for i in $( ifconfig | grep 'inet addr' | awk '{print $2}'| sed 's#addr:##g' );
do
IPADDR[$index]=$i
let "index += 1"
done
index=0
for i in $( ifconfig | grep 'ppp' | awk '{print $1}' );
do
echo $i
let "index += 1"
done
Result:
ppp0 addr:IP
ppp1 addr:IP
ppp2 addr:IP