Grep search for a string by pattern and then find part of this string inside of another file - linux

I have log file where I look for strings like:
tail -n 1000 -f logfile.log | grep -i "host"
and then I receive strings like these:
host2 %host-DEREG: host c459.cf00.1105 is deregistered on E0/1:60.
Could I choose mac addresses from these strings and look for strings with these mac addresses inside of another file?


There is no macaddress in your example
grep `tail -n 1000 -f logfile.log | grep -i "host" | grep -o "[a-f0-9][a-f0-9][a-f0-9][a-f0-9]\.[a-f0-9][a-f0-9][a-f0-9][a-f0-9]\.[a-f0-9][a-f0-9][a-f0-9][a-f0-9]"` anotherfile

Related

Extract IPs from Pcap Which are Matching the String

How can I extract IPs from a PCAP file whose packet has any of these hex strings.
--hex-string '|AF0DFT5F14|'
--hex-string '|AF0DFT5F25|'
--hex-string '|AF0DFT5F45|'
I just need the list of IPs.
I tried to search a lot on web but couldn't get any specific about this. I found out to extract all the IPs from a pcap file using the below command
( tcpdump -n -r m.pcap | grep IP | cut -f 3 "-d " | cut -f1-4 -d.; tcpdump -n -r m.pcap | grep IP | cut -f 5 "-d " | cut -f1-4 -d. ) | sort -u
Grep, awk, anything can do

Parsing nmap -oG output using sed

I have a logfile
...
Host: 111.222.121.123 (111.222.121.123.deploy.static.akamaitechnologies.com) Ports: 80/open/tcp//http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/, 443/open/tcp//ssl|http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/
Host: 1.2.3.4 () Ports: 80/open/tcp//http//cloudflare/, 443/open/tcp//ssl|https//cloudflare/, 2052/open/tcp//clearvisn?///, 2053/open/tcp//ssl|http//nginx/, 2082/open/tcp//infowave?///, 2083/open/tcp//ssl|http//nginx/, 2086/open/tcp//gnunet?///, 2087/open/tcp//ssl|http//nginx/, 2095/open/tcp//nbx-ser?///, 2096/open/tcp//ssl|http//nginx/, 8080/open/tcp//http-proxy//cloudflare/, 8443/open/tcp//ssl|https-alt//cloudflare/, 8880/open/tcp//cddbp-alt?///
Host: 2.3.4.5 (a104-96-1-61.deploy.static.akamaitechnologies.com) Ports: 53/open/tcp//domain//(unknown banner: 29571.61)/
...
I need to extract and convert IPs and http ports to the following format
1.2.3.4:80,443,2083
There are just two types of port fields in the logfile
80/open/tcp//http
2083/open/tcp//ssl|http
Tried to use sed but without success. I ended up with this dysfunctional command
cat ../host_ports.txt | sed -rn 's/Host: ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*?([0-9]{1,5}\/open\/tcp\/\/http|[0-9]{1,5}\/open\/tcp\/\/ssl\|http).*/\1 \2/p'

First handle the repeating ports, and next replace Host/Port to the desired format.
sed -r 's/(Ports:|,) ([0-9]*)[^,]*/\1\2/g;s/Host: ([^ ]*).*Ports:/\1:/' ../host_ports.txt
EDIT:
First I gave all ports of a line with http somewhere, now limit the result to ports with http in its description.
sed -nr 's/Ports: /, /;
s/, ([0-9]*)[^,]*http[^,]*/,\1/g;
s/,[^,]*\/[^,]*//g;
s/Host: ([^ ]*)[^,]*,/\1:/p' ../host_ports.txt

This script will do it for you, and you don't need sed :
#!/bin/bash
while read -r line; do
if echo $line | grep -q "http"; then
host=$(echo "$line" | grep -Po '(?<=^Host: )[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+')
ports=$(echo "$line" | grep -Po '[0-9]*((?=\/open\/tcp\/\/http)|(?=\/open\/tcp\/\/ssl\|http))' | tr '\n' ',')
echo "$host:${ports:0:-1}"
fi
done < ../log
The first grep will catch the IP address, with the help of Look behind. the -P is to use perl like regex, and the -o is to output only the matching string
The second regex is much like the first, but uses look after instead of look behind. It will only capture ports which are followed by /open/tcp//http or /open/tcp//ssl|http. The tr right after will replace newlines with commas.
the ${ports:0:-1} is just to eliminate the trailing comma.
Hope this helps!

Count lines of CLI output in linux

Hi have the following command:
lsscsi | grep HITACHI | awk '{print $6}'
I want that the output will be the number of lines of the original output.
For example, if the original output is:
/dev/sda
/dev/sdb
/dev/sdc
The final output will be 3.

Basically the command wc -l can be used to count the lines in a file or pipe. However, since you want to count the number of lines after a filter has been applied I would recommend to use grep for that:
lsscsi | grep -c 'HITACHI'
-c just prints the number of matching lines.
Another thing. In your example you are using grep .. | awk. That's a useless use of grep. It should be
lsscsi | awk '/HITACHI/{print $6}'

how to match hostname machine

please advice how to match the following hostname
machine hostname Should be according to the following rule
<a-z word ><number><a-z character/s>
real example
star1a
linux25as
machine2b
linux5a
solaris300C
unix9c
please advice how to machine these hostname with grep
I have for now this syntax
hostname | grep -c '[a-z][1-2][a-z]'
but these syntax not work on all my examples
on solaris the option egrep -E not works
hostname | grep -E '\b[a-z]+[0-9][a-z]+'
grep: illegal option -- E
Usage: grep -hblcnsviw pattern file . . .
Broken Pipe
try the second option ( on solaris machine ):
hostname
swu2a
hostname | grep "^[a-z]\+[0-9][a-z]\+$"
not matched!!!
I also try this:
hostname
swu2a
hostname | grep '[a-z]\+[0-9]\+[a-zA-Z]\+'
NOT MATCHED!!!

Here is an awk using same regex as the grep posted here uses.
awk '/[a-z]+[0-9]+[a-zA-Z]+/'
star1a
linux25as
machine2b
linux5a
solaris300C
unix9c
If you need to make sure there is nothing else in the line, only the words above, use:
awk '/^[a-z]+[0-9]+[a-zA-Z]+$/'
^ marks start of line.
$ marks end of line.

You can use the following pattern:
grep '^[a-z]\+[0-9]\+[a-zA-Z]\+$'
Note that you can use the the return value of grep to decide whether the pattern matches or not, you don't need to use the -c option. Like this:
if [ hostname | grep '^[a-z]\+[0-9]\+[a-zA-Z]\+$' >/dev/null 2>&1 ] ; then
echo "host name OK"
fi

Grep - returning both the line number and the name of the file

I have a number of log files in a directory. I am trying to write a script to search all the log files for a string and echo the name of the files and the line number that the string is found.
I figure I will probably have to use 2 grep's - piping the output of one into the other since the -l option only returns the name of the file and nothing about the line numbers. Any insight in how I can successfully achieve this would be much appreciated.
Many thanks,
Alex

$ grep -Hn root /etc/passwd
/etc/passwd:1:root:x:0:0:root:/root:/bin/bash
combining -H and -n does what you expect.
If you want to echo the required informations without the string :
$ grep -Hn root /etc/passwd | cut -d: -f1,2
/etc/passwd:1
or with awk :
$ awk -F: '/root/{print "file=" ARGV[1] "\nline=" NR}' /etc/passwd
file=/etc/passwd
line=1
if you want to create shell variables :
$ awk -F: '/root/{print "file=" ARGV[1] "\nline=" NR}' /etc/passwd | bash
$ echo $line
1
$ echo $file
/etc/passwd

Use -H. If you are using a grep that does not have -H, specify two filenames. For example:
grep -n pattern file /dev/null

My version of grep kept returning text from the matching line, which I wasn't sure if you were after... You can also pipe the output to an awk command to have it ONLY print the file name and line number
grep -Hn "text" . | awk -F: '{print $1 ":" $2}'

Resources