I have this file:
this is line 1 192.168.1.1
this is line 2 192.168.1.2
this is line 3 192.168.1.2
this is line 4 192.168.1.1
this is line 5 192.168.1.2
I would like to get, in a bash script, all lines (with tabs) which contains, for example, the pattern "192.168.1.1". By this way, I would get:
this is line 1 192.168.1.1
this is line 4 192.168.1.1
But i don't want this result:
this is line 1 192.168.1.1 this is line 4 192.168.1.1
I tried it with sed without success:
var='192.168.1.1'
body=`sed -n -e '/$var/p' $file`
echo $body
Thanks beforehand!
awk to the rescue!
$ awk -v var='192.168.1.1' '$NF==var' file
this is line 1 192.168.1.1
this is line 4 192.168.1.1
Following sed may help you on same.
var="your_ip"
sed -n "/$var/p" Input_file
Or in awk following will help on same.
var="your_ip"
awk -v var="$var" 'var==$NF' Input_file
A simple grep command could do that:
grep 192.168.1.1 filename
Or a more "complex" grep command could do that and write it to another file:
grep 192.168.1.1 filename > new_filename
Hope this helps!
Assuming the data format is well defined like you said:
"this is line <seq> <ip>"
you could do simply this:
cat file | egrep "192.168.0.1$"
Related
I have a logfile
...
Host: 111.222.121.123 (111.222.121.123.deploy.static.akamaitechnologies.com) Ports: 80/open/tcp//http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/, 443/open/tcp//ssl|http//AkamaiGHost (Akamai's HTTP Acceleration|Mirror service)/
Host: 1.2.3.4 () Ports: 80/open/tcp//http//cloudflare/, 443/open/tcp//ssl|https//cloudflare/, 2052/open/tcp//clearvisn?///, 2053/open/tcp//ssl|http//nginx/, 2082/open/tcp//infowave?///, 2083/open/tcp//ssl|http//nginx/, 2086/open/tcp//gnunet?///, 2087/open/tcp//ssl|http//nginx/, 2095/open/tcp//nbx-ser?///, 2096/open/tcp//ssl|http//nginx/, 8080/open/tcp//http-proxy//cloudflare/, 8443/open/tcp//ssl|https-alt//cloudflare/, 8880/open/tcp//cddbp-alt?///
Host: 2.3.4.5 (a104-96-1-61.deploy.static.akamaitechnologies.com) Ports: 53/open/tcp//domain//(unknown banner: 29571.61)/
...
I need to extract and convert IPs and http ports to the following format
1.2.3.4:80,443,2083
There are just two types of port fields in the logfile
80/open/tcp//http
2083/open/tcp//ssl|http
Tried to use sed but without success. I ended up with this dysfunctional command
cat ../host_ports.txt | sed -rn 's/Host: ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*?([0-9]{1,5}\/open\/tcp\/\/http|[0-9]{1,5}\/open\/tcp\/\/ssl\|http).*/\1 \2/p'
First handle the repeating ports, and next replace Host/Port to the desired format.
sed -r 's/(Ports:|,) ([0-9]*)[^,]*/\1\2/g;s/Host: ([^ ]*).*Ports:/\1:/' ../host_ports.txt
EDIT:
First I gave all ports of a line with http somewhere, now limit the result to ports with http in its description.
sed -nr 's/Ports: /, /;
s/, ([0-9]*)[^,]*http[^,]*/,\1/g;
s/,[^,]*\/[^,]*//g;
s/Host: ([^ ]*)[^,]*,/\1:/p' ../host_ports.txt
This script will do it for you, and you don't need sed :
#!/bin/bash
while read -r line; do
if echo $line | grep -q "http"; then
host=$(echo "$line" | grep -Po '(?<=^Host: )[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+')
ports=$(echo "$line" | grep -Po '[0-9]*((?=\/open\/tcp\/\/http)|(?=\/open\/tcp\/\/ssl\|http))' | tr '\n' ',')
echo "$host:${ports:0:-1}"
fi
done < ../log
The first grep will catch the IP address, with the help of Look behind. the -P is to use perl like regex, and the -o is to output only the matching string
The second regex is much like the first, but uses look after instead of look behind. It will only capture ports which are followed by /open/tcp//http or /open/tcp//ssl|http. The tr right after will replace newlines with commas.
the ${ports:0:-1} is just to eliminate the trailing comma.
Hope this helps!
I have a list like the following:
Name_JR_1
1.1.1.1
Name_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3
If I want to chose all associated name with following numerical syntax I can look for a pattern and print the matching line plus the after context or next line using the -A1 option, as follows:
grep "JR" -A1 file_name
and this will print what I want:
Name_JR_1
1.1.1.1
I need a way to invert this however, where I can REMOVE all entries which match the search pattern. However using the -v option with this syntax doesen't give me the results I want:
grep -v "JR" -A1 file_name
What I want the output to be like after this command is as follows:
Names_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3.
Try:
$ awk '/JR/{getline;next} 1' file
Name_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3
How it works
/JR/{getline;next}
This selects the lines containing JR. For those lines, this instructs awk to get the next line (getline) and then to skip the rest of the commands and start over on the following line (next).
1
For any lines that don't have JR in them, this prints the line.
Another awk:
$ awk '/JR/||f==1{f=!f;next}1' file
Name_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3
If we see JR or flag is up, reverse the flag and skip to next line.
You can also use sed
$ sed '/JR/{N;d;}' ip.txt
Name_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3
N will add next line to pattern space and then d will delete it
use N;N for two more lines, N;N;N for three more lines and so on
For a generic solution with awk
$ awk '/JR/{c=2} !(c && c--)' ip.txt
Name_SR_1
2.2.2.2
Name_NONE_1
3.3.3.3
here 2 is count of matching line and one line after
so for -A2 equivalent, you'd need c=3
See Printing with sed or awk a line following a matching pattern for plenty of cases related to this
I have a txt file with a list of ip addresses against domain names. eg;
1.1.168.192 example1.example1.net
2.1.168.192 example2.example2.net
3.1.168.192 example3.example3.net
.....
12.1.168.192 example12.example12.net
I can't get my sed command to change the output to;
192.168.1.1 example1.example1.net
192.168.1.2 example2.example2.net
192.168.1.3 example3.example3.net
....
192.168.1.12 example12.example12.net
sed command i'm using is
sed -r 's/^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$/\4.\3.\2.\1/'
using it as
cat filename | sed -r 's/^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$/\4.\3.\2.\1/'
The only problem is that you've included an anchor $ in your pattern, which tries to match the end of each line but fails. You just need to remove it:
$ sed -r 's/^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})/\4.\3.\2.\1/' file
192.168.1.1 example1.example1.net
192.168.1.2 example2.example2.net
192.168.1.3 example3.example3.net
Note that I'm passing the file name as an argument to sed, thereby avoiding a useless use of cat.
awk version
awk '{split(".",i,$1);printf "%d.%d.%d.%d %s\n",i[4],i[3],i[2],i[1],$2}' YourFile
$ sed -r 's/([^.]+)(\.[^.]+)(\.[^.]+)\.([^ ]+)/\4\3\2.\1/' file
192.168.1.1 example1.example1.net
192.168.1.2 example2.example2.net
192.168.1.3 example3.example3.net
192.168.1.12 example12.example12.net
I have many configuration files ,
the line that start with LINE word have IP address
My target to read the line that start with LINE word from the file and print only the IP address
The problem is that IP address can be in any field in the line so I can’t capture the IP according to field number
example
grep LINE file1.txt
LINE /home/Ariate/run.pl "Voda STS 4 Test - " "102841" && ssh 17.77.170.130 -p 2022
grep LINE file2.txt
LINE /home/Ariate/run.pl 137.77.170.30 "Voda STS 4 Test - " "102841" && ssh ACTIVE
please advice how to capture the IP address from the line ( solution can be also with perl one liner )
expected results
echo $IP_FROM_LINE
17.77.170.130
echo $IP_FROM_LINE
137.77.170.30
perl -MRegexp::Common=net -lne 'print $1 if /^LINE.*\b($RE{net}{IPv4})/'
Using this grep -oE:
grep -oE '\d+\.\d+\.\d+\.\d+' file
17.77.170.130
137.77.170.30
OR else:
grep -oP '\d+\.\d+\.\d+\.\d+' file
The following will get you the desired IP addresses:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file
To place the result in a variable as request, you'll need to iterate of the results as follows:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file |
while read IP_FROM_LINE ; do
echo $IP_FROM_LINE
done
grep -oE '[0-9]{2,3}(\.[0-9]{2,3}){3}'
matches
17.77.170.130
137.77.170.30
or
grep -oP '\d{2}(\.\d{2}){3}'
if your grep supports -P option.
both of them works with the data you have given.
But if you want really worried of what to be matched, use
grep -Eo '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)'
which would match excat ip addresses.
In a very large file I need to find the position (line number) of a string, then extract the 2 lines above and below that string.
To do this right now - I launch vi, find the string, note it's line number, exit vi, then use sed to extract the lines surrounding that string.
Is there a way to streamline this process... ideally without having to run vi at all.
Maybe using grep like this:
grep -n -2 your_searched_for_string your_large_text_file
Will give you almost what you expect
-n : tells grep to print the line number
-2 : print 2 additional lines (and the wanted string, of course)
You can do
grep -C 2 yourSearch yourFile
To send it in a file, do
grep -C 2 yourSearch yourFile > result.txt
Use grep -n string file to find the line number without opening the file.
you can use cat -n to display the line numbers and then use awk to get the line number after a grep in order to extract line number:
cat -n FILE | grep WORD | awk '{print $1;}'
although grep already does what you mention if you give -C 2 (above/below 2 lines):
grep -C 2 WORD FILE
You can do it with grep -A and -B options, like this:
grep -B 2 -A 2 "searchstring" | sed 3d
grep will find the line and show two lines of context before and after, later remove the third one with sed.
If you want to automate this, simple you can do a Shell Script. You may try the following:
#!/bin/bash
VAL="your_search_keyword"
NUM1=`grep -n "$VAL" file.txt | cut -f1 -d ':'`
echo $NUM1 #show the line number of the matched keyword
MYNUMUP=$["NUM1"-1] #get above keyword
MYNUMDOWN=$["NUM1"+1] #get below keyword
sed -n "$MYNUMUP"p file.txt #display above keyword
sed -n "$MYNUMDOWN"p file.txt #display below keyword
The plus point of the script is you can change the keyword in VAL variable as you like and execute to get the needed output.