I am comparing two files and desired results will be in new file. I am using output.write(line). I want desired results in a list but it should be in same pattern e.g:
source ip > destination ip
1.1.1.1 > 2.2.2.2
test.py:
import re
input1 = open("file1.txt","r")
input2 = open("file2.txt","r")
output = open("newfile.txt","w")
file2_data = input2.read()
for line in input1:
regex_search = re.search(r"\d.\d.\d.\d", line)
source_ip = regex_search[0]
if re.search(source_ip,file2_data)==None:
output.write(line)
file1.txt:
source ip > destination ip
0.0.0.0 > 1.1.1.1
5.5.5.5 > 4.4.4.4
file2.txt:
source ip
3.3.3.3
6.6.6.6
0.0.0.0
The above code is just comparing source ip. The uncommon data will be stored in a newfile.txt like this:
newfile.txt:
source ip > destination ip
5.5.5.5 > 4.4.4.4
Thus, Just I want newfile.txt data into list
Related
I know how to sort a text file alphabetically but I'm trying to do more than sorting (i.e. grouping).
I'm trying to create a Unix Shell script that formats /etc/hosts file in my organization in the following format:
From:
Xsb ip
aabc ip
A2bc ip
Eexg ip
exx ip
Fxg ip
To:
### A
aabc ip
a2bc ip
### E
eexg ip
exx ip
### F
fxg ip
### X
xsb ip
Then I'll create another script to add new hostname lines but for now I'm not sure what's the most compact way to do this. I thought perhaps I'll need to loop on all initial letters of the hostnames with a for loop but your expert advice is highly appreciated on the shortest way.
Following script maybe achieve your need.
tr A-Z a-z | sort | awk '{\
if (last != substr($0, 0, 1)) {\
print "### "toupper(substr($0,0,1))"\n"$0; last = substr($1, 0, 1)\
} else {\
print $0\
}\
}'
tr convert letters to lowercase
sort sort text alphabetically
awk add '###' when first letter of line different from next line
Hope this can help you.
I have two files: one contains a list of AP names, and another list contains AP names again, but this time the controller IP for each AP is listed before the AP's name.
File 1:
AP1
AP2
Ap3
AP4
Ap5
Ap6
...
File 2:
1.1.1.1,Ap1
2.2.2.2,Ap2
3.3.3.3,Ap3
4.4.4.4,Ap4
6.6.6.6,Ap6
...
How can I match up the names from file 1 with the names in file 2 so that the output resembles the following?
1.1.1.1,Ap1
2.2.2.2,Ap2
3.3.3.3,Ap3
4.4.4.4,Ap4
IP Not Found,Ap5
6.6.6.6,Ap6
I was thinking that I could use the comm command, but I do not know of a good way to only compare the names and not the IPs. I could also just grep for every single name, but that would take forever (there are around 8,000 AP names).
The join command will do the job (note that both files have to be sorted by AP name first; your example data already is, but if your real-world data isn't, run the first file through sort -f, and the second file through sort -f -t , -k 2).
join -i -t , -1 1 -2 2 -a 1 -o 2.1,1.1 -e "IP Not Found" file1.txt file2.txt
-i means ignore case, -t , means fields are separated by commas, -1 1 means join on the first (only) field of the first file; -2 2 means join on the second field of the second file. -a 1 means include rows from the first file that don't have any matches. -o 2.1,1.1 specifies the output format: the first field of the second file (IP), then the first field of the first file (AP). -e "IP Not Found" means to output "IP Not Found" in place of an empty field.
This will output
1.1.1.1,AP1
2.2.2.2,AP2
3.3.3.3,Ap3
4.4.4.4,AP4
IP Not Found,Ap5
6.6.6.6,Ap6
This awk snippet should do it:
awk 'BEGIN{FS=","}
(FNR==NR){a[tolower($2)]=$0}
(FNR!=NR){if (a[tolower($1)]!="")
print a[tolower($1)]
else
print "IP Not Found," $1}' file2.txt file1.txt
producing in your case:
1.1.1.1,Ap1
2.2.2.2,Ap2
3.3.3.3,Ap3
4.4.4.4,Ap4
IP Not Found,Ap5
6.6.6.6,Ap6
I have this command which outputs 2 columns separated by ⎟. First column is the number of occurrence, second is the IP address. And the whole thing is sorted by ascending # of occurrence.
awk '{ips[$1]++} END {for (ip in ips) { printf "%5s %-1s %-3s\n", ips[ip], "⎟", ip}}' "${ACCESSLOG}" | sort -nk1
19 ⎟ 76.20.221.34
19 ⎟ 76.9.214.2
22 ⎟ 105.152.107.118
26 ⎟ 24.185.179.32
26 ⎟ 42.117.198.229
26 ⎟ 83.216.242.69
etc.
Now i would like to add a third column in there. In the bash shell, if you do, for instance:
host 72.80.99.43
you'll get:
43.99.80.72.in-addr.arpa domain name pointer pool-72-80-99-43.nycmny.fios.verizon.net.
So for every IP appearing in the list, i want to show in the third column its associated host. And i want to do that from within awk. So calling host from awk and passing it the parameter ip. And ideally, skipping all the standard stuff and only showing the hostname like so: nycmny.fios.verizon.net.
So my final command would look like this:
awk '{ips[$1]++} END {for (ip in ips) { printf "%5s %-1s %-3s %20s\n", ips[ip], "⎟", ip, system( "host " ip )}}' "${ACCESSLOG}" | sort -nk1
Thanks
You wouldn't use system() since you want to combine the shell command output with your awk output, you'd call the command as a string and read it's result into a variable with getline, e.g.:
awk '{ips[$1]++}
END {
for (ip in ips) {
cmd = "host " ip
if ( (cmd | getline host) <= 0 ) {
host = "N/A"
}
close(cmd)
printf "%5s %-1s %-3s %20s\n", ips[ip], "⎟", ip, host
}
}' "${ACCESSLOG}" | sort -nk1
I assume you can figure out how to use *sub() to get just the part of the host output you care about.
I have such textfile:
313 "88.68.245.12"
189 "87.245.108.11"
173 "84.134.230.11"
171 "87.143.88.4"
158 "77.64.132.10"
....
I want to grep only the IP from the first 10 lines, run whois over the IP adress and from that output I want to grep the line where it says netname.
How can I achieve this?
Just loop through the file with while - read:
while IFS='"' read -r a ip c
do
echo "ip: $ip"
whois "$ip" | grep netname
done < <(head -10 file)
This is giving IFS='"' so that the field separator is a double quote ". This way, the values within double quotes will be stored in $ip.
Then, we print the ip and perform the whois | grep thing.
Finally, we feed the loop with head -10 file, so that we just read the first 10 lines.
Does anyone know of a way to replace blanks with 0's? Here's what im trying to do...
Basically i have a script that pulls an ip address and manipulates the address to make a port number out of it.
192.168.202.3 = Port 23
what i need is a smart enough sed command to add 2 0's in front of the 3 making it a full value.
192.168.202.3 = Port 2003
or:
192.168.202.003 = Port 2003
The catch is, if the number already exists then i dont want it to add 0's..
192.168.202.254 = Port 2254
instead of:
192.168.202.254 = Port 200254
Any ideas on how to do it?
Relevant Portion of the script:
# Retrieve local-ipv4 address from meta-data
GET http://169.254.169.254/latest/meta-data/local-ipv4 > /metadata
# Create a manipulated version of ipv4 to use as a port number
sed "s/192.168.20//" /metadata > /metaport
sed -i "s/\.//g" /metaport
If you have another way without using sed im open for those suggestions as well!!
Thanks!
I would prefer using awk for number manipulation rather than sed
awk -F'.' '{printf "%03s%03s\n", $3, $4}' /metadata | cut -c3-6 > /metaport
Input IP:
192.168.202.3
192.168.202.23
192.168.202.254
Output Port:
2003
3023
2254
EDIT
More concise awk only solution avoiding need of cut (Suggested by Jonathan Leffler)
awk -F'.' '{printf "%d%03d\n", $3 % 10, $4}' /metadata > /metaport
If the input file contains only an IP address, then brute force and ignorance can do the job:
sed -e 's/\([0-9]\)\.\([0-9]\)$/& = Port \100\2/' \
-e 's/\([0-9]\)\.\([0-9][0-9]\)$/& = Port \10\2/' \
-e 's/\([0-9]\)\.\([0-9][0-9][0-9]\)$/& = Port \1\2/'
The first expression deals with 1 digit; the second with 2 digits; the third with 3.
Given input data:
192.168.202.3
192.168.203.13
192.168.202.003
192.168.202.254
the output is:
192.168.202.3 = Port 2003
192.168.203.13 = Port 3013
192.168.202.003 = Port 2003
192.168.202.254 = Port 2254
If you have a different input data format, you have to work harder to isolate the relevant section of the IP address, but you should really, really show what the input data looks like.
Just for fun, bash:
while IFS=. read a b c d; do
printf "%d%03d\n" $((c%10)) $d
done <<END
192.168.202.3
192.168.202.003
192.168.209.123
127.0.0.1
END
2003
2003
9123
0001
Given the description -- only insert two zeros when we only have 2 digits into the port the following should work:
sed -r '/Port [0123456789]{2}$/s/Port (.)/\100/'
So this only matches when Port is followed by 2 digits. If it does match, replace the first digit with that digit and two zeros.
If you need to handle 3 digits, another match section that does just 3 digits could be trivially added.