I want to retrieve all the hostnames associated with the IP.
I ran into an error while using requests module in python. And the data that the error contains I want that data.
First I got the IP of of youtube.com I wanted to get the webpage using ip address only so using ping I got the IP address of youtube.com
IP 172.217.163.78
Then I made the request
import requests
session_ = requests.Session()
res_ = session_.get('https://172.217.163.78')
ERROR
\Python3.7.2\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='172.217.163.78', port=443):
Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError("hostname '172.217.163.78'
doesn't match either of '*.google.com', '*.android.com',
'*.appengine.google.com', '*.cloud.google.com', '*.g.co', '*.gcp.gvt2.com', '*.ggpht.cn', '*.google-analytics.com',
'*.google.ca', '*.google.cl', '*.google.co.in',
'*.google.co.jp', '*.google.co.uk', '*.google.com.ar',
'*.google.com.au', '*.google.com.br', '*.google.com.co',
'*.google.com.mx', '*.google.com.tr', '*.google.com.vn',
'*.google.de', '*.google.es', '*.google.fr', '*.google.hu',
'*.google.it', '*.google.nl', '*.google.pl', '*.google.pt',
'*.googleadapis.com', '*.googleapis.cn',
'*.googlecommerce.com', '*.googlevideo.com', '*.gstatic.cn',
'*.gstatic.com', '*.gstaticcnapps.cn', '*.gvt1.com',
'*.gvt2.com', '*.metric.gstatic.com', '*.urchin.com',
'*.url.google.com', '*.youtube-nocookie.com',
'*.youtube.com', '*.youtubeeducation.com',
'*.youtubekids.com', '*.yt.be', '*.ytimg.com',
'android.clients.google.com', 'android.com',
'developer.android.google.cn',
'developers.android.google.cn', 'g.co', 'ggpht.cn', 'goo.gl',
'google-analytics.com', 'google.com', 'googlecommerce.com'
, 'source.android.google.cn', 'urchin.com', 'www.goo.gl',
'youtu.be', 'youtube.com', 'youtubeeducation.com',
'youtubekids.com', 'yt.be'")))
Is there a way to get all the above hostnames, and is there a way to solve this problem SSLError that i'm getting
Does this work for you?
import requests
session_ = requests.Session()
res_ = session_.get('https://172.217.163.78', verify=False)
print (res_.url)
#Output
# http://www.google.com
Have you concerned using socket instead of request?
import socket
from pprint import pprint
ip_address = socket.gethostbyname('www.abc.com')
pprint (socket.gethostbyaddr(ip_address))
sys.exit(0)
OUTPUT
('www.pitchsharktank.com',
['250.132.181.199.in-addr.arpa',
'www.disneycampusrep.com',
'www.disneycollateral.com',
'www.disneyinternsinfo.com',
'www.missiontimekeeper.com',
'www.watchdisneyjunior.go.com',
'www.disneycastingscout.com',
'www.disneyimaginations.com',
'www.disneyprogramsblog.com',
'www.disneyvacationclub.disney.go.com',
'www.watchdisneychannel.go.com',
'www.wdwcollegeprogramecard.com',
'www.wdwcollegeprogramecard2.com',
'www.disneyinternationalepresentation.com',
many many more here],
['199.181.132.250'])
UPDATE
I spent more than half the day looking into the hostname issue. I have found that this problem is complex, especially for IP addresses assigned to Google.
For example:
ping www.youtube.com
PING youtube-ui.l.google.com (64.233.185.93): 56 data bytes
64 bytes from 64.233.185.93: icmp_seq=0 ttl=41 time=19.820 ms
#############################################################
nslookup www.youtube.com
www.youtube.com canonical name = youtube-ui.l.google.com.
Name: youtube-ui.l.google.com
Address: 172.217.11.142
Name: youtube-ui.l.google.com
Address: 64.233.176.190
Name: youtube-ui.l.google.com
Address: 64.233.177.91
Name: youtube-ui.l.google.com
Address: 64.233.177.93
Name: youtube-ui.l.google.com
Address: 64.233.177.190
Name: youtube-ui.l.google.com
Address: 64.233.185.91
Name: youtube-ui.l.google.com
Address: 64.233.185.93
Name: youtube-ui.l.google.com
Address: 64.233.185.136
Name: youtube-ui.l.google.com
Address: 64.233.185.190
Name: youtube-ui.l.google.com
Address: 74.125.138.190
Name: youtube-ui.l.google.com
Address: 74.125.196.91
Name: youtube-ui.l.google.com
Address: 108.177.122.91
Name: youtube-ui.l.google.com
Address: 108.177.122.93
Name: youtube-ui.l.google.com
Address: 108.177.122.136
Name: youtube-ui.l.google.com
Address: 108.177.122.190
Name: youtube-ui.l.google.com
Address: 172.217.0.78
#############################################################
dig www.youtube.com
;; QUESTION SECTION:
;www.youtube.com. IN A
;; ANSWER SECTION:
www.youtube.com. 9446 IN CNAME youtube-ui.l.google.com.
youtube-ui.l.google.com. 132 IN A 64.233.185.190
youtube-ui.l.google.com. 132 IN A 74.125.138.136
youtube-ui.l.google.com. 132 IN A 74.125.138.190
youtube-ui.l.google.com. 132 IN A 74.125.196.91
youtube-ui.l.google.com. 132 IN A 74.125.196.93
youtube-ui.l.google.com. 132 IN A 172.217.0.78
youtube-ui.l.google.com. 132 IN A 172.217.0.142
youtube-ui.l.google.com. 132 IN A 172.217.2.46
youtube-ui.l.google.com. 132 IN A 172.217.4.14
youtube-ui.l.google.com. 132 IN A 172.217.164.78
youtube-ui.l.google.com. 132 IN A 173.194.219.136
youtube-ui.l.google.com. 132 IN A 173.194.219.190
youtube-ui.l.google.com. 132 IN A 64.233.177.91
youtube-ui.l.google.com. 132 IN A 64.233.177.93
youtube-ui.l.google.com. 132 IN A 64.233.177.136
youtube-ui.l.google.com. 132 IN A 64.233.177.190
The IP address that you provided in your original question doesn't appear in either the ping, nslookup or dig results.
In Apple Safari, Google Chrome and Mozilla Firefox the URL https://172.217.163.78 returns https://www.google.com.
The ShowIP extension in Mozilla Firefox shows that the IP address for https://www.youtube.com is 2607:f8b0:4002:c00::88, which is an IPv6 address.
I also found this:
FQDN: youtube.com
Domain Name: youtube.com
Name servers: ns1.google.com
ns2.google.com
ns3.google.com
ns4.google.com
IP numbers: 2404:6800:4003:805::200e
2404:6800:4004:801::200e
2404:6800:4006:809::200e
2607:f8b0:4004:802::200e
2607:f8b0:4005:807::200e
2607:f8b0:400a:804::200e
2800:3f0:4001:80a::200e
2a00:1450:4009:812::200e
2a00:1450:400b:c01::be
74.125.193.91
74.125.193.93
74.125.193.136
74.125.193.190
172.217.7.238
172.217.24.78
172.217.30.78
216.58.195.78
216.58.197.174
216.58.203.110
216.58.206.46
216.58.217.46
LOOKUP fully qualified domain names:
import socket
##############################################
# IP addresses linked to YouTube on 01-28-2019
##############################################
ip_addresses =['2404:6800:4003:805::200e',
'2404:6800:4004:801::200e',
'2404:6800:4006:809::200e',
'2607:f8b0:4004:802::200e',
'2607:f8b0:4005:807::200e',
'2607:f8b0:400a:804::200e',
'2800:3f0:4001:80a::200e',
'2a00:1450:4009:812::200e',
'2a00:1450:400b:c01::be',
'74.125.193.91',
'74.125.193.93',
'74.125.193.136',
'74.125.193.190',
'172.217.7.238',
'172.217.24.78',
'172.217.30.78',
'216.58.195.78',
'216.58.197.174',
'216.58.203.110',
'216.58.206.46',
'216.58.217.46']
for iP_address in ip_addresses:
fully_qualified_domain_name = socket.getfqdn(str(iP_address))
print (fully_qualified_domain_name)
# OUTPUT
sin10s06-in-x0e.1e100.net
nrt12s02-in-x0e.1e100.net
syd09s15-in-x0e.1e100.net
iad23s58-in-x0e.1e100.net
sfo07s16-in-x0e.1e100.net
sea15s08-in-x0e.1e100.net
2800:3f0:4001:80a::200e
lhr35s10-in-x0e.1e100.net
2a00:1450:400b:c01::be
ig-in-f91.1e100.net
ig-in-f93.1e100.net
ig-in-f136.1e100.net
ig-in-f190.1e100.net
iad23s58-in-f14.1e100.net
sin10s06-in-f14.1e100.net
gru06s34-in-f14.1e100.net
sfo07s16-in-f78.1e100.net
nrt12s02-in-f14.1e100.net
syd09s15-in-f14.1e100.net
lhr35s10-in-f14.1e100.net
sea15s08-in-f14.1e100.net
As you can see, not one of these FQDN equals youtube.com.
I also decided to lookup the IP addresses assigned to YouTube through ARIN. the code below loops through those addresses.
import socket
import ipaddress
from dns import reversename, resolver
# IPv4 addresses listed at whois.arin.net for YouTube
# 64.15.112.0/20 = number of hosts 4,096
# 104.237.160.0/19 = number of hosts 8,192
# 208.65.152.0/22 = number of hosts 1,024
# 208.117.224.0/19 = number of hosts 8,192
youtube_IPv4_addresses = ['64.15.112.0/20','104.237.160.0/19', '208.65.152.0/22', '208.117.224.0/19']
# IPv6 addresses listed at whois.arin.net for YouTube
# Start Range: 2620:11a:a000:0:0:0:0:0
# End Range: 2620:11a:a0ff:ffff:ffff:ffff:ffff:ffff
# No. of host: 309485009821345068724781056
youtube_IPv6_addresses = ['2620:11A:A000::/40']
#############################################
# This function is designed to query the IPv4
# address blocks assigned to YouTube for their
# corresponding PTR records, which are used
# for the Reverse DNS (Domain Name System)
#############################################
def get_ipv4_hostnames():
for network in youtube_IPv4_addresses:
ip_addresses = ipaddress.IPv4Network(network)
for ip_address in ip_addresses:
try:
rev_name = reversename.from_address(str(ip_address))
reversed_dns = str(resolver.query(rev_name,"PTR")[0])
print (reversed_dns)
except Exception as error:
print ('The following error occurred: \n {}'.format(error))
#############################################
# WARNING WARNING WARNING WARNING WARNING
#############################################
# There are 309485009821345068724781056
# possible hosts within the IPv6 address
# range assigned to YouTube at ARIN
#############################################
# WARNING WARNING WARNING WARNING WARNING
#############################################
#############################################
def get_ipv6_hostnames():
for network in youtube_IPv6_addresses:
ip_addresses = ipaddress.IPv6Network(network)
for ip_address in ip_addresses:
try:
fully_qualified_domain_name = socket.getfqdn(str(ip_address))
if fully_qualified_domain_name != str(ip_address):
print (fully_qualified_domain_name)
except Exception as error:
print('The following error occurred: \n {}'.format(error))
Related
Hi we are trying to use NodeJS to return IP address WHOIS information before we send the requesting IP address to the rest of our app - That part is easy.
However the part that is not easy is, selecting only the Organization part of the whois information.
for example this is a whois and what it returns
whois 137.184.236.168
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object
refer: whois.arin.net
inetnum: 137.0.0.0 - 137.255.255.255
organisation: Administered by ARIN
status: LEGACY
whois: whois.arin.net
changed: 1993-05
source: IANA
# whois.arin.net
NetRange: 137.184.0.0 - 137.184.255.255
CIDR: 137.184.0.0/16
NetName: DIGITALOCEAN-137-184-0-0
NetHandle: NET-137-184-0-0-1
Parent: NET137 (NET-137-0-0-0-0)
NetType: Direct Allocation
OriginAS: AS14061
Organization: DigitalOcean, LLC (DO-13)
RegDate: 2019-11-13
Updated: 2020-04-03
Comment: Routing and Peering Policy can be found at https://www.as14061.net
Comment:
Comment: Please submit abuse reports at https://www.digitalocean.com/company/contact/#abuse
Ref: https://rdap.arin.net/registry/ip/137.184.0.0
OrgName: DigitalOcean, LLC
OrgId: DO-13
Address: 101 Ave of the Americas
Address: FL2
City: New York
StateProv: NY
PostalCode: 10013
Country: US
RegDate: 2012-05-14
Updated: 2022-05-19
Ref: https://rdap.arin.net/registry/entity/DO-13
OrgAbuseHandle: ABUSE5232-ARIN
OrgAbuseName: Abuse, DigitalOcean
OrgAbusePhone: +1-347-875-6044
OrgAbuseEmail: abuse#digitalocean.com
OrgAbuseRef: https://rdap.arin.net/registry/entity/ABUSE5232-ARIN
OrgTechHandle: NOC32014-ARIN
OrgTechName: Network Operations Center
OrgTechPhone: +1-347-875-6044
OrgTechEmail: noc#digitalocean.com
OrgTechRef: https://rdap.arin.net/registry/entity/NOC32014-ARIN
OrgNOCHandle: NOC32014-ARIN
OrgNOCName: Network Operations Center
OrgNOCPhone: +1-347-875-6044
OrgNOCEmail: noc#digitalocean.com
OrgNOCRef: https://rdap.arin.net/registry/entity/NOC32014-ARIN
The only thing we are interested in is Organization: DigitalOcean, LLC (DO-13)
As we want to drop all IP addresses from this host provider.
We noticed that we have been successful at stopping Google and AWS via using host command but Digital Ocean does not work this way and we need to do it via Whois.
I know in NodeJS I would request the information
exec("whois "+ip, (error, stdout, stderr) => {
console.log(stdout);
}
Could use a regular expression:
const organizationPattern = /^organization:\s*(.+)$/im;
const match = organizationPattern.exec(stdout);
const organization = match ? match[1] : 'unknown';
console.log(organization);
The context in /var/lib/misc/dnsmasq.leases like below
1646438467 12:03:f2:19:41:4d 192.168.4.239 * 01:12:03:f2:19:41:4d
lease time: 1646438467
MAC address: 12:03:f2:19:41:4d
IP address: 192.168.4.13
hostname: *
but what is the 01:12:03:f2:19:41:4d? It likes like 01: and Mac address
https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html does not mention that
In the following code I would like to capture the instance ID and then the body underneath it. The pattern is recurring so that there are multiple instances with the same bodies. I can't seem to figure out how to get it to continue past the newline segments.
import re
config = '''
Instance: evpn-a
VLAN ID: 123, MAC address: 00:05:86:71:05:f0
Source: irb.0, Rank: 1, Status: Active
State: <Local-MAC-Only Local-Gateway Remote-Adv-Allowed>
IP address: 192.168.10.251
VLAN ID: 123, MAC address: 00:05:86:71:ab:f0
Source: 20.1.1.2, Rank: 1, Status: Active
State: <Remote-Gateway Local-Adv-Allowed Local-Adv-Done>
IP address: 192.168.10.252
L3 route: 192.168.10.252/32, L3 context: bridge-vrf (irb.0)
Instance: evpn-b
VLAN ID: 123, MAC address: 00:05:86:71:05:f0
Source: irb.0, Rank: 1, Status: Active
State: <Local-MAC-Only Local-Gateway Remote-Adv-Allowed>
IP address: 192.168.10.251
VLAN ID: 123, MAC address: 00:05:86:71:ab:f0
Source: 20.1.1.2, Rank: 1, Status: Active
State: <Remote-Gateway Local-Adv-Allowed Local-Adv-Done>
IP address: 192.168.10.252
L3 route: 192.168.10.252/32, L3 context: bridge-vrf (irb.0)
'''
evpn_obj_list = re.compile(r'Instance:\s+(\S+)(.*?)(?:\S+|\Z)',re.S|re.M).findall(config)
evpn = evpn_obj_list
print(evpn)
The result I get from the above is:
[('evpn-a', '\n\n'), ('evpn-b', '\n\n')]
You may use
rx = re.compile(r'^Instance:\s+(\S+)\s*(.*?)(?=\n\s*Instance:\s|\Z)', re.S|re.M)
evpn_obj_list = rx.findall(config)
See the regex demo.
Details
^ - start of a line
Instance: - a string
\s+ - 1+ whitespaces
(\S+) - Group 1: any one or more non-whitespace chars
\s* - 0+ whitespaces
(.*?) - Group 2: any 0 or more chars, as few as possible
(?=\n\s*Instance:\s|\Z) - a positive lookahead that requires a newline, 0+ whitespaces, Instance:, a whitespace OR the end of file immediately to the right of the current location.
See the Python demo yielding
[('evpn-a', 'VLAN ID: 123, MAC address: 00:05:86:71:05:f0\n Source: irb.0, Rank: 1, Status: Active\n State: <Local-MAC-Only Local-Gateway Remote-Adv-Allowed>\n IP address: 192.168.10.251\n\nVLAN ID: 123, MAC address: 00:05:86:71:ab:f0\n Source: 20.1.1.2, Rank: 1, Status: Active\n State: <Remote-Gateway Local-Adv-Allowed Local-Adv-Done>\n IP address: 192.168.10.252\n L3 route: 192.168.10.252/32, L3 context: bridge-vrf (irb.0)'), ('evpn-b', 'VLAN ID: 123, MAC address: 00:05:86:71:05:f0\n Source: irb.0, Rank: 1, Status: Active\n State: <Local-MAC-Only Local-Gateway Remote-Adv-Allowed>\n IP address: 192.168.10.251\n\nVLAN ID: 123, MAC address: 00:05:86:71:ab:f0\n Source: 20.1.1.2, Rank: 1, Status: Active\n State: <Remote-Gateway Local-Adv-Allowed Local-Adv-Done>\n IP address: 192.168.10.252\n L3 route: 192.168.10.252/32, L3 context: bridge-vrf (irb.0)\n')]
I'm trying to setup a DNS Server using Debian but I keep getting errors when I do nslookup like SERVFAIL or REFUSED.
I want to use 3 virtual machines (VM1, VM2 and VM3) and call them that by those names in the DNS Server, I'm using VMWare Workstation 11.
Here is my configuration:
named.conf.options
options {
directory "/var/cache/bind";
additional-from-auth no;
additional-from-cache no;
// If there is a firewall between you and nameservers you want
// to talk to, you may need to fix the firewall to allow multiple
// ports to talk. See http://www.kb.cert.org/vuls/id/800113
// If your ISP provided one or more IP addresses for stable
// nameservers, you probably want to use them as forwarders.
// Uncomment the following block, and insert the addresses replacing
// the all-0's placeholder.
forwarders {
192.168.207.2;
192.168.207.133;
};
//========================================================================
// If BIND logs error messages about the root key being expired,
// you will need to update your keys. See https://www.isc.org/bind-keys
//========================================================================
dnssec-validation yes;
allow-recursion{127.0.0.1;};
auth-nxdomain no; # conform to RFC1035
listen-on-v6 { any; };
};
named.conf.local
zone "linux.local"{
type master;
file "etc/bind/db.linux.local";
};
zone "207.168.192-in-addr.arpa"{
type master;
file "etc/bind/db.207.168.192";
};
db.linux.local
;
; SOA
;
$TTL 1h
# IN SOA vm1.linux.local. root.linux.local. (
1 ; Serial number (YYYYMMDDnn)
1h ; Slave refresh
15m ; Slave retry
2w ; Slave expire
1h ; Cache TTL
)
;
; NS RECORDS
;
# IN NS vm1.linux.local.
;
; A RECORDS
;
linux.local. IN A 192.168.207.133
# IN A 192.168.207.133
vm1 IN A 192.168.207.133
vm3 IN A 192.168.207.135
vm2 IN A 192.168.207.130
vmware iN A 192.168.207.2
db.207.168.192
$TTL 1h
# IN SOA vm1.linux.local. root.linux.local. (
1;
1h;
15m;
2w;
1h;
)
IN NS vm1.linux.local.
133 IN PTR linux.local.
133 IN PTR vm1.linux.local.
135 IN PTR vm2.linux.local.
130 IN PTR vm3.linux.local.
2 IN PTR vmware.linux.local.
Here is the nslookup for VM1 and linux.local:
root#debian:/etc/bind# nslookup vm1
Server: 192.168.207.133
Address: 192.168.207.133#53
** server can't find vm1: REFUSED
root#debian:/etc/bind# nslookup linux.local
Server: 192.168.207.133
Address: 192.168.207.133#53
** server can't find linux.local.linux.local: SERVFAIL
maybe the access is limited. try edit the file named.conf, change or add options allow-query { any;};
run
rndc-confgen >> /etc/named.conf
This should fix the issue.
I'm running bind 9 in a CentOS 6.3 machine and trying do setup and internal DNS server.
I'm really new to this and need to set this up for my company, the guy responsible for this just quited and I'm my own for a while. Thanks in advance.
I've tried a few tutorials and so far this is what I have:
named.conf file
options {
listen-on port 53 { 127.0.0.1; 192.168.0.24; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; 192.168.0.0/24;};
recursion yes;
dnssec-enable yes;
dnssec-validation yes;
dnssec-lookaside auto;
/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";
managed-keys-directory "/var/named/dynamic";
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
zone "." IN {
type hint;
file "named.ca";
};
zone "local.logits.me" IN {
type master;
file "fwd.local.logits.me";
allow-update {none;};
};
zone "0.168.192.in-addr.arpa" IN {
type master;
file "rev.local.logits.me";
allow-update {none;};
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
fwd.local.logits.me file
$TTL 86400
# IN SOA web.local.logits.me. root.local.logits.me. (
2013102113 ; serial
3600 ; refresh
1800 ; retry
604800 ; expire
86400 ; minimum Time to Lie (TTL)
)
# IN NS web.local.logits.me.
# IN A 192.168.0.24
web IN A 192.168.0.24
rev.local.logits.me file
$TTL 86400
# IN SOA web.local.logits.me. root.local.logits.me. (
2013102113 ; serial
3600 ; refresh
1800 ; retry
604800 ; expire
86400 ; minimum time to live (TTL)
)
# IN NS web.local.logits.me.
# IN PTR local.logits.me.
web IN A 192.168.0.24
24 IN PTR web.local.logits.me.
And this is what I get when I try nslookup
[root#localhost ~]# nslookup local.logits.me
Server: 192.168.0.24
Address: 192.168.0.24#53
** server can't find local.logits.me: NXDOMAIN
[root#localhost ~]# nslookup web.local.logits.me
Server: 192.168.0.24
Address: 192.168.0.24#53
** server can't find web.local.logits.me: NXDOMAIN
[root#localhost ~]# nslookup 192.168.0.24
Server: 192.168.0.24
Address: 192.168.0.24#53
** server can't find 24.0.168.192.in-addr.arpa.: NXDOMAIN
What is wrong?
I figured it out. Someone (either me or the guy before me) put the files on the wrong folder.
So I was editing files on '/' instead of the named root path given in '/etc/sysconfig/named'.
The root path is '/var/named/chroot', so I moved the config files there and everything works fine.