I'm writing some software that analyzes registered domain names and looks for trends. I'm experimenting with some machine learning to help predict what domain names will be purchased in the future based on what types of domains are being registered.
I've been looking around searching for a way to download "all" of the registered domains that exist, but I haven't been able to find a way to do so.
It's easy for me to query individual domain names using the whois command line tool, for example:
$ whois google.com
Domain Name: GOOGLE.COM
Registry Domain ID: 2138514_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2018-02-21T18:36:40Z
Creation Date: 1997-09-15T04:00:00Z
Registry Expiry Date: 2020-09-14T04:00:00Z
Registrar: MarkMonitor Inc.
Registrar IANA ID: 292
Registrar Abuse Contact Email: abusecomplaints#markmonitor.com
Registrar Abuse Contact Phone: +1.2083895740
Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
Domain Status: serverDeleteProhibited https://icann.org/epp#serverDeleteProhibited
Domain Status: serverTransferProhibited https://icann.org/epp#serverTransferProhibited
Domain Status: serverUpdateProhibited https://icann.org/epp#serverUpdateProhibited
Name Server: NS1.GOOGLE.COM
Name Server: NS2.GOOGLE.COM
Name Server: NS3.GOOGLE.COM
Name Server: NS4.GOOGLE.COM
DNSSEC: unsigned
URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2018-03-20T03:16:59Z <<<
For more information on Whois status codes, please visit https://icann.org/epp
NOTICE: The expiration date displayed in this record is the date the
registrar's sponsorship of the domain name registration in the registry is
currently set to expire. This date does not necessarily reflect the expiration
date of the domain name registrant's agreement with the sponsoring
registrar. Users may consult the sponsoring registrar's Whois database to
view the registrar's reported date of expiration for this registration.
TERMS OF USE: You are not authorized to access or query our Whois
database through the use of electronic processes that are high-volume and
automated except as reasonably necessary to register domain names or
modify existing registrations; the Data in VeriSign Global Registry
Services' ("VeriSign") Whois database is provided by VeriSign for
information purposes only, and to assist persons in obtaining information
about or related to a domain name registration record. VeriSign does not
guarantee its accuracy. By submitting a Whois query, you agree to abide
by the following terms of use: You agree that you may use this Data only
for lawful purposes and that under no circumstances will you use this Data
to: (1) allow, enable, or otherwise support the transmission of mass
unsolicited, commercial advertising or solicitations via e-mail, telephone,
or facsimile; or (2) enable high volume, automated, electronic processes
that apply to VeriSign (or its computer systems). The compilation,
repackaging, dissemination or other use of this Data is expressly
prohibited without the prior written consent of VeriSign. You agree not to
use electronic processes that are automated and high-volume to access or
query the Whois database except as reasonably necessary to register
domain names or modify existing registrations. VeriSign reserves the right
to restrict your access to the Whois database in its sole discretion to ensure
operational stability. VeriSign may restrict or terminate your access to the
Whois database for failure to abide by these terms of use. VeriSign
reserves the right to modify these terms at any time.
The Registry database contains ONLY .COM, .NET, .EDU domains and
Registrars.
Domain Name: google.com
Registry Domain ID: 2138514_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2018-02-21T10:45:07-0800
Creation Date: 1997-09-15T00:00:00-0700
Registrar Registration Expiration Date: 2020-09-13T21:00:00-0700
Registrar: MarkMonitor, Inc.
Registrar IANA ID: 292
Registrar Abuse Contact Email: abusecomplaints#markmonitor.com
Registrar Abuse Contact Phone: +1.2083895740
Domain Status: clientUpdateProhibited (https://www.icann.org/epp#clientUpdateProhibited)
Domain Status: clientTransferProhibited (https://www.icann.org/epp#clientTransferProhibited)
Domain Status: clientDeleteProhibited (https://www.icann.org/epp#clientDeleteProhibited)
Domain Status: serverUpdateProhibited (https://www.icann.org/epp#serverUpdateProhibited)
Domain Status: serverTransferProhibited (https://www.icann.org/epp#serverTransferProhibited)
Domain Status: serverDeleteProhibited (https://www.icann.org/epp#serverDeleteProhibited)
Registry Registrant ID:
Registrant Name: Domain Administrator
Registrant Organization: Google LLC
Registrant Street: 1600 Amphitheatre Parkway,
Registrant City: Mountain View
Registrant State/Province: CA
Registrant Postal Code: 94043
Registrant Country: US
Registrant Phone: +1.6502530000
Registrant Phone Ext:
Registrant Fax: +1.6502530001
Registrant Fax Ext:
Registrant Email: dns-admin#google.com
Registry Admin ID:
Admin Name: Domain Administrator
Admin Organization: Google LLC
Admin Street: 1600 Amphitheatre Parkway,
Admin City: Mountain View
Admin State/Province: CA
Admin Postal Code: 94043
Admin Country: US
Admin Phone: +1.6502530000
Admin Phone Ext:
Admin Fax: +1.6502530001
Admin Fax Ext:
Admin Email: dns-admin#google.com
Registry Tech ID:
Tech Name: Domain Administrator
Tech Organization: Google LLC
Tech Street: 1600 Amphitheatre Parkway,
Tech City: Mountain View
Tech State/Province: CA
Tech Postal Code: 94043
Tech Country: US
Tech Phone: +1.6502530000
Tech Phone Ext:
Tech Fax: +1.6502530001
Tech Fax Ext:
Tech Email: dns-admin#google.com
Name Server: ns1.google.com
Name Server: ns4.google.com
Name Server: ns2.google.com
Name Server: ns3.google.com
DNSSEC: unsigned
URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/
>>> Last update of WHOIS database: 2018-03-19T20:13:36-0700 <<<
The Data in MarkMonitor.com's WHOIS database is provided by MarkMonitor.com for
information purposes, and to assist persons in obtaining information about or
related to a domain name registration record. MarkMonitor.com does not guarantee
its accuracy. By submitting a WHOIS query, you agree that you will use this Data
only for lawful purposes and that, under no circumstances will you use this Data to:
(1) allow, enable, or otherwise support the transmission of mass unsolicited,
commercial advertising or solicitations via e-mail (spam); or
(2) enable high volume, automated, electronic processes that apply to
MarkMonitor.com (or its systems).
MarkMonitor.com reserves the right to modify these terms at any time.
By submitting this query, you agree to abide by this policy.
MarkMonitor is the Global Leader in Online Brand Protection.
MarkMonitor Domain Management(TM)
MarkMonitor Brand Protection(TM)
MarkMonitor AntiPiracy(TM)
MarkMonitor AntiFraud(TM)
Professional and Managed Services
Visit MarkMonitor at http://www.markmonitor.com
Contact us at +1.8007459229
In Europe, at +44.02032062220
For more information on Whois status codes, please visit
https://www.icann.org/resources/pages/epp-status-codes-2014-06-16-en
--
The WHOIS data contains everything I need, but I can't find a way to download the WHOIS data for all currently registered domains.
Is there some way for me to get this data? I feel like it must be publicly available somewhere since the whois CLI tool can so easily query the info.
What am I missing here?
TL;DR: You can not (download all "whois" data).
(side preliminary note: "whois data", while often used is kind of incorrect. You use the whois protocol with a whois client to query a whois server at a registry, and more specifically here a domain name registry, that stores contact data about domain names it sponsors. For the same reason there is no "whois database".)
Now for the long sad story:
It is not possible for many obvious technical and non technical reasons. And you are deeply mistaken if you think the whois CLI command is simple (see my other answer here: https://unix.stackexchange.com/a/407030/211833 for details on that point)
First your question makes no sense for all TLDs at once. You have at least to separate ccTLDs from gTLDs.
1) ccTLDs
ccTLDs have often stricter rules about privacy on personal data and this ought to be even stricter with ongoing European regulations such as GDPR.
Basically some of them already forbid to have access to the complete list of domain names (which is often refered as the "zonefile") which has no personal data, so there is no way you will get access to all the content and the personal data.
You may try to approach some and ask if there is anything possible like for research studies, but I doubt you will be successful and you will need to deal with each ccTLD registry separately as they each deal with their own content (all data on the domain names in the TLD they manage)
2) gTLDs
For them, the situation is quite different.
First, since things are by default more liberal (no protection of personal data), you will see that many registrars/companies provide proxy/privacy services which means that even in a whois query output you will not see much useful data.
But still due to GDPR and assimilated, things are changing. Do a whois on godaddy.com for example and watch all these stars for contact names and emails, and hence the need to go to a website.
However registrars and registries are under contract with ICANN. Which means they both have some requirements, and they are uniform.
First, all registries are mandated to give access to their zonefiles. It is often done throught the CZDA, for which you can find details on ICANN website. Note that it is in fact the list of all domain names publishes, not exactly the list of all domain names registered as you can register a domain name and not put it visible on the DNS.
As for the contact data, that is the rest of the information visible in whois, there are other points not wellknown.
See the registrar agreement at https://www.icann.org/resources/pages/approved-with-specs-2013-09-17-en and specially section 3.3.6 that provide bulk acces to registrar "whois" data. Note how it is tied to some money (USD$10 000) and comes with various limitations on what you can do with it.
Remember that you would need to do it per registrar, so in the gTLDs world that is more than 1000 of them.
There is no equivalent provisions in the registry agreements for public bulk access (see https://newgtlds.icann.org/sites/default/files/agreements/agreement-approved-31jul17-en.html).
Things are complicated because as up today and for some months yet, .COM/.NET remains a thin registry that is one without the contact data stored at registry level, only at registrars.
Also all the above will change in the coming months/years because of the new regulations and also because RDAP, a new protocol, is slated to replace whois at one point. RDAP will allow far greater level of granularity on the access given and the amount of data returned.
Of course, in all cases above, nothing technically forbids anyone to just do regular whois queries and store the results locally. As you can see in a whois output your use of the data is constrained by various limits and bulk querying whois servers always expose you to the risk of being blacklisted or at least heavily rate limited.
Note that for the input (which names to query the whois server for), it is easy to start with zonefiles, even cross TLDs (if site.example exists you can try also site.test even if you do not have .test zonefile), or search engines queries, or dictionaries, etc.
Multiple companies do that and provide tools to search their data, like to do reverse queries and things like that. Maybe some could deliver you bulk results, but certainly not for free.
You can get the Whois record for millions of the registered domains (Active & Inactive) by using WhoisFreaks Database. https://whoisfreaks.com/. It provides well-parsed Whois domain information such as domain registration details, domain registrar details, registrant details, administrative contact, technical contact, server names, domain status, registry data, etc.
WhoisFreaks Database is One of the biggest whois databases with whois
information for 1000+ TLDs (gTLDs & ccTLDs), 422M+ tracked domains,
555M+ whois records.
Well parsed and normalized whois data records which are available in
CSV file format easy to read and integrate into any business system.
Accurate and updated, the WhoisFreaks database is updated on the
daily basis.
WhoisFreaks database provides whois records from 1986.
Custom whois domain data is also available on the basis of Registrant
Specific Domains, Country-Specific Domains, and TLDs Specific
Domains.
Over 4.5 Million domains are registered each month.
WhoisFreaks database contains only unique Whois records of the
domains.
WhoisFreaks JSON response looks like this:
{
"status": true,
"domain_name": "google.com",
"query_time": "2021-03-12 08:03:53",
"whois_server": "whois.markmonitor.com",
"domain_registered": "yes",
"create_date": "1997-09-15",
"update_date": "2019-09-09",
"expiry_date": "2028-09-13",
"domain_registrar": {
"iana_id": "292",
"registrar_name": "MarkMonitor, Inc.",
"whois_server": "whois.markmonitor.com",
"website_url": "http://www.markmonitor.com",
"email_address": "abusecomplaints#markmonitor.com",
"phone_number": "+1.2083895770"
},
"registrant_contact": {
"company": "Google LLC",
"state": "CA",
"country_name": "United States",
"country_code": "US",
"email_address": "Select Request Email Form at https://domains.markmonitor.com/whois/google.com"
},
"administrative_contact": {
"company": "Google LLC",
"state": "CA",
"country_name": "United States",
"country_code": "US",
"email_address": "Select Request Email Form at https://domains.markmonitor.com/whois/google.com"
},
"technical_contact": {
"company": "Google LLC",
"state": "CA",
"country_name": "United States",
"country_code": "US",
"email_address": "Select Request Email Form at https://domains.markmonitor.com/whois/google.com"
},
"name_servers": [
"ns1.google.com",
"ns2.google.com",
"ns3.google.com",
"ns4.google.com"
],
"domain_status": [
"clientTransferProhibited",
"clientDeleteProhibited",
"serverDeleteProhibited",
"serverTransferProhibited",
"serverUpdateProhibited",
"clientUpdateProhibited"
],
"whois_raw_domain": "\nDomain Name: google.com\nRegistry Domain ID: 2138514_DOMAIN_COM-VRSN\nRegistrar WHOIS Server: whois.markmonitor.com\nRegistrar URL: http://www.markmonitor.com\nUpdated Date: 2019-09-09T08:39:04-0700\nCreation Date: 1997-09-15T00:00:00-0700\nRegistrar Registration Expiration Date: 2028-09-13T00:00:00-0700\nRegistrar: MarkMonitor, Inc.\nRegistrar IANA ID: 292\nRegistrar Abuse Contact Email: abusecomplaints#markmonitor.com\nRegistrar Abuse Contact Phone: +1.2083895770\nDomain Status: clientUpdateProhibited (https://www.icann.org/epp#clientUpdateProhibited)\nDomain Status: clientTransferProhibited (https://www.icann.org/epp#clientTransferProhibited)\nDomain Status: clientDeleteProhibited (https://www.icann.org/epp#clientDeleteProhibited)\nDomain Status: serverUpdateProhibited (https://www.icann.org/epp#serverUpdateProhibited)\nDomain Status: serverTransferProhibited (https://www.icann.org/epp#serverTransferProhibited)\nDomain Status: serverDeleteProhibited (https://www.icann.org/epp#serverDeleteProhibited)\nRegistrant Organization: Google LLC\nRegistrant State/Province: CA\nRegistrant Country: US\nRegistrant Email: Select Request Email Form at https://domains.markmonitor.com/whois/google.com\nAdmin Organization: Google LLC\nAdmin State/Province: CA\nAdmin Country: US\nAdmin Email: Select Request Email Form at https://domains.markmonitor.com/whois/google.com\nTech Organization: Google LLC\nTech State/Province: CA\nTech Country: US\nTech Email: Select Request Email Form at https://domains.markmonitor.com/whois/google.com\nName Server: ns3.google.com\nName Server: ns2.google.com\nName Server: ns1.google.com\nName Server: ns4.google.com\nDNSSEC: unsigned\nURL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/\n>>> Last update of WHOIS database: 2021-03-11T22:57:36-0800 <<<\n\nFor more information on WHOIS status codes, please visit:\n https://www.icann.org/resources/pages/epp-status-codes\n\nIf you wish to contact this domainâs Registrant, Administrative, or Technical\ncontact, and such email address is not visible above, you may do so via our web\nform, pursuant to ICANNâs Temporary Specification. To verify that you are not a\nrobot, please enter your email address to receive a link to a page that\nfacilitates email communication with the relevant contact(s).\n\nWeb-based WHOIS:\n https://domains.markmonitor.com/whois\n\nIf you have a legitimate interest in viewing the non-public WHOIS details, send\nyour request and the reasons for your request to whoisrequest#markmonitor.com\nand specify the domain name in the subject line. We will review that request and\nmay ask for supporting documentation and explanation.\n\nThe data in MarkMonitorâs WHOIS database is provided for information purposes,\nand to assist persons in obtaining information about or related to a domain\nnameâs registration record. While MarkMonitor believes the data to be accurate,\nthe data is provided \"as is\" with no guarantee or warranties regarding its\naccuracy.\n\nBy submitting a WHOIS query, you agree that you will use this data only for\nlawful purposes and that, under no circumstances will you use this data to:\n (1) allow, enable, or otherwise support the transmission by email, telephone,\nor facsimile of mass, unsolicited, commercial advertising, or spam; or\n (2) enable high volume, automated, or electronic processes that send queries,\ndata, or email to MarkMonitor (or its systems) or the domain name contacts (or\nits systems).\n\nMarkMonitor reserves the right to modify these terms at any time.\n\nBy submitting this query, you agree to abide by this policy.\n\nMarkMonitor Domain Management(TM)\nProtecting companies and consumers in a digital world.\n\nVisit MarkMonitor at https://www.markmonitor.com\nContact us at +1.8007459229\nIn Europe, at +44.02032062220\n--",
"registry_data": {
"domain_name": "GOOGLE.COM",
"query_time": "2021-03-12 08:03:52",
"whois_server": "whois.verisign-grs.com",
"domain_registered": "yes",
"create_date": "1997-09-15",
"update_date": "2019-09-09",
"expiry_date": "2028-09-14",
"domain_registrar": {
"iana_id": "292",
"registrar_name": "MarkMonitor Inc.",
"whois_server": "whois.markmonitor.com",
"website_url": "http://www.markmonitor.com",
"email_address": "abusecomplaints#markmonitor.com",
"phone_number": "+1.2083895740"
},
"name_servers": [
"NS2.GOOGLE.COM",
"NS1.GOOGLE.COM",
"NS4.GOOGLE.COM",
"NS3.GOOGLE.COM"
],
"domain_status": [
"clientDeleteProhibited",
"clientTransferProhibited",
"serverDeleteProhibited",
"serverTransferProhibited",
"serverUpdateProhibited",
"clientUpdateProhibited"
],
"whois_raw_registery": "\n Domain Name: GOOGLE.COM\n Registry Domain ID: 2138514_DOMAIN_COM-VRSN\n Registrar WHOIS Server: whois.markmonitor.com\n Registrar URL: http://www.markmonitor.com\n Updated Date: 2019-09-09T15:39:04Z\n Creation Date: 1997-09-15T04:00:00Z\n Registry Expiry Date: 2028-09-14T04:00:00Z\n Registrar: MarkMonitor Inc.\n Registrar IANA ID: 292\n Registrar Abuse Contact Email: abusecomplaints#markmonitor.com\n Registrar Abuse Contact Phone: +1.2083895740\n Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited\n Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited\n Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited\n Domain Status: serverDeleteProhibited https://icann.org/epp#serverDeleteProhibited\n Domain Status: serverTransferProhibited https://icann.org/epp#serverTransferProhibited\n Domain Status: serverUpdateProhibited https://icann.org/epp#serverUpdateProhibited\n Name Server: NS1.GOOGLE.COM\n Name Server: NS2.GOOGLE.COM\n Name Server: NS3.GOOGLE.COM\n Name Server: NS4.GOOGLE.COM\n DNSSEC: unsigned\n URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/\n>>> Last update of whois database: 2021-03-12T07:03:38Z <<<\n\nFor more information on Whois status codes, please visit https://icann.org/epp\n\nNOTICE: The expiration date displayed in this record is the date the\nregistrar's sponsorship of the domain name registration in the registry is\ncurrently set to expire. This date does not necessarily reflect the expiration\ndate of the domain name registrant's agreement with the sponsoring\nregistrar. Users may consult the sponsoring registrar's Whois database to\nview the registrar's reported date of expiration for this registration.\n\nTERMS OF USE: You are not authorized to access or query our Whois\ndatabase through the use of electronic processes that are high-volume and\nautomated except as reasonably necessary to register domain names or\nmodify existing registrations; the Data in VeriSign Global Registry\nServices' (\"VeriSign\") Whois database is provided by VeriSign for\ninformation purposes only, and to assist persons in obtaining information\nabout or related to a domain name registration record. VeriSign does not\nguarantee its accuracy. By submitting a Whois query, you agree to abide\nby the following terms of use: You agree that you may use this Data only\nfor lawful purposes and that under no circumstances will you use this Data\nto: (1) allow, enable, or otherwise support the transmission of mass\nunsolicited, commercial advertising or solicitations via e-mail, telephone,\nor facsimile; or (2) enable high volume, automated, electronic processes\nthat apply to VeriSign (or its computer systems). The compilation,\nrepackaging, dissemination or other use of this Data is expressly\nprohibited without the prior written consent of VeriSign. You agree not to\nuse electronic processes that are automated and high-volume to access or\nquery the Whois database except as reasonably necessary to register\ndomain names or modify existing registrations. VeriSign reserves the right\nto restrict your access to the Whois database in its sole discretion to ensure\noperational stability. VeriSign may restrict or terminate your access to the\nWhois database for failure to abide by these terms of use. VeriSign\nreserves the right to modify these terms at any time.\n\nThe Registry database contains ONLY .COM, .NET, .EDU domains and\nRegistrars."
}
}
I hope it would be the best solution that will help you in purchasing the domain names.
Disclaimer: I work for WhoisFreaks API & Database.
For this you can also try the website www.whoisdb.co, Here you will get instant access to a 100M+ whois database. You will get whois information like
"num","domain_name","query_time","create_date","update_date","expiry_date","domain_registrar_id","domain_registrar_name","domain_registrar_whois","domain_registrar_url","registrant_name","registrant_company","registrant_address","registrant_city","registrant_state","registrant_zip","registrant_country","registrant_email","registrant_phone","registrant_fax","administrative_name","administrative_company","administrative_address","administrative_city","administrative_state","administrative_zip","administrative_country","administrative_email","administrative_phone","administrative_fax","technical_name","technical_company","technical_address","technical_city","technical_state","technical_zip","technical_country","technical_email","technical_phone","technical_fax","billing_name","billing_company","billing_address","billing_city","billing_state","billing_zip","billing_country","billing_email","billing_phone","billing_fax","name_server_1","name_server_2","name_server_3","name_server_4","domain_status_1","domain_status_2","domain_status_3","domain_status_4"
Some key hights
Download more than 50000 whois leads of newly registered domains everyday
Every day, more than 100,000 domains are added to the list created with whois leads information.
You can even use their API or Webhooks to get the databse
All database can also be downloaded in the form of xls file or zip file
OUTGOING WEBHOOK
When the event below is triggered inside the software, send and POST a JSON to the provided URL.
Preview of JSON
{
"webhook_event": "new_domain_registered",
"num": "value",
"domain_name": "value",
"query_time": "value",
"create_date": "value",
"update_date": "value",
"expiry_date": "value",
"domain_registrar_id": "value",
"domain_registrar_name": "value",
"domain_registrar_whois": "value",
"domain_registrar_url": "value",
"registrant_name": "value",
"registrant_company": "value",
"registrant_address": "value",
"registrant_city": "value",
"registrant_state": "value",
"registrant_zip": "value",
"registrant_country": "value",
"registrant_email": "value",
"registrant_phone": "value",
"registrant_fax": "value",
"administrative_name": "value",
"administrative_company": "value",
"administrative_address": "value",
"administrative_city": "value",
"administrative_state": "value",
"administrative_zip": "value",
"administrative_country": "value",
"administrative_email": "value",
"administrative_phone": "value",
"administrative_fax": "value",
"technical_name": "value",
"technical_company": "value",
"technical_address": "value",
"technical_city": "value",
"technical_state": "value",
"technical_zip": "value",
"technical_country": "value",
"technical_email": "value",
"technical_phone": "value",
"technical_fax": "value",
"billing_name": "value",
"billing_company": "value",
"billing_address": "value",
"billing_city": "value",
"billing_state": "value",
"billing_zip": "value",
"billing_country": "value",
"billing_email": "value",
"billing_phone": "value",
"billing_fax": "value",
"name_server_1": "value",
"name_server_2": "value",
"name_server_3": "value",
"name_server_4": "value",
"domain_status_1": "value",
"domain_status_2": "value",
"domain_status_3": "value",
"domain_status_4": "value"
}
This website allows to download lists of registered domain names: https://networksdb.io .
Some are free, some are paid for. It allows also to access aggregates WHOIS data for IP blocks to find which companies own what address blocks and the other way around.