I'm trying to find a way to list all registered domains under a top-level domain (TLD). I.e. everything under .com, .net, etc. All the tools I find only applies to finding subdomains under a domain.
The information you seek isn't openly available. However, there are a few options you can try:
You might want to try inquiring at the respective registries directly about getting access to the Zone files. However, the process can take weeks and some registries choose not to offer access at all. For newer GTLDs you can apply at ICANN's Centralized Zone Data Service. You might need to provide a good reason to access the full lists. The Zone file can only be pulled once a day, though, so for more up to date information the only option is a paid service.
Whois API offers the entire whois database download in major GTLDs (.com, .net, .org, .us, .biz, .mobi, etc). It also provides archived historic whois database in both parsed and raw format for download as CSV files, as well as a daily download of newly registered domains.
A similar, popular question exists already but the answers and links are a bit outdated.
Related
I have one G Suite account with three domains (mainly many emails).
Now I would like to separate one domain to other fresh G Suite account. Of course with historical emails.
Anybody know how to do this? It's possible without G Suite support team?
There are different ways of doing this and yes, you can do it yourself. Please consider that it can be tricky depending on the amount of data you want to move.
The main steps are:
Create a new account with a temporary domain (the same domain can not be in two consoles at the same time)
Migrate all the contents from your current account to the new one. You have different options to do this. The cheapest one is the Data Migration Service (DMS) that will only allow you to migrate only email using the IMAP protocol (so you need to know the users' passwords). Google support for the DMS is best effort, so if you have time and budget I recommend to use a commercial tool (my tool of choice here is Cloud Migrator) that is also able to migrate calendar items and Google Drive files using the Google APIs (so it is transparent for the users).
In a cut-off date that you agree with your users you remove the domain from the original console, you add it to the new one and perform a mass rename (my tool of choice here is GAM).
There are many variables that can make the process much more complex that are difficult to describe in a single answer and this kind of activities usually require a dedicated project and a (small) team: I really suggest you to get some help.
A word of warning:
I did this on Bluehost (transferring G Suite ownership of a domain from one Bluehost account to another) through Bluehost customer support.
It took 3 separate calls to Bluehost to get everything fully moved over. Make sure that your hosting provider sends you an email confirmation that everything happened successfully, because as a reseller of a Google product, they may not have complete authority over transferring ownership of Google's product.
Best regards.
Also in reference to the domain go to Google Domain and you can see all the options you have in reference to your domain website etc. It is very self-explanatory.
wc.
Searched the web and unable to find a solution. I have an umbraco site using IIS to host on a Windows server. Any ideas on approach to block users accessing site outside the UK? Htaccess approach would be too slow.... thank you in advance!
That's quite hard to do accurately, as you could have someone based in the UK using a European network provider, which means that they might appear to come from say Holland instead of the UK. It's also possible for people to spoof their location fairly easily if they really want to get at your site.
As Lex Li mentions there are plenty of commercial databases and tools for looking up a user's location, but the accuracy of these varies considerably, not to mention the fact that some of them only support IPv4. Any of these options are going to be slow though, as you'll have to check on every request. You also have to make sure you keep the databases up to date.
Another option would be to proxy your site through something like CloudFront or CloudFlare which both support blocking traffic by country.
To PROGRAMMATICALLY verify if a domain exists I do the following:
DNS Query it and see if it resolves. If it does, it's obviously registered. So no need for step 2. If it doesn't, it might STILL be registered. So a whois check is required.
Backtrack from whois.iana.org and see if the designated whois server knows the domain or not.
Well, whois is not really meant for bulk checking. Not to mention that the RFC has only 4 pages and there's no clear specifications as to the format or even the encoding of the data. So you pretty much have to train the parser for each specific answer format (server).
Is there a way to circumvent the whois query and check (as close to the metal as possible) if the domain is registered in another (publicly available) standardized (preferably free or affordable) way? And not by downloading the TLD zone file or using third-party APIs (as they have a bad habit of snatching domains that you check before you get to register them). :)
I know registrars have their own protocol but I'm not sure if it's open to public use.
There isn't really any good way to do this accurately without looking at zone files or checking directly with the registry, unfortunately.
Registrars typically use a protocol like EPP to talk to a registry, check name availability and place orders. It's unlikely that anyone other than an accredited registrar would be permitted to use this protocol, but it may be worth checking with the registry that manages the TLDs you are interested in, e.g. Verisign.
I'd (personally) be wary of relying too much on DNS queries or WHOIS lookups to ascertain whether a particular domain exists, as both can produce inaccurate results from time to time. For example, certain TLDs have name servers configured for any unregistered domain name (they often direct you to the registry's website). The Vietnamese registry is one example of this. WHOIS lookups can fail for any number of reasons, so lack of a record is not concrete evidence of the domain's availability.
I have a website, hosted on a shared server.
Through CPANEL, I am provided with a few traffic analysis logs and tools.
None seem to provide what I'm looking for.
For each day, I'd like to see log file with a list of unique visitors.
Under each unique visitor (by IP address), I'd like to see the following information
geographic location (based on IP address)
information to help determine if the visitor was a bot or human
the page URLs they requested (including the exact time of request)
explanation of my application:
I run a forum on my site. I'd like a better understanding of who is visiting, when they visit and how
they navigate through my forum pages (topics, posts etc.)
I would appreciate some direction on how to develop this (a script is probably best)
I would (and do) use Google Analytics as it gives you exactly what you are asking for and a whole lot more (like being able to see live what is happening). It requires you to add some javascript code to the application (which for so many today, plugins are available).
If no plugin is available, see https://support.google.com/analytics/answer/1008080?hl=en
This approach to your end result will typically be a lot easier than trying to create your own log analyser and installing it on a shared cPanel server.
There are online tools (i.e. http://www.swola.com/) for discovering newly available domains.
Given a corpus of domain names, it is easy to periodically check their records and raise a flag when one becomes available. However, checking periodically the records for all registered domain names in the world sounds excessive/impractical.
What is an efficient way to programmatically discover such domains? How such tools work?
I don't have enough experience with the data to determine how up to date it is (for example if someone renews a domain a week before it expires, how quickly is it removed from the list?) but I discovered that pool.com has a database of upcoming domains sorted by expiration date. It's a relatively simple matter to create a cron job that downloads this file, parses it for the specific domains you want (for example, domains that expire tomorrow, domains with only characters a-z, domains up to 9 characters, domains with KEYWORD in them, .coms, etc) and emails you a daily report.
One thing that I would like to hear about is how others are ordering these domains as soon as they become available. I know some registrars offer that service but usually at a great expense.
Here's my script, I threw it together in a half an hour but hopefully it will be useful to you.