2

I am looking for a fast wat to test if a domain name is in use, hence has a site attached to it. I have a huge list of domain names and need to check which ones have a site attached to it or not.

If I know that there is a site attached to it then I don't need to do any time consuming whois lookups, etc.

My initial idea was to check for HTTP header code and see if the header codes return 200, 301, 302... if is is something else there is a high chance that there is no site attached or that the domain name is available. The code snipper I am using is:

$headers = @get_headers( $url);
$headers = (is_array($headers)) ? implode( "\n ", $headers) : $headers;
return (bool)preg_match('#^HTTP/.*\s+[(200|301|302)]+\s#i', $headers);

However this process is pretty slow.

Next I've tried using the PHP function checkdnsrr to check if a DNS record exist. I am using the following code snipper.

if (checkdnsrr($domain . ".", "A") == false) {
   echo "Domain might be available";
} else {
   echo "Domain is in use";
}

Unfortunately it gives me a lot of "false" information saying that the domain might be available. The same goes for the following code snippet:

if (gethostbyname($domain) != $domain) {
  echo "DNS Record found";
} else {
  echo "NO DNS Record found";
}

A WHOIS lookup is a possibility also however with all the new TLD (.frl, .xyz, etc.) this might be a huge task to setup.

Another idea I had was to ping the domain name in question and if I get a response I know the domain name is "in use". Not sure if this is the best approach therefor I am asking you is there a a fast but more reliable way to check if domain is in use (or registered)?

Marc Witteveen
  • 743
  • 2
  • 10
  • 25
  • Note: There's not much you can do to make a WHOIS fast. You're reliant on the speed of the servers you're querying. If domain *registration* is an absolute requirement, WHOIS it is. – ceejayoz Apr 24 '16 at 20:05
  • Hi, I've updated my question. I am not looking for WHOIS data but a simple indicator that a domain is used or not (don't care of it is registered). On network level I would send a ping to a server/domain, but I am not sure if this is the most efficient way to do it. – Marc Witteveen Apr 24 '16 at 20:23
  • What are you actually trying to accomplish with this, then? What's the user case? – ceejayoz Apr 24 '16 at 20:24
  • I've got a huge dynamic list of domain names and I want to know which ones are in "use" and which ones are not. If the domain name is not in use I will investigate manually to see check out the domain name (e.g. backlinks, previous website trough internet archive, see if domain is available, etc.). – Marc Witteveen Apr 24 '16 at 20:28
  • 1
    Without a WHOIS, you're going to run into exactly the problem you described - domains that are registered but don't host a website. These are very, very common. So, you either deal with a lot of false negatives with your `checkdnsrr` or `gethostbyname` techniques, or you do a WHOIS. – ceejayoz Apr 24 '16 at 20:29
  • Before I dive into Google: are you maybe aware of a PHP library that can do WHOIS lookups for all the new TLD's? – Marc Witteveen Apr 24 '16 at 20:40
  • https://github.com/phpWhois/phpWhois looks fairly recently updated. – ceejayoz Apr 24 '16 at 20:47

1 Answers1

0

You could try parse "http://whois.domaintools.com/" . $domain

and look for the keyword "Domain Available"

Brian Min
  • 265
  • 4
  • 10
  • That would be an idea but I think if I make a lot of request to their site in a relative short time period they might ban my IP address. – Marc Witteveen Apr 24 '16 at 20:25