Hi
Is there a database that exists online for me to find out how many urls have a particular word used within them? For instance, say I want to find out how many domain names use the word “dog,” is there a way to find that out?
Thanks
1 Like
The closest I can think of is the Google Search Operator “related:” eg.
related:dog
https://www.google.com/search?q=related%3Adog&ie=utf-8&oe=utf-8
1 Like
Why do you want to do this?
1 Like
You need to remember that the web is growing so fast that even Google now has less than 10% of pages in its database. Of course no other resource has more than that so that’s the best you could get.
Yes, the number of possible domains in human terms might as well be considered infinite and the data never stable.
I was curious so I looked into it a bit more.
I first looked at ICANN but only noticed “legal” stuff
Then I thought maybe Whois might have something
This “Whois” site has several APIs and Services https://www.whoisxmlapi.com/
It might be more data than needed and out of budget depending on the situation, but it looks promising enough to justify a “contact” to find out more
1 Like
I am just curious how many websites use a particular word. 
I’ve looked a few times before and I couldn’t find anything. I am surprised there isn’t an easily queryable database of taken URLs out there.
If Google with all the resources they have working 24/7 is falling further and further behind in finding URLs (they got as high as about 17% of the web listed but have now fallen back below 10%). then who would have the resources to produce such a list?
The number of new URLs being added is currently about three times as many in a given period as Google can add to their list and the rate is accelerating.
Not that every IP has a domain name, But even with the inadequate IPV4 the number is extreme i.e.
https://en.wikipedia.org/wiki/IPv4
IPv4 uses 32-bit (four-byte) addresses, which limits the address space to 4294967296 (2^32) addresses.
And now with IPV5 the number is much more
https://en.wikipedia.org/wiki/IPv6
IPv6 uses a 128-bit address, theoretically allowing 2^128, or approximately 3.4×10^38 addresses. The actual number is slightly smaller, as multiple ranges are reserved for special use or completely excluded from use. The total number of possible IPv6 address is more than 7.9×10^28 times as many as IPv4, which uses 32-bit addresses and provides approximately 4.3 billion addresses.
I can’t even begin to envision a database capable of containing 4.3 billion records let alone 79000000000000000000000000000 times more than that
1 Like
Of course some IP addresses have hundreds of domain names so changes are that there are several times as many domain names as there are IPv4 addresses.
1 Like
This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.