Does anybody know anything about a search engine called Abrave? They keep spidering my sites, but they don’t respect robots.txt or “nofollow” links, which makes me a little suspicious of them. Other than this, I’ve never heard of them.
They are an aspiring startup company, seem really shady to me . Contact them and tell them to put your site on a “no-follow” list so they don’t crawl, rank or index anything on your site.
It is a directory too.
Me too. Even more so now I’ve just discovered the domain is up for sale.
I didn’t realise I could do that - I’ll give it a try. But will they take any notice, given that they ignore my robots.txt files?
If you don’t try you’ll never know
I’ve contacted them and await their reply.
I found this on their site:
As of release 2.7 (5th March 2010), a correction has been implemented to the Abrave Spider to fix a problem where on occasion disallowed pages listed in the robots.txt files of some websites were still being accessed even though these were correctly not indexed by Abrave.
Doesn’t inspire much confidence.
I’ve had a reply and they say they’ve removed my sites from their search results. They are less clear about whether they will refrain from crawling them in future. We shall see.