Unknown bot

After fought against bandwith usage with Baidu I am now fighting against an “Unknown bot” at least it is is how SmarterStats describes that bots who is hammering my website. So far this month 28Gb of bandwith was used by that bot, just on par with Google. But as it appears in my stats as unknown I can’t stop or at least somehow slow it down.

A couple of days ago I banned the IP address but today I am having an “Unknown bot” from IP

I contacted SingleHop, the organization behing but I didn’t have an answer about the actual people behind this ip. I am now contacting Choopa.com for the other IP.

Any clue on which bot it could be ?

There are lots of nasty spam bots out there, so my guess is it’s one of them. You could perhaps try to tell it to go away in your robots file, but I doubt it would take any notice.


Out of interest, have you got caching set up on your site?

Not sure how to find that out. I am using IIS by the way

I got answer: one bot was Blexbot and the other one was [URL=“https://ahrefs.com/robot/”]Ahrefs

Do you know what CMS you are using? Assuming that your site is based on ASP.NET, most CMS’s will have caching enabled by default, but it is a good idea to check. Caching will go a long way.

Home made coding

Banning the IP address is probably the best thing you can do. The bot might well come back from another address, so you will need to be vigilant, and keep on banning it. (Normally, I would suggest that you don’t bother, but if it really is stealing 28 GB a month - which sounds like a heck of a lot - you do need to do something about it.)


I second @Mikl’s comments on this. I’d also look into setting up a caching layer for your site. 28GB is a lot, even for a huge website.

It is not even a huge website, it’s pretty small with low traffic according to Analytics. I just a look at SmarterStats and found out that traffic from China is causing me those headache. 91% of the traffic is coming from there burning 32Gb of bandwith whereas my website is a directory of business for a region in Chile ! (using 2gb of bandwith) !!

It might be worth just banning the whole darned range of IPs for China. Some sites keep an updated list of them, such as IP2Nation.

I thought of suggesting the same thing, but I didn’t, mainly because China is a huge market, and there might well be potential visitors from there that you wouldn’t want to lose. Of course, if you’re sure that your site contains nothing that would interest a genuine Chinese visitor, that’s another matter, but I wouldn’t jump to that conclusion.

By the way, the two IP addresses that Corobori mentioned are both in the United States.


:lol: Yes, the only way to stop this completely is to ban everyone … which no doubt will happen some time in the future when we all get sick of the www.

China is surely a huge market but I doubt Chinese people have much interest in a website in Spanish which is focused on a particular Chilean region. Surely not to attract 977,322 visitors in August so far whereas visitors from Chile were 27,209 (according to SmarterStats). By the way Google Analytics doesn’t “see” any significant traffic from China, actually it says 3 (three) visitors from China for the same period.

I know the 2 IPs I pointed to originally are from the US but I guess I bumped I wouldn’t have noticed them if I hadn’t had the main issue with China which is siphonning off all this bandwidth, I just saw that for august so far it’s 80Gb.

In that case, it might well make sense to block all IPs from China.


Definitely something going wrong…

This is by month: oh boy I’d love having actually 1M visitors in a month on my website !!!