.htaccess to stop msnbot?

Server Config
1

Hi,

For the last two days, our forum and our wiki has been taken offline for hours on end by what appears to be heavy bot traffic from the msnbot. Some searching around seems to suggest this is no longer the actual msnbot but third-parties renting the servers?

We’ve tried to restrict the msnbot via robots.txt, but it doesn’t seem to respect that. We also tried an .htaccess rewrite, like this:

RewriteCond %{HTTP_USER_AGENT} msnbot [NC]
RewriteRule . - [F]

This doesn’t seem to work either. It may be that the bot doesn’t always send its user agent, perhaps? Has anyone had any experience getting this particular bot to leave your site alone?

2

You did put

RewriteEngine On

in your .htaccess I assume?

3

Yes, sorry! Was just quoting the relevant bit, but good point. :slight_smile:

4

Okay :slight_smile:
So, are you sure the bot has “msnbot” in it’s name, exactly spelled like you did? What does your access log say?

5

If you know bot IP then you can block it in your .htaccess by adding code

<Limit GET POST>
order deny,allow
deny from x.x.x.x
allow from all
</Limit>

x.x.x.x = physical IP of bot

6

I need to get the access logs from my host, she said msnbot but I guess we need to confirm for sure. Will try the IP approach as well, thanks. :slight_smile:

Though it does seem like we have some other on-going issues with apache, so perhaps the bots were just the straw that broke the camels back.