Redirect Bad/Spam Traffic

[Contrid]

Hey guys!

I hope this is the right forum for this.

Someone created approximately 2000+ bad/spam backlinks to a specific page on our site from porn sites and all sorts of other bad sources. I don't know what the reason or purpose of this is and whether it is competition trying to harm us or what not.

The result is that it is and has hurt our Google SERPs so we need to do something about it. Right now I've created a 301 redirect to a 404 page for this specific page on our site so that it would go through to a 404 Not Found.

I assume the best way is to do a 410 Gone header so that Google would know but I'm not sure so I need your help please.

Since they link to this specific page, the problem seems easy to solve.
What do we do? Where do we redirect to and how do we redirect (which status)?
What is your recommendation?

Thank you for your input, greatly appreciated!

[ServerStorm]

Hi Contrid,

The most effective way to block spammer/bad bots is to use a firewall that supports intrusion detection, DOS attacks and host filtering.

However you may not have access to such a beast so you can do some things with your .htaccess code.

You can build a list of bad HTTP_REFERERs each being like:

/* check if the referer is a given bad domain */
RewriteCond %{HTTP_REFERER} baddomain1\.com [OR,NC]
RewriteCond %{HTTP_REFERER} baddomain2\.com [NC]
/* block them */
RewriteRule .? - [F,L]

You can filter traffic from certain countries is covered nicely here: http://www.sitepoint.com/forums/show...ht=bad+traffic

You can block bad robots except the ones you want. This is hit or miss as bots don't always get caught but it can help:

/* If it is the robots text then skip */
RewriteRule ^robots\.txt - [S=1]

/* If any of these AGENTS and not the search engines then block */
RewriteCond %{HTTP_USER_AGENT} ^$ [OR,NC]
RewriteCond %{HTTP_USER_AGENT} spider [OR,NC]
RewriteCond %{HTTP_USER_AGENT} crawl [OR,NC]
RewriteCond %{HTTP_USER_AGENT} bot [NC]
RewriteCond %{HTTP_USER_AGENT} !(googlebot|bingbot|msnbot) [NC]
RewriteRule .? - [F]

A great article with good advice can be found here: http://en.linuxreviews.org/HOWTO_sto...sing_.htaccess

As your original quest is to use 410 or a 301, do a 410; eventually the search engines will learn to not index this page and crawl it.

Hope this helps,
Steve

[dklynn]

Steve,

Strange: You correctly used the OR flag in the second code group but not the first. Please correct as you want to ban bad1 OR bad2 OR ... OR badn.

Regards,

DK

[ServerStorm]

Steve,

Strange: You correctly used the OR flag in the second code group but not the first. Please correct as you want to ban bad1 OR bad2 OR ... OR badn.

Regards,

DK

Whoops, well I've fixed it. Thanks DK!

[dklynn]

Steve,

I **KNOW** that you know better but it's better for you to fix your code than anyone else post a "rebuttal."

The hurrier I go the behinder I get, too.

Regards,

DK

[Contrid]

Hi Steve,

Thank you for the advice, greatly appreciated.

I cannot check the referrer as there are hundreds.
I just shut down the specific page/URL with a 404 for now.