SitePoint Sponsor

User Tag List

Results 1 to 6 of 6
  1. #1
    Working on it... Contrid's Avatar
    Join Date
    Apr 2006
    Location
    Online
    Posts
    955
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    Unhappy Redirect Bad/Spam Traffic

    Hey guys!

    I hope this is the right forum for this.

    Someone created approximately 2000+ bad/spam backlinks to a specific page on our site from porn sites and all sorts of other bad sources. I don't know what the reason or purpose of this is and whether it is competition trying to harm us or what not.

    The result is that it is and has hurt our Google SERPs so we need to do something about it. Right now I've created a 301 redirect to a 404 page for this specific page on our site so that it would go through to a 404 Not Found.

    I assume the best way is to do a 410 Gone header so that Google would know but I'm not sure so I need your help please.

    Since they link to this specific page, the problem seems easy to solve.
    What do we do? Where do we redirect to and how do we redirect (which status)?
    What is your recommendation?

    Thank you for your input, greatly appreciated!
    And so I got lost in code...completely asphyxiated by it...

    Premium WordPress plugins - Tribulant Software

  2. #2
    Foozle Reducer ServerStorm's Avatar
    Join Date
    Feb 2005
    Location
    Burlington, Canada
    Posts
    2,699
    Mentioned
    89 Post(s)
    Tagged
    2 Thread(s)
    Hi Contrid,

    The most effective way to block spammer/bad bots is to use a firewall that supports intrusion detection, DOS attacks and host filtering.

    However you may not have access to such a beast so you can do some things with your .htaccess code.

    You can build a list of bad HTTP_REFERERs each being like:
    Code:
    /* check if the referer is a given bad domain */
    RewriteCond %{HTTP_REFERER} baddomain1\.com [OR,NC]
    RewriteCond %{HTTP_REFERER} baddomain2\.com [NC]
    /* block them */
    RewriteRule .? - [F,L]
    You can filter traffic from certain countries is covered nicely here: http://www.sitepoint.com/forums/show...ht=bad+traffic

    You can block bad robots except the ones you want. This is hit or miss as bots don't always get caught but it can help:

    Code:
    /* If it is the robots text then skip */
    RewriteRule ^robots\.txt - [S=1]
    
    /* If any of these AGENTS and not the search engines then block */
    RewriteCond %{HTTP_USER_AGENT} ^$ [OR,NC]
    RewriteCond %{HTTP_USER_AGENT} spider [OR,NC]
    RewriteCond %{HTTP_USER_AGENT} crawl [OR,NC]
    RewriteCond %{HTTP_USER_AGENT} bot [NC]
    RewriteCond %{HTTP_USER_AGENT} !(googlebot|bingbot|msnbot) [NC]
    RewriteRule .? - [F]
    A great article with good advice can be found here: http://en.linuxreviews.org/HOWTO_sto...sing_.htaccess

    As your original quest is to use 410 or a 301, do a 410; eventually the search engines will learn to not index this page and crawl it.

    Hope this helps,
    Steve
    Last edited by ServerStorm; Jan 30, 2013 at 11:12.
    ictus==""

  3. #3
    Certified Ethical Hacker silver trophybronze trophy dklynn's Avatar
    Join Date
    Feb 2002
    Location
    Auckland
    Posts
    14,604
    Mentioned
    19 Post(s)
    Tagged
    2 Thread(s)
    Steve,

    Strange: You correctly used the OR flag in the second code group but not the first. Please correct as you want to ban bad1 OR bad2 OR ... OR badn.

    Regards,

    DK
    David K. Lynn - Data Koncepts is a long-time WebHostingBuzz (US/UK)
    Client and (unpaid) WHB Ambassador
    mod_rewrite Tutorial Article (setup, config, test & write
    mod_rewrite regex w/sample code) and Code Generator

  4. #4
    Foozle Reducer ServerStorm's Avatar
    Join Date
    Feb 2005
    Location
    Burlington, Canada
    Posts
    2,699
    Mentioned
    89 Post(s)
    Tagged
    2 Thread(s)
    Quote Originally Posted by dklynn View Post
    Steve,

    Strange: You correctly used the OR flag in the second code group but not the first. Please correct as you want to ban bad1 OR bad2 OR ... OR badn.

    Regards,

    DK
    Whoops, well I've fixed it. Thanks DK!
    ictus==""

  5. #5
    Certified Ethical Hacker silver trophybronze trophy dklynn's Avatar
    Join Date
    Feb 2002
    Location
    Auckland
    Posts
    14,604
    Mentioned
    19 Post(s)
    Tagged
    2 Thread(s)
    Steve,

    I **KNOW** that you know better but it's better for you to fix your code than anyone else post a "rebuttal."

    The hurrier I go the behinder I get, too.

    Regards,

    DK
    David K. Lynn - Data Koncepts is a long-time WebHostingBuzz (US/UK)
    Client and (unpaid) WHB Ambassador
    mod_rewrite Tutorial Article (setup, config, test & write
    mod_rewrite regex w/sample code) and Code Generator

  6. #6
    Working on it... Contrid's Avatar
    Join Date
    Apr 2006
    Location
    Online
    Posts
    955
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi Steve,

    Thank you for the advice, greatly appreciated.

    I cannot check the referrer as there are hundreds.
    I just shut down the specific page/URL with a 404 for now.
    And so I got lost in code...completely asphyxiated by it...

    Premium WordPress plugins - Tribulant Software


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •