SitePoint Sponsor

User Tag List

Results 1 to 3 of 3
  1. #1
    ********* Scotland Saltire's Avatar
    Join Date
    Jan 2000
    Location
    Scotland
    Posts
    660
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Preventing certain parts of page from being spidered

    I would like to prevent the header of a page being spidered, but want the rest of the content to be spidered. I remember a while ago somebody requested to code to do this but a search for it did not prove fruitful.

    For instance,

    <b>Hello</b>
    <b>Test</b>

    I just want "Test" to be spidered, so what code do I put round "Hello" to prevent it from being spidered?

  2. #2
    We like music. weirdbeardmt's Avatar
    Join Date
    May 2001
    Location
    Channel Islands Girth: Footlong
    Posts
    5,882
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I know we talked about this over ICQ Saltire, but figured some people might like to know about my suggestions...

    either a) have a look at www.robotstxt.org to view info on how to control robot/spider access. As far as I can tell from that, you can only disallow access to an entire page.

    Secondly you could write the code you don't want spidered in JS (using document.write) as spiders won't index script.
    I swear to drunk I'm not God.
    Matt's debating is not a crime
    Hint: Don't buy a stupid dwarf Clicky

  3. #3
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,939
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes... the only way to do it is to use javascript.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •