SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    SitePoint Member
    Join Date
    Apr 2000
    Location
    Bhopal, M.P., India
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hello,

    I am listing my site with a search engine, but i want to secure a few HTML documents i.e. I do not want them to be listed in the search results of the search engine even if someone types the related keywords. How do i do that?

    Thanks in advance.

  2. #2
    SitePoint Author Kevin Yank's Avatar
    Join Date
    Apr 2000
    Location
    Melbourne, Australia
    Posts
    2,571
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    There's no way to absolutely guarantee that a page won't be indexed, but the following link explains a method that most search engines will respect:
    http://info.webcrawler.com/mak/proje...ion-admin.html


    ------------------
    -Kevin Yank.
    http://www.SitePoint.com/
    Helping Small Business Grow Online!

  3. #3
    SitePoint Wizard
    Join Date
    Jul 1999
    Location
    Chicago
    Posts
    2,629
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    That'll teach you how to make robots.txt files, but they aren't acceptable for blocking out search engine spiders.

    Many, many large sites disable robots.txt including IBM: http://www.ibm.com/robots.txt and Sun: http://www.sun.com/robots.txt . It looks almost as if Sun is hiding the fact that they have subscriber-only downloads (I may be wrong) by saying they aren't.

    The bottom line is, if a robot can view it and know what not to index (i.e. secret files), then anyone can know and visit them.

  4. #4
    SitePoint Zealot
    Join Date
    May 2000
    Posts
    150
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Hello,

    You could place the relevant docs in a new directory and password protect them right ?

    Arpith


    ------------------
    http://www.arpith.com/

  5. #5
    SitePoint Wizard
    Join Date
    Jul 1999
    Location
    Chicago
    Posts
    2,629
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You certainly could. Shalini3010, what level of protection do you need? From the search engines or from all visitors? If it's from all visitors, go to http://www.realmz.net/showarticle.ph...b6940e5c&aid=1 (on RAGE's site...he's a member in these forums)

    ------------------
    Who the heck is Gen. Failure and why does he want to read my hard drive?
    Soon to come: Aspology.com


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •