Preventing the search engine to index some html pages in my site

[shalini3010]

Hello,

I am listing my site with a search engine, but i want to secure a few HTML documents i.e. I do not want them to be listed in the search results of the search engine even if someone types the related keywords. How do i do that?

Thanks in advance.

[Kevin Yank]

There's no way to absolutely guarantee that a page won't be indexed, but the following link explains a method that most search engines will respect:
http://info.webcrawler.com/mak/proje...ion-admin.html


------------------
-Kevin Yank.
http://www.SitePoint.com/
Helping Small Business Grow Online!

[qslack]

That'll teach you how to make robots.txt files, but they aren't acceptable for blocking out search engine spiders.

Many, many large sites disable robots.txt including IBM: http://www.ibm.com/robots.txt and Sun: http://www.sun.com/robots.txt . It looks almost as if Sun is hiding the fact that they have subscriber-only downloads (I may be wrong) by saying they aren't.

The bottom line is, if a robot can view it and know what not to index (i.e. secret files), then anyone can know and visit them.

[arpith]

Hello,

You could place the relevant docs in a new directory and password protect them right ?

Arpith


------------------
http://www.arpith.com/

[qslack]

You certainly could. Shalini3010, what level of protection do you need? From the search engines or from all visitors? If it's from all visitors, go to http://www.realmz.net/showarticle.ph...b6940e5c&aid=1 (on RAGE's site...he's a member in these forums)

------------------
Who the heck is Gen. Failure and why does he want to read my hard drive?
Soon to come: Aspology.com