I have recently become acquainted with robots.txt files, but have found them of quite limited use. Is there a way of excluding all spiders/robots from all files except index.htm, except for the Freefind spider which I use to allow people to internally search my site?

------------------
--
Alexis