i need to detect search engine bots/spiders/crawlers or whatever they're called.
at the moment my (php) script checks the user agent for -
"bot" and "spider" and "crawl"
so if the user agent contains those charecters (regardless of upper/lowercase) it'll detect correctly
can anyone see any problems with this? are there search engine bots that don't contain the above three words?
are there even normal browser that do contain the above three words (i doubt that but you never know)?