SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    SitePoint Enthusiast
    Join Date
    Apr 2003
    Location
    PHP World
    Posts
    57
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    how i can know if the visitor is spider

    hello

    how i can know if the visitor is spider ? any tips

  2. #2
    SitePoint Zealot ashutosh_sns's Avatar
    Join Date
    Aug 2004
    Location
    New Delhi , india
    Posts
    119
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Ashutosh Tripathi
    Home Page

  3. #3
    SitePoint Evangelist
    Join Date
    Feb 2005
    Posts
    581
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You could possibly put all known spiders into a CSV file and then check the user agent against the values in that file each time the script is called. Could get complicated/messy, though.
    I will not flame the newbies,
    I will not flame the newbies,
    I will flame the newbies...
    Table free is the way to be!

  4. #4
    SitePoint Zealot
    Join Date
    Feb 2005
    Posts
    136
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    technically you can't, unless you had a master list of all spiders.

    that would be like trying to guess my nationality by hearing my name. you could probably guess right most of the time, but you could never be 100% sure.

  5. #5
    SitePoint Enthusiast
    Join Date
    Oct 2004
    Location
    Australia
    Posts
    84
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    The short answer is: 'You can't tell whether a visitor is a spider or not'.

    The long answer is:
    You could do any or all of the following:
    • Keep a list of known spiders and compare the browser identity to that
    • Keep a list of known browsers and assume anything that is not a browser is a spider
    • Match certain keywords that usually indicate a crawler - eg spider, crawler etc.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •