SitePoint Sponsor

User Tag List

Results 1 to 3 of 3
  1. #1
    SitePoint Addict photoshopfreak's Avatar
    Join Date
    Aug 2004
    Location
    uk
    Posts
    267
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Whats this robot?

    Inktomi Slurp 1636+503 9.13 MB 20 Dec 2004 - 20:21
    MSNBot 1911+84 103.59 MB 20 Dec 2004 - 19:09
    Googlebot 1122+30 14.53 MB 20 Dec 2004 - 05:42
    Unknown robot (identified by 'crawl') 120+11 4.51 MB 20 Dec 2004 - 06:22
    Unknown robot (identified by hit on 'robots.txt') 0+130 19.42 KB 20 Dec 2004 - 17:24
    AskJeeves 22+21 160.57 KB 20 Dec 2004 - 10:13
    Alexa (IA Archiver) 30+7 1.10 MB 17 Dec 2004 - 19:33
    Unknown robot (identified by 'robot') 2+1 8.49 KB 16 Dec 2004 - 12:39
    ht://Dig 0+2 216 Bytes 20 Dec 2004 - 11:00

    I have that robot that is in orange on two of my sites log files, what is it?

  2. #2
    SitePoint Wizard silver trophy someonewhois's Avatar
    Join Date
    Jan 2002
    Location
    Canada
    Posts
    6,364
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You can google ht://Dig to get http://www.htdig.org/... not sure if that helps though. Seems like it's just another small/insignificant search engine, though I could be wrong.

  3. #3
    SitePoint Enthusiast
    Join Date
    Nov 2004
    Location
    Pittsburgh, PA
    Posts
    71
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ht://Dig is software that is used to put an search interface to your website.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •