SitePoint Sponsor

User Tag List

Results 1 to 13 of 13
  1. #1
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    googlebot & help with Analog report

    Hi folks...

    I have a question that involves interpretation of my Analog reports from my web host. This is the report that I got fro this month so far:

    Listing browsers with at least 1 request for a page, sorted by the number of requests for pages.

    #: #reqs: #pages: browser
    --: -----: ------: -------
    1: 96591: 3607: MSIE
    : 76928: 2950: MSIE/6
    : 19533: 600: MSIE/5
    : 130: 57: MSIE/4
    2: 3247: 98: Netscape
    : 2284: 83: Netscape/6
    : 963: 15: Netscape/4
    3: 407: 73: http:
    : 407: 73: http://www
    4: 56: 56: InternetSeer.com
    5: 3221: 37: Mozilla
    : 2675: 30: Mozilla/1
    : 546: 7: Mozilla/0
    6: 10: 10: Scooter-3.2
    7: 174: 8: Netscape (compatible)
    8: 533: 7: Konqueror
    : 533: 7: Konqueror/2
    9: 496: 7: WebTV
    : 496: 7: WebTV/2
    10: 4: 4: ESLoop
    : 4: 4: ESLoop/0
    11: 3: 3: Googlebot
    : 3: 3: Googlebot/2
    12: 3: 3: Lynx
    : 3: 3: Lynx/2
    13: 70: 1: Opera
    : 30: 1: Opera/5
    : 72406: 0: [not listed: 1 browser]

    Does this mean that the googlebot has seen my site? Has it spidered my site? How can I check?

    The reason I am concerned is that my site was down for 24 hrs when my host unilaterally decided to re-assign IP addresses without asking me. My site was down for 24-36 hrs. Would the googlebot have shown up here if it had not been able to get through?

    A worried newbie webmaster...

    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  2. #2
    SitePoint Addict jbradley's Avatar
    Join Date
    May 2002
    Location
    Huntsville, AL
    Posts
    224
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I don't believe it would be shown there if it hadn't visited your site. Someone correct me if I'm wrong though. You could check your raw access logs to be sure about that because I'm not absolutely certain.

    Jason
    Jason Bradley
    Post your project and receive FREE bids from professionals!
    naSnap Freelance Marketplace

  3. #3
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    how do I examine access logs

    I've downloaded the raw access log from my web host...it is an ms-dos application. How do I view it?



    p
    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  4. #4
    SitePoint Addict jbradley's Avatar
    Join Date
    May 2002
    Location
    Huntsville, AL
    Posts
    224
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: how do I examine access logs

    Originally posted by Philip Toews
    I've downloaded the raw access log from my web host...it is an ms-dos application. How do I view it?



    p
    Should be just a text file I would think. If not, try to download it again and when it asks you to save it, save it as a text file. That way you can just open Notepad and view it. Some hosts also have an interface setup so that you can view it on the web.

    Jason
    Jason Bradley
    Post your project and receive FREE bids from professionals!
    naSnap Freelance Marketplace

  5. #5
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes, Googlebot visited your site.

    No need to view your access logs, thats what Analog is doing for you.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  6. #6
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    yes googlebot visited

    OK...I'm happy then! Thanks for answering my question. What do those stats mean exactly...that the googlebot(s) came and checked out three pages?

    So the googlbots came...what did they do? I'm serious...I'm curious to know what happens when the bots hit your site...

    How can I tell if the whole site was spidered?

    How can I find out what my search engine ranking is?

    How long after the spider crawls does my site get a listing?

    Sorry for these newbie questions...but I gotta learn somewhere!

    TIA

    ps. anywhere to suggest I can learn more about the arcane world of log reports and search engine ranking?
    Last edited by Philip Toews; Jul 4, 2002 at 03:23.
    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  7. #7
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It looks like Googlebot visited 6 pages. Google has a whole array of spiders that they send out, so your site can be visited by 15 or so different IP addresses that are all Googlebot.

    You won't know how you rank from this Crawl until the end of July. Google takes 3 weeks or so to analyze information from a crawl before updating their index.

    You should use a more sophisticated stat program. I've got an article coming out on SitePoint on July 16th that is about traffic analysis, until then I'll just say that I recommend AXS

    http://www.xav.com/scripts/axs/

    for SE ranking check out this article:

    http://www.promotionbase.com/article/762/
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  8. #8
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Thanks Chris

    Appreciate that...

    I'll go check out the article...

    p
    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  9. #9
    SitePoint Enthusiast
    Join Date
    Feb 2002
    Posts
    35
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Your raw logs file is not a ms-dos file. It's a text file with a name that ends in .com

    Maybe you're downloading a file named yourdomain.com and windows thinks it's a dos program. Just rename it to something like yourdomain.com.txt and read it with notepad.

  10. #10
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    did it crawl my whole site

    OK...the googlebot hit my site on 6 pages. Did it follow all the links on the 6 pages and spider my whole site?
    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  11. #11
    One website at a time mmj's Avatar
    Join Date
    Feb 2001
    Location
    Melbourne Australia
    Posts
    6,282
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    No. If it did, you would see that in your logs too.

    It appears that google hit six pages on your site. This does not mean it has added you to its directory, although this is probably what will happen soon. There will be some delay. In about a month google may return to spider some more of your site too.
    [mmj] My magic jigsaw
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    The Bit Depth Blog Twitter Contact me
    Neon Javascript Framework Jokes Android stuff

  12. #12
    SitePoint Addict Philip Toews's Avatar
    Join Date
    Dec 2001
    Location
    Kuala Belait, Brunei
    Posts
    367
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    should I just keep submitting pages?

    While waiting for the mysterious and mercurial googlebot to return should I just go ahead and submit 5 other pages from my site a day....?
    Philip Toews Professional esl Educator and ASP.NET wannabe

    http://www.philiptoews.com
    philip@philiptoews.com

  13. #13
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    No, you only need to submit one page, one time. Assuming there is applicable navigation between all of your pages.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •