You Have to Try This on Google.

Matt Mickiewicz

Google is now offering a new glimpse at exactly how their spider views your Website.

Try this:
1. Login to your Google account (AdWords, Gmail or Analytics)
2. Enter your Website URL at the bottom of this page.
3. Verify your Website by creating a uniquely-named, but empty HTML file and uploading it to your server. Click on the “verify” link to view instructions on how to do this. More info about verifying your site.
4. Click the “View stats” link

So what exactly can you find out?

  • The most popular queries that your Website shows up for
  • The top queries from which you get clickthroughs
  • Crawl statistics: Pages succesfully crawled, pages blocked by robots.txt, pages that generated HTTP errors or were unreachable
  • The PageRank distribution within your site
  • Various indexing stats (pages indexed, etc.)

This is an awesome troubleshooting tool for all Webmasters, and something that hopefully the other search engines will copy soon.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • pdxi

    There isn’t anywhere at the bottom of this page to enter a URL… ?

  • http://www.SitePoint.com Matt Mickiewicz

    Look for: “If you don’t have a Sitemap…”

  • Dan

    Matt – I think you missed something, and pdxi may be right. It looks like you have to be logged into Google to get to the page you mentioned. Otherwise, you get redirected to a login page.

  • http://www.SitePoint.com Matt Mickiewicz

    You’re right – it seems that even a GMail account will work though. I’ve updated the instructions.

  • Anonymous

    One hang up on their clever verification process. If you use something like the Google tool “Blogger”, you can’t upload a file. :O

  • Anonymously

    Hmmm….

    For “Query stats” I received – “Data is not available at this time. Please check back later for statistics about your site.”

    Guess I’ll check back – because that does not compute… ;)

    Thanks!

  • http://www.lukkesplace.com lukeurtnowski

    Help Matt!
    I’ve signed up and found my web site (http://www.lukesplace.com)
    (when I go to stats and click on the first link (Indexed pages in your site) I get 183 results showing different pages of the site. Does that mean that the google search engine has indexed 183 pages on my site (Is that good?)
    How do I use this utillity to its fullest potential?

  • http://rtasks.com SRTech

    I noticed this earlier today, but it seems that some of the things it lists, like most popular queries, don’t agree with my stats.

  • http://www.maxhyatt.com MystaMax

    Luke those are good questions, and I’d like to know whats the best way to utilize this. I actually got it up and running in about 3 minutes. Pretty straightforward. Thanks.

  • Anonymously

    Round Two…

    Ok – so know Google see something, but umm – so what, I knew that.

    The “Top search queries” and “Top search query clicks” gives no raw data (e.g. number of clicks per XXX, number of queries per XXX, etc.)

    The top queries could have a million queries per second and the second place one could have TWO a YEAR…

    Also, under “” I’m getting “URLs timed out” even thought Google’s “Distribution” bar says 5% of the sites URLs are timing out…

    Guess when they say BETA they’re not kidding…

    Oh well, any data Google give out is well not data.

  • Anonymously

    I need some sleepzzz ;)

  • http://www.designity.nl peach

    I thought this would be cool when I read about it in the forums yesterday, but your post convinced me to sign up.

    The stats are really cool, I can even preview the PageRank of my new website with this new interface, some clever people just have to find out how to interpret these new pagerank visualisations.

  • Anonymous
  • http://boyohazard.net Octal

    Neat. A lot of work for me to do considering the query stats for work’s site!

  • craig34

    You said that this would enable us to see the Pagerank distribution? I signed up and can’t see where that data is. Help?

  • http://www.rage3d.gr/ Gonik

    This is cool! Great webmaster resource. Just tested on my site and works great ;)

    Thanx Matt.

  • WebDevGuy

    this is incredible! thanks for showing this

  • http://www.dotcomwebdev.com chris ward

    how strange… lets me log in, but then won’t let me past the CAPTCHA… says my username/password credentials are incorrect!?!

  • SebastianX

    The top query stats are from Google’s click tracking on the SERPs. The tracking scripts aren’t in place permanently, so it’s just a snapshot. The PageRank distribution is interesting, it seems it’s based on the real PageRank used in rankings, not outdated toolbar stuff. As for the crawler problem reports, I monitor them since 8/30/2005 and they are accurate (more info). Unfortunately they don’t show the source of broken links yet.

  • Pingback: Ringfahndung Blog

  • Pingback: ryanwalker.org » Blog Archive » See what Google knows about your site with Google Sitemap

  • Pingback: blackrimglasses.com » SitePoint Blogs » You Have to Try This on Google.

  • http://www.SitePoint.com Matt Mickiewicz

    lukeurtnowski: It depends on your site… If you know how many pages you have on your Website, you can compare it against the number that Google has indexed, and arrive at a percentage. If Google is missing a significant number of pages, you might consider submitting a full “site map” to Google. Details of how-to do this are at: https://www.google.com/webmasters/sitemaps/docs/en/submit.html

    craig34: Look under stats -> Page analysis for a neat little graph.

    Since the service is still in beta, you might have to revisit the page to see the numbers as I’ve noticed it occasionally goes offline.

  • Pingback: Tech Junction » Blog Archive » Site stats from googly sitemaps

  • John

    See the Google Sitemaps how-to at A Consuming Experience for detailed information re: making this work.

  • Pingback: Basic Thinking Blog » Google Sitemap Statistics

  • Pingback: Scott Randolph.net » Coolest Hack ever. EVER.

  • Pingback: FreshBlog

  • Pingback: o [cc] do [caiocesar] na [www]

  • Anonymously

    Round Three…

    Well I check back and Google had deleted the information the had and replaced it again with “Data is not available at this time. Please check back later for statistics about your site.”

    Also, I tried Google Analytics (e.g. Urchin Hosted for Free)
    http://www.google.com/analytics/

    And ran into the same issue of reports not being generated real-time nor are they archive… Pretty freaken lame even if it is “free.”

    I mean the point of web services is that they are “on-demand” not when ever we get around to it. If their load is so high, then they could atleast say when the next generation will be rendered and cache the old data…

    Oh well.

  • nocomment

    Nice post.. any help with getting listed better on google I will do.

  • Rick

    Regarding Pagerank Distribution… Question is, what does Google consider High, Medium and Low pageranks? In the sitemaps interface, looks like I have a page or two showing High, but the highest PR pages (according to the toolbar & datacenter checks) I have on this relatively new site are PR3. Hope that is not considered High. Would love for these results to be somewhat forward looking and indicate I have a PR5+ showing up near term, but I’m not holding my breath.

  • Pingback: Queeby » Blog Archive » Stuff you don’t know about your site (and Google does)

  • Anonymously

    Regarding Pagerank Distribution… Question is, what does Google consider High, Medium and Low pageranks? In the sitemaps interface, looks like I have a page or two showing High, but the highest PR pages (according to the toolbar & datacenter checks) I have on this relatively new site are PR3. Hope that is not considered High. Would love for these results to be somewhat forward looking and indicate I have a PR5+ showing up near term, but I’m not holding my breath.

    I agree – I mean google as XXX number of you pages indexed & PRzero to PR10, seems like they should just list all of the pages and their PR and have a report that summarize which fall into what PR… in mean they know you are the webmaster of the site, not a competitor.

  • Anonymously

    By the way, WTF with all the SPAM post…

    Also – this is an english site – please stop commenting in German, Hebrew, & etc… (even if I can read it…)

  • Pingback: Mine Road » New Google Sitemaps Toys

  • Vodstock

    For me, the really valuable info is the top search queries compared to the top search query clicks – the two lists should be as similar as possible, I believe.

    For example, for my site, ‘vodcast’ is at the top of both lists, meaning my site is listed high when searching for ‘vodcast’ and when people search for ‘vodcast’, they often click on my site. However, ‘video podcast’ is in my query clicks list but not in my search queries list, effectively meaning my site is relevant to people searching for ‘video podcasts’ but I’m not appearing high enough in Google’s results. In other words, I need to optimise my site for the term ‘video podcast’.

    To summarise, terms that are in the right (clicks) list but not in the left (queries) list are terms you need to optimise your website for.

  • MountainUU

    Very Cool!

    - MountainUU
    -

  • Mike

    very nice

  • Pingback: robert denton - a day » Blog Archive »

  • Pingback: davidbisset.com » View Your Own Site Stats On Goggle

  • www.obrienit.com

    Excellent. Really worked well, thanks for the tip.

  • Pingback: Silver Ronin » See what Google knows about your site with Google Sitemap

  • Pingback: Images of Broken Light

  • SS

    What if i have a blog that i want to check the stats for? How will i upload that Googlexxxx.html file then? Is it possible to use Google SiteMap for blogs on Blogspot? Gurus….any suggestions?

  • Pingback: Khaled’s Blog » Blog Archive » Weekly Link Collection for 19/11/2005

  • Pingback: Blaine Collins Blog » Blog Archive » links for 2005-11-18

  • John

    SS, see A Consuming Experience for lobbying blogger to ask for verification. Right now, there’s no way to make it work. They need to integrate sitemaps with blogspot more effectively….

  • Pingback: This Damn Blog » Blog Archive » What Google thinks of This Damn Blog

  • Pingback: Adjustafresh :: Uncommon Sense :: A blog for smart people :: Blog Archive » links for 2005-11-18

  • Arthur Abon

    Tried it and it actually works. Consistent with the SERP and PR data that I have been checking these past few months.

  • http://www.custom-toolbars.com milette

    One problem with verification can be your handling of ’404′ pages. (Page not found). For all of my sites, my 404 handler is a redirect to the root page of the site, and issuance of a 200 return code. (OK)

    During the verification, Google checks for the blank page, and then also checks for a non-existing page to see if you have this feature enabled.

    No major problem if you have access to the server — just temporarily set it to issue normal 404 failures, get verified, and then switch it back the way it was.

  • Grail

    Allow me to join the clamour of voices shouting the same thing … What about Blogger … Anyone got any ideas? Can use site maps (see this link) but can’t verify so can’t get the spidery goodness of the full strength stats … Ideas?

  • Sy Ali

    I learned of this through the O’Reilly newsletter. It’s too busy at the moment for me to get results right away but I’m sure the Google goodness will be nice.

  • James

    How we analize the traffic through google analytics…How will we get that code which we’ll have to put into our own site?

  • James

    PLz tell me the working process of google analytics..

  • nedy

    i didnt work for me :-(

    also the links are down