the will usually only index the top page of a site unless you tell them not to. Which is a good thing. Basically they'll index the first page of a site that you submit, if you want them to index each page, submit each page, and watch your ranking plummit. They'll follow the links on your site, if they lead to other areas of your site they ignore them, if they lead to other sites they'll index those sites and so on and so forth.
I've been using the Alta Vista page count that I found here... http://www.northernwebs.com/set/alta_vista.html
I'm only getting one page back for my site http://www.scepticism-inc.com
Why? As I understood it spiders went down through the index page to the lower pages indexing them all, unless you TOLD them not to (in the header).
Am I wrong?
Do they stop on the index page by default unless told to go deeper.
If they need to go deeper can someone please post the bit I need to stick in the header to get it to work.
All you have to do is submit your main page to Altavista and eventually they will spider your site and possible add the pages to their index. It is possible your site might appear first in the canadian index.
Plebius Press - A progressive perspective, and some scripts too.
"Insert favorite quote here"