We have around 1.3m total pages, Google currently crawls on average 87k a day and our average page load is 1.7 seconds. Out of those 1.3m pages(1.2m being “spun up”) google has only indexed around 368k and our SEO person is telling us that if we speed up the pages they will crawl the pages more and thus will index more of them.
I personally don’t believe this. At 87k pages a day Google has crawled our entire site in 2 weeks so they should have all of our pages in their DB by now and I think they are not index because they are poorly generated pages and it has nothing to do with the speed of the pages. Am I correct? Would speeding up the pages make Google crawl them faster and thus get more pages indexed?
Speeding up the page load might make a small improvement to the crawl rate, but it won’t be massive. (You might think that 87,000 visits a day times 14 days equals 1.2 million pages, but that assumes it’s only visiting each page once. What is more likely is that some pages, like the home page, are being revisited over and over again, so you won’t get full coverage that quickly)
Realistically, how do you think Google is going to react to a site with over a million pages? OK, so Wikipedia has 10 times that, but it’s a pretty exceptional case. A newer site without that kind of authority or history and over a million pages is going to be ringing alarm bells like there’s aliens invading. How on earth have you created over a million pages anyway?
Without having looked at your site, I’d lay heavy money that your content, quality and structure is a much bigger problem than your loading speed.
That’s true in terms of ranking the sites, certainly. Although in this case I doubt they would apply any kind of penalty – a page load of under 2 seconds (if that’s genuine for a clean load off the server) is not bad at all.