SEO-Friendly Infinite Scroll

Shaumik Daityari

As a developer, at some point you may have to decide between the old fashioned pagination and the trendy new infinite scroll, using the latest technologies, inspired by internet giants like Facebook and Pinterest. It depends on your requirements, but it also may come down to your (or your client’s) personal choice.

Infinite can be an efficient way for users to browse content, but it has a lot of disadvantages and can be the wrong choice on certain types of websites.

One of the primary problems with infinite scroll is the fact that it is, in its barest form, not SEO friendly. When you implement infinite scroll, new content is loaded via Ajax. Unless you take specific measures, this renders the content invisible to search engines. So how then do we make infinite scroll SEO friendly? Let’s discuss a few ways we can do this, including one discussed by Google on the Webmaster Central blog.

Include a Sitemap

If the content you’re loading via infinite scroll is separated into various distinct pages, or units, that have their own URLs, search crawlers like Googlebot may not be able to find that content unless it’s linked somewhere in an accessible spot on one of your already-indexed pages.

A sitemap is one way you can ensure a search crawler can find every page, including the content loaded via infinite scroll functionality. A sitemap is a list of pages in a website, usually in XML format. If a part of your website is inaccessible to web crawlers, you can provide a sitemap to inform the crawler of the full list of pages. For more info about sitemaps and how to create one, you can refer to this tutorial on Tuts+.

Google Webmaster Tools — Add a Sitemap

If you have an account on Google Webmaster Tools, you can use the service to test and submit your sitemap to ensure that a crawler doesn’t miss your site due to errors in your sitemap or other factors.

Using Pagination + Infinite Scroll

This is the method, as mentioned earlier, that Google discusses in a recent blog post. Interestingly, it involves providing the pagination alternative to infinite scroll within the same page! Let’s look into the details.

Using HTML5’s History API

When you load new content, Google recommends that you use HTML5’s History API to change the URL of the page to accommodate the page number when the user scrolls to initiate the loading of new new content. For instance, on loading page 3, the current URL would change to http://myexample.com/?page=3. Similarly, on scrolling up to view the previous content, the URL would change to http://myexample.com/?page=2.

Including Pagination Along with Infinite Scroll

In addition to changing the URL as content is scrolled through, Google suggests that you provide your list of pages at the bottom, which are links to the same pages with page numbers in the URL (e.g. http://myexample.com/?page=3). Just like in traditional pagination, when someone clicks on a page, the page refreshes, and the contents of page 3 are shown. At this point, the infinite scroll functionality will still work, along with the HTML5 History API changes outlined above. So if the user scrolls up or down, new content is shown and the page number is changed on the URL.

The Benefits

So what does this mean for search crawlers? Well, when a crawler visits your site, the content of the first page is shown with page numbers at the bottom. Since the crawler cannot trigger infinite scroll, no new content is going to be loaded. However, because the pagination section is still included on the page, the crawler is able to access all the content. This also ensures that while searching, when someone searches for a particular item on page 3, they land directly on the content and not somewhere else.

This method has the added bonus of making deep linking to infinite scroll content easy. So if I want to share a particular item on the 4th page with someone, I can simply copy the URL (which would be on page 4 at the time) and when someone goes to that URL, they would be able to see exactly what I want them to see.

John Mueller’s Demo

A live demo of the implementation of this idea of infinite scroll with pagination is provided by Webmaster Trends Analyst John Mueller.

Infinite Scroll with Pagination Demo

And as mentioned, you can read a full description of John’s demo on the Google Webmaster Central Blog.

Conclusion

On one of my recent articles discussing implementing infinite scroll with jQuery, someone commented that as more sites start using this technique, the better it will be for those that don’t. This may very well be true.

Google’s own suggestions, along with what I’ve discussed here should provide more incentive for those who want to use the technique but not suffer from SEO limitations.

If you’ve looked at Google’s suggestions or have had to deal with this on one of your own websites or applications, let us know in the comments.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • http://jeanphilippeboily.com/ Jean-Philippe Boily

    You could also simply hide the pagination by the way, so the user does not see any useless pagination and it just works for Google, too. I think this is cleaner as it removes useless UI. My 2 cents.

  • Daniel Laidler

    I would imagine all paginated pages should contain a canonical tag pointing to the first page also. This will prevent needless indexing of the paginated pages… Hiding pagination but leaving the links in the code could loosely be seen as being manipulative.

    I think offering pagination that is clickable, and doesn’t simply keep disappearing off the page every time the endless scroll is auto-triggered offers a benefit to the user. EG/ The SitePoint homepage has a grid of 6 x 4 articles, but if I wanted an article that was produced a two week ago, this could mean a lot of needless scrolling.

  • http://newsviews.satya-weblog.com/ Satya Prakash

    The only problem I faced in this auto pagination is when I want to see Footer. Many websites have kept Footer and there is no way to read the footer info until you are at the last page. Or may be I do not know any hidden secret to reach at the last page using any shortcut!

    • http://dada.theblogbowl.in/ Shaumik Daityari

      Hi Satya,

      One thing that you can do is fix the size of the container of the items in infinite scroll (with overflow auto). That way, unless you use the scrollbar of the container, you can always go to the footer!

      • http://newsviews.satya-weblog.com/ Satya Prakash

        Thanks for the reply. But I do not think anyone want two scroll bars near to each other.

        I think, only solution is not to have footer in this case.

        • http://dada.theblogbowl.in/ Shaumik Daityari

          Yes, you could avoid having a footer altogether (like Facebook does, with the footer information like copyright on the right side).

          However, another thing that comes to my mind is the John Mueller’s demo. The page size is fixed (therefore no scroll bar) and the items of the lazy scroll are loaded in a container div.

          • http://newsviews.satya-weblog.com/ Satya Prakash

            yes, it is easy and to put footer info on sidebar or have one entry for all footer info in Top nav.

  • http://www.purplepier.com.br Carlos Garcia

    Agreed, sometime infite scroll can not be the better choice.

  • Matt M.

    I can’t stand infinite scrolling. With the exception of sites like Twitter and Facebook, where I have very little interest in revisiting or viewing every “page,” it annoys me to the point of leaving for another site. I fail to see any benefit to the user (other than not having to click “next” or “previous”). I have no way of knowing how long I’ll be scrolling and I can’t bookmark any “page.”

    • Roman

      Not being able to bookmark any “page” is resolved by using the HTML5 history API (for browsers that support it, of course), but I am also annoyed by infinite scrolling because I want to know when I will reach the end of the content. The psychological effect is strong and unpleasant.

      • Dmitri

        How useful is it to know you have 1000 pages to go? Would you not rather see the actual number of items?

        • Roman

          That would be more useful, yes.

    • Dmitri

      I respectfully disagree. The pages numbers and links to them make no sense for any dynamic content. If you want to revisit or bookmark any of the links within the infinite scroll, you do it for the actual link, not the whole page showing the link. It is exactly with pages that you don’t have a clue how far you have to jump. It is much easier to scroll down quickly and then use page search for a string to quickly find what you want. There is no way you can do that with pages.

      • Matt M.

        You’re right. I don’t bookmark a list of posts when I’m interested in a specific post. I do however like to do that when I discover a new site. If they have a lot of content, I generally can’t finish going through the entire list in one sitting. So I bookmark the page I stop at. As far as I know, I can’t do that with infinite scrolling, so I generally don’t come back to a site like that.