Reducing HTTP Requests: An idea for a plugin

Cal Henderson from Flickr recently posted Serving Javascript Fast, where he talks about the approach they use at Flickr to reduce the number of HTTP requests per page and efficiently propagating new changes to assets.

In Rails 1.1 we get the propagation of changed assets for free: all URLs to static assets (javascripts, stylsheets, images) get their last modified timestamp appended to their URL. When an asset gets updated, the URL changes and all clients download the latest version.

For example, the following code:

<%= stylesheet_link_tag 'common' %>

will generate something similar to:

<link href="/stylesheets/common.css?1148952578" media="screen" rel="Stylesheet" type="text/css" />

Though not as efficient as Flickr’s approach, it’s simple and works for most people.

The other problem Cal addressed, reducing the number of HTTP requests, is not something we get for free in Rails. In fact, the default setup for a Rails application that uses script.aculo.us has 4 javascript files. These four files don’t often change (only when you upgrade Rails), they are often all needed together and are often updated at the same time. Sound like a good candidate for reducing HTTP requests? Wouldn’t it be great if when we deployed our application to a production server we could concatenate these together and have our views point to the concatenated version, and it all happens automatically. Sounds like a good candidate for a plugin… let’s call it… CombinedAssets!

CombinedAssets would consist of 2 parts:

  1. A rake task which will generate the concatenated files
  2. A view helper to output different asset URLs on the production server

In your environment.rb file you’d declare some sets of assets:


CombinedAssets.javascripts[:rails] = %w(prototype controls dragdrop effects)
CombinedAssets.stylesheets[:yui] = %w(fonts reset grids)

Then in your view you could simply call:


<%= javascript_include_tag combined_javascripts(:rails) %>
<%= stylesheet_link_tag combined_stylesheets(:yui) %>

The output of which would look something like the following in development:


<script src="/javascripts/prototype.js?1147878861" type="text/javascript"></script>
<script src="/javascripts/controls.js?1147878861" type="text/javascript"></script>
<script src="/javascripts/dragdrop.js?1147878861" type="text/javascript"></script>
<script src="/javascripts/effects.js?1147878861" type="text/javascript"></script>
<link href="/stylesheets/fonts.css?1147878863" media="all" rel="Stylesheet" type="text/css" />
<link href="/stylesheets/reset.css?1147878863" media="all" rel="Stylesheet" type="text/css" />
<link href="/stylesheets/grids.css?1147878863" media="all" rel="Stylesheet" type="text/css" />

and the following in production:


<script src="/javascripts/prototype_controls_dragdrop_effects.js?1147878861" type="text/javascript"></script>
<link href="/stylesheets/fonts_reset_grids.css?1147878863" media="all" rel="Stylesheet" type="text/css" />

That’s 5 less HTTP requests for new visitors.

Of course we’d also need a rake task that actually concats the files together, and when combined with Capistrano you could have this happen automatically on the production servers during deployment.

With a language as succinct as Ruby and a framework as flexible as Rails, half the fun is coming up with the most elegant solution. The fact that’s its usually so simple to implement is an added bonus.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • cal

    be careful that you’re sending out cache freshness headers with those assets – browsers wont cache URLs with querystrings unless told otherwise by response headers.

  • http://www.lopsica.com BerislavLopac

    Basically, that’s just reinventing browser cache, but on the server side…

  • http://www.toolmantim.com timlucas

    Cal: Yep, but that’s really only a prob if you’re serving them from something other than Apache, lighttpd etc. A comment on Cal’s article (different Cal?) has more information: The State of Browser Caching.

  • http://www.toolmantim.com timlucas

    BerislavLopac: Not sure exactly what you mean. Forcing new assets to download via a URL change means that you can serve them with expiry dates far into the future, allowing browsers to use their cache without any HTTP chatter.

  • http://aplosmedia.com/ Eric.Coleman

    I was under the impression that IE & firefox would both cache urls with query strings. It’s safari & opera that will not.

    Am I wrong in this assumption?

  • http://www.toolmantim.com timlucas

    Eric.Coleman: I haven’t tested them, but according to Mike D and Cal’s comments there shouldn’t be a problem.

  • HR

    Query string or not, you should always set the corresponding headers if you’re serving a file that can be cached. Especially in this case, where you should set a “Expires” header to a very distant date and/or a “Cache-control” with a huge “max-age” to avoid unnecessary revalidations.

    BTW, I don’t know RoR, but doesn’t this technique interfere with the page cache? if the CSS file changes, shouldn’t all the cached files referring to it be invalidated?

  • http://www.toolmantim.com timlucas

    HR: I really didn’t cover the whole HTTP expiration side of things as I was mainly trying to discuss the idea of mashing assets together at deployment time–but you’re right; you do definitely want to control the expires and max-age to take advantage of this kind of setup. If this plugin was ever to come to fruition I’d hope that Apache and lighttpd code snippets would accompany it.

    In terms of page caching, you’re right in that if the HTML never changes then browsers won’t re-request the asset. You either have to expire page caches or stay dynamic.

  • HR

    I’m sorry, my previous post came out so wrong. After re-reading it I realize it sounds angry and it really wasn’t my intention ;) The article’s good and the idea is sound, I just wanted to point out to other readers that defining revalidation headers is good practice. Sorry again, next time I will post _after_ a good night of sleep, not before.

    BTW, thanks for the link to Mark Nottingham’s blog, this is a must read.

  • scottbecker

    After reading Cal Henderson’s article on Vitamin Serving Javascript Fast I was immediately inspired to create a plugin to easily facilitate this in Ruby on Rails. I whipped up most of it right then.

    Seeing your article and Chris Williams blog post kicked it into high gear, and I finally got around to polishing it for release today.

    Told myself I wasn’t allowed to eat today until I finished. How’s that for inspiration?

    Here you go!

    MergeJS: http://synthesis.sbecker.net/pages/merge_js

    - Scott

  • scott becker

    By the way, the Rails plugin has been updated and has some nice new features (stylesheet support, per asset timestamps, etc).

    The name also changed from MergeJS to AssetPackager.

    http://synthesis.sbecker.net/pages/asset_packager

  • http://thedeparted.blogs.ie Khalil

    Hello,
    Just wondering if anyone could help. I did something stupid and started my blog using a numerical archive system, and now I'd like to change it so that the post title is part of the URL for SEO reasons. Is there any WordPress plugins that anyone knows of that could switch it without sending Googlers to invalid pages? Maybe some sort of redirector to the correct page?

    Thanks.