When we want to compare frameworks, more often than not we do so using a Hello, World benchmark. After all, they're simple, objective, and measurable. But the Hello, World benchmark misses the vast majority of features that a framework provides.

Recently I went looking for comparisons of JavaScript templating solutions, and I found this little gem. I found this interesting because of the way they went about evaluating their choices:

The Test

To pick between the 18 templating options, we assigned each option to a small team of front-end engineers and setup a test: render a (very simplified) LinkedIn profile.

...

The List

We gave each team a list of features to look for in their assigned templating solution. The idea was to fill out a score, from one (poor) to five (excellent), for each item:
  • DRY: how DRY is the templating technology? is there support for code-reuse and partials?
  • i18n: is there support for translations and multiple languages?
  • Hot reload: are changes visible immediately or is there a compile/deploy cycle?
  • Performance: how long does it take to render in the browser and server?
  • Ramp-up time: how is the learning curve?
  • Ramped-up productivity: once you've ramped-up, how fast can you build things?
  • Server/client support: can the same template be rendered both client-side and server-side?
  • Community: is there an active community using this project? Can you google issues?
  • Library agnostic: are there dependencies on other JS libraries, such as jQuery or Mootools?
  • Testable: how hard is it to write unit and integration tests?
  • Debuggable: is it possible to step through the code while it's running to track down errors?
  • Editor support: is there an editor with auto-complete, syntax highlighting, error checking, etc?
  • Maturity: is this a relatively stable project or still experimenting and churning?
  • Documentation: how is the documentation?
  • Code documentation: do the templates encourage/require documentation/comments?
I think this list of features for evaluation is phenomenal. If ever we were to do a thorough comparison of the various frameworks available, I imagine the list for evaluation would look very similar to this.

P.S. If anyone is interested in organizing a thorough framework comparison along these lines, then feel free to drop a quick reply saying so. If enough people are interested, this could be doable.