Tom’s Hardware recently completed one of the most thorough browser tests ever undertaken. The Web Browser Grand Prix pitches IE9, Firefox 6, Chrome 13, Safari 5.1 and Opera 11.5 against one another on Windows 7 and Mac OS X using the same hardware.
A total of 32 tests with between three and five iterations were recorded to evaluate load times, rendering speed, plug-in performance, memory usage, standards conformance and reliability. The results should interest web developers and anyone who uses a browser on a regular basis.
On Your Marks. Get Set. Go.
Startup times were measured with a single tab (Google home page) and eight tabs (Google, Facebook, YouTube, Yahoo!, Twitter, MSN, Amazon, and Wikipedia). For a single tab, Chrome on Windows 7 at 0.8 seconds was marginally faster than Safari on OS X.
Opera was the slowest browser to open a single tab, but was only half a second behind. In addition, Opera was the fastest at opening eight tabs — just 1.46 seconds compared to 7.93 seconds for Safari on Windows 7.
Page Load Pentathlon
The next test evaluated page load times. Chrome was the winner on both OSs with an average of 778ms on Windows and 692ms on OS X. Unsurprisingly, the webkit-powered Safari was only a few milliseconds behind.
IE9 fared well and came in third at just over 100ms slower. On Window 7, this was followed by Opera and Firefox last at 1,129ms. On OS X, Firefox did better with 953ms compared to Opera’s 1,190ms.
Performance Bench Press
A number of browser benchmarks were executed including Futuremark Peacekeeper, webkit’s SunSpider, Microsoft’s Maze Solver, Mozilla’s Kraken and Dromaeo.
Chrome performed well in most but, surprisingly, it was beaten by IE, Firefox and Opera in webkit’s own SunSpider test. If you were hoping for an outright winner, you’ll be disappointed. Browsers can come first one benchmark then last in another. The only shocking result is Firefox’s CSS performance in Maze Solver — it took an average of 73.3 seconds on Windows and 94.0 seconds on OS X compared to 3 seconds for Chrome on both OSs.
The following tests evaluated Flash, Java and Silverlight, HTML5, Hardware Acceleration and WebGL performance.
You’d expect most plugins to run at similar speeds and that’s true for Silverlight. However, Chrome can be found toward the bottom of the Flash league and is beaten by most other browsers — especially Opera. Firefox tops the Java table.
The HTML5 and hardware acceleration benchmarks are dominated by Firefox with IE close behind.
Master of Memory
Memory usage causes some of the biggest disagreements among power users. For a single tab, IE9 used the least resources — just 25.7Mb. On Windows, this was followed by Chrome (30.2Mb), Safari (41.7Mb), Firefox (57.9Mb) and Opera (67.3Mb).
OS X browsers required far more memory. Opera was the most efficient with 79.7Mb, followed by Safari (96.6Mb), Chrome (109.1Mb) and Firefox (128.1Mb).
A heavy-load test was then performed with 40 open tabs. Safari was the most efficient with 644.4Mb on Windows and 731.2Mb on OS X. On Windows, this was followed by Firefox (745.4Mb), Opera (789.2Mb), IE (858.5Mb) and Chrome (991.8Mb). The biggest shock comes from the OS X versions of Firefox and Chrome: they require 1,185.7Mb and 1,802.3Mb respectively. I’d heard Mac users complaining but hadn’t realized it was quite that bad.
The next test closed 39 of the tabs and monitored how well memory was reclaimed after five minutes. Unsurprisingly, Chrome performed best: its single process-per-tab model returns almost all memory.
On Windows, IE9 held on to 196.4Mb, followed by Safari (324.1Mb), Firefox (390.3Mb) and Opera (399.9Mb). Again, OS X users are hit hard — Safari retained 618.4Mb while Firefox released little more than 135Mb.
The conformance benchmark examined W3C standards using ACID3, HTML5Test.com, CSS3 Selectors and ECMAScript test262.
There’s little between the browsers. Chrome, Firefox and Safari achieved similar scores followed by Opera and IE9 in last place. All offer a good level of standards support and cross-browser compatibility issues are becoming rarer.
Who Takes Gold?
The review provides placing tables, analysis tables and announces the winners…
Windows 7 Champions:
- Gold: Chrome 13 — a large set of wins discount the weaknesses
- Silver: Firefox 6 — the highest number of non-winning strong scores
- Bronze: IE9 — although it is falling further behind as competing browsers are updated faster
- Opera 11.5 — it’s very close to IE9, but let down by poor memory management and no hardware acceleration
- Safari 5.1 — the least number of wins and most losses, primarily because the Windows version has not kept pace with OS X.
OS X Champions:
- Gold: Safari 5.1 — great performance all-round
- Silver: Chrome 13 — lacks the speed, reliability and memory management of its Windows sibling
- Bronze: Opera 11.5 — couldn’t quite match the webkit browsers
- Firefox 6 — close to Opera, but shocking memory usage
Perhaps the most startling result is how close the five competing browsers are. With the possible exception of some dubious OS X memory management, all the browsers offer a good level of standards support and performance.
However, everyone’s experience will be different depending on their hardware, the number of applications they typically run, and how many tabs they open. Try them all and pick one that’s best for you.