Programming

How to Build the Best Browser Test Suite

    Craig Buckler
    Share

    your browser test suiteTesting a website used to be so much easier. You would develop and test your code in Internet Explorer and be totally oblivious to anything but major HTML, CSS or JavaScript errors. Even the most conscientious developers would only test a couple of other minority browsers.

    Today, we are faced with 5 mainstream browsers and, to complicate matters further, each of those has multiple versions. Usage statistics vary from site to site, but the following browsers should cater for 99% of users:

    • Microsoft IE6 and IE7 (IE8 coming soon)
    • Mozilla Firefox 2 and 3 (Firefox 3.5 coming soon)
    • Apple Safari 3 (Safari 4 coming soon)
    • Google Chrome 1
    • Opera 9

    Need to get closer to 100% user coverage? Throw in IE5, IE5.5, Netscape 6, Mozilla 1.7, mobile Opera mini, a PlayStation 3, a PSP, a few historical versions of Mac Safari, and a couple of screen readers. This is a daunting list: testing your web application could take longer than building it!

    Fortunately, it is easy to install multiple versions of Firefox, Safari and Opera on a single PC: just choose a different installation folder. It is early days for Chrome, but I suspect Google will issue automatic updates so most users will have the latest version.

    As always, the tough one is Internet Explorer. The browser is tightly integrated into Windows so only one version can be installed. Vista users can not downgrade and must use either IE7 or IE8. However, there are a number of ways to test multiple browsers on a single PC.

    Online Browser Simulators

    Several online systems can be used to show how your site works across various combinations of OS and browser:

    There is nothing to install, however, your site must be live, testing is a slow process, and the better services charge a fee. You only see a screenshot so debugging is tricky and you certainly will not discover JavaScript errors or other subtle problems.

    These systems are undoubtedly useful, but it is probably better to use them towards the end of your testing process.

    Install Multiple Versions of Internet Explorer

    Multiple versions of IE can be installed on a single PC. Standalone versions of IE6 (and any version back to v3.0) can be downloaded from the evolt.org browser archive. XP users can simply extract the files and create a shortcut to iexpore.exe. Unfortunately, they do not work in Vista.

    An alternative is the Xenocode Browser Sandbox which provides IE6, IE7, IE8, Firefox 2, Firefox 3, Chrome, Opera and Safari. These are single .exe files that use virtualization technology to ensure the browser runs in its own segregated environment without affecting your core system.

    Finally, there is IETester from DebugBar. This is certainly the easiest application to install and use. It offers tabbed browsing with all versions of IE from 5.5 to 8.0 beta.

    IETester

    Whilst these tools are useful, you should be aware that you are not running ‘real’ installations of IE. The problems include:

    • There are some interface quirks and features such as the browser history, cookies and the cache may not work as you expect.
    • Filters and transitions fail in IE5.5/6.0 so 24-bit PNGs and transparency effects will not appear. There is a workaround for XP standalone browsers, but neither Xenocode or IETester appear to implement a solution.
    • The old browser might use the latest IE7 JavaScript engine.
    • Conditional CSS comments will fail or become confused – which is another good reason to avoid them!

    Use a Virtual Machine

    The most reliable way to run an old version of IE is to use a dedicated PC with a clean installation of Windows. However, it is not necessary to fill your office with clunky old desktops: you can use virtual machine software to emulate any number of PCs on a single machine. Popular VM software includes:

    • VMware: one of the first and best. Confusingly, VMware Server is free whereas VMware Workstation is a more advanced commercial product offering snapshots and cloning.
    • VirtualBox: an excellent open source product owned by Sun Microsystems. It feels as fast as VMware, although lacks a few minor features such as file drag-and-drop between the host and guest OS.
    • Microsoft Virtual PC: a solid free VM, although it is not as fast or fully-featured as its competitors.

    Creating a virtual machine

    After installing the software, you create a new PC then boot-up with an Window installation disk or ISO in the virtual CD drive. Any version of Windows from 98 to XP should be adequate for installing old versions of IE.

    There are several other advantages to using virtual machines:

    • Virtual disks can be copied and backed up. You can trash the OS and revert to a previous image in seconds.
    • You can install older browsers or try out new software without polluting your main PC.
    • You can install other operating systems such as Linux and test a range of other browsers (Lynx is great for assessing accessibility).

    Building your browser suite takes time and none of the solutions are perfect. However, the newly released Microsoft SuperPreview could be the answer to many browser testing problems…

    Have I missed your preferred method of testing multiple browsers?