Why Usability Testing is Vital to Success

Kerry Butters

Usability is something that’s at the forefront of most designers’ minds, whether they’re front or back-end developers. It’s also something that should be considered by content creators, SEO professionals, marketers and anyone that’s involved in a design project.

I’ve talked before about taking a ‘mobile first’ approach and this is to some extent due to the issues that can emerge when designing a site with desktop in mind. If it’s then later discovered that a desktop-targeted site performs terribly on mobile and tablets, then it could mean scrapping the design and starting from scratch. This does of course cost in terms of time and money; the modern consumer demands a great experience across device and that means that the site/app has to perform in a variety of ways.

What is Usability?

An obvious question perhaps. But a necessary one nonetheless, as there is so much talk surrounding the User Experience (UX) and User Interface (UI) at the moment that it all becomes a little difficult to separate.

There is actually no need to separate them. Good design should incorporate all of the above and this will provide a site that, provided content is delivered well too, is highly usable.

According to the Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests by Jeffrey Rubin and Dana Chisnell:

“[W]hat makes something useable is the absence of frustration in using it” and further, “when a product or service is truly usable, the user can do what he or she wants to do the way that he or she expects to be able to do it, without hindrance, hesitation or questions”.

Why Carry Out Usability Testing?

While it’s common sense to have first planned out a strategy and know your target audience, there is never any guarantee that a design will be received to great acclaim in the first instance.

This means that a testing period is necessary to ensure that at launch day, you have a fully functioning product that you already know your target audience will love. Yes, you may have tested it yourself, and on a variety of devices, but then you could be a teeny bit biased, so you need feedback from other, less biased sources and these should be potential end-users.

As Steve Krug puts it, usability is really, when you get down to it, the end-user saying “don’t make me think”.

So it’s vital that testing is carried out in order to ensure that the content and layout of the site appeal to the visitor and is actionable, so that they don’t have to think about the next step, just click on a button that takes them where they want to go.

Planning the Strategy

There are several approaches you can take to usability testing and these include three broad categories:

  1. Explorative
  2. Assessment
  3. Comparative

Number one is used in the early stages of design so that the basic framework can be accessed in order to establish if it fits with the target audience and their expectations. The second category takes place during the design process and again, concentrates on the underlying technology of the site, rather than content.

By this, I mean clickable areas, navigation and overall site architecture. At this point it’s useful to test performance too, although this will need to be tested again once the content is put into place.

Thirdly, this is a means of using more than one version of the basic site and testing it on potential users to see which performs best in their eyes, which is of course the goal at the end of the day.

Whilst I say that these are ‘categories’ of usability, that’s not to say you should choose one and ignore the rest. Usability testing should be ongoing throughout the design process, from the stages of planning, the initial prototype and when the site is launched. Even then, it’s something that should be monitored frequently, especially in the first instance.

Getting Started with Usability Testing

Firstly, decide who you are going to carry out testing on. Stakeholders? Potential users through a Beta-testing operation? Other designers? Usability experts even, such as the above-mentioned Mr. Krug?

In the first instance, let’s think about the actual aesthetics of the site, alongside ease of navigation.

Paper Prototype

This is a type of testing that can be very useful in the planning stages, as it requires no real expenditure, yet can give a good idea of how the site should be designed by garnering user feedback from initial, sketched designs.

According to Jakob Nielsen: “Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.”

This, he believes, is due to the ‘overlap’ in what all testers will find usable and so based upon his mathematical equation, more than 5 users is a fundamental waste of time. Better to test every element on those 5, rather than use a huge pool of testers that will all say much the same thing, with a few variations on how they say it.

That is certainly worth bearing in mind and illustrates that a huge budget is not necessarily the answer. However, do attempt to find testers that don’t have skewed perspectives due to being family members/friends and so on.

Getting back to paper testing, this allows you to show initial site layout and design at scale, without putting in a huge amount of work in the first instance. Once you have presented the sketches and given users the opportunity to tell you their initial thoughts, you can alter design appropriately.


  • Clickable areas
  • Content layout & delivery
  • Navigation

Paper prototypes should be created for a variety of screen sizes and this will also allow you to think about content and layout in the early stages and how these will work going forward.

A/B Testing

This allows you to present two different (but similar) mock-ups to see what the user finds actionable, which language tends to draw them in, colors and branding and how this affects the choices a user makes when looking at the designs.

The idea with A/B Testing is that the tester doesn’t get to see both designs, or the results could prove useless, as they are naturally then biased towards one or the other.

A/B testing can be followed up with a survey (as can paper testing), asking the user what they liked and disliked about the design, with questions that are intended to show, rather than tell, you what works well and what doesn’t with each prototype.

There are tools on the market that can help you with usability testing, such as usabilla and given the increasing interest in usability I’m sure that these are far from the only solution.


The USEFul Framework

This is still a conceptual framework which allows for extensive usability testing using a specially designed framework which will be web-based, although it’s still in prototype stage right now. The concept has been published in the International Journal of Human Computer Interaction and is the brainchild of Alexiei Dingli & Justin Mifsud. The framework is intended to address usability and automatically identify when usability issues are violating standard guidelines.


While it is still in prototype, it’s something to look out for in the future as according to its creators: “The USEFul framework was able to correctly identify 91.48% of the guideline-related usability violations when compared to Nielsen and Tahir’s manual evaluation. It is also significant to state that a code inspection demonstrated that 8.52% discrepancy was mainly due to the poor coding of the tested web sites.”

Further Methods and Ongoing Testing

There are a few schools of thought when it comes to usability testing, many of them totally valid and worth carrying out individually at different stages. As Nielsen points out, if his calculations are to be relied upon, it needn’t involve a huge budget and amount of people.

The key is in knowing who is going to be using the site and this is something that the marketing representative of the client should have already arrived upon. This means that for the designer, it’s necessary in the planning stage to know the buyer persona in order to properly address them in the design, layout and architecture.

Preference Testing

This is another type of testing where you are putting two similar mock-ups in front of the test subject and asking them to make a choice. This is quite strongly based in psychology and the way in which people react emotionally to color and language.
This is where it’s vital to know the target audience, as different languages and colors apply to different demographics.

The Importance of Continuous Testing

Usability tests don’t end with the official launch of a website and designers have known this for a long time. Webmasters have a long tradition of using reporting contact forms for issues that occur on sites such as broken links, for example.

However, these days, it’s not enough to just ask users to do this, it’s also important that new information is gathered as the site begins to take hold after launch.

Indeed, Steve Krug advises that testing should be carried out once a month in order to allow new information to be gathered and any potential issues to be resolved quickly.

Certainly these days it’s not a good idea to rest on your laurels, a good site is always going to be an ongoing project, so long as the client understands the need for maintenance and testing.

What it Means to Web and Business

We’re all concerned with making the web a useful and interesting place that gives the user the best possible experience. However, we also have to bear in mind the commercial perspective of the client and so it’s important to realize that usability testing is going to give good ROI.

A well thought out site that’s simple to use, appealing to the audience and accessible is one that is going to be visited more frequently than its poorer, less well tested neighbor. And that, at the end of the day, will result in more conversions.