Heuristic Evaluation – a Step By Step Guide Article

Share this article

Evaluation and testing is an important part of your Website development process. Usability tests gather data about the usability of your site by a group of users performing specific tasks.
What is Heuristic Evaluation?
Heuristic Evaluation (originally proposed by Nielsen and Molich, 1990) is a discount method for quick, cheap, and easy evaluation of the user interface. The process requires that a small set of testers (or “evaluators”) examine the interface, and judge its compliance with recognised usability principles (the “heuristics”). The goal is the identification of any usability issues so that they can be addressed as part of an iterative design process. Heuristic Evaluation is popular in Web development circles because it requires few resources in terms of money, time or expertise. So any developer can enjoy the benefits of usability testing – not just those with thousands to spend on a professional assessment. Heuristic Evaluation is characterised by:
  • Small test scenarios that use paper mock-ups or screen shots, which can easily be changed from one test situation to the next
  • An informal basis for assessment that doesn’t require psychologists
  • A high success rate – so only a handful of testers are needed
  • A few key guidelines
How can I Use Heuristic Evaluation on my Site?
1. Plan Your Evaluation How will you test your interface? Heuristic Evaluation typically employs one of the three main approaches: 1. Develop a set of tasks and ask your evaluators to carry them out. Identify and test the tasks that are critical to your site’s success – you’ll want all visitors to be able to perform these – and any elements expected to cause difficulty for your site visitors. 2. Provide evaluators with the goals of the system, and allow them to develop their own tasks. An example goal might be “users should be able to find out how much product x costs.” Evaluators can then break this goal down into appropriate tasks, and test each in turn. 3. Ask evaluators to assess your dialogue elements. Ask evaluators to go through the interface a number of times and examine and assess the efficacy of those elements of your Website that contribute to a dialogue with your site visitors. Choosing which method to use will depend on you, the time that you have available, and on your evaluators. For example, if you were evaluating with young children, the most appropriate method would be to develop a set of tasks and ask them to carry them out. Children will find this much more achievable than trying to develop their own tasks, or assessing your Website elements without any obvious aims. 2. Choose your Evaluators The more evaluators you use, the more usability problems you’ll reveal. However, studies on the subject have shown that the benefit/cost ratio decreases at about five evaluators. So who should these evaluators be?
  • Those with experience – if you can find 5 evaluators who are experts in software ergonomics, and in the field in which the software is applied, a well-planned evaluation program will typically find 81%-90% of usability problems with your interface
  • Those without experience – if you don’t have 5 free experts at your fingertips, don’t worry. A student with no knowledge of software ergonomics will find 22% to 29% of usability problems.
Heuristic Evaluation is known to find more than 90% of usability problems if it’s performed by 3 to 5 experienced people… but remember, one evaluator is better than none! 3. Review the Heuristics Once you’ve decided which approach you’ll take, and you’ve selected your evaluators, you’ll need to brief these people on the ten heuristics you want them to assess your site against. Note: Examples of what they should look out for can be found in the Useful Websites section at the end of this article.
1. Visibility of System Status Probably the two most important things that site visitors need to know are:
  • “Where am I?” and
  • “Where can I go next?”.
So it’s essential that your interface keeps users informed about what’s going on. To test this, your evaluators should look for appropriate feedback within a reasonable time following each user interaction. For example, once a user clicks the ‘Submit’ button on your order form, within a few seconds they’ll require feedback that tells them their order has been received. This feedback might appear in the form of a separate page, or popup, which also contains a ‘back to site’ link indicating where the user can go next. 2. Match Between the System and the Real World The system should speak the users’ language, using words, phrases and concepts that are familiar to the user, rather than system-oriented terms. Even though you might use what is considered standard jargon for the topic on which your site focuses, consider including a further simplification or explanation of the words you’ve used. Your evaluators should make sure you’ve followed real-world conventions, and that your information appears in a natural and logical order. A real world concept applied on the Web is the shopping cart. On many sites, you click once to select an item (the equivalent of picking it up off the shelf in a real store), click again to “add to basket” (or place it in your trolley) and then a third time to confirm your intention to buy (or move to the checkout). 3. User Control and Freedom Site visitors often choose system functions by mistake, and will need a clearly marked “emergency exit” to leave the unwanted page without having to go through an extended dialogue. While there’s a definite need for order to exist in your site, a greater degree of user control may be required to cater to the needs of more experienced users. Your evaluators should ensure that your site meets the control requirements of both first-time and experienced users. An example of a control element might be a “home” button that appears on every page. It’s a simple way to let users feel in control of the system – they know they can “go home” (or opt-out) at any stage in the process. 4. Consistency and Standards Users should not have to wonder whether different words, situations, or actions mean the same thing. it’s best to follow the uniform and/or platform conventions to which your users are accustomed. If the user want to return to the main page then label your link “Home” or “Homepage”, rather than some obscure reference. 5. Error Prevention Even better than good error messages is a careful design that prevents a problem from occurring in the first place. The best way to avoid errors is to conduct testing, more testing, and even more testing. However, if errors do occur, try to provide user friendly messages in natural language rather than code. 6. Recognition Rather than Recall Make sure objects, actions, and options are highly visible. Your site visitors shouldn’t have to remember information between different parts of their dialogue with your site. Instructions for use of the system should be visible – or at least easily retrievable – whenever your users need them. This increases the chance that your visitors will be able to recognise where they are, so they won’t have to retrace their path from the home page. For example, if you create a Website with a lot of submenus, then use a system that will let the users know what section they are in at all times. You could do this by leaving a breadcrumb trail, or maybe applying a color scheme that differentiates the various sections. 7. Flexibility and Ease of Use Accelerators, which may be unseen by the novice user, can often speed up the interaction for the expert user, and allow the system to cater to both types of visitors. You might, for instance, allow users to tailor frequent actions. Take Amazon. They save the personal information that’s provided by customers upon purchase. Then, each time the customer makes another purchase, they can retrieve their information with a single click. In this way, Amazon provides customers with a way to avoid filling in an extensive form each time they buy a product at the store. 8. Aesthetic and Minimalist Design Extraneous information on a page is a distraction and a slow-down. Make rarely needed information accessible via a link so that the details are available, but don’t interfere with the more relevant content. If your “Contact Us” page contains a form, as well as all your physical contact details such as address, telephone number etc., there’s no need to also include a map with extensive instructions on how to get to your premises. Instead, this can be provided on a linked but separate page – not everyone who fills out the contact form will wish to see the map every time they visit the page. 9. Help Users Recognise, Diagnose, and Recover from Errors Errors will occur despite all your efforts to prevent them. Your error messages should be expressed in plain language with no codes or jargon. They should detail the problem, and constructively suggest a solution. For example, if a form is completed incorrectly, your error message should alert your visitor to this, identify which fields will need to be refilled, and perhaps highlight those fields when the user returns to complete the form after they dismiss the error message. 10. Help and Documentation Ideally, every online system could be used without documentation, However, it may be necessary to provide help and documentation to cater to the needs of all users, and be on the safe side. Your evaluators should check to make sure that help and other documentation:
  • is easy to search
  • focuses on the user’s task
  • lists the concrete steps users need to carry out to achieve their goals
  • isn’t too large
Often, documentation is fully integrated into a Website. There should be links from the main help sections into specific subsections, and vice versa. Help could even be fully integrated into each page so that users never feel like assistance is too far away. 4. Conducting the Evaluation Conduct the evaluation using either of these methods:
  • Individual Evaluation – each evaluator reviews the interface individually and reports problems to you. Individual evaluation is easily conducted over the Internet. It will pick up more problems than group evaluation, but takes a lot more time to complete.
  • Group Evaluation – evaluators review the interface as a team, while you record the problems. Evaluators do not have to agree on a problem – but every issue they identify should be recorded. Group evaluation requires more planning than does individual evaluation, as all evaluators need to be assembled, however, the evaluation need only be conducted once as all the evaluators can complete their tasks at the same time.
The most basic form of evaluation is to choose a random page from your site and see if your evaluators can:
  • tell where they are
  • tell what they can/should do next
5. Analysing your Results Once your evaluators have:
  • worked their way through the tasks or goals you set,
  • evaluated each of these in light of the ten heuristics, and
  • provided their feedback,
you’ll need to compile all the information. Remove any duplicates and combine similar issues. What’s left will be a set of problems or comments that you can address to improve your site’s usability.
Remember the Golden Rule!
The golden rule of usability is: There’s no such thing as a “user error”! Make sure you clarify every problem your evaluators identify – ask questions so that you understand the specific nature of the difficulties they encountered. And remember:
  • don’t argue with your evaluators,
  • don’t try to “explain away” problems they identified
If an evaluator found an aspect of your site confusing, then it’s more than likely that your Website visitors might have problems with it too.
More Information
Why You Only Need to Test with 5 Users http://www.useit.com/alertbox/20000319.html (Accessed June 2001) Introduction to Usability http://www.usabilityfirst.com/intro/index.txl (Accessed June 2001) Heuristic Evaluation System Checklist http://www.stcsig.org/usability/topics/articles/he-checklist.html (Accessed June 2001)

Frequently Asked Questions on Heuristic Evaluation

What are the key principles of heuristic evaluation?

Heuristic evaluation is a usability inspection method that helps to identify usability problems in the user interface (UI) design. It involves evaluators examining the interface and judging its compliance with recognized usability principles (the “heuristics”). These principles include visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, help users recognize, diagnose, and recover from errors, and help and documentation.

How many evaluators are needed for a heuristic evaluation?

The number of evaluators can vary depending on the complexity of the system and the resources available. However, research suggests that the best results are obtained with 3 to 5 evaluators. This is because different evaluators will find different sets of usability problems.

How is a heuristic evaluation conducted?

A heuristic evaluation is conducted in three stages. First, the evaluators familiarize themselves with the system and the heuristics. Then, they evaluate the system independently, identifying any usability issues that violate the heuristics. Finally, the evaluators meet to discuss their findings and prioritize the issues based on their severity.

What are the benefits of heuristic evaluation?

Heuristic evaluation is a cost-effective and efficient method for identifying usability problems in a user interface. It can be conducted early in the design process, allowing for issues to be identified and addressed before the system is fully developed. It also provides a structured approach to usability testing, which can help to ensure that all important aspects of the user interface are evaluated.

What are the limitations of heuristic evaluation?

While heuristic evaluation is a valuable tool in usability testing, it does have some limitations. It relies on the expertise of the evaluators, and different evaluators may identify different sets of problems. It also does not involve actual users, so it may not identify all potential usability issues. Finally, it can sometimes lead to false positives, where features are identified as problems when they are not.

How do you choose the right heuristics for your evaluation?

The choice of heuristics can depend on the specific context and goals of your evaluation. However, a good starting point is the set of ten heuristics proposed by Jakob Nielsen, which cover a broad range of usability principles. These can be supplemented or modified as needed to fit your specific needs.

How do you prioritize the findings of a heuristic evaluation?

The findings of a heuristic evaluation are typically prioritized based on their severity. This can be determined by considering factors such as the impact of the problem on the user, the frequency of the problem, and the potential for the problem to be easily corrected.

Can heuristic evaluation be used in combination with other usability testing methods?

Yes, heuristic evaluation can be effectively combined with other usability testing methods, such as user testing, cognitive walkthroughs, and expert reviews. This can provide a more comprehensive view of the usability of a system.

How often should a heuristic evaluation be conducted?

The frequency of heuristic evaluations can depend on various factors, such as the complexity of the system, the rate of changes to the system, and the resources available. However, it is generally recommended to conduct a heuristic evaluation at several stages throughout the design process, to catch and address usability issues as early as possible.

What is the role of a heuristic evaluator?

A heuristic evaluator is responsible for examining a user interface and identifying any usability issues that violate recognized usability principles or “heuristics”. They need to have a good understanding of the system being evaluated, the heuristics being used, and the principles of usability. They also need to be able to effectively communicate their findings and recommendations.

Nicky DaninoNicky Danino
View Author

Nicky is a Community administrator for the SitePoint Forums. She's an advocate of accessibility and her research has been presented at international conferences. Nicky loves to travel, especially to Gibraltar, and is friends with anyone who offers her ice-cream or chocolate.

Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week
Loading form