Creating Comfortable Usability Tests

A usability test is simply observing a person using your web site or application in a guided and intentional way with the goal of learning how to improve.

The contents of the test itself are important, but your test will suffer mightily if the participants don’t feel comfortable and at ease. Whether you are in a conference room or at a coffee shop with a laptop, there are some things you should consider before jumping into a usability test:

Be Prepared!

Your participants will be able to tell if you’re not prepared and it will set the tone for the rest of the test. This is a new situation for them and knowing that you’re in control makes everything feel a little more normal. These three steps can help make sure you’re prepared:

  1. Create your rough draft. Write an introduction so you start the meeting with confidence and assurance. Identify the questions you want answered about your site. Then, write down specific tasks for your participants to complete that might reveal the answers to these questions.
  2. Pre-test. Run a sample test or two with those tasks exactly like the real thing. A coworker, friend, or family member is great for this. Take note of where the test flow breaks down, where you are most uncomfortable, and if there are any spots where the participant misunderstands you. Make sure all of the technology you’ll be using works as expected.
  3. Tweak. Review your notes about where the holes were in the rough draft. Polish the introduction and practice reading it to yourself a couple more times. Reorganize task order if necessary (especially if doing one task gives clues to how to do another task), tweak wording, and triple check that you are not presenting tasks in a leading way.

Set Expectations

You’re putting someone in a situation with a lot of unknowns, so telling a participant what to expect up front helps calm nerves. Having someone watch you use a web site can feel like a personal invasion (especially if the content is personal).

Be very clear that everything is confidential and that you are not testing the person; you are testing the product. There is nothing they can do wrong. Reaffirm them throughout the session when they do a particularly good job verbalizing thoughts and asking rhetorical questions because this will bring about more of the same.

On the other hand, the more opinionated folks might relish the opportunity to tell you everything that’s wrong with the site. This can quickly devolve into exaggeration and/or exasperation. Use your introduction as a way to help each person understand what type of feedback is helpful and what isn’t. You are watching them and listening to them speak their thoughts, asking specific questions about what the site is. This is different from asking for their opinion and you can nicely explain that the test is built in a way that helps your findings maintain integrity.

Sound Like a Human

Communicating with someone by reading a script off of a page can be off-putting, particularly for your in-person participants. If you find that you are miserable reading a script, then try speaking extemporaneously from a list of bullet points. That way, you’ll cover everything but you won’t risk sounding like a robot.

If you do use bullets, just practice before you start testing so that you’re not introducing new concepts or thoughts by accident. A script is in some ways easier and always more consistent, but can be damaging to your rapport if it doesn’t come off as authentic.

Stay Flexible

Your participants may veer off of your expected path now and then and end up doing a task that you had saved for later. In this case, I like to let them finish it and then switch up the test on the fly. They won’t know the difference and you will avoid causing the frustration of having to do the same thing twice (nobody likes to do that!).

You’ll probably also realize after a couple tests that some of the tasks you’ve created aren’t helpful or aren’t clear. This gives you bad information and can bring the whole test to a screeching halt as your volunteer tries to interpret your request. Improve the task for future tests and don’t schedule all of your tests back-to-back. That way, you’ll have time for reflection and improvement.

Above all, it’s important to let the participants know that you respect their time and appreciate their involvement. Shake their hands, offer them refreshments, give them a comfortable place to sit, ask how they are doing — treat them as you would an important client. It will set the right tone and help them understand how crucial their job is!

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • NTBoss

    Our team is working on a website and part of the project plan is for us to test usability with a group of volunteers. I am glad to have come across your article…we usually test using people the client knows or people who are part of the project.

  • Etienne

    Insightful article but it seems it’s too focused on in-person user testing. What about remote, non-moderated testing? what about online user testing? Do the same rules apply?

    If you’re interested in user tests, check out Justinmind Usernote, an online tool to share and discuss wireframes and prototypes with your users and you can even carry out user tests (both online and in-person) as it integrates most user tesing tools (ClickTale, UserZoom, Usabilla…).

    http://www.justinmind.com