When it comes to usability testing, too many of us resemble Captain Louis Renault at the end of Casablanca. He’s the one instructing his staff to “round up the usual suspects.” Sound familiar?
What follows are a few simple ways to round up a more interesting — and truly useful — cast of characters for your next investigation.
Key Takeaways
- Identify the target user groups for your usability tests by analyzing your content and understanding who the project was designed for. Include people who fit both the current and desired profile of typical site users in your tests.
- While the rule of 5 (testing with 5 participants can uncover 85% of usability problems) is popular, it’s recommended to expand the number of participants to around 15 to cover various user groups and account for no-shows.
- Utilize different sampling techniques to get a good mix of participants. These include random sampling, quota sampling, opportunity sampling, and snowball sampling. Each has its own benefits and drawbacks and can be chosen based on the specific needs of the usability test.
- Incentives such as money, food, school credit, gifts or gift certificates, or even a sincere thank you note can motivate people to participate in your usability tests. It’s also important to approach potential participants professionally, avoid using the word “test” to prevent anxiety, and reassure them about your privacy policies.
Finding the Right Participants
The first step is to figure out which user groups you want to hear feedback from. Analyze your content. Be clear about the audiences you’re trying to reach. Step back and really assess your goals:
- Who was the project designed for?
- Is there any special equipment, knowledge, or background necessary to appreciate it?
- How do you want the user to benefit from your site or application?
Assess what you know about your current site users. Use site statistics as well as customer feedback to figure out your typical audience. Is it primarily middle-aged, male, American? Are you content to reach that group, or would you like to shift the profile in some way? For example, would you like to attract younger, older, richer, poorer, more or less highly educated users?
In your testing, you’ll want to include people who fit both the current and desired profile of typical site users.
How Many People Do I Need?
Jakob Nielsen has popularized the rule of 5: the theory that 85% of a site’s problems can be found with as few as 5 participants.
It’s certainly true that you can learn a lot in just a few interviews. But it’s hard to get input that’s representative of all your various user groups with such a small number. I would recommend you expand your number of participants to around 15, if only to account for a few inevitable no-shows, and to cover your various user groups.
Finding the appropriate people can be a major hurdle to performing frequent and effective usability testing. Recruiting agency costs can be substantial: in 2003, Nielsen found that the average agency fee was $107 per participant. If that’s not in your budget, then you will have to do some outreach of your own.
Sampling Techniques
You will need enough users to fill your sample population with the correct mix
of experience, skills, and demographic characteristics. Usability test results are only valid if the participants reflect your current or desired profile of typical site users. Here are a few sampling techniques to ensure that you get a good mix of participants.
Random Sampling
Random sampling requires that you define a targeted population of users, then ensure that every member of that population has an equal chance of being chosen. It is considered the best way to get an unbiased, representative sample, because it ensures that you don’t just approach the people you know and like. The tricky part, of course, is defining your population. Random sampling can also be a time-consuming and expensive approach if you have a very large target group.
But if you’re testing a company’s intranet site, for example, then you have a well-defined population. You could put all employees’ names into a hat and draw out names until you have the desired number of participants. Voila: a random sample.
If you’re to test an existing ecommerce site, why not ask the marketing department for a list of current customers, and select randomly from that list?
Quota Sampling
You may already have used quota sampling even if you aren’t familiar with the term. It basically means that you grab whomever you can get, subject to certain quotas. For example, if you’re interviewing 20 people total, then perhaps no more than 10 should be men (depending on your current and desired user profiles).
Once you’ve reached your quota for any given factor, move on to the next: age, income, race, preferred browser, whatever. Although not truly random, this approach is widely used commercially and very common in usability testing.
Opportunity Sampling
Opportunity sampling is a popular technique because it is relatively easy — you simply position yourself somewhere you are likely to find users who fit your desired criteria, and take your sample from people who are available at the time. Although it has its flaws, opportunity sampling can be a very useful technique when you are trying to target a very specific group in a short period of time. Just be aware that this opportunistic approach is far from a random method, since people with strong opinions are usually more willing to co-operate. The same is true of people who have plenty of time on their hands — like kids, or adults who are unemployed — and therefore may not be the demographic you’re looking for.
But if you are interested in the opinions of economists, for example, it makes sense to position yourself in a place where you’re likely to run into economists – perhaps a university department building or an economics conference.
If you are interested in improving the accessibility of your site and need to find users with visual impairments, consider contacting rehabilitation agencies. They may be able to connect you with some of their clients, or perhaps an assistive technology instructor who would be happy to promote the cause of Website accessibility. Another resource to consider is your local university’s Office of Disability Services.
Snowball Sampling
Snowball sampling is a great way to expand your typical pool of participants. It’s often recommended as a tool to find populations who are harder to reach. If your desired characteristic is rare, it might be extremely difficult or time-consuming to find representative users through traditional methods.
Find one or two key people who fit the profiles you’re looking for and ask them to recommend people similar to them who would be willing to be interviewed. By moving steadily away from the people you know, you can reduce bias and expand your group of potential testers.
Snowball sampling is often used for exploratory, qualitative research, usually conducted through interviews. Therefore, it lends itself well to usability testing.
Enticements
Money
Money is everyone’s favorite motivator, of course. If you have the budget to pay participants for their time, then you’ll find people are much more willing to help you. Money is also a great time-saver. A temp agency can help you find users with the desired skills and background. But if there is no budget allocated for usability testing (often the case), never fear. You have other options.
Food
Homemade snacks go a long way toward convincing people that you appreciate their time. I recommend bar cookies – here’s a recipe for oatmeal chocolate-chip bars that contains so much butter it is practically foolproof.
School Credit
If your targeted user groups include students, see if you can convince a computer instructor to offer extra credit for participating in usability testing.
Gifts or Gift Certificates
Microsoft, for example, offers a variety of software programs to its usability testing participants. If your company or client sells products or services, consider giving your usability participants their choice of gift.
Thank You
A sincere note of thanks for users’ time is always appreciated. They will be especially glad if you know their boss’s name and make sure to cc him or her.
How to Approach Participants
- Introduce yourself. First, let them know who you are. Once they know your name, and that you represent a reputable company or client, they will be more inclined to help.
- Avoid using the word “test.” People get anxious about tests. It can make participants feel like they are being tested, not the site. Make it clear through your careful use of language that it is your site that is being evaluated, not their Web skills. Rather than “participate in usability testing,” use phrases like “try out our site” and “give us your feedback.” Let them know that they are the experts here.
- Reassure them about your privacy policies. Make it clear that no identifying information will be attached to user’s comments. This protects your participants and encourages honesty.
Privacy is particularly important if you are asking coworkers to test an intranet site. Employees need to be assured that their supervisors won’t learn about their criticisms of the site, both for their peace of mind and for the effectiveness of your usability testing. The goal is honest feedback.
Conclusion
The more usability test participants you can round up, the more likely you are to conduct frequent and effective tests of your Web projects. To that end, it’s worthwhile to think carefully about each project’s intended audience, hone your sampling and recruiting techniques, improve your spiel, and offer adequate compensation. With a little planning, you can easily round up more than just the usual suspects.
For more on this topic, see:
- http://www.psybox.com/web_dictionary/Samplingtechniques.htm
- http://www.ahref.com/guides/design/199806/0615jef4.html
- http://www.useit.com/alertbox/20030120.html
Frequently Asked Questions about Usability Test Participants
How can I find the right participants for my usability test?
Finding the right participants for your usability test involves identifying your target audience and recruiting individuals who fit that profile. You can use various methods such as social media, online forums, or recruitment agencies. It’s important to ensure that your participants represent a diverse range of user types to get a comprehensive understanding of your product’s usability.
How many participants do I need for a usability test?
The number of participants you need for a usability test can vary depending on the complexity of your product and the diversity of your user base. However, research suggests that testing with five users can uncover about 85% of usability problems.
How do I incentivize participants for a usability test?
Incentives can be a great way to encourage participation in your usability test. This could be in the form of cash, gift cards, or discounts on your product or service. It’s important to ensure that the incentive is appropriate and appealing to your target audience.
What should I include in a usability test screener?
A usability test screener should include questions that help you identify suitable participants. This could include demographic information, experience with similar products, and availability for testing. The screener should be concise and easy to understand.
How do I ensure the usability test is unbiased?
To ensure your usability test is unbiased, it’s important to recruit a diverse range of participants and to avoid leading questions during the test. It’s also crucial to analyze the results objectively and consider all feedback, even if it contradicts your expectations.
How do I prepare participants for a usability test?
Preparing participants for a usability test involves providing them with clear instructions about the test process, what is expected of them, and how their data will be used. It’s also helpful to create a comfortable testing environment and reassure participants that they are testing the product, not being tested themselves.
How do I analyze the results of a usability test?
Analyzing the results of a usability test involves identifying patterns in user behavior, noting any difficulties or frustrations experienced by participants, and considering how these insights could inform improvements to your product. It’s important to consider both quantitative and qualitative data in your analysis.
Can I conduct a usability test remotely?
Yes, remote usability testing is a viable option, especially for digital products. This involves participants completing the test on their own devices, often while being observed via screen-sharing software. Remote testing can be a cost-effective way to reach a wider range of participants.
What is the role of a moderator in a usability test?
A moderator in a usability test guides the participant through the test process, ensuring they understand the tasks and providing assistance if needed. The moderator also observes and notes the participant’s behavior and feedback.
How can I improve the effectiveness of my usability tests?
Improving the effectiveness of your usability tests involves careful planning, recruiting the right participants, and thorough analysis of the results. It’s also important to iterate on your testing process, incorporating feedback from participants and making adjustments as necessary.
Elizabeth is Web content manager for the American Foundation for the Blind. She helps AFB determine the needs of many audiences -- people who are blind or visually impaired, their friends and family members, seniors, kids, teachers and other professionals -- and ensures that those needs are met through ongoing usability testing.