This article is a companion piece to my newly launched incrementalux podcast and is based on my learning from interviewing people much smarter than I am.
In Pursuit of the Perfect Product
While user-testing has always had a place in the web development process, perhaps it’s the explosion of smartphones and internet-enabled devices that has really brought user-testing into the spotlight.
In the recent past, several design methodologies have focused on feedback from the end user. Lean UX places measuring feedback as one of its core tenets. User centered design puts the user,.. well, in the center.
Being a UX designer involves a lot of testing and gathering feedback. There are now many products in the marketplace that enable us to analyze and review our users. But, before you wear out that credit card on a raft of user testing software and services, you’ll need (at least) an absolute beginner’s understanding of the what, when, and how of user testing.
That’s what I want to tackle today!
Let’s start with the easiest question to answer.
When to Test
User testing is most effective when utilized before any effort goes into development. It helps to uncover issues on a conceptual or thinking level, before you begin coding or building. It helps to do a lot of testing before you commit to building a solution. Test early, test often is a mantra that has been hammered into all of our heads. But, testing early alone is not sufficient.
How to Test
The 5 second test
Goal: A simple yes/no test to establish whether an idea is viable The quickest and easiest way to gather feedback on designs is the 5 second test. Show your design to your team members, friends or spouses for 5 seconds and ask them questions like:- What do you think this design is about?
- What is the primary action you could take?
- What did you like about it?
- What did you dislike about it?
Guerilla testing
Goal: Gather feedback on a design from random people at common places‘Get out of the building’ is a fairly common phrase heard in the design community.
Guerilla testing is a fancy name for showing up at a coffee shop and randomly selecting people and asking them (politely) for feedback on designs.
This is a great way to get feedback from a variety of sources. Perfect for customer-facing applications that require feedback from a varied demographic.
The downside to guerilla testing is when people give you irrelevant feedback so they can get free coffee, in exchange. This can be mitigated by observing the users and their behavior. Using the notes from observation in combination with their verbal responses helps understand issues with the design better.
Longitudinal Study
Goal: Gather feedback from the same person repeatedly, over a period of time A lot of focus is given to initial impressions and metrics around user tasks like register, sign up, pay. This focus transcends what your role is in the industry.- Product managers focus on user onboarding
- Marketers focus on landing page and conversion rates
- Developers focus on first load time or time to first byte
Binge watching on Netflix
There are several use cases for testing with users who engage with an application for longer periods of time. Journal studies request users to log their experience within applications over a period of time. This way you are not only gathering feedback for “first impressions”. You are also optimizing for repeated uses. A longitudinal study is when you gather feedback from the same person over a period of time for the same application. You’re literally journaling a customer’s lifetime within the application.
Think of how your experience differs between the first time you used a service like netflix and how you use it currently. The initial experience is all about acquainting ourselves with the interface but later on it’s all about binge watching movies. The expectations and requirements of the user changes over usage and time. So does the feedback associated with the user.
Gather feedback on feature changes
A longitudinal study is also a great way to gather feedback on changes to an existing product and how users’ perception of the app changes based on new additions and deletions to the core product.
- How does the new google logo make people feel?
- How do people respond to it as soon as it’s released?
- How About a month later?
- What happens when you remove a feature?
- How do they rate their experience after such a change?
System usability scale
Goal: A standard metric/scale to gather and measure feedback Speaking of ratings, one of the best ways to gather feedback is to use a standard scale. The system usability scale is a great tool to gather subjective feedback.
- How does the user feel about the design?
- Did they find it easy?
- Did they feel confident accomplishing their tasks?
- Would they recommend it to a friend?
What to Test
Usually, you test users on tasks that are core to the website’s functionality. For e-commerce websites, that’d be discovering and purchasing products. Some of the tasks could be like:- Search for a jacket in your favorite color and purchase it
- Find trousers from your favorite brand
- Complete a checkout for 3 items you find interesting
- Time on task – How long did it take the user to complete the task?
- Success on task – How often were users able to successfully complete the tasks?
- Failure rate – How often did they fail?
Last words
User testing is a powerful method to test the effectiveness of your screens. It is usually recommended that you use a combination of qualitative and quantitative techniques to measure user feedback. P.S. If you are interested in learning more about user testing and user research, you should check out this episode on incrementalux podcast.Frequently Asked Questions (FAQs) about User Testing
What is the importance of user testing in product development?
User testing is a critical aspect of product development. It provides insights into how users interact with a product, what they like or dislike about it, and how it can be improved. User testing helps to identify usability issues that may not be apparent to the developers or designers. It also helps to ensure that the product meets the needs and expectations of the users, thereby increasing user satisfaction and product success.
How often should user testing be conducted?
The frequency of user testing depends on the nature of the product and the stage of development. However, it is generally recommended to conduct user testing at various stages of product development, from the initial concept stage to the final product stage. This helps to identify and address usability issues at an early stage, thereby reducing the cost and time of product development.
What are some common methods of user testing?
There are several methods of user testing, including usability testing, A/B testing, surveys, interviews, and focus groups. Usability testing involves observing users as they interact with the product, while A/B testing involves comparing two versions of a product to see which one performs better. Surveys and interviews are used to gather feedback from users, and focus groups involve a group of users discussing their experiences with the product.
How can I recruit participants for user testing?
Participants for user testing can be recruited in several ways. You can use your existing user base, recruit through social media or online forums, or use a user testing service that provides access to a pool of participants. It’s important to select participants who represent your target user group to ensure that the results of the testing are relevant to your product.
How can I analyze the results of user testing?
The results of user testing can be analyzed in several ways. You can use qualitative methods, such as observing user behavior and listening to user feedback, or quantitative methods, such as analyzing metrics like task completion rates and error rates. It’s important to analyze the results in the context of your product goals and user needs to identify actionable insights.
What are some common challenges in user testing and how can they be addressed?
Some common challenges in user testing include recruiting participants, managing the testing process, and analyzing the results. These challenges can be addressed by using a structured approach to user testing, using tools and services that facilitate user testing, and having a clear understanding of your product goals and user needs.
How can user testing improve the user experience?
User testing can improve the user experience by identifying usability issues and user needs that may not be apparent to the developers or designers. By addressing these issues and needs, you can make your product more user-friendly, intuitive, and satisfying to use.
Can user testing be conducted remotely?
Yes, user testing can be conducted remotely. This can be done through video conferencing, screen sharing, or using a remote user testing service. Remote user testing can be a cost-effective and convenient way to conduct user testing, especially when the participants are located in different geographical locations.
What is the role of a moderator in user testing?
A moderator plays a crucial role in user testing. They guide the participants through the testing process, ask questions to gather feedback, and ensure that the testing is conducted in a structured and unbiased manner. A good moderator can help to obtain valuable insights from the user testing.
How can I ensure the validity of user testing results?
The validity of user testing results can be ensured by using a representative sample of users, conducting the testing in a controlled and unbiased manner, and analyzing the results in the context of your product goals and user needs. It’s also important to conduct multiple rounds of testing to confirm the findings.
Vinay RaghuVinay Raghu is a UI/UX developer passionate about experimenting with the latest frontend technologies. He is an avid open source contributor and loves meetup groups. You can learn more about him at viii.in.
