Responsive Web Design: Real User Testing

Have you ever tried testing your Responsive Website with real users? If not, it is definitely worth considering. You may be surprised by the results.

User Testing vs Technical Testing

Before we begin, some clarification about different types of testing. This article will not focus on technical testing, where the aim is to determine how the site performs on a wide range of devices, or in different circumstances such as with different bandwidths. Instead, we are going to focus on user testing. The aim of this type of testing is to determine whether real users are able to successfully use the product as required. In this case, the product is your website or web application. You can conduct user-testing to determine if users are able to navigate around your site, search for content, fill in a form, purchase a product or perform any other site-related tasks.

Why test your site with real users?

All of us have our own opinions about how products should work. When we test them ourselves, no matter how experienced we are, our views are always biased to some degree. Only actual users can really tell you whether your product meets their needs. For this reason, it is important to test any product as early as possible in the process and as frequently as possible throughout the process.

Task-based Testing

Task-based testing is a very powerful user testing process. Users are asked to perform a series of tasks while interacting with the actual product. Each task is monitored and scored to determine if the user is able to successfully complete the task or not. If users are not able to complete a task, the product may have a potential problem that needs to be addressed. The actual tasks are determined by the required outcomes. If you want to test the purchasing process within an online shopping website, then tasks could include finding a specific product, adding the product to a cart, and completing the purchase process. The best results come from testing a small set of users (around 5 users) and running as many small tests as you can afford.

Controlled Testing or Free Exploration

The testing process can be conducted in a variety of ways. At one end of the spectrum, the entire process is tightly controlled. Users are not allowed to interact with the product until they are given instructions by the tester. This means that you are able to receive detailed feedback from the users at each point in the process before moving on. Users can be asked about their expectations before they click to a new “screen” or a new function. At the other end of the spectrum, users can be allowed to explore the product as they wish. They are prompted to provide feedback as they interact with the product. Each method has strengths and weaknesses. The “controlled testing” method allows you to closely observe each interaction. The “free exploration” method allows you to observe users performing in a more natural manner.

Using Different Media

User testing can be undertaken at any stage in the planning, design or development phases, and can employ different media:

• paper-based wireframes
• onscreen wireframes
• onscreen prototypes
• fully functional website

Before Testing Responsive Websites

Before jumping into user testing with responsive websites, it’s important to know who your current and potential future users may be, and how they are likely to access the website. For example, are they likely to visit the site on desktop and mobile? Mobile only? Mobile and tablet? Each of these options can affect how you choose to conduct the user testing. For example, if users frequently use both dektop and mobile, it may be worth conducting user testing on wide screen and mobile as part of the process. If users frequently use desktop only or mobile only, you may want to run user testing with two separate groups.

Responsive Website user-testing challenges

Responsive websites present challenges that are quite different to those faced by traditional wide screen user testing. For example:

• Users may be disconcerted that the mobile version of the site looks different to the wide screen version
• Users may not be familiar with small screen interactions such as scrolling, swiping etc
• Users may be familiar with specific mobile apps and expect your website to operate in a similar way.
• Users may be familiar with specific platforms (iPhone or Android) and this may affect their expectations.
• Users may not be familiar with linearized content, which is common for responsive websites when displayed on small screens.

Some Lessons Learned

1. Make the user feel “at home”

When conducting responsive website user testing, it is important to provide devices that are familiar to the user. We try to provide a range of different devices, such as iPhone, Android and a small set of tablet devices. In some cases, we also allow users to perform the testing process on their own device, if they prefer. This ensures they aren’t struggling with the device at the same time that they are trying to “learn” your product.

2. Users want to “play” on mobile devices

When conducting user testing on mobile devices, we have found that users immediately want to “play” – they want to grab the device and click around the website or web application. The very first mobile user testing session we ever did was a disaster. We had intended to follow a “tightly controlled” process where the user was only able to look at one screen at a time. Instead, the user picked up the device and immediately began jumping around the site. This can be solved by using the “free exploration” method, or giving the user clear guidelines before commencing.

3. Observing detailed interactions

Small screen devices mean smaller and more intimate interactions. This means that you have to closely observe the user at all times. Videoing them and replaying it later for analysis may help with this. There is actually a range of different methods that can be used to capture the testing process. You can record iPhone and iPad devices using a combination of Air Flow and screencast software such as Camtasia or Screenflow. You may also want to record “over the shoulder” or “head-cam” video of the user as they move through the site. This allows you to see the users’ interaction experience from their point of view on top of the screen, rather than just recording the data from the screen itself. You can also hook up the device to a Data Projector. We have found this useful when we have other people observing the testing process. These observers can view the process live, but from a different room.

Conclusion

It’ s very important to test your responsive websites with users – just as you would any other product. This is the only way you will be able to see how real users interact with your site. It takes some planning and effort, but it is definitely worthwhile.

Need more?

If you are based in Australia, you can come to the Max Design series of workshops touring capital cities during March 2013, where we address key aspects of Building Responsive Websites. 

Frequently Asked Questions about Responsive Web Design and Real User Testing