5 Myths About Data-driven Design

Daniel Schwarz
Daniel Schwarz
Share
It’s not unusual for designers to omit data-driven design from their workflow, simply because of the unfair myths surrounding analytics and the way that data-driven design is conducted.
Psychology studies and UX studies are mistakenly thought to be sufficient enough as an informant, since the results are tested in a controlled environment and defined by large sets of data. But while the information is certainly useful (I mean, it’s fine to regard the results of user studies as best practices, for example), what’s missing from these experiments is the fact that the test subjects aren’t necessarily our users. Without data collected from our audience, it’s easy to make assumptions or be without the other half of the story. Let’s take a look at 5 myths about data-driven design that might change the way you look at using analytics for UX design.

Myth #1: “Analytics Aren’t My Responsibility”

Oh, hell no 🙅‍♂️. Data is definitely 100% your responsibility, regardless of what your primary role is in a team. If you’re a marketer, you need to know conversion rates to see if your marketing strategy is working. If you’re a developer
, you need to know conversion rates to see if your code is bugging out somewhere. If you’re a designer, you need to know conversion rates to see if the user experience is optimal. What marketers, developers and designers all have in common is the same business goal of helping users convert. Product managers have to care about all of those things, although each team member collectively has a responsibility to improve metrics using the skill and knowledge of their field. TL;DR: analytics are everyone’s responsibility.

Myth #2: Data = Numbers

Data can be the heatmap from a user test, the answers from a customer survey, a scrap of feedback from a teammate. Anything that informs our decision-making is data. However, many of us make our first mistake when we assume that usability testing and surveying are alternatives
to analytics, where in fact analytics lives and breathes at the center of all user research. Analytics helps you ask the right questions for your customer survey, define the right user groups for usability testing, and identify where exactly the UX is falling short so that you can run A/B tests on those areas and find design solutions. TL;DR: data isn’t just numbers, but analytics comes first.

Myth #3: Data-driven Design Requires a Finished Product

It’s very wrong to assume that we can only collect data on finished products. Product design today is leaner than it’s ever been before, with “shipping” becoming something of a spectacle thanks to discovery apps like BetaList and Product Hunt
. Nowadays, it’s not unusual to ship an app, website or feature multiple times as it naturally moves through its alpha, beta and launch stages. With each ship, data and feedback can be collected on a massive scale while even acquiring subscribers, customers and/or a social following along the way. These early adopters can be what make or break your initial success. TL;DR: data can be collected at any time.

Myth #4: Data-driven Design Kills Creativity

Analytics tells you where the best opportunities are leaking out of your site, but it doesn’t outright tell you how fix them. Most of the time there are multiple winning strategies, and that’s where classic creativity comes into play. Creativity is needed to come up with effective solutions to turn a losing game into a winning game, using analytics to drive smart decisions. Here’s an insightful way to think of analytics: Analytics puts you in the position of a football manager, allowing you to see things from a sideline perspective. However, it’s up to you to draw conclusions on how the game is progressing and find ways to turn the game around if needed. So, while data-driven design doesn’t let you run wild, it doesn’t hinder your creativity either. It reduces the number of feasible options and tightens the boundaries, which is actually a good thing, because when there are too many directions to choose from, it can be hard to reach a destination (scientists call this analysis paralysis
in their psychology studies). TL;DR: data eliminates bad choices, not all choices.

Myth #5: Conclusions Are Objective

Data holds the clues, but it requires a detective to analyze those clues and solve the mystery of what your users are looking for. Let’s say, for example, that your website has a high bounce rate. What’s fact is that many users are entering your website and leaving without visiting another web page. What’s not fact is the assumption that users hate your site. It may be that users find exactly what they’re looking for and leave satisfied within seconds (👌👌👌). Perhaps they googled a question, and your website answered it perfectly. It’s up to you to dive deeper and look into referral data, search terms and user demographics to try and identify what the user came to achieve, and then decide whether or not they achieved it. Data is objective, but the conclusions drawn are sub
jective. With analytics, it’s easy to see what you want to see. No data is impervious to bias, and even the most intelligent analytics software can’t explain exactly what your users want. Now that you know this, you can be more consciously aware of what analytics can and can’t do. It’s a tool, not a solution. To learn in-depth about UX Analytics, check out SitePoint’s book Researching UX: Analytics.

Frequently Asked Questions about Data-Driven Design Myths

What is data-driven design and why is it important?

Data-driven design is a process that uses information gathered from both quantitative and qualitative sources to inform how a product should be designed. It is important because it allows designers to make decisions based on actual user data and behavior, rather than assumptions. This leads to a more effective and user-friendly product.

What are some common misconceptions about data-driven design?

Some common misconceptions include the belief that data-driven design stifles creativity, that it’s only about A/B testing, or that it’s too time-consuming. In reality, data-driven design can enhance creativity by providing a clear direction, it involves various types of data analysis, and the time invested can lead to significant improvements in the end product.

How does data-driven design differ from other design approaches?

Unlike other design approaches that may rely heavily on intuition or personal preference, data-driven design is grounded in user behavior and feedback. This makes the design process more objective and focused on the user’s needs and preferences.

Can data-driven design limit creativity?

Contrary to popular belief, data-driven design does not limit creativity. Instead, it provides a framework within which creativity can be applied effectively. By understanding user behavior and preferences, designers can create innovative solutions that also meet user needs.

Is data-driven design only about A/B testing?

No, data-driven design involves much more than just A/B testing. While A/B testing can be a valuable tool, data-driven design also involves analyzing user behavior, conducting user interviews, usability testing, and more.

Is data-driven design too time-consuming?

While data-driven design can require a significant investment of time, the benefits often outweigh the costs. By making design decisions based on data, you can increase the likelihood of creating a successful product, which can save time and resources in the long run.

How can I start implementing data-driven design in my work?

Start by gathering data about your users, such as their behavior, preferences, and feedback. Use this data to inform your design decisions. Remember, data-driven design is not about ignoring your intuition, but about supplementing it with user data.

Can data-driven design be applied to any product?

Yes, data-driven design can be applied to any product, regardless of its nature or complexity. The key is to gather relevant data about your users and use it to inform your design decisions.

What are some challenges in implementing data-driven design?

Some challenges include gathering sufficient and relevant data, interpreting the data correctly, and balancing data-driven decisions with other considerations such as business goals or technological constraints.

What are some resources to learn more about data-driven design?

There are many resources available online, including articles, tutorials, and courses. Some recommended resources include the Nielsen Norman Group, UX Collective, and the Interaction Design Foundation.