“Any UX Designer not using data is not effectively doing their job”
— -Jeff Gothelf, author of LeanUX, Sense & Respond and Forever Employable
Most UX Designers agree that it is critical to use metrics to not only understand the health of a product but also how users are using a product. Without data, product meetings often devolve into an opinion driven debate where the highest-ranking person in the meeting often gets their way. Everyone agrees that this is NOT informed decision making. However, Product Design teams often face significant barriers in order to truly become data-driven. Below is an overview of how to overcome the most common barriers to measuring UX.
The 6 barriers Designers face when trying to access data
The 6 Most Common Barriers Product Design teams face when measuring User Experience:
The data doesn’t exist: The toughest barrier teams can face is, simply, that the data doesn’t exist. Your company may be collecting some data, but it’s not collecting the metrics that YOU need. Either no one set up or installed a way to capture the right metrics or your process puts your measurement needs in the back of a long development queue.
Differing levels of access to the data: How a Product Designer is able to obtain data can vary by organization. For example, sometimes the data is captured, however, it’s buried in a database, and the only way to get it is to have an engineer or business insight team create a custom query. This can then turn into a game of begging for favors and constant follow up. Another issue around access can be limited allocation of a central business insights team. Some teams are allotted specific hours per month to obtain data or reports, which isn’t sufficient to work in an agile organization. Other organizations ration user licenses to analytics tools.
Usability of data: Many Product Designers complain that while the data is available, it is shared in a format that makes it very challenging to put into a meaningful story that tells stakeholders what is happening. The team might be collecting data, but it’s still of no use to the Product Designer.
Basic data collection tools are being utilized: If data is being collected but the metrics captured are too rudimentary, you might not be able to get answers to the questions you have in order to make a decision of what to do next. Just because some data is being collected, doesn’t mean it’s usable or helpful to you as a Designer.
Complex data tools are the source of truth: When Product Designers do get access to the analytics tools, they often find the UX of these tools to be confusing and laborious.Most analytics tools are designed for Data Scientists or Business Analysts who are heavily quantitative. The use case for these personas is much more complex than what a Product Designer needs. That is why it can feel brutally painful when a Product Designer navigates through the screens trying to string together various data queries.
Prevention from accessing data: Lastly, some organizations gate-keep the data. These organizations do not democratize metrics across the organization to foster data-driven decision making. A Product Designer in these organizations is often granted access to an automated report or gets reports emailed to them from a Product Manager. But this approach often results, again, in missing the metrics the Product Designer needs to do their job effectively not to mention potential delays in the delivery of the data (i.e. monthly reports).
So how does a Product Designer overcome these barriers?
Ask the 5 key questions before starting any design project. You can read about these questions here. Set up a process to review these 5 key questions with your team before starting any new project to ensure you set yourself and your team up for success.
Advocate for design specific metrics. It can be hard to go against the grain at your organization, but it’s imperative to find your voice or empower other members on your team to advocate for tools that capture metrics that YOU need. If everyone can access the metrics they need (without bothering another team) and feel empowered to do their work- it’s a win-win.
Find a tool that works within your realistic environment.Implementing a tool that requires a lot of developer support if your team is already pressed for development resources, is not going to work. A tool that is complex and requires a lot of training if your UX team is already strapped for time, is not going to work. The tools that you choose to collect your metrics should be as simple as you’d want your product to be for your customers. Finally, it is important to take a candid assessment of your team’s data analytic skills. Find a tool that meets them at the level for their skillset. Not everyone will be a power user!
———
This article is based on a conversation with Jeff Gothelf.
