UX Certification
Get hands-on practice in all the key areas of UX and prepare for the BCS Foundation Certificate.
Over the last few months, I’ve worked with three clients who have each adopted a different approach to usability evaluation. These approaches are like different lenses used to observe the customer experience. No single approach is adequate on its own, but in combination the three approaches form a powerful strategy.
Organisation A is in the banking sector. As with many companies in the banking sector, the firm has undergone an enormous amount of change in the last 5 years. As part of that change, the organisation has re-oriented itself around customers and they have a large (and growing) user experience team.
Prior to the development of the user experience team, most of the bank’s knowledge about customers came from market research carried out by external agencies. This meant that internal teams tended to base their product decisions on aggregate data and they often felt one-step removed from customers. As a reaction to this, the team’s manager encouraged her London-based team to get face-to-face with customers in usability testing sessions. These sessions take three forms:
This team’s strategy is all about qualitative, in-depth, one-to-one sessions. This approach works well for the organisation because it ensures the team gets first-hand exposure to users rather than having user needs filtered through a third party. It also ensures that the team can carry out quick, focused tests that support the organisation’s agile way of working.
As a web-based start-up, Company B has a very different culture from Company A and this is reflected in their approach to usability testing. The company CEO is driven by the numbers and he has recruited a user experience manager in his image. The manager of the user experience team doesn’t see a lot of point in investing in lab facilities when there are so many useful tools on the web. His team uses remote, unmoderated tools for usability testing. They carry out these kinds of research:
This team’s strategy is mainly about using quantitative usability data to steer the organisation. The team also ensure the organisation stays grounded in real user behaviour by encouraging the team to view videos from the remote, unmoderated, 'think aloud' sessions. This approach works well for this organisation because it is strongly driven by quantitative data.
Company C is a large retailer with a number of bricks and mortar stores. With its history in retail outlets, this organisation favours data from natural, real world use, not the kind of data that comes from the scripted studies favoured by Companies A and B. Company C knows that moving a product to the end of an aisle in a physical store increases its sales, and they are continually on the prowl for similar behaviours customers exhibit with its web site. This firm wants to know how to optimise the details of each element on its web page. This means the design team wants to know what real customers are doing with their web site right now.
This design team favours the kind of quantitative, real world usage data that comes from A/B and multivariate testing. They use Google Analytics to:
This approach suits the company culture because the web site is treated as just another retail outlet (though an extremely profitable one) that submits its sales returns at the end of each week.
One-to-one moderated usability testing solves the problem of getting the design team exposed to customers so they make better design decisions. Remote, unmoderated usability testing solves the problem of having quantitative data to choose between design ideas. Measuring real-world use solves the problem of knowing how customers use your system right now.
But none of these is an ideal strategy on its own.
The best kind of user research is triangulated. Triangulation is like observing your users through different lenses. Sometimes you want to be up close and personal. Other times you want to sit back and observe the crowd.
One difficulty is that different organisations (and different user researchers) tend to favour one or other of these three approaches, so it’s rare to find a company observing its users through all three lenses. The good news is that if your organisation favours just one of these approaches, you’ll be able to seriously improve design outcomes by showing the design team how to look through a different lens. Train all three lenses on your users and you’ll have the beginnings of an unbeatable usability evaluation strategy.
Dr. David Travis (@userfocus) has been carrying out ethnographic field research and running product usability tests since 1989. He has published three books on user experience including Think Like a UX Researcher. If you like his articles, you might enjoy his free online user experience course.
Gain hands-on practice in all the key areas of UX while you prepare for the BCS Foundation Certificate in User Experience. More details
This article is tagged usability testing.
Our most recent videos
Our most recent articles
Let us help you create great customer experiences.
We run regular training courses in usability and UX.
Join our community of UX professionals who get their user experience training from Userfocus. See our curriculum.
copyright © Userfocus 2021.
Get hands-on practice in all the key areas of UX and prepare for the BCS Foundation Certificate.
We can tailor our user research and design courses to address the specific issues facing your development team.
Users don't always know what they want and their opinions can be unreliable — so we help you get behind your users' behaviour.