Get hands-on practice in all the key areas of UX and prepare for the BCS Foundation Certificate.
Heard these before? ‘Market research uses hundreds of people. How come you can get answers with just 5?’ ‘Our product is aimed at everyone, so we can use ourselves as users.’ ‘Users don’t know what they want’ ‘Apple doesn’t do user research so why should we?’ ‘Our agency does all of this for us.’ Here’s how to successfully counter each of these objections.
I was meeting with a potential client a few weeks back who wanted a usability test. “Tell me about your users,” I asked, hoping that I could then use this information as the basis for a recruitment screener.
“Well, it’s aimed at everyone really, so you don’t need to recruit any special kind of user,” came the reply.
Red flag. I was just formulating a response when he said:
“Anyway, that’s good news for you as you’ll need quite a few users!” he said
Putting on my best ‘actually-you’ll-be-surprised-by-this’ face, I said, “Often, we find we can get lots of insight with as few as five people.”
He laughed. “Five!” he said. “Our marketing department use hundreds. How do you seriously expect to get decent results with a handful?”
It turned into a long meeting.
Has this happened to you? With user experience being such a growth field, I find I’m increasingly meeting with people who claim to want some UX magic sprinkled on their user interface but appear not to know many of the basic tenets of UX. Sometimes these people are senior managers in an organisation. Other times they may be project managers.
The problem with these kinds of discussion is that, if you’re not careful, you end up giving your internal or external client what they ask for rather than what they need. And when what they’ve asked for doesn’t give them the results they want, they don’t come back for any future work.
It doesn’t have to be this way. Here’s some of the common objections I’ve heard, along with some tactful ways to help you convince managers — whether they’re inside or outside your organisation — to see the light.
Market research is based on opinions. Opinions vary from person to person. It would be ludicrous for a political pollster to sample 5 people in an attempt to predict an election. And even if we take a single person, his or her opinions will change over time, depending on what’s in the news, the other experiences they have, and how we phrase the question.
To reduce the inherent variability in opinion data, we need to sample a large group of people. For example, if there are 10,000 people who use our product, and we want to know how many of them think Facebook is easy to use, we’ll need to randomly sample 370 to achieve a sampling error within 5%.
In contrast, user experience research is based on behaviour. Behaviour turns out to be remarkably consistent from person to person. For example, if you watch 5 people approach a door, and 4 of them attempt to pull it when the door needs to be pushed, you know there’s something wrong with the design. You don’t need to randomly sample 370 people to draw this conclusion. You observe that the door has a pull handle, and it’s probably that that’s causing the problem. So you replace the pull handle with a push panel, and see if you’ve fixed the problem.
User experience researchers can get away with small samples because they are looking for behavioural insights, not opinions.
This one contains so many flawed assumptions that you’ll need to take a deep breath before answering it.
First up, we have the ‘aimed at everyone’ assumption. Just because everyone can use your product, it doesn’t mean that everyone will. The downside of a product with ‘something for everyone’ is that it has ‘everything for no one’. Even if your product will be used by a wide variety of users, focusing on a small group of users first will result in a product that’s much more likely to be successful.
The best evidence for this comes from Geoffrey Moore’s 'Crossing the Chasm’, a marketing book published over 20 years ago but now enjoying a renaissance as part of the Lean Startup movement. Moore shows that whenever truly innovative high-tech products are first brought to market, they initially have some success in an early market made up of technology enthusiasts and visionaries. But most then fall into a chasm, during which sales falter and often plummet. To cross the chasm, high-tech products must first be adopted by niche customers who find the product a total solution to their specific needs. He calls this niche a ‘beachhead segment’ and this is the group of customers you should aim to satisfy first (for example, by developing personas).
The second assumption — that you can use the person at the next desk as your user — is equally flawed. With the exception of intranets, it’s very rare that internal staff are the target market for the product you’re designing. Real users are almost certainly less tech-savvy, much less knowledgeable about the product domain and a lot less tolerant of the product’s shortcomings than internal users will be.
Here’s an interesting story I came across that shows the value of listening to users. In the early 1950s Leo Fender decided to amplify solid steel-string guitars. He wasn’t a guitar player so he interviewed guitarists and watched them play to help him understand what was important.
One guitarist he met, called Dick Dale, kept pushing Fender for an amplified guitar that could be heard over a crowd of people. But every time Fender built one, the guitarist kept breaking it with his playing. Fender couldn't understand why.
One night Fender went down to the Rendezvous Ballroom in Balboa, California with his friend Freddie Tavares. Fender stood in the middle of four thousand screaming and dancing Dick Dale fans and said to Freddie: “I now know what Dick Dale is trying to tell me. Back to the drawing board”. (You can get an idea of Dick Dale's playing style from this clip on YouTube).
I sometimes wonder if the real inventor of contextual inquiry was Leo Fender.
When managers ask you to design in isolation from users, it's rather like being asked to buy a book for someone else to enjoy reading on holiday. Just because you like a certain author doesn't mean someone else will enjoy reading the book. You'll only be able to get the right book if you know something about the person, either by spending some time with them or by asking questions.
Perhaps you’ve been in a meeting with someone who repeats the famous Henry Ford quotation: “If I had asked people what they wanted, they would have said a faster horse.” This is normally said with a dismissive wave of the hand, indicating that this is conclusive proof that speaking to customers in the early stages of design has no value.
If you want to get fired, you could just respond that there’s no evidence that Henry Ford ever actually said this. On the other hand, if you want to keep your job and do the research, you should agree. “You’re right,” you should say, “users don’t know what they want. So instead of asking them, I plan to transport them to the future and see what we can learn by watching them use our new concept”.
User experience research isn’t about finding out what people like or dislike. And it's not about asking users to design your interface. It’s about seeing the difficulties users face when trying to use the design you’ve invented.
By their very nature, project managers are often focused on solutions. They are often preoccupied with making decisions on what to build, how to build it, what new features it should have etc… An unintended consequence of this is that they sometimes fail to take the long view and consider the problems they are solving. One role of the user experience practitioner is to help project managers take this long view.
This is closely related to the previous objection. Steve Jobs has famously said that Apple “doesn’t do market research” and more recently Sir Jonathan Ive said: “We don’t do focus groups — that is the job of the designer. It’s unfair to ask people who don’t have a sense of the opportunities of tomorrow from the context of today to design.”
There’s nothing new here of course. The problem is that people conflate market research with user research. Apple have found market research methods, like focus groups, to be ineffective ways of finding out what people want from technology or how they’ll use it. (We said the same thing 8 years ago in one of our most popular articles. This is why, unlike most of our competitors, we don’t do focus group research).
But this doesn’t mean Apple doesn’t do user research. In the famous ‘Playboy’ interview in 1985, Jobs said: “We’ve done studies that prove that the mouse is faster than traditional ways of moving through data or applications,” and there’s lots of evidence of usability testing being carried out in the early development of the Mac.
Another of my favourite examples comes from Apple’s User Interface Group. They needed to prototype a portable computer for architects and the first design questions centred on the size and weight of the device. So they stuffed a pizza box with bricks to match the expected weight of the computer and asked an architect to carry it about. They then used techniques from user research to observe how the architect carried the “computer”, noted down the other things he carried, and identified the tasks he carried out. These observations changed the design team’s thinking towards a softer form.
By ‘agency’ here I’m referring to the company that designs and implements your web site or product. An agency will typically provide an all-in-one design service that also includes UX research. It’s understandable that managers expect their design agency to have the UX base covered because that’s one of the things they’re paying for.
Now, there are good and bad agencies and I don’t want to tar them all with the same brush. But in my experience there are some flaws behind this assumption.
These 5 objections aren’t exhaustive. There are many others that never made it into this article including:
There are many misconceptions about user experience, and as a result some project managers still do not fully embrace UX. Why not try preparing your own set of responses to these kinds of objections and start to better educate your clients?
Thanks to Philip Hodgson for the Leo Fender story and for making improvements to this article.
Dr. David Travis (@userfocus) has been carrying out ethnographic field research and running product usability tests since 1989. He has published three books on user experience including Think Like a UX Researcher. If you like his articles, you might enjoy his free online user experience course.
Gain hands-on practice in all the key areas of UX while you prepare for the BCS Foundation Certificate in User Experience. More details
Every month, we share an in-depth article on user experience with over 10,000 newsletter readers. Want in? Sign up now and get free, exclusive access to our reports and eBooks.
Our most recent articles
copyright © Userfocus 2018.
Get hands-on practice in all the key areas of UX and prepare for the BCS Foundation Certificate.
We can tailor our user research and design courses to address the specific issues facing your development team.
Users don't always know what they want and their opinions can be unreliable — so we help you get behind your users' behaviour.