It doesn’t matter if it’s your first usability test or your hundredth; there are always things you can improve to make the most of the time with your users.
- Avoid using why in a direct, reflexive manner: We of course want to know why users do things on websites and in applications.But when we ask why directly, we risk putting the participant on the defensive.
We attempt to soften the probing why questions by using gentler but more verbose questions:
- Instead of “Why did you click on that icon?” ask “Can you tell me what about the icon led you to click on it?”
- Instead of, “Why did you choose number 6?” ask “Can you briefly explain why you chose a 6 when deciding to recommend or not recommend the product to a friend?”
- Minimize the planting of ideas: Usability tests now seem to involve more than just the classic task-based studies. They often involve getting at concepts, feature acceptance and interest. While we want to know if users would use a product, feature or brand, using the product or company name could possibly plant the idea in their head. We might not know if it was something the user would have mentioned without our cue.For example, instead of asking: “Is Samsung one of the top three companies you consider when purchasing a laptop?” ask: “What three companies come to mind when you considering purchasing a laptop?” Then count the number of times the participants mention Samsung without any seeding from the facilitator. Bonus: Use confidence intervals around the percent of times a company is mentioned by each participant to estimate the prevalence in the entire user population.
- Minimize Yes and No Questions: When probing users on their intentions and actions, we ask a lot of questions. Try to phrase questions so they can’t easily be answered with short and often uninformative yes and no responses. Yes and No questions of course sneak in all the time, so don’t worry if you find yourself asking them. (There’s a lot to keep track of in a usability testing session.) When they do sneak in or it’s awkward not to ask a Yes or No question, be ready to follow up with another question (which of course won’t be “Why?).
- Don’t rely too heavily on the “Would you?” questions: Usually, we really want to know if a feature is something that has to be built, fixed or nixed. The natural question is to ask is if users would use the feature (a yes/no question about the future). There’s nothing inherently wrong with asking people if they’d do something; it’s just that people are notoriously bad at predicting their future behavior.What’s more, when participants are being paid for their opinion, there is a natural resistance to overtly acknowledging that something is useless, broken or not helpful to them. Consequently, you’re likely to receive more positively biased responses. Consider phrasing questions in a way that gives the user permission to be critical. Ask them questions like: “If you could only keep one feature?”, or “If you had to cut two things, what would they be?”
- Turn user questions around: Learn the art of gently deflecting questions back on the user. Users will often ask a question about a feature or navigation element. Or, they will ask the ever popular “Did I do that right?” Instead of answering directly with a Yes or No, or ignoring the participant entirely, ask them “Do you think it was the correct selection?” or “What about that makes you unsure you’ve selected the correct option?” By turning the questions gently back toward the user, you can usually glean a bit more about those elusive intentions, mental models and motivations.
You don’t need to be a trained psychotherapist to conduct effective usability sessions, but it always helps to refine the art of understanding human behavior and intentions when looking to improve the customer experience.
