The Right Tool for the User Research Method
Jeff Sauro • November 8, 2011
There are many great methods for gathering insights from users. And there are definitely many software programs and services to help.
Instead of making a long list of them I've presented the few I use the most when conducting user research.
Lab Based Usability Test
The classic method of usability testing involves bringing users into a lab (usually with a one way mirror) and observing them as they attempt tasks.
I typically use a screen recorder like Camtasia
and a custom web-application called Usability Logger
for recording notes and collecting metrics for stand alone tests or more complex mult-product within subjects tests.
There are also free screen recorders like CamStudio
which have worked for me in the past and the more sophisticated Morae
which includes more recording control and data collection. You can go low budget on this too and use a conference room and pad of paper.
Moderated Remote Usability Test
Inviting participants to attempt tasks in a usability test while they share their screen from their home or work is a growing method in User Research
. I use GoTo Meeting
for the screen sharing and recording the audio and video.
There are several screen recording solutions out there including WebEx
and Adobe Connect
but I've found that GoTo Meeting tends to work on the most users most of the time and the recording is relatively easy. If you don't have the budget then Mikogo
is free, offers a free conference service and it's worked for me when GoTo meeting has not. The main downfall is that you cannot record audio with the screen (yet).
Unmoderated Remote Usability Test
Having users complete tasks without the need for a moderator's time or expensive lab costs continues to grow in popularity. Good software can make the process even easier.
In most studies I use Userzoom
For a low price Loop11
also offers a reliable task-based navigation solution and core reporting. Disclosure: Loop11 and Userzoom are both sponsors of MeasuringUsability.com
Video of Users
When you want a qualified panel of users to test a website and you want to actually see what they're doing then both Usertesting.com
are good solutions. Userlyrtics also offers recordings of the user's facial expressions which can be helpful for branding and awareness studies.
I've often combined both Usertesting.com and Userzoom to have recordings of users interacting with the site, detailed heat maps and a rich metrics to analyze. Disclosure: Usertesting.com and Userlytics are both sponsors of MeasuringUsability.com.
Nothing makes survey data quite as meaningful as comparing it to a standardized benchmark
. For a standardized measure of website usability, credibility, loyalty and appearance I use the SUPR-Q
. For a standardized measure of usability for software, cell-phones, hardware and IVR systems I use the System Usability Scale (SUS)
Predicting Task Times
When shaving seconds or minutes off a task is paramount (such as in software that's used in call centers) getting an estimate of skilled users is one of the best ways to measure improvements
. Keystroke level modeling is a method that's been refined over the last 3 decades and provides a quick and reliable way for estimating task times
and assessing productivity improvements.
I use a method called composite KLM and the software I developed available at www.compositeKLM.com
to generate quick and reliable estimates. Cogtool
is a free tool developed at Carnegie Mellon. It requires more of an investment to learn and is especially suited for mocking up new interfaces.
When you need to know how to categorize your navigation or what to name the labels then card sorting is a great method. You can go low tech and actually use index cards but there's software that makes both delivery and analysis easier.
Web-based solutions like Optimal Workshop's Optimal Sort
and Userzoom's card sorting tool
also allow you to collect data from users located around the world. Another Disclosure: Optimal Workshop is a sponsor of MeasuringUsability.com.
Surveys continue to be a popular and efficient way of collecting feedback quickly. I created my own survey platform UsableSurveys.com
which allows me to collect both simple survey data (multipl-choice questions and open-text) as well as more complex A/B concept testing and top-task ratings.
For complex branching and tracking SurveyGizmo
offer good solutions. The ubiquitous Survey Monkey
is also an option when there's $0 budget.
When I need users recruited and sent to an unmoderated evaluation I rely on panel agencies to recruit and deliver users. I've had good experiences with Toluna
for the general Internet population, car buyers and IT decision makers. For hard to find users (like hardware engineers) Userlytics
offers a competitive service and even Usertesting.com
has come through for me for a retail study.