Usability, Customer Experience & Statistics

Six controversial topics in usability

Jeff Sauro • March 28, 2011

Every field has its set of hot-button issues and usability is no exception. Here are six topics that tend to generate some passionate discussions.

1.    Quantifying usability: Usability is all about the user (i.e. people). Talk of using numbers to describe human computer interaction gets some upset. Usability is typically considered a qualitative activity and not the place for cold-number crunching. Throw some probability and statistics and you've really got some folks going.

2.    Reliability & quality of Usability Evaluators: Ask a group of independent usability professionals to examine an interface and identify the problems.  You'll get a list of problems with only a small overlap.  This has been demonstrated in several comparative usability evaluations.  Should we question the veracity of usability claims? Do such discussions hurt the credibility of the field?

3.    Certification:  Want to call yourself a usability professional? Maybe you should be certified. Will a certification board and process lift the quality of work or just add a layer of bureaucracy that doesn't mean more than the piece of paper it's printed on? 

4.    Unmoderated testing versus lab-based moderated testing:  There are dozens of tools  to collect data from users quickly. Are such tools a threat to practitioners? Can we trust the data?

5.    Effectiveness of Heuristic Evaluations: Does the method of having a usability expert review an interface against a set of heuristics do more harm than good?  Are heuristic evaluations 99% bad? Are such discount-usability methods effective or should they be cleared off the shelves?

6.    Sample size:  There are few things which generate more controversy that the number of users you need to test in a usability evaluation. Some just say "a lot," others say 5 and others do their best to avoid talking sample sizes all together.  The real answer depends on a few factors but there are solid mathematical ways for finding the optimal sample size for usability tests. Of course, using such methods to find the optimal sample size may get some upset (see controversy 1).

About Jeff Sauro

Jeff Sauro is the founding principal of MeasuringU, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 5 books on statistics and the user-experience.
More about Jeff...

Learn More

You Might Also Be Interested In:


Posted Comments

There are 1 Comments

April 13, 2011 | Chris Poteet wrote:

You know what's really a shame? That "interesting" rating system you have for these posts. 

Post a Comment


Your Name:

Your Email Address:


To prevent comment spam, please answer the following :
What is 3 + 3: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[6390 Subscribers]

Connect With Us

Our Supporters

Loop11 Online Usabilty Testing

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Use Card Sorting to improve your IA


Jeff's Books

Customer Analytics for DummiesCustomer Analytics for Dummies

A guidebook for measuring the customer experience

Buy on Amazon

Quantifying the User Experience 2nd Ed.: Practical Statistics for User ResearchQuantifying the User Experience 2nd Ed.: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download