Usability, Customer Experience & Statistics

Blogs & Articles

Understanding and appreciating the consequences of sampling error and statistical significance is one thing. Conveying this concept to a reader is another challenge. Picking the "right" visualization is a balance between knowing your audience, working with conventions in your field, and not overwhelming your reader. Here are six ways to indicate sampling error and statistical significance to the consumer of your research.

6 Ways to Visualize Statistical Significance

Jeff Sauro • December 6, 2016

Understanding and appreciating the consequences of sampling error and statistical significance is one thing. Conveying this concept to a reader is another challenge. Picking the "right" visualization is a balance between knowing your audience, working with conventions in your field, and not overwhelming your reader. Here are six ways to indicate sampling error and statistical significance to the consumer of your research.[Read More]

One-way mirrors are an enduring symbol of interrogation, psychology experiments, focus groups, and usability tests. Researchers once considered it essential, but in the last few years I

Reflecting on the One-Way Mirror

Jeff Sauro • November 28, 2016

One-way mirrors are an enduring symbol of interrogation, psychology experiments, focus groups, and usability tests. Researchers once considered it essential, but in the last few years I've seen companies moving away from the mirror. Here are some advantages and disadvantages of the one-way mirror.[Read More]

Latent, or hidden, variables differ from observed variables in that they aren

The Difference between Observed and Latent Variables

Jeff Sauro • November 22, 2016

Latent, or hidden, variables differ from observed variables in that they aren't measured directly. Instead we use observed variables and mathematically infer the existence and relationship of latent variables. This is the core method behind many powerful techniques such as: factor analysis (finding underlying constructs), cluster analysis (card sorting), latent class analysis (segmentation analysis) and structural equation modeling (verifying constructs).[Read More]

Facilitation is a valuable skill for collecting data from participants. It

The Facilitation Spectrum: From Babysitter to Therapist

Jeff Sauro • November 15, 2016

Facilitation is a valuable skill for collecting data from participants. It's used extensively with several methods including usability tests, in-depth interviews, and focus groups. Not all studies involving a facilitator are created equal though. A good facilitator needs to adjust his or her facilitation style based on the type of study and particular research goals. You can think of the facilitator styles needed for each type of study on a spectrum, from babysitter to therapist. The amount of talking, probing, assisting, and investigating the reasons behind actions and utterances depends on where your study is on this spectrum. Identifying the role and expectation for the facilitator ensures you're getting the right data to answer your research questions and helps studies run smoothly.[Read More]

A lot of work goes into planning a study, from customer surveys and unmoderated usability studies to market segmentations.
Without enough of the right participants agreeing to participate and completing your study, the generalizability of your findings are limited. Here are five approaches you can use to get the right people to participate in your studies. In many cases you can combine these approaches to achieve higher participation rates.

5 Ways to Increase Study Participation Rates

Jeff Sauro • November 8, 2016

A lot of work goes into planning a study, from customer surveys and unmoderated usability studies to market segmentations. Without enough of the right participants agreeing to participate and completing your study, the generalizability of your findings are limited. Here are five approaches you can use to get the right people to participate in your studies. In many cases you can combine these approaches to achieve higher participation rates.[Read More]

Having participants think aloud as they use an interface is a cornerstone technique of usability testing. It

The Origins and Evolution of Thinking Aloud

Jeff Sauro • November 1, 2016

Having participants think aloud as they use an interface is a cornerstone technique of usability testing. It's been around for much of the history of user research. It's used to help uncover problems in an interface by allowing a researcher to understand the thought process of a user. To better understand both the method and its application, it helps to know where it came from and how it's evolved. And like the field of User Experience in general, the roots of thinking aloud has its genesis in other fields with over a century of history and evolution.[Read More]

There

5 Visualization Techniques for Managing the Customer Experience

Jeff Sauro • October 25, 2016

There's a plethora of both big and small data. It comes in both qualitative and quantitative forms. In all cases, visualizing data is one of the most effective ways to uncover relationships and patterns, which leads to better insights and improvements in user experiences. Here are five techniques to use to better visualize your data.[Read More]

A key driver analysis (KDA) allows you to identify what features or aspects have the biggest impact on an outcome variable such as likelihood to recommend, brand attitudes, and UX quality. Here are 10 things to know about this powerful technique we use to help prioritize findings in surveys.

10 Things to Know about a Key Driver Analysis

Jeff Sauro • October 18, 2016

A key driver analysis (KDA) allows you to identify what features or aspects have the biggest impact on an outcome variable such as likelihood to recommend, brand attitudes, and UX quality. Here are 10 things to know about this powerful technique we use to help prioritize findings in surveys.[Read More]

The effectiveness of surveys starts with the quality of the questions. Question writing is an art and a science. You need to balance your needs and the needs of the organization commissioning the survey with the burden on the respondents. Here

12 Tips For Writing Better Survey Questions

Jeff Sauro • October 11, 2016

The effectiveness of surveys starts with the quality of the questions. Question writing is an art and a science. You need to balance your needs and the needs of the organization commissioning the survey with the burden on the respondents. Here's a summary of 12 useful guidelines we use.[Read More]

An affinity diagram is a visual technique for organizing seemingly disconnected information. It can be low-tech (sticky notes) or high-tech if you want to make it digital (and potentially involve remote members). Affinity diagramming is a inductive technique in that you start from the bottom with fragments of ideas, behaviors, and observations and end up with "top" groups and relationships. Its main benefit is in taking what may seem like disconnected information and finding potentially hidden associations and patterns to reveal latent themes using a low-tech and quick approach.

How and When to Use an Affinity Diagram

Jeff Sauro • October 4, 2016

An affinity diagram is a visual technique for organizing seemingly disconnected information. It can be low-tech (sticky notes) or high-tech if you want to make it digital (and potentially involve remote members). Affinity diagramming is a inductive technique in that you start from the bottom with fragments of ideas, behaviors, and observations and end up with "top" groups and relationships. Its main benefit is in taking what may seem like disconnected information and finding potentially hidden associations and patterns to reveal latent themes using a low-tech and quick approach.[Read More]

Biases can be particularly pernicious because they

9 Biases That Affect Survey Responses

Jeff Sauro • September 27, 2016

Biases can be particularly pernicious because they're harder to spot than more glaring problems in surveys (like non-mutual exclusivity or double-barreled questions). In fact, there are not always clear remedies to the many biases that can affect your results. However, often just being aware of them is enough to help mitigate unwanted effects. Here are 9 common biases I've documented from the literature and our experience conducting surveys to watch out for and in some cases ideas on how to fix them.[Read More]

Requiring responses increases the burden on the respondent, which in turn may lead to increased abandonment. But should required responses be avoided at all costs? The actual effect on survey completion rates is unclear and in some cases required responses may actually increase the response rate. To improve response rates, reducing the length of the survey (number of questions) will likely have a bigger effect on the response rate than the number of required responses. More research is needed to disentangle the effects of survey length and required responses to a variety of survey types. If there is one clear conclusion about required responses is that the advice that you should never have mandatory responses is overstated. Much like the notorious three clicks to content rule, there are so many exceptions that it shouldn

Pros and Cons of Requiring Survey Responses

Jeff Sauro • September 19, 2016

Requiring responses increases the burden on the respondent, which in turn may lead to increased abandonment. But should required responses be avoided at all costs? The actual effect on survey completion rates is unclear and in some cases required responses may actually increase the response rate. To improve response rates, reducing the length of the survey (number of questions) will likely have a bigger effect on the response rate than the number of required responses. More research is needed to disentangle the effects of survey length and required responses to a variety of survey types. If there is one clear conclusion about required responses is that the advice that you should never have mandatory responses is overstated. Much like the notorious three clicks to content rule, there are so many exceptions that it shouldn't be added to the survey playbook. [Read More]

Google Analytics is an essential tool to understand some key aspects of your website traffic. But it can

4 Things UX Research Tells You that Google Analytics Doesn't

Jeff Sauro • September 12, 2016

Google Analytics is an essential tool to understand some key aspects of your website traffic. But it can't tell you everything. Combining GA with UX research methods help you better understand who your users are, their intent for visiting (goals), what they think of the site (attitudes), problems being encountered, and what to fix.[Read More]

Innovation can mean improving the customer experience or adding new features or functions to address customer needs.Innovation is more than just an epiphany moment from a solo genius. It

4 Principles to Help Innovate and Improve the Customer Experience

Jeff Sauro • September 6, 2016

Innovation can mean improving the customer experience or adding new features or functions to address customer needs.Innovation is more than just an epiphany moment from a solo genius. It's usually the result of much effort and more failure from many people.While no magic method can consistently deliver innovation, a few principles can help. Here are four principles (along with some links) to help spur innovation and improve the customer experience.[Read More]

Standardized questionnaires are generally more reliable and valid than homegrown ones. Many also offer the ability to convert raw scores into percentile ranks. Sometimes you may feel a previously validated questionnaire needs a change (e.g. item wording, scale labels). But before you make changes, consider whether you are measuring more than one thing and whether the existing data you get is good enough. If you decide to make changes, test the original and new items with a set of participants and examine the correlation.

Can You Change A Standardized Questionnaire?

Jeff Sauro • August 30, 2016

Standardized questionnaires are generally more reliable and valid than homegrown ones. Many also offer the ability to convert raw scores into percentile ranks. Sometimes you may feel a previously validated questionnaire needs a change (e.g. item wording, scale labels). But before you make changes, consider whether you are measuring more than one thing and whether the existing data you get is good enough. If you decide to make changes, test the original and new items with a set of participants and examine the correlation.[Read More]

Knowing how many participants you should use is your UX research study is not a simple question. What

Getting Started Finding the Right Sample Size

Jeff Sauro • August 23, 2016

Knowing how many participants you should use is your UX research study is not a simple question. What's particularly difficult about learning how to compute the right sample size is that books and articles can get overly technical; it's hard to know whether the advice is relevant to an applied research setting. In general, it can be hard to know where to start. This article includes five steps to help you start computing the right sample size for your next study.[Read More]

Contextual inquiries involve observation and semi-structured interviews to understand how participants use a product in a particular context. They are one of a number of methods used to help identify unmet user needs. They provide insight into: problems people are trying to solve, points of friction and how people go about completing tasks to accomplish goals. This article includes some background and 8 tips for making contextual inquires more successful.

The Essentials of a Contextual Inquiry

Jeff Sauro • August 16, 2016

Contextual inquiries involve observation and semi-structured interviews to understand how participants use a product in a particular context. They are one of a number of methods used to help identify unmet user needs. They provide insight into: problems people are trying to solve, points of friction and how people go about completing tasks to accomplish goals. This article includes some background and 8 tips for making contextual inquires more successful.[Read More]

If people encounter problems in the online checkout process there

Does Improving Website Navigation Pay Off?

Jeff Sauro • August 8, 2016

If people encounter problems in the online checkout process there's a clear link to conversion rates and the bottom line. But if users can't even find what they are looking for, the best checkout process won't matter. The cost of poor findability is high but does improving it really pay off? Here are five points backed by research to emphasize the importance of measuring and improving website findability.[Read More]

It

13 Things to Consider When Offering Survey Incentives

Jeff Sauro • August 1, 2016

It's no fun planning a party no one comes to. The same can be said about surveys that hardly any one takes. In the never-ending quest to collect data from customers through branded or unbranded surveys, we inevitably encounter the question: should we offer some type of incentive for people to take the survey? Here are 13 things to consider when deciding whether an incentive is the way to go.[Read More]

While it

Controlling for Brand Attitudes in UX Studies

Jeff Sauro • July 26, 2016

While it's realistic to have a mix of brand lovers and haters in a UX study, you'll often want to understand how much the experience differs despite these variations in participants' brand attitudes. Brand favorability has a strong and statistically significant impact on UX metrics. The correlation is strong enough that it often masks the differences between interfaces being compared. To control for the effects of brand favorability you can use an ANCOVA (preferred) or create a segment of low and high brand favorability participants to see whether the patterns in metrics hold.[Read More]

The mobile user experience is no longer a niche industry. It

15 Mobile UX Facts and Insights for 2016

Jeff Sauro • July 19, 2016

The mobile user experience is no longer a niche industry. It's hard to talk about the user experience of websites and software and not consider the impact and experience of mobile devices. We've presented a number of facts and insights in 2013 and 2015. Here's an updated and expanded version culled from many sources on the internet and our own mobile usability studies.[Read More]

The System Usability Scale (SUS) is the most used questionnaire to measure perceptions of usability. The SUS

5 Ways to Use the System Usability Scale (SUS)

Jeff Sauro • July 12, 2016

The System Usability Scale (SUS) is the most used questionnaire to measure perceptions of usability. The SUS's wide adoption and good psychometric properties means it's a good instrument to start measuring with. Here are five uses of the System Usability Scale: After a usability test, in a survey, with prototypes, for focused functionality, on mobile apps and more.[Read More]

Statistics can be daunting, especially for UX professionals who aren

5 Steps for Getting Started with Statistics for Research

Jeff Sauro • July 5, 2016

Statistics can be daunting, especially for UX professionals who aren't particularly excited about the idea of using numbers to improve designs. But like any skill that can be learned, it takes some time to understand statistical concepts and put them into practice. Here's five steps for getting started: understanding data types, sampling error, confidence intervals, statistical significance and knowing what test to use.[Read More]

Excel is an invaluable tool for analyzing and displaying data. In Part 1 I covered some essentials Excel skills, such as conditionals, absolute references and the fill handle. In this second part I

Essential Excel Skills For Researchers Part 2

Jeff Sauro • June 28, 2016

Excel is an invaluable tool for analyzing and displaying data. In Part 1 I covered some essentials Excel skills, such as conditionals, absolute references and the fill handle. In this second part I'll cover a few more shortcuts and advanced functionalities that mimic database manipulations (without needing to know SQL).[Read More]

Excel is a powerful program. It

Essential Excel Skills For Researchers Part 1

Jeff Sauro • June 21, 2016

Excel is a powerful program. It's like an onion, peeling back layers to reveal increasingly specialized functions. It can take years to master. But after you get beyond the basic idea of the spreadsheet and simple formulas, you should master the following functions and techniques that make data analysis more efficient and powerful. Part 1 of 2.[Read More]

A lot goes into a benchmarking study, and any of those things can go wrong. It

5 Common Mistakes in UX Benchmark Studies

Jeff Sauro • June 14, 2016

A lot goes into a benchmarking study, and any of those things can go wrong. It's important to get the details right because you'll use the benchmarks to make comparisons over time and the wrong decisions can have an impact that last for years.Here are five of the more common mistakes made when conducting benchmark studies and what you can do to prevent them.[Read More]

While it

How to Assign the Severity of Usability Problems

Jeff Sauro • June 7, 2016

While it's generally straightforward to count how many times you observe a problem in a usability test, assigning severity ratings to problems can be more challenging. There are often too many problems to fix. Severity ratings help development teams prioritize what does and doesn't get fixed. Here are five steps to follow for establishing a system for assigning problem severity after observing problems in a usability test: Agree on a severity scale; Train evaluators; Have at least two evaluators rate independently; Evaluate agreement; Average or reconcile the differences in ratings.[Read More]

While it

Should All Graphs Start at 0?

Jeff Sauro • May 31, 2016

While it's a good idea to have best practices with displaying data in graphs, the "show the zero" is a rule that clearly can be broken. But showing or not showing the zero alone is not sufficient to declare a graph objective or conversely "deceptive." There isn't an objective graph. But if people want to deceive, they will and the unfocused reader may certainly be swayed. But often displaying the data with or without the zero can provide a different perspective for readers to reconsider the context and the implications.[Read More]

Yes, of course you can. But it depends on who you ask! It

Can You Take the Mean of Ordinal Data?

Jeff Sauro • May 24, 2016

Yes, of course you can. But it depends on who you ask! It's a common question and point of contention when measuring human behavior using multi-point rating scales. Whether someone tells you it's permissible to take the average of ordinal data depends on their view of measurement theory—and not all people agree. Using the mean of ordinal data is fine; just be careful not to make interval or ratio statements about your data -- even researchers who take a more relaxed view of averaging ordinal data would disagree with that practice.[Read More]

The successful researcher, regardless of title, should understand how to combine traditional market research and UX research activities for the best results. If you

Combining UX Research with Market Research

Jeff Sauro • May 17, 2016

The successful researcher, regardless of title, should understand how to combine traditional market research and UX research activities for the best results. If you're in either role, you should understand the tools and techniques that help define what customers think and what they do—and that means blending methods and mindsets.[Read More]

Showing the latest 30 posts. Show All 385 Blog Posts

Newsletter Sign Up

Receive bi-weekly updates.
[6510 Subscribers]

Connect With Us

Our Supporters

Loop11 Online Usabilty Testing

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Usertesting.com

Use Card Sorting to improve your IA

About Jeff Sauro

Jeff Sauro is the founding principal of MeasuringU, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 5 books on statistics and the user-experience.
More about Jeff...

.

Jeff's Books

Customer Analytics for DummiesCustomer Analytics for Dummies

A guidebook for measuring the customer experience

Buy on Amazon

Quantifying the User Experience 2nd Ed.: Practical Statistics for User ResearchQuantifying the User Experience 2nd Ed.: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to the 2nd Ed. of Quantifying the User ExperienceExcel & R Companion to the 2nd Ed. of Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.