Usability, Customer Experience & Statistics

Blogs & Articles

Requiring responses increases the burden on the respondent, which in turn may lead to increased abandonment. But should required responses be avoided at all costs? The actual effect on survey completion rates is unclear and in some cases required responses may actually increase the response rate. To improve response rates, reducing the length of the survey (number of questions) will likely have a bigger effect on the response rate than the number of required responses. More research is needed to disentangle the effects of survey length and required responses to a variety of survey types. If there is one clear conclusion about required responses is that the advice that you should never have mandatory responses is overstated. Much like the notorious three clicks to content rule, there are so many exceptions that it shouldn

Pros and Cons of Requiring Survey Responses

Jeff Sauro • September 19, 2016

Requiring responses increases the burden on the respondent, which in turn may lead to increased abandonment. But should required responses be avoided at all costs? The actual effect on survey completion rates is unclear and in some cases required responses may actually increase the response rate. To improve response rates, reducing the length of the survey (number of questions) will likely have a bigger effect on the response rate than the number of required responses. More research is needed to disentangle the effects of survey length and required responses to a variety of survey types. If there is one clear conclusion about required responses is that the advice that you should never have mandatory responses is overstated. Much like the notorious three clicks to content rule, there are so many exceptions that it shouldn't be added to the survey playbook. [Read More]

Google Analytics is an essential tool to understand some key aspects of your website traffic. But it can

4 Things UX Research Tells You that Google Analytics Doesn't

Jeff Sauro • September 12, 2016

Google Analytics is an essential tool to understand some key aspects of your website traffic. But it can't tell you everything. Combining GA with UX research methods help you better understand who your users are, their intent for visiting (goals), what they think of the site (attitudes), problems being encountered, and what to fix.[Read More]

Innovation can mean improving the customer experience or adding new features or functions to address customer needs.Innovation is more than just an epiphany moment from a solo genius. It

4 Principles to Help Innovate and Improve the Customer Experience

Jeff Sauro • September 6, 2016

Innovation can mean improving the customer experience or adding new features or functions to address customer needs.Innovation is more than just an epiphany moment from a solo genius. It's usually the result of much effort and more failure from many people.While no magic method can consistently deliver innovation, a few principles can help. Here are four principles (along with some links) to help spur innovation and improve the customer experience.[Read More]

Standardized questionnaires are generally more reliable and valid than homegrown ones. Many also offer the ability to convert raw scores into percentile ranks. Sometimes you may feel a previously validated questionnaire needs a change (e.g. item wording, scale labels). But before you make changes, consider whether you are measuring more than one thing and whether the existing data you get is good enough. If you decide to make changes, test the original and new items with a set of participants and examine the correlation.

Can You Change A Standardized Questionnaire?

Jeff Sauro • August 30, 2016

Standardized questionnaires are generally more reliable and valid than homegrown ones. Many also offer the ability to convert raw scores into percentile ranks. Sometimes you may feel a previously validated questionnaire needs a change (e.g. item wording, scale labels). But before you make changes, consider whether you are measuring more than one thing and whether the existing data you get is good enough. If you decide to make changes, test the original and new items with a set of participants and examine the correlation.[Read More]

Knowing how many participants you should use is your UX research study is not a simple question. What

Getting Started Finding the Right Sample Size

Jeff Sauro • August 23, 2016

Knowing how many participants you should use is your UX research study is not a simple question. What's particularly difficult about learning how to compute the right sample size is that books and articles can get overly technical; it's hard to know whether the advice is relevant to an applied research setting. In general, it can be hard to know where to start. This article includes five steps to help you start computing the right sample size for your next study.[Read More]

Contextual inquiries involve observation and semi-structured interviews to understand how participants use a product in a particular context. They are one of a number of methods used to help identify unmet user needs. They provide insight into: problems people are trying to solve, points of friction and how people go about completing tasks to accomplish goals. This article includes some background and 8 tips for making contextual inquires more successful.

The Essentials of a Contextual Inquiry

Jeff Sauro • August 16, 2016

Contextual inquiries involve observation and semi-structured interviews to understand how participants use a product in a particular context. They are one of a number of methods used to help identify unmet user needs. They provide insight into: problems people are trying to solve, points of friction and how people go about completing tasks to accomplish goals. This article includes some background and 8 tips for making contextual inquires more successful.[Read More]

If people encounter problems in the online checkout process there

Does Improving Website Navigation Pay Off?

Jeff Sauro • August 8, 2016

If people encounter problems in the online checkout process there's a clear link to conversion rates and the bottom line. But if users can't even find what they are looking for, the best checkout process won't matter. The cost of poor findability is high but does improving it really pay off? Here are five points backed by research to emphasize the importance of measuring and improving website findability.[Read More]

It

13 Things to Consider When Offering Survey Incentives

Jeff Sauro • August 1, 2016

It's no fun planning a party no one comes to. The same can be said about surveys that hardly any one takes. In the never-ending quest to collect data from customers through branded or unbranded surveys, we inevitably encounter the question: should we offer some type of incentive for people to take the survey? Here are 13 things to consider when deciding whether an incentive is the way to go.[Read More]

While it

Controlling for Brand Attitudes in UX Studies

Jeff Sauro • July 26, 2016

While it's realistic to have a mix of brand lovers and haters in a UX study, you'll often want to understand how much the experience differs despite these variations in participants' brand attitudes. Brand favorability has a strong and statistically significant impact on UX metrics. The correlation is strong enough that it often masks the differences between interfaces being compared. To control for the effects of brand favorability you can use an ANCOVA (preferred) or create a segment of low and high brand favorability participants to see whether the patterns in metrics hold.[Read More]

The mobile user experience is no longer a niche industry. It

15 Mobile UX Facts and Insights for 2016

Jeff Sauro • July 19, 2016

The mobile user experience is no longer a niche industry. It's hard to talk about the user experience of websites and software and not consider the impact and experience of mobile devices. We've presented a number of facts and insights in 2013 and 2015. Here's an updated and expanded version culled from many sources on the internet and our own mobile usability studies.[Read More]

The System Usability Scale (SUS) is the most used questionnaire to measure perceptions of usability. The SUS

5 Ways to Use the System Usability Scale (SUS)

Jeff Sauro • July 12, 2016

The System Usability Scale (SUS) is the most used questionnaire to measure perceptions of usability. The SUS's wide adoption and good psychometric properties means it's a good instrument to start measuring with. Here are five uses of the System Usability Scale: After a usability test, in a survey, with prototypes, for focused functionality, on mobile apps and more.[Read More]

Statistics can be daunting, especially for UX professionals who aren

5 Steps for Getting Started with Statistics for Research

Jeff Sauro • July 5, 2016

Statistics can be daunting, especially for UX professionals who aren't particularly excited about the idea of using numbers to improve designs. But like any skill that can be learned, it takes some time to understand statistical concepts and put them into practice. Here's five steps for getting started: understanding data types, sampling error, confidence intervals, statistical significance and knowing what test to use.[Read More]

Excel is an invaluable tool for analyzing and displaying data. In Part 1 I covered some essentials Excel skills, such as conditionals, absolute references and the fill handle. In this second part I

Essential Excel Skills For Researchers Part 2

Jeff Sauro • June 28, 2016

Excel is an invaluable tool for analyzing and displaying data. In Part 1 I covered some essentials Excel skills, such as conditionals, absolute references and the fill handle. In this second part I'll cover a few more shortcuts and advanced functionalities that mimic database manipulations (without needing to know SQL).[Read More]

Excel is a powerful program. It

Essential Excel Skills For Researchers Part 1

Jeff Sauro • June 21, 2016

Excel is a powerful program. It's like an onion, peeling back layers to reveal increasingly specialized functions. It can take years to master. But after you get beyond the basic idea of the spreadsheet and simple formulas, you should master the following functions and techniques that make data analysis more efficient and powerful. Part 1 of 2.[Read More]

A lot goes into a benchmarking study, and any of those things can go wrong. It

5 Common Mistakes in UX Benchmark Studies

Jeff Sauro • June 14, 2016

A lot goes into a benchmarking study, and any of those things can go wrong. It's important to get the details right because you'll use the benchmarks to make comparisons over time and the wrong decisions can have an impact that last for years.Here are five of the more common mistakes made when conducting benchmark studies and what you can do to prevent them.[Read More]

While it

How to Assign the Severity of Usability Problems

Jeff Sauro • June 7, 2016

While it's generally straightforward to count how many times you observe a problem in a usability test, assigning severity ratings to problems can be more challenging. There are often too many problems to fix. Severity ratings help development teams prioritize what does and doesn't get fixed. Here are five steps to follow for establishing a system for assigning problem severity after observing problems in a usability test: Agree on a severity scale; Train evaluators; Have at least two evaluators rate independently; Evaluate agreement; Average or reconcile the differences in ratings.[Read More]

While it

Should All Graphs Start at 0?

Jeff Sauro • May 31, 2016

While it's a good idea to have best practices with displaying data in graphs, the "show the zero" is a rule that clearly can be broken. But showing or not showing the zero alone is not sufficient to declare a graph objective or conversely "deceptive." There isn't an objective graph. But if people want to deceive, they will and the unfocused reader may certainly be swayed. But often displaying the data with or without the zero can provide a different perspective for readers to reconsider the context and the implications.[Read More]

Yes, of course you can. But it depends on who you ask! It

Can You Take the Mean of Ordinal Data?

Jeff Sauro • May 24, 2016

Yes, of course you can. But it depends on who you ask! It's a common question and point of contention when measuring human behavior using multi-point rating scales. Whether someone tells you it's permissible to take the average of ordinal data depends on their view of measurement theory—and not all people agree. Using the mean of ordinal data is fine; just be careful not to make interval or ratio statements about your data -- even researchers who take a more relaxed view of averaging ordinal data would disagree with that practice.[Read More]

The successful researcher, regardless of title, should understand how to combine traditional market research and UX research activities for the best results. If you

Combining UX Research with Market Research

Jeff Sauro • May 17, 2016

The successful researcher, regardless of title, should understand how to combine traditional market research and UX research activities for the best results. If you're in either role, you should understand the tools and techniques that help define what customers think and what they do—and that means blending methods and mindsets.[Read More]

When creating a survey, one important variable to consider is whether to brand the survey as coming from your organization or having a third-party research firm host and send an anonymous survey. While a branded survey is likely to increase response rates, it comes with some drawbacks too.

The Pros and Cons of a Branded Survey

Jeff Sauro • May 10, 2016

When creating a survey, one important variable to consider is whether to brand the survey as coming from your organization or having a third-party research firm host and send an anonymous survey. While a branded survey is likely to increase response rates, it comes with some drawbacks too.[Read More]

Graphs of difference scores are a helpful visualization technique to help highlight differences compared to displaying raw scores. Their main advantage can also be a disadvantage as even small differences can look important. In customer research and analysis in general, while the graph maker can help, it

Visualizing Data: Raw vs Difference Scores

Jeff Sauro • May 3, 2016

Graphs of difference scores are a helpful visualization technique to help highlight differences compared to displaying raw scores. Their main advantage can also be a disadvantage as even small differences can look important. In customer research and analysis in general, while the graph maker can help, it's ultimately up to the viewer to understand if differences depicted have an impact that's meaningful or more modest.[Read More]

Whether you

Create a UX Measurement Plan

Jeff Sauro • April 26, 2016

Whether you're introducing how to measure user experience to an organization or trying to advance the maturity of your UX practice, you need a plan for measuring and improving the user experience. With a good idea about who your users are and how to collect data from them, here's a high-level plan to start measuring and then improving the user experience in 9 steps.[Read More]

Benchmarking is part of maintaining a healthy user experience for your website, app or product. Here is a list of common points to discuss and decide when embarking on a benchmarking project.

A Checklist For Planning a UX Benchmark Study

Jeff Sauro • April 19, 2016

Benchmarking is part of maintaining a healthy user experience for your website, app or product. Here is a list of common points to discuss and decide when embarking on a benchmarking project.[Read More]

All too often efforts get stymied in a quest for perfect data, the perfect metric, or the perfect method—what a lot of people call planning paralysis. Don

Better to be Approximately Right than Exactly Wrong

Jeff Sauro • April 12, 2016

All too often efforts get stymied in a quest for perfect data, the perfect metric, or the perfect method—what a lot of people call planning paralysis. Don't let a quest for perfect data prevent you from collecting any data! Look for sound approximations that get you to a "good enough" place that accomplishes the job and answers your research questions.[Read More]

It

5 Ways to Find Out More About Your Customers

Jeff Sauro • April 5, 2016

It's fundamental to creating both a usable customer experience and a better business: you need to know who your customers are. It however can be surprisingly difficult for organizations to connect with their customers to collect information. This article describes five methods (each with their strengths and weaknesses) for collecting key demographic and psychographic information about your customers.[Read More]

Your job title doesn

6 Best Practices for Using Numbers to Inform Design

Jeff Sauro • March 22, 2016

Your job title doesn't have to be "researcher" or "statistician" to use data to drive design decisions.You can apply some best practices even when numbers aren't your best friend. Here are six best practices for using numbers to inform your design efforts that don't require a career change or advanced degree in math. [Read More]

As if the Net Promoter Score didn

Is the Net Promoter Score a Percentage?

Jeff Sauro • March 15, 2016

As if the Net Promoter Score didn't already stir up enough strong opinions about whether it's the "right" metric for organizations, now there's a new controversy: how to display it. Is it an NPS of 25% or 25? Do you add or omit the percentage sign? Here are my thoughts on this % tempest in a teapot.[Read More]

To loosely quote Lord Kelvin, when we can measure something and express it in numbers, we understand and manage it better. Measuring usability allows us to better understand how changes in usability affect customer satisfaction and loyalty. Usability can and should be measured on mobile apps, enterprise accounting software, early stage prototypes, or mature websites. While devices and users will differ, here are ten core concepts to understand when measuring usability that are likely to remain constant.

10 Essentials of Measuring Usability

Jeff Sauro • March 8, 2016

To loosely quote Lord Kelvin, when we can measure something and express it in numbers, we understand and manage it better. Measuring usability allows us to better understand how changes in usability affect customer satisfaction and loyalty. Usability can and should be measured on mobile apps, enterprise accounting software, early stage prototypes, or mature websites. While devices and users will differ, here are ten core concepts to understand when measuring usability that are likely to remain constant.[Read More]

With the proliferation of big data, the number of statistical tests we can perform seems endless. But the number of fluke discoveries we

How to Handle Multiple Comparisons

Jeff Sauro • March 1, 2016

With the proliferation of big data, the number of statistical tests we can perform seems endless. But the number of fluke discoveries we're likely to detect has increased as well. One of the better known methods for managing this false positive rate is the Bonferroni correction, however, it tends to be too conservative and introduces too many false negatives. A better approach that balances both false positives and false negatives is the Benjamini-Hochberg method which is explained with examples in this article.[Read More]

In many cases, the judgment from multiple people collected independently and then aggregated, is better than even the best individual judgment. The idea of aggregating results is a powerful methodological tool that can smooth out unusual forecasts, scientific conclusions, and judgments from experts and novices alike. It

The Benefits of Aggregating Judgment

Jeff Sauro • February 23, 2016

In many cases, the judgment from multiple people collected independently and then aggregated, is better than even the best individual judgment. The idea of aggregating results is a powerful methodological tool that can smooth out unusual forecasts, scientific conclusions, and judgments from experts and novices alike. It's the power behind meta-analysis and using the average of several polls to predict the winner of an election. It can also be applied to user and customer research as well.[Read More]

Showing the latest 30 posts. Show All 374 Blog Posts

Newsletter Sign Up

Receive bi-weekly updates.
[6306 Subscribers]

Connect With Us

Our Supporters

Use Card Sorting to improve your IA

Loop11 Online Usabilty Testing

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Usertesting.com

About Jeff Sauro

Jeff Sauro is the founding principal of MeasuringU, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 5 books on statistics and the user-experience.
More about Jeff...

.

Jeff's Books

Customer Analytics for DummiesCustomer Analytics for Dummies

A guidebook for measuring the customer experience

Buy on Amazon

Quantifying the User Experience 2nd Ed.: Practical Statistics for User ResearchQuantifying the User Experience 2nd Ed.: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.