There are 100 Examples discussed and worked out in the book: Quantifying the User Experience: Practical Statistics for User Research. In this companion book, each example is revisited, with the answer provided with an explanation, and then there are illustrated steps with numbered screen shots to show you:
 Which calculator and statistical test to use
 Page numbers to refer to the worked example in the main book
 How to enter the data into the calculator
 How to enter the data and interpret the solution in R
 Where to find online calculators which also provide the solution
Who Should BuyUser Researchers, Marketers, Designers and Developers: If you have an interest in providing a quantifiable improvement in an interface, you need this stepbystep guide along with the supporting tools (the excel calculator) or for more advanced researchers download R.Table of ContentsContents 3
Chapter 1: About this Book 10
Chapter 2: How to Get What You Need 12
Getting and using the custom Excel tool 12
Getting and using the custom R functions 12
Chapter 3: How Precise Are Our Estimates? Confidence Intervals 17
Abstract 17
Example 1: Binomial confidence interval around success rate 17
Example 2: Binomial confidence interval around success rate 20
Example 3: Confidence interval for a set of SUS scores 21
Example 4: Confidence interval for a summary of SUS scores 24
Example 5: Compute the standard error for a set of SUS scores 26
Example 6: Confidence interval from set of Likert responses 27
Example 7: Confidence interval from summary of Likert responses 30
Example 8: Mean and median of a set of times 31
Example 9: Mean and median of a set of times 33
Example 10: Computing log times and the geometric mean 35
Example 11: Computing the mean and median of a set of raw times 37
Example 12: Computing logtime mean and standard deviation 39
Example 13: Confidence interval of log times from summary 40
Example 14: Confidence interval of times after log conversion 42
Example 15: Converting raw times to log times 44
Example 16: Computing log time mean and standard deviation 45
Example 17: Confidence interval of log times from summary 47
Example 18: Confidence interval around median task time 48
Example 19: Confidence interval around median task time 51
Exercise 1: Confidence interval for completion rate 54
Exercise 2: Confidence interval for median task time (small sample) 56
Exercise 3: Confidence interval for median task time (large sample) 60
Exercise 4: Confidence interval for SUS scores 62
Exercise 5: Confidence interval for problem occurrence rate 65
Chapter 4: Did We Meet or Exceed Our Goal? 68
Abstract 68
Example 1: Confidence interval for completion rate 68
Example 2: Comparison of success rate with benchmark (small sample) 71
Example 3: Comparison of success rate with benchmark (small sample) 74
Example 4: Comparing success rate benchmark to confidence interval 77
Example 5: Comparing success rate to benchmark (large sample) 80
Example 6: Comparing success rate benchmark to confidence interval 82
Example 7: Comparing success rate to benchmark (large sample) 85
Example 8: Comparing benchmark to confidence interval 87
Example 9: Comparing mean SUS score to benchmark 89
Example 10: Comparing SUS benchmark to confidence interval 92
Example 11: Comparing mean SUS score to benchmark 94
Example 12: Comparing SUS benchmark to confidence interval 96
Example 13: Comparing agreement rate to benchmark 98
Example 14: Comparing agreement benchmark to confidence interval 100
Example 15: Comparison of mean Likert score with benchmark 102
Example 16: Compute the Net Promoter Score® from a set of Likelihood to Recommend ratings 105
Example 17: Comparison of completion times to benchmark 107
Example 18: Comparison of completion times to benchmark 111
Exercise 1: Comparison of completion rate to benchmark (small sample) 115
Exercise 2: Comparison of completion rate to benchmark (large sample) 118
Exercise 3: Comparison of mean SUS to benchmark 121
Exercise 4: Comparison of mean Likert score to benchmark 124
Exercise 5: Comparison of completion times to benchmark 127
Chapter 5: Is There a Statistical Difference between Designs? 131
Abstract 131
Example 1: Paired ttest of SUS questionnaire data (continuous, dependent) 131
Example 2: Paired ttest of completion time data (continuous, dependent) 137
Example 3: Independent groups ttest of SUS data (continuous, independent) 140
Example 4: Independent groups ttest of time data (continuous, independent) 144
Example 5: Chisquared test of 2x2 contingency table (binary, independent) 147
Example 6: Various tests of 2x2 contingency table (binary, independent) 151
Example 7: Comparison of success rates (binary, independent) 154
Example 8: Comparison of conversion rates (binary, independent) 156
Example 9: Comparison of success rates (binary, dependent) 159
Example 10: Comparison of success rates (binary, dependent) 163
Exercise 1: Comparison of SEQ ratings (continuous, independent) 166
Exercise 2: Comparison of conversion rates (continuous, independent) 169
Exercise 3: Comparison of recommendation scores (continuous, independent) 172
Exercise 4: Comparison of promotion rates (binary, independent) 174
Exercise 5: Comparison of success rates (binary, dependent) 178
Exercise 6: Comparison of SUS questionnaire scores (continuous, dependent) 182
Chapter 6: What Sample Sizes Do We Need? Part I – Summative Studies 185
Abstract 185
Example 1: Estimate of continuous data 185
Example 2: Estimate of time (small sample) 188
Example 3: A realistic example given estimate of variability (continuous) 192
Example 4: An unrealistic example (continuous) 193
Example 5: No estimate of variability (continuous) 195
Example 6: Comparison with a benchmark (continuous) 196
Example 7: Withinsubjects comparison of an alternative (continuous) 198
Example 8: Betweensubjects comparison of an alternative (continuous) 200
Example 9: Increasing power beyond 50% (continuous) 202
Example 10: Large sample estimate of success rate with no prior estimate of p (binomial) 205
Example 11: Large sample estimate of success rate with a prior estimate of p (binomial) 207
Example 12: Small sample estimate of success rate (binomial) 209
Example 13: AdjustedWald binomial confidence interval 211
Example 14: Benchmark proportion (binomial) 214
Example 15: Independent proportions (binomial) 216
Example 16: Independent proportions (binomial) 218
Example 17: Independent proportions (binomial) 220
Example 18: Dependent proportions (binomial) 221
Example 19: CI around difference in dependent proportions (binomial) 224
Exercise 1: Estimate of continuous variable 227
Exercise 2: Test against benchmark (continuous) 229
Exercise 3: Withinsubjects test of difference (continuous) 231
Exercise 4: Betweensubjects test of difference (continuous) 233
Exercise 5: Withinsubjects test of difference (continuous) 235
Exercise 6: Estimate of success rate (binomial) 238
Exercise 7: Comparison of independent proportions (binomial) 241
Exercise 8: Comparison of dependent proportions (binomial) 244
Chapter 7: What Sample Sizes Do We Need? Part II – Formative Studies 249
Abstract 249
Example 1: Sample size for formative usability study 249
Example 2: Analysis of 01 problem discovery matrix 251
Example 3: Analysis of 01 problem discovery matrix 253
Example 4: Additional problem discovery analyses 255
Exercise 1: Sample size for formative usability study 257
Exercise 2: Sample size for formative usability study 258
Exercise 3: Probability of discovery of events of given p and n 259
Exercises 46: Analysis of 01 problem discovery matrix 262
Chapter 8: Standardized Usability Questionnaires 265
Abstract 265
Exercise 1: Analysis of matrix of PSSUQ data 266
Exercise 2: More analysis of matrix of PSSUQ data 271
Exercise 3: Analysis of matrix of SUS data 275
Exercise 4: More analysis of matrix of SUS data 278
Chapter 9: Six Enduring Controversies in Measurement and Statistics 281
Abstract 281
Exercise 1: Sample size for withinsubjects comparison of continuous measure 282
Exercise 2: Likelihood of getting x significant results when alpha = .05 284
Chapter 10: Final Words 288
References 289
Appendix: The R Functions 290
About the Authors 335
James R. (Jim) Lewis 335
Jeff Sauro 336
Select Sample Screen ShotsCompanion Cover
 StepbyStep in Excel
 Screen Shots and Easy to Follow Results
 Real World Problems and Soluations with referring page numbers


