Measuring the User Experience: Collecting, Analyzing, and Presenting UX Metrics
- Length: 384 pages
- Edition: 3
- Language: English
- Publisher: Morgan Kaufmann
- Publication Date: 2022-03-07
- ISBN-10: 0128180803
- ISBN-13: 9780128180808
- Sales Rank: #116551 (See Top 100 Books)
Measuring the User Experience: Collecting, Analyzing, and Presenting UX Metrics, Third Edition provides the quantitative analysis training that students and professionals need. This book presents an update on the first resource that focused on how to quantify user experience. Now in its third edition, the authors have expanded on the area of behavioral and physiological metrics, splitting that chapter into sections that cover eye-tracking and measuring emotion. The book also contains new research and updated examples, several new case studies, and new examples using the most recent version of Excel.
Copyright Contents Dedication Preface References Acknowledgments A Special Note From Cheryl Tullis Sirois Biographies 1 Introduction 1.1 What is User Experience? 1.2 What are User Experience Metrics? 1.3 The Value of UX Metrics 1.4 Metrics for Everyone 1.5 New Technologies in User Experience Metrics 1.6 Ten Myths About UX Metrics Myth 1: Metrics Take Too Much Time to Collect Myth 2: UX Metrics Cost Too Much Money Myth 3: UX Metrics Are Not Useful When Focusing on Small Improvements Myth 4: UX Metrics Don’t Help Us Understand Causes Myth 5: UX Metrics Are Too Noisy Myth 6: You Can Just Trust Your Gut Myth 7: Metrics Don’t Apply to New Products Myth 8: No Metrics Exist for the Type of Issues We Are Dealing With Myth 9: Metrics Are Not Understood or Appreciated by Management Myth 10: It’s Difficult to Collect Reliable Data With a Small Sample Size 2 Background 2.1 Independent and Dependent Variables 2.2 Types of Data 2.2.1 Nominal Data 2.2.2 Ordinal Data 2.2.3 Interval Data 2.2.4 Ratio Data 2.3 Descriptive Statistics 2.3.1 Measures of Central Tendency 2.3.2 Measures of Variability 2.3.3 Confidence Intervals 2.3.4 Displaying Confidence Intervals as Error Bars 2.4 Comparing Means 2.4 Independent Samples 2.4.2 Paired Samples 2.4.3 Comparing More Than Two Samples 2.5 Relationships Between Variables 2.5.1 Correlations 2.6 Non-Parametric Tests 2.6.1 The Chi-Square Test 2.7 Presenting Your Data Graphically 2.7.1 Column or Bar Graphs 2.7.2 Line Graphs 2.7.3 Scatterplots 2.7.4 Pie or Donut Charts 2.7.5 Stacked Bar Graphs 2.8 Summary 3 Planning 3.1 Study Goals 3.1.1 Formative User Research 3.1.2 Summative User Research 3.2 UX Goals 3.2.1 User Performance 3.2.2 User Preferences 3.2.3 User Emotions 3.3 Business Goals 3.4 Choosing the Right UX Metrics 3.4.1 Completing an eCommerce Transaction 3.4.2 Comparing Products 3.4.3 Evaluating Frequent Use of the Same Product 3.4.4 Evaluating Navigation and/or Information Architecture 3.4.5 Increasing Awareness 3.4.6 Problem Discovery 3.4.7 Maximizing Usability for a Critical Product 3.4.8 Creating an Overall Positive User Experience 3.4.9 Evaluating the Impact of Subtle Changes 3.4.10 Comparing Alternative Designs 3.5 User Research Methods and Tools 3.5.1 Traditional (Moderated) Usability Tests 3.5.2 Unmoderated Usability Tests 3.5.3 Online Surveys 3.5.4 Information Architecture Tools 3.5.5 Click and Mouse Tools 3.6 Other Study Details 3.6.1 Budgets and Timelines 3.6.2 Participants 3.6.3 Data Collection 3.6.4 Data Cleanup 3.7 Summary 4 Performance Metrics 4.1 Task Success 4.1.1 Binary Success Calculating Confidence Intervals for Binary Success 4.1.2 Levels of Success How to Collect and Measure Levels Of Success How to Analyze and Present Levels of Success 4.1.3 Issues in Measuring Success 4.2 Time-On-Task 4.2.1 Importance of Measuring Time-on-Task 4.2.2 How to Collect and Measure Time-on-Task Turning on and off the Clock Tabulating Time Data 4.2.3 Analyzing and Presenting Time-on-Task Data Ranges Thresholds Distributions and Outliers 4.2.4 Issues to Consider When Using Time Data Only Successful Tasks or all Tasks? Using a Concurrent Think-Aloud Protocol Should you tell the Participants about the Time Measurement? 4.3 Errors 4.3.1 When to Measure Errors 4.3.2 What Constitutes an Error? 4.3.3 Collecting and Measuring Errors 4.3.4 Analyzing and Presenting Errors Tasks with a Single Error Opportunity Tasks with Multiple Error Opportunities 4.3.5 Issues to Consider When Using Error Metrics 4.4 Other Efficiency Metrics 4.4.1 Collecting and Measuring Efficiency 4.4.2 Analyzing and Presenting Efficiency Data Lostness 4.4.3 Efficiency as a Combination of Task Success and Time 4.5 Learnability 4.5.1 Collecting and Measuring Learnability Data 4.5.2 Analyzing and Presenting Learnability Data 4.5.3 Issues to Consider When Measuring Learnability What is a Trial? Number of Trials 4.6 Summary 5 Self-Reported Metrics 5.1 Importance of Self-Reported Data 5.2 Rating Scales 5.2.1 Likert Scales 5.2.2 Semantic Differential Scales 5.2.3 When to Collect Self-Reported Data 5.2.4 How to Collect Ratings 5.2.5 Biases in Collecting Self-Reported Data 5.2.6 General Guidelines for Rating Scales 5.2.7 Analyzing Rating-Scale Data 5.3 Post-Task Ratings 5.3.1 Ease of Use 5.3.2 After-Scenario Questionnaire 5.3.3 Expectation Measure 5.3.4 A Comparison of Post-Task Self-Reported Metrics 5.4 Overall User Experience Ratings 5.4.1 System Usability Scale 5.4.2 Computer System Usability Questionnaire 5.4.3 Product Reaction Cards 5.4.4 User Experience Questionnaire 5.4.5 AttrakDiff 5.4.6 Net Promoter Score 5.4.7 Additional Tools for Measuring Self-Reported User Experience 5.4.8 A Comparison of Selected Overall Self-Reported Metrics 5.5 Using SUS to Compare Designs 5.6 Online Services 5.6.1 Website Analysis and Measurement Inventory 5.6.2 American Customer Satisfaction Index 5.6.3 OpinionLab 5.6.4 Issues With Live-Site Surveys 5.7 Other Types of Self-Reported Metrics 5.7.1 Assessing Attribute Priorities 5.7.2 Assessing Specific Attributes 5.7.3 Assessing Specific Elements 5.7.4 Open-Ended Questions 5.7.5 Awareness and Comprehension 5.7.6 Awareness and Usefulness Gaps 5.8 Summary 6 Issues-Based Metrics 6.1 What is a Usability Issue? 6.1.1 Real Issues Versus False Issues 6.2 How to Identify an Issue 6.2.1 Using Think-Aloud From One-on-One Studies 6.2.2 Using Verbatim Comments From Automated Studies 6.2.3 Using Web Analytics 6.2.4 Using Eye-Tracking 6.3 Severity Ratings 6.3.1 Severity Ratings Based on the User Experience 6.3.2 Severity Ratings Based on a Combination of Factors 6.3.3 Using a Severity Rating System 6.3.4 Some Caveats About Rating Systems 3.4 Analyzing and Reporting Metrics for Usability Issues 6.4.1 Frequency of Unique Issues 6.4.2 Frequency of Issues per Participant 6.4.3 Percentage of Participants 6.4.4 Issues by Category 6.4.5 Issues by Task 6.5 Consistency in Identifying Usability Issues 6.6 Bias in Identifying Usability Issues 6.7 Number of Participants 6.7.1 Five Participants Is Enough 6.7.2 Five Participants Is Not Enough 6.7.3 What to Do? 6.7.4 Our Recommendation 6.8 Summary 7 Eye Tracking 7.1 How Eye Tracking Works 7.2 Mobile Eye Tracking 7.2.1 Measuring Glanceability 7.2.2 Understanding Mobile Users in Context 7.2.3 Mobile Eye Tracking Technology 7.2.4 Glasses 7.2.5 Device Stand 7.2.6 Software-Based Eye Tracking 7.3 Visualizing Eye Tracking Data 7.4 Areas of Interest 7.5 Common Eye Tracking Metrics 7.5.1 Dwell Time 7.5.2 Number of Fixations 7.5.3 Fixation Duration 7.5.4 Sequence 7.5.5 Time to First Fixation 7.5.6 Revisits 7.5.7 Hit Ratio 7.6 Tips for Analyzing Eye Tracking Data 7.7 Pupillary Response 7.8 Summary 8 Measuring Emotion 8.1 Defining the Emotional User Experience 8.2 Methods to Measure Emotions 8.2.1 Five Challenges in Measuring Emotions 8.3 Measuring Emotions Through Verbal Expressions 8.4 Self-Report 8.5 Facial Expression Analysis 8.6 Galvanic Skin Response 8.7 Case Study: the Value of Biometrics 8.8 Summary 9 Combined and Comparative Metrics 9.1 Single Ux Scores 9.1.1 Combining Metrics Based on Target Goals 9.1.2 Combining Metrics Based on Percentages 9.1.3 Combining Metrics Based on Z-Scores 9.1.4 Using SUM: Single Usability Metric 9.2 Ux Scorecards and Framework 9.2.1 UX Scorecards 9.2.2 UX Frameworks 9.3 Comparison to Goals and Expert Performance 9.3.1 Comparison to Goals 9.3.2 Comparison to Expert Performance 9.4 Summary 10 Special Topics 10.1 Web Analytics 10.1.1 Basic Web Analytics 10.1.2 Click-Through Rates 10.1.3 Drop-off Rates 10.1.4 A/B Tests 10.2 Card-Sorting Data 10.2.1 Analyses of Open Card-Sort Data 10.2.1.1 Hierarchical Cluster Analysis 10.2.1.2 Multidimensional Scaling 10.2.1.3 How Many Participants Are Enough for a Card-Sorting Study? 10.2.2 Analyses of Closed Card-Sort Data 10.3 Tree Testing 10.4 First Click Testing 10.5 Accessibility Metrics 10.6 Return-on-Investment Metrics 10.7 Summary 11 Case Studies 11.1 Thinking Fast and Slow in the Netflix TV User Interface Background Methods Participant Interviews Materials Procedure Example scenarios Eye-Tracking Results Discussion Thinking Fast Thinking Slow Impact Biography 11.2 Participate/Compete/Win (Pcw) Framework: Evaluating Products and Features in the Marketplace 11.2.1 Introduction 11.2.2 Outlining Objective Criteria Participate Compete Win 11.2.3 Feature Analysis Generating the Feature List Calculating a Feature Importance Score Choosing a Competitive Product Calculating a Feature Availability/Value Score 11.2.4 “PCW” (Summative) Usability Testing Overall Versus Workflow Success Rate Prototype Versus Live Site Testing Overall Program Success Biographies 11.3 Enterprise UX Case Study: Uncovering the “UX Revenue Chain” 11.3.1 Introduction 11.3.1 Metric Identification and Selection Participants 11.3.2 Methods Top Task Identification Top Task Force Ranking Survey Task-based Benchmark Study Pre-redesign 11.3.3 Analysis 11.3.4 Results 11.3.5 Conclusion Biography 11.4 Competitive UX Benchmarking of four Healthcare Websites 11.4.1 Methodology Data collection through UserZoom for both studies Experimental Design for the Quantitative N = 200 study Metrics and Key Performance Indicators 11.4.2 Results qxScore Overall Results Task Details Pre-Versus Post-Site Perception 11.4.3 Summary and Recommendations 11.4.4 Acknowledgment and Contributions 11.4.5 Biography 11.5 Closing the SNAP Gap 11.5.1 Field Research 11.5.2 Weekly Reviews 11.5.3 Application Questions 11.5.4 Surveys 11.5.5 Testing Prototypes 11.5.6 Success Metric 11.5.7 Organizations GoInvo Massachusetts Department of Transitional Assistance 11.5.8 Biography 12 Ten Keys to Success 12.1 Make the Data Come Alive 12.2 Don’t Wait to be Asked to Measure 12.3 Measurement is Less Expensive than you Think 12.4 Plan Early 12.5 Benchmark Your Products 12.6 Explore Your Data 12.7 Speak the Language of Business 12.8 Show Your Confidence 12.9 Don’t Misuse Metrics 12.10 Simplify Your Presentation Index
Donate to keep this site alive
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.