Designing, Running, and Analyzing Experiments

Product type
Logo Coursera (CC)
Provider rating: starstarstarstar_halfstar_border 6.6 Coursera (CC) has an average rating of 6.6 (out of 5 reviews)

Need more information? Get more details on the site of the provider.

Description

When you enroll for courses through Coursera you get to choose for a paid plan or for a free plan

  • Free plan: No certicification and/or audit only. You will have access to all course materials except graded items.
  • Paid plan: Commit to earning a Certificate—it's a trusted, shareable way to showcase your new skills.

About this course: You may never be sure whether you have an effective user experience until you have tested it with users. In this course, you’ll learn how to design user-centered experiments, how to run such experiments, and how to analyze data from these experiments in order to evaluate and validate user experiences. You will work through real-world examples of experiments from the fields of UX, IxD, and HCI, understanding issues in experiment design and analysis. You will analyze multiple data sets using recipes given to you in the R statistical programming language -- no prior programming experience is assumed or required, but you will be required to read, understand, and modify co…

Read the complete description

Frequently asked questions

There are no frequently asked questions yet. If you have any more questions or need help, contact our customer service.

When you enroll for courses through Coursera you get to choose for a paid plan or for a free plan

  • Free plan: No certicification and/or audit only. You will have access to all course materials except graded items.
  • Paid plan: Commit to earning a Certificate—it's a trusted, shareable way to showcase your new skills.

About this course: You may never be sure whether you have an effective user experience until you have tested it with users. In this course, you’ll learn how to design user-centered experiments, how to run such experiments, and how to analyze data from these experiments in order to evaluate and validate user experiences. You will work through real-world examples of experiments from the fields of UX, IxD, and HCI, understanding issues in experiment design and analysis. You will analyze multiple data sets using recipes given to you in the R statistical programming language -- no prior programming experience is assumed or required, but you will be required to read, understand, and modify code snippets provided to you. By the end of the course, you will be able to knowledgeably design, run, and analyze your own experiments that give statistical weight to your designs.

Who is this class for: This class is at the level of advanced undergraduates or first-year graduate students in fields related to human-computer interaction, computer science, interaction design, user experience design, user-centered design, or human-technology behavior studies.

Created by:  University of California, San Diego
  • Taught by:  Scott Klemmer, Associate Professor

    Cognitive Science & Computer Science
  • Taught by:  Jacob O. Wobbrock, Associate Professor

    The Information School
Basic Info Course 7 of 8 in the Interaction Design Specialization Commitment 9 weeks, 3-4 hours/week Language English How To Pass Pass all graded assignments to complete the course. User Ratings 3.6 stars Average User Rating 3.6See what learners said Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from your peers

Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends, colleagues, and employers.

University of California, San Diego UC San Diego is an academic powerhouse and economic engine, recognized as one of the top 10 public universities by U.S. News and World Report. Innovation is central to who we are and what we do. Here, students learn that knowledge isn't just acquired in the classroom—life is their laboratory.

Syllabus


WEEK 1


Basic Experiment Design Concepts



In this module, you will learn basic concepts relevant to the design and analysis of experiments, including mean comparisons, variance, statistical significance, practical significance, sampling, inclusion and exclusion criteria, and informed consent. You’ll also learn to think of an experiment in terms of its participants, apparatus, procedure, and design & analysis. This module covers lecture videos 1-2.


2 videos, 1 reading expand


  1. Video: 01. What You Will Learn in this Course
  2. Reading: All Course Materials
  3. Video: 02. Basic Experiment Design Concepts

Graded: Understanding the Basics

WEEK 2


Tests of Proportions



In this module, you will learn how to analyze user preferences (or other tallies) using tests of proportions. You will also get up and running with R and RStudio. Topics covered include independent and dependent variables, variable types, exploratory data analysis, p-values, asymptotic tests, exact tests, one-sample tests, two-sample tests, Chi-Square test, G-test, Fisher’s exact test, binomial test, multinomial test, post hoc tests, and pairwise comparisons. This module covers lecture videos 3-9.


7 videos expand


  1. Video: 03. Description of a Study of User Preferences
  2. Video: 04. Getting Started with R and RStudio
  3. Video: 05. Exploring Data and a First Test of Proportions
  4. Video: 06. Understanding and Reporting Your First Statistical Test
  5. Video: 07. Exact Tests, Asymptotic Tests, and the Binomial Test
  6. Video: 08. One-Sample Tests of Proportions
  7. Video: 09. Two-Sample Tests of Proportions

Graded: Understanding Tests of Proportions
Graded: Doing Tests of Proportions

WEEK 3


The T-Test



In this module, you will learn how to design and analyze a simple website A/B test. Topics include measurement error, independent variables as factors, factor levels, between-subjects factors, within-subjects factors, dependent variables as responses, response types, balanced designs, and how to report a t-test. You will perform your first analysis of variance in the form of an independent-samples t-test. This module covers lecture videos 10-11.


2 videos expand


  1. Video: 10. Experiment Design Concepts in a Simple A/B Test
  2. Video: 11. Analyzing a Simple A/B Test with a T-Test

Graded: Understanding Experiment Designs
Graded: Doing Independent-Samples T-Tests

WEEK 4


Validity in Design and Analysis



In this module, you will learn about how to ensure that your data is valid through the design of experiments, and that your analyses are valid by understanding and testing for their assumptions. Topics include how to achieve experimental control, confounds, ecological validity, the three assumptions of ANOVA, data distributions, residuals, normality, homoscedasticity, parametric versus nonparametric tests, the Shapiro-Wilk test, the Kolmogorov-Smirnov test, Levene’s test, the Brown-Forsythe test, and the Mann-Whitney U test. This module covers lecture videos 12-15.


4 videos expand


  1. Video: 12. Designing for Experimental Control
  2. Video: 13. Data Assumptions and Distributions
  3. Video: 14. Testing for ANOVA Assumptions
  4. Video: 15. Mann-Whitney, a Nonparametric T-Test

Graded: Understanding Validity
Graded: Doing Tests of Assumptions

WEEK 5


One-Factor Between-Subjects Experiments



In this module, you will learn about one-factor between-subjects experiments. The experiment examined will be a between-subjects study of task completion time with various programming tools. You will understand and analyze data from two-level factors and three-level factors using the independent-samples t-test, Mann-Whitney U test, one-way ANOVA, and Kruskal-Wallis test. You will learn how to report an F-test. You will also understand omnibus tests and how they relate to post hoc pairwise comparisons with adjustments for multiple comparisons. This module covers lecture videos 16-18.


3 videos expand


  1. Video: 16. Description of a Study for a Oneway ANOVA
  2. Video: 17. Analyzing and Reporting a Oneway ANOVA
  3. Video: 18. Kruskal-Wallis, a Nonparametric Oneway ANOVA

Graded: Understanding Oneway Designs
Graded: Doing Oneway ANOVAs

WEEK 6


One-Factor Within-Subjects Experiments



In this module, you will learn about one-factor within-subjects experiments, also known as repeated measures designs. The experiment examined will be a within-subjects study of subjects searching for contacts in a smartphone contacts manager, including the analysis of times, errors, and effort Likert-type scale ratings. You will learn counterbalancing strategies to avoid carryover effects, including full counterbalancing, Latin Squares, and balanced Latin Squares. You will understand and analyze data from two-level factors and three-level factors using the paired-samples t-test, Wilcoxon signed-rank test, oneway repeated measures ANOVA, and Friedman test. This module covers lecture videos 19-23.


5 videos expand


  1. Video: 19. Description of a Study for a Oneway Repeated Measures ANOVA
  2. Video: 20. Counterbalancing Repeated Measures Factors
  3. Video: 21. Long-Format and Wide-Format Data Tables
  4. Video: 22. The Paired T-Test and Wilcoxon Signed-Rank Test
  5. Video: 23. Analyzing a Repeated Measures ANOVA and Friedman Test

Graded: Understanding Oneway Repeated Measures Designs
Graded: Doing Oneway Repeated Measures ANOVAs

WEEK 7


Factorial Experiment Designs



In this module, you will learn about experiments with multiple factors and factorial ANOVAs. The experiment examined will be text entry performance on different smartphone keyboards while sitting, standing, and walking. Topics include mixed factorial designs, interaction effects, factorial ANOVAs, and the Aligned Rank Transform as a nonparametric factorial ANOVA. This module covers lecture videos 24-27.


4 videos expand


  1. Video: 24. Description of a Study for a Factorial ANOVA
  2. Video: 25. Understanding Interaction Effects
  3. Video: 26. Analyzing a Factorial ANOVA
  4. Video: 27. The ART, a Nonparametric Factorial ANOVA

Graded: Understanding Factorial Designs
Graded: Doing Factorial ANOVAs

WEEK 8


Generalizing the Response



In this module, you will learn about analyses for non-normal or non-numeric responses for between-subjects experiments using Generalized Linear Models (GLM). We will revisit three previous experiments and analyze them using generalized models. Topics include a review of response distributions, nominal logistic regression, ordinal logistic regression, and Poisson regression. This module covers lecture videos 28-29.


2 videos expand


  1. Video: 28. Introduction to Generalized Linear Models
  2. Video: 29. Analyzing Three Generalized Linear Models

Graded: Understanding Generalized Linear Models
Graded: Doing Generalized Linear Models

WEEK 9


The Power of Mixed Effects Models



In this module, you will learn about mixed effects models, specifically Linear Mixed Models (LMM) and Generalized Linear Mixed Models (GLMM). We will revisit our prior experiment on text entry performance on smartphones but this time, keeping every single measurement trial as part of the analysis. The full set of analyses covered in this course will also be reviewed. This module covers lecture videos 30-33.


4 videos expand


  1. Video: 30. Introduction to Mixed Effects Models
  2. Video: 31. Analyzing a Linear Mixed Model
  3. Video: 32. Analyzing a Generalized Linear Mixed Model
  4. Video: 33. Course in Review

Graded: Understanding Mixed Effects Models
Graded: Doing Mixed Effects Models
There are no reviews yet.

Share your review

Do you have experience with this course? Submit your review and help other people make the right choice. As a thank you for your effort we will donate £1.- to Stichting Edukans.

There are no frequently asked questions yet. If you have any more questions or need help, contact our customer service.