Generalizability Crisis Session

Duration: 60 minutes

Learning Objectives

• Understand the concept of generalizability in research
• Recognize the factors that contribute to the generalizability crisis
• Discuss the implications of the generalizability crisis in various fields

Introduction

• What is the generalizability crisis?
• What factors contribute to it?

Activity 1

Interactive activity on LearningApps.org

Activity 2 - Case Studies

Discuss problems relating to design that you can identify in the following case studies.

1. A study on heart medication tested mostly on young, healthy individuals.
   ____________________

2. A study on depression prevalence in college students at a predominantly white university.
   ____________________

3. A cancer treatment study conducted in one urban hospital only.
   ____________________

4. A study on teaching method effectiveness conducted in a culturally homogeneous school.
   ____________________

5. A study on anxiety therapy using only self-reported measures.
   ____________________

6. A health condition prevalence study based on a single-day survey.
   ____________________

7. A mental health intervention study showing effects in one trial but not replicated.
   ____________________

Activity 3 - Implications

Reflect on the case studies and research you’re familiar with — what are the implications of these flaws?
Now look at some specific examples. Can you add anything else?

Match the Examples to the Titles

Titles:

• Some groups might be disadvantaged regarding depression treatment
• Lack of representation of diverse populations in clinical trials
• Some groups overlooked in genetic research
• Inaccurate treatment recommendations for women
• Black skin conditions not as easy to treat

Examples:

1. Some dermatological treatments are less effective for black patients due to skin differences.
2. Clinical trials often exclude women, minorities, and older adults.
3. Cardiovascular research focused on men leads to poor treatment for women.
4. Genetic research overemphasizes European ancestry populations.
5. Mental health research based on Western populations may not generalize.

Activity 4 - Strategies to Address the Problem

Look back at the case studies and reflect on how these flaws could be addressed. Brainstorm how to mitigate these issues in your field.