A Student’s Guide to Open Science: Using the Replication Crisis to Reform Psychology

A child of the replication crisis

When Open University Press approached me to write a book on the “replication crisis in psychology”, I was filled with excitement as I knew that discussion of replication issues is scarce within teaching and psychology textbooks. If they don’t teach it, then I could! But I also knew it had to be more than the so-called ‘crisis’; a term associated with pessimism. It had to be about how far the discipline of psychology has come, and the opportunities that open research can bring to reform research training and the wider culture.

I am a ‘child of the replication crisis’. I completed my PhD in 2016 on ‘stereotype threat’; an influential phenomenon that suggests negative societal stereotypes deplete performance. But I got stuck time and time again trying to replicate these classic effects. I felt like a failure. I wish I had known what I know now: that the experiences I was facing were more common than what was gleaned from the published literature. Science is not perfect and nor should it be, but this was not what I was taught through my undergraduate education. So I wrote a book – A Student’s Guide to Open Science – for my younger self. In other words, for the talented and passionate students studying Psychology. By knowing our history – the good parts but also the messy parts – our students represent the grassroots that can make this discipline the best it can be!

What is this book about and why is it needed?

By now, many researchers have heard about the ‘replication crisis’ that has swept research disciplines, and the fast-paced improvements brought about by open research reform. However, there is still a gap in educational practices – in the teaching of the replication crisis – which FORRT and its members strive to overcome. Furthermore, when trying to learn about the replication crisis and open science, it can feel overwhelming and there are lots of different resources in many different places. With this in mind, I wrote a “A Student’s Guide to Open Science” as a way of pulling together all of these various pieces of information and useful resources that are currently out there.

The book walks students (and their educators) through a recent history of the replication crisis in psychology, the causes and drivers of crises (including academic incentive structures, questionable research practices, and cognitive biases), and open science reform, its benefits and associated challenges. Importantly, to ensure that students understand how to put these new practices into practice, it also includes a handy, digestible guide to implementing open science practices such as preprints, preregistration, Registered Reports, and open materials, data, and code. It is my hope that this book becomes an essential guide to navigating the replication crisis.

Pedagogic resources within the book

This book is primarily aimed at students studying psychology at both undergraduate and postgraduate level, but throughout I include useful pedagogic resources that educators can use to teach the replication crisis and open science reform. For example, there is an activity where students plan to replicate a study and can understand the various difficulties they may stumble upon in their quest. Here, they find their favourite article from the journal it was published and walk themselves through different questions to assess whether there is enough information to perform an exact replication – what is the aim of the study? Do the researchers specify their main hypotheses? What is the research design? Are the measures they used detailed enough to understand how they were scored? Do they report whether (and how) they screened or cleaned their data? (see Box 1).

Box 1. The seat of a replicator. An example pedagogic activity from “A Student’s Guide to Open Science”.

This same pedagogic activity can then be expanded to teach about open science reform. Specifically, using the notes that students have taken from their “seat of a replicator activity”, they can then have a go at preregistering their replication using a template from a verified repository (e.g., AsPredicted.org/Open Science Framework). These templates can be downloaded from each repository and students can either complete these as a formative (e.g., seminar activity) or summative assessment (e.g., written report; see Box 2).

Box 2. Pedagogic activity: preregistering a replication study.

In addition to these pedagogic resources, the book also contains top tips for implementing open science practices into student’s research training, including for preprints, preregistration, Registered Reports, and open materials, data, and code. Whilst the focus is mainly on quantitative research, it also provides links and resources to qualitative research. In addition, throughout it discusses the many different organisations and initiatives that have shaped my own learning of open science (see Box 3), thus providing a detailed overview of everything we have learnt so far from the advent of the replication crisis through to current open science reform.

Box 3. Key resources to get started with open science.

Become a student of open science!

The replication crisis and open science reform is not currently embedded within the teaching of psychology. It is my hope that with all of these wonderful new and ever-evolving resources, today’s students become critical consumers of science and pave the way to creating a truly robust, transparent, replicable, reproducible and representative psychological science.

Author Bio & Contact:

Dr Charlotte Pennington is a Lecturer in Psychology at Aston University, Birmingham, a Fellow of the Higher Education Academy, and a member of FORRT. Her book, “A Student’s Guide to Open Science: Using the Replication Crisis to Reform Psychology” is available in all good bookstores and inspection copies are available from McGraw Hill for review. If you have trouble accessing it, please let Charlotte know via the contact details below and she will do her best to help.

Email: c.pennington@aston.ac.uk

Twitter: @drcpennington

Mastodon: @drcpennington@mastodon.social