Course on Open Science 101

Main Text

The replication crisis has raised many questions for researchers, but it also poses unique challenges for those teaching about science and the scientific process. We now know that many scientific studies cannot be accurately reproduced or their findings replicated, and this has raised concerns about the validity and reliability of scientific research, leading to calls for increased transparency and reproducibility in scientific inquiry.

For those involved in teaching science, it is therefore not possible to simply continue as we have been, for example, accepting and presenting published scientific works as the “truth” or final word on any given phenomenon. Rather, we need to take a step back and teach students about the entirety of the research process, both in its idealised form, and addressing the complex reality of a scientific process that is impacted by cognitive biases, human behaviour, and incentive structures.

Introducing Students to Open Science

These lectures and tutorials introduce students to the scientific process and open research practices, in order to ensure that psychological research is as rigorous, transparent and, ultimately, as reproducible as possible. We also consider problematic practices that have frequently occurred in psychological research, and discuss the impact they have had on psychological knowledge, as well as learning how to avoid such problematic practices in the future. The module content was designed to be delivered at a first-year undergraduate level, and can also be delivered in a pretty modular fashion, allowing the tutor to pick and choose which sessions best suit their programme or student needs.

Some might wonder if the first year of a degree programme is too early to be learning about open science. Shouldn’t we be teaching them the traditional approach first and then, once they have this grounding, get into open science practices later in their degree? I would strongly disagree with such an approach. There are several reasons why it’s important to teach open science to undergraduate students, and at the earliest possible opportunity. First, open science promotes transparency and reproducibility in research, which are essential principles of scientific inquiry. By teaching students about open science, we can help ensure that they are conducting research in a responsible and ethical manner. Why would we first teach them an approach to research that doesn’t integrate these ideals? It makes no sense to me.

In exposing students to open research and to questions of research integrity, they get to see not only the unvarnished nature of scientific inquiry, warts and all, but also to see the true potential of applying open research practices. They can see how open science can make research more collaborative and inclusive, for example by highlighting how to share data openly, so they can more easily share their findings with others, and also being able to build upon the work of their peers. Teaching open science can also help prepare students for the rapidly changing landscape of scientific research, where open and collaborative approaches are becoming increasingly important. Open research skills are frequently highlighted in academic job requirements, as well as cited as best practice by education institutions, charitable foundations, and research funders.

The Course Content

The content presented here was jointly developed with colleagues at Lancaster University (Dr. Marina Bazhydai, Dr. Sally Linkenauger, Dr. Neil McLatchie), as part of two undergraduate psychology modules, formally listed as Research Integrity and Open Science I (PSYC123) and Research Integrity and Open Science II (PSYC124).

Because the courses assume no knowledge of the research process, the course first focusses on the scientific method, thinking about experimental design, and questions of confounds, reliability, and validity. Once we have that under our belts we move on to contrasting “traditional” approaches to research vs an “open science” approach, and cover topics of study preregistration, replications, and sample size considerations.

We then turn to some of the thornier questions science and scientists have been faced with: How do we determine what is true in science? How do we know which theories are well supported by evidence and which ones are not? How can we tell if researchers are trying to pull the wool over our eyes? To start to think about these questions we look at learning how to spot and avoid questionable research practices, in favour of practices that are open, transparent and reproducible. In this way, we think about some of the problems faced by researchers, such as cognitive biases, researcher flexibility in reporting outcomes, and even fraud. We ask how we can assess research findings in the face of such problems?

These modules will show students how scientific practices in the field have changed rapidly in recent years, with renewed interest in openness and transparency. While we focus on a range of problems that impact our interpretation of research findings, students will also develop an understanding of tools that can help overcome and prevent these issues from arising (e.g., study preregistration, sharing of data, meta-analyses etc.).

For tutors and instructors, lecture slides were designed to be covered in about 50 minutes, and the labs can also be completed in around 50 minutes, although some will take a little longer. It’s also worth noting that these modules don’t involve any statistical programming, focussing more on open research concepts, but can (and are) taught alongside modules more focussed on learning statistics through R. There is also some flexibility in the order that topics can be delivered. For example, Meta-Analysis is a standalone topic that could be moved later in the course, or handled separately as part of a more advanced research methods module.

The course content has an open licence (CC BY), so we encourage instructors to use and modify the content to meet their needs. For instructors, answer sheets are available for the majority of lab exercises, so please just email if you’d like a copy. If you do use any of the materials, we would love to hear about it! Indeed, we would be grateful for any feedback you have to offer.

Course content is available on the Open Science Framework, so to download everything go here for PSYC123, and here for PSYC124.

Contact information:

Dr. Dermot Lynott, Department of Psychology, Maynooth University (Email: dermot.lynott@mu.ie; Homepage.)