Methodological Advances in Behavioral Research: Crowdsourcing Science

Abstract

The results of many published studies across many scientific domains are not easily reproduced by independent laboratories. For example, an initiative by Bayer Healthcare to replicate 67 pre-clinical studies led to a reproducibility rate of 20-25% (Prinz et al., 2011), and researchers at Amgen were only able to replicate 6 of 53 influential cancer biology studies (Begley & Ellis, 2012). Similar replication failures have been reported in social and cognitive psychology (Ebersole et al., 2015; Klein et al., 2014; Open Science Collaboration, 2015). This PhD Boot camp introduces PhD students in the behavioral sciences to 1) the ongoing “crisis of confidence” in science, 2) typical methodological challenges of conducting replications, 3) the philosophy of science and statistical background of replications, 4) highly collaborative approaches to replication, in which findings are replicated in independent laboratories before (rather than after) they are published. As part of the boot camp, students will be organized into replication teams and take part in a crowdsourced pre-publication independent replication project on which they will be creditedas co-authors. Participation in the bootcamp is free, the replications will be funded by a grant from INSEAD, and the infrastructure for data collection is already in place. Crowdsourcing research involves recruiting numerous scientific teams to achieve large-scale projects no single team could feasibly carry out. Leveraging crowds of researchers increases the statistical power and generalizability of research designs, reduces investigator error and bias, and enhances scientific transparency. Actively participating in a large-scale replication effort provides an opportunity for students to experience the power of a crowd of researchers firsthand. Lecture topics will include the scientific crisis caused by high-profile replication failures, publication bias, questionable research practices, the open data movement, and crowdsourced replication efforts, among others.