On Not Confusing the Tree of Trustworthy Statistics with the Greater Forest of Good Science: A Comment on Simmons et al.’s Perspective on Pre-registration

Abstract

In this commentary on Simmons, Nelson, and Simonsohn (this issue), we examine their rationale for pre-registration within the broader perspective of what good science is. We agree that there is potential benefit in a system of pre-registration if implemented selectively. However, we believe that other tools of open science such as the full sharing of study materials and open access to underlying data, provide most of the same benefits—and more (i.e., the prevention of outright fraud)—without risking the potential adverse consequences of a system of pre-registration. This is why we favor these other means of controlling type I error and fostering transparency. Direct replication, as opposed to conceptual replication, should be encouraged as well.