The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too)

Abstract

Most discussions of rigor and replication focus on empirical practices (methods used to collect and analyze data). Typically overlooked is the role of conceptual practices: the methods scientists use to arrive at and articulate research hypotheses in the first place. This article discusses how the conceptualization of research hypotheses has implications for methodological decision-making and, consequently, for the replicability of results. The article identifies three ways in which empirical findings may be non-replicable, and shows how all three kinds of non-replicability are more likely to emerge when scientists take an informal conceptual approach, in which personal predictions are equated with scientific hypotheses. The risk of non-replicability may be reduced if scientists adopt more formal conceptual practices, characterized by the rigorous use of “if–then” logic to articulate hypotheses, and to systematically diagnose the plausibility, size, and context-dependence of hypothesized effects. The article identifies benefits that are likely to arise from more rigorous and systematic conceptual practices, and identifies ways in which their use can be encouraged to be more normative within the scholarly culture of the psychological sciences.