Pre-registration and Results-Free Review in Observational and Qualitative Research

Abstract

Amidst rising concern about publication bias, pre-registration and results-blind review have grown rapidly in use. Yet discussion of both the problem of publication bias and of potential solutions has been remarkably narrow in scope: publication bias has been understood largely as a problem afflicting quantitative studies, while pre-registration and results-blind review have been almost exclusively applied to experimental or otherwise prospective research. This chapter examines the potential contributions of pre-registration and results-blind review to qualitative and quantitative retrospective research. First, the chapter provides an empirical assessment of the degree of publication bias in qualitative political science research. Second, it elaborates a general analytic framework for evaluating the feasbility and utility of pre-registration and results-blind review for confirmatory studies. Third, through a review of published studies, the paper demonstrates that much observational—and, especially, qualitative—political science research displays features that would make for credible pre-registration. The paper concludes that pre-registration and results-blind review have the potential to enhance the validity of confirmatory research across a range of empirical methods, while elevating exploratory work by making it harder to disguise discovery as testing.