BOB REED: Replications Can Make Things Worse? Really?

In a recent article in Slate entitled “The Unintended Consequences of Trying to Replicate Research,” IVAN ORANSKY and ADAM MARCUS from Retraction Watch argue that replications can exacerbate research unreliability.  The argument assumes that publication bias is more likely to favour confirming replication studies over disconfirming studies. To read more, click here. This is the same argument that Michele Nuijten makes in her guest blog for *TRN,*which you can read here.

Whether this is a real concern depends on the replication policies at journals.  At least two economics journals have publication policies that explicitly state they are neutral towards the conclusion of replication studies.  In their “Call for Replication Studies”, Burmann et al. state: “Public Finance Review will publish all … kinds of replication studies, those that validate and those that invalidate previous research” ( see here).  And the journal Economics: The Open-Access, Open-Assessment E-Journalstates: “The journal will publish both confirmations and disconfirmations of original studies. The only consideration will be quality of the replicating study” ( see here).

Further, in their recent study, “Replications in Economics: A Progress Report” ( see here), Duvendack et al. find that most published replication studies in economics disconfirm the original research.  So while it is possible that replications could make things worse, perhaps this is more a worry in theory than in practice.  At least in economics.

Share this:

• Click to share on X (Opens in new window) X
• Click to share on Facebook (Opens in new window) Facebook

Like Loading…