As part of a major replication and robustness project of articles in the American Economic Review, this fall I assigned students in my Masters Macro course at the New Economic School (Moscow) to replicate and test robustness for Macro papers published in the AER. In our sample of AER papers, 66% had full data available online, and the replicated results were exactly the same as in the paper 72% of the time.
The replication crisis has elicited a number of recommendations, from betting on beliefs, to open data, to improved norms in academic journals regarding replication studies. In our recent working paper, âA Call for Out-of-Sample Testing in Macroeconomicsâ ( available at SSRN), we argue that a renewed focus on out-of-sample tests will significantly mitigate the issues with replication, and we document the fact that out-of-sample tests are absent from entire literatures in economics.