[NOTE: This is a repost of a blog that Doug Campbell wrote for his blogsite at douglaslcampbell.blogspot.co.nz] Stephen Hsu has a nice blog post on this topic. He writes about this common pattern: (1) Study reports results which reinforce the dominant, politically correct, narrative. (2) Study is widely cited in other academic work, lionized in the popular press, and used to advance real world agendas.
As part of a major replication and robustness project of articles in the American Economic Review, this fall I assigned students in my Masters Macro course at the New Economic School (Moscow) to replicate and test robustness for Macro papers published in the AER. In our sample of AER papers, 66% had full data available online, and the replicated results were exactly the same as in the paper 72% of the time.