What’s wrong with statistical tests – and where do we go from here?

Abstract

This chapter considers problems with null hypothesis significance testing (NHST). The literature in this area is quite large. D. Anderson, Burnham, and W. Thompson (2000) recently found more than 300 articles in different disciplines about the indiscriminate use of NHST, and W. Thompson (2001) lists more than 400 references about this topic. As a consequence, it is possible to cite only a few representative works. After review of the debate about NHST, the author argues that the criticisms have sufficient merit to support the minimization or elimination of NHST in the behavioral sciences. The author offers specific suggestions along these lines. Some concern alternatives that may replace or supplement NHST and thus are directed at researchers. Others concern editorial policies or educational curricula. Few of the recommendations given are original in that many have been made over the years by various authors. However, as a set they deal with issues often considered in separate works. For simplicity, the context for NHST assumed is reject-support (RS) instead of accept-support (AS). The RS context is more common, and many of the arguments can be reframed for the AS context.