Statistical significance in psychological research.

Abstract

MOST THEORIES IN THE AREAS OF PERSONALITY, CLINICAL, AND SOCIAL PSYCHOLOGY PREDICT ONLY THE DIRECTION OF A CORRELATION, GROUP DIFFERENCE, OR TREATMENT EFFECT. SINCE THE NULL HYPOTHESIS IS NEVER STRICTLY TRUE, SUCH PREDICTIONS HAVE ABOUT A 50-50 CHANCE OF BEING CONFIRMED BY EXPERIMENT WHEN THE THEORY IN QUESTION IS FALSE, SINCE THE STATISTICAL SIGNIFICANCE OF THE RESULT IS A FUNCTION OF THE SAMPLE SIZE. CONFIRMATION OF 1 DIRECTIONAL PREDICTION GENERALLY BUILDS LITTLE CONFIDENCE IN THE THEORY BEING TESTED. MOST THEORIES SHOULD BE TESTED BY MULTIPLE CORROBORATION AND MOST EMPIRICAL GENERALIZATIONS BY CONSTRUCTIVE REPLICATION. STATISTICAL SIGNIFICANCE, PERHAPS THE LEAST IMPORTANT ATTRIBUTE OF A GOOD EXPERIMENT, IS NEVER A SUFFICIENT CONDITION FOR CLAIMING THAT (1) A THEORY HAS BEEN USEFULLY CORROBORATED, (2) A MEANINGFUL EMPIRICAL FACT HAS BEEN ESTABLISHED, OR (3) AN EXPERIMENTAL REPORT OUGHT TO BE PUBLISHED