American Scientist

To the Editors:

In her column “Do We Really Need the S-word?” (Macroscope, January–February), Megan Higgs points out many of the unfortunate consequences of the excessive use of the term “statistically significant” in current scientific literature. Much of the difficulty stems from the fact that the meaning of “significant” in everyday language bears little resemblance to its meaning in the context of statistical inference, and I agree with her thesis that its rampant use needs to be curtailed.

Although deploring the use of the s -word to “objectivize inference” and “classify results,” Higgs provides little in the way of examples to help readers along the path toward s -word elimination. On page 6 of her article, she suggests that “if we mean that the two-sided p-value is less than 0.05, then let us just say that.” One can ask whether “just saying that” would be much of an improvement over the current state of affairs, because reasonable statistical sophistication is needed to understand what this statement means in the context of a given study. Perhaps a better approach might be to encourage use of the word “chance” in the results section of scientific papers. For example, in a study of two independent samples comparing their sample means when the p-value is 0.038, we might say: “In the absence of a real difference between the populations being compared, a difference as large as was observed in this study would occur by chance only 3.8 percent of the time.” As an alternative, we might say, “In the absence of a real difference between the populations being compared, random sampling would produce a difference as large as that observed in this study only 3.8 percent of the time.”

If we were explaining the results of an analysis in which a salary of y dollars was regressed on x years of experience, and the fitted slope coefficient was $950 per year with a p-value of 0.011, we might say: “Average salary increased by $950 for each additional year of experience, and in the absence of a relationship in the population, an effect as large as this one would occur by chance only about 1 percent of the time.”

The big advantage of replacing the s -word with the chance word is that the use of the word “chance” in everyday language has much the same meaning as it does in the context of statistical inference. Consequently, the transfer of results from the scientific literature to the media read by the general public should be less problematic. And even within the bounds of the readership of scientific journals, more readers are likely to make reasonable statistical interpretations of observed differences if they are explained in terms of chance rather than p-values. It is my sincere hope that Higgs’s article will stimulate some discussion on how we might make progress toward elimination of the s-word.

Don Holbert
Retired Professor of Biostatistics
East Carolina University
Greenville, NC