The Cult of Statistical Significance shows, field by field, how "statistical significance," a technique that dominates many sciences, has been a huge mistake. The authors find that researchers in a broad spectrum of fields, from agronomy to zoology, employ "testing" that doesn't test and "estimating" that doesn't estimate. The facts will startle the outside reader: how could a group of brilliant scientists wander so far from scientific magnitudes? This study will encourage scientists who want to know how to get the statistical sciences back on track and fulfill their quantitative promise. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots.
"McCloskey and Ziliak have been pushing this very elementary, very correct, very important argument through several articles over several years and for reasons I cannot fathom it is still resisted. If it takes a book to get it across, I hope this book will do it. It ought to."
—Thomas Schelling, Distinguished University Professor, School of Public Policy, University of Maryland and 2005 Nobel Prize Laureate in Economics
"With humor, insight, piercing logic and a nod to history, Ziliak and McCloskey show how economists—and other scientists—suffer from a mass delusion about statistical analysis. The quest for statistical significance that pervades science today is a deeply flawed substitute for thoughtful analysis. This hollow pursuit, kept alive by mechanical, conformist thinking, has led to grave and obvious errors. Yet few participants in the scientific bureaucracy have been willing to admit what Ziliak and McCloskey make clear: the emperor has no clothes."
—Kenneth Rothman, Professor of Epidemiology, Boston University School of Health
"[Steve Ziliak and Deirdre McCloskey] explain to us why the misunderstanding of statistical significance has lead to bad government policy making and how one particularly famous brewery employed the technique to improve the pints we enjoy today."
—Tim Harford, BBC
"Despite appearing to be a book of limited appeal - it is after all a book that looks at a set of statistical techniques - it is one that has immense social implications. We live in an age where ideologies have largely been cast aside and instead we are governed increasingly by a class of politicians and civil servants who aim for 'evidence-based' policy-making. When that evidence is based on statistically significant results that ignore any quantification of results then we all have reason to pay attention."
—London Book Review
"Persuading professionals that their procedures are wrong is a long and lonely task. McCloskey, joined later by Ziliak, has been conducting such a crusade against the misuse of significance testing for over 25 years. This book presents their argument, gives lots of examples of the adverse consequences of misuse, and provides some history of the controversy, which dates from the origins of mathematical statistics."
—Ron P. Smith, Journal of Economic Issues
"The Cult of Statistical Significance has virtues that extend beyond its core message. It is clearly written and should be accessible to those who have neither formal training in statistics nor a desire to secure any. It is full of examples that illustrate why it is the strength of relationships and not their statistical significance that mainly matters."
—Richard Lempert, Law and Social Inquiry
"A clear trade-off: how much confidence [in a result] is "enough" depends on the costs of further research and the benefits of extra precision. Ziliak and his co-author Deirdre McCloskey argue in The Cult of Statistical Significance that most academic disciplines have forgotten this trade-off . . . A sharp line for statistical significance makes no sense, and it has a cost."
—Tim Harford, The Financial Times
"If not Fisherian significance, what should be the Holy Grail of statistics? Ziliak and McCloskey . . . answer: "Oomph." We should identify quantities that matter and measure them, not merely determine whether they can be distinguished from the null (meaning no effect) at some predetermined likelihood level. The validity of this point I take to be virtually self-evident. Yet statistical tests that ignore quantity remain pervasive, as the authors demonstrate through quantitative analyses of the contents of some very prestigious journals of economics, psychology, and medicine."
—Theodore Porter, Science
"The book is a model of scholarship, transparent in its method, wide-reaching in its disciplinary expertise, and highly literate, including occasional haiku poems and humor such as, 'If the variable doesn't fit/you may not have to acquit.' The authors convincingly argue that environmental quality, jobs, and even lives are at stake."
—M. H. Maier, Glendale Community College, Choice
Listen: Interview Forbes | 12/30/2015
Read: Article Forbes | 1/6/2016
LISTEN: Author Steve Ziliak on beer-o-nomics, BBC Radio 4 More or Less with Tim Harford
1.23.2009 | 7:02 mins 2MB
–REVIEW OF "THE CULT," by Aris Spanos, Virginia Tech, Erasmus Journal of Philosophy and Economics, 1:1
–"RESPONSE TO ARIS SPANOS," by authors Ziliak and McCloskey, Erasmus Journal of Philosophy and Economics, 1:1
Read: Review Kvams | 7/28/2009
Read: Article Machine Design | 3/16/2009
Read: Blog Post Conflicted | 2/28/2009