Banishing the notion of things that are “statistically significant” means that uncertainty is destined to become a more significant part of our lives.

Though in a very deep sense we are a civilization governed by statistics, most people show little interest in a mathematical discipline that many consider to be a dry, inhuman science. Statistics, as a professional activity, has made its way into the core of our scientific, economic, political and even philosophical culture. In the world of serious decision making — whether in the corporate world, sciences or political marketing — ideas and initiatives not supported by statistics tend to be dismissed in favor of those that are, even if no one (including statisticians) really understands how statistics can be expected to produce meaning.

The American Statistical Association (ASA) has come forward with an act of public humility that few people among the public will pay attention to. It has now admitted that over-reliance on statistics may be dangerous for our health. In an interview with the title, “Time to say goodbye to ‘statistically significant’ and embrace uncertainty, say statisticians,” we learn, for example, that “relying on statistical significance alone often results in weak science” and that, contrary to the illusion many have maintained about statistical evidence, “pure objectivity can never be achieved.”

In a world that is preparing to integrate artificial intelligence (AI) into every level of institutional decision-making, this could indicate a methodological breakthrough with far-reaching effects. AI both uses and produces statistics to make the decisions we so willingly accept to delegate to it. By acknowledging that uncertainty is more certain than supposed statistical truth, we may begin to situate our own decision-making responsibilities, based on factors other than numbers alone.

Here is today’s 3D definition:

Statistically significant:

Indicating that a certain representation of quantitative data may serve to justify ideas or initiatives that we fail to understand or, in some cases, refuse to understand   

Contextual note

At the center of statistical reasoning is something called “p-value,” a calculation embodied in a mathematical formula that has no intrinsic meaning — or “significance” — because it attributes a numerical value to the immaterial notion of comparative probability. The ASA’s intention of dislodging p-value from its place at the center of the world of statistics represents a Copernican change of perspective for statisticians. It has the potential to become the cause of an identity crisis in the field. But it also has significance for the world we live in, especially when considering the drama — if not trauma — building around the perspective of AI taking over so much critical decision-making.

Among the problems that have led to the replacement of the notion of “statistically significant” by the acceptance of uncertainty are practices that have been used and abused extensively in the pharmaceutical sector. The article cites “p-hacking (manipulating the data until statistical significance can be achieved)” and “perverse incentives especially in the academy that encourage ‘sexy’ headline-grabbing results.”

But the issue strikes even deeper into our civilizational values as statistician Nicole Lazar, the interviewee of the article, acknowledges this fact containing vast cultural significance: “Categorization and categorical thinking are the fundamental problems, not the p-value in and of itself.” When we apply mathematical reasoning to human problems, our dependence on both language and the pragmatics of human activity force us to call things and ideas by names we invent and to relate them to each other by grouping them in categories, or what psychologists call “cognitive boundaries.” This has never been truer than in the digital civilization that we now depend on. 

Historical note

Although the debate about the reliability of statistics and the dominant methodologies has been going on for decades, the article makes an important point: “The tone now is different, perhaps because of the more pervasive sense that what we’ve always done isn’t working.”

In the sciences, it can take decades or even centuries to notice “that what we’ve always done isn’t working.” During that time, we have a tendency to think that because it is the accepted science of the day, it can be assumed to be true and that its truth is established by the fact that everything we do with it works. Or seems to work — until we realize it doesn’t.

This is what Thomas Kuhn, tracing the history of science, called a “paradigm shift.” When Copernicus found flaws in the elaborate Ptolemaic description of the functioning of a universe in which the sun was deemed to revolve around the Earth, he could initiate a shift in the understanding of astronomy and the forces that govern our material existence.

Why might change in the status of what is “statistically significant” seem important today? The world should welcome any moment when experts and authorities engage in what Lazar describes as making “an attempt to start a deeper conversation about the best ways forward for science and statistics.” If science becomes more reliable, we all benefit. Accepting uncertainty seems to be as necessary today as it was for Copernicus, who had become uncertain about the otherwise reliable description of the movement of celestial bodies around what we all perceive to be the stable Earth.

But the consequences of this revolution in statistics may go further and help us to understand something about the paradigm shift we are currently experiencing, without necessarily realizing it. Democracy has become, in many people’s eyes and especially for political decision-makers, government by and through statistical significance. Elections have become essentially statistical competitions. And between elections, the media interpret “significance” through polling that produces statistics to predict statistics. Consequently, national elections have turned into the equivalent of global sporting contests, with the media claiming that each one will be the “fight of the century,” a fight that produces “moral significance” telling us how we will be governed. In the aftermath of those elections, the pundits analyze them as if the resulting statistics told us what we, the electors, were thinking, often representing it as a simple choice of a type of society.

That is where we are today, with the certainty that the British people chose to leave the European Union in 2016, whatever the consequences, or that the American people had decided to make America great again through blind nationalism. We trust the statistical results, at least for the duration between two elections. And we accept to live, work and socialize within the cognitive boundaries of the categories established by the parties and their marketers.

In this final week of March 2019, we might ask British Prime Minister Theresa May whether she, for one, is ready to “embrace uncertainty.”

*[In the age of Oscar Wilde and Mark Twain, another American wit, the journalist Ambrose Bierce, produced a series of satirical definitions of commonly used terms, throwing light on their hidden meanings in real discourse. Bierce eventually collected and published them as a book, The Devil’s Dictionary, in 1911. We have shamelessly appropriated his title in the interest of continuing his wholesome pedagogical effort to enlighten generations of readers of the news.]

The views expressed in this article are the author’s own and do not necessarily reflect Fair Observer’s editorial policy.