Criminal Stupidity of Statistical Science

Nassim Nicholas Taleb

Time to understand a few facts about small probabilities [criminal stupidity of statistical science]

The Japanese Nuclear Commission had the following goals set in 2003: " The mean value of acute fatality risk by radiation exposure resultant from an accident of a nuclear installation to individuals of the public, who live in the vicinity of the site boundary of the nuclear installation, should not exceed the probability of about 1x10^6 per year (that is , at least 1 per million years)".

That policy was designed only 8 years ago. Their one in a million-year accident occurred about 8 year later. We are clearly in the Fourth Quadrant there.

I spent the last two decades explaining (mostly to finance imbeciles, but also to anyone who would listen to me) why we should not talk about small probabilities in any domain. Science cannot deal with them. It is irresponsible to talk about small probabilities and make people rely on them, except for natural systems that have been standing for 3 billion years (not manmade ones for which the probabilities are derived theoretically, such as the nuclear field for which the effective track record is only 60 years).

1) Small probabilities tend to be incomputable; the smaller the probability, the less computable. (Forget the junk about "Knightian" uncertainty, all small probabilities are incomputable). (See TBS, 2nd Ed., or Douady and Taleb, Statistical undecidability, 2011.)

2) Model error causes the underestimation of small probabilities & their contribution (on balance, because of convexity effects). Any model error, just as any undertainty about flying time causes the expected arrival to be delayed (you rarely land 4 hours early, more often 4 hours late on a transatlantic flight, so "unforeseen" disturbances tend to delay you). See my argument about second order effects with my paper. [INTUITION: uncertainty about the model used for calculation of random effects causes a second layer of randomness, causing small probabilities to rise on balance].

3) The problem is more acute in Extremistan, particularly the manmade part. The probabilities are undestimated but the consequences are much, much more underestimated.

4) As I wrote, because of globalization, the costs of natural catastrophes are increasing in a nonlinear way.

5) Casanova problem (survivorship bias in probability): If you compute the frequency of a rare event and your survival depends on such event not taking place (such as nuclear events), then you underestimated that probability. 

6) Semi-technical Examples: to illustrates the point (how models are Procrustean beds):

Case 1: Binomial

Take for example the binomial distribution with B[N, p] probability of success (avoidance of failure), with N=50. When p moves from 96% to 99% the probability quadruples. So small imprecision around the probability of success (error in its computation, uncertainty about how we computed the probability) leads to enormous ranges in the total result. This shows that there is no such thing as "measurable risk" in the tails, no matter what model we use.

Case 2: More scary. Take a Gaussian, with the probability of exceeding a certain number, that is, . 1- Cumulative density function.. Assume mean = 0, STD= 1. Change the STD from 1 to 1.1 (underestimation of 10% of the variance). For the famed "six sigmas", the area in the tails explodes by 2400%. For the areas above 10 sigmas (common in economics), the area explodes by trillions.