The Law of Regression To The Tail How To PDF
The Law of Regression To The Tail How To PDF
3 1
The Law of Regression to the Tail: How to Mitigate Covid-19, Climate Change,
and Other Catastrophic Risks
By Bent Flyvbjerg, BT Professor and Chair of Major Programme Management, University of Oxford
Regression to the mean is nice and reliable, regression to the tail is reliably scary. We live in the age of regression to the
tail. It is only a matter of time until a pandemic worse than covid-19 will hit us, and climate more extreme than any we
have seen so far. What are the basic principles for navigating such risks, for government, business, and the public?
Galton illustrated his principle by the example that parents who are tall tend to have children who
grow up to be shorter than their parents, closer to the mean of the population, and vice versa for short
parents.
In another example, made famous by Nobel-Prize winner in economics Daniel Kahneman, pilots who
performed well on recent flights tended to perform less well on later flights, closer to the mean of
performance over many flights. This was not because the pilots’ skills had deteriorated, but because
their recent good performance was due not to an improvement of skills but to lucky combinations of
random events.
There is nothing as practical as a theory that is correct. Regression to the mean has been proven
mathematically for many types of statistics and is highly useful in health, insurance, to casinos, and in
risk management, e.g., for flight safety.
But regression to the mean presupposes that a population mean exists. For some random events of
great social consequence this is not the case.
Size-distributions of pandemics, floods, forest fires, earthquakes, wars, terrorist attacks, crimes, and IT
investments, e.g., have no population mean, or the mean is ill defined due to infinite variance. In other
words, mean and/or variance do not exist. Regression to the mean is a meaningless concept for such
distributions, whereas what one might call "regression to the tail" is meaningful and consequential.
Regression to the tail applies to any distribution with non-vanishing probability density towards
infinity. The frequency of new extremes and how much they exceed previous records is decisive for
how fat-tailed a distribution will be, e.g., whether it will have infinite variance and mean. Above a
certain frequency and size of extremes, the mean increases with more events measured, with the mean
eventually approaching infinity instead of converging. In this case, regression to the mean means
regression to infinity, i.e., a non-existent mean. Deep disasters – e.g., earthquakes, tsunamis,
pandemics, and wars – tend to follow this type of distribution.
I suggest we name this phenomenon – that events return to the tail in sufficient size and frequency for
the mean to not converge – "the law of regression to the tail." The law depicts a situation with many
extreme events, and no matter how extreme the most extreme event is, there will always be an event
even more extreme than this. It is only a matter of time until it appears.
Regression to the Tail, © Copyright Bent Flyvbjerg, draft 2.3 2
In any given situation, prudent decision makers and their risk managers must be able to decide
whether they face a situation with regression to the mean (mild Gaussian risk) or regression to the tail
(extreme fat-tailed risk), and – most importantly – never mistake the former for the latter. This is a
difficult task, because a host of cognitive and other biases – including simple wishful thinking – trick us
into seeing mild risk when risk is in effect wild.
To illustrate, consider the current covid-19 pandemic. Cirillo and Taleb (2020) have argued that
pandemics (measured by number of deaths) seem to follow a Generalized Pareto Distribution. The law
of regression to the tail is consequently pertinent, with three important implications.
First, the covid-19 pandemic was entirely predictable. Indeed, the pandemic was predicted years ago
by people as different as Nassim Nicholas Taleb, author of Incerto, philanthropist Bill Gates, and
numerous epidemiologists who are now, deservedly, having a field day as what-did-I-say prophets,
after being ignored for years by government, business, and media.
Second, it is clear to anyone who understands regression to the tail what the main mitigating measures
should be once a crisis develops, namely: (a) cut the tail (by breaking the chain of transmissions
through lockdowns, personal protection equipment, testing, development of vaccines, etc.) and (b) the
precautionary principle (rather a lockdown too many than one too few) – rolled out immediately, at speed,
and at scale, worldwide. The closing of wet markets and changes to the food processing industry will
help prevent crisis from developing in the first place.
Early mitigation and prevention pay back thousandfold when facing regression to the tail.
Unfortunately, China’s leadership delayed mitigation by trying at first to suppress information about
the virus. Then, once the data were released, leaders elsewhere – including in the US and the UK –
were slow to realize they faced extreme risks. Consequently, they were sluggish in making the right
decisions. Siloed government also slowed progress.
Third, contingencies must be in place to allow speedy scale-up. When leaders finally understood that
covid-19 was a fat-tailed phenomenon and began to make the proper decisions, it turned out that
health services, government, and businesses were dismally underprepared, to a degree that things as
basic as supplies of face masks, gowns, and other protective gear for health workers immediately ran
out. The lack of reserves made it impossible to scale effectively and fast – just like a bank without
reserves would fail in a crisis.
Due to failure on each of these three points, mitigation in many places was late, slow, and at insufficient
scale, that is, the opposite of what is needed when facing regression to the tail, with devastating
consequences in terms of lives lost, suffering, and wealth destruction.
First, everyone needs to be honest about, and keep in mind, that there will be more pandemics in the
future, and that one of these will be worse than the covid-19 pandemic. This uncomfortable fact follows
directly from the power-law distribution of pandemics and the associated law of regression to the tail.
Second, once leaders and citizens understand that pandemics involve regression to the tail, they will
also understand how to handle the next pandemic: by cutting its tail and employing the precautionary
principle immediately, at speed, and at scale, with the necessary contingencies in place.
The two lessons are general. They apply not only to pandemics, but to all phenomena that are subject
to the law of regression to the tail, for instance: floods, forest fires, earthquakes, tsunamis, snow
avalanches, crime, wars, terrorist attacks, blackouts, bankruptcies, and cybercrime, together with less
disastrous but financially highly risky ventures like hosting the Olympics, building nuclear power
plants, high-speed rail systems, hydroelectric dams, new cities, and even something as apparently
innocuous as procuring new IT systems, the latter being a serious bug in current worldwide
digitization efforts.
The massive stimulus spending programs that governments – including the USA and China – use to
restart depressed economies typically comprise giant construction and investment projects with fat-
tailed financial risks, like megaprojects in IT, transportation, energy, water, education, housing, health,
and defense.
Some projects are more fat tailed than others, i.e., they are more susceptible to the law of regression to
the tail. Data analytics should be used to separate fat-tailed projects from thin-tailed ones, and stick
with the latter whenever possible. The data and the analytics to do this exist and are ready for use.
For instance, nuclear power plants and large hydroelectric dams are bespoke, slow to build, and fat
tailed for financial risk; whereas wind farms and energy storage are modular, fast, and thin tailed. By
choosing wind over nuclear, the risk of regression to the tail will be significantly reduced, and climate
goals reached sooner. Elon Musk, CEO of Tesla, and Ørsted, a wind farm company, understand this.
Big Energy does not.
Every investment alternative must be assessed in this manner to ensure that stimulus spending becomes
a boost instead of a drag on the economy. The latter happens more often than we like to think, when
delays and cost overruns become so large that the net benefits of stimulus programs turn negative.
Table 1: Top 10 phenomena that are subject to the law of regression to the tail, ranked after fatness of tails. The higher
on the list, the fatter the tail, and the larger and more frequent regressions to the tail will be. All phenomena have infinite
variance. The table shows phenomena for which data were available.
Phenomenon Described in
3. Wars (number of battle deaths per capita of involved nations) Newman 2005
7. Bankruptcies (percent of firms per year per industry) Hong et al. 2007
These are the four basic principles for mitigating risk in the age of regression to the tail.
If we follow these principles proactively, regression to the tail will be mostly manageable. If we don't,
tail events will come back to haunt us, over and over, causing unnecessary carnage while we scramble
reactively to put in place measures that anyone who understands regression to the tail would agree
should have been ready long before we ended up in the tail, as with the covid-19 pandemic.
The law of regression to the tail further tells us what the focus for climate mitigation must be: (a)
identifying which mitigation measures are particularly scalable at blitz-like speeds and which are not,
and (b) accelerating and ramping up measures that are, while ruthlessly scrapping those that are not,
because we have no time to waste.
If we do this – i.e., if we truly understand the urgency of the law of regression to the tail for climate
change – we have a chance to survive this particular tail risk. If we don't, it will likely mean farewell to
the world as we know it, in a mass destruction of lives and wealth, making covid-19 seem like a picnic
in comparison.
References
Cirillo, Pasquale and Nassim Nicholas Taleb, 2020, "Tail Risk of Contagious Diseases," arXiv, 18
April, https://arxiv.org/abs/2004.08658.
Clauset, A., Shalizi, C. R. and Newman, M. E., 2009, "Power-Law Distributions in Empirical Data,"
SIAM review, 51(4), 661-703.
Flyvbjerg, Bent, Alexander Budzier, Dirk W. Bester, and Daniel Lunn, in progress, "Digitization
Disasters: Towards a Theory of IT Investment Risk."
Flyvbjerg, Bent, Alexander Budzier, and Daniel Lunn, in progress, "Regression to the Tail: Why the
Olympics Blow Up."
Hong, B. H., Lee, K. E. and Lee, J. W., 2007, "Power Law in Firms Bankruptcy," Physics Letters
A, 361(1-2), 6-8.
Maillart, T. and Sornette, D., 2010, "Heavy-Tailed Distribution of Cyber-Risks," The European Physical
Journal B, 75(3), 357-364.
Malamud, B. D. and Turcotte, D. L., 2006, "The Applicability of Power-Law Frequency Statistics to
Floods," Journal of Hydrology, 322(1-4), 168-180.
Newman, M. E., 2005, "Power Laws, Pareto Distributions and Zipf's Law," Contemporary Physics, 46(5),
323-351.