Books & arts

Does probability exist?

expedition to grab the eggs of emperor pen-
guins (Aptenodytes forsteri) in midwinter, first
detailed by Apsley Cherry-Garrard in The Worst
Journey In the World (1922). I’m very proud to
tell a new story about that well-worn old tale,
from my own physical research on site.
And then maybe I’ll tell a story about the
Probably not — but it is useful to act as if it does.
plan to slow the melting of glaciers in Antarc- By David Spiegelhalter

L
tica by drawing the water out from underneath
them. It’s now being investigated by a team
of glaciologists, as well as governance and ife is uncertain. None of us know what started corresponding in the 1650s that
finance people, to make sure that it doesn’t is going to happen. We know little of any rigorous analysis was made of ‘chance’
look like scientists coming in out of left field what has happened in the past, or is events. Like the release from a pent-up
and telling people how to save the world. happening now outside our immediate dam, probability has since flooded fields as
experience. Uncertainty has been called diverse as finance, astronomy and law — not
What do you think about AI? the ‘conscious awareness of ignorance’1 — be to mention gambling.
My feeling is that ‘artificial intelligence’ is a it of the weather tomorrow, the next Premier To get a handle on probability’s slipperiness,
public-relations name that obscures what’s League champions, the climate in 2100 or the consider how the concept is used in modern
really going on. It’s artificial for sure. But as identity of our ancient ancestors. weather forecasts. Meteorologists make pre-
for intelligence — the term is so broad that In daily life, we generally express uncertainty dictions of temperature, wind speed and quan-
you immediately get lost in it. So AI is a poor in words, saying an event “could”, “might” or tity of rain, and often also the probability of
name. If it was called ‘extremely rapid compu- “is likely to” happen (or have happened). But rain — say 70% for a given time and place. The
tation’, or ‘assisted data analysis’ or ‘cognitive uncertain words can be treacherous. When, first three can be compared with their ‘true’
prosthesis’ then that would de-emphasize the in 1961, the newly elected US president John values; you can go out and measure them. But
magical portions of it. You would be talking F. Kennedy was informed about a CIA-spon- there is no ‘true’ probability to compare the last
about what to do with it, not about making up sored plan to invade communist Cuba, he with the forecaster’s assessment. There is no
a human mind or a consciousness. commissioned an appraisal from his military ‘probability-ometer’. It either rains or it doesn’t.
There’s so much bad science fiction that top brass. They concluded that the mission What’s more, as emphasized by the philoso-
anthropomorphizes AI to the point at which had a 30% chance of success — that is, a 70% pher Ian Hacking2, probability is “Janus-faced”:
it has agency and malevolence. Machines are chance of failure. In the report that reached the it handles both chance and ignorance. Imagine
not going to get to consciousness using large president, this was rendered as “a fair chance”. I flip a coin, and ask you the probability that it
language models, which is simply optimiza- The Bay of Pigs invasion went ahead, and was will come up heads. You happily say “50–50”,
tion. And it’s easier to imitate human sentences a fiasco. There are now established scales for or “half”, or some other variant. I then flip the
than we thought it was because we’re predict- converting words of uncertainty into rough coin, take a quick peek, but cover it up, and
able. So, the Turing test turns out to be a rela- numbers. Anyone in the UK intelligence com- ask: what’s your probability it’s heads now?
tively low bar. All you have to do is fool human munity using the term ‘likely’, for example, Note that I say “your” probability, not “the”
beings, and we are very gullible. should mean a chance of between 55% and probability. Most people are now hesitant to
I had a grand time writing Aurora (2015), 75% (see go.nature.com/3vhu5zc). give an answer, before grudgingly repeating
about a journeying starship, written mainly Attempts to put numbers on chance and “50–50”. But the event has now happened, and
from the perspective of an AI. The AI, called uncertainty take us into the mathematical there is no randomness left — just your igno-
Ship, represents my thinking on how things realm of probability, which today is used rance. The situation has flipped from ‘aleatory’
might get interesting. It’s running a starship confidently in any number of fields. Open any uncertainty, about the future we cannot know,
and a human says to it: ‘keep a narrative account science journal, for example, and you’ll find to ‘epistemic’ uncertainty, about what we cur-
of the trip’. The computer doesn’t know what to papers liberally sprinkled with P values, con- rently do not know. Numerical probability is
do and has to figure it out. fidence intervals and possibly Bayesian poste- used for both these situations.
It still might not be consciousness, but Ship is rior distributions, all of which are dependent There is another lesson in here. Even if there
pretty eloquent by the end of the novel, pretty on probability. is a statistical model for what should happen,
self-aware. Pretty much like human conscious- And yet, any numerical probability, I will this is always based on subjective assump-
ness, with the starship as its body and its peo- argue — whether in a scientific paper, as part tions — in the case of a coin flip, that there are
ple like its gut microbiome. But we’re talking of weather forecasts, predicting the outcome two equally likely outcomes. To demonstrate
500 years from now with a quantum computer. of a sports competition or quantifying a health this to audiences, I sometimes use a two-
What could happen? Well, one doesn’t know. risk — is not an objective property of the world, headed coin, showing that even their initial
but a construction based on personal or collec- opinion of “50–50” was based on trusting me.
Do you have a tive judgements and (often doubtful) assump- This can be rash.
message for scientists? tions. Furthermore, in most circumstances, it
Scientists need to speak as a group. When all is not even estimating some underlying ‘true’ Subjectivity and science
the scientific institutions say, together, ‘we, quantity. Probability, indeed, can only rarely My argument is that any practical use of prob-
the scientific community, the ones who keep be said to ‘exist’ at all. ability involves subjective judgements. This
you alive, the ones who are your doctors and doesn’t mean that I can put any old numbers on
provide your food, say this has to be done’, Chance interloper my thoughts — I would be proved a poor proba-
that’s powerful. Probability was a relative latecomer to math- bility assessor if I claimed with 99.9% certainty
ematics. Although people had been gambling that I can fly off my roof, for example. The objec-
Interview by Anne Pichon. with astragali (knucklebones) and dice for tive world comes into play when probabilities,
This interview has been edited for length and millennia, it was not until the French mathe- and their underlying assumptions, are tested
clarity. maticians Blaise Pascal and Pierre de Fermat against reality (see ‘How ignorant am I?’); but

560 | Nature | Vol 636 | 19/26 December 2024

SÉBASTIEN THIBAULT

that doesn’t mean the probabilities themselves interval of 19–49%). The P value — the calcu- the signal is so strong that a model allowing,
are objective. lated probability of observing such an extreme say, the underlying risk to vary between par-
Some assumptions that people use to assess relative risk, assuming a null hypothesis of no ticipants will make little difference to the
probabilities will have stronger justifications underlying difference in risk — can be calcu- overall conclusions. If the results were more
than others. If I have examined a coin carefully lated to be 0.0001, or 0.01%. marginal, however, it would be appropriate to
before it is flipped, and it lands on a hard sur- This is all standard analysis. But the precise do extensive analysis of the model’s sensitivity
face and bounces chaotically, I will feel more confidence level and P value rely on more to alternative assumptions.
justified with my 50–50 judgement than if than just assuming the null hypothesis. It also To exercise the much-quoted aphorism,
some shady character pulls out a coin and depends on all of the assumptions in the sta- “all models are wrong, but some are useful”4.
gives it a few desultory turns. But these same The dexamethasone analysis was particularly
strictures apply anywhere that probabilities “Probability is ‘Janus-faced’: useful because its firm conclusion changed
are used — including in scientific contexts, in clinical practice and saved hundreds of thou-
which we might be more naturally convinced
it handles both chance and sands of lives. But the probabilities that the
of their supposed objectivity. ignorance.” conclusion was based on were not ‘true’ — they
Here’s an example of genuine scientific, and were a product of subjective, if reasonable,
public, importance. Soon after the start of the assumptions and judgements.
COVID-19 pandemic, the RECOVERY trials tistical model, such as the observations being
started to test therapies in people hospitalized independent: that there are no factors that Down the rabbit hole
with the disease in the United Kingdom. In one cause people treated more closely in space and But are these numbers, then, our subjective,
experiment, more than 6,000 people were ran- time to have more-similar outcomes. But there perhaps flawed estimates of some underlying
domly allocated to receive either the standard are many such factors, whether it’s the hospital ‘true’ probability, an objective feature of the
care given in the hospital they were in, or that in which people are being treated or changing world?
care plus a dose of dexamethasone, an inex- care regimes. The precise value also relies on I will add the caveat here that I am not talking
pensive steroid3. Among those on mechanical all of the participants in each group having about the quantum world. At the sub-atomic
ventilation, the age-adjusted daily mortality the same underlying probability of surviving level, the mathematics indicates that cause-
risk was 29% lower in the group allocated dex- 28 days. This will differ for all sorts of reasons. less events can happen with fixed probabil-
amethasone compared with the group that None of these false assumptions necessarily ities (although at least one interpretation
received only standard care (95% confidence mean that the analysis is flawed. In this case, states that even those probabilities express a

Nature | Vol 636 | 19/26 December 2024 | 561

Books & arts

How ignorant am I?
The need to evaluate the accuracy of the Questions Your confidence
probabilities we assign to things became 1. Which contains more water by that your answer is
clear when weather forecasters started percentage? correct (out of 10): 5 6 7 8 9 10
giving probabilities of precipitation. In 1951, (A) Human brain
meteorologist Glenn Brier developed the (B) Human blood Score if you
Brier score as a way to assess predictions12, are right: 0 9 16 21 24 25
and it can be adapted to see how good or 2. Who published their key work first?
bad you are at assessing your degree of (A) Charles Darwin Score if you
confidence about facts. (B) Gregor Mendel are wrong: 0 –11 –24 –39 –56 –75
A good Brier score depends on
a probability assessor both being 3. Which is the bigger planet? The scoring is deliberately harsh (it
discriminatory, so they give some confident (A) Venus uses the square of the prediction error).
judgements, but also calibrated, so that (B) Earth By punishing failure more than rewarding
of the situations in which they state success, honesty is encouraged.
‘70% probability’, they are right around 4. Which molecule has more atoms? If you ended up with a negative total,
70% of the time. This idea turns out to (A) Caffeine you did worse than a complete ignoramus
be fundamentally important when we (B) Aspirin who just answered 5 to every question.
consider the meaning of these subjective People with an exaggerated sense of their
judgements (see main text). 5. Which has the higher melting point? own knowledge tend to end up with large
For each of the following questions, decide (A) Gold negative scores. Those with an awareness of
which answer you feel is most likely to be (B) Silver their own doubts tend to mainly use 5s, 6s
correct, and then quantify your confidence or 7s, and might end up with a small positive
on a scale from 5 to 10. For example, if Now, check your judgements against the score. People who actually know a lot, or are
you are certain that answer (A) is correct, answers on the facing page (also do this extremely lucky, get higher scores. This type
you should give it 10/10, but if you are only quiz online at go.nature.com/3vakwqj). The of exercise is used to train forecasters to be
around 70% sure, then it gets 7/10. If you have following table shows how you should score less over-confident, and have insight into
no idea, then give 5/10 to either choice. yourself when the true answer is revealed. their own thought processes.

relationship with other objects or observers, This seems practically unverifiable. In every other situation in which probabili-
rather than being intrinsic properties of quan- There is a limited range of well-controlled, ties are used, however — from broad swathes
tum objects)5. But equally, it seems that this repeatable situations of such immense of science to sports, economics, weather, cli-
has negligible influence on everyday observ- complexity that, even if they are essentially mate, risk analysis, catastrophe models and
able events in the macroscopic world. deterministic, fit the frequentist paradigm so on — it does not make sense to think of
I can also avoid the centuries-old arguments by having a probability distribution with our judgements as being estimates of ‘true’
about whether the world, at a non-quantum predictable properties in the long run. These probabilities. These are just situations in
level, is essentially deterministic, and whether include standard randomizing devices, such which we can attempt to express our per-
we have free will to influence the course of as roulette wheels, shuffled cards, spun sonal or collective uncertainty in terms of
events. Whatever the answers, we would still coins, thrown dice and lottery balls, as well as probabilities, on the basis of our knowledge
need to define what an objective probability pseudo-random number generators, which and judgement.
actually is. rely on non-linear, chaotic algorithms to give
Many attempts have been made to do this Matters of judgement
over the years, but they all seem either flawed “Any practical use of This all just raises more questions. How do
or limited. These include frequentist proba- we define subjective probability? And why
bility, an approach that defines the theoret- probability involves are the laws of probability reasonable, if they
ical proportion of events that would be seen subjective judgements.” are based on stuff we essentially make up? This
in infinitely many repetitions of essentially has been discussed in the academic literature
identical situations — for example, repeating for almost a century, again with no universally
the same clinical trial in the same population numbers that pass tests of randomness. agreed outcome.
with the same conditions over and over again, In the natural world, we can throw in the One of the first attempts was made in 1926
like Groundhog Day. This seems rather unre- workings of large collections of gas molecules by the mathematician Frank Ramsey at the
alistic. The UK statistician Ronald Fisher sug- which, even if following Newtonian physics, University of Cambridge, UK. He ranks as the
gested thinking of a unique data set as a sample obey the laws of statistical mechanics; and person in history I would most like to meet.
from a hypothetical infinite population, but genetics, in which the huge complexity of He was a genius whose work in probability,
this seems to be more of a thought experi- chromosomal selection and recombina- mathematics and economics is still considered
ment than an objective reality. Or there’s the tion gives rise to stable rates of inheritance. fundamental. He worked only in the mornings,
semi-mystical idea of propensity, that there is It might be reasonable in these limited devoting his after-hours to a wife and a lover,
some true underlying tendency for a specific circumstances to assume a pseudo-objective playing tennis, drinking and enjoying exu-
event to occur in a particular context, such as probability — ‘the’ probability, rather than ‘a’ berant parties while laughing “like a hippo-
my having a heart attack in the next ten years. (subjective) probability. potamus” (he was a big man, weighing in at

562 | Nature | Vol 636 | 19/26 December 2024

 108 kilograms). He died in 1930 aged just 26,
probably, according to his biographer Cheryl
Misak, from contracting leptospirosis after
swimming in the River Cam6.
Ramsey showed7 that all the laws of proba-
bility could be derived from expressed pref-
erences for specific gambles. Outcomes have
assigned utilities, and the value of gambling
on something is summarized by its expected
utility, which itself is governed by subjective
numbers expressing partial belief — that is,
our personal probabilities. This interpreta-
tion does, however, require an extra specifica-
tion of these utility values. More recently, it’s
been shown8 that the laws of probability can
be derived simply by acting in such a way as to
maximize your expected performance when
using a proper scoring rule, such as the one
shown in the text box “How ignorant am I?”.
Attempts to define probability are often
rather ambiguous. In his 1941–2 paper ‘The
Applications of Probability to Cryptography’,
for example, Alan Turing uses the working
definition that “the probability of an event on
certain evidence is the proportion of cases in
MICHAEL OCHS ARCHIVES/GETTY

which that event may be expected to happen

given that evidence”9. This acknowledges that
practical probabilities will be based on expec-
tations — human judgements. But by “cases”,
does Turing mean instances of the same obser-
vation, or of the same judgements?
The latter has something in common with
frequentist definition of objective probability, John F. Kennedy approved a US invasion of Cuba on the basis of imprecise probabilities.
just with the class of repeated similar observa-
tions replaced by a class of repeated similar can instead take a pragmatic approach. Rather convictions, we should act as if events were
subjective judgements. In this view, if the prob- ironically, de Finetti himself provided the driven by objective chances.
ability of rain is judged to be 70%, this places it most persuasive argument for this approach It is extraordinary that such an important
in the set of occasions in which the forecaster in his 1931 work on ‘exchangeability’, which body of work, underlying all of statistical sci-
assigns a 70% probability. The event itself is resulted in a famous theorem that bears his ence and much other scientific and economic
expected to occur in 70% of such occasions. name11. A sequence of events is judged to be activity, has arisen from such an elusive idea.
This is probably my favourite definition. But exchangeable if our subjective probability for And so I will conclude with my own aphorism. In
the ambiguity of probability is starkly demon- each sequence is unaffected by the order of our everyday world, probability probably does
strated by the fact that, after nearly four cen- our observations. De Finetti brilliantly proved not exist — but it is often useful to act as if it does.
turies, there are many people who won’t agree that this assumption is mathematically equiv-
with me on that. alent to acting as if the events are independ- David Spiegelhalter is emeritus professor of
ent, each with some true underlying ‘chance’ statistics at the University of Cambridge, UK.
Pragmatic approach of occurring, and that our uncertainty about He is the author of The Art of Uncertainty: How
When I was a student in the 1970s, my mentor, that unknown chance is expressed by a subjec- to Navigate Chance, Ignorance, Risk and Luck
statistician Adrian Smith, was translating the tive, epistemic probability distribution. This (Penguin, 2024).
Italian actuary Bruno de Finetti’s Theory of is remarkable: it shows that, starting from a
Probability10. De Finetti had developed ideas of specific, but purely subjective, expression of 1. Smithson, M. Ignorance and Uncertainty: Emerging
Paradigms (Springer, 1989).
subjective probability at around the same time 2. Hacking, I. The Emergence of Probability (Cambridge
as Ramsey, but entirely independently. (They Answers to the quiz Univ. Press, 2006).
were very different characters: in contrast to 1. A (The brain is about 73% water, blood is 3. The RECOVERY Collaborative Group N. Engl. J. Med. 384,
693–704 (2021).
Ramsey’s staunch socialism, in his youth de about 50% water) 4. Box, J. E. P. J. Am. Stat. Assoc. 71, 791–799 (1976).
Finetti was an enthusiastic supporter of Italian 2. A (Darwin published On the Origin 5. Rovelli, C. Helgoland: The Strange and Beautiful Story of
dictator Benito Mussolini’s style of fascism, of Species in 1859, Mendel published Quantum Physics (Penguin, 2022).
6. Misak, C. Frank Ramsey: A Sheer Excess of Powers (Oxford
although he later changed his mind.) That Experiments on Plant Hybridization in 1866) Univ. Press, 2020).
book begins with the provocative statement: 3. B (Venus’s radius is 6,052 kilometres, 7. Ramsey, F. P. in The Foundations of Mathematics and other
“probability does not exist”, an idea that has Earth’s is 6,378 kilometres) Logical Essays 156–198 (Routledge, 1926).
8. Lindley, D. V. Int. Stat. Rev. 50, 1–11 (1982).
had a profound influence on my thinking over 4. A (Caffeine has 24 atoms, aspirin 21 9. Turing, A. M. UK National Archives record HW 25/37
the past 50 years. atoms) (1941–2).
In practice, however, we perhaps don’t have 5. A (Gold’s melting point is 1,064 °C, 10. de Finetti, B. Theory of Probability (Wiley, 1974).
11. de Finetti, B. Atti della R. Accad. Naz. Lincei 4, 251–299
to decide whether objective ‘chances’ really silver’s is 962 °C) (1931).
exist in the everyday non-quantum world. We 12. Brier, G. W. Mon. Weather Rev. 78, 1–3 (1950).

Nature | Vol 636 | 19/26 December 2024 | 563