iScience ll

OPEN ACCESS

Review
Pain, from perception
to action: A computational perspective
Zhe Sage Chen1,2,3,4,* and Jing Wang2,3,4,5,*

SUMMARY
Pain is driven by sensation and emotion, and in turn, it motivates decisions and
actions. To fully appreciate the multidimensional nature of pain, we formulate
the study of pain within a closed-loop framework of sensory-motor prediction.
In this closed-loop cycle, prediction plays an important role, as the interaction be-
tween prediction and actual sensory experience shapes pain perception and sub-
sequently, action. In this Perspective, we describe the roles of two prominent
computational theories—Bayesian inference and reinforcement learning—in
modeling adaptive pain behaviors. We show that prediction serves as a common
theme between these two theories, and that each of these theories can explain
unique aspects of the pain perception-action cycle. We discuss how these compu-
tational theories and models can improve our mechanistic understandings of
pain-centered processes such as anticipation, attention, placebo hypoalgesia,
and pain chronification.

INTRODUCTION
Pain is complex and multidimensional. It includes the sensory-discriminative, affective-emotional, and
cognitive-evaluative components and is the result of dynamic interactions of multiple central and periph-
eral neural processes.1,2 Acute pain serves to protect us from predictable harm. However, when acute pain
persists into a chronic phase, it contributes to the dual public health crises of under-treatment and opioid
overuse and addiction. A detailed understanding of pain mechanisms is necessary to address these health-
1Department of Psychiatry,
care issues.
New York University
Grossman School of
In the brain, there is no ‘‘pain cortex’’. Instead, a distributed network of cortical-subcortical areas contrib- Medicine, New York, NY
utes to pain processing.3–5 During pain processing, sensory-motor integration occurs, and this integration 10016, USA
2Department of
further interacts with memory to guide decision-making. Pain experiences are subjective and highly
Neuroscience and
variable across subjects, and are dependent on contextual, social, and cultural factors.6 Furthermore, Physiology, New York
pain encompasses both hedonic and motivational aspects of an unpleasant experience.7 Pain serves as University Grossman School
of Medicine, New York, NY
a ‘‘teaching signal’’ to avoid or minimize predictable harm and to modulate physiological (e.g., homeostat-
10016, USA
ic, autonomic, endocrine, and immune) responses that further affect motivated behaviors (e.g., exploration 3NeuroscienceInstitute, NYU
and social interaction). To fully understand these complex factors, it is imperative to develop a principled Grossman School of
computational framework and model individual components. Medicine, New York, NY
10016, USA
4Interdisciplinary
Pain
The mammalian pain system consists of ascending and descending pathways, including the peripheral
Research Program, NYU
nerves, spinal cord, and cerebral cortex. To date, multimodal neuroimaging and multisite electrophysi- Langone Health, New York,
ology techniques have been used to uncover pain mechanisms of peripheral and central nervous sys- NY 10016, USA
tems.8–11 In contrast to a wealth of experimental data, however, there is a relative lack of computational 5Department of
Anesthesiology,
theories and modeling in pain research.12–14 Several reviews on pain have focused on nociception and Perioperative Care and Pain
negative emotion,15,16 expectation,17 motivational decision and action,18 as well as reinforcement and con- Medicine, New York
trol.13,19 Predictive coding and Bayesian models for perception have been widely presented in the litera- University Grossman School
of Medicine, New York, NY
ture,20–23 but only recently been adopted in pain research.12,24–32 Gradually, a more general computational 10016, USA
framework is also beginning to emerge to accommodate the exponential growth of data in pain research. *Correspondence:
[email protected]
In this Perspective, we systematically appraise the role of computational theories and models for studying a (Z.S.C.),
[email protected]
wide range of pain-associated processes, with a goal to present a coherent theme that establishes the (J.W.)
computational foundation for both theorists and experimentalists in pain research. Specifically, we will https://doi.org/10.1016/j.isci.
focus on theories of predictive coding and reinforcement learning (RL), both of which share a common 2022.105707

iScience 26, 105707, January 20, 2023 ª 2022 The Author(s). 1

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
 ll iScience
OPEN ACCESS Review

machinery of ‘‘prediction’’ and ‘‘prediction error’’. Predictive coding and Bayesian theory offer a predictive
inference framework that can be used to understand pain perception across time. Meanwhile, RL provides
a mechanistic framework to understanding how the brain supports pain-related actions and decisions.13
Together, these two prediction-driven theories form a computational foundation for the ‘‘pain percep-
tion-action cycle’’ and goal-oriented decision-making.

MOTIVATION FOR NEW COMPUTATIONAL APPROACHES IN PAIN RESEARCH

Good theories can not only deepen understanding of biological processes but also generate new predic-
tions to validate current and future experiments.33 An experimental design may be complex, and data
interpretation often requires making some basic assumptions and inferences. On the other hand, modeling
also requires information extraction and abstraction (e.g., an internal world model) to identify the common
computational principle or mechanism (Figure 1A). There are several reasons to call for new theories and
models to study pain. First, computational approaches can complement and overcome some challenges in
the current pain experiments, including the lack of quantifiable readout in complex pain behaviors and the
tendency to emphasize sensation over action in pain experiences. Furthermore, theory can potentially
harmonize the findings across species and modalities, and fill in the knowledge gap in both circuit and
behavioral-level understandings. Theories or models may also help constrain experimental variables and
guide experimental designs.

A second reason for studies on theoretical modeling is that a number of older theories on pain, such as the
intensity theory, specificity theory, pattern theory, and gate control theory (see ref. 36,37), have been
outdated. For instance, the intensity theory believes that pain results from excessive stimulation of the
sense of touch, and defining pain as an emotion that occurs when the stimulus intensity and central sum-
mation are greater than the threshold. The specificity theory of pain argues that the summation of pain re-
ceptors produces nerve impulses which are transmitted to a pain-specific system in the brain, and views
nociception independently of other sensory pathways. The pattern theory of pain hypothesizes that spatio-
temporal pattern of impulses from the peripheral nerves encodes the pain sensation and intensity, and
these patterns only occur with intense stimulation. The gate control theory proposes that non-nociceptive
neurons ascending to the dorsal horn of the spinal cord activate local inhibitory neurons which in turn in-
hibits smaller-diameter nociceptive pain fibers, thereby halting the transmission of sensory information to
the brain.38 However, all of these older theories do not sufficiently account for the integration of active
sensing, cognition, and decision-making, and may thus underestimate the importance of dynamic changes
in nociceptive/sensory/motor variables in real-world environments. In other words, pain is not only a multi-
sensory experience but also involves at least partially conscious choices and active behavioral responses. A
complete understanding of pain requires models that can integrate sensory experience with cognitive
choices and motor actions.

Finally, learning and adaptive behaviors are essential to every aspect of pain processing, with examples
ranging from pain anticipation, perception, avoidance, recuperation, and chronification. All these adaptive
behaviors are driven by learning.13,19 Pain is a teaching signal. Throughout evolution, animals develop the
ability to adapt their bodily or physiological responses to detect and avoid predictable injury and pain.
Hence, a new computational framework is urgently needed to incorporate such learning paradigms into
the understanding of pain.

COMPUTATIONAL FRAMEWORK FOR STUDYING THE PAIN PERCEPTION-ACTION CYCLE

The perception-action cycle is found in all goal-directed behaviors. 39 Within this cycle, the brain needs to
execute a series of cognitive functions including sensory assessment, attention, memory, and sensorimotor
integration. This perception-action cycle can be used as a conceptual model for pain processing.40 A pain
perception-action cycle has several key components: sensory-motor integration, motivation, and learning.
The principle of prediction comprises many aspects in the pain perception-action cycle: predictive coding
can be used to explain pain perception and expectation, whereas RL provides conceptual understanding
for motivation-driven action. Indeed, prediction is fundamental to numerous pain behaviors including
pain anticipation, pain avoidance, fear, pain catastrophizing, chronic pain aversion, and placebo anal-
gesia.16,41–43 A mismatch between prediction and sensory inputs generates a surprise signal representing
prediction error (PE), which further drives aversive learning associated with nociception, resulting in both
adaptive and maladaptive action. In this section, we will review several computational theories that are
essential to prediction and learning for understanding the pain perception-action cycle (Table 1).

2 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

A B C

D E

Figure 1. Examples of computational theories and models

(A) Levels of abstraction and update of theory and model via error feedback.
(B) Schematic diagram of predictive coding for pain perception.
(C) The perception-action cycle is intertwined with internal prediction loops, where different stages of prediction can be applied in pain-related processing.
(D) Schematic diagram of reinforcement learning (RL). The agent receives nociceptive input xk(t) at the k-th trial, updates the value function, selects the
action, receives the reward (or punishment), produces a prediction error (PE), which is further used to update the weights and value function.
(E) Schematic diagram of the computational model with arbitration between model-based and model-free learning systems (Wang et al., 2018, ref. 34, figure
modified with permission, CCBY 4.0 license).
(F) The predicted response outcome (PRO) model (Alexander and Brown, 2011, ref. 35, figure reproduced with permission, Springer Nature). (i) The model
learns predictions of future outcomes (e.g., error or correct feedback) based on task-related cues (S) signaling the onset of a trial is presented. Over the
course of a task, the model learns a timed prediction (V) of possible responses and outcomes (r). The temporal difference learning signal (d) is decomposed
into its positive and negative components (uP and uN, respectively), indicating unpredicted occurrences and unpredicted non-occurrences, respectively. (ii)
uN accounts for typical effects observed in the PFC from human imaging studies. Conflict and error likelihood panels show activity magnitude aligned on trial
onset; error and error unexpectedness panels show activity magnitude aligned on feedback. Model activity (vertical axis) is in arbitrary units. HEL, high error
likelihood; LEL, low error likelihood. (iii) Typical time courses for components of the PRO model.

Our main objective of this Perspective article is to educate pain researchers on computational approaches
for pain studies. Before reviewing detailed pain applications of computational models, we hope to present
a high-level description of theories and models to familiarize pain researchers with basic mathematical
backgrounds and necessary technical jargons. Many models are conceptually connected, while hybrid
models can also be employed in pain applications. To link these theories with the various pain applications,
we aim to provide multiple lines of evidence and neural correlates of PEs in pain circuits by presenting
model discussions.

iScience 26, 105707, January 20, 2023 3

 ll iScience
OPEN ACCESS Review

Table 1. Theories and models for understanding the pain perception-action cycle

Theories and Models Computation Pain applications

Predictive coding Prediction of sensory Pain perception,
or motor signals sensorimotor integration
Bayesian theory Prediction of posterior Pain perception,
and posterior mean placebo analgesia
Reinforcement learning Prediction of value function Decision-making, planning,
avoidance, pain conditioning,
pain chronification
Avoidance/escape Prediction of value function, Pain anticipation/avoidance,
learning integration of multi-factors pain-related fear conditioning
Associative learning Modulation of ‘‘associability’’ Attention modulation for
strength pain, fear-conditioned
pain modulation, pain extinction
Predicted Action selection and Pain-associated decision-making
response-outcome prediction
(PRO) model
Motivation-Decision Weighting the utility Decision-making in cost-benefit
model function conflicting or competitive choices

Predictive coding and Bayesian theory: Prediction for sensory perception and motor
response to pain
To address uncertainties (i.e., ‘‘what’’, ‘‘when’’, and ‘‘where’’) in a dynamic environment, the brain uses prior
information to guide both sensory perception and motor output.44–46 A useful paradigm to describe this
process is predictive coding. In this paradigm, sensing and action describe the inversion of an internal
generative percept model, which continuously adapts to additional sensory perceptions and/or motor
preparation. Such predictive coding paradigms have provided insight into perceptual inference, motor
control, and multisensory integration.21,23,47,48 In this view, sensing is active and predictive, and attention
can be used to bias the perceptual selection of multiple sensory inputs.

In predictive coding, the bottom-up (sensory, nociceptive, and proprioceptive) input and top-down
(expectation and attention) output are integrated to compute a PE, which is further used as a form of feed-
back to update the internal model for subsequent prediction (Figure 1B).

Prediction = Input + gain 3 PE

The PE represents a surprise factor, and the uncertainty of this surprise factor can also be characterized by
its variance or the inverse of gain in a system. A small PE and/or large variance would call for a small correc-
tion; in contrast, a large PE and small variance would call for a big update. Predictive coding can success-
fully explain both perception and action, and thus the perception-action cycle consists of two intertwining
internal prediction loops (Figure 1C).

Bayesian theory further generalizes predictive coding with a feature of probabilistic inference. In the case
of nociception, bottom-up sensory inputs from the periphery produce a likelihood for perceptive and
behavioral response, whereas neurons in the brain which undergo plasticity as the results of prior expe-
riences can also provide top-down modulatory outputs to further shape these responses. Take pain
perception (including placebo hypoalgesia) as an example, how do we ultimately perceive pain (‘‘pain
belief’’) does not only depend on how the nociceptive input, sensory input (such as vision), and propri-
oceptive input signaling alarms (‘‘likelihood of injury or harm’’) but also on our attention, expectation,
context, and environment (‘‘prior’’). Through evolution and experiences, our brains have built empirical
internal predictive mapping models for the likelihood of an event to occur. Such directional neural circuit
response can be characterized by a generative predictive model (denoted by a probability distribution
P(observed | model parameter)), where Bayesian inference can be used to compute the most likely
causes of observed neural response (i.e., posterior) by optimally integrating prior experience and
observed nociceptive inputs.

4 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

Box 1. Bayesian inference

Let x denote the nociceptive input, and z denote the pain percept. Given the prior expectation pðzÞ and the likelihood
pðxjzÞ, the goal of Bayesian inference is to estimate the posterior

pðzÞpðxjzÞ
pðz j xÞ = fpðzÞpðx j zÞðexact BayesianÞ
pðxÞ
In the presence of a latent variable h, the posterior estimation needs to integrate over the distribution pðhÞ. For
computational simplicity, pðhÞ can be approximated by a Gaussian distribution Nðhjmh ; sh Þ with mean mh and SDsh
Z Z
 
pðzjxÞ f pðzÞpðxjz; hÞpðhÞdhz pðzÞpðxjz; hÞNðhjmh ; sh Þdh: Gaussian approximation

Alternatively, one can compute the lower bound of the marginal log likelihood log pðxÞ by introducing a variational
posterior distribution qðz; hÞ, such that
Z Z
log pðxÞ = log dh dzpðhÞpðz; xjhÞ

Z Z
pðhÞpðz; xjhÞ
= log dh dzqðz; hÞ
qðz; hÞ
Z Z
pðhÞpðz; xjhÞ  
R dh dzqðz; hÞlog hF qðz; hÞ ðfree energyÞ
qðz; hÞ
where the free energy F ðqðz; hÞÞ approximates the lower bound of log pðxÞ.

Let’s consider a simple example with the prior pðxÞ  Nðm1 ; s1 Þ and likelihood pðxjzÞ  Nðm2 ; s2 Þ; then the posterior
s2 s2 s2 s2
pðzjxÞ is also Gaussian, with the new variance s2 1+ s2 2 and new mean s2 +2 s2 m1 + s2 +1 s2 m2 . Let x1 = 1
s21
and x2 = 1
s22
denote the
1 2 1 2 1 2

precision (inverse variance) parameters; then the new precision is x = x1 + x2 , and the new posterior mean estimate is
weighted by their respective precision parameters: m = xx1 m1 + xx2 m2 . Alternatively, the posterior mean can be rear-
ranged as an incremental update form from the prior: m = m1 + xx2 ðm2  m1 Þ, where the relative precision x2
x can be
viewed as a learning rate, and ðm2  m1 Þ can be viewed as a prediction error (PE).

PosteriorfPrior 3 Likelihood
Z
fPrior 3 Pðobserved j latentÞ 3 PðlatentÞ dðlatentÞ

From which the pain belief can be inferred from the statistics of posterior distribution (e.g., mean and vari-
ance). Importantly, this often forms an inferential loop as the effect of such information may further drive
top-down modulation. In this inference model, it is common to introduce a latent variable, which can be
either a continuous or discrete random variable, to account for unobserved neural processes (e.g., atten-
tion and modulatory input). In the above pain perception example, this is equivalent to averaging out all
possibilities of unknowns given their respective priors in the inference. When all the unknowns are equally
possible (i.e., noninformative priors), then the posterior computation becomes simplified. In a general
case, however, exact inference of the full posterior distribution is computationally intractable (for either
computer or brain) and often relies on certain variational approximations, producing different classes of
approximate Bayesian inference methods (Box 1). A special case of predictive coding is the Kalman filter,
which is an optimal Bayesian estimator for the linear Gaussian system.

An effort that combines predictive coding and Bayesian theory is Bayesian predictive coding,22 where the
predicted sensory input is equivalent to the mathematical expectation with respect to the posterior of the
input
Z
   
Prediction = Input 3 Posterior input d input

ZZ
      
f Input 3 P input 3 P observed  latent; input 3 PðlatentÞ d input dðlatentÞ

iScience 26, 105707, January 20, 2023 5

 ll iScience
OPEN ACCESS Review

Box 2. Temporal difference reinforcement learning

One of the most popular RL models is the temporal difference RL (TDRL) model.57 In the simplest form of TDRL model,
the value function V ðsÞ is associated with a state s, and represents the expected value of the temporally discounted
sum of all future rewards:
 
V ðtÞ = E g0 rðtÞ + g1 rðt + 1Þ + g2 rðt + 2Þ + /
where 0 < g % 1 is a discount factor for the reward. At the k-th step (or trial) of the perception-action cycle, the TDRL
model is used to update the value function

V ðsk + 1 Þ ) V ðsk Þ + a½rk + 1 + gV ðsk + 1 Þ  V ðsk Þ

where r denotes the (positive, zero or negative) reward, and 0 < a < 1 is a small learning rate parameter. When g = 0,
the TDRL rule is reduced to the Rescorla-Wagner rule. The error term dk hrk + 1 + gVðsk + 1 Þ  V ðsk Þ, derived from the
difference in predictions over successive time steps, is called the reward PE or temporal difference (TD) error. This error
is known to be represented in the midbrain dopamine (DA) neurons, and has been implicated in other cortical
processing.

In a TDRL model, learning may occur continuously from moment to moment and is not artificially constrained to
specific trial numbers. Algorithmically, the temporal credit assignment problem can be resolved efficiently by the
eligibility traces (Figure 1D). In a generalized form of TD(l) model, the value function is represented by a linear func-
tional approximation between the feature x and weight w: V ðtÞ = w$xðtÞ, the weights are updated by the following
TDRL rule:

DwðtÞ = adðtÞeðtÞ; eðtÞ = leðt  1Þ + xðtÞ

where 0 < l < 1 is a decay parameter that determines the plasticity window; a recently active stimulus will have high
corresponding eligibility traces, and accommodate for a larger update. When l = 0, the standard TD(0) learning is
recovered.

The computation can be understood through a hierarchical inference process.

How do predictive coding and Bayesian theory help our understanding of pain perception? To date, these
two theories have been advocated to account for behavioral and neural data in pain processing.12,27,49–53 In
the context of inference for pain, PE and prediction may be computed in different brain regions. A detailed
discussion on neural correlates of these PE signals will be presented in the next section. Additionally, com-
munications of PE and prediction signaling can be manifested in gamma and alpha/beta oscillations in the
bottom-up and top-down pathways, respectively; one possibility is within the S1-ACC circuits.54 The PE is
computed in the higher frequency (such as gamma) oscillations, whereas the prediction is updated in the
lower frequency (such as alpha and beta) oscillations, as shown in experimental findings in many pain
studies.27,55,56

Reinforcement learning: Prediction of the pain-aversive value

Motivation forms another key component of the pain perception-action loop. RL is an important tool
to understand motivated actions and decision-making.57 Through experience-based learning, a biolog-
ical agent receives an input from the environment and seeks an action in a certain state to maximize
the reward and pleasure (such as pain relief) or to minimize the cost. The problem of determining
the action that leads to a goal-directed outcome can be formulated as a temporal credit assignment
problem. For instance, if a runner has a long-term objective to run a marathon, then a series of planning
and action will be made to reach the goal, which ultimately results in a large emotional reward.
The same runner will experience physical pain during training; yet the runner is motivated to endure
the pain.

Through learning the associations between stimuli (‘‘state’’), actions, and the occurrence of pleasant
or unpleasant events, RL is capable of predicting future rewards or punishments.58 An RL algorithm
is model-free if it only replies on real samples from the environment and never uses generated
predictions of next state and next reward to alter behavior. One of the most popular model-free RL
models is known as temporal difference RL (TDRL) (Box 2 and Figure 1D). The basic idea of the RL
model is to formulate a value function and use it to compute the reward PE to guide action

6 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

selection; the value function is further updated from the reward PE as explained in the following
equation.

Predicted value function = old value function þ gain x reward PE

In this case, the reward PE can be used to increase the propensity to perform actions resulting in higher-
than-expected rewards, as well as to update the value function and future predictions. Another model-
free RL model is the actor-critic model, which generalizes the TDRL to allow policy iteration, which
simultaneously allows action learning (actor) and value learning (critic).57 In short, the actor-critic model
separates PE from the action selection process: the critic learns to calculate PE, and uses it to predict
the value of the current environmental state, and the actor uses the PE to learn to select between
competing possible actions. The dorsolateral and ventral striatum have been implicated for the actor
and critic roles, respectively.59 RL has been successfully adopted in a series of human experiments in the
context of pain.60,61 Specifically, the ventral striatum and the anterior insula encode PE signals predicted
by RL models in a fMRI study of higher order aversive conditioning, where humans were instructed to learn
a computational strategy to predict pain.60 In contrast, the amygdala and midbrain tended to reflect
reward-like signals predicted by RL models, where humans learned to generate expectations of pain re-
lief.61 Detailed discussions of neural correlates of PE signals in distributed pain circuits will be presented
in the next section.

Although reward and pain represent pleasant and unpleasant experiences, respectively, they can both
trigger motivated behaviors through RL models. The nature of reinforcement can be either positive or
negative. Positive reinforcement works by introducing a reward to increase the likelihood of a behavior,
whereas negative reinforcement learning involves the removal of an unpleasant stimulus (e.g., pain
relief).7,62 In the midbrain mesolimbic dopamine system, dopaminergic (DA) signaling was found in
the presence of not only appetitive stimuli (‘‘reward’’) but also aversive stimuli (‘‘pain’’),63–66 where
salience-coding DA signaling is similar for pain and reward. On the other hand, DA signaling can repre-
sent hedonic value or the sign of the learning signal in reward or pain relief.62,67 However, reward and
pain involve distinct circuit pathways: aversive DA signaling involves ventral tegmental area (VTA)/
cortical projections, whereas reward DA signaling involves VTA/ventral striatal or VTA/nucleus ac-
cumbens (NAc) projections.68 These two systems are dissociable and yet also interconnected.7,69 Addi-
tionally, DA signaling has been found to be important for motivational avoidance behavior; neuroimag-
ing has suggested that dopamine dysfunction plays a role in chronic pain.70–73 Chronic pain decreases
the activity of VTA DA neurons, reduces motivation for natural rewards, and drives anhedonia-like
behavior.74 The RL theory may thus provide a useful framework to model DA-related chronic pain and
addiction behaviors.13,75

How can RL theories help explain strategies of value learning and action learning in the context of pain?
Pain presents as a strong motivational drive to impact decision-making, and there is also a neurobiological
overlap between decision-making and pain.18,76–78 The adaptive pain behaviors in animals and humans
may have originated evolutionarily to promote survival by minimizing harm, and may further involve higher
order cognitive processes and planning. Therefore, the decision on how to plan an action to escape from or
endure potential harm in order to maximize future reward can be formulated as a temporal credit assign-
ment problem.13 A number of top-down pathways have been implicated in the control or modulation of
pain-related decision-making processes. These include the projections from the prefrontal and midcingu-
late areas to mid- and hindbrain structures such as the periaqueductal gray (PAG) and NAc,41,79 as well as
the projections to the insula cortex and thalamus.80–82

Associative learning: Modulation of PE and association for pain

The ability to learn associations between cues and pain allows the prediction and avoidance of experiences
of pain and injury.83 Operant conditioning (or instrumental conditioning) is a type of associative learning
through which the strength of a behavior is modified by reinforcement or punishment. Similar to RL, asso-
ciative learning uses PE to guide learning; but it also applies a modulating factor (such as attention) to
directly or indirectly change the association between a conditioned stimulus (CS) and an unconditioned
stimulus (US).84 The Rescorla-Wagner model is a special class of associative learning model that uses
the signed PE to determine the effectiveness of USs.85 Another class of associative learning model employs
the unsigned PE to determine the eligibility of CSs.86–88 PE signaling has been found in the amygdala in the
context of fear.89,90

iScience 26, 105707, January 20, 2023 7

 ll iScience
OPEN ACCESS Review

How does associative learning relate to pain? Pain is a primary reinforcer in associative learning in animal
models of fear conditioning.18 During a Pavlovian fear-conditioning paradigm, the expected shock prob-
ability of a noxious stimulus and associability (‘‘uncertainty’’) were estimated from an associative learning
model.91 Human imaging studies have also found a role of associative learning in pain expectation as
well as in placebo responses.42,92,93 Recently, Zhang and colleagues proposed an associative learning
model for tonic pain relief in an escape-learning paradigm. In this study, they found that the uncertainty
(‘‘associability’’) signal in the associative learning model was encoded in the pregenual anterior cingulate
cortex that controls the level of tonic pain aversion, and the error signals in the dorsal putamen explained
active seeking of pain relief.94 Furthermore, another human neuroimaging study has shown that associative
learning signals—the learned associability and PE—are correlated with fMRI responses in amygdala-stria-
tal regions, corresponding to the fear learning circuit.95

Avoidance learning and escape learning: Responses to aversive pain stimuli

Adaptive pain behaviors can be explained by various instrumental learning theories driven by reward or
punishment. There is a critical difference between reward- and punishment-based learning because of
the distinct nature and asymmetry of how the brain process appetitive and aversive outcomes. Specifically,
avoidance (or escape) learning is referred to the scenario in which animals or humans can learn a response
to avoid (or terminate) experiencing an aversive and painful stimulus. For instance, in avoidance learning, a
mouse can avoid the electric shock by moving to the opposite chamber after the CS is presented; in escape
learning, a mouse responds to the US instead of the CS by escaping to the opposite chamber. In avoidance
learning, the action can be active or passive (i.e., inaction, as action in the presence of warning stimuli has
previously led to pain).

How do we develop computational models for avoidance learning behaviors in response to aversive pain
stimuli? Avoidance learning can be formulated as a temporal difference model similar to RL,96 and can be
used to model pain-avoidance conditioning of a non-noxious stimulus.97,98 Recently, a hybrid model-based
and model-free avoidance learning paradigm was proposed for a two-step aversive decision-making
task,99 in which the avoidance behaviors are controlled by two different systems: a ‘‘model-free’’ habit sys-
tem and a ‘‘model-based’’ cognitive system. The model-free system bears a form of a specialized RL model,
whereas the model-based system learns a forward internal model of state transition; the reliabilities of the
two models are evaluated from their respective PEs, and the ultimate value function consists of a weighted
sum of the two value functions (VMF and VMB) of two systems (Figure 1E). Additionally, Jepma and col-
leagues used model-free RL to fit the behavioral data of participants performing an instrumental pain-
avoidance learning task; they also found that pain PEs were encoded in subcortical and limbic brain re-
gions, whereas no-pain PEs were encoded in frontal and parietal cortical regions.34

What is the neural basis of avoidance learning? The neural substrates of avoidance learning are composed
by a WHEN system (limbic circuits, including the amygdala, hippocampus, cingulate cortex, and limbic
thalamus) and a WHAT system (motor circuits). Pain and fear are two major factors that influence motiva-
tional behaviors: pain motivates recuperative behaviors that promote learning, whereas fear activates
defensive behaviors to guard against dangers. These two motivational systems are thought to serve
distinct and competitive functions, and as such, a perceptual-defensive-recuperative model has been
developed to account for the relationship between fear and pain.100 The amygdala and striatum are two
subcortical circuits primarily involved in avoidance learning18,101: the amygdala is responsible for learning
about aversive outcomes, and the striatum is implicated in reward-related reinforcement of actions to
avoid negative outcomes. The brainstem/amygdala and anterior insula/amygdala pathways play an
important role in avoidance of punishment.102 Recent data have also shown that pharmacological manip-
ulations had no effect on the neural encoding PEs, suggesting that pain-avoidance learning is supported by
separate threat- and safety-learning system, and that dopamine and endogenous opioids regulate
learning from successfully avoided pain.34

Prediction models for action selection and decision-making in conflicts for pain
Another aspect of the pain perception-action cycle is action. Therefore, it is equally important to develop
computational models to understand the process of action and its motivation. The anterior cingulate cor-
tex (ACC) is a central hub for pain sensation and action that connects with the somatosensory cortex, pa-
rietal cortex, and supplemental motor area (dorsal pathway) as well as the insula, amygdala, and NAc
(ventral pathway). The ACC has been implicated in monitoring conflicts and errors, and using the error

8 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

feedback to guide actions in aversive scenarios.103 Specifically, an RL-motivated computational model

known as the predicted response-outcome (PRO) model has been proposed for the ACC, as part of the
medial prefrontal cortex (mPFC), to predict the action outcome (Figure 1F).35 The PRO model is a gener-
alization of the temporal difference RL model. However, unlike the classic RL model that learns a scalar pre-
diction of the value of the current state, the PRO model learns predictions for multiple possible outcomes.
In this case, the model output represents a temporally discounted prediction of various outcomes in pro-
portion to their probability of occurrence, and the aggregates of outcome predictions generated by this
model minus the observed outcomes is explained as ‘‘negative surprise’’, which reflects the probability
of an expected non-occurrence, or a ‘‘positive surprise’’, reflecting the probability of an expected occur-
rence.104–106 These surprise signals can be used to guide learning and action prediction. For example,
Alexander and Brown used PRO and hierarchical error representation models to demonstrate the role of
mPFC in action prediction, in which the PE representation and action selection are interleaved with bot-
tom-down and top-down processing.106

The action in pain perception-action cycle is sometimes driven by conflicting motivations. Pain affects
social decision-making in humans. Animals also need to learn prioritized choices to survive when encoun-
tering competitive goals (e.g., safety, food, escape, and mating) in the presence of conflicts (e.g., pain,
hunger, predator threat, and fight). A utility function of each choice can be introduced to guide deci-
sion-making.107 The ‘‘decision utility’’ describes the usefulness that one perceives and uses for decision-
making, which is rooted in the classical behavioral economics theory.108 Given multiple competitive
choices, a Motivation-Decision model of pain can explain how animals and humans learn to reevaluate
the cost-benefit conflict to prioritize decisions.17,109 According to this model, anything that has higher rele-
vance for survival than pain will receive action preference over pain-related reflexes (by inhibiting nocicep-
tion); therefore, neural circuits that process multiple conflicting motivations will integrate information to
provide selective outputs to drive immediate attention. This Motivation-Decision model also allows us
to characterize the pain perception-action loop, as it predicts that top-down expectations may either
enhance or inhibit nociceptive transmission and further influence pain perception and future behavioral
responses.

NEURAL CORRELATES OF PREDICTION ERRORS IN PAIN CIRCUITS

The past few decades have witnessed rapid progress and deeper understanding on the neurobiology of
pain.3,5,110–115 Human neuroimaging has enabled us to study ‘‘correlational’’ changes in brain activity or
functional connectivity between a large number of brain areas during various pain states (Figure 2A). Ex-
amples of brain regions that respond to acute noxious stimuli and that take part in the pathology of
chronic pain include the prefrontal cortex (PFC), ACC, midcingulate cortex (MCC), anterior insula, pos-
terior insula (PI), thalamus, basolateral amygdala (BLA), hypothalamus, striatum, PAG, and rostral ventro-
medial medulla (RVM). Meanwhile, studies in rodent models have enabled us to dissect the causal roles
of pain pathways to show how specific neural circuits, such as the ACC, insula cortex, BLA, and prelimbic
(PL) cortex (human analog of dorsolateral PFC), play critical roles in pain processing and regulation
(Figure 2A and Box 3).

Neural evidence for brain region activity correlating to the computational models is a strong support for
the theoretical models. To date, experimental research has shown that many types of neurons can carry
reward PE signals,127 starting from dopamine neurons in the VTA and norepinephrine neurons in the locus
coeruleus, as well as pyramidal neurons and interneurons from the superior colliculus, frontal cortex, PAG,
and the striatum (Figure 2A). For instance, in the primary sensory cortex, motor cortex, and prefrontal cor-
tex, neuronal expressions of PE have been shown to represent the difference between predicted and
observed sensory, motor, or executive signals.21,48 The PE signals can appear in the form of unsigned
PEs (i.e., representing the magnitude of an unexpected outcome) or in the form of signed PEs (i.e., the di-
rection of such deviation). Neural ‘‘PE’’ signals can also appear as a generalized form that goes beyond the
reward PE or state PE.128

Two prominent examples of neural correlates of PEs are found in the amygdala and ACC in animal studies.
In one study, McHugh and colleagues showed that aversive prediction signals are found in the mouse
BLA.129 Specifically, amygdalar responses during fear conditioning evoked by footshock progressively
decreased, whereas responses evoked by the auditory cue that predicted footshock concomitantly
increased. The unexpected footshock produced larger amygdala responses than expected footshock.

iScience 26, 105707, January 20, 2023 9

 ll iScience
OPEN ACCESS Review

S1
S2

PC
PFC Th
Insula ala
m
Striatum us
OFC HIPP

S1

ACC S2 PPC

Insula
dlPFC Thalamus
NAc
AMY
OFC PAG
Cerebellum
RVM

S1 MCC
PL

ACC ACC
ACC PI

C
PF
NAc
MD PAG

RVM

Figure 2. Brain regions implicated in representations, prediction, and regulation of pain

(A) Top left and bottom left: The lateral and medial views of the brain map show the key nodes of pain matrix in the brainstem, cerebellum, and cerebral
cortex. The table on the right summarizes representative studies from human functional imaging studies (*) and electrophysiology studies in animals (#), or
computer simulations (A).
(B) Illustrations of several causally identified direct pathways in pain regulation.

The magnitude of the amygdalar response to the footshock predicted behavioral responses in the
following day, and the omission of expected footshock led to a decreased amygdala response below base-
line, suggesting a negative aversive PE signal. ACC neurons not only can encode pain-related negative
affect but they are also responsible for learning underlying recognition of pain-predictive cues and avoid-
ance.130 The ACC can also encode PE and surprise signals.105,131,132 Elston and colleagues reported that
error-related feedback integration is associated with increased inputs from the midbrain to the ACC,
and the error is predictive of subsequent behavioral adaptation.133

Meanwhile, human neuroimaging studies over the last two decades, meanwhile, have identified the roles of
a broad range of brain regions in prediction and PE during pain processing. For example, Talmi et al. re-
ported that reward PEs are also expressed in a ventromedial prefrontal region extending into the orbito-
frontal cortex (OFC), ACC, ventral striatum, hippocampus, bilateral insula, and posterior cingulate cor-
tex.134 Additionally, Baliki et al. found that the positive activations of NAc in healthy subjects encode the

10 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

Box 3. Causal dissection of cortical and subcortical pathways in pain regulation

Unlike peripheral and spinal nervous systems where nociceptive information flow follows a relatively linear ascending
pathway,116 the brain does not have a single distinct region responsible for integration of nociceptive signals. Instead,
a distributed network of cortical and subcortical regions collectively processes and integrates nociceptive signals to
give rise to the overall pain experience. Several cortico-cortical and cortico-subcortical pathways have been causally
identified in rodent pain studies (Figure 2B). In the S1/ACC pathway, chronic pain enhances the cortico-cortical
connection, whereas optogenetic modulation of this projection regulates aversive responses to pain.117 In the
pathway from mediodorsal (MD) thalamus to ACC, the weakened excitation of ACC neurons to MD inputs causes
excitation/inhibition (E/I) imbalance in pain; activating MD inputs elicited pain-related aversion, whereas direct inhi-
bition of subcortically projecting ACC neurons reproduces the same effect.118 In the afferent MCC/PI pathway, the
MCC does not mediate acute pain sensation and pain affect, but regulates nociceptive hypersensitivity.119 In the
ACC/RVM pathway, direct cortico-spinal modulation by optogenetics causes behavioral pain sensitization, whereas
inhibiting the projection induces analgesic effects.120 In the PL/NAc pathway, the prefrontal-striatal pathway pro-
vides an important regulation to pain.79,121,122 In the PFC/PAG pathway, neuropathic pain alters excitability and
local connectivity of PFC-PAG neurons,123 and optogenetic activation of this pathway produces analgesic and anti-
anxiety effects.124 Furthermore, it has been shown that the BLA/PFC/PAG/spinal pathway was critical for
development of mechanical and thermal hypersensitivity in neuropathic pain. Nerve injury strengthens synaptic input
from the BLA onto inhibitory PL interneurons, whereas the connectivity changes after pain behavior by reducing de-
scending modulation of spinal pain signals.125 Finally, the insula/BLA pathway is critical for the formation of
empathic pain. The selective activation or inhibition of the insula/BLA pathway strengthens or weakens the intensity
of observational pain, respectively.126

predictive value of pain relief and anticipate its analgesic potential on chronic pain.41 In an instrumental
pain-avoidance task, Roy et al. found that pain PEs are encoded in the human PAG.135 Specifically, they
found that the ventromedial prefrontal cortex provides an expectation-input to the PAG (Figure 3A), which
further relays PE signals to other prefrontal regions important for behavioral regulation (e.g. the dorsome-
dial prefrontal cortex). Using a probabilistic thermal pain paradigm, Geuter and colleagues found that the
anterior insula responses to cued pain stimuli strictly follow the response patterns according to the predic-
tive coding model, whereas the PI directly encodes the stimulus intensity.50 Furthermore, the dorsal ante-
rior insula represents pain intensity and expectations, whereas the ventral anterior insula additionally rep-
resents the absolute PE.136 Recently, using a mixed US (painful heat or loud sound) experimental protocol,
Horing and Buchel reported that the anterior insula correlates with unsigned intensity PEs, irrespective of
modality (pain vs. sound), indicating an unspecific aversive surprise signal; whereas the dorsal PI encodes
the signed intensity PEs, which are modality specific.53 Together, these results from the human insula cortex
offer a novel interpretation of aberrant pain processing as disturbed weighting of predictions and PEs. In a
follow-up human EEG study, the same research group reported that the alpha-to-beta frequency band ac-
tivity in the frontal cortical area is associated with the stimulus intensity expectation, followed by a negative
modulation of the gamma-band activity by absolute PEs.137 Additionally, Ploghaus et al. studied brain ac-
tivity during different types of thermal pain mismatch and observed that when painful thermal stimulation
was unexpected (i.e., positive PE), fMRI BOLD signals were elevated in the hippocampus, superior frontal
gyrus (GFs), superior parietal gyrus (GPs), and cerebellum.83 Activation of these brain regions, however,
was not found in the mismatch between the expectation and delivery of nonpainful heat stimulation.
Furthermore, when painful thermal stimulation was unexpectedly omitted (i.e., negative PE), the BOLD re-
sponses decreased in the left GPs and increased in the hippocampus, GFs, and cerebellum. In human EEG
recordings, it has been demonstrated that the peak amplitude of laser-evoked potentials (LEPs) reflects the
surprise of painful stimulus rather than the intensity of the stimulus, supporting a macroscopic neural corre-
late of PE.138 Together, PE signals appear to be distributed across multiple brain regions, and the differ-
ences in local responses can cause overall circuit changes to process both sensory and affective compo-
nents of pain.

APPLICATIONS OF COMPUTATIONAL MODELS TO UNDERSTAND DISTINCT PAIN

EXPERIENCES
Pain anticipation
Learning about environmental cues that predict noxious stimuli can modify our response to such stimuli.
Pain anticipation represents an important example of such learning paradigm. The anticipation of impend-
ing pain drives pain catastrophizing and other enhanced aversive responses. The anticipation of a future

iScience 26, 105707, January 20, 2023 11

 ll iScience
OPEN ACCESS Review

A B

C D

Figure 3. Computational models for studies of pain

(A) Dynamic causal modeling (DCM) for identifying plausible brain nodes of effective connectivity among value-encoding and aversive PE-encoding regions
(Roy et al., 2014, ref. 135, figure reprinted with permission, Springer Nature).
(B) Computational models to characterize self-fulfilling prophecies in pain (Jepma et al., 2018, ref. 12, figure adapted with permission, Springer Nature). (i) RL
model for capturing effects of cued-based expectations on pain and confirmation bias on expectation updating. Perceptual inference of pain is jointly
determined by nociceptive input, expected pain, and initial belief of the cue. (ii) Predictive coding is visualized by a graphical model and formulated as a
linear Gaussian state-space model (parameterized by three variables fm; sj ; sε g). Each node is a random variable, and the arrow indicates the statistical
dependency between random variables, the uncertainty is characterized by a Gaussian distribution with mean and SD parameters. Bayesian inference
produces iterative updating of the posterior distribution of pain. (iii) Expectation updating as a function of PE sign. Negative (aversive) and positive
(appetitive) PEs represent the lower-than-expected and higher-than-expected pain, respectively. There was a significant main effect of cue type but no
significant interaction between PE and cue type.
(C) Bayesian model prediction for placebo hypoalgesia (Büchel et al., 2014, ref. 49, figure adapted with permission, Elsevier). The uncertainty of expectation
or pain stimulus is characterized by the precision (inverse of variance) parameter. (i) Impact of the precision of prior expectation on posterior prediction in
placebo hypoalgesia experiments. Distributions of prior expectation (red), sensory observation (blue), and posterior of perceived pain (green) are shown
with respect to the visual analog scale (VAS) rating. (ii) Impact of the precision of sensory input on posterior prediction in placebo hypoalgesia experiments.
(D) Effects of attention increases orientation selectivity (response profile amplitude) and mismatch selectivity (putative PE) for four test conditions. Time
0 denotes the stimulus onset (Smount et al., 2019, ref. 139, CC BY license).

reward in the form of pain relief, meanwhile, can result in a range of responses from healthy exercises to
harmful drug addiction. Human neuroimaging studies have shown that the mPFC and insula cortex may
encode pain anticipation.110,140 These studies further suggest that distinct neural activities may underlie
pain and the anticipation of pain.

Prediction and PE provide a natural explanation for pain anticipation. Recently, Iigaya and colleagues pro-
posed that the total value of the reward predictive cue may be the sum of the anticipation of a future reward
and the reward value itself.141 Specifically, at the neural circuit level, the authors argued that the ventrome-
dial PFC tracks the value of anticipatory utility, midbrain DA correlates with information that enhances
anticipation, whereas the hippocampus mediates the functional coupling between the PFC and midbrain.
Thus, the intensity of pain anticipation is mechanistically summed by predictions as well as PE. Further-
more, Story et al. found that pain anticipation may carry a cost in decision-making, and a delayed outcome
can motivate individuals to choose an expedited punishment while waiting for the reward.142 These two
theories explain the potential unpleasantness or aversiveness of pain anticipation itself. Evidence from
pain studies also suggests that the degree of predictability may modulate the degree of anticipation.
For instance, the differences in pain rating and differences in event-related potential responses between
high and low-intensity painful stimulations were significantly reduced when the intensity of the noxious
stimulus was random and uncertain.143

12 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

A prior history of pain experiences can drive future expectations, and such expectations can in turn influ-
ence both perception and learning. Furthermore, bidirectional interactions between expectations and
experiences can result in self-reinforcing behaviors. Both RL and Bayesian theories have been applied to
understand how pain anticipation drives behaviors. Jepma and colleagues examined the modulation of
sensory perception in a human fMRI study and compared RL and Bayesian inference models while fitting
their experimental data (Figure 3B); they argued that the RL model explains pain perception as a mecha-
nistic process involving prediction and error correction, whereas the Bayesian model explains pain percep-
tion in terms of probabilistic inference.12 This interpretation is still debatable since Bayesian predictive
coding models can describe prediction and error correction, whereas RL can also be formulated within a
Bayesian framework.

Placebo and nocebo effects

Since the experience of pain is influenced by one’s belief and expectation, placebo treatment that has no
intrinsic pharmacological effect may produce analgesia by modifying expectation.144,145 Psychological and
neurobiological mechanisms of placebo analgesia have been well studied,146 and predictive coding
models have been adopted for these mechanistic inquiries.147 One theory for placebo hypoalgesia is
the integration of bottom-up nociceptive signals with top-down prior prediction of pain and/or pain re-
lief.49 Bayesian model posits that the brain makes inference or prediction by integrating the uncertainty
of expectation for pain or pain relief and the uncertainty of nociceptive information. Thus, Büchel et al.
showed that the uncertainty of expectations or sensory information is vital for subjective pain perception
(Figure 3C).49 Application of the Bayesian theory has led to several important discoveries. First, modulatory
neurotransmitters including opioid receptor agonists have been shown to influence the precision of expec-
tation to modulate pain perception.148,149 Second, a disruption of top-down prediction from prefrontal
area as a result of neurodegeneration can exacerbate PEs and consequently lead to a stronger influence
of ascending nociceptive signals.150–152 The theory of Bayesian integration in placebo hypoalgesia has
been tested in a number of human neuroimaging experiments.50,51,153 Meanwhile, RL theories can also
explain persistent pain modulation after the discontinuation of reinforcement, another feature of placebo
effects. Indeed, fMRI experiments have shown that analgesic expectations are associated with PE signals in
the ventral striatum, whereas the suppression of striatal PEs mediated by the PFC leads to reduced updat-
ing of treatment expectancies and decreased extinction of placebo hypoalgesia.154

Additionally, the view of placebo effects as a form of modification of expectation suggests the possibility to
update individual expectations. Thus, positive expectations of pain relief can drive placebo analgesia and
the reduction of pain, and these placebo effects further depend on the length of exposure to prior effective
interventions.155 In contrast, nocebo effects are defined as adverse events related to negative expectations
and learning processes that are involved in the modulation of the descending pain pathways. Nocebo hy-
peralgesia, as an opposite os placebo analgesia, is defined by the role that negative expectations of
increased pain intensity play in amplifying the overall pain perception.156,157 Computational models play
a similar role in understanding nocebo effects. In a human neuroimaging study, Tinnermann and col-
leagues showed that the value information (such as the price of medication) increased nocebo effects,
and this effect was mediated by neural interactions between the rostral ACC (rACC), PAG, and spinal
cord.158 Furthermore, the rACC deactivation predicted the strength of reported pain increase during no-
cebo treatment, serving an inhibitory function on the descending pain. Together, these results not only
highlight the importance of value representation in higher cortical areas such as the ACC for pain modu-
lation but they also support the role of predictive coding theory in understanding placebo and nocebo
effects.

Attention and pain

An important cognitive component of pain is attention.159 Pain demands attention; at the same time, atten-
tion or distraction can also modulate both sensory and affective expression of pain.160,161 Attention to pain
increases pain sensitivity, whereas distraction from pain reduces it.162 Specifically, when subjects were
distracted during painful stimulation, brain areas associated with the affective division of the ACC, OFC,
and PAG showed increased activation, whereas the insula, thalamus, and cognitive division of the ACC
showed reduced activation.163,164 Within the predictive coding framework, attention serves as an important
variable to promote neural encoding of PEs.139 Specifically, attention optimizes the expected precision of
predictions and increases the selectivity for neural responses to a relevant stimulus (Figure 3D). Therefore,
attention and prediction are two inherently related processes. Conversely, pain can disrupt attentional

iScience 26, 105707, January 20, 2023 13

 ll iScience
OPEN ACCESS Review

performance in both healthy subjects and patients with chronic pain.165 A noxious and salient stimulus can
be viewed as an independent attentional target, and it competes with other attentional targets in the pre-
frontal circuit during bottom-up and top-down interactions of the prefrontal and subcortical regions.166–168
A conflict between attention and pain avoidance demands a reevaluation of cognitive priority to determine
behavioral outputs. At the circuit level, if we adopt the conventional view that the PFC leads the main circuit
to predict outcomes, then changes in attention, through changes in prefrontal activities, can modify the
motivational gain during pain perception and reshape behavioral response. Furthermore, timing is critical
for attention, and attentional modulation provides a mechanism whereby even slight changes in neural ac-
tivity within the prediction-action cycle, if properly placed in a specific temporal context, can result in large
behavioral shifts. For example, a recent finding has shown that low-frequency (2 Hz) optogenetic activation
of the prelimbic (PL) cortex slightly decreased the basal firing rates of pain-regulatory PFC neurons, but it
effectively inhibited pain behaviors of freely behaving rats.169 Together, these results support the role of
PFC as a modifier of attention and in turn a gain controller for the prediction of behavioral outcomes in
pain states.170 Evidence for prefrontal regulation of pain behavioral control also allows the adoption of a
broad range of prediction-based computational models, such as differential reinforcement model,171 to
unravel the underpinning of attention and pain.

Pain chronification
Acute pain protects us from danger and bodily harm. Chronic pain, on the other hand, represents a path-
ological state. In many cases, chronic pain continues even when the initial tissue injury has already been
repaired. Chronic pain is in general defined by pain lasting longer than three months, and two hallmark fea-
tures are frequently observed: hypersensitivity to peripheral noxious stimuli of lower intensity or non-
noxious stimuli. The pathogenesis of chronic pain has been widely studied at peripheral, spinal, and brain
levels.172–175 In animal and human studies, repeated noxious stimulations can result in abnormal neural re-
sponses to noxious stimuli or at baseline across cortical (e.g., the somatosensory cortex, insula, PFC, and
ACC) and subcortical areas (e.g., the thalamus, amygdala, and NAc). In human fMRI studies, during pain
chronification, the brain undergoes a gradual switch from a predominantly somatosensory pain circuit to
one dominated by the affective-processing circuit.70,176,177 Multiple lines of evidence have pointed to
the vital role of mesolimbic or cortico-limbic system in the development, amplification, and prediction
of chronic pain.79,178 The interaction of mesolimbic circuitry and prefrontal circuitry determines the condi-
tion of pain transition to an emotional state, where the limbic circuits provide modulatory signals for
learning.179 During this process, the negative motivational value of nociception may increase while the
value of reward of pain relief may decrease.180

While molecular and circuit mechanisms of chronic pain have been well studied,111,172,181 computational
theories including Bayesian learning and RL can further contribute to the understanding of chronic pain,
particularly the transition from acute to chronic pain.40,182 A long-established view of pain chronification
theory is operant learning,183,184 which hypothesizes that positive and negative reinforcement of acute
pain behaviors leads to the development of chronic pain.185,186 Apkarian and colleagues proposed that
pain chronification is due to the persistence of the memory of pain and/or the inability to extinguish this
memory of pain evoked by an initial injury.182 According to this theory, pain chronification corresponds
to a state of continuous learning, in which aversive emotional associations are established with incidental
peripheral inputs.179,182,187 To emphasize the role of maladaptivity in brain circuits, Baliki and Apkarian pro-
posed a theory to view chronic pain as a repeated failure of avoidant behavior, and postulated that disrup-
tion of adaptations in a mesolimbic threshold process that gates the transformation of nociceptive activity
to conscious pain is viewed as a key factor for chronic pain development.16 Specifically, they deconstructed
chronic pain into four phases: predisposition, injury, transition, and maintenance. While healthy persistent
pain can be treated and recovered, chronic pain represents a maladaptive plastic change in the sensory
processing pathway that consequently leads to reorganization of the neocortical circuits. Additionally,
the transition from acute to chronic pain can cause low DA delivery in the mesolimbic system.72,73 Since
DA signaling contributes to avoidance behavior in the presence of noxious stimuli, a deficit in DA signaling
can impair motivated behavior over time.66 Neurobiological and behavioral findings can benefit from
computational theories and models to explain the chronicity and chronification of pain.

For instance, the prediction of injury and post-injury inference can be formulated as an inference and con-
trol question.19 The optimal control theory, as a special case of RL, can help model the perceived uncon-
trollability (or unpredictability) and gradual information restriction leading to pathological chronic pain.

14 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

Therefore, the control-as-inference view may model pain chronification as a series of maladaptive learning
(e.g., wrong associations), maladaptive integration (e.g., lesion-induced information loss), and maladaptive
priors (e.g., pessimistic belief or reduced subjective controllability) (see ref. 188 for a review).

During the transition process, a combination of physical events, memory of past events, and emotions can
force the brain develop an actor model (i.e. action learning) and a critic model (i.e. value learning) for
optimal long-term action selection. Additionally, persistent, maladaptive learning may cause a cascading
effect or vicious body-mind circle (e.g., pain/stress/pain). Overtime, the goal-directed action
(‘‘Stimulus-Action-Outcome’’ association) may gradually reduce to a habit (an autonomous ‘‘Stimulus-
Response’’ association) that is beyond volitional control. The basal ganglia—the area responsible for habit
learning, also plays a role in nociception and pain.189 Together, these computational models provide po-
tential tools for modeling different putative mechanisms of pain chronification, whereas associative
learning may be also used for modeling pain extinction or unlearning unpleasant emotion.190,191

Phantom pain
Phantom pain is defined as pain that occurs in an area of the body that is no longer present. Brain mech-
anisms for phantom pain have focused on cortical reorganization,192 as well as maladaptive plasticity in
both peripheral and central nervous systems.193 A deeper understanding of such neural plasticity in phan-
tom pain can be characterized by computational models.194 For instance, Bostrom and colleagues
proposed a physiologically plausible computational model to study the maladaptive reorganization of
the primary somatosensory cortex (S1).195 This model assumes that the S1 dynamically self-organizes based
on the neural input, and spontaneous activity in the sensory cortex is the result of discrete neuronal noise
and coherent activity. Computational simulations for this model showed that both the amount of reorga-
nization and the level of S1 activity during phantom movements were enhanced, suggesting that phantom
pain and persistent representation may share a common mechanism driven by an abnormally enhanced
spontaneous activity of deafferented peripheral nociceptive channels.

Understanding of the origin of phantom pain has further benefited from computational modeling,
especially models of prediction and PE. According to a stochastic entanglement model,196 a disruption
in the ‘‘sensory-motor-prediction’’ loop causes pain perception and sensorimotor circuitry to be patholog-
ically linked and in turn activated despite the lack of a real nociceptive input. This pathologic link creates a
misinformation of prediction in terms of the ‘‘perceived location’’ and ‘‘imagined motor execution’’. This
model can explain several features for phantom pain as well as its therapy, including mirror treatment
where a subject is placed in front of a mirror and asked to reimagine the missing limb.197 This model
also predicts that mirror therapy can effectively induce reactivation of the original central and peripheral
circuitry involved in motor control of the missing body part, resulting in gradual neuroplasticity to disen-
tangle the maladaptive pain-processing circuitry. Finally, phantom limb pain can be explained by maladap-
tive information integration, where the brain fails to access persistent congruent multisensory information
that is required to infer the state-action value function as a result of sensorimotor reorganization, an impor-
tant factor likely contributing to pathological chronic pain.174 Such maladaptive learning can be potentially
modeled by the actor-critic RL paradigm.

Conclusion
In this Perspective, we have presented a broad range of theoretical and computational models, with the
common theme being that PE and prediction play a central role in understanding pain as a closed loop
of perception and action. Prediction and learning-driven computational models can explain a broad
range of adaptive pain behaviors, including pain anticipation, pain relief as negative reinforcement, pla-
cebo and nocebo effects, and chronic pain. Therefore, computational modeling, in combination with
neuroimaging in humans and circuit dissection studies in animal models, can help advance our under-
standing of pain.

ACKNOWLEDGMENTS
We thank B. Seymour and G. Iannetti for some valuable discussions and M. Rockholt for feedback and
proofreading. We acknowledge the funding support from the US National Science Foundation (CBET-
1835000, Z.S.C. and J.W.), the National Institutes of Health (R01-NS121776, Z.S.C. and J.W.; R01-
GM115384, J.W.), and the NYU Interdisciplinary Pain Research Program (Z.S.C. and J.W.).

iScience 26, 105707, January 20, 2023 15

 ll iScience
OPEN ACCESS Review

AUTHOR CONTRIBUTIONS
Z.S.C. conceived and supervised experiments, analyzed and interpreted the data, and wrote the paper.
J.W. edited the paper. Z.S.C. and J.W. acquired funding.

DECLARATION OF INTERESTS
The authors declare no competing interests.

REFERENCES
1. Rainville, P., Duncan, G.H., Price, D.D., 14. Lang, V.A., Lundh, T., and Ortiz-Catalan, M. 29. Tabor, A., and Burr, C. (2019). Bayesian
Carrier, B., and Bushnell, M.C. (1997). Pain (2021). Mathematical and computational learning models of pain: a call to action.
affect encoded in human anterior cingulate models for pain: a systematic review. Pain Curr. Opin. Behav. Sci. 26, 54–61.
but not somatosensory cortex. Science 277, Med. 22, 2806–2817.
968–971. 30. Eckert, A.-L., Pabst, K., and Endres, D.M.
15. Wiech, K., and Tracey, I. (2009). The (2022). A Bayesian model for chronic pain.
2. Price, D.D. (2000). Psychological and neural influence of negative emotions on pain: Front. Pain Res. 3, 966034.
mechanisms of the affective dimension of behavioral effects and neural mechanisms.
pain. Science 288, 1769–1772. Neuroimage 47, 987–994. 31. Kiverstein, J., Kirchhoff, M.D., and Thacker,
M. (2022). An embodied predictive
3. Vogt, B.A. (2005). Pain and emotion 16. Baliki, M.N., and Apkarian, A.V. (2015). processing theory of pain experience. Rev.
interactions in subregions of the cingulate Nociception, pain, negative moods, and Philos Psychol. 2022, 11–26.
gyrus. Nat. Rev. Neurosci. 6, 533–544. behavior selection. Neuron 87, 474–491.
32. Ishikawa, R., and Izawa, J. (2022). The
4. Mano, H., and Seymour, B. (2015). Pain: a 17. Fields, H.L. (2018). How expectations computational neuroanatomy of predictive
distributed brain information network? PLoS influence pain. Pain 159, S3–S10. dynamics of pain perception. Preprint at
Biol. 13, e1002037. BioRxiv. https://doi.org/10.1101/2022.04.
18. Wiech, K., and Tracey, I. (2013). Pain, 13.488260.
5. Tan, L.L., and Kuner, R. (2021). Neocortical decisions, and actions: a motivational
circuits in pain and pain relief. Nat. Rev. perspective. Front. Neurosci. 7, 46. 33. Levenstein, D., Alvarez, V.A.,
Neurosci. 22, 458–471. Armarasingham, A., et al. (2022). On the role
19. Seymour, B., and Mancini, F. (2020). of theory and modeling in neuroscience.
Hierarchical models of pain: inference, J. Neurosci. to appear. https://arxiv.org/
6. Koban, L., Jepma, M., López-Solà, M., and
information-seeking, and adaptive control. pdf/2003.13825.pdf.
Wager, T.D. (2019). Different brain networks
mediate the effects of social and Neuroimage 222, 117212.
conditioned expectations on pain. Nat. 34. Jepma, M., Roy, M., Ramlakhan, K., van
Commun. 10, 4096. 20. Knill, D.C., and Pouget, A. (2004). The Velzen, M., and Dahan, A. (2022). Different
Bayesian brain: the role of uncertainty in brain systems support learning from
neural coding and computation. Trends received and avoided pain during human
7. Leknes, S., and Tracey, I. (2008). A common
Neurosci. 27, 712–719. pain-avoidance learning. Elife 11, e74149.
neurobiology for pain and pleasure. Nat.
Rev. Neurosci. 9, 314–320.
21. Bastos, A.M., Usrey, W.M., Adams, R.A., 35. Alexander, W.H., and Brown, J.W. (2011).
Mangun, G.R., Fries, P., and Friston, K.J. Medial prefrontal cortex as an action-
8. Mouraux, A., and Iannetti, G.D. (2018). The (2012). Canonical microcircuits for predictive outcome predictor. Nat. Neurosci. 14,
search for pain biomarkers in the human coding. Neuron 76, 695–711. 1338–1344.
brain. Brain 141, 3290–3307.
22. Aitchison, L., and Lengyel, M. (2017). With or 36. Perl, E.R. (2007). Ideas about pain, a
9. Corder, G., Ahanonu, B., Grewe, B.F., without you: predictive coding and Bayesian historical view. Nat. Rev. Neurosci. 8, 71–80.
Wang, D., Schnitzer, M.J., and Scherrer, G. inference in the brain. Curr. Opin.
(2019). An amygdalar neural ensemble that Neurobiol. 46, 219–227. 37. Moayedi, M., and Davis, K.D. (2013).
encodes the unpleasantness of pain. Theories of pain: from specificity to gate
Science 363, 276–281. 23. Spratling, M.W. (2017). A review of control. J. Neurophysiol. 109, 5–12.
predictive coding algorithms. Brain Cogn.
10. Xiao, Z., Martinez, E., Kulkarni, P.M., Zhang, 112, 92–97. 38. Melzack, R., and Wall, P.D. (1965). Pain
Q., Hou, Q., Rosenberg, D., Talay, R., Shalot, mechanisms: a new theory. Science 150,
L., Zhou, H., Wang, J., and Chen, Z.S. (2019). 24. Anchisi, D., and Zanon, M. (2015). A 971–979.
Cortical pain processing in the rat anterior Bayesian perspective on sensory and
cingulate cortex and primary cognitive integration in pain perception and 39. Fuster, J.M. (2004). Upper processing stages
somatosensory cortex. Front. Cell. placebo analgesia. PLoS One 10, e0117270. of the perception-action cycle. Trends
Neurosci. 13, 165. Cogn. Sci. 8, 143–145.
25. Wiech, K. (2016). Deconstructing the
11. Acuña, M.A., Kasanetz, F., De Luna, P., and sensation of pain: the influence of cognitive 40. Ingvar, M. (2015). Learning mechanisms in
Nevian, T. (2020). Cortical representation of processes on pain perception. Science 354, pain chronification—teachings from
pain by stable dedicated neurons and 584–587. placebo research. Pain 156, S18–S23.
dynamic ensembles. Preprint at BioRxiv.
https://doi.org/10.1101/2020.11.02.364778. 26. Ongaro, G., and Kaptchuk, T.J. (2019). 41. Baliki, M.N., Geha, P.Y., Fields, H.L., and
Symptom perception, placebo effects, and Apkarian, A.V. (2010). Predicting value of
12. Jepma, M., Koban, L., van Doorn, J., Jones, the Bayesian brain. Pain 160, 1–4. pain and analgesia: nucleus accumbens
M., and Wager, T.D. (2018). Behavioural and response to noxious stimuli changes in the
neural evidence for self-reinforcing 27. Ploner, M., Sorg, C., and Gross, J. (2017). presence of chronic pain. Neuron 66,
expectancy effects on pain. Nat. Hum. Brain rhythms of pain. Trends Cogn. Sci. 21, 149–160.
Behav. 2, 838–855. 100–110.
42. Buhle, J.T., Stevens, B.L., Friedman, J.J., and
13. Seymour, B. (2019). Pain: a precision for 28. Tabor, A., Thacker, M.A., Moseley, G.L., and Wager, T.D. (2012). Distraction and Placebo:
reinforcement learning and control. Neuron Körding, K.P. (2017). Pain: a statistical Two Separate Routes to Pain Control23,
101, 1029–1041. account. PLoS Comput. Biol. 13, e1005142. pp. 246–253.

16 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

43. Petrini, L., and Arendt-Nielsen, L. (2020). 59. van der Meer, M.A., and Redish, A.D. (2011). 72. Serafini, R.A., Pryce, K.D., and Zachariou, V.
Understanding pain catastrophizing: Ventral striatum: a critical look at models of (2020). The mesolimbic dopamine system in
putting pieces together. Front. Psychol. 11, learning and evaluation. Curr. Opin. chronic pain and associated affective
603420. Neurobiol. 21, 387–392. comorbidities. Biol. Psychiatry 87, 64–73.

44. Körding, K.P., and Wolpert, D.M. (2006). 60. Seymour, B., O’Doherty, J.P., Dayan, P., 73. Yang, S., Boudier-Revéret, M., Choo, Y.J.,
Bayesian decision theory in sensorimotor Koltzenburg, M., Jones, A.K., Dolan, R.J., and Chang, M.C. (2020). Association
control. Trends Cogn. Sci. 10, 319–326. Friston, K.J., and Frackowiak, R.S. (2004). between chronic pain and alterations in the
Temporal difference models describe mesolimbic dopaminergic system. Brain Sci.
45. De Ridder, D., Vanneste, S., and Freeman, higher-order learning in humans. Nature 10, 701.
W. (2014). The Bayesian brain: phantom 429, 664–667.
percepts resolve sensory uncertainty. 74. Markovic, T., Pedersen, C.E., Massaly, N.,
Neurosci. Biobehav. Rev. 44, 4–15. 61. Seymour, B., O’Doherty, J.P., Koltzenburg, Vachez, Y.M., Ruyle, B., Murphy, C.A.,
M., Wiech, K., Frackowiak, R., Friston, K., and Abiraman, K., Shin, J.H., Garcia, J.J., Yoon,
46. Alhussein, L., and Smith, M.A. (2021). Motor Dolan, R. (2005). Opponent appetitive- H.J., et al. (2021). Pain induces adaptations
planning under uncertainty. Elife 10, e67019. aversive neural processes underlie in ventral tegmental area dopamine
predictive learning of pain relief. Nat. neurons to drive anhedonia-like behavior.
Neurosci. 8, 1234–1240. Nat. Neurosci. 24, 1601–1613.
47. Clark, A. (2013). Whatever next? Predictive
brains, situated agents, and the future of
62. Navratilova, E., Atcherley, C.W., and 75. Redish, A.D. (2004). Addiction as a
cognitive science. Behav. Brain Sci. 36,
Porreca, F. (2015). Brain circuits encoding computational process gone awry. Science
181–204.
reward from pain relief. Trends Neurosci. 38, 306, 1944–1947.
48. Koster-Hale, J., and Saxe, R. (2013). Theory 741–750.
76. Wiech, K., Vandekerckhove, J., Zaman, J.,
of mind: a neural prediction problem. Tuerlinckx, F., Vlaeyen, J.W.S., and Tracey, I.
Neuron 79, 836–848. 63. Becerra, L., Breiter, H.C., Wise, R., Gonzalez,
R.G., and Borsook, D. (2001). Reward (2014). Influence of prior information on pain
circuitry activation by noxious thermal involves perceptual decision-making. Curr.
49. Büchel, C., Geuter, S., Sprenger, C., and Biol. 24, R679–R681.
stimuli. Neuron 32, 927–946.
Eippert, F. (2014). Placebo analgesia: a
predictive coding perspective. Neuron 81, 77. Wiech, K., Eippert, F., Vandekerckhove, J.,
1223–1239. 64. Navratilova, E., Xie, J.Y., Okun, A., Qu, C.,
Eyde, N., Ci, S., Ossipov, M.H., King, T., Zaman, J., Placek, K., Tuerlinckx, F., Vlaeyen,
Fields, H.L., and Porreca, F. (2012). Pain J.W.S., and Tracey, I. (2022). Cortico-
50. Geuter, S., Boll, S., Eippert, F., and Büchel, brainstem mechanisms of biased
relief produces negative reinforcement
C. (2017). Function dissociation of stimulus perceptual decision-making in the context
through activation of mesolimbic reward-
intensity encoding and predictive coding of of pain. J. Pain 23, 680–692.
valuation circuitry. Proc. Natl. Acad. Sci. USA
pain in the insula. Elife 6, e24770.
109, 20709–20713.
78. Salcido, C.A., Geltmeier, M.K., and Fuchs,
51. Grahl, A., Onat, S., and Büchel, C. (2018). P.N. (2018). Pain and decision-making:
65. Navratilova, E., and Porreca, F. (2014).
The periaqueductal gray and Bayesian interrelated through homeostasis. Open
Reward and motivation in pain and pain
integration in placebo analgesia. Elife 7, Pain J. 11, 31–40.
relief. Nat. Neurosci. 17, 1304–1312.
e32930.
79. Lee, M., Manders, T.R., Eberle, S.E., Su, C.,
66. Taylor, A.M.W., Becker, S., Schweinhardt, P., D’amour, J., Yang, R., Lin, H.Y., Deisseroth,
52. Hoskin, R., Berzuini, C., Acosta-Kane, D., El- and Cahill, C. (2016). Mesolimbic dopamine
Deredy, W., Guo, H., and Talmi, D. (2019). K., Froemke, R.C., and Wang, J. (2015).
signaling in acute and chronic pain: Activation of corticostriatal circuitry relieves
Sensitivity to pain expectations: a Bayesian implications for motivation, analgesia, and
model of individual differences. Cognition chronic neuropathic pain. J. Neurosci. 35,
addiction. Pain 157, 1194–1198. 5247–5259.
182, 127–139.
67. Smith, K.S., Berridge, K.C., and Aldridge, 80. Wiech, K., Lin, C.-S., Brodersen, K.H., Bingel,
53. Horing, B., and Büchel, C. (2022). The human J.W. (2011). Disentangling pleasure from
insula processes both modality- U., Ploner, M., and Tracey, I. (2010). Anterior
incentive salience and learning signals in insula integrates information about salience
independent and pain-selective learning brain reward circuitry. Proc. Natl. Acad. Sci.
signals. PLoS Biol. 20, e3001540. into perceptual decisions about pain.
USA 108, E255–E264. J. Neurosci. 30, 16324–16331.
54. Song, Y., Yao, M., Kemprecos, H., Byrne, A., 68. Lammel, S., Ion, D.I., Roeper, J., and 81. Gehrlach, D.A., Dolensek, N., Klein, A.S.,
Xiao, Z., Zhang, Q., Singh, A., Wang, J., and Malenka, R.C. (2011). Projection-specific Roy Chowdhury, R., Matthys, A., Junghänel,
Chen, Z.S. (2021). Predictive coding models modulation of dopamine neuron synapses M., Gaitanos, T.N., Podgornik, A., Black,
for pain perception. J. Comput. Neurosci. by aversive and rewarding stimuli. Neuron T.D., Reddy Vaka, N., et al. (2019). Aversive
49, 107–127. 70, 855–862. state processing in the posterior insular
cortex. Nat. Neurosci. 22, 1424–1437.
55. Arnal, L.H., and Giraud, A.L. (2012). Cortical 69. Becker, S., Gandhi, W., and Schweinhardt, P.
oscillations and sensory predictions. Trends (2012). Cerebral interactions of pain and 82. Tu, Y., Fu, Z., Mao, C., Falahpour, M.,
Cogn. Sci. 16, 390–398. reward and their relevance for chronic pain. Gollub, R.L., Park, J., Wilson, G., Napadow,
Neurosci. Lett. 520, 182–187. V., Gerber, J., Chan, S.T., et al. (2020).
56. van Pelt, S., Heil, L., Kwisthout, J., Distinct thalamocortical network dynamics
Ondobaka, S., van Rooij, I., and Bekkering, 70. Baliki, M.N., Petre, B., Torbey, S., Herrmann, are associated with the pathophysiology of
H. (2016). Beta and gamma-band activity K.M., Huang, L., Schnitzer, T.J., Fields, H.L., chronic low back pain. Nat. Commun.
reflect predictive coding in the processing and Apkarian, A.V. (2012). Corticostriatal 11, 4347.
of causal events. Soc. Cogn. Affect. functional connectivity predicts transition to
Neurosci. 11, 973–980. chronic back pain. Nat. Neurosci. 15, 83. Ploghaus, A., Tracey, I., Clare, S., Gati, J.S.,
1117–1119. Rawlins, J.N., and Matthews, P.M. (2000).
57. Sutton, R.S., and Barto, A.G. (2021). Learning about pain: the neural substrate of
Reinforcement Learning: An Introduction, 71. Martikainen, I.K., Nuechterlein, E.B., Peciña, the prediction error for aversive events.
2nd edition (MIT Press). M., Love, T.M., Cummiford, C.M., Green, Proc. Natl. Acad. Sci. USA 97, 9281–9286.
C.R., Stohler, C.S., and Zubieta, J.K. (2015).
58. Maia, T.V. (2009). Reinforcement learning, Chronic back pain is associated with 84. Holland, P.C., and Schiffino, F.L. (2016).
conditioning, and the brain: successes and alterations in dopamine neurotransmission Mini-review: prediction errors, attention and
challenges. Cogn. Affect. Behav. Neurosci. in the ventral striatum. J. Neurosci. 35, associative learning. Neurobiol. Learn.
9, 343–364. 9957–9965. Mem. 131, 207–215.

iScience 26, 105707, January 20, 2023 17

 ll iScience
OPEN ACCESS Review

85. Rescorla, R.A., and Wagner, A.R. (1972). A 98. Yang, Z., Jackson, T., and Huang, C. (2016). 115. Finnerup, N.B., Kuner, R., and Jensen, T.S.
theory of Pavlovian conditioning: variations Neural activation during anticipation of near (2021). Neuropathic pain: from mechanisms
in the effectiveness of reinforcement and pain-threshold stimulation among the pain- to treatment. Physiol. Rev. 101, 259–301.
nonreinforcement. In Classical Conditioning fearful. Front. Neurosci. 10, 342.
II: Current Research and Theory, A.H. Black 116. Todd, A.J. (2010). Neuronal circuitry for pain
and W.F. Prokasy, eds. (Appleton Century 99. Wang, O., Lee, S.W., O’Doherty, J., processing in the dorsal horn. Nat. Rev.
Crofts), pp. 64–99. Seymour, B., and Yoshida, W. (2018). Model- Neurosci. 11, 823–836.
based and model-free pain avoidance
86. Mackintosh, N.J. (1975). A theory of learning. Brain Neurosci. Adv. 2. 117. Singh, A., Patel, D., Li, A., Hu, L., Zhang, Q.,
attention: variations in the associability of 2398212818772964. Liu, Y., Guo, X., Robinson, E., Martinez, E.,
stimuli with reinforcement. Psychol. Rev. 82, Doan, L., et al. (2020). Mapping cortical
276–298. 100. Bolles, R.C., and Fanselow, M.S. (1980). A integration of sensory and affective pain
perceptual-defensive-recuperative model pathways. Curr. Biol. 30, 1703–1715.e5.
87. Pearce, J.M., and Hall, G. (1980). A model for of fear and pain. Behavioral and Brain Sci. 3,
Pavlovian learning: variations in the 291–323.
effectiveness of conditioned but not of 118. Meda, K.S., Patel, T., Braz, J.M., Malik, R.,
unconditioned stimuli. Psychol. Rev. 87, 101. Schlund, M.W., and Cataldo, M.F. (2010). Turner, M.L., Seifikar, H., Basbaum, A.I., and
532–552. Amygdala involvement in human avoidance, Sohal, V.S. (2019). Microcircuit mechanisms
escape and approach behavior. through which mediodorsal thalamic input
Neuroimage 53, 769–776. to anterior cingulate cortex exacerbates
88. Pearce, J.M., Kaye, H., and Hall, G. (1982).
pain-related aversion. Neuron 102, 944–
Predictive accuracy and stimulus
959.e3.
associability: development of a model for 102. Seymour, B., Singer, T., and Dolan, R. (2007).
Pavlovian learning. In Quantitative Analyses The neurobiology of punishment. Nat. Rev.
of Behavior. Ballinger, M.L. Commons, R.J. Neurosci. 8, 300–311. 119. Tan, L.L., Pelzer, P., Heinl, C., Tang, W.,
Herrnstein, and A.R. Wagner, eds., Gangadharan, V., Flor, H., Sprengel, R.,
pp. 241–255. 103. Carter, C.S., Braver, T.S., Barch, D.M., Kuner, T., and Kuner, R. (2017). A pathway
Botvinick, M.M., Noll, D., and Cohen, J.D. from midcingulate cortex to posterior insula
89. Johansen, J.P., Tarpley, J.W., LeDoux, J.E., (1998). Anterior cingulate cortex, error gate nociceptive hypersensitivity. Nat.
and Blair, H.T. (2010). Neural substrates for detection, and the online monitoring of Neurosci. 20, 1591–1601.
expectation-modulated fear learning in the performance. Science 280, 747–749.
amygdala and periaqueductal gray. Nat. 120. Chen, T., Taniguchi, W., Chen, Q.Y., Tozaki-
Neurosci. 13, 979–986. 104. Alexander, W.H., and Brown, J.W. (2014). A Saitoh, H., Song, Q., Liu, R.H., Koga, K.,
general role for medial prefrontal cortex in Matsuda, T., Kaito-Sugimura, Y., Wang, J.,
90. Roesch, M.R., Esber, G.R., Li, J., Daw, N.D., event prediction. Front. Comput. Neurosci. et al. (2018). Top-down descending
and Schoenbaum, G. (2012). Surprise! 8, 69. facilitation of spinal sensory excitatory
Neural correlates of pearce-Hall and transmission from the anterior cingulate
rescorla-wagner coexist within the brain. 105. Alexander, W.H., and Brown, J.W. (2019). cortex. Nat. Commun. 9, 1886.
Eur. J. Neurosci. 35, 1190–1200. The role of the anterior cingulate cortex in
prediction error and signaling surprise. Top. 121. Martinez, E., Lin, H.H., Zhou, H., Dale, J., Liu,
91. Taylor, V.A., Roy, M., Chang, L., Gill, L.-N., Cogn. Sci. 11, 119–135. K., and Wang, J. (2017). Corticostriatal
Mueller, C., and Rainville, P. (2018). Reduced regulation of acute pain. Front. Cell.
fear-conditioned pain modulation in 106. Alexander, W.H., and Brown, J.W. (2018). Neurosci. 11, 146.
experienced meditators: a preliminary Frontal cortex function as derived from
study. Psychosom. Med. 80, 799–806. hierarchical predictive coding. Sci. Rep. 122. Zhou, H., Martinez, E., Lin, H.H., Yang, R.,
8, 3843. Dale, J.A., Liu, K., Huang, D., and Wang, J.
92. Walteros, C., Sánchez-Navarro, J.P., Muñoz, (2018). Inhibition of the prefrontal projection
M.A., Martı́nez-Selva, J.M., Chialvo, D., and 107. Berridge, K.C., and Aldridge, J.W. (2008). to the nucleus accumbens enhances pain
Montoya, P. (2011). Altered associative Decision utility, the brain, and pursuit of sensitivity and affect. Front. Cell. Neurosci.
learning and emotional decision making in hedonic goals. Soc. Cogn. 26, 621–646. 12, 240.
fibromyalgia. J. Psychosom. Res. 70,
294–301. 108. von Neumann, J., and Morgenstern, O.
123. Cheriyan, J., and Sheets, P.L. (2018). Altered
(1953). Theory of Games and Economic
excitabilitty and local connectivity of mPFC-
93. Zunhammer, M., Spisák, T., Wager, T.D., Behavior (Princeton Univ. Press).
PAG neurons in a mouse model of
and Bingel, U.; Placebo Imaging
neuropathic pain. J. Neurosci. 38,
Consortium (2021). Meta-analysis of neural 109. Fields, H.L. (2006). A Motivation-Decision
4829–4839.
systems underlying placebo analgesia from Model of Pain: The Role of Opioids (IASP
individual participant fMRI data. Nat. Press).
Commun. 12, 1391. 124. Yin, J.B., Liang, S.H., Li, F., Zhao, W.J., Bai,
110. Tracey, I., and Mantyh, P.W. (2007). The Y., Sun, Y., Wu, Z.Y., Ding, T., Sun, Y., Liu,
94. Zhang, S., Mano, H., Lee, M., Yoshida, W., cerebral signature for pain perception and H.X., et al. (2020). dmPFC-vlPAG projection
Kawato, M., Robbins, T.W., and Seymour, B. its modulation. Neuron 55, 377–391. neurons contribute to pain threshold
(2018). The control of tonic pain by active maintenance and antianxiety behaviors.
relief learning. Elife 7, e31949. 111. Costigan, M., Scholz, J., and Woolf, C.J. J. Clin. Invest. 130, 6555–6570.
(2009). Neuropathic pain: a maladaptive
95. Zhang, S., Mano, H., Ganesh, G., Robbins, response of the nervous system to damage. 125. Huang, J., Gadotti, V.M., Chen, L., Souza,
T., and Seymour, B. (2016). Dissociable Annu. Rev. Neurosci. 32, 1–32. I.A., Huang, S., Wang, D., Ramakrishnan, C.,
learning processes underlies human pain Deisseroth, K., Zhang, Z., and Zamponi,
conditioning. Curr. Biol. 26, 52–58. 112. Veinante, P., Yalcin, I., and Barrot, M. (2013). G.W. (2019). A neuronal circuit for
The amygdala between sensation and activating descending modulation of
96. Moutoussis, M., Bentall, R.P., Williams, J., affect: a role in pain. J. Mol. Psychiatry 1, 9. neuropathic pain. Nat. Neurosci. 22,
and Dayan, P. (2008). A temporal difference 1659–1668.
account of avoidance learning. Network 19, 113. Legrain, V., Iannetti, G.D., Plaghki, L., and
137–160. Mouraux, A. (2011). The pain matrix 126. Zhang, M.-M., Geng, A.-Q., Chen, K., Wang,
reloaded: a salience detection system for J., Wang, P., Qiu, X.T., Gu, J.X., Fan, H.W.,
97. Urien, L., Xiao, Z., Dale, J., Bauer, E.P., Chen, the body. Prog. Neurobiol. 93, 111–124. Zhu, D.Y., Yang, S.M., et al. (2022).
Z., and Wang, J. (2018). Rate and temporal Glutamatergic synapses from the insular
coding mechanisms in the anterior cingulate 114. Peirs, C., and Seal, R.P. (2016). Neural cortex to the basolateral amygdala encode
cortex for pain anticipation. Sci. Rep. circuits for pain: recent advances and observational pain. Neuron 110, 1993–
8, 8298. current views. Science 354, 578–584. 2008.e6.

18 iScience 26, 105707, January 20, 2023

iScience ll
Review OPEN ACCESS

127. Schultz, W., and Dickinson, A. (2000). and the utility of anticipation. Sci. Adv. 6, 156. Colloca, L., and Benedetti, F. (2007).
Neuronal coding of prediction errors. Annu. eaba3828. Nocebo hyperalgesia: how anxiety is turned
Rev. Neurosci. 23, 473–500. into pain. Curr. Opin. Anaesthesiol. 20,
142. Story, G.W., Vlaev, I., Seymour, B., Winston, 435–439.
128. Chumbley, J.R., Burke, C.J., Stephan, K.E., J.S., Darzi, A., and Dolan, R.J. (2013). Dread
Friston, K.J., Tobler, P.N., and Fehr, E. and the disvalue of future pain. PLoS 157. Tracey, I. (2010). Getting the pain you
(2014). Surprise beyond prediction error. Comput. Biol. 9, e1003335. expect: mechanisms of placebo, nocebo
Hum. Brain Mapp. 35, 4805–5485. and reappraisal effects in humans. Nat.
143. Peng, W., Huang, X., Liu, Y., and Cui, F. Med. 16, 1277–1283.
129. McHugh, S.B., Barkus, C., Huber, A., (2019). Predictability modulates the
Capitão, L., Lima, J., Lowry, J.P., and anticipation and perception in pain in both 158. Tinnermann, A., Geuter, S., Sprenger, C.,
Bannerman, D.M. (2014). Aversive self and others. Soc. Cogn. Affect. Neurosci. Finsterbusch, J., and Büchel, C. (2017).
prediction error signals in the amygdala. 14, 747–757. Interactions between brain and spinal cord
J. Neurosci. 34, 9024–9033. mediate value effects in nocebo
144. Wager, T.D., Rilling, J.K., Smith, E.E., hyperalgesia. Science 358, 105–108.
130. Johansen, J.P., Fields, H.L., and Manning, Sokolik, A., Casey, K.L., Davidson, R.J.,
B.H. (2001). The affective component of pain Kosslyn, S.M., Rose, R.M., and Cohen, J.D. 159. Villemure, C., and Bushnell, C.M. (2002).
in rodents: direct evidence for a contribution (2004). Placebo-induced changes in FMRI in Cognitive modulation of pain: how do
of the anterior cingulate cortex. Proc. Natl. the anticipation and experience of pain. attention and emotion influence pain
Acad. Sci. USA 98, 8077–8082. Science 303, 1162–1167. processing. Pain 95, 195–199.

131. Brown, J.W., and Braver, T.S. (2005). Learned 145. Bingel, U. (2020). Placebo 2.0: the impact of 160. Seminowicz, D.A., and Davis, K.D. (2007). A
predictions of error likelihood in the anterior expectations on analgesic treatment re-examination of pain-cognition
cingulate cortex. Science 307, 1118–1121. outcome. Pain 161, S48–S56. interactions: implications for neuroimaging.
Pain 130, 8–13.
132. Hyman, J.M., Holroyd, C.B., and Seamans, 146. Colloca, L., Klinger, R., Flor, H., and Bingel,
J.K. (2017). A novel neural prediction error U. (2013). Placebo analgesia: psychological 161. Kucyi, A., and Davis, K.D. (2015). The
found in anterior cingulate cortex ensemble. and neurobiological mechanisms. Pain 154, dynamic pain connectome. Trends
Neuron 95, 447–456.e3. 511–514. Neurosci. 38, 86–95.
133. Elston, T.W., Croy, E., and Bilkey, D.K. 147. Atlas, L.Y. (2021). A social affective 
162. Bushnell, M.C., Ceko, M., and Low, L.A.
(2019). Communication between the neuroscience lens on placebo analgesia. (2013). Cognitive and emotional control of
anterior cingulate cortex and ventral Trends Cogn. Sci. 25, 992–1005. pain and its disruption in chronic pain. Nat.
tegmental area during a cost-benefit Rev. Neurosci. 14, 502–511.
reversal task. Cell Rep. 26, 2353–2361.e3.
148. Friston, K. (2010). The free-energy principle:
a unified brain theory? Nat. Rev. Neurosci. 163. Tracey, I., Ploghaus, A., Gati, J.S., Clare, S.,
134. Talmi, D., Dayan, P., Kiebel, S.J., Frith, C.D.,
11, 127–138. Smith, S., Menon, R.S., and Matthews, P.M.
and Dolan, R.J. (2009). How humans
(2002). Imaging attention modulation of
integrate the prospects of pain and reward
149. Kocagoncu, E., Klimovich-Gray, A., Hughes, pain in periaqueductal gray in humans.
during choice. J. Neurosci. 29, 14617–14626.
L.E., and Rowe, J.B. (2021). Evidence and J. Neurosci. 22, 2748–2752.
135. Roy, M., Shohamy, D., Daw, N., Jepma, M., implications of abnormal predictive coding
Wimmer, G.E., and Wager, T.D. (2014). in dementia. Brain 144, 3311–3321. 164. Bantick, S.J., Wise, R.G., Ploghaus, A., Clare,
Representation of aversive prediction errors S., Smith, S.M., and Tracey, I. (2002). Imaging
in the human periaqueductal gray. Nat. 150. Conte, A., Khan, N., Defazio, G., Rothwell, how attention modulates pain in humans
Neurosci. 17, 1607–1612. J.C., and Berardelli, A. (2013). using functional MRI. Brain 125, 310–319.
Pathophysiology of somatosensory
136. Fazeli, S., and Büchel, C. (2018). Pain-related abnormalities in Parkinson disease. Nat. 165. Yoshino, A., Otsuru, N., Okada, G., Tanaka,
expectation and prediction error signals in Rev. Neurol. 9, 687–697. K., Yokoyama, S., Okamoto, Y., and
the anterior insular are not related to Yamawaki, S. (2021). Brain changes
aversiveness. J. Neurosci. 38, 6461–6474. 151. Corbett, A., Husebo, B., Malcangio, M., associated with impaired attention function
Staniland, A., Cohen-Mansfield, J., in chronic pain. Brain Cogn. 154, 105806.
137. Strube, A., Rose, M., Fazeli, S., and Büchel, Aarsland, D., and Ballard, C. (2012).
C. (2021). Spatial and spectral characteristics Assessment and treatment of pain in people 166. Buschman, T.J., and Miller, E.K. (2007). Top-
of expectations and prediction errors in pain with dementia. Nat. Rev. Neurol. 8, 264–274. down versus bottom-up control of attention
and thermoception. Elife 10, e62809. in the prefrontal and posterior parietal
152. Rukavina, K., Leta, V., Sportelli, C., Buhidma, cortices. Science 315, 1860–1862.
138. Iannetti, G.D., Hughes, N.P., Lee, M.C., and Y., Duty, S., Malcangio, M., and Ray
Mouraux, A. (2008). Determinants of laser- Chaudhuri, K. (2019). Pain in Parkinson’s 167. Rossi, A.F., Pessoa, L., Desimone, R., and
evoked EEG responses: pain perception or disease: new concepts in pathogenesis and Ungerleider, L.G. (2009). The prefrontal
stimulus saliency? J. Neurophysiol. 100, treatment. Curr. Opin. Neurol. 32, 579–588. cortex and the executive attention. Exp.
815–828. Brain Res. 192, 489–497.
153. Geuter, S., Koban, L., and Wager, T.D.
139. Smout, C.A., Tang, M.F., Garrido, M.I., and (2017). The cognitive neuroscience of 168. Ong, W.Y., Stohler, C.S., and Herr, D.R.
Mattingley, J.B. (2019). Attention promotes placebo effects: concepts, predictions, and (2019). Role of the prefrontal cortex in pain
the neural encoding of prediction errors. physiology. Annu. Rev. Neurosci. 40, processing. Mol. Neurobiol. 56, 1137–1166.
PLoS Biol. 17, e2006812. 167–188.
169. Dale, J., Zhou, H., Zhang, Q., Martinez, E.,
140. Ploghaus, A., Tracey, I., Gati, J.S., Clare, S., 154. Schenk, L.A., Sprenger, C., Onat, S., Colloca, Hu, S., Liu, K., Urien, L., Chen, Z., and Wang,
Menon, R.S., Matthews, P.M., and Rawlins, L., and Büchel, C. (2017). Suppression of J. (2018). Scaling up cortical control to inhibit
J.N. (1999). Dissociating pain from its striatal prediction errors by the prefrontal chronic pain. Cell Rep. 23, 1301–1313.
anticipation in the human brain. Science cortex in placebo hypoalgesia. J. Neurosci.
284, 1979–1981. 37, 9715–9723. 170. Wiech, K., Kalisch, R., Weiskopf, N., Pleger,
B., Stephan, K.E., and Dolan, R.J. (2006).
141. Iigaya, K., Hauser, T.U., Kurth-Nelson, Z., 155. Colloca, L., Petrovic, P., Wager, T.D., Ingvar, Anterolateral prefrontal cortex mediates the
O’Doherty, J.P., Dayan, P., and Dolan, R.J. M., and Benedetti, F. (2010). How the analgesic effect of expected and perceived
(2020). The value of what’s to come: neural number of learning trials affects placebo control over pain. J. Neurosci. 26, 11501–
mechanisms of coupling prediction error and nocebo responses. Pain 151, 430–439. 11509.

iScience 26, 105707, January 20, 2023 19

 ll iScience
OPEN ACCESS Review

171. Turner, K.M., and Parkes, S.L. (2020). 180. Porreca, F., and Navratilova, E. (2017). 190. Büchel, C., and Dolan, R.J. (2000). Classical
Prefrontal regulation of behavioral control: Reward, motivation and emotion of pain and fear conditioning in functional
evidence from learning theory and its relief. Pain 158, S43–S49. neuroimaging. Curr. Opin. Neurobiol. 10,
translational approaches in rodents. 219–223.
Neurosci. Biobehav. Rev. 118, 27–41. 181. Kuner, R., and Kuner, T. (2021). Cellular
circuits in the brain and their modulation in 191. Koenen, L.R., Pawlik, R.J., Icenhour, A.,
172. Basbaum, A.I., Bautista, D.M., Scherrer, G., acute and chronic pain. Physiol. Rev. 101, Petrakova, L., Forkmann, K., Theysohn, N.,
and Julius, D. (2009). Cellular and molecular 213–258. Engler, H., and Elsenbruch, S. (2021).
mechanisms of pain. Cell 139, 267–284. Associative learning and extinction of
182. Apkarian, A.V., Baliki, M.N., and Geha, P.Y. conditioned threat predictors across
173. Denk, F., McMahon, S.B., and Tracey, I. (2009). Towards a theory of chronic pain. sensory modalities. Commun. Biol. 4, 1000.
(2014). Pain vulnerability: a neurobiological Prog. Neurobiol. 87, 81–97.
perspective. Nat. Neurosci. 17, 192–200. 192. Flor, H., Elbert, T., Knecht, S., Wienbruch, C.,
183. Fordyce, W.E. (1976). Behavioral Methods Pantev, C., Birbaumer, N., Larbig, W., and
for Chronic Pain and Illness (Mosby). Taub, E. (1995). Phantom-limb pain as a
174. Kuner, R., and Flor, H. (2017). Structural
perceptual correlate of cortical
plasticity and reorganisation in chronic pain.
184. Hölzl, R., Kleinböhl, D., and Huse, E. (2005). reorganization following arm amputation.
Nat. Rev. Neurosci. 18, 113–130.
Implicit operant learning of pain Nature 375, 482–484.
sensitization. Pain 115, 12–20.
175. Zhang, Q., Manders, T., Tong, A.P., Yang, R., 193. Flor, H., Nikolajsen, L., and Staehelin
Garg, A., Martinez, E., Zhou, H., Dale, J., Jensen, T. (2006). Phantom limb pain: a case
Goyal, A., Urien, L., et al. (2017). Chronic 185. Gatzounis, R., Schrooten, M.G.S., Crombez,
G., and Vlaeyen, J.W.S. (2012). Operant of maladaptive CNS plasticity. Nat. Rev.
pain induces generalized enhancement of Neurosci. 7, 873–881.
aversion. Elife 6, e25302. learning theory in pain and chronic pain
rehabilitation. Curr. Pain Headache Rep. 16,
117–126. 194. Hsu, E., and Cohen, S.P. (2013).
176. Bingel, U., Schoell, E., Herken, W., Büchel, Postamputation pain: epidemiology,
C., and May, A. (2007). Habituation to painful 186. Borsook, D., Youssef, A.M., Simons, L., mechanisms, and treatment. J. Pain Res. 6,
stimulation involves the antinociceptive Elman, I., and Eccleston, C. (2018). When 121–136.
system. Pain 131, 21–30. pain gets stuck: the evolution of pain
chronification and treatment resistance. 195. Boström, K.J., de Lussanet, M.H.E., Weiss,
177. Hashmi, J.A., Baliki, M.N., Huang, L., Baria, Pain 159, 2421–2436. T., Puta, C., and Wagner, H. (2014). A
A.T., Torbey, S., Hermann, K.M., Schnitzer, computational model unifies apparently
T.J., and Apkarian, A.V. (2013). Shape 187. Mansour, A.R., Farmer, M.A., Baliki, M.N., contradictory findings concerning phantom
shifting pain: chronification of back pain and Apkarian, A.V. (2014). Chronic pain: the pain. Sci. Rep. 4, 5298.
shifts brain representation from nociceptive role of learning and brain plasticity. Restor.
to emotional circuits. Brain 136, 2751–2768. Neurol. Neurosci. 32, 129–139. 196. Ortiz-Catalan, M. (2018). The Stochastic
entanglement and phantom motor
178. Thompson, J.M., and Neugebauer, V. 188. Seymour, B., Crook, R., and Chen, Z.S. execution hypotheses: a theoretical
(2019). Cortico-limbic pain mechanisms. (2022). Post-injury pain and behaviour: a framework for the origin and treatment of
Neurosci. Lett. 702, 15–23. control theory perspective. Nat. Rev. phantom limb pain. Front. Neurol. 9, 748.
Neurosci. to appear.
179. Apkarian, V.A., Hashmi, J.A., and Baliki, 197. Ramachandran, V.S., and Rogers-
M.N. (2011). Pain and the brain: specificity 189. Chudler, E.H., and Dong, W.K. (1995). The Ramachandran, D. (1996). Synaesthesia in
and plasticity of the brain in clinical chronic role of the basal ganglia in nociception and phantom limbs induced with mirrors. Proc.
pain. Pain 152, S49–S64. pain. Pain 60, 3–38. Biol. Sci. 263, 377–386.

20 iScience 26, 105707, January 20, 2023