Neuroimaging Studies of Memory

John Jonides
Tor D. Wager
David T. Badre

University of Michigan

Address correspondence to:

Dr. John Jonides
Department of Psychology
University of Michigan
525 E. University Ave.
Ann Arbor, MI 48109-1109
Telephone: 734-764-0192
Fax: 734-994-7157
E-mail: [email protected]

Preparation of this manuscript was supported in part by a grant from the National Insitute of Aging to the
University of Michigan.

Neuroimaging Studies of Memory 1

 good chocolate); of spatial information we have stored
Some Introductory Concepts for navigating around our world; and so on. In addition,
many pieces of information are stored that we don’t
Types of Memory. One of the critical
normally retrieve consciously, but that nevertheless
properties that makes the human mind so
guide our everyday behavior, such as the rules of
extraordinarily suited to understanding and dealing with
language or habitual actions in which we engage every
the world is its ability to shift in time—to model the
day.
future and reconstruct the past. Reconstructing the
Larry Squire of the University of California and
past requires memory, and memory is fundamental to
Endel Tulving of the University of Toronto have
nearly any cognitive skill. It is involved in complex
proposed schemes that summarize the various forms of
processes such as problem-solving, and it is involved
long-term memory. One way of synthesizing and
in even what seem to be the simplest skills, such as
expanding these schemes is shown in Figure 1. The
recognizing a familiar face. The role played by
figure shows that there are two broad divisions of long-
memory in cognition is complex enough that not just a
term memory: declarative and procedural. Declarative
single memory system will do. Humans and other
memory refers to the facts and events that we can
animals have several memory systems with different
retrieve at will, often consciously. By contrast,
characteristics and different neural implementations,
procedural memory refers to stored information that has
and these systems, acting in concert, contribute to the
an impact on our behavior, but that is not willfully
human mind's tremendous adaptability.
retrieved. Consider, for example, the concept of a
At the broadest level, one can distinguish
bicycle. A declarative memory you might have of a
between "working memory" and "long-term memory."
bicycle is that it is blue, has 21 gears, mountain-terrain
Working memory refers to the system that stores a
tires, two handbrakes, and so on. These are all facts that
small amount of information for a brief span of time.
can be willfully retrieved from memory. By contrast,
Information stored in working memory is then used in
you also have stored information that allows you to ride
the service of other cognitive tasks. For example, if
your bicycle—a task that any 6-year old will tell you is
you were solving an arithmetic problem such as 817 +
not easy. This information is not consciously
723 without the benefit of writing anything on paper,
retrievable; indeed, it is a nontrivial problem in physics
working memory would be used to store the problem,
and kinesiology to describe just how people are able to
store the intermediate steps in the addition, and store
ride a two-wheeled bicycle without falling over. The
the final solution. In addition to temporary storage, an
contrast between these two sorts of memory is a contrast
important component of working memory is what is
between declarative and procedural memory. Perhaps the
called "executive processing:" the set of operations
most compelling evidence that procedural knowledge is
that permit one to manipulate the contents of working
different from declarative knowledge is that patients with
memory. In the previous example, executive processes
damage to their hippocampi and surrounding medial
would be involved in switching attention from one
temporal lobes can learn new procedural skills, even
column of addition to another and in organizing the
though they cannot encode where they learned the skill
order of steps to arrive at a final sum. While there is
or remember any details of having practiced it, even when
as yet no overall agreement about a full list of
that practice occurred very recently. Other patients with
executive processes, they can generally be thought of
cerebellar damage can remember the practice sessions,
as operations that regulate the processes operating on
but their skills on most motor tasks do not improve. This
the contents of working memory, processes such as
pattern of deficits, called a double dissociation, helps
selective attention to relevant information (more about
define procedural and declarative processes as distinct
this shortly).
types of memory.
In contrast to the short duration and small
Declarative memory itself comes in two forms.
capacity of working memory, long-term memory is a
One is called episodic memory, or memory for specific
system with very long-duration memory traces and
events, and it consists of memory traces that are
very large storage capacity. In our mental arithmetic
accompanied by memory for the context in which they
example above, long-term memory would be the
were formed. Each piece of episodic memory has a
repository of the facts of addition, which would be
source tag associated with it, possibly including the time
needed to solve the mental arithmetic problem. Of
and place of memory formation and other details about
course, it stores much more than that. For example, it
the context. When retrieving an episodic memory, one
is the repository of all the words we know in our
can retrieve either the item itself given information about
language; of sensory information that we all have
the source, or the source, given information about the
stored for untold numbers of events (e.g., the taste of a
item. For example, you may recall where and when you

Neuroimaging Studies of Memory 2

purchased your current bicycle, or given the time and influence on the path of activity in the brain early in the
place, you may recall the features of the bicycle that processing sequence. The best example of this is the
you purchased. The other category of declarative visual system. Spatial information about a visual
memory is semantic: This type of memory consists of stimulus is selectively routed to a dorsal stream of
the vast store of facts and events that you have in long- information processing that mainly includes the parietal
term memory, regardless of whether you can retrieve lobes, whereas information about shape and other
when and where you learned them. For example, you nonspatial object-features of the same stimulus is
may remember the fact that bicycles can be mountain processed by a ventral stream in the occipital and inferior
bikes, racing bikes, hybrid bikes, and so on, yet you temporal lobes. Generalizing from this example, we can
may not be able to recall when or where you learned say that the nature of incoming information will
this semantic fact Procedural memories also influence the path of processing that the information
are of various sorts. There are skills, for example, takes in the brain. Beyond this, though, there is also an
such as riding a bicycle. There are classically influence of the task with which a person is faced, as
conditioned responses, which entail a previous pairing many different operations may be performed on any
of an unconditioned with a conditioned stimulus to given type of material. For example, one can process a
yield a conditioned response. And there are cases of word by noting its meaning or by noting whether it is
priming, in which a previously learned piece of printed in uppercase or lowercase letters. These very
information facilitates processing of some new piece different types of processing on the very same stimulus
of information. Psychological measures of priming, yield different patterns of activation in the brain, as we
such as decreases in response time to recognize a shall see below.
previously viewed word, indicate that a trace of the Once encoded, information is retained for some
previously learned piece of information is affecting period of time. Consistent with the fundamental
current cognitive processing—even if there is no distinction between working and long-term memory, the
conscious recollection of having seen the word before. length of the retention interval will in large part
Another important dimension of memory, determine which of these systems is most heavily
whether working or long-term, is the type of involved. Retrieval after short retention intervals—up to,
information being stored. As we shall see below, the perhaps, intervals as long as 30 seconds to one minute—
brain circuitry involved in a memory task honors the uses working memory. Retrieval of information stored
type of information that is stored and retrieved. for longer periods will under most circumstances
Perhaps the most frequently studied case of this necessitate the involvement of long-term memory
concerns the distinction between linguistic storage. Which memory systems is engaged will be
information (such as letters, words, sentences, and revealed by the circuitry that is activated. Working
stories) and visual or spatial information (such as a memory engages circuitry in frontal and parietal cortices
scene, an object, a face, or a spatial environment). most prominently, whereas long-term memory requires
There is by now ample evidence that the two involvement of frontal and parietal circuitry as well as
hemispheres of the brain are differentially activated by hippocampal and parahippocampal mechanisms.
these two types of information, with the left Just as encoding different types of material
hemisphere specialized for verbal information and the engages different mechanisms, storage of different types
right for visual or spatial information in most humans. of material also requires different mechanisms. This has
Types of processes. Memory entails three been demonstrated most handsomely in the contrast
cognitive operations: encoding, storage, and retrieval. between verbal and visual material, which respectively
These terms refer to the sequence in which memory activate left and right hemisphere structures
processes are thought to occur. Entering information predominantly. This distinction has been demonstrated
is first put into the proper internal code and a new for both working memory and long-term memory, as we
memory trace is formed (encoding). Encoding is shall see below.
followed by storage of the information for some Once encoded and stored, information in
period of time. This stage may include consolidation, memory can then be retrieved as needed. Suppose, for
or alteration of memory traces to make them last example, that you ask a person to memorize a list of
longer and be easily retrievable. Retrieval is the words. Retrieval can be accomplished in several ways.
process of reporting information from storage. You might simply ask the person to recall as many of the
The nature of encoding depends on two words as possible (free recall). Or you might guide
factors: the type of material that is involved in the recall by giving some of the words on the list as hints and
memory task and the task that is performed with that asking the person to recall the others (cued recall). Or
material. The type of material exerts a strong you might present the person with a longer list of words,

Neuroimaging Studies of Memory 3

some of which were presented on the original list and item-recognition task. In this task, participants are
some not, and the person has to decide which is which presented a small number of target items, typically
(recognition). Any of these procedures requires the randomly selected letters, to store for a retention
person to access the stored information in memory and interval of several seconds. Following this interval, a
produce an explicit response that depends on that single probe item is presented and participants must
stored information. For this reason, these are often decide whether this item was a member of the
called explicit tests of memory. However, there are memorized set. When participants engage in this task in
also implicit tests. Suppose, for example, that you PET and fMRI settings, there are a number of easily
presented someone a list of words and later flashed the replicable sites of activation compared to a control
same words and new words, one by one, so briefly that condition that does not require memory at all or in which
they were difficult to identify. If your subject were the memory requirement is minimal. One frequent site
more accurate in identifying words that had been of activation is in posterior parietal cortex, typically
presented on the original list than ones that hadn't more prominently in the left hemisphere than the right.
(which is what happens in this perceptual identification In addition, a set of activations appears in frontal areas,
situation), then one could conclude that the original including inferior frontal gyrus on the left, premotor
words were stored in memory even though no explicit cortex (more prominently on the left than on the right),
retrieval of them was ever demanded. The process of and supplementary motor cortex. These brain regions,
storage and use of information without explicit and all other major regions discussed throughout this
memory is called priming. Evidence from PET and chapter, are shown in Figure 2.
fMRI suggests that implicit and explicit tests of The frontal cortical areas that are activated in
memory recruit different brain areas, as reviewed this task are quite similar to those activated in a task that
below. requires making judgments of rhyming, a task that
With these preliminaries about memory in presumably requires producing a speech-like
place, we are now in a position to review what representation. So, it is likely that these frontal areas are
neuroimaging evidence has contributed to ones involved in rehearsal, which involves internally
understanding basic mechanisms of human memory. generating and regenerating a speech-like code for the
stored verbal material. The posterior parietal sites have
Working Memory been suggested as sites for the storage of verbal
information as well as for switching attention between
one item and another.
The canonical model of working memory is
The purported dissociation between the frontal
due originally to Alan Baddeley, and it is this model
and parietal sites is nicely supported by a study that used
that has been investigated in detail using neuroimaging
a different task involving verbal working memory, the 2-
methods. The model claims a fundamental distinction
back task. In this task, participants see a series of letters
between short-term storage of information and
presented at a pace of one every 2.5 seconds, and they
executive processes that manipulate this information.
must judge for each whether it matches in identity the
This general view is supported by the existence of
one that appeared two letters back in the series. This task
patients who have intact short-term storage, but
clearly requires storage and rehearsal of each letter, as
deficits in executive processes; this pattern of
well as other processes that we discuss below.
impairments contrasts with that of other patients who
Compared to a task in which participants must simply
have deficits in executive processing but intact short-
judge whether each letter in the series matches a single
term storage. Such a double dissociation suggests that
target (say, the letter "P"), the 2-back task produces
the circuitry of storage and executive processing are
activations in regions similar to the item-recognition
separable, and imaging studies have confirmed this
task. This is as it should be if both tasks involve storage
separability.
and rehearsal. Beyond this, though, the 2-back task has
Short-term Storage. The short-term storage
also been compared to another condition, one in which
of information in working memory appears to be
participants had to silently rehearse letters to themselves
accomplished via two mechanisms: one that retains
with little storage requirement (e.g., say the letter "P" for
information and another that "rehearses" that
3 seconds, followed by silently saying the letter "M", and
information in order to keep the memory traces active
so on). Subtraction of the activation in this Rehearsal
during a retention interval. This is perhaps best
condition from that in the 2-back condition revealed
illustrated for verbal information. A task that has been
much lower activation in the frontal areas. The Rehearsal
used frequently to study the mechanisms of verbal
condition is presumed to involve the explicit production
working memory in neuroimaging experiments is the
of silent speech. Subtracting the activations in this

Neuroimaging Studies of Memory 4

condition from those in the 2-back condition reduces part with allocating attention to several items in memory.
frontal but not parietal activation; therefore, one can However, the differences in activations suggest that the
conclude that the frontal activations in the 2-back and mechanisms by which information is stored and
other verbal working memory tasks must reflect an rehearsed may be different. Indeed, there is evidence of
inner rehearsal process as part of those tasks. These a similarity in circuitry between processes mediating
same frontal regions are also activated in tasks that spatial working memory and those mediating shifts of
require a recall response, so they are not unique to the attention to various locations in the visual field when
peculiarities of the item-recognition task or the 2-back stimuli are being perceived. This leads to the conclusion
matching task. that spatial rehearsal may amount to a successive
Just as we can identify the frontal sites used in allocation of attention to internal representations of
verbal rehearsal, we can also identify the parietal sites spatial locations, a process possibly mediated by
used in verbal storage. Evidence that the parietal sites premotor mechanisms near the frontal eye fields. This
are used in part for storage comes from a study in region, together with parietal cortex, may also play a role
which subjects memorized a set of nonsense letter in maintaining the representations of the spatial locations
strings (e.g., “MAVER”), and then kept these items in as well, a conclusion that is consistent with lesion
memory during a retention interval of some 50 studies and electrophysiological studies of monkeys in
seconds, during which they were PET scanned. After spatial working memory tasks. So, we can see that
the scan, they had to retrieve the items to be sure that although storage and rehearsal are common features of
they had been accurately stored. Scanning during just spatial and verbal working memory, they appear to be
the retention interval allows one to isolate storage implemented in the brain in different ways.
processes, or at least to concentrate scanning on Of course, visual information that is stored need
storage. One study using this procedure found not be spatial in nature. Features such as the shape of an
posterior parietal activations, leading to the conclusion object or its color are not spatial, even though they are
that these activations reflected storage processes, and visual. As described above, the brain honors this
not encoding or retrieval processes. distinction in simple visual processing, and indeed,
Storage and rehearsal should not be restricted neuroimaging research suggests that spatial memory and
to verbal information, of course, if they are general memory for other visual information are processed
properties of working memory as Baddeley supposed. differently in the brain, as well. One experiment that
Indeed, many studies have investigated the storage and demonstrates this used pictures of three faces presented
rehearsal circuitry used for spatial information as well. sequentially in three different spatial locations. After a
The clearest result of these studies is that the circuitry retention interval, a probe picture was presented in one
activated by spatial information in a working memory of the locations. When subjects were tested on their
task is quite different from that activated by verbal working memory for objects, they had to decide whether
information, even when the tasks are quite similar and the probe face was the same as any of the previous three;
only the material differs. For example, in an analog to when they were tested on spatial working memory, they
the item recognition task, subjects are presented a set had to decide whether the probe was in the same location
of dots on a screen and asked to store their locations in as one of the original faces. The elegance of this design
memory. Following a retention interval of several is that it involves the very same stimuli, and only the
seconds, they are presented with a single probe dot, nature of the memory task changes. The results show
and their task is to decide whether it appears at the that this change in task produces an important difference
same location as one of the locations they have stored. in brain activation: The object task activated regions of
This task has the same formal structure as the item- dorsolateral prefrontal cortex, whereas the spatial task
recognition task for letters, yet it yields activations activated a region posterior to this, in premotor cortex.
that are quite different. In common are activations in Beyond this, a meta-analysis of several spatial and object
posterior parietal and premotor cortex, although with a working memory tasks suggests that there is also a
tendency for greater activation in the right than the left dorsal-ventral difference in activation in posterior
hemisphere. However, quite different are activations cortex. Spatial working memory tasks activate more
in occipital cortex, superior frontal cortex, and dorsal structures in posterior cortex, while object
inferior frontal cortex, most prominently in the right working memory tasks activate more ventral structures.
hemisphere.
The common activations in parietal and Executive Processes. In addition to storage
premotor cortex between verbal and spatial versions of components, the model of working memory proposed by
the task suggest that there are some processes in Baddeley includes a component due to executive
common between the tasks, possibly having to do in processes. Although there is not yet a clear taxonomy of

Neuroimaging Studies of Memory 5

executive processes in hand, descriptions of them includes an inhibitory component, as described above. In
typically include: (a) focusing attention on relevant addition to this executive process, the letters that are
information and inhibiting attention from irrelevant stored in memory also have to be tagged by their order of
information; (b) scheduling processes in tasks that appearance so that the subject can keep in mind which
require multiple processes; (c) planning and one is 2-back, which 1-back, which 3-back, and so on.
prioritizing a sequence of steps to meet some goal; (d) Thus, the 2-back task must recruit an executive process
updating and checking the contents of working responsible for temporally tagging information, a sort of
memory; and (e) coding internal representations for short-term episodic memory requirement. Indeed, the 2-
time or place of occurrence. All of these processes back task shows evidence of activations in dorsolateral
involve manipulation of information that is temporarily prefrontal cortex in addition to other sites that may well
stored in working memory. Research on executive be responsible for temporal tagging. The dorsolateral
processes using neuroimaging techniques has revealed prefrontal activation that arises in this task seems to be a
a heavy contribution of frontal mechanisms regardless common broad site of activation in many tasks that
of the executive process in question. require manipulating the information stored in working
As an example, recall the verbal item- memory, and so this leads to the general conclusion that
recognition task. In that task, subjects are presented a prefrontal mechanisms may be responsible for a wide
set of letters that they have to retain for several array of executive processes.
seconds, after which they have to decide whether a Summary of Working Memory. Overall the
probe letter matches one of the letters in memory. neuroimaging research concerned with working memory
Several recent studies have introduced an inhibitory has reliably revealed a set of structures that may be
component in this task in the following way. Trials important for storage, rehearsal, and executive
were included in which the distractor probes (probes processes. Posterior parietal mechanisms have been
that did not match an item in the current memory set) implicated in the storage of verbal material, and
were letters that did match a letter in the memory set prefrontal ones concerned with language processing have
from the previous trial. Thus, these probes were been implicated in the rehearsal of stored verbal
relatively familiar because they had recently been material. For spatial material, the sites of storage and
memorized. This design creates a situation in which rehearsal are different, but nonetheless, one can
participants have a sense of familiarity about the probe conclude that there are storage and rehearsal processes
item, but they must remember that it doesn’t match the for nonverbal material as well, but that these may be
memory set on the current trial. On such trials, implemented via non-linguistic mechanisms. Finally,
subjects take longer to give a “no match” response. various sites in prefrontal cortex, most prominently
Both PET and fMRI studies show that there is a site in dorsolateral prefrontal areas, have been documented in
left lateral prefrontal cortex that is activated on these the mediation of executive processes. Thus, the
trials, and the activation occurs most prominently at psychological architecture proposed by Baddeley in his
the time the probe is presented. Furthermore, older model of working memory seems amply supported by a
subjects, who show a greater interference effect on brain architecture that may honor the same distinctions
these trials, also show less activation at this left lateral among processes.
site. And patients with damage to this area show a
dramaticaly increased interference effect compared to Episodic Memory
patients with damage elsewhere in frontal cortex.
As described above, episodic memory can be
Taken together, this evidence suggests that the left
defined as memory for information that is associated
lateral site is involved in resolving the conflict
with a time and place of occurrence. Take as an example
between familiarity and source memory that arises on
a semantic fact: you may know that the turn of the
these trials.
century French impressionist painter Claude Monet lived
Another example of a task in which executive
and worked for many years in his provincial home at
processes interact with storage processes is the 2-back
Giverny. This fact is in the domain of semantic memory.
task described above. Recall that in this task, single
However, your memory of learning this fact in your art
letters are presented in succession, and subjects must
history course would be an episodic memory. Episodic
judge whether each letter matches the one two earlier
memory is often studied in a controlled laboratory
in the sequence. To succeed at this task, one has to not
setting using recognition or recall tasks, described in the
only store the recent stream of letters, but one also has
introduction to this chapter. These tasks require memory
to update this stored set as new letters are presented,
for a source code (e.g., time or place of occurrence) that
dropping older letters and adding newer ones. This task
is the essence of episodic memory. In the context of
is similar to the item-interference task in that it

Neuroimaging Studies of Memory 6

neuroimaging, the encoding and/or retrieval phases of shown hippocampal activity. Recent neuroimaging
these tasks are scanned using PET or fMRI and then evidence has shown, however, a selective response of the
compared to a control task with a diminished or absent hippocampus to novelty. In one experiment, participants
demand on memory. were shown pictures of indoor and outdoor scenes while
These studies have identified a set of regions in the MRI scanner. They were required to judge whether
underlying episodic memory. These include medial each scene was an indoor scene or an outdoor scene and
temporal structures, such as hippocampus and remember the pictures for a later test. In some scans, the
parahippocampal areas, prefrontal cortex, anterior same two pictures were repeated many times so that
cingulate cortex, cerebellum, and parietal and superior participants became very familiar with them. During
temporal association cortices (shown in Figure 2). other scans, the scenes were entirely novel and
Important hemispheric, regional, and functional unfamiliar. Comparing the unfamiliar scans to the
differences exist, however, between the encoding and familiar scans showed activity in parahippocampal gyrus
retrieval phases of episodic memory. In addition to bilaterally. Given this fact, it would seem that the medial
exploring these differences, neuroimaging studies have temporal lobe is particularly responsive to novel
also begun to examine cases in which this system stimuli—a finding consistent with the intuition that most
performs inadequately. episodic memory encoding occurs on the first
Episodic Encoding. As discussed above, presentation of new material.
memory entails three important general stages: The function of left prefrontal cortex appears to
encoding, storage, and retrieval. At the encoding stage, involve processing the context (or source) in which new
processes must be involved that create an internal code information is learned. An event-related fMRI
for a piece of information, and then attach a context (a experiment has studied the different functions of left
place or time) to the new memory. prefrontal and hippocampal mechanisms in episodic
Several neuroimaging studies have scanned memory. Event-related fMRI allows the examination of
participants while they perform some task to encode a areas of the brain that are active in response to different
set of items. For example, participants might be asked events occurring within the context of a single cognitive
to make a judgment about whether a word represents an task. In this experiment, participants were required to
animal or vegetable, an encoding task that requires learn word pairs in which the first word served as
access to the semantics of the word. Alternatively, a semantic context for a second word, for example
subject might simply be asked to memorize a set of “athlete-boxer". Participants were presented with several
items and be tested on them later. Subsequent testing of these word pairs during each scan. Sometimes the
of the items confirms whether subjects have context for a word would change, as in “dog-boxer”.
effectively encoded the items. These studies show that Other times the word would change as in “dog-labrador”.
encoding involves left prefrontal cortex, hippocampus, Both word and context could also be new, or both could
parahippocampal cortex, anterior cingulate, and some be old. This design permitted independent manipulation
superior temporal cortex. Further experimentation, of the novelty and the context of the item to be learned
including converging evidence from neuropsychology during encoding. The hippocampus was active when
and other experimental paradigms, has begun to either the context word or related word was new, and it
examine the role of each of these regions and their was most active when both were new. This corroborates
relationships to one another. the idea that the hippocampus is involved in processing
The hippocampus and surrounding areas have novel items. The left prefrontal cortex was most active
long been associated with memory. Evidence from when an old context was attached to a new word or a new
both animal studies and studies of brain-damaged context to an old word. This finding suggests that the
patients has shown that damage to the hippocampus can prefrontal cortex is involved in representing the context
result in amnesia, one form of which is caused by of the item to be remembered, a function that is critical
damage to medial temporal structures such as the for episodic memory.
hippocampus and parahippocampal gyrus. Though We can test whether the effectiveness of
amnesics are typically able to retrieve memories from encoding is related to the brain activations that reflect
their distant past, they show a profound deficit in the encoding by varying what is called the depth of
ability to form new memories, a phenomenon known as processing participants apply to material. It is well
anterograde amnesia. For this reason, the hippocampus known that evaluating the semantic content of material
is thought to be involved with the encoding and (‘deep encoding’) leads to more elaborate processing and
consolidation of long-term memories. a longer lasting memory trace than evaluating the
In line with this, many neuroimaging studies physical features of material (‘shallow encoding’). One
using the encoding paradigms described above have experiment that takes advantage of this effect required

Neuroimaging Studies of Memory 7

subjects to judge whether a word was abstract or during the study phase. Hence, it is necessary only to
concrete (deep encoding) or whether it was printed in access the source and not the item as well. In recall
upper or lower case characters (shallow encoding). tests, it is necessary to generate the item as we ll.
Deeply encoded words were remembered better than Neuroimaging studies of both recall and recognition
words encoded shallowly, replicating previous typically show activity in right prefrontal cortex,
behavioral results. When the two conditions were hippocampus, medial as well as inferior parietal cortex,
compared, it was found that there was greater activity anterior cingulate and cerebellum. There are some
in hippocampus and left prefrontal cortex for deep important variations in this pattern, however, that are
encoding, suggesting that both areas are more discussed below.
vigorously involved with deep than shallow encoding. The hippocampus is typically considered to be
This result by itself does not indicate that involved in the consolidation of long term memories, as
more activity in prefrontal cortex and hippocampus discussed above. Although this function implies that the
produces better behavioral performance; it only hippocampus should not be involved in retrieval, some
indicates that depth of encoding and activation are studies have found it activated during retrieval tasks. To
correlated. To address the performance question, test whether the effort required for retrieval might
several studies have directly examined the relationship influence activation of the hippocampus, one study varied
between performance on retrieving an individual item the amount of effort required to search memory. In a
in memory and brain activation while encoding that “high-recall” condition, words were deeply encoded and,
item. After being scanned, the participants had to hence, less effortfully retrieved. When the recall phase
recognize the encoded items, and they were grouped by was scanned, this manipulation revealed activity in the
whether they were correctly or incorrectly retrieved, hippocampus bilaterally, supporting the view that the
an indication of good or poor encoding. This hippocampus is involved in effortless, conscious recall.
comparison revealed activity bilaterally in In a “low-recall” condition, words were encoded more
hippocampus and in left prefrontal cortex. Hence, it superficially, and hence, required more effortful
would seem important that for effective encoding, not retrieval. Scanning during this more effortful recall phase
only must the information be consolidated effectively showed bilateral prefrontal but not hippocampal
by the hippocampus; it is also important that the activation. Finding that the prefrontal cortex is involve d
prefrontal cortex assist in processing the context. in effortful retrieval is consistent with the view that one
To summarize, encoding recruits a set of function of the prefrontal cortex is to implement
regions that include left prefrontal cortex, retrieval strategies. The hippocampus, by contrast, may
hippocampus, parahippocampal gyrus, parietal cortex, be involved in relatively more automatic retrieval.
and anterior cingulate (as shown in Figure 2). These Certain neuroimaging studies of episodic
regions appear to be involved in tranforming retrieval have found not only increased activity in right
information into a mental code in the brain which can prefrontal cortex but decreased activity in other areas
later be retrieved. Two processes entailed by this task such as left prefrontal cortex. Based on this effect, some
are the consolidation of a novel item by the have suggested that episodic retrieval is not just an active
hippocampus and processing its context by the left process of search and retrieval, but involves the active
prefrontal cortex. The extent to which the information inhibition of certain regions of the brain by other areas
being encoded can be effectively recovered at a later of the brain. By this model, right prefrontal cortex could
time is strongly dependent on the depth of encoding be actively inhibiting left frontal regions as well as
which seems to have an effect on the activity of the inferior temporal regions, areas that sometimes show
hippocampus and left prefrontal cortices. deactivations in retrieval tasks. In the case of the
Episodic Retrieval. Retrieval of episodic temporal regions, for example, this might indicate the
memory is mediated by regions that are generally suppression of language processes during episodic
functionally and anatomically distinct from those used retrieval. This effect has been termed "ensemble
in encoding. Most neuroimaging studies of retrieval inhibition", and suggests that episodic retrieval may be
use a similar task design as used in studies of carried out, in part, by inhibitory processes.
encoding, in which participants must study a set of Retrieval processing involves an interplay
items and are subsequently tested for their memory of between the right prefrontal cortex and hippocampus in
the items. The difference is that participants are the implementation of search strategies and conscious,
scanned while they retrieve (recall or recognize) rather effortless retrieval respective ly. The involvement of
than while they encode the material. In recognition other areas of the brain such as the precuneus, parietal
tasks, an item is shown, and it is the task of the cortex, anterior cingulate, and cerebellum have yet to be
participant to indicate whether that item was presented fully elucidated, so much further research is required on

Neuroimaging Studies of Memory 8

this problem. Hippocampal activation has been found unilaterally on
Synthesis: The HERA model and its the right and left, and bilaterally in tasks of both encoding
extension. There appear to be stable differences in the and retrieval. The pattern of activity in the hippocampus
activations accompanying encoding versus retrieval. is best described as being dependent on the type of
The most striking pattern is in the activity of the material being encoded or retrieved, not on encoding and
prefrontal cortex. Most studies of encoding have retrieval by themselves. This is shown by systematic
shown activity in left prefrontal cortex, at a more patterns of activation depending on whether verbal or
anterior site. Whereas, most studies of retrieval have nonverbal stimuli are used in an experiment. Most
shown activity in right prefrontal cortex, also at a more experiments using verbal information have shown
anterior site. This hemispheric difference in predominantly left hippocampal activity. In contrast, the
prefrontal activity in episodic memory is typically right hippocampus or both hippocampi may be more
referred to as the Hemispheric Encoding/Retrieval active in encoding visual information. For example, a
Asymmetry model, or HERA. study in which people retrieved information about a
Other areas of the brain have also been spatial route through a town activated bilateral
included recently in the HERA model. For example, hippocampus. It should be noted as well that the material
the left cerebellum seems more active during retrieval specificity of activations extends to prefrontal cortex.
than during encoding. The cerebellum’s anatomical One area of prefrontal cortex often observed in studies
connections are predominantly with the contralateral of episodic memory does not follow HERA, but rather
prefrontal cortex (via the thalamus), so the coupling of depends on whether the stimulus material is verbal or
right prefrontal and left cerebellar activations is not visuospatial, the former producing activation on the left
surprising. What function the cerebellum might be and the latter on the right. So, even within prefrontal
serving in the context of episodic retrieval is unclear. cortex, one region obeys the description given by the
The cerebellum has long been associated with motor HERA model and another does not.
coordination and visuomotor skill learning. It is Overall, it does appear that many patterns of
possible that the cerebellum is serving one of these activity demonstrated in tasks of episodic memory
general roles in effortful retrieval, but its exact role follow an asymmetric hemispheric pattern in regard to
remains to be elucidated. encoding and retrieval. In prefrontal cortex, and in
The association cortices have also gained temporal and parietal association areas, activity in the
some attention in regard to the HERA model. The left left hemisphere is associated with encoding processing
temporal cortex has been found to be activated in some and activity in the right hemisphere is associated with
studies of encoding while activation in right or retrieval processing. The cerebellum also shows a
bilateral parietal cortex has been documented during hemispheric asymmetry, but this is the reverse pattern,
retrieval. These findings, though not entirely with left cerebellum engaged during retrieval. There are
uncontroversial, seem to follow the HERA pattern, exceptions to this pattern, and these are seen in the
with encoding being a left hemisphere function, in this hippocampus, posterior regions, and some anterior
case in the temporal lobe, and retrieval being a right regions of prefrontal cortex as well. The patterns of
hemisphere function, in parietal cortex. There is a activity in these regions are dependent on the modality of
great deal of speculation as to what these areas are the information being processed—verbal information
doing exactly. Some accounts claim that they are lateralized to the left hemisphere and visuo-spatial
involved in some way in the execution of special lateralized to the right—rather than encoding and
encoding or retrieval strategies. In the case of the retrieval processes.
temporal cortex, this might be attaching some kind of False memories. Memories of our personal
mneumonic code to items to help ease of retrieval experiences are extremely vulnerable because memory is
later on. Others suggest that parietal cortex is involved often not reconstructive but constructive. When
in perceptual aspects of retrieval, such as mental retrieving an episodic memory, we try to reproduce the
imagery. Further study is necessary to fully understand event as closely as possible, constructing the most
the functions subserved by these regions, as well as the plausible approximation. Consequently, we often
way that they interact with the other areas of the brain incorporate aspects of the event that are close to the
that are active during episodic encoding and retrieval. original but not exactly it, and we can even insert
It must be noted that there are important information that never occurred at all. This vulnerability
exceptions to the HERA model’s general description can even go so far as to produce elaborate situations that
of the patterns of brain activity during tasks of episodic never actually happened, though the person might swear
memory. The model does not do well in predicing that they did.
patterns of activation in the hippocampus. Neuroimaging studies have begun to examine

Neuroimaging Studies of Memory 9

differences in brain activation comparing retrieval of a most readily studied and because it is more difficult,
true past event from false memory for an event that did given the normal course of learning, to study encoding or
not occur. Most of these studies have used a task in storage of semantic memory.
which participants are shown lists of words to study. Verbal Semantic Memory. Many of the
After a long retention period, the participants are asked concepts that make up our semantic knowledge are coded
to perform a recognition task, indicating which words in the form of language, probably because we are such
on a second list were present on the first list. Some of intensely linguistic creatures. These concepts come
the words on the second list that were not present on from various categories, of course, such as living things
the first list, called foils, are semantically related to and nonliving things, distinctions that we can readily
the words on the first list. For example, the words make for many concepts. There is evidence from
“pajama, bed, night” might have appeared on the first patients with focal brain injury that the brain seems to
list, but the word “sleep” might appear at the time of honor some of these categorical distinctions among
the recognition test. Semantically related foils were concepts. For example, there are patients who appear to
often falsely recognized as having appeared on the have lost their ability to identify living things, such an
originally studied list. Furthermore, participants often elephant or a flower, even though they are still capable of
rated their confidence that the words had been on the identifying nonliving things, such as tools. One
original list as highly as they rated their confidence interpretation of this result is that the brain’s
that actual words had appeared. Thus, it appears as if organization of semantic memory is, in part, organized by
the participants had created false memories of broad categories. To test whether this is so in normal
semantically related foil words. adults as well as brain-injured adults, one study used PET
Neuroimaging studies have compared to examine what areas of the brain are active during the
activations due to falsely recognized words to retrieval of three different semantic categories.
activations due to correctly recognized words. One Participants were given several scans in each of which
difference that emerges is activation in frontal cortex they were asked to name photographs of either famous
during true recognition. This is consistent with other people, animals, or tools. As predicted, different brain
retrieval studies, as reviewed above. Another feature activations resulted from naming each kind of stimulus.
of activation is that words on the recognition test that Naming famous people produced activity in the most
had actually been presented sometimes caused anterior part of temporal cortex, called the temporal
activation in the primary sensory cortex of the pole. Naming animals produced activity in a more
modality in which they had appeared. For example, a posterior area of temporal cortex in inferior and middle
word presented aurally, when tested at the time of temporal gyri. Naming tools showed activity in an even
recognition, might show activation in superior more posterior portion of the inferior temporal gyrus. In
temporal cortex. By contrast, words that did not general, the more specific the item that had to be named,
appear showed no such sensory cortex activation. the more anterior the activation in temporal cortex.
Thus, there is apparently a neural signature that permits These findings are consistent with the notion of a visual
one to distinguish actually presented words from processing stream that spreads from occipital cortex into
semantically related foils, even if the subject does not temporal cortex, moving from general classification to
access this signature in her recognition judgments. more specific categorization in anterior temporal cortex.
Semantic Memory A related PET study of object and face naming
Episodic memory is distinguished by the fact provides corroborating evidence for these findings and
that it requires not only the retrieval of an item from further distinguishes between activations involved in the
memory but also a source or context for that item. But identification of specific faces versus simply
many times, we retrieve a fact with no knowledge of its recognizing stimuli as faces. In one condition,
context, as when we can identify various types of participants were asked to make gender discrimination
bicycles, without knowing when and where we learned judgments of familiar and unfamiliar faces. Relative to a
about the various types. Semantic memory can be control condition, this task produced activations broadly
defined as memory for facts about the world, naked of in extrastriate occipital cortex. Only when participants
their source context. This kind of knowledge plays a identified faces of famous people, and so had to make a
critical role in all forms of cognition, from language to specific face identification, were the temporal poles
reasoning to problem-solving. Hence, semantic activated. Also activated were other structures in
memory is an important topic for study. Most studies temporal cortex (fusiform gyri, right lingual and
of semantic memory have focused on the retrieval of parahippocampal gyri, and left middle temporal gyrus) as
semantic information from memory because this is well as orbitofrontal cortex.

Neuroimaging Studies of Memory 10

 Object naming in this study shared some generation of semantic concepts. Further study of this
common areas of activation with face naming, verb-generation situation compared a task in which
including orbitofrontal cortex, left middle temporal participants had to name a verb for each noun presented
gyrus and left fusiform gyrus. Converging evidence to two control conditions—reading words and passively
from animal and lesion studies may shed light on the viewing words. Multiple subtraction conditions were
roles of these regions. Neuropsychological data used to identify areas related to the motor execution of
suggest that the left middle temporal gyrus may be speech (motor cortex), word reading (left insula), and
necessary for naming, but not recognizing, objects and verb generation (left frontal cortex, anterior cingulate,
faces. Lesions of orbitofrontal cotext have been and right cerebellum). This study also revealed changes
related to visual memory impairments in animals. in activation with practice on this task, as reviewed
Finally, the fusiform gyri appear to be involved in below. The constellation of regions that were activated
recall of both faces and objects, with face recognition in this study probably included a complex combination of
activating the right gyrus and object recognition the areas involved in attention, inferential reasoning, willed
left. Interestingly, the fusiform gyrus has also been action, episodic encoding, and working memory, but
implicated in the perception of faces and objects, so most prominently, there was significant activation in left
the regions responsible for semantic memory for this prefrontal cortex and elsewhere that could be attributed
type of information may be similar to the region used to semantic retrieval.
to perceive it. This hypothesis is supported by
intracortical electrode studies done on patients, in Cognitive Skill Learning
preparation for possible surgery to treat epilepsy.
Investigating practice-related changes in the
These recordings showed that face recognition elicited
verb-generation task provides a convenient segue into a
electrical acitvity in the fusiform gyrus, and that
discussion of skill acquisition—another vital aspect of
electrical stimulation in this same region resulted in an
memory. After 15 minutes of practice on the verb
inability of the patients to name a face for the duration
generation task, 90% of the verbs that participants
of stimulation.
generated were rote responses that had been consistently
These complementary results suggest that
associated with the nouns. Participants no longer had to
face perception and recognition share a common
search through memory for a novel association; instead,
substrate, and that the boundary between perception
they could quickly recall a response they gave previously.
and semantic memory may be indistinct for this type of
Analysis of the difference between the PET images early
material. Studies of object naming more generally
and late in the verb generation task showed that with
show that semantic memory about concrete objects
practice, activity decreased in the anterior cingulate, left
appears to be organized, at least to some degree, in
prefrontal cortex, bilateral inferior frontal gyrus, left
cortical modules devoted to particular types of
temporal cortex, and right cerebellum—the very areas
remembered material, and these become more specific
that were active when comparing verb generation to word
moving from posterior to anterior brain regions.
reading. In fact, after practice the PET images were
Semantic memory for words must take on
indistinguishable from those of word reading. Increases
more than just a simple recognition and naming
in activation with practice were observed in the
function. Indeed, when faced with an object it is often
precuneus and cuneus on the medial wall of the posterior
more useful to know what can be done with that object,
cerebrum, and in right superior parietal cortex. These
than just what the name of the object is. This type of
shifts in activation could be due to decreased demand on
information, as well as information about cosntructs
attention and effort, decreased searching of semantic
that are not concrete, are within the domain of
memory, or some other factor.
semantic memory and have been directly studied.
Without further evidence, it is hard to
To examine this sort of semantic memory, a PET
distinguish among these. Certainly there is evidence that
experiment was designed that required participants to
some of these areas are involved in other cognitive
generate an associated verb for each word in a list of
processes. For example, the left prefrontal cortex is
nouns. For example, shown a picture of an apple, a
activated in verbal working memory tasks and in the
participant might respond, "eat". This experiment
encoding of long-term memories, as we reviewed above.
revealed a preferential role of lateral and inferior
The anterior cingulate is also activated in some
frontal cortex in the generation of verbs associated
overpracticed motor tasks, particularly when they might
with visually presented objects. It has been suggested
require attending, making inferences about a pattern, or
that verbs are at the core of semantic structure, and
anticipating future stimuli or feedback. One region that
hence activation in frontal cortical areas might be an
may have a clearer role in the practice effect seen in the
indication of which areas are critical in mediating the

Neuroimaging Studies of Memory 11

verb-generation task is the insula, an area located memory. This study suggests that different learning
anatomically near the region responsible for speech regimens may have a profoundly different effect on the
output, and an area that showed increased activation in brain circuitry recruited to the task, at least for a task
the verb generation task with practice. This activation requiring categorization processes.
may be an indication of increasing automaticity in Learning Procedural Skills
producing verb-associations given nouns as stimuli, a Procedural knowledge, as described above,
kind of stimulus-response connection that developed consists of knowledge of how to do something. It
even over the course of relatively little practice. includes all the behaviors shown as examples of
Of course, one might ask whether skill in the procedural knowledge in Figure 1, all of which share in
verb generation task is a good example of cognitive common that they do not require explicit retrieval of
skill learning in general. The shifts in activation in this information; rather, they require the person to tap
task seem to result not from an improved ability to memory in the service of some other task, such as riding
generate novel verbs, but rather from the ability to call a bike taps memory to coordinate the muscles in various
up from memory the same verb the subject gave on the ways so that balance can be maintained. This is often
last trial, a kind of automatic stimulus-response called “implicit” memory. Viewed this way, it is clear
mapping. It is not yet understood how true cognitive that a vast number of motor skills are mediated by
‘skills’, such as the ability to make inferences and procedural memory. Of course, procedural memory
manipulate abstract concepts—are learned, except that must develop over the course of practice, and the
their appearance in children seems to parallel changes that occur with practice are often highly
development of the frontal lobes. However, a great specific, improving performance on precisely those tasks
deal of what we normally consider to be cognitive we practice. Although this may seem to limit our
skills, such as expertise in chess, can be explained as behavioral repertoire, in fact, once we have learned a
the formation and retrieval of ever larger and more sequence of motor movements, we can quickly learn to
complex sets of associations in semantic memory. adapt these movements to similar situations.
A final example of cognitive skill learning Skill learning occurs when a new movement or
comes from a PET study of categorization. sequence of movements is acquired, and performance
Experimental studies have shown that people learn to becomes both faster and more accurate with training.
categorize objects in several ways: through application Sometimes, this learning might involve the acquisition of
of rules, learning of specific exemplars of a category, new sensory-motor mappings. Studies of motor skill
and implicit learning of an average or ‘prototype’ of learning have focused on either the acquisition of some
the category. Patients with medial temporal damage, new sequence of motor movements, such as a sequence
who have virtually no remaining episodic memory, fail of finger taps, or on the acquisition of some sensory-
on rule and exemplar-based categorization but learn motor mapping, such as guiding a visual cursor with
prototype-based categorization as readily as do normal joystick movements.
participants. In the study, people classified pictures of A note: it is often difficult to separate the brain
contrived animals based on previous practice with areas involved in learning a skill from those involved in
similar (but not identical) animals. One practice group other aspects of processing that change along with skill
learned to categorize animals by a rule, and the other learning. As people learn a skill, for example, they
group learned the categories by trial and error. The devote less attention to the task, and so neural circuitry
rule group categorized the new animals presented responsible for focusing attention is less involved. At
during scanning by applying the rule. The trial-and- the same time, the incidence of errors decreases, so the
error group categorized animals during PET scanning brain activity related to error detection and correction
based on their similarity to the animals they saw during will decrease. The brain regions responsible for
training, an exemplar-based strategy. Only rule-based mediating task performance may also change as skills
categorization activated bilateral parietal cortex, right become automatic. All these changes obscure the
prefrontal cortex, and bilateral supplementary cortex, interpretation of neuroimaging studies of the learning
possibly reflecting greater working memory demands, process by making it difficult to determine which
attention shifting, and retrieval and application of the activations are due to skill-learning per se and which are
rule. Exemplar-based categorization activated left due to other processes.
extrastriate cortex and left cerebellum. The Motor skill learning. Execution of motor
extrastriate activation may reflect greater use of a tasks such as tapping your fingers in a particular
perceptually based memory trace in the exemplar- sequence or maintaining contact between a target moving
based strategy, consistent with the involvement of in a circular pattern and a hand-held marker activate large
extrastriate visual cortex in studies of implicit

Neuroimaging Studies of Memory 12

regions of cerebral cortex and subcortical structures, other structures, including changes in primary motor
including sensory motor cortex (primarily the primary cortex, in more depth. Unpracticed performance
motor cortex), supplementary motor area (SMA), activated a network of areas often associated with the
premotor area (PMA), putamen, cerebellum, and planning and execution of movements: contralateral
sensorimotor thalamus (see Figure 2). Traditionally, primary motor cortex and putamen, bilateral PMA and
and consistent with data on the anatomical layout of the SMA (with more activity on the contralateral side) and
motor system, neuroimaging studies have found cerebellum. With practice, activity decreased in the
activations in these areas on the side contralateral to lateral portion and deep nuclei of the cerebellum,
hand movement (with ipsilateral activation of the supporting the view that the cerebellum is important in
cerebellum because of its crossed connections to sequence learning and may be less important in execution
cortex). However, recent evidence indicates that of highly learned sequences.
complex movement of even one hand can activate Subsequent studies examined activations during
motor areas bilaterally. For example, in one a sequential finger-tapping task in which participants had
experiment, participants rotated two metal balls at a to learn the correct sequence by trial and error. The first
constant rate in either their right or left hand. These study compared activations between new sequences,
complex movements activated the regions mentioned learned sequences, and a resting control. Similar to the
above, including significant bilateral activations in the earlier studies, performance of learned sequences with
postcentral gyrus, traditionally considered primary some degree of automaticity activated contralateral (left)
somatosensory cortex, and intraparietal sulcus. primary motor cortex, PMA, SMA, putamen, and bilateral
Notably missing from this list of activations is the cerebellar hemispheres, vermis, deep cerebellar nuclei,
basal ganglia, which appear to be preferentially anterior cingulate, parietal cortex, and ventrolateral
activated during performance of learned sequences of thalamus. Of course, as sequence performance was
movements. compared with a resting condition, this network includes
Areas involved in the learning of motor skills areas reponsible for processes irrelevant to sequence
are largely the same areas as those used during task learning. When compared to rest, performance of new
performance. However, some regions become active sequences showed, in addition to these areas, increases
only during initial learning, or only after performance in prefrontal cortex and more extensive activation of
has become automatic and requires little effort. These cerebellum. When compared to the practiced sequence
changes are specific to the type of task—they vary directly, learning of a new sequence produced activations
depending on whether automatic performance involves in bilateral PMA, cerebellum, anterior cingulate,
increased or decreased use of sensory cues and prefrontal cortex, and medial thalamus. It seems likely
whether learning involves learning a new movement, a that the requirement to infer the correct sequence based
sequence of movements, or a sensory-motor on error feedback was responsible for the activation of
association. the anatomically interconnected prefrontal-anterior
Sequence learning. Among the first motor cingulate-medial thalamus network. However, as we will
learning tasks studied is the sequential tapping task, in see later, it is possible that the cerebellum also
which participants must tap the fingers of their contributes to error detection and correction.
dominant hand (all studies reviewed here used right- In another revealing manipulation, participants
handed subjects) in a pre-specified sequence. An early were asked to pay attention to their finger movements
study scanned participants doing sequential finger while performing a highly learned sequence. Attention
tapping at three stages in the learning process. The resulted in the reactivation of the anterior cingulate and
three levels corresponded to an initial learning phase prefrontal cortex. Comparing learning of a new sequence
when a skill is first being learned, the phase when a with a control condition that required a similar level of
skill becomes automatic after significant practice, and attention and similar decision and motor processes, they
skilled performance after performance level has found activation of the caudate nucleus and the
reached its asymptote. The ipsilateral (right) cerebellum. This result indicates that these two
cerebellum was activated in all three conditions, and structures may be important in learning a new sequence,
the activation per movement in the cerebellum as opposed to other task-related processes. Together,
decreased as training progressed. The striatum was these studies suggest that the basal ganglia and
also activated during advanced practice, suggesting a cerebellum, and possibly the PMA, are involved in skill
role for the basal ganglia in the development of learning, whereas anterior cingulate and prefrontal areas
automaticity. are involved in attention and higher-level control
Subsequent studies of sequential finger processes.
tapping have examined the role of the cerebellum and

Neuroimaging Studies of Memory 13

 A more controlled version of the tapping task increased activation in the right ventrolateral frontal
is the Repeated Sequence Task. In this task, cortex. Explicit versus implicit knowledge of newly
participants see a cue appear above one of four learned sequences resulted in increased activation of the
squares, and they must touch that square as quickly as right cerebellum and left ventrolateral frontal cortex.
possible. As a particular sequence appears more The sequence-learning studies reviewed here
frequently, participants become faster in pressing the implicate the cerebellum and basal ganglia in initial
appropriate squares. The faster responses for the learning and automatization of new skills. Three studies
learned sequence indicates that implicit learning of the found significant activity in the basal ganglia with
sequence has occurred, even though participants have practiced, but not novel, sequences. Two other studies
no explicit, declarative memory for the sequence. found basal ganglia activity in newly learned sequences,
Doyon and coworkers, using this paradigm, one relative to a resting control and the other relative to a
compared PET activation among several conditions, free-selection tapping task. The cerebellum appears to
including different amounts of learning, and included a be involved in early learning of the sequence, but it is
condition in which subjects were given explicit also active during skilled performance relative to rest.
knowledge of the sequence prior to scanning. Its activity may increase as participants gain skill if that
Performance on highly learned ve rsus random skill involves making speeded responses to visual cues.
sequences resulted in increased activation in the right Notably, none of these PET studies showed
(ipsilateral) ventral striatum, right cerebellum, anterior changes in primary motor cortex as a function of
cingulate bilaterally, right medial parietal cortex, and practice. However, some researchers have shown that
right extrastriate cortex. Decreases were found in the skill learning produces nonmonotonic changes in the
ventrolateral frontal, frontopolar, and lateral parietal strength and spread of activation within primary motor
cortices, all on the right side. As with previous cortex. These researchers scanned participants learning
studies, basal ganglia activity increased when one of two similar finger tapping sequences using fMRI.
performance was highly learned. These changes are Initially, a habituation effect to sequence performance
consistent with recent animal models suggesting a role was found: activity in primary motor cortex was lower
for the basal ganglia in performance of movement when it was performed later in the scanning block. After
sequences. 30 minutes of practice, activity in primary motor cortex
When compared with newly learned was higher when it was performed later in the scanning
sequences, highly learned sequences showed increased block, but only for practiced sequences—possibly
activity in the cerebellum, suggesting a role for the reflecting fast learning processes that set up later
cerebellum in sequence performance or in the consolidation of a new motor sequence. After four
development of automaticity. This finding contrasts weeks of daily practice, more areas in primary motor
with previous sequence learning studies, which found cortex were recruited during performance of the learned
that cerebellar activity decreases as performance sequence, showing that practice resulted in a true
becomes automatic, so further research is necessary to expansion of the cortical area recruited in primary motor
investigate the role of cerebellum. It is possible that cortex.
the cerebellum has multiple roles and plays a part both Learning sensory-motor associations. What
in initial learning and in later retrieval of sequences. are strikingly missing from analyses of sequence
An alternative explanation for cerebellar activity in the learning, but apparent in other motor learning tasks, are
learned – unlearned comparisons in these studies is changes in activation of premotor and supplementary
that the cerebellum is required for speeding up motor cortices. Studies of sequential finger-tapping
movements while maintaining a criterion level of primarily require the learning of associations between
accuracy. In the early stages of training, it may be movements that form a sequence. This kind of learning
difficult to keep movements in time, resulting in may be fundamentally different from learning that
cerebellar activation that decreases as it becomes requires improvement in coordination or learning an
easier to perform the task at the requisite speed. This altogether new motor movement. In addition, the finger
hypothesis highlights the fact that neuroimaging results tapping task requires a series of internally generated
may be interpreted in multiple ways. movements rather than movements elicited by sensory
Newly and highly learned sequences in the cues. We have already seen that learning in the Repeated
Repeated Sequence Test were also examined before Sequence Test, which involves visuo-motor associative
and after participants were given explicit knowledge of learning, may produce a different pattern of cerebellar
the sequence. Explicit knowledge of highly learned activity than learning of internally generated movement
sequences relative to implicit performance decreased sequences. Perhaps examination of non-sequence motor
signal in the right (ipsilateral) cerebellum and learning, including studies of rotor pursuit, trajectory

Neuroimaging Studies of Memory 14

movements, joystick movements, and maze tracing the maze studies, initial performance relied heavily on
may help resolve discrepancies. somatosensory feedback from the cutout maze, and so
Rotor pursuit involves maintaining contact PMA was heavily recruited. After the maze was well
between a hand-held stylus and a target moving in a learned, a coordinated, internally generated movement
circular pattern. Several studies have compared rotor could be used to successfully trace the maze without
pursuit with visual tracking of the stylus, identifying reliance on sensory feedback. This type of internally
changes due to skill learning of the appropriate hand generated movement appears to be the domain of the
movement. An initial study showed increases with SMA. Consistent with this view, neuropsychological
practice in primary motor cortex, SMA, and the evidence shows that patients with PMA damage are
pulvinar nucleus. In a later study, PET scans over two impaired in sensory-cued motor learning.
days of learning revealed a similar pattern of activation A study in which partipants had to write the
on day 1, this time including practice-related changes letter “R” found increases in right PMA and right parietal
in ipsilateral cerebellum, cingulate, and inferior cortex, possibly indicating enhanced attention to sensory
parietal cortex. On day 2, after an extensive practice feedback. Again, the activation was unexpectedly
period, the rotor pursuit – visual tracking subtraction ipsilateral to the hand used. When participants wrote the
revealed changes in the putamen and parietal cortex letter as quickly as possible, a network including left
bilaterally, and the left (contralateral) PMA. primary motor, PMA, SMA, and right putamen were
Improvement on the task was correlated with increased activated. These experiments also compared writing
activity in premotor, prefrontal, and cingulate areas, novel ideograms with the right hand to a baseline in
and decreased activity in visual processing areas. which participants watched the figures being drawn.
An opposite pattern of of changes in the PMA Novel ideograms, as compared to baseline, activated left
and SMA were observed in a series of studies of maze primary motor cortex and right cerebellum. Practiced
tracing. In the maze tracing task, participants practiced ideograms minus baseline activated right PMA, left
tracing cutout maze patterns with their eyes closed. SMA, and right cerebellum. Once again, cerebellar
Tracing a novel maze with the right hand in a clockwise activity appeared during the initial learning and
pattern (relative to a control) resulted in increases in automatization phases of new visuomotor patterns, but
right PMA and left cerebellum. Practice diminished was not apparent when writing the letter “R”, a highly
both these loci of activity: PET scanning after 10 practiced ideogram.
minutes of practice produced increases in SMA and These studies show increases in PMA when
decreases in right PMA and left cerebellum. Contrary sensory feedback or external cues are critical for task
to expectations, training affected activity in the performance. Right PMA increases appear to be
ipsilateral cerebrum and contralateral cerebellum. A modulated by task demands—e.g., asking participants to
follow-up study examined maze tracing with the left be accurate—and independent of the hand used. In the
hand in a counter-clockwise pattern and, strikingly, study of unilateral two -ball rotation mentioned above,
produced the same results. A recent third study positive correlations were found between unilateral PMA
confirmed that only primary motor cortex and anterior activity during movements of either hand and skill
cerebellum showed activation depending on which hand improvement for the ipsilateral hand. Skill improvement
was used, suggesting that learning and performance in was measured during non-scanning intervals by
the maze-tracing task requires an abstract comparing the maximum rotation speed of the balls just
representation of movements and patterns not directly after the first scan and just before the last scan. The
tied to motor activity. strongest correlation was between right PMA activity and
Contradictory effects of practice in similar skill improvement for the right hand. Correlations
tasks, such as that found in PMA and SMA in the between left PMA and skill improvement for the left
studies reviewed above, may reflect something of the hand were also positive and significant.
underlying functions of premotor brain regions. It has The cerebellum appears to play a particular role
been suggested that the SMA is active in the internal in the learning of new visuomotor associations. In a
generation of responses, and PMA is preferentially classic study, participants made joystick movements to
active when responses are contingent upon sensory align a cursor with a visual target in one of three
cues. That distinction can be applied to neuroimaging joystick-cursor mappings: normal mapping, reversed
studies of human skill learning. In the rotor pursuit mapping in which joystick movements caused the cursor
studies, learning a new motor skill resulted in to move in the opposite direction, and random mapping
increased activity in SMA. After the movement was with no relationship between joystick and cursor
automatic, PMA and basal ganglia were activated in movement. The PET camera in this study was centered
adapting the movement to the motion of the stylus. In on the cerebellum, to allow maximum sensitivity to

Neuroimaging Studies of Memory 15

changes in this structure. When the mapping between temporal gyrus. Decreases were found in a number of
joystick movements and cursor movements was areas, including left lingual gyrus, bilateraloccipital
reversed, cerebellar activity was high during initial cortex, and right cerebellum, superior and inferior
learning and decreased as performance improved. parietal, inferior temporal, and pulvinar. The deactivation
When the mapping between joystick movements and of right superior parietal cortex and occipital cortex
cursor movements was random, no learning occurred could reflect decreased involvement of attention, which
and cerebellar activity remained high. The authors is associated with right parietal activation in a number of
concluded that the cerebellum contributes to studies, or it could result from a decreased need to rely
visuomotor skill learning by participating in the on visuospatial transformations. Increases in the left
detection and correction of errors. It should be noted, fusiform gyrus, activated in at least one other letter
in closing, that the cerebellar contribution to skill recognition study and in studies of object recognition,
learning may be primarily in sensory-motor could reflect a shift to direct letter-recognition
association tasks. Patients with cerebellar atrophy can processes. Overall, the results are consistent with a shift
improve on some motor skill tasks, but they fail to in strategy from sensory transformation to direct
coordinate and adapt movements to environmental recognition of mirror words.
contexts. Implicit memory. All the studies of skill
Learning perceptual skills. Research on learning discussed above involved both explicit and
the psychophysics of perceptual learning suggests that implicit memory for motor activity. Although
long-term learning of skills is not limited to motor procedural in nature, in most of the tasks participants
areas. This research suggests three striking findings: were allowed to develop conscious recognition of the
that learning to detect a visual stimulus is specific to a sequences and movements to be learned. In one study,
retinal location, that it only occurs if the stimulus is explicit awareness of a sequence was associated with
behaviorally relevant, and that effects of learning changes in the cerebellum, ventrolateral frontal cortex,
remain robust after nearly three years without practice. and medial frontal cortex. The remainder of the changes
Although there has been relatively little with skill learning, found prominently in parts of the
neuroimaging research on perceptual learning, the cerebellum, primary motor cortex, SMA, PMA, and basal
research that has been conducted is illuminating. ganglia, presumably reflect changes in procedural motor
Perhaps the first study was an fMRI experiment of processes: the kind you can do, but cannot describe—
mirror reading. Participants viewed words and and, if you are amnesiac, may not remember that you ever
matched nonwords printed backwardsand decided learned.
whether the stimulus was a real word. Improvement in Implicit memory is not limited to motor and
mirror reading is still detectable after two months of perceptual learning, however. Exposure to words,
practice, and changes in performance are evident after pictures, faces, and other stimuli influences later
a year without practice. The researchers compared processing of this information, even though subjects may
mirror-reading with normal reading and practiced not ‘remember’ any of the trained items. This
mirror-words with unpracticed ones. Mirror reading unconscious memory is independent of explicit
compared to normal reading produced increases in recollection, meaning that the amount of explicit
blood flow in a number of areas, including medial and memory one has for a stimulus does not predict the
lateral occipital cortex, right superior parietal cortex, amount of implicit memory. Also, implicit memory is
bilateral fusiform gyrus, pulvinar, and cerebellum, not affected by depth of encoding as is explicit memory.
particularly on the right side. Decreases were found in The two forms of memory appear to be separate species
nearby regions of some of the same areas, including that operate independently of one another.
medial occipital and right superior parietal cortex, as Neuroimaging results support the conclusion that
well as in precuneus and bilateral middle/superior explicit memory and implicit memory are mediated by
temporal gyrus. The occipital, fusiform, and pulvinar separate circuits in the brain—explicit memory by
areas form an anatomically interconnected network hippocampus, prefrontal cortex, and related circuitry, and
associated with visual processing, object recognition, implicit memory by perceptual areas similar to those
and visual attention. The authors suggested that these discussed in the previous section.
activations were due to visual transformation of the One of the most common ways to study implicit
letters in unpracticed mirror reading. retrieval is the word-stem completion task. In the
With practice, mirror reading activation encoding phase of the task, participants are given a list of
increased in the precuneus, left superior parietal words and asked to make some judgment, semantic or
cortex and fusiform, and right cerebellum and superior otherwise, about the words. Subsequently, the
participants are given a list of incomplete words—such

Neuroimaging Studies of Memory 16

as the first three letters—and asked to complete the behavioral evidence from normal human subjects, and
fragments with whatever word comes to mind. If the evidence from animal models. Taken together, these
fragments match words that they viewed before, sources of evidence have sketched the outlines of
participants are more likely to complete the stem with complex of memory systems. These systems in their
the previously viewed word, whether or not they interaction form a seamless whole capable of dealing
consciously remember it. with a variety of cognitive problems. Our memory
Daniel Schacter and coworkers predicted that apparatus has multiple components (working memory,
when only implicit memory is involved in a task, stem long-term memory), multiple representations of
completion will not activate the hippocampus, the information in different formats (e.g., verbal versus
structure critical to explicit episodic memory. They spatial), multiple retrieval schemes (explicit versus
asked participants to count the number of ‘T’ junctions implicit), multiple circuitries, and multiple processes
in a list of words, and compared activation during (encoding, retention, and retrieval). We are far from
stem-completion of words from the list with novel getting our arms around a complete description of this
words. Participants did not explicitly remember the system, but continued use of the full array of
words, but their performance showed priming investigative tools promises much new progress in the
effects—and their PET scans revealed no hippocampal current millenium.
activation, but showed decreases in extrastriate (visual)
cortex. In another study, they compared stem
completion when participants had encoded words
deeply (when they were required to make semantic
judgments) or superficially (when they counted ‘T’
junctions). Explicit memory and hippocampal
activation during stem completion were greater for the
deeply processed words. Other studies have
replicated this basic finding with object naming and
categorization tasks. These results taken together
indicate a dissociation between explicit and implicit
memory; explicit memory involves activation of
hippocampal structures, while implicit memory
involves deactivation of posterior neocortex.
Studies in other domains of semantic
knowledge, such as object categorization and object
naming, show similar priming-related reductions in
activation. One explanation of the decrease in
activations is that repeated presentations lead to more
efficient perceptual processing and lower levels of
activation. However, in an auditory word-stem
completion task studied by Schacter and colleagues,
auditory presentation of words and auditory stem
completion produced the same rCBF decreases in
bilateral extrastriate cortex, as well as decreases in
right anterior medial prefrontal cortex, right angular
gyrus, and precuneus. Thus, priming effects on
extrastriate cortex appear to be modality-independent.
Some Concluding Remarks
There has been remarkable progress in the
study of memory during the last half of the twentieth
century. The development of neuroimaging techniques
has contributed in no small part to this progress. These
techniques have provided a source of evidence about
the relationship between brain structure and
psychological function that complements evidence
from the study of human patients with brain injury,

Neuroimaging Studies of Memory 17

Figure Captions

1. A taxonomy of various forms of long-term

memory

2. Brain diagrams highlighting the major

structures discussed in the text. The upper
figure shows a lateral view of a left
hemisphere of the human brain, and the lower
figure shows a view of the right hemisphere as
seen from the midline of the brain. Major
structures of relevance to memory are labeled.

Neuroimaging Studies of Memory 18

 Dorsal / Superior
Primary motor area (M1)
Supplementary motor area Superior parietal lobule
Inferior parietal lobule:
Premotor area
Supramarginal
Primary motorgyrus
area (M1)
Angular gyrus
Dorsolateral prefrontal cortex

Rostral / Caudal/
Anterior Posterior

Occipital cortex
Frontal pole

Inferior frontal
gyrus

Orbital gyrus

Temporal pole Left cerebellar hemisphere

Superior temporal gyrus

Middle temporal gyrus
Inferior temporal gyrus
Ventral / Inferior

Neuroimaging Studies of Memory 19

 Figure 2, con’t.

Dorsal / Superior

Supplementary motor area

Primary motor area (M1)
Anterior cingulate gyrus Superior parietal lobule

Precuneus

Posterior cingulate gyrus

Caudate nucleus
(basal ganglia)

Rostral / Cuneus
Anterior Caudal/
Posterior

Gyrus rectus

Fimbria (connects to Lingual gyrus

hippocampus)

Fusiform gyrus
Parahippocampal Occipitotemporal
Inferior temporal gyrus (lateral to occipito-
gyrus gyrus
temporal gyrus, not
Ventral / Inferior visible)

Neuroimaging Studies of Memory 20