Neuromethods 119

Massimo Filippi Editor

fMRI
Techniques
and Protocols
Second Edition
NEUROMETHODS

Series Editor
Wolfgang Walz
University of Saskatchewan
Saskatoon, SK, Canada

For further volumes:

http://www.springer.com/series/7657
fMRI Techniques and Protocols

Second Edition

Edited by

Massimo Filippi
Neuroimaging Research Unit, INSPE, Division of Neuroscience,
San Raffaele Scientific Institute and Vita-Salute San Raffaele
University, Milan, Italy
Editor
Massimo Filippi, MD, FEAN
Neuroimaging Research Unit
Institute of Experimental Neurology
Division of Neuroscience
San Raffaele Scientific Institute and
Vita-Salute San Raffaele University
Milan, Italy

Department of Neurology
Division of Neuroscience
San Raffaele Scientific Institute and
Vita-Salute San Raffaele University
Milan, Italy

ISSN 0893-2336 ISSN 1940-6045 (electronic)

Neuromethods
ISBN 978-1-4939-5609-8 ISBN 978-1-4939-5611-1 (eBook)
DOI 10.1007/978-1-4939-5611-1

Library of Congress Control Number: 2016947708

© Springer Science+Business Media New York 2016

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction
on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation,
computer software, or by similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not
imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and
regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to
be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty,
express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

This Humana Press imprint is published by Springer Nature

The registered company is Springer Science+Business Media LLC New York
Preface to the Series

Experimental life sciences have two basic foundations: concepts and tools. The Neuromethods
series focuses on the tools and techniques unique to the investigation of the nervous system
and excitable cells. It will not, however, shortchange the concept side of things as care has
been taken to integrate these tools within the context of the concepts and questions under
investigation. In this way, the series is unique in that it not only collects protocols but also
includes theoretical background information and critiques which led to the methods and
their development. Thus it gives the reader a better understanding of the origin of the
techniques and their potential future development. The Neuromethods publishing program
strikes a balance between recent and exciting developments like those concerning new ani-
mal models of disease, imaging, in vivo methods, and more established techniques, includ-
ing, for example, immunocytochemistry and electrophysiological technologies. New
trainees in neurosciences still need a sound footing in these older methods in order to apply
a critical approach to their results.
Under the guidance of its founders, Alan Boulton and Glen Baker, the Neuromethods
series has been a success since its first volume published through Humana Press in 1985. The
series continues to flourish through many changes over the years. It is now published under
the umbrella of Springer Protocols. While methods involving brain research have changed a
lot since the series started, the publishing environment and technology have changed even
more radically. Neuromethods has the distinct layout and style of the Springer Protocols
program, designed specifically for readability and ease of reference in a laboratory setting.
The careful application of methods is potentially the most important step in the process
of scientific inquiry. In the past, new methodologies led the way in developing new disci-
plines in the biological and medical sciences. For example, Physiology emerged out of
Anatomy in the nineteenth century by harnessing new methods based on the newly discov-
ered phenomenon of electricity. Nowadays, the relationships between disciplines and meth-
ods are more complex. Methods are now widely shared between disciplines and research
areas. New developments in electronic publishing make it possible for scientists that
encounter new methods to quickly find sources of information electronically. The design of
individual volumes and chapters in this series takes this new access technology into account.
Springer Protocols makes it possible to download single protocols separately. In addition,
Springer makes its print-on-demand technology available globally. A print copy can there-
fore be acquired quickly and for a competitive price anywhere in the world.

Saskatoon, Canada Wolfgang Walz

v
Preface

fMRI has gained a remarkable role as a tool for studying brain function due to its capability
to provide an invaluable insight into the mechanisms through which the human brain
works in healthy individuals and to explore the mechanisms associated with recovery of
function or clinical deterioration in patients with different neurological and psychiatric
conditions. The utility of this technique to monitor the effects of pharmacologic and reha-
bilitative treatments has also been recently demonstrated.
The second edition of this book aims at providing an up-to-date review of the main
methodological aspects of fMRI, as well as a state-of-the-art summary of the achievements
obtained by its application to the study of central nervous system functioning in the clinical
arena. Future evolutions of fMRI techniques are also discussed. The contributors of this
volume are all worldwide renowned scientists and physicians with a broad experience in the
technical development and clinical use of fMRI. Although the field is ample, based on a
series of very different disciplines and expanding at a dramatic pace every day, I believe that
this book provides an adequate background against which to plan and design new studies
to advance our knowledge on the physiology of the normal human brain and its change
following tissue injury.
Part I of the volume is aimed at providing the basic knowledge for the understanding
of the technical aspects of fMRI. It covers the basic principles of MRI and fMRI, the differ-
ent options that can be used to set up an fMRI experiment, and the steps of fMRI analysis,
from the preparation of data to the achievement of interpretable results. This part is there-
fore essential to introduce the readers to the “fMRI world” and make them able to inter-
pret with enough criticism the results of their own experiments. A chapter is devoted to the
advantages, caveats, and pitfalls of fMRI data acquired using high-field MR scanners. In
addition, although still in its infancy, the assessment of brain connectomics with functional
and structural imaging techniques is considered at length, given its potential for improving
the understanding of normal and pathological brain function.
Part II provides an overview of the main results derived from the application of fMRI
to the study of healthy individuals. Given its noninvasiveness, safety, and repeatability, fMRI
is rapidly replacing, whenever possible, other functional techniques, such as positron emis-
sion tomography, to image the function of the normal brain. In addition, due to its spatial
resolution, fMRI is commonly preferred to neurophysiological techniques to locate with
precision the areas activated during the performance of experimental tasks. What has been
achieved in the analysis of the main human functional systems with fMRI is illustrated,
including, among many other aspects, behavior, language, memory, emotion, sensation,
pain, vision, and hearing.
Part III is more clinically oriented and illustrates the main findings obtained by the
application of fMRI to assess the role of brain plasticity in the major neurological and psy-
chiatric conditions. The first chapter is devoted to fMRI studies of multiple sclerosis, since
there is a growing body of evidence that brain functional reorganization has an important
role, at least in same phases of the disease, in limiting the clinical consequences of MS-related
irreversible tissue damage. Therefore, MS can be viewed as a “model” to understand how

vii
viii Preface

pathology can affect the patterns of brain recruitment. The results obtained in other white
matter conditions, including isolated demyelinating myelitis and vasculitides, are then pre-
sented. The second chapter deals with stroke studies, which have shown consistently that
reorganization of surviving neuronal networks is one of the key factors underlying recovery
of function. The experimental caveats to be faced when studying patients with severe clini-
cal impairment are also reviewed. The following chapters cover psychiatric and neurode-
generative diseases, a field where fMRI is providing important pieces of information not
only for the understanding of the mechanisms underlying disease pathophysiology and
genesis of symptomatology, but also for planning and monitoring novel treatment strate-
gies. Then, two conditions, i.e., epilepsy and tumors, where fMRI is gaining an important
role in the presurgical evaluation of patients, are discussed. The last contribution of this
part describes the potential and some preliminary, but nevertheless promising, results on
the use of fMRI in the monitoring of pharmacological treatments and motor
rehabilitation.
Part IV is a glimpse into the future and presents novel approaches for the integration
of fMRI data with measures of damage assessed using structural MR techniques and the
application of fMRI to image spinal cord function. Finally, results derived from the applica-
tion of graph analysis to assess network abnormalities in patients with several neurological
and psychiatric disorders, including dementia, amyotrophic lateral sclerosis, multiple scle-
rosis, and schizophrenia, are presented.
The hope that has inspired this book is that it will be of help to clinicians and research-
ers in their daily life activity by providing a “user-friendly” summary of the field and the
necessary background against which to plan and carry out future and successful studies.
This is, indeed, an ever-growing and exciting field of research, where we have reached a lot
in the past few years, but where there is still a long journey ahead of us.

Milan, Italy Massimo Filippi

Contents

Preface to the Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

PART I BOLD FMRI: BASIC PRINCIPLES

1 Principles of MRI and Functional MRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Ralf Deichmann
2 Introduction to Functional MRI Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Luis Hernandez-Garcia, Scott Peltier, and William Grissom
3 Selection of Optimal Pulse Sequences for fMRI. . . . . . . . . . . . . . . . . . . . . . . . 69
Mark J. Lowe and Erik B. Beall
4 High-Field fMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Alayar Kangarlu
5 Experimental Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Hugh Garavan and Kevin Murphy
6 Preparing fMRI Data for Statistical Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . 155
John Ashburner
7 Statistical Analysis of fMRI Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Mark W. Woolrich, Christian F. Beckmann, Thomas E. Nichols,
and Stephen M. Smith
8 Dynamic Causal Modeling of Brain Responses . . . . . . . . . . . . . . . . . . . . . . . . 241
Karl J. Friston
9 Brain Atlases: Their Development and Role in Functional Inference . . . . . . . . 265
John Darrell Van Horn and Arthur W. Toga
10 Graph Theoretic Analysis of Human Brain Networks. . . . . . . . . . . . . . . . . . . . 283
Alex Fornito

PART II FMRI APPLICATION TO MEASURE BRAIN FUNCTION

11 Functional MRI: Applications in Cognitive Neuroscience . . . . . . . . . . . . . . . . 317
Mark D’Esposito, Andrew Kayser, and Anthony Chen
12 fMRI of Language Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
Jeffrey R. Binder
13 Neuroimaging Approaches to the Study of Visual Attention . . . . . . . . . . . . . . 387
George R. Mangun, Yuelu Liu, Jesse J. Bengson, Sean P. Fannon, Nicholas E.
DiQuattro, and Joy J. Geng
14 fMRI of Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
Federica Agosta, Indre V. Viskontas, Maria Luisa Gorno-Tempini,
and Massimo Filippi

ix
x Contents

15 fMRI of Emotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

Simon Robinson, Ewald Moser, and Martin Peper
16 fMRI of Pain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
Emma G. Duerden, Roberta Messina, Maria A. Rocca, Massimo Filippi
and Gary H. Duncan
17 fMRI of the Sensorimotor System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
Massimo Filippi, Roberta Messina, and Maria A. Rocca
18 Functional Imaging of the Human Visual System . . . . . . . . . . . . . . . . . . . . . . 545
Guy A. Orban and Stefania Ferri
19 fMRI of the Central Auditory System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573
Deborah Ann Hall and Aspasia Eleni Paltoglou

PART III FMRI CLINICAL APPLICATION

20 Application of fMRI to Multiple Sclerosis and Other
White Matter Disorders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
Massimo Filippi and Maria A. Rocca
21 fMRI in Cerebrovascular Disorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639
Nick S. Ward
22 fMRI in Psychiatric Disorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 657
Erin L. Habecker, Melissa A. Daniels, Elisa Canu, Maria A. Rocca,
Massimo Filippi and Perry F. Renshaw
23 fMRI in Neurodegenerative Diseases: From Scientific Insights
to Clinical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699
Bradford C. Dickerson, Federica Agosta, and Massimo Filippi
24 fMRI in Epilepsy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741
Rachel C. Thornton, Louis André van Graan, Robert H. Powell,
and Louis Lemieux
25 fMRI in Neurosurgery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 801
Oliver Ganslandt, Christopher Nimsky, Michael Buchfelder,
and Peter Grummich
26 Pharmacological Applications of fMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 817
Paul M. Matthews
27 Application of fMRI to Monitor Motor Rehabilitation . . . . . . . . . . . . . . . . . . 833
Steven C. Cramer and Jessica M. Cassidy

PART IV FUTURE FMRI DEVELOPMENT

28 Multimodal Fusion of Structural and Functional Brain Imaging Data . . . . . . . 853
Jing Sui and Vince D. Calhoun
29 Functional MRI of the Spinal Cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 871
Patrick Stroman and Massimo Filippi
30 Clinical Applications of the Functional Connectome . . . . . . . . . . . . . . . . . . . . 893
Massimo Filippi and Maria A. Rocca

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905
Contributors

FEDERICA AGOSTA • Neuroimaging Research Unit, Institute of Experimental Neurology,

Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San Raffaele
University, Milan, Italy
JOHN ASHBURNER • Wellcome Trust Centre for Neuroimaging, Institute of Neurology,
University College London, London, UK
ERIK B. BEALL • Imaging Institute, Cleveland Clinic, Cleveland, OH, USA
CHRISTIAN F. BECKMANN • Oxford University Centre for Functional Magnetic Resonance
Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford, UK
JESSE J. BENGSON • Center for Mind and Brain, University of California, Davis, CA, USA;
Department of Psychology, Sonoma State University, Sonoma, CA, USA
JEFFREY R. BINDER • Language Imaging Laboratory, Department of Neurology and
Biophysics, The Medical College of Wisconsin, Milwaukee, WI, USA
MICHAEL BUCHFELDER • Department of Neurosurgery, Universitätsklinikum Erlangen,
Erlangen, Germany
VINCE D. CALHOUN • The Mind Research Network and Lovelace Biomedical and
Environmental Research Institute, Albuquerque, NM, USA; Electrical and Computer,
University of New Mexico, Albuquerque, NM, USA; Departments and Neurosciences,
University of New Mexico, Albuquerque, NM, USA
ELISA CANU • Neuroimaging Research Unit, Institute of Experimental Neurology,
Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San Raffaele
University, Milan, USA
JESSICA M. CASSIDY • Department of Neurology, University of California, Irvine, Orange,
CA, USA
ANTHONY CHEN • Helen Wills Neuroscience Institute, University of California, Berkeley,
CA, USA
STEVEN C. CRAMER • Department of Neurology, University of California, Irvine, Orange,
CA, USA; Department of Anatomy & Neurobiology, University of California, Irvine,
Orange, CA, USA
MARK D’ESPOSITO • Helen Wills Neuroscience Institute, University of California, Berkeley,
CA, USA
MELISSA A. DANIELS • Brain Imaging Center, McLean Hospital, Belmont, MA, USA
RALF DEICHMANN • University Hospital, ZNN, Brain Imaging Center, Frankfurt, Germany
NICHOLAS E. DIQUATTRO • Center for Mind and Brain, University of California, Davis,
CA, USA
BRADFORD C. DICKERSON • Gerontology Research Unit, Massachusetts General Hospital,
Charlestown, MA, USA
EMMA G. DUERDEN • Groupe de recherche sur le système nerveux central, Université de
Montréal, Montréal, QC, Canada
GARY H. DUNCAN • Groupe de recherche sur le système nerveux central, Université de
Montréal, Montréal, QC, Canada
SEAN P. FANNON • Center for Mind and Brain, University of California, Davis, CA, USA;
Department of Psychology, Folsom Lake College, Folsom, CA, USA

xi
xii Contributors

STEFANIA FERRI • Dipartimento di Neuroscienze, Universita degli Studi di Parma, Parma,

Italy
MASSIMO FILIPPI • Neuroimaging Research Unit, Institute of Experimental Neurology,
Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San
Raffaele University, Milan, Italy; Department of Neurology, Division of Neuroscience,
San Raffaele Scientific Institute and Vita-Salute San Raffaele University, Milan, Italy
ALEX FORNITO • Brain and Mental Health Laboratory, Monash Institute of Cognitive and
Clinical Neuroscience, School of Psychological Sciences and Monash Biomedical Imaging,
Monash University, Melbourne, Australia
KARL J. FRISTON • Wellcome Centre for Neuroimaging, Institute of Neurology, University
College London, London, UK
OLIVER GANSLANDT • Neurochirurgische Klinik, Klinikum Stuttgart, Stuttgart, Germany
HUGH GARAVAN • Department of Psychiatry, University of Vermont, Burlington, VT, USA
JOY J. GENG • Center for Mind and Brain, University of California, Davis, CA, USA;
Department of Psychology, University of California, Davis, CA, USA
MARIA LUISA GORNO-TEMPINI • Department of Neurology, Memory and Aging Center,
University of California, San Francisco, CA, USA
LOUIS ANDRÉ VAN GRAAN • Department of Clinical and Experimental Epilepsy, Institute of
Neurology, University College London, London, UK
WILLIAM GRISSOM • Biomedical Engineering Department, Vanderbilt University, Nashville,
TN, USA
PETER GRUMMICH • Department of Neurosurgery, Universitätsklinikum Erlangen,
Erlangen, Germany
ERIN L. HABECKER • Brain Imaging Center, McLean Hospital, Belmont, MA, USA
DEBORAH ANN HALL • MRC Institute of Hearing Research, Nottingham, UK
LUIS HERNANDEZ-GARCIA • University of Michigan Functional MRI Laboratory and
Biomedical Engineering Department, University of Michigan, Ann Arbor, MI, USA
JOHN DARRELL VAN HORN • USC Mark and Mary Stevens Neuroimaging and Informatics
Institute, Keck School of Medicine of USC, University of Southern California, Los Angeles,
CA, USA
ALAYAR KANGARLU • Columbia University and New York State Psychiatric Institute, New
York, NY, USA
ANDREW KAYSER • Helen Wills Neuroscience Institute, University of California, Berkeley,
CA, USA
LOUIS LEMIEUX • Department of Clinical and Experimental Epilepsy, Institute of
Neurology, University College London, London, UK
YUELU LIU • Center for Mind and Brain, University of California, Davis, CA, USA
MARK J. LOWE • Imaging Institute, Cleveland Clinic, Cleveland, OH, USA
GEORGE R. MANGUN • Center for Mind and Brain, University of California, Davis, CA,
USA; Department of Psychology, University of California, Davis, CA, USA; Department
of Neurology, University of California, Davis, CA, USA
PAUL M. MATTHEWS • Department of Medicine and Centre for Neurotechnology,
Hammersmith Hospital, Imperial College, London, UK; GlaxoSmithKline Clinical
Imaging Centre, Hammersmith Hospital, Imperial College, London, UK
ROBERTA MESSINA • Neuroimaging Research Unit, Institute of Experimental Neurology,
Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San Raffaele
University, Milan, Italy; Department of Neurology, Division of Neuroscience, San
Raffaele Scientific Institute and Vita-Salute San Raffaele University, Milan, Italy
 Contributors xiii

EWALD MOSER • High Field MR Centre, Medical University of Vienna, Austria; Division
MR-Physics, Center for Medical Physics and Biomedical Engineering, Vienna, Austria
KEVIN MURPHY • School of Psychology, Cardiff University, Cardiff, Wales, UK
THOMAS E. NICHOLS • Oxford University Centre for Functional Magnetic Resonance
Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford, UK
CHRISTOPHER NIMSKY • Neurochirurgische Klinik, Klinikum Stuttgart, Stuttgart,
Germany
GUY A. ORBAN • Dipartimento di Neuroscienze, Universita degli Studi di Parma, Parma,
Italy
ASPASIA ELENI PALTOGLOU • MRC Institute of Hearing Research, Nottingham, UK
SCOTT PELTIER • University of Michigan Functional MRI Laboratory and Biomedical
Engineering Department, University of Michigan, Ann Arbor, MI, USA
MARTIN PEPER • General and Biological Psychology Section, Faculty of Psychology,
Philipps-Universität Marburg, Marburg, Germany
ROBERT H. POWELL • Department of Clinical and Experimental Epilepsy, Institute of
Neurology, University College London, London, UK
PERRY F. RENSHAW • Brain Institute, University of Utah, Salt Lake City, UT, USA
SIMON ROBINSON • High Field MR Centre, Medical University of Vienna, Austria;
Department of Biomedical Imaging and Image-guided Therapy, Medical University
of Vienna, Austria
MARIA A. ROCCA • Neuroimaging Research Unit, Institute of Experimental Neurology,
Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San Raffaele
University, Milan, Italy; Department of Neurology, Division of Neuroscience, San
Raffaele Scientific Institute and Vita-Salute San Raffaele University, Milan, Italy
STEPHEN M. SMITH • Oxford University Centre for Functional Magnetic Resonance
Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford, UK
PATRICK STROMAN • Department of Diagnostic Radiology, c/o Center for Neuroscience
Studies, Queen's University, Kingston, ON, Canada; Department of Physics, c/o Center
for Neuroscience Studies, Queen's University, Kingston, ON, Canada
JING SUI • Brainnetome Center and National Laboratory of Pattern Recognition, Institute
of Automation, Chinese Academy of Sciences, Beijing, China; The Mind Research
Network and Lovelace Biomedical and Environmental Research Institute, Albuquerque,
NM, USA
RACHEL C. THORNTON • Department of Clinical and Experimental Epilepsy, Institute of
Neurology, University College London, London, UK
ARTHUR W. TOGA • USC Mark and Mary Stevens Neuroimaging and Informatics
Institute, Keck School of Medicine of USC, University of Southern California,
Los Angeles, CA, USA
INDRE V. VISKONTAS • Department of Neurology, Memory and Aging Center, University of
California, San Francisco, CA, USA
NICK S. WARD • Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University
College London, London, UK
MARK W. WOOLRICH • Oxford University Centre for Functional Magnetic Resonance
Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford, UK
 Part I

BOLD fMRI: Basic Principles

 Chapter 1

Principles of MRI and Functional MRI

Ralf Deichmann

Abstract
This chapter describes the basics of magnetic resonance imaging (MRI) and functional MRI (fMRI). It is
aimed at beginners in the field and does not require any previous knowledge. Complex technical issues are
made plausible by presenting plots and figures, rather than mathematical equations.
The part dealing with the basics of MRI covers spins, spin alignment in external magnetic fields, the
magnetic resonance effect, field gradients, frequency encoding, phase encoding, slice selection, k-space,
gradient echoes, and echo-planar imaging.
The part dealing with fMRI covers transverse relaxation times, the basics of the blood oxygen level-
dependent (BOLD) contrast, and the hemodynamic response.

Key words Spin, Field gradients, Frequency encoding, Phase encoding, Slice selection, k-Space,
Gradient echo, Echo-planar imaging, Transverse relaxation time, Blood oxygen level-dependent

1 Basic Physical Principles

1.1 Spins in an The first question arising is “What do we actually see in MRI?”
External Magnetic In general, we see protons. A proton is the nucleus of the hydro-
Field gen atom. Hydrogen is the most common element in tissue, so if we
are able to detect the presence of protons and display them with a
certain spatial resolution, it is fair to say that we can “see” tissue.
The detection of protons is based on a physical property
called the “spin.” A correct description of spins is only possible
with quantum mechanics and would be beyond the scope of this
book, so may it suffice to say that a spin is quite similar to a com-
pass needle. In particular, a compass needle carries a “magnetiza-
tion” which enables it to align in an external magnetic field and
produce a magnetic field itself (for example, a compass needle
can be used to attract small iron particles). A spin possesses an
elementary magnetization (albeit a tiny one), so it behaves in a
similar way.
Let us consider a simple object containing protons, for example a
container with water (Fig. 1). If there is no external magnetic field

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_1, © Springer Science+Business Media New York 2016

3
4 Ralf Deichmann

Fig. 1 (Left) Without an external magnetic field (labeled B) the spins point in dif-
ferent directions, the contributions of their respective magnetization vectors can-
cel out, so there is no net magnetization (labeled M). (Right) Inside an external
magnetic field B the spins align in parallel or anti-parallel direction, with a slight
majority of spins in parallel direction, giving rise to a net magnetization M

(usually labeled B), the spins will point in different directions, the con-
tributions of their respective magnetization vectors will cancel out, so
there is no net magnetization (usually labeled M). If, however, this
object is placed into an external magnetic field B (e.g., into the bore
of an MR scanner), the spins will align. According to the laws of quan-
tum mechanics, this alignment is either parallel or anti-parallel to the
external magnetic field, so once again one might assume that the sin-
gle magnetization vectors will cancel each other. However, a slight
majority of spins prefers the parallel direction. This results in a macro-
scopic net magnetization M which is parallel to B (Fig. 1, right). The
idea is now: any measurement of M would correspond to the detec-
tion of the presence of protons in the object. If M is measured with a
spatial resolution, we can display the result as an image. This is exactly
what is done in MR imaging. The measurement of M is based on a
physical effect which will be described in Subheading 1.2.

1.2 The Larmor Let us assume that we disturb the realigned spins in a way that (at
Precession least for a short time) the magnetization vector M is not parallel to
B but tilted by a certain angle (how this can be achieved will be
discussed in Subheading 1.3). In this case, an interesting process
begins: the tilted magnetization vector rotates around the direction
of the external magnetic field. This movement, which resembles
closely the behavior of a spinning top, is called precession (Fig. 2).
The frequency is called Larmor frequency. It is important to note
that the Larmor frequency f is proportional to the external field B.
f = g B. (1)
In this formula, γ is the gyromagnetic ratio with a value of
42.58 MHz/T for hydrogen.
 Principles of MRI and Functional MRI 5

Fig. 2 Behavior of the tilted magnetization: Precession with the Larmor frequency

This effect is of major interest because during the precession,

the spins send out an electromagnetic wave with the Larmor fre-
quency. For field strengths that are common on clinical MR scan-
ners (1.5 and 3 T), the Larmor frequencies are about 63 MHz and
127 MHz, respectively. As the reader may know, the tuning on
standard frequency modulation (FM) radios ranges from 88 to
108 MHz, so it is fair to say that the signal sent out by the spins is
similar to an ordinary FM radio broadcasting signal. For this rea-
son, it is also called radiofrequency (RF) signal. It can be detected
with an appliance very similar to an FM tuner. This is exactly the
way how the magnetization M and thus the presence of protons are
detected in MR scanners. In Subheading 1.3, it will be described
how the magnetization can be tilted.

1.3 The Magnetic The next physical effect is more or less the opposite of the one that
Resonance Effect has just been discussed. This time, the object is exposed to an exter-
nal RF signal which has exactly the Larmor frequency and is pro-
duced by a kind of “inbuilt FM broadcasting station” (Fig. 3, part 1).
This signal will tilt the magnetization which subsequently starts to
precede (Fig. 3, part 2). During precession, an RF signal is being sent
out which can be detected with a kind of “FM tuner” (Fig. 3, part 3).
It should be noted that this concept only works if the incoming
RF signal has exactly the spins’ Larmor frequency. Otherwise, the
magnetization will not be tilted and it is impossible to detect a signal.
Thus, we are dealing with a resonance effect, and this explains why
the imaging technique based on this effect is called magnetic reso-
nance imaging (MRI). The nuclear magnetic resonance effect was
first described independently by Bloch and Purcell in 1946 [1, 2].
We have now covered all the basic physical effects that are
required to understand how MRI works. To summarize it, all MR
experiments are based on the following principles:
● Put the object to be imaged into a strong external magnetic
field B. The spins will align and create a net magnetization M
which is parallel to B.
● Knowing B, calculate the Larmor frequency f = γB and send an
external RF pulse which has exactly this frequency. This will tilt
the magnetization. After a short time, the external RF has
served its duty and can be switched off.
6 Ralf Deichmann

Fig. 3 An external radiofrequency (RF) pulse that has exactly the Larmor fre-
quency tilts the magnetization (1). The tilted magnetization starts to precede (2)
and sends out an RF pulse with the Larmor frequency itself (3)

● The magnetization vector now precedes. Switch on your FM

tuner. If you detect a signal, you have detected the presence
of protons.
Subheading 2 will deal with the question how spatial resolu-
tion can be achieved.

2 A One-Dimensional MR Experiment

Let us assume we have a closed box containing two glasses of water,

one being half full and the other one full (Fig. 4a).
We now have to answer the following questions:
1. Are there any glasses inside the box or is it empty?
2. How many glasses are inside the box?
3. What is the exact location of the glasses?
4. What are the relative filling levels?
Of course: we are not allowed to open the box! Our investigation
must rely purely on the physical principles we have discussed so far.
The whole procedure will be discussed step by step as follows:
Step 1. We put the box into the MR scanner, i.e., into a strong
magnetic field B. In both glasses, spins will align, resulting in a
net magnetization M in the first glass and, due to the larger
number of protons, 2 M in the second glass (Fig. 4b).
Step 2. We calculate the Larmor frequency f = γB and send an exter-
nal RF pulse which has exactly this frequency. This will cause a
tilt of the magnetization vectors in both glasses which conse-
quently start to precede, sending out an RF signal with the
same frequency f (Fig. 4c). This signal can be detected with a
kind of FM tuner, so we can answer at least the first question:
the box is sending out a signal, so it cannot be empty. However,
so far we are not able to comment on the number and locations
of the glasses, because they send out signals with the same fre-
quency and we can only detect the sum signal outside the box.
 Principles of MRI and Functional MRI 7

Fig. 4 (a) Two glasses of water inside a closed box. (b) Magnetization inside both glasses after placing the box
inside an external magnetic field. (c) Reaction to an external radiofrequency (RF) pulse: the magnetization vec-
tors in both glasses start to precede and send out RF signals with the Larmor frequency. (d) Effect of an exter-
nal field gradient: the magnetic field strength at the position of both glasses is different, so radiofrequency
signals with different Larmor frequencies are being sent

Step 3. This is the crucial step: we switch on a so-called gradient field.

This simply means that we modify for a certain time the external
magnetic field B in a way that it is no longer constant across the
box but increases (slightly) from the left-hand to the right-hand
side (Fig. 4d). In particular, the field strengths at the positions
of the first and the second glass are now different, assuming the
values B1 and B2, respectively. Since the Larmor frequency
depends on the magnetic field strength, the magnetization vec-
tors continue their precession with different frequencies f1 and
f2. Once again, we measure the sum signal outside the box.
8 Ralf Deichmann

As the next step, a frequency analysis of this signal is performed.

This means that the signal is decomposed into its spectrum of fre-
quency components. In the present case, this analysis yields the
following results:
● The signal contains two frequency components, so there must
be two glasses inside the box.
● We can measure the absolute values of the frequencies f1 and f2.
According to Eq. 1, we can calculate from these frequencies
the field strengths B1 and B2 at the positions of the first and the
second glass, respectively. Since we know how we modified the
magnetic field B, we can deduce from this the exact positions
of the glasses.
● The amplitude of the frequency component f2 is twice the ampli-
tude of the frequency component f1. From this we can deduce
that there must be twice the number of spins in the second glass,
so we also obtain information about the relative filling levels.
In summary, we have answered all the questions above with-
out opening the box, purely by using the concepts of magnetic
resonance.
If you could follow this section, you have understood the
basics of MR imaging.
The use of field gradients for spatial encoding was proposed by
Lauterbur in 1973 [3].
In textbooks and publications, MR experiments are usually
described by special plots, the so-called pulse diagrams. For the
experiment described above, the respective pulse diagram is shown
in Fig. 5, comprising an RF axis and a gradient axis. The entries on
the RF axis correspond to the initial RF pulse which tilts the
magnetization and the acquired signal. The entries on the gradient
axis show that during signal acquisition the gradient Gx is switched
on, i.e., during this time the external magnetic field is modified in
a way that it increases linearly in a certain spatial direction (the
x-direction of an arbitrarily chosen coordinate system). Because
this gradient is switched on during the readout process, it is also
referred to as read gradient.

Fig. 5 Schematic plot of a one-dimensional MR experiment, showing a radiofre-

quency (RF) axis and a gradient (Gx) axis
 Principles of MRI and Functional MRI 9

Fig. 6 Frequency spectrum resulting from the Fourier transform. The frequency
corresponds to the position of the originating spin

3 The Fourier Transform

The frequency analysis of a signal is a mathematical process called

Fourier transform. A Fourier transform yields the frequency spec-
trum of the signal, i.e., the amplitudes of the various frequency
components. The frequency spectrum for the setup described
above (two glasses in a box) is shown in Fig. 6.
As explained above, a spin’s Larmor frequency depends on
its local magnetic field strength and therefore on its position
within the box, as long as the linear field gradient is switched
on. It is therefore fair to say that the frequency spectrum shows
a one-dimensional image of the scanned object. In the special
case of Fig. 6, the two glasses, their respective positions, their
relative filling levels, and even their diameters are clearly
displayed.

4 The Gradient Echo

The gradient echo technique is of major importance in fMRI.

Let us consider the three experiments described in Fig. 7.
Experiments will be discussed on a purely phenomenological basis
first, in particular the experimental setup and the respective signal
behaviors. The explanation will be given afterwards (Fig. 8) .
In the first experiment (Fig. 7, part 1), an initial RF pulse tilts
the magnetization. As expected, precession starts and a signal can
be acquired immediately after sending the RF pulse. This signal has
a relatively long duration.
In the second experiment (Fig. 7, part 2), the signal is
acquired while a gradient is switched on. In this case, the signal
decays much faster. The reason for this will be given below. For
the time being it is sufficient to note that obviously inhomogene-
ities of the static magnetic field, as created by a gradient, cause a
more rapid signal decay.
10 Ralf Deichmann

Fig. 7 Schematic sketch of three MR experiments: in the absence of any gradients, a long signal can be
observed (1). In the presence of a gradient, the signal decays more rapidly (2). If the gradient is inverted, a
gradient echo occurs (3)

The third experiment (Fig. 7, part 3) starts off like the second
one, resulting in the same fast signal decay. However, after a while
the gradient is inverted (a negative gradient Gx means that the
magnetic field strength decreases in x-direction, rather than
increasing). This leads to a striking phenomenon: the signal which
seemed to have disappeared completely, suddenly comes back.
This effect is called gradient echo.
It is relatively simple to explain these effects. For illustration,
Fig. 8a shows the precession of four individual spins at different
positions in the first experiment after the initial RF pulse (the respec-
tive magnetization vectors are seen “from top”). Although the spins
are located at different positions, they are exposed to the same mag-
netic field because there are no gradients. Thus, they precede with
the same Larmor frequency; their magnetization vectors are always
parallel and add up to a relatively strong net magnetization.
In theory, this should go on forever. In practice, the signal decays
due to transverse relaxation effects which will be discussed later.
Figure 8b shows the respective sketch for the second experi-
ment. Due to the field gradient, the spins are exposed to different
field strengths and precede with different Larmor frequencies. After
a relatively short time, they are completely dephased, i.e., the mag-
netization vectors cancel each other and there is no net magnetiza-
tion. The result is a fast signal decay, as depicted in Fig.7 (part 2) .

Fig. 8 (continued) dephase, reducing the duration of the signal. (c) Explanation of the third experiment: the
gradient inversion leads to a change from anti-clockwise to clockwise rotation, so spins rephase. A strong
signal, the so-called gradient echo can be observed when the magnetization vectors are parallel again
Fig. 8 (a) Explanation of the first experiment: spins at different positions still have the same Larmor frequencies
as the magnetic field is homogeneous. As a consequence, their magnetization vectors remain parallel and sum
up to a strong net magnetization over a relatively long time. (b) Explanation of the second experiment: spins
at different positions have different Larmor frequencies due to the field gradient. Their magnetization vectors
12 Ralf Deichmann

Fig. 9 Signal losses due to the finite gradient rise time: by the time the gradient
has reached its full amplitude, the signal has decayed considerably

Figure 8c shows the respective sketch for the third experi-

ment. Due to the gradient, the spins dephase, as described above.
However, when the gradient is inverted the spins turn backwards,
maintaining their frequencies. As a result, they rephase (i.e., they
return to their original positions), resulting in a realignment which
corresponds to a reappearance of the signal. This is the origin of
the gradient echo.
The gradient echo helps to overcome a typical problem in
MR imaging.
According to Fig. 5, a signal has to be acquired while the
read gradient is switched on. However, due to technical limita-
tions, gradients have a certain rise time, i.e., a certain time delay
(typically several hundreds of μs or even several ms) is required
to ramp up the gradient (Fig. 9, shaded area). This leads to a
dilemma: on one hand, one has to wait for the gradient to reach
its plateau level before signal acquisition can start, because only
then there is a well-defined relationship between the position of
a spin and its Larmor frequency, as required to deduce spatial
information from the frequency spectrum. On the other hand,
during the process of ramping up the gradient spins start to
dephase, so we will have lost a considerable part of the signal by
the time the acquisition starts, resulting in a poor image
quality.
This problem can be overcome by using the gradient echo
concept as shown in Fig. 10: by means of an initial negative
gradient, spins are dephased deliberately. Rephasing and the
occurrence of a gradient echo take place during the plateau of
the read gradient, so we can measure a strong signal at a time
when the gradient is constant. Gradient echo sequences are
widely used in MR imaging.
The experiment described in Fig. 10 is one-dimensional, i.e.,
it allows for spatial resolution in one direction (the x-direction)
only. If the gradient axes are chosen as shown in Fig. 11 (left), the
result is a profile in anterior/posterior direction (Fig. 11, right).
The full extent of the imaged object in y- and z-direction is pro-
jected onto the x-axis.
 Principles of MRI and Functional MRI 13

Fig. 10 Solution of the problem imposed by the finite gradient rise times: the
initial negative gradient creates a gradient echo and thus a strong signal at a
time when the read gradient has reached its full amplitude. The maximum of the
echo occurs when the shaded areas are identical

Fig. 11 If the gradient axes are chosen as shown on the left-hand side, the result of the one-dimensional
experiment with a read gradient in x-direction is a profile in anterior/posterior direction

5 The k-Space

Before we move on to two-dimensional experiments (which allow

us to obtain real images), the so-called k-space will be introduced.
The k-space is a very useful concept when it comes to describing
and understanding MR sequences.
Imagine a gradient is turned on for a certain time. During this
time, a single data point is acquired (Fig. 12). The k-value of this
data point is the area under the preceding gradient (shaded), i.e.,
the area under the gradient up to the time point of acquisition.
This is simply a definition.
14 Ralf Deichmann

Fig. 12 Definition of a data point’s k-value as the area under the gradient before
the data point is sampled

Let us now analyze our one-dimensional MR experiment as

depicted in Fig. 10. The signal acquisition consists basically of the
acquisition of a series of discrete data points (Fig. 13) with differ-
ent k-values.
The first data point is preceded by the negative dephasing gra-
dient, so it has a negative k-value. The next data point “sees” a
combined preparation: the negative dephasing gradient, followed
by the first bit of the positive read gradient. Thus, its k-value (being
the sum of individual areas under the preceding gradient) is still
negative, but slightly higher than for the first data point. The sub-
sequent data points have increasing k-values. The central data point
is acquired when the shaded areas in Fig. 13 are identical. Thus, its
k-value is zero. As described above, this is where the center of the
gradient echo occurs, i.e., this data point will have the highest signal
amplitude. The subsequent data points have increasing, positive
k-values and the maximal k-value is attained for the last data point.
We learnt above that spatial resolution in one direction is
achieved by acquiring a signal while a gradient in this direction is
switched on. We further know that this gradient should be pre-
ceded by a negative dephasing gradient to obtain a gradient echo.
The k-value concept allows us now to move on to an alternative
formulation. However, please note that this formulation is basically
identical to the previous one:
Spatial resolution in one direction is achieved by acquiring a
series of data points with different k-values, ranging from a nega-
tive to a positive value. Maximum signal is attained for the data
point for which the respective k-value is zero.
This concept was introduced because it makes it much easier to
understand how two-dimensional imaging works. Basically, we
need spatial resolution and thus gradients in two directions (the
x- and the y-direction), so in general we have to attribute a kx and
a ky value to each data point, corresponding to the areas under the
respective preceding gradients (Fig. 14).
To visualize these k-values, we can create a coordinate system
with the axes kx and ky and insert the data point at the respective
position (Fig. 15). This is the so-called k-space. In the example of
Fig. 14, there is a relatively large positive kx and a relatively small
ky, so the position of this data point in k-space would be more or
less as shown in Fig. 15.
 Principles of MRI and Functional MRI 15

Fig. 13 Gradient echo experiment: the data points constituting the gradient echo
have increasing k-values, ranging from a negative to a positive value. The k-value
is zero for the central data point where the shaded areas are identical and the
echo has maximum amplitude

Fig. 14 Definition of a data point’s kx- and ky-values as the areas under the
respective gradients before the data point is sampled

Fig. 15 Description of a data point’s kx- and ky-values in k-space

16 Ralf Deichmann

6 Two-Dimensional Acquisition

It is now easy to extend the one-dimensional concept described

above to two dimensions:
Spatial resolution in two directions (x, y) is achieved by acquir-
ing a series of data points with different combinations of kx - and
ky -values, filling the two-dimensional k-space as shown in Fig. 16.
Maximum signal is attained for the central data point for which
both k-values are zero.
Let us now consider the experiment depicted in Fig. 17. It
closely resembles the experiment discussed in Fig. 13: the RF axis
and the Gx axis are identical. This means that the data points have
increasing kx-values, ranging from a negative to a positive value.
However, there is in addition a Gy axis, showing a negative gradi-
ent which is switched on and off before the acquisition starts. Since
this gradient is off during acquisition, all data points have the same
(negative) ky-value, corresponding to the area under Gy. Thus, the
ky-value is constant, whereas the kx-value increases. This means
that the experiment acquires a single horizontal line in k-space.
To obtain spatial resolution in two dimensions, we have to
acquire several horizontal lines in k-space. This means that we have
to repeat the experiment from Fig. 17 several times with different
amplitudes of the gradient Gy. In textbooks and publications, this
is usually depicted as shown in Fig. 18: the gradient Gy appears as
a “ladder” with an arrow, which means that the acquisition is
repeated several times with different discrete Gy values, stepping
from a minimum to a maximum value.
It should be noted that the gradient Gy is also referred to as
phase gradient or phase encoding gradient.
The experiment described in Fig. 18 is two-dimensional, i.e., it
allows for spatial resolution in two directions (the x- and

Fig. 16 The basis of two-dimensional imaging: several data points with different
combinations of kx- and ky-values have to be sampled, filling the two-
dimensional k-space
 Principles of MRI and Functional MRI 17

Fig. 17 A subset of a two-dimensional MR experiment. The echo covers a single

horizontal line in k-space

Fig. 18 Schematic description of a full two-dimensional MR experiment: the

experiment shown in the previous figure is repeated several times with different
values of the gradient Gy, so several horizontal lines in k-space are covered

y-direction). If the gradient axes are chosen as shown in Fig. 19

(left), the result is an image in the axial plane (Fig. 19, right). The
full extent of the imaged object in z-direction is projected onto this
plane, i.e., all axial slices are still added up.

7 Slice-Selective Excitation

So far, we have covered spatial resolution in two dimensions. To

obtain spatial resolution in the third dimension, it would be useful
to have a kind of “intelligent” excitation pulse which tilts the mag-
netization only inside a slice of interest. This would mean that only
spins within this slice could contribute to the signal, and we could
18 Ralf Deichmann

Fig. 19 If the gradient axes are chosen as shown on the left-hand side, the result
of the two-dimensional experiment with a read gradient in x-direction and a
phase gradient in y-direction is an image in the axial plane, showing an overlay
of all axial slices

subsequently employ the experiment described above to obtain

spatial resolution in the remaining two directions. These slice-
selective excitation pulses exist. They are based on the following
principles. Let us assume we want to excite an axial slice through
the brain (Fig. 20). As a first step, we switch on a gradient in the
spatial direction perpendicular to this slice (z-direction).
Consequently, the Larmor frequency of the spins will depend on
their position: spins inside the slice of interest have a certain Larmor
frequency f0, whereas spins in upper/lower parts of the brain have
higher/lower Larmor frequencies. If we send an RF pulse with the
frequency f0 while the gradient is switched on, it will tilt the mag-
netization only inside the slice of interest (please remember that
spin excitation is a resonance effect and affects only those spins
whose Larmor frequency matches the frequency of the incoming
RF pulse). In summary, we can say that an RF pulse which is trans-
mitted while a field gradient is turned on causes a slice-selective
excitation. After this special kind of excitation, we can proceed as
shown in Fig. 18 to achieve spatial resolution in two dimensions
within the slice of interest.
The complete imaging experiment with spatial resolution in
three dimensions is shown in Fig. 21. Basically, it is similar to
Fig. 18, but comprises a slice selective excitation, including the
slice gradient Gz.
The latter requires some further explanations. As shown in
Fig. 7 (part 2) and Fig. 8b, gradients cause dephasing of the spins
and thus signal losses. The second half of the slice gradient (i.e.,
the part of the slice gradient that comes after sending the RF pulse)
would have a similar effect. To avoid signal losses, a negative
rephasing gradient has been added after the slice gradient which
 Principles of MRI and Functional MRI 19

Fig. 20 The basis of slice selective excitation: due to the slice gradient Gz spins
have spatially dependent Larmor frequencies. A radiofrequency pulse with fre-
quency f0 can only excite spins whose Larmor frequency corresponds to f0. These
spins are located in a plane perpendicular to the gradient direction

Fig. 21 Schematic description of a full three-dimensional MR experiment with

slice selective excitation

compensates this effect, as described for the gradient echo. Full

compensation is approximately achieved if the shaded areas on the
Gz axis in Fig. 21 are identical.
The experiment described in Fig. 21 is three-dimensional, i.e.,
it allows for spatial resolution in all directions. If the gradient axes
are chosen as shown in Fig. 22 (left), the result is an image in the
axial plane (Fig. 22, right) with a finite slice thickness.
20 Ralf Deichmann

Fig. 22 If the gradient axes are chosen as shown on the left-hand side, the result
of the three-dimensional experiment with a read gradient in x-direction, a phase
gradient in y-direction, and a slice gradient in z-direction is an image in the axial
plane with a finite slice thickness

8 Echo-Planar Imaging

The experiment described in Fig. 21 can be relatively time consum-

ing, because it requires an RF pulse for each single echo, i.e., for
each line in k-space. Given the duration of RF pulses (typically sev-
eral ms) it would be far more efficient to acquire all echoes after a
single excitation pulse. One of these single shot sequences, dubbed
echo-planar imaging (EPI), was developed by Mansfield [4] and is
nowadays widely used in functional imaging experiments. It is based
on the acquisition of multiple gradient echoes as described in Fig. 23.
The initial part of this experiment is identical to the one shown
in Fig. 13: after the RF pulse, a negative gradient causes dephasing
of the spins. During the subsequent positive read gradient, a
gradient echo is acquired. The k-values of the data points consti-
tuting this echo increase, ranging from a negative to a positive
value. Afterwards, the read gradient is inverted. It is obvious that
this will result in another gradient echo with decreasing k-values,
which means that the second echo covers the same k-values as the
first one, only in reverse order. After a further gradient inversion, a
third echo can be acquired which covers the k-values in exactly the
same way as the first one. Of course, this is only a one-dimensional
experiment, because no gradients in y-direction are used.
A two-dimensional expansion is shown in Fig. 24. As before, a
series of gradient echoes is acquired by means of an oscillating read
gradient. However, a phase gradient with negative amplitude is
switched before the first echo (shaded), so all data points constituting
the first echo have the same (negative) ky-value, covering a horizontal
line in k-space (Fig. 24, bottom). Between the acquisition of the first
and the second echo, a very short phase gradient pulse is switched (a
 Principles of MRI and Functional MRI 21

Fig. 23 Acquisition of multiple gradient echoes by successive read gradient inversion. All echoes cover the
same k-values, but the order is reversed when comparing even and odd echoes

Fig. 24 The basis of echo planar imaging: due to intermediate “blips” (shaded gradient pulses) the echoes have
increasing ky-values, thus covering different lines in k-space

so-called blip, shaded in Fig. 24). The ky-value of the second echo is
determined by the sum of the areas under the initial phase gradient
and this blip, so ky is increased and the second echo covers another
horizontal, slightly “higher” line in k-space in reverse direction. This
concept of intermediate blips is maintained throughout the remain-
ing acquisition, resulting in a meander-like journey through k-space.
22 Ralf Deichmann

Fig. 25 A complete echo planar imaging experiment. The central echo which
covers the center of k-space has the highest amplitude

Figure 25 shows the complete EPI experiment, including a slice

gradient Gz (with a subsequent negative rephasing gradient) for
slice-selective excitation. The echoes have different amplitudes due
to their different positions in k-space, with the highest amplitude
for the echo covering the center of k-space, as described before.
The main advantage of the EPI sequence is its speed due to the
use of a single RF pulse only, with typical acquisition times of
50–100 ms per slice. Another advantage is its susceptibility to the
blood oxygen level-dependent (BOLD) effect [5, 6] which is
exploited in the majority of functional imaging studies and will be
discussed in a later section.

9 The Transverse Relaxation Times T2 and T2*

The first experiment in Fig. 7 (part 1) describes the acquisition of

an MR signal after sending an excitation pulse, assuming the
absence of any field gradients. According to Fig. 8a, spins at differ-
ent locations will have the same Larmor frequency, so there are no
dephasing effects, and in theory there should be no signal loss at
all. However, the signal will still decay due to an effect called trans-
verse relaxation. This is caused by the spin-spin interaction: in the
classical view, the spins randomly exchange small amounts of
energy, resulting in minor fluctuations of their Larmor frequencies
and thus in gradual signal dephasing, even in otherwise perfectly
homogeneous fields. The signal decays exponentially with a time
constant called the transverse relaxation time T2. In white matter
and gray matter, T2 has an approximate value of about 100 ms.
In reality, the signal would decay with a time constant even
shorter than T2. This is due to the following effect: tissue is not a
 Principles of MRI and Functional MRI 23

homogeneous piece of matter, but consists of several, often micro-

scopic components, for example arterioles and venules. In general,
these components have slightly different magnetic properties, so
they distort the magnetic field and create microscopic field gradi-
ents of varying amplitude and direction. As shown in Fig. 7 (part 2)
and Fig. 8b, the presence of field gradients speeds up the signal
decay due to spin dephasing. Thus, the signal decays with the effec-
tive transverse relaxation time T2* which is shorter than T2. This
time constant depends on the scanner field strength. In white and
gray matter, T2* amounts to about 70 ms at 1.5 T and 45 ms at 3 T.
For several applications (including the most common fMRI
techniques), it is useful to acquire so-called T2* weighted images,
i.e., images where the local signal intensity depends on the local
T2* value. The degree of T2* weighting can be influenced by modi-
fying a certain acquisition parameter, the echo time (TE) .

10 The Echo Time

All MR experiments that were discussed so far are based on the

same concept: an initial excitation pulse tilts the magnetization.
After a certain time (during which one or more gradients are
switched on and off) a signal is acquired. The time delay between
excitation and acquisition is called the echo time (TE) (Fig. 26).
To achieve a certain degree of T2* weighting, TE must be cho-
sen carefully. This is depicted in Fig. 27, showing a fast (dashed line)
and a slow (bold line) T2* decay, and three different choices for TE.
The first choice (TE much shorter than T2*) would be prob-
lematic, because signal amplitudes are almost identical, so T2* con-
trasts would be poor.
The second choice (TE similar to T2*) would yield a much bet-
ter T2* contrast.
The third choice (TE much longer than T2*) is again problem-
atic: signal has decayed in both compartments, so the image would
show noise only, but hardly any structures.
As an example, Fig. 28 shows T2* weighted brain images
acquired with TE values of 10 ms, 50 ms, and 200 ms. At 10 ms,
contrasts are relatively low. At 50 ms, a nice T2* contrast can be
observed (arrow). This is due to an increased iron content which

Fig. 26 Definition of the echo time (TE)

24 Ralf Deichmann

Fig. 27 Signal decay in compartments with a long T2* value (solid line) and a short
T2* value (dashed line). If a short echo time (TE) is chosen (1), there is a high
signal amplitude, but hardly any contrast between both compartments. For an
intermediate TE (2), there is a good contrast and still a sufficient signal ampli-
tude. For a long TE (3), the signal in both compartments has decayed

Fig. 28 T2* weighted brain images acquired with echo time (TE) values of 10, 50, and 200 ms. At 10 ms, con-
trasts are relatively low. At 50 ms, a nice T2* contrast can be observed (arrow). At 200 ms, signal has mostly
decayed and only cerebrospinal fluid is visible

gives rise to magnetic field distortions and therefore reduces the

T2* value. At a TE of 200 ms, signal has mostly decayed and only
cerebrospinal fluid is visible due to its longer T2* value.

11 The Basis of the BOLD Contrast

The majority of fMRI techniques are based on the BOLD contrast

which will be explained in this section.
The BOLD contrast is closely linked to two physical phenom-
ena, called “diamagnetism” and “paramagnetism.” A full discus-
sion would be beyond the scope of this book. To understand the
BOLD effect, it is sufficient to know the following facts: if a dia-
magnetic substance is brought into an external magnetic field, it
tends to decrease slightly this field, whereas a paramagnetic sub-
stance tends to increase it. This means that the close vicinity of
paramagnetic and diamagnetic substances causes a local distortion
of the magnetic field near the interface.
 Principles of MRI and Functional MRI 25

Tissue is mainly diamagnetic. In contrast, blood contains a

certain level of deoxyhemoglobin (i.e., hemoglobin that does not
have oxygen attached) which is paramagnetic. Thus, the presence
of blood in tissue means a close vicinity of substances with different
magnetic properties, giving rise to microscopic field distortions. As
explained above, the resulting field gradients cause spin dephasing
and lower the T2* value. In summary one may say that due to the
presence of deoxyhemoglobin, the signal intensity of tissue is
slightly reduced in T2* weighted images.
After neuronal activation, blood is locally hyperoxygenated,
corresponding to a wash-out of deoxyhemoglobin and an increased
concentration of oxyhemoglobin. In contrast to deoxyhemoglo-
bin, oxyhemoglobin is diamagnetic, so it has similar magnetic
properties as tissue, leading to a more homogeneous magnetic
field and an increased signal intensity in T2* weighted images.
Thus, the basic concept of the BOLD effect can be summa-
rized as follows: the hemodynamic response to brain activation con-
sists in a decrease in deoxyhemoglobin and an increase in oxyhemoglobin,
resulting in an increased field homogeneity and thus a higher sig-
nal intensity in a series of T2* weighted images. Therefore, quanti-
fication of this signal enhancement allows for the detection of
neuronal activation.
In reality, the physiology of the BOLD effect is more complex
and depends on the following parameters: the cerebral blood flow
(CBF), the cerebral blood volume (CBV), and the metabolic rate
of oxygen consumption (CMRO2). After a stimulus, the CBF goes
up to deliver more oxygen to the site of neuronal activation, caus-
ing the BOLD effect as explained above. On the other hand, the
CMRO2 is increased, so more oxygen is consumed, which reduces
the BOLD effect. The question arises if the first effect outpaces the
second one, which is a prerequisite for blood hyperoxygenation
and thus the detectability of the BOLD signal.
The exact physiology of the BOLD response is still controver-
sial and several models have been proposed. For a more detailed
overview, the reader is referred to the literature [7]. In the follow-
ing section, one of the most common models will be explained.
Figure 29 is a simplified sketch, showing how the physiological
parameters are affected by neuronal activation and how their interac-
tion influences the signal intensity in T2* weighted images. Directly
after the stimulus, CMRO2 goes up, causing increased oxygen con-
sumption and thus increased concentration of deoxyhemoglobin. As
explained above, deoxyhemoglobin lowers the signal intensity in T2*
weighted images, so there is an initial signal reduction with a duration
of about 1 s, the so-called initial dip. It should be noted that this effect
is small and not always present. About 1 s after the stimulus, the brain
reacts by increasing the CBF, transporting oxygen to the site of activa-
tion. Fortunately, this effect outpaces the increase in CMRO2, so blood
becomes in fact hyperoxygenated. At the same time, the CBV is
26 Ralf Deichmann

Fig. 29 Change of physiological parameters, the concentration of deoxyhemoglobin, and the T2* weighted
signal amplitude in response to neuronal activation

increased. As this parameter determines the total amount of blood, an

increased CBV causes an increased quantity of deoxyhemoglobin.
However, this effect is again outpaced by the increase in CBF, and for
a period of 4–6 s blood is hyperoxygenated, giving rise to a positive
BOLD response. After this, CMRO2 and CBF return to their baseline
values. The relaxation of CBV is somewhat slower, so for a certain time
there is an increased concentration of deoxyhemoglobin, resulting in a
post-stimulus signal undershoot with a duration of about 30 s.
In summary, the right-hand side of Fig. 29 shows the complete
signal behavior following neuronal activation, the so-called hemo-
dynamic response function (HRF): after the initial dip, there is a
strong positive BOLD response, followed by a small negative sig-
nal undershoot.

12 Choice of TE in fMRI Experiments

The question arises, which value of TE should be chosen to maxi-

mize the BOLD signal, and how to set up the parameters of the
EPI sequence described above to achieve this value.
Figure 30 shows the theoretical dependence of the BOLD sen-
sitivity on TE (which is given in units of T2*). The results are not
surprising and correspond to the previous discussion (see Fig. 27):
at short TE, the BOLD sensitivity is low because there is hardly
any T2* weighting. At long TE, the BOLD sensitivity goes down
because the signal decays due to transverse relaxation effects.
 Principles of MRI and Functional MRI 27

Fig. 30 Dependence of the blood oxygen level-dependent (BOLD) sensitivity on

the chosen echo time (TE)

Maximum BOLD sensitivity is achieved when TE equals T2* (about

70 ms at 1.5 T and 45 ms at 3 T).
However, there is a fundamental problem with EPI sequences
used in fMRI experiments: the magnetic field is usually distorted in
brain areas that are close to air/tissue interfaces, in particular in the
orbitofrontal cortex (due to the vicinity of the nasal sinus) and the
temporal lobes (due to the vicinity of the ear canals). In these areas,
local macroscopic field gradients lower the T2* value, resulting in sig-
nal losses in EPI sequences with relatively long TE values. This shows
the basic dilemma with fMRI experiments based on the BOLD effect:
on one hand, T2* weighting is required to be able to detect the hemo-
dynamic response to neuronal activation. On the other hand, T2*
weighting causes severe signal losses in certain brain areas. Therefore,
it is advisable to keep TE as short as possible to avoid signal losses in
these areas, but still as long as necessary to detect a BOLD signal.
According to Fig. 30, a decent BOLD sensitivity can still be achieved
if TE corresponds to about 2/3 of T2*. The general advice is there-
fore to use a TE of about 50 ms at 1.5 T, and 30 ms at 3 T.
The next question is how TE can be defined for the EPI
sequence. As shown in Fig. 25, EPI implies the acquisition of a
series of gradient echoes after a single excitation pulse, i.e., each
echo has a different echo time. However, as explained above these
echoes have different amplitudes, with maximum signal strength
for the echo that covers the center of k-space (usually the central
echo in the series if symmetric sampling is used). Therefore, the TE
value of an EPI sequence is defined as the echo time of this central
echo (Fig. 31). This shows that in EPI sequences TE is not simply
28 Ralf Deichmann

Fig. 31 Definition of echo time (TE) for an echo-planar imaging (EPI) sequence

a wasted waiting time (as one might have expected from Fig. 26),
but can be used for the acquisition of the first half of the echo train,
being another reason why EPI allows for a high temporal resolu-
tion in fMRI experiments.

References
1. Bloch F, Hansen WW, Packard M (1946) 5. Ogawa S, Lee TM, Nayak AS, Glynn P (1990)
Nuclear induction. Phys Rev 69:127 Oxygenation-sensitive contrast in magnetic res-
2. Purcell EM, Torrey HC, Pound RV (1946) onance image of rodent brain at high magnetic
Resonance absorption by nuclear magnetic fields. Magn Reson Med 14:68–78
moments in a solid. Phys Rev 69:37–38 6. Kwong KK, Belliveau JW, Chesler DA et al
3. Lauterbur PC (1973) Image formation by (1992) Dynamic magnetic resonance imaging
induced local interactions: examples employing of human brain activity during primary sensory
nuclear magnetic resonance. Nature stimulation. Proc Natl Acad Sci USA 89:
242:190–191 5675–5679
4. Mansfield P (1977) Multiplanar image forma- 7. Buxton RB, Uludag K, Dubowitz DJ, Liu TT
tion using NMR spin echoes. J Phys C (2004) Modeling the hemodynamic response
10:L55–L58 to brain activation. NeuroImage 23:S220–S233
 Chapter 2

Introduction to Functional MRI Hardware

Luis Hernandez-Garcia, Scott Peltier, and William Grissom

Abstract
The chapter gives an overview of peripheral devices commonly used in fMRI experiments, and it addresses
the principles, performance aspects, and specifications of fMRI hardware. The general guidelines for
MR-compatible hardware are also discussed. The target audience is quite broad and mathematical descrip-
tions are kept to a minimum and qualitative descriptions are favored whenever possible.

Key words Functional MRI, Hardware, Peripheral devices, MRI, Multimodal acquisition,
Neuroimaging

1 Introduction

This chapter is concerned with both the MRI hardware compo-

nents and the multitude of peripherals that are necessary for func-
tional MRI. Our primary aim is to describe the different components
of each subsystem and to identify the important features and
parameters. We hope this chapter will be of some use to those who
want to get a deeper understanding of the electronics involved, but
we mostly want to convey how each of the parts is responsible for
the quality (or lack thereof) of the images and the experiment in
general. Thus we will try to give minimum requirements for the
performance of each component and describe what happens when
those requisites are not met.
There is a myriad of subsystems to explore and we cannot pos-
sibly do justice to all of them in this chapter, so we will limit our-
selves to the main ones and to those that most commonly affect the
performance of the system.
While this chapter primarily addresses the principles, perfor-
mance aspects, and specifications of the functional MRI hardware,
it is important that we keep in mind that the objective is to carry
out experiments on human subjects performing cognitive tasks.
Thus we will also keep in mind the ergonomics and safety charac-
teristics of the equipment.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_2, © Springer Science+Business Media New York 2016

29
30 Luis Hernandez-Garcia et al.

Our target audience is fairly broad so we will try to keep

mathematical descriptions to a minimum and give qualitative
descriptions whenever possible. However, some of the descrip-
tions and the requirements make a lot more sense in the context
of the mathematical description of image acquisition and recon-
struction. We also hope that this chapter can serve as a good
introduction to each topic for those interested in the specific sub-
jects. In a way, we approach this chapter as if we were trying to
give advice to someone who is considering setting up a functional
MRI facility and/or establishing a set of quality control protocols
for such a facility. We will be careful to leave our descriptions as
general as possible and to avoid endorsing specific vendors or
commercial products.
In this second edition, we have done our best to update the
contents to reflect the rapid progress in MR technology. In the last
few years, we have seen considerable advances in the areas of paral-
lel imaging, high-field magnets and gradient performance, and
widespread adoption of these technologies. There are also exciting
new developments in the construction of superconducting mag-
nets that are reducing the need for liquid Helium. Complementary
brain activity monitoring techniques are being used in conjunction
with MRI, such as fNIRS, and new brain stimulation techniques
are on the horizon, such as transcranial Direct Current Stimulation.

2 The MRI Scanner Environment

Let us begin by considering the surroundings of the MRI scanner.

When deciding on the layout and location of an MRI scanner, there
are several important questions one must ask. The first one is the
scanner’s purpose. Will it be used for clinical purposes or will it be
dedicated to research on healthy subjects? In the case of research-
dedicated scanners, one must think about a number of specific fac-
tors when designing the layout of the fMRI laboratory. This includes
dressing rooms, and testing rooms where the subjects can be trained
on the experimental paradigm prior to scanning. Whenever possi-
ble, it is important to make sure the control room is large enough
to accommodate the needs of the researchers using it. It is not
uncommon for fMRI experiments to require multiple pieces of cus-
tom stimulation/recording equipment in the control room and for
several investigators to be present during the experiment, so that
extra bench space and “elbow room” is very advantageous. When
scanning clinical populations there are additional precautions and
considerations, like the presence of MR compatible first aid equip-
ment. We will not address that aspect in detail, as it can be an exten-
sive discussion that will vary from case to case.
The first thing one notices about an MRI scanner is how big it
is (the magnet alone can take up a space of roughly 4 × 4 × 6 m) and
 Introduction to Functional MRI Hardware 31

Fig. 1 These photos were taken during the installation of an MRI scanner at Resurgens Orthopaedics in Atlanta,
GA on the 19th floor of their Crawford facility. Courtesy of Resurgens Orthopaedics, Atlanta, GA

one must find a site with sufficient space for the scanner. Access is
also important, as the main magnet is typically delivered and
installed in one piece. Furthermore, MRI magnets are filled with
cryogens, which are delivered in large dewars. Thus there must be
a wide path to the loading dock that avoids stairs and is clear of
obstacles. Typically, MRI scanners are housed in basements or
ground floors near the loading docks of hospitals, although excep-
tions exist, as the one shown in Fig. 1.
Next one must consider how the magnetic field will affect its sur-
roundings. Before the magnet is installed, one must consider how far
the magnetic field will extend. Most people who work with MRI are
familiar with the enormous forces that the main magnetic field can
exert on objects in its proximity (i.e. the magnet room). Modern MRI
scanners are actively shielded and contain the magnetic field fairly well
within the magnet room. Even with shielding, sometimes a subtle but
significant magnetic field can extend beyond the walls of the magnet
room. Thus it is important to keep in mind two questions: how well
the magnetic field is contained, and what sort of equipment is in the
rooms adjacent to the magnet room. The first question is usually
answered in terms of the location of the “5 Gauss line.”
The United States FDA regulates that the general public (any-
one not working with an MRI scanner or being scanned) not be
exposed to static magnetic fields over 5 G (5 × 10−4 T), and thus the
5 G boundary must be contained inside the magnet room. MRI
scanner vendors will typically provide contour plots of the m ­ agnetic
field superimposed on the blueprints of the room and provide con-
sultation on the location of the scanner. One must realize, however,
that smaller magnetic fields will extend beyond the walls of the
32 Luis Hernandez-Garcia et al.

magnet room, so it is important to note how quickly the field decays

and what is located on those adjacent rooms. Subtle magnetic fields
can affect electronic equipment in many ways. For example, a mov-
ing charge in the presence of a magnetic field will experience a force
perpendicular to the magnetic field. This effect was most obvious in
CRT monitors (now obsolete), whose images would be skewed by
the magnetic field, and in the performance of computer hard drives
and magnetic media. In fact, magnetic media, including the mag-
netic strips on credit cards, floppy disks, …etc. are typically erased
when taken into the magnet room. Another example is that the life
span of light bulbs near MRI scanners tends to be quite short because
of the vibration of the filaments caused by switching the direction of
the current in the presence of a large magnetic field. Hence, DC
lights are often used in MRI scanner rooms to avoid this problem.
Pace makers, neurostimulators, and implants must be kept outside
the 5 G line, unless they have been tested and classified as MR com-
patible. A number of publications exist [1–3] and are updated regu-
larly with classification of MR compatible devices.
Containment of the magnetic field can be achieved by two
different kinds of shielding. Passive shields consist of building a
­symmetric box around the magnet out of thick iron walls (Fig. 2)
that contain the magnetic field. Alternatively, active shields can
be built as secondary electromagnets concentrically placed around
the main magnet. The shield magnets are built such that the field

Fig. 2 A passively shielded 4.0 T magnet encased in a hexagonal iron cage to partially contain the main mag-
netic field
 Introduction to Functional MRI Hardware 33

they produce points in the opposite direction of the main mag-

netic field. The strength of the shielding field is typically about
half of the main field. For example, a 3 T actively shielded magnet
can often be a 4.5 T magnet surrounded by a second, concentric
and opposing 1.5 T magnet. Such a system would have a 3 T field
inside of the bore, and because of the inverse square dependence
of the magnetic field on the distance to the coil, the field outside
of the magnets bore is dramatically reduced. Figure 3 shows a
plot of the magnetic fields produced by the main magnet as a
function of the distance to the center of the bore. The field pro-
duced by the shield and the net sum of both fields are superim-
posed on the same plot.
One must also consider how the environment will affect the
MRI scanner. In this regard, the most important issue is the pres-
ence of electromagnetic noise sources. MRI scanners construct
images from radio frequency (RF) electromagnetic signals. Thus,
radio stations, cell phones, and other wireless communications will
interfere with the MRI experiment and severely degrade image
quality unless they are properly isolated. MRI rooms are usually
encased in a copper shield box that blocks external RF radiation,
and contains the MRI’s RF radiation as well. The quality of the
shield is critically important to the performance of the scanner and
it must be tested carefully before the magnet is ramped up to field.
Typically attenuations for RF shielding are 100 dB at the operating
frequency range. A slightly defective soldered seam between cop-
per sheets, or a nail going through the copper sheeting are

Active Shielding
15
Original B field
B field after shielding
10
Magnetic field (T esla)

-5 Operating region
(inside of bore)
-10

-15
-300 -200 -100 0 100 200 300
Distance from the center of the bore

Fig. 3 A plot of the magnetic field strength along the radial direction. The thick vertical lines denote the location
of the coil’s windings. The thinner line denotes the operating region where the sample is placed. Ideally, this
region should have a flat magnetic field, and additional fields (shims) are necessary
34 Luis Hernandez-Garcia et al.

Fig. 4 The streaks in the image are caused by the presence of electromagnetic
frequency noise at a single frequency. This is typically introduced by the pres-
ence of badly shielded electronic equipment in the room. The AC power supply
running at 60 Hz to the device is in this case the culprit of the artifact

sufficient to let RF noise into the room that ruins images, so test-
ing of the room shielding must be stressed. Figure 4 shows an
example of the effects of an RF noise source on an MR image
Good shielding of the room is not enough. MR-related
electronic equipment produces RF noise and can act as an
antenna that passively carries RF noise from the outside into the
room. There are also a number of peripherals that are needed
for functional MRI in order to provide stimulation and record
data from the subject (see Sect. 4). It is preferable to keep all
electronics out of the magnet room, but if the electronic equip-
ment must be inside, it must be tested thoroughly for RF noise.
If it is indeed noisy, care must be taken to shield the equipment
to prevent image artifacts (Copper mesh is very useful for build-
ing RF shields).
Consider the case of a button response box. Typically, one
keeps the bulk of the electronic components out of the magnet
room, but the buttons themselves must be in the scanner and they
need to communicate with the response recording electronics.
Even if fiber-optic technology is used to carry signals into the
room, RF noise can enter through the same opening as the fiber
optic cabling. The solution is to build a “penetration panel” into
the shield. This is a panel on the wall that is outfitted with wave-
guides and filtered connectors. The role of a waveguide is to block
any electromagnetic radiation that is not parallel with the direction
of the waveguide, and at the same time, to guide the EM waves
 Introduction to Functional MRI Hardware 35

Fig. 5 On the left is the penetration panel that connects the MRI scanner electronics to the magnet hardware.
The image on the right is a second penetration panel used for all the additional stimulus/response equipment
needed for fMRI

produced inside it. The filtered connectors typically remove high

frequency radiation that may be carried by the cabling of the
peripherals. Examples of penetration panels used for scanner
cabling and for general, user-specific peripherals are shown in
Fig. 5. Additional precautions against RF contamination is the
used of twisted-pair and coaxial cabling to contain the fields pro-
duced inside the transmission lines.
MR scanner equipment must be kept in stable temperature
and humidity conditions, as these can change the performance of
the electronics and the field strength. All the amplifiers and com-
puter equipment required for MR imaging can generate a signifi-
cant amount of heat, so it is important that the environmental
temperature-regulating equipment be powerful enough to han-
dle it. Changes in the scanner performance can mask changes in
brain activity so the scanner’s performance be maintained as con-
stant as possible.
To put things in perspective, the MRI electronics equipment
produces approximately 50,000 BTU per hour. Typical requisites
for the temperature and humidity in the room are variability of less
than 3 °C per hour and 5 % per hour, respectively. Normal operat-
ing ranges are in the 15–32 °C temperature range and 30–70 %
humidity range. Of particular interest are the gradient coils, since
they can heat up significantly as a result of the large currents that
run through them. As gradient coils heat up, their performance is
36 Luis Hernandez-Garcia et al.

degraded and thus they require its own cooling system to keep
them stable. This system is usually a cooling loop involving a water
chiller that can remove about 14,000 BTU per hour.1
It is also noteworthy that the scanner’s performance can be
affected by other unexpected environmental factors. Outside mag-
netic fields and vibrations can be an issue. For example, nearby
construction can produce vibrations that will affect the MR signal’s
stability if the floor is not adequately mechanically damped. It is
thus important to carry out vibration tests of the site before instal-
lation proceeds. Large moving objects, such as nearby trains can
also generate magnetic fields that affect the scanner’s stability [4]
One other issue that can cause a great deal of grief to investiga-
tors is the production of small electromagnetic spikes inside the
magnet room. These occur when metal objects in the room vibrate
(typically because of the gradients) and bang against each other or
when arcing occurs across badly-soldered connections in home-
made equipment. The RF receiver hardware is sensitive enough to
pick up these spikes. Spikes in the k-space data translate into stripe
patterns in the images (Fig. 6). It is thus very important to make
sure that all metal equipment is well secured.
A useful option to consider for a functional MRI lab is a mock
MRI scanner. This is advantageous since it allows subjects to get

Fig. 6 A T2*-weighted image showing white pixel artifact. Spikes in k-space

results in sinusoidal patterns in image spike

1
These numbers are based on the specification of a 3 T scanner by General
Electric (MR750).
 Introduction to Functional MRI Hardware 37

used to the fMRI experience prior to their actual scans, in a safe

environment. This can help alleviate subjects’ apprehension and
claustrophobia, and lead to reduced head motion and improved
task performance.
For completeness, replication of the entire MR suite would be
preferable, but usually this is limited by available space and
resources. The most important factors to reproduce are the spatial
dimensions, audio environment, and visual stimulus presentation
of the MRI scanner.
It is important to have the subject familiarize themselves with
the restrictive space in the MR scanner. This includes the inner
diameters of both the bore of the main magnet and the head coil
being used. These restrictions, combined with the distance the
subject travels into the magnet from the home position of the
patient bed, combine to give the overall physical experience. MR
or CT patient beds and scanner housing may be recycled for this
use, or the mock MR scanner can be built from scratch as the one
shown in Figs. 7 and 8.
The audio environment of the scanner is also an important
consideration. Having an audio recording of the actual scanner to
play in the mock MR scanner will let the patient adjust to the jar-
ring transition when the scanner starts playing sequences, and the
to the volume of this noise throughout the scan. Inclusion of audio
feedback can also demonstrate to the subjects that the scanner
operator will be able to communicate with them between scans.
A duplication of the fMRI visual stimulus presentation can also
serve to acclimatize the subjects. The standard forward- or rear-­
projection systems used to present visual stimuli are relatively easy
to replicate in a non-MR environment, and allow the subject to get
used to task presentation during the scan.

M1

M2
M1 - Bore Inner diameter
M2 – Head coil Inner diameter
M3 distance from isocenter to edge of bore

M3

Fig. 7 Diagram of mock scanner showing critical dimensions

38 Luis Hernandez-Garcia et al.

Fig. 8 Example of a mock MRI scanner (University of Michigan)

An MR mock scanner can help to increase patient comfort and

task performance, especially in some target populations (e.g., indi-
viduals with autism, children), and can be used for screening (e.g.,
individuals with claustrophobia, large physical dimensions). It can
also be useful for the training of fMRI lab personnel or the designing,
troubleshooting, or rehearsing of complicated fMRI experiments.

3 The MRI Scanner

We can divide the components of an MRI scanner into four cate-

gories. The magnet, the magnetic field gradients, the RF Transmit/
Receive hardware, and the data acquisition electronics.

3.1 The Magnet We will begin by considering the magnet. Typical MRI magnets,
like the one shown in Fig. 9, are large solenoid coils made of super-
conducting metal (niobium alloys, typically). They are kept cooled
at approximately 4 K by liquid Helium in order to achieve and
maintain superconductivity.2 The magnet is “ramped up” to field
by introducing a current through a pair of leads that produces the
desired magnetic field. Once the specified current has been built
up, the circuit is closed such that the current “re-circulates”
through the coil constantly and there is no need to supply more
power to it. It is crucial to maintain the low temperature to prevent
the coil from resisting the current flow.
A new, exciting development is the “dry magnet.” The term
refers to super-conducting magnets that are cooled without liquid

2
For more information on superconductivity, see [5]
 Introduction to Functional MRI Hardware 39

Fig. 9 A 3T magnet during installation at the University of Michigan’s FMRI

Laboratory

helium. This new generation of magnets uses cryo-coolers [6] that

run constantly in order to maintain the temperature of the magnet
in the super-conducting range. Cryo-coolers still use helium,
although in its gaseous state and in much smaller quantities. A
major advantage of this cooling technology is that it allows ramp-
ing the magnet up and down much more safely, quickly, and inex-
pensively, since no liquid helium fills are required in the process. At
the time of this writing and to our knowledge, these are only cur-
rently available from one vendor of MRI systems (MR solutions)
and only for preclinical systems. It is expected that this technology
will be adopted for clinical MRI systems in the near future, but
most MRI scanners in the world are still cooled by liquid Helium.
If the windings ever warm up and become resistive (i.e., they
lose their super-conducting state), they dissipate the electric power
as heat. This very undesirable event is termed “quenching.” As the
magnet’s windings become more resistive, the very high current
circulating through the coil produces heat, such that the liquid
helium that is responsible for maintaining the superconducting
temperature boils off. This rapid boiling of the helium quickly
exacerbates the problem and the superconducting state is quickly
lost. The very large currents can potentially produce enough heat
to melt or severely damage the windings. The greatest danger
however, is that the rapid rate of helium boiling can build up a
great deal of pressure in the magnet and also flood the room with
helium gas and suffocate whoever is there. While helium is not
toxic, it displaces the oxygen in the room. Thus, it is crucial that
the magnet be outfitted with emergency vents (manufacturers of
MRI scanners typically include emergency quench ventilation sys-
tems). Additionally, magnet rooms are outfitted with oxygen
40 Luis Hernandez-Garcia et al.

sensors that sound an alarm when the oxygen level falls below safe
levels. It must be stressed that all personnel be trained in emer-
gency quench procedures in case the emergency systems fail.
Having considered what can happen when the magnet fails, let us
now return to the more cheerful subject of what the magnet can do.
The key parameter in the magnet is its field strength (B0), as it
determines many properties of the images. Primarily, field strength
determines the amount of spins that align with and against the
field. The higher the field strength, the larger the population of
aligned spins that can contribute to the MR signal. More specifi-
cally, the population of spins aligned with the magnetic field (n+)
and against it (n−) is described by the Boltzmann equation

æ n+ ö æ g hB0 ö
ç ÷ = exp ç ÷ (1)
è n- ø è kT ø

where γ is the gyromagnetic constant for the material, h is Plank’s

constant, k is Boltzmann’s constant, T is the temperature of the
sample, and B0 is the strength of the magnetic field. Hence, the
higher the field, the more spins will contribute to the signal and
thus yield a higher the signal to noise ratio (SNR) .
The field strength also determines the resonance frequency of
the spins, ω0, in a linear fashion according to the Larmor
equation.

v 0 = g B0 (2)

Where γ is again the gyromagnetic constant, which is specific for

the nucleus in question. The resonant frequency will in turn deter-
mine the characteristics of the RF transmit and receiver subsystems
(see Sect. 3.3). Field strength also affects both the longitudinal and
transverse relaxation rates of the materials via the resonant fre-
quency as predicted by the equations

1 ì tc 4t c ü
aí + 2 2 ý
(3)
T1 î1 + w0 t c 1 + 4w0 t c þ
2 2

and

1 ì 5t c 2t c ü
a í3t c + + 2 2 ý
(4)
T2 î 1 + w0 t c 1 + 4w0 t c þ
2 2

where τc is the correlation time (a measure of the tumbling rate

and frequency of collisions between molecules) of the species.
Note that T1 is more heavily dependent on B0 than T2.
While T1 and T2 typically get longer, T2* gets shorter at higher
fields. Recall that T2* is the rate of transverse signal loss accounting
for both T2 and macroscopic field inhomogeneity, that is
 Introduction to Functional MRI Hardware 41

1 1 1 1
= + = + gDB0 (5)
T2 * T2 T2 ¢ T2

where T2′ is the relaxation due purely to the field inhomogeneity.

This inhomogeneity in the magnetic field is usually produced by
inhomogeneity in the magnetic susceptibility across the sample,
e.g., the air in the ear canals has very different susceptibility than
the water in brain tissue. The distortion of the magnetic field
caused by magnetic susceptibility is described by

B0¢ = B0 (1 - c ) (6)

where χ is the magnetic susceptibility of the sample, B0 is the origi-

nal magnetic field and B0′ is the resulting magnetic field after con-
sidering the magnetic susceptibility. Thus the change in the field
gets worse as the magnetic field increases.
The implications for functional imaging are that most imaging
and spectroscopy applications benefit in terms of SNR, and that
the bold oxygen level-dependent (BOLD) effect is more pro-
nounced at higher fields. But, as is common in MR, there is a
tradeoff. As the field increases, and T2* effects get shorter, suscep-
tibility artifacts get much more pronounced. This is particularly
significant, as the BOLD effect is observed by T2* weighted imag-
ing, which is very sensitive to susceptibility artifacts.
Arterial Spin Labeling techniques [7] also benefit from higher
field strength, as the longer T1 means longer duration of the label.
Another major practical implication of working at a higher field is
that the resonant frequency of protons is proportionally higher,
and thus RF pulses deposit more power into the subject. The US
FDA regulates the amount of RF power that can be used on a
human subject cannot exceed 1.5 W/Kg.
The higher frequency of the pulses also means a shorter wave-
length and the formation of standing waves in the sample during
transmission. Hence, it is more challenging to achieve uniform
excitation patterns across the imaging slice and parts of the imag-
ing slice appear artificially brighter than others. Figure 10 (left)
shows an example of this phenomenon (usually referred to as
“dielectric effects”) in brain images at 3 T [8–10]. The right panel
of the figure shows the corrected image.
To put things in context, at the time of this writing, T2*
weighted imaging techniques required for BOLD fMRI are fairly
challenging at 7 T and not many groups are doing human work at
these high fields, although there is an increasing trend toward
7 T. Presently there are only two research groups that have 9 T
human imaging systems. At this time, 7 T magnets are predomi-
nantly used for small animal research systems. The “de facto” stan-
dard field strength for human fMRI systems in the last few years has
become 3 T, although many sites still use 1.5 T scanners for fMRI.
42 Luis Hernandez-Garcia et al.

Fig. 10 Illustration of the dielectric effect on a high-resolution, T1 weighted, SPGR image. The center of the
image appears brighter, because of the formation of standing waves in the RF pattern. As a result, the center
of the field of view receives a higher flip angle than the periphery of the image. The image on the right has
been corrected by removing the low frequency spatial oscillation with a 2D FIR filter

Besides the strength, one must consider the spatial homogene-

ity and temporal stability of the magnetic field. The magnetic field’s
homogeneity is crucial since the lack of it translates into severe
image distortions. One challenge is that the shape of the magnetic
field changes when an object (i.e., the subject) is introduced into
the field. Consequently, MRI systems are usually outfitted with a set
of small pieces of iron installed around the bore of the magnet,
referred to as “passive shims.” The location of the passive shims is
carefully chosen to make the field more homogeneous. In order to
adjust the field homogeneity for individual subjects, additional elec-
tromagnets whose field can be dynamically changed when the
patient are used. These are referred to as active shims and the pro-
cess of shaping the field is referred to as “shimming.” There are
many types of shim coils that are used to superimpose magnetic
fields for shimming purposes. The shim coils are designed to pro-
duce spatial magnetic field gradients. These fields are typically
shaped as linear, quadratic, and higher order functions of spatial
position. While typical clinical scanning procedures require adjust-
ments to the linear shims from patient to patient, it is our experi-
ence that T2* weighted (BOLD) fMRI benefits greatly from higher
order shimming. Modern scanners are equipped with automatic
shimming procedures [11] that can typically achieve homogeneities
over a 1500 cm3 region of less than 20 Hz RMS, approximately.
In addition to being homogeneous, it is quite important that
the magnetic field be as constant as possible over time. The field
tends to drift over time due to a number of factors, among them
temperature of the room and the equipment, as mentioned previ-
ously. These drifts are typically subtle and slow enough that they
do not affect clinical / structural imaging. FMRI, however, is
 Introduction to Functional MRI Hardware 43

based on subtle signal changes over time and therefore, drifts act as
significant confounds, especially in slow paradigms. Statistical and
signal processing tools do exist to reduce these drifts effects, but it
is much more desirable that they be reduced during acquisition.
Unfortunately, there are many sources of drift in the MRI hard-
ware, so it is important that the magnet undergo extensive stability
testing before it becomes operational and that quality control tests
including stability measurements be performed regularly. The
scanner’s stability can be measured on a phantom over a small
region of interest. Figure 11 illustrates a typical stability test.
The physical configuration and shape of the magnet also plays
an important role in many of these parameters. While “open” MRI
systems exist and are used for large or claustrophobic subjects,
their field strength is typically not sufficient for functional MRI
applications and their use is limited to clinical applications that do
not demand high-quality imaging. Among the closed bore sys-
tems, one can choose between short and long bore systems. Short
bore systems are intended for head-only applications and can
sometimes offer improved performance over smaller regions. Long

Fig. 11 A typical stability test showing the time course and its frequency content in a phantom (above)
44 Luis Hernandez-Garcia et al.

bore systems, although more cumbersome, achieve greater field

homogeneity over a larger area, which is beneficial for some appli-
cations, such as arterial spin labeling.
When considering magnets, it is crucial that we also consider
safety issues. The most obvious issue is the very powerful force that
magnetic fields of the magnitude required for MRI exert on fer-
romagnetic objects. These forces are inversely proportional to the
square of the distance between the object and the dipole, and are
directly proportional to the mass of the metal in question. recall
that the magnetic field produced by a current is described by the
Biot-Savart law:

m0 IdL ´ r
B (r ) = ò (7)
4p r
3

where I is the current, L is a unit vector in the direction of the cur-

rent, r is another unit vector pointing from the location of the wire
to the location of interest, and |r| is the distance from the location
of interest to the current source.
One must obviously be very careful to keep ferromagnetic
objects out of the magnet room. Typical accidents occur when
someone forgets about small metallic objects in their pocket and
they fly out of their pocket and strike someone. Accidents some-
times happen because there may be a very subtle force on the
object at a specific location in the room leading an unsuspecting
investigator to believe that the object is not ferromagnetic.
However, moving the object a very small amount toward the mag-
net can translates into a very rapid increase of the magnetic field,
since the magnetic field increases with the inverse of the square of
the distance, as mentioned. A small step in the wrong direction
while carrying a ferromagnetic object can be the difference between
a gentle tug on the object and the object being launched into the
bore of the magnet, to the horror of the investigator and the sub-
ject. It is thus paramount that strict screening procedures be fol-
lowed before allowing people into the magnet room. Sometimes
small bar magnets and airport security style metal detectors are
used to verify the absence of ferromagnetic objects on the subject’s
body or to test allegedly MR compatible equipment.
These forces can also affect metal implants in the subject’s bod-
ies. Pacemakers, neurostimulators, and other implanted electronic
devices are likely to malfunction putting the subject at great risk. It
is thus crucial that subjects be thoroughly screened for the presence
of implants, shrapnel, or other metal sources in their bodies. Having
said that, many modern implants are built of titanium and nonreac-
tive materials that are not ferromagnetic and are therefore “MR
compatible.” A number of publications exist cataloging medical
devices and their MR compatibility according to model and manu-
facturer [1, 2] (and on the web: http://www.mrisafety.com/).
 Introduction to Functional MRI Hardware 45

3.2 Magnetic Field In order to produce images, an MRI scanner needs a spatially varying
Gradients magnetic field (see image reconstruction chapter) under tight control
by the user. This is accomplished by using an additional set of coils
that add extra magnetic fields to the main field. A set of such coils is
shown in Fig. 12. By supplying customized current waveforms to
these coils, the user can change the distribution of the magnetic
field’s shape at will. In broad terms, by varying gradient’s strength
can be varied over time during the pulse sequence, one can obtain
MR signals whose phase distribution is a function of the spatial distri-
bution of the sample.
The ideal gradient set is capable of quickly changing the mag-
netic field as a linear function of spatial location along each of the
Cartesian axes. Typical gradients in clinical and functional MRI are
between 10 to 40 mT/m, but specialized gradient inserts exist that
can produce larger gradients (in the range of 100 mT/m). Small
bore animal systems can be outfitted with more powerful gradients
(up to approximately 400 mT/m). The main challenges in MRI
gradient design and construction usually consist of producing lin-
ear gradients in space and time, and the production of eddy cur-
rents. Motivated by the need to achieve finer spatial resolution and
better axon fiber tracking through diffusion tensor images (DTI) ,
Massachusetts General Hospital has developed a high-performance
gradient insert that can achieve up to 300 mT/m in a human sys-
tem, whereas standard clinical gradients rarely exceed 50 mT/m.
This system is utilized at present primarily for experiments con-
cerning the “Human Connectome Project” [12] (www.human-
connectomeproject.org).
The spatial linearity of the gradients must be maintained over
the volume of the sample to be imaged, or the images will appear
warped (although these distortions can be corrected during recon-
struction if the true shape of the gradient is known). The spatial

Fig. 12 Gradient coils from Doty Scientific (reproduced with permission of Doty Scientific)
46 Luis Hernandez-Garcia et al.

linearity of the gradient fields is primarily a function of the shape of

the gradient coils, and a great deal of effort goes into their design
and construction (we will not go into those details here). Typical
MRI scanner head gradients can maintain 95 % linearity over 30 cm.
The gradient coils must also be able to produce magnetic field
gradients very quickly and accurately. The inductive nature of the
coils causes their response to the input currents to be severely
dampened. In order to correct this problem, typical gradient cur-
rents are “pre-compensated” in order to produce the desired wave-
form [13, 14]. An example of precompensation is illustrated in
Fig. 13. Most maintenance or quality assurance protocols include
gradient linearity and pre-compensation.
The rate at which a gradient is achieved is referred to as the
slew rate. Slew rates of about 200 T/m/s can be generally achieved.
However, there are FDA limitations (these are determined by the
imaging sequence type and the duration of the stimulation.
Typically the maximum allowed rate of change in magnetic field is
approximately 20 T/s), since the sudden changes in the magnetic
field can induce currents in the peripheral nervous system, causing
muscle contractions and unpleasant or even painful sensations in
the patient. This phenomenon is referred to as peripheral nerve
stimulation (PNS) .

Input Current

Uncompensated
Compensated

Output Gradient

Uncompensated
Compensated

Fig. 13 Simplified illustration of gradient current compensation. The inductive

effects of the gradient coils are partially corrected by modifying the input cur-
rents to the coil
 Introduction to Functional MRI Hardware 47

Active shielding of gradients is necessary to contain the gradient

fields and reduce the interactions between gradient coils and other
conductors in the scanner. The principles are the same as the active
shielding of the main magnetic field (see Sect. 3.1). In other words, a
coil producing an opposite gradient field is built around the main gra-
dients in order to cancel the fields outside the area of interest [15, 16].
Gradient vibration and noise are another issue to consider in
gradient design. As the gradients are rapidly turned on and off,
especially in echo planar sequences, they experience torques due to
the presence of the main magnetic field. Thus the coils vibrate vio-
lently thus causing the familiar banging MRI sounds. The sound
levels are quite loud and require that the subject’s wear ear plugs.
Active shielding can help reduce the vibrations and acoustic noise
produced by the gradients [16].

3.3 RF Hardware MRI scanners employ RF hardware to generate oscillating mag-

netic fields that cause the magnetization vector to tip into the trans-
verse plane, and for signal reception. To create signal, an MRI
scanner uses a powerful amplifier (generally 1–25 kW) to drive an
excitation coil with a large pulse of electric current. In the reception
stage, the receive coil is used to pick up the MR signal, which is
then processed to create an image. The components of the RF chain
that an fMRI user should pay attention to are the transmit and
receive coils. Historically, the transmit and receive coils used for
fMRI were the same. Today, given the widespread use of ­parallel
imaging to improve image quality via reduced acquisition time,
fMRI experiments typically use the scanner’s body RF coil for trans-
mit, and a set of multiple coils contained in a single housing that sit
close to the head for receive. This approach is referred to as “paral-
lel imaging” [17, 18] and, in FMRI, it’s used to improve image
quality via reduced acquisition time. Head coils can take many
shapes and functional forms, however, for the purposes of SNR and
image homogeneity comparisons, there are two main classes of
coils: single-channel and multichannel or phased-array coils.
An RF coil is a resonant circuit, and can be modeled as a simple
loop containing an inductor, a capacitor, and a resistor (Fig. 14).
The inductor and capacitor represent actual circuit components
lumped together with the inductance and capacitance of the sam-
ple. When a source of electrical current that oscillates at the cir-
cuit’s resonant frequency is placed across its terminals, the
impedances of the inductor and capacitor cancel, and the coil
delivers the maximal amount of energy to the sample. In a recipro-
cal manner, current measured at the coil terminals due to energy
radiated by the sample will be of maximum amplitude when that
energy oscillates at the coil’s resonant frequency. Because the fre-
quency at which biological spins oscillate is determined by the
main magnetic field strength via the Larmor relationship, an RF
coil must be “tuned” to resonate at this frequency.
48 Luis Hernandez-Garcia et al.

Fig. 14 Series Resistor-Inductor-Capacitor (RLC) circuit representing an RF coil

and a biological sample

Single-channel surface coils are the simplest RF coil design.

These are typically used to image small areas close to the surface of
the skin, as their penetration depth drops sharply with distance
from the coil. They consist of a simple loop that induces an oscil-
lating magnetic field when a current is passed through it.
Although the phase information is routinely discarded in image
reconstruction, MR signals are inherently vector quantities. RF
coils are used to generate an oscillating magnetic field, and the sig-
nal emitted back by the sample is a rotating electromagnetic field.
However, a simple surface coil can only produce and detect only
one component of that vector: the component that is ­perpendicular
to the plane of the coil. Quadrature coils typically consist of two
perpendicular coils that transmit at 90° phase from each other. The
resulting magnetic field is the vector sum of the two perpendicular
fields. Quadrature reception can also be achieved with the same
coils by adding phase to one of the channels in the receive chain.
The most popular quadrature-channel head coils generally take
on a “birdcage” design (Fig. 15a). This classic design provides
good SNR and image homogeneity characteristics, owing to the
unique nature of the magnetic field that it creates [19]. Another
type of single-channel coil commonly used in studies of the occipi-
tal cortex is a quadrature occipital coil (Fig. 15b), which provides
high SNR in this localized region of the brain, and has a compact
design compared to full head coils, allowing the experimenter
greater freedom in stimulus hardware setup.
In contrast, multichannel or phased-array coils (Fig. 15c) are
composed of a set of discrete and independent “surface” coils,
arranged around the head. Generally, these coils can be brought
into a tighter conformation around the head, which improves
SNR. Taken alone, images obtained with individual surface coils
will possess lower SNR and poor image homogeneity compared to
a birdcage coil, however, when images from the coils are combined
in a sum-of-squares reconstruction, excellent SNR can be achieved
[20], though image homogeneity will still be worse than for a bird-
cage coil. Furthermore, most phased-array coils can be used only
 Introduction to Functional MRI Hardware 49

Fig. 15 (a) Transmit/receive birdcage head coil. (b)Receive-only quadrature occipital coil (c) 8-channel phased-­
array coil

for reception, so a separate coil (usually the scanner’s body coil)

must be employed for excitation, which can result in degraded
image homogeneity, as well as increased SAR in areas outside the
head. The central advantage to phased-array coils is that they allow
the use of parallel imaging techniques, such as SENSE [17] and
GRAPPA [21], when paired with RF signal chains capable of
receiving multiple channels simultaneously, which are now com-
monly available from most MR vendors. Parallel imaging exploits
the inhomogeneity of images obtained with surface coils to accel-
erate image acquisition, which results in a reduction of artifacts in
fMRI images, as in Fig. 16. This comes at the cost of reduced
SNR. Phased-array head coils used for fMRI commonly have 32
individual coil elements, and can be effectively used to reduce
image acquisition time by a factor of 2–4. Scanners with up to 128
receive channels are available from MR vendors.
A birdcage coil generally provides a lot of flexibility in stimulus
presentation setup, due to the large amount of room within the
coil. One can use goggles, projector/mirror systems, and a range
of other solutions in conjunction with a birdcage coil (as we discuss
50 Luis Hernandez-Garcia et al.

Fig. 16 Comparison of fMRI images obtained using conventional (a) and parallel imaging (b) in the inferior
brain. In this example, parallel imaging with an 8-channel phased-array coil was used to reduce data readout
time by a factor of 2. This reduced signal loss and image distortions due to susceptibility, particularly in the
region indicated by the arrows. (Courtesy of Yoon Chung Kim, University of Michigan Functional MRI Laboratory)

later). In comparison, current phased-array coil designs limit pre-

sentation options, since the coils forming it are placed closer to the
head, which may prevent the use of visors. However, phased-array
coils are generally compatible with the most commonly used
projector-­ based visual stimulus setups. When using visor-based
stimulus systems, one must ensure that the electronics of the visor
are properly shielded to prevent image artifacts of the type shown
in Fig. 4. The “streaking” artifact shown in this image was caused
by a visor with an electromagnetic “leak,” that coupled to the
receive coil and manifested as a false MR signal. The form of the
artifact will depend on the pulse sequence used; in this example the
streak is localized, while in another pulse sequence the artifact may
be spread over the entire image.
In the near future, the design of multichannel receive coils is
likely to be in part driven by simultaneous multislice imaging, a
recent parallel imaging-based scan acceleration technique in which
multiple slices in a volume are excited and read out simultaneously,
leading to a scan acceleration factor equal to the number of
simultaneously-­excited slices [22, 23]. Whereas conventional par-
allel imaging is applied to accelerate the acquisition of each slice’s
data individually and thus requires a circumferential density of
receive coils around the head in the plane of the slice, a simultane-
ous multislice acquisition requires a density of coils in the slice
dimension (typically head-foot in fMRI) to successfully separate
 Introduction to Functional MRI Hardware 51

the simultaneously-excited slices’ signals. Using appropriate coils

and sequences, acceleration factors of 2–5 are common in simulta-
neous multislice exams [24].
Finally, it is now a common option for 3 T MRI scanners to be
equipped with two transmit channels, which are used to indepen-
dently adjust the signals transmitted into the two ports of the scan-
ners’ integrated quadrature body coil in order to produce more
uniform RF fields, on a subject-to-subject basis. While this capability
is primarily driven by the need for more uniform RF fields in body
MRI, it may also provide some benefit in improving field uniformity
in brain MRI [25]. Furthermore, multichannel transmit methods
are currently being developed by several researchers to alleviate
through-plane signal loss artifacts in the lower brain [26, 27].

3.4 Computing Functional MRI experiments generate large datasets. The raw data
Resources alone for a single subject, scanned for one hour, weighs in between
100 and 200 MB. Combine this with the space needed for image
reconstructions and analysis, and one should budget a storage and
scratch space of around 2 GB for every hour of scanning. In this
section, we will provide some guidelines and suggestions on how
to set up a computing environment to handle all this data. The two
major factors influencing the design of your environment will be
(1) money, and (2) the expertise available, in terms of computer
systems and MRI data processing.
The data stream in a typical laboratory consists of four main
stages. The first stage is the MRI scanner, which produces either
raw, unreconstructed data, or reconstructed images that are ready
for post-processing and analysis. If a lab has an MR physicist at its
disposal, then the former case is often true, since an MR physicist
may develop image reconstruction codes that provide improved
image quality over vendor-provided software, and that can add in
improved reconstruction techniques as they are developed.
Assuming images have been reconstructed, the second stage con-
sists of slice timing correction, realignment, coregistration, warp-
ing, and smoothing, which are all operations that prepare the
dataset for statistical modeling and analysis. The third stage is sta-
tistical modeling and analysis. The fourth stage is data backup,
though it is advisable to make backups of data at more than one
point in the stream.
The majority of fMRI labs maintain one powerful workstation
or computer cluster to do the bulk of their processing. Users can
log into this computer from their own machines to initiate process-
ing or view and download the preprocessed data for local analysis.
The central advantage to this model is that the large and compli-
cated software packages used to process fMRI need only be main-
tained on one machine, which greatly simplifies maintenance. A
secondary advantage is that this model allows users greater flexibil-
ity in choosing the operating system of the computers they use; the
52 Luis Hernandez-Garcia et al.

workstation can be running a Unix derivative, which has benefits in

terms of networking, stability, and the availability of fMRI software
packages, while the users can be using Windows PC’s or
Macintoshes, which are generally more user-friendly systems. All
computers that will be involved in processing or storing data
should be connected with the fastest network possible, such as a
gigabit network.
The central processing workstation should have as much RAM
as possible. By today’s computing standards, each CPU should
have at least 2 GB RAM available to it, so that it may store the
entire dataset and related files. The reason for this is to minimize
the frequency with which the computer accesses its hard disk pro-
cessing, which costs large amounts of time. The second main con-
sideration is the number of CPU’s the computer should have. This
will largely depend on the number of simultaneous processing
streams one expects to have running on the computer. The third
consideration is storage. The main computer should be connected
to a large data storage device, so that all current experiments can
be instantly accessed, without requiring reloading of backed-up
data. This device should be a set of hard disks configured in a
redundant manner, such as a RAID array.
One of the central computing dilemmas that an fMRI lab will
continually deal with is making data backups. There are two main
questions to answer here: (1) at what points in the processing
stream should backups be made, and (2) what form should back-
ups take? Backups of the initial MR data are absolutely necessary,
since an experimenter may be asked to reproduce their results at a
later date, or bugs may be found in the post-processing stream
(stage two), which will require re-processing the original data.
After this stage in the stream, the choice of where to do backups
will depend on the amount of backup space available and the speed
with which the second and third stages may be executed, should
the analyzed data be lost. The more points at which backups are
made, the more quickly an experimenter could recover after a data
loss or processing interruption. A laboratory also has many options
in choosing backup forms, and it may be best to use a combination
of them. Perhaps the simplest backup form is mirroring the data,
i.e., storing the data in another set of hard disks whose sole pur-
pose is to store backed-up and archived data. This solution is par-
ticularly simple in that backups can be fully automated, and
instantly accessed. The other two main options are tape storage
and optical media (DVD). While automated machines may be pur-
chased to manage tape backups, hard disk capacity is rapidly out-
stripping tape capacity, and the mechanical nature of these machines
makes them prone to frequent failure. On the other hand, DVD
backups are more reliable and cheap, but they require human
interaction to load DVD’s and execute burning software. A backup
schedule will also have to be worked out by the laboratory.
 Introduction to Functional MRI Hardware 53

Because computer components (i.e., disks, CPU’s, and video

cards) frequently fail or need replacement, it is advisable to set
aside part of the lab’s initial capital for yearly computer mainte-
nance. It is also advisable to purchase extended warranties for the
computers, as they will be heavily used and if they fail, this can save
the lab a significant amount of money in the long run. Furthermore,
as technology advances, the lab should build into their operational
budget the cost for new machines every few years.

4 Stimulus Presentation and Behavioral Data Collection Devices

There is a small but increasing number of manufacturers of stimu-

lus presentation hardware and software. We will not review specific
products or vendors but limit ourselves to describe the important
characteristics of these devices.
Some components of the stimulation and response recording
equipment must go inside and/or near the magnet. These must not
be susceptible to magnetic forces for obvious safety reasons.
Additionally, the presence of metals in headphones and head
mounted displays can generate field distortions that cannot be com-
pensated by shimming, even if they are not ferromagnetic. This
results in severe image degradation and it is thus important that the
devices be thoroughly tested on phantoms for image degradation.
As we alluded to before, electronic equipment in the scanner
room must be adequately shielded to avoid introducing RF noise
into the system. For example, LCD displays for visual stimulation
inside the magnet are typically encased in a fine wire mesh that
acts as a Faraday cage to contain RF leakage. Other audiovisual
electronic equipment used in the MR environment, such as pro-
jectors and button response units are typically encased in brass or
aluminum for the same reasons. Regardless of the manufacturers’
best intentions, sometimes the shielding is not adequate or
becomes damaged over time in subtle ways. Just as in the case of
the room’s RF shielding, an exposed wire or bad shielding con-
nections can produce severe RF contamination of the images.
Thus, it is paramount that the stimulation devices be checked
upon purchase and periodically for RF leaks that may develop
during delivery, installation, or daily use.
There are different technologies commercially available for
MR compatible visual stimulation. The simplest approach is per-
haps an LCD projector outfitted with narrow focus lenses that
project the images into a back projection screen placed inside the
bore of the magnet. The subject then can see the display through
a set of mirrors that are mounted on the head coil assembly. The
main advantages of this approach are simplicity and lower cost.
The disadvantages are related to positioning issues and a reduced
visual field for the subject.
54 Luis Hernandez-Garcia et al.

Another approach to MR compatible visual stimulation is fiber

optic display visors. This sort of display system is based on a optical
signal converter that carries an SVGA quality image through an
array of micro optic fibers to a head mounted display inside the
scanner’s head coil. This approach is very attractive in that there
are no electronic components that need to be installed inside the
scanner room and the display can be placed very accurately in front
of the subject’s eyes, maximizing the available visual field. The
drawbacks are that in additional to being expensive, the fiber optics
used in the array are very fine and brittle, so that regardless of the
high quality of fabrication, there will always be a small number of
broken fibers that result in dead pixels or small streaks in the image.
One of the more popular approaches is to display the images
on a shielded LCD screen mounted in front of the subject’s head.
This screen can be either a large one that is mounted outside the
scanner’s RF coil, or a small one in a visor that the subject wears
inside the coil. The advantages of this are the large visual field and
ease of use of the system. The drawbacks are the high cost and the
interactions between the display electronics and the magnet.
Some of these devices become dimmer when placed inside the
magnetic field. Additionally, if any RF leaks develop, they severely
degrade the images, especially in the visor type systems since they
sit inside of the RF coil.
Auditory stimulation is typically performed in the MR envi-
ronment through two different kinds of headphones: pressure
waveguide types, and shielded piezoelectrics. Both of these are
highly effective devices. The pressure waveguide headphones keep
the speakers outside of the magnet’s bore and the sound is carried
through rigid tubing into the headphones. The piezoelectric head-
phones are akin to standard speaker technology but use piezoelec-
trics to produce the vibrations. They require RF shielding of the
cables and the electronics to prevent artifacts. Perhaps the biggest
challenge for auditory stimulation is reduction of the MRI scan-
ner’s noise. There is very limited space inside the scanner’s head
coil for building an effective muffler into the headphones but fairly
effective noise reduction (typically around 30 dB) can be achieved.
The headphones’ acoustic insulation is sometimes achieved by gel
padding that attenuates the sound very effectively by forming a
tight seal around the ear. Caution must be used as the gel in the
padding produces an MR signal and is visible in the images so it
must be taken into consideration during registration and normal-
ization of structural images. The gel’s resonant frequency is typi-
cally not the same as water and produces some off-resonance
artifacts, but these tend to be mild.
MR compatible microphones for patient communication and
verbal response recording are typically based on piezoelectric tech-
nology and require both electronic and acoustic shielding to reduce
the scanner sound. To our knowledge of the present state of the art,
 Introduction to Functional MRI Hardware 55

the acoustic shielding of the microphone from the scanner sound is

somewhat effective, but communication with the subject during
the scan is still challenging. Some systems are equipped with active
noise cancellation with limited success. Consequently, investigators
often use pulse sequences with “quiet” (i.e., no gradient pulses)
periods during the subject response time instead [28, 29].
Other response recording devices are primarily “button response
units (BRU)” that are built into hand rests and strapped to the sub-
ject’s hands. They typically carry only DC currents through twisted
pair cables and RF noise is not an issue as the electronics to drive the
system are kept outside the scanner room. There are a number of
other response units, such as MR c­ ompatible joysticks and keyboards
that are manufactured by small companies. While these are typically
safe and effective, one should test all such equipment immediately
upon purchase not only for functionality but for RF leakage, and
ferromagnetic forces. Periodic RF testing of peripherals should be an
integral of the fMRI facility’s QA procedures.

5 Subject Monitoring

When running an fMRI experiment, it can be desirable to monitor

and record subjects’ status and peripheral signals during an fMRI
experiment, to use as correlates of subject behavior or as nuisance
signals in data correction. Some possibilities include monitoring
cardiorespiratory rhythms, galvanic skin response, head motion, or
eye-tracking. As stressed in the previous sections, all these consid-
erations should fit with the comfort and safety of the subject.
In general, when considering recording peripheral signals on
fMRI subjects, one should pay attention to: synchronization with
the MR scanner, adequate sampling of the signal in question, and
avoiding introducing signal noise in both the MR data and the
recorded peripheral signals.

5.1 Scanner In order to match the recorded external signals with the fMRI data
Synchronization being recorded, synchronization with the start of the scan must be
achieved. This can be done using a TTL pulse to/from the scanner
from/to the external device or recording media. For instance, a
logic pulse from the MR scanner to the computer recording physi-
ological noise can be set to trigger the recording sequence.
Commercial MR scanners from the main vendors (GE, Siemens,
Philips) all have the capability to send or receive TTL sync pulses.

5.2 Physiological A limit to the effectiveness of functional MRI in detecting activa-

Monitoring tion is the presence of physiological noise, which can equal or
exceed the desired signal changes in an fMRI experiment [30].
These physiological fluctuations that are present during an fMRI
scan can obscure the BOLD activity that the researcher is trying to
56 Luis Hernandez-Garcia et al.

detect. In addition, monitoring physiological rates can help as sec-

ondary reaction measures (such as monitoring the cardiac rate vari-
ability during a stress experiment).

5.3 Cardiac Monitoring the cardiac waveform can be achieved in several ways.
Monitoring The most common solutions are pulse oximeters or ECG patches.
The primary cardiac harmonic frequency lies in the 0.5–2.0 Hz
range, with both the first and secondary harmonics shown to affect
the fMRI signal [31].
Pulse oximetry refers to indirectly monitoring the oxygen lev-
els in the extremities to monitor the cardiac waveform. This is most
often accomplished in fMRI labs by using an LED and photodiode
that clips to the subject’s finger, connected to a data acquisition
board (see Fig. 17). Several MR scanner vendors offer this as part
of the MR system (GE, Siemens), and stand-alone monitoring
units from commercial vendors are also available (Invivo, Biopac).
Normal setup with compliant subjects allows adequate sampling of
cardiac rhythm, as seen in Fig. 18. Drawbacks include the fact that

Fig. 17 Pulse oximeter for indirect measure of cardiac waveform

2000

1500

1000

500

-500
0 5 10 15 20 25 30

Fig. 18 Cardiac waveform acquired during an fMRI scan. (Data acquired on a 3.0 T GE scanner, using a pulse
oximeter with a sampling rate of 40 Hz)
 Introduction to Functional MRI Hardware 57

QRS
Complex

ST
PR Segment
T
P Segment

PR Interval
Q
S
QT Interval

Fig. 19 Example of ECG patch (Courtesy of Invivo, www.invivocorp.com). With a schematic of a typical QRS
waveform

subject motion may corrupt the signal, and motor tasks may be
impeded with the oximeter placed on the finger (alternative place-
ment on the ear or toe is possible).
ECG patches located over the heart allow high-fidelity monitor-
ing of cardiac electrical activity. This allows identification of features
beyond the simple cardiac peaks, such the QRS complex during the
depolarization of the ventricles (Fig. 19). Disadvantages of ECG
recording include increased setup complexity, and patient comfort.

5.4 Respiratory The respiratory rhythm has a normal frequency range of 0.1–0.5 Hz.
Monitoring Motion of the chest during respiration combined with the changes
in oxygen saturation in the lungs lead to a modulation of the local
magnetic field that can affect the phase of the MR signal at the posi-
tion of the head during scanning (Fig. 20), which can lead to modu-
lation of the recorded MR signal intensity. Mitigation of respiratory
effects on the MR signal can include scanning during breath-holds,
modified pulse sequences to sample and account for the modulation
in magnetic field [32] and recording of the respiratory signal to use
as nuisance covariates in post-processing analysis [33].
Monitoring respiratory rhythm can be accomplished by using
a plethysmograph (pressure belt) around the waist of a subject, like
the one shown in Fig. 21, or by using a nasal cannula to monitor
expired CO2 concentration. A sample respiratory waveform is
shown in Fig. 22.

5.5 Galvanic Skin Galvanic skin response (GSR) is a measure of the electrical resistance
Response (GSR) of the skin, a physical property that has been shown to increase in
response to subject arousal, mental effort, or stress. It is monitored
58 Luis Hernandez-Garcia et al.

Fig. 20 Phase difference between inspiration and expiration for a coronal slice

Fig. 21 Plethysmograph belt for measuring respiration

3500

3000

2500

2000

1500
0 10 20 30 40 50 60

Fig. 22 Respiratory waveform acquired during an fMRI scan. (Data acquired on a 3.0 T GE scanner, using a
pulse oximeter with a sampling rate of 40 Hz)
 Introduction to Functional MRI Hardware 59

by measuring a voltage drop across the skin using paired electrodes.

This can have the same motion sensitivity and motor task complica-
tion as the pulse oximeter if the electrodes are placed on the fingers.
These problems are usually reduced by placing the electrodes
between the second and third knuckles, instead of on the fingertip.

5.6 Head Motion Severe head motion during an fMRI scan can severely corrupt the
Tracking data. Besides minimizing patient motion using cushioning and
restraints, a measure of head motion may also be collected to help
correct data in post-processing. This may be done using modified
pulse sequences (PACE), or by using external motion tracking
[33–35]. Again, patient comfort and visual path should be taken
into consideration.

5.7 Eye Tracking Patient gaze and fixation time is important for several types of
fMRI paradigms and pathological types. In addition, eye motion
can be a source of variance in fMRI scans [36]. Thus, tracking eye
position can be desirable. The common method is to monitor the
position of infrared (IR) light that is reflected off the eye of the
subject. This involves transmitting IR light to the subject’s eye,
and then recording it, using MR-compatible equipment. Several
vendors provide hardware solutions including both long-range and
short-range cameras. These systems typically have on the order of
0.1–1° spatial resolution and accuracy, with working ranges of
10–25° horizontally and vertically, and 60–120 Hz sampling rate.
In acquiring a physical eye-tracking system for an fMRI lab,
consideration should be given to the optical path for the eye-­
tracking system, taking into account the MR bore, head coil, and
visual stimulus presentation system; signal integrity of the MR
data; ease of setup for the MR techs and scanners. This will also
involve peripheral equipment located in the scanner room or con-
trol room: usually a camera, power supply, video monitor for real-­
time display of the subject’s eye, and a PC.

6 Multimodal fMRI

Acquiring other neurophysiological measures with fMRI data can

complement the excellent spatial resolution and depth penetration
of fMRI with modalities that have superior temporal resolution
and different sensitivities to the underlying neuronal activity. In
the following sections, we will expand upon complementary
modalities that allow investigations of response (e.g., EEG, fNIRS)
and stimulation (e.g., TDCS, TMS).

6.1 FMRI-EEG The simultaneous acquisiton of electroencephalography (EEG)

data with fMRI data allows the higher temporal sampling rate of
EEG (~5000 Hz) to be combined with the superior resolution (~
mm) and depth penetration of fMRI.
60 Luis Hernandez-Garcia et al.

Several factors must be accounted for when setting up a

simultaneous EEG/fMRI acquisition, chief among them safety
and ­signal quality. In the following, we will try to touch on most of
the considerations one will make when selecting and setting up
EEG-­fMRI hardware.
Several companies have MR-compatible EEG hardware com-
mercially available (Brain Products, Neuroscan, EGI). A common
EEG setup includes an electrode cap, connected to a signal ampli-
fier and recording device. Any part of this setup that is inside the
MR scanner room within the 5 G line must be nonferrous, and the
amount of metal must be kept to a minimum, with care exercised
around all metallic components (this includes electrode leads on
the cap and skin, batteries in amplifiers, etc.). Fiber optics can be
used for signal transmission after amplification, with recording
devices located in the scanner control room.
Much planning is required to integrate the EEG with the
MR. The head coil used for MR acquisition may affect the physical
setup. For instance, an open birdcage coil may allow placement of the
EEG cap cord and amplifier above the head of the subject, with no
obstruction of the subject’s field of view. With an alternative phased-
array coil that is closed at one end, this setup may not be possible.
Also, for time synchronization, it may be necessary to use the
TTL pulse from the scanner (mentioned in Sect. 5.1) to trigger the
EEG recording device at each TR.
Finally, for fusion of the fMRI and EEG data, accurate electrode
locations on the head should be recorded and transferred to the struc-
tural MR images to be used as references for localization. Commercial
head position recording systems are available (Brainsight). These sys-
tems can record points on the subject’s head using infrared positional
markers, and coregister these coordinates with the structural MR
scans of the subject. Subsequent coregistration of the fMRI and struc-
tural MR data allows direct overlay and source localization using both
between the MR, fMRI, and EEG data.
The EEG equipment should not adversely affect the MR data,
if proper materials and shielding are used. Images with and with-
out the EEG equipment should be inspected for any introduction
of AC line noise (~60 Hz) or localized variance in the structural
and fMRI images.
The MR equipment will affect the EEG recording, due to the
high-field environment, and the application of gradients during
the MR acquisition (Fig. 23). However, the MR gradient artifact
can be corrected for in post-processing, using either available soft-
ware (EEGLAB, http://www.sccn.ucsd.edu/eeglab/) or adapt-
ing techniques such as PCA or ICA.

6.2 Functional Functional MRI allows investigation of the hemodynamic response

Near-Infrared to neural activity, thus allowing localization of brain regions involved
Spectroscopy (fNIRS) in a cognitive task. However, fMRI is not a naturalistic setting.
 Introduction to Functional MRI Hardware 61

Fig. 23 EEG recorded during fMRI acquisition, before (top) and after (bottom) MR artifact correction

Functional near-infrared spectroscopy (fNIRS) offers a possible

solution for portable neuroimaging in the field. It offers subsecond
temporal sampling, with spatial resolution on the order of centime-
ters. For multimodal work with fMRI, it is attractive as it samples
the same hemodynamic changes as BOLD fMRI imaging, namely
the change in relative levels of oxygenated and deoxygenated blood,
using the differential absorption of infrared laser light. An image of
a commercially available FNIRS system can be seen in Fig. 24.
Generally, fNIRS operates by using infrared light source and
detector pairs. While the light used is in the infrared range, and thus
the majority of the setup may be thought of as fiber-optic and
largely MR compatible, the internal construction of the probes may
contain non-MR compatible materials (such as metal coatings in
reflectors). Careful consideration must thus be used for a fNIRS-­
fMRI combined experiment. Several vendors offer MR-compatible
probes (EGI, Imagent), or custom setups can be constructed in-­
house [37]. In terms of induced noise, attention must be paid to
induced MR artifacts due to probe construction. However, due to
62 Luis Hernandez-Garcia et al.

Fig. 24 Example MRI compatible fNIRS setup (Cortech Solutions, www.cortech-

solutions.com/Products/NI/NI-OM), showing light guides, example subject setup
configuration, and receiving equipment

the optical nature of the fNIRS signal, it is not affected by the MR

environment, offering one advantage over EEG-fMRI acquisition.

6.3 Brain Stimulation A fundamental strategy used in cognitive neuroscience is to stimu-

and fMRI late the brain in some way and then observe how it responds.
Functional MRI provides such observations of the brain’s responses.
While most studies use sensory (audio, visual, tactile … etc.) cues
and cognitive tasks as a form of stimulation, one can also stimulate
human brains directly and noninvasively inside an MR scanner.
One such technique is transcranial magnetic stimulation (TMS),
which has great potential not only as research tool but also as a ther-
apeutic device [38–40]. The principle behind TMS is that a large
current waveform is driven through a coil placed adjacent to the
tissue of interest. The current in turn induces an electromagnetic
field depending on the rate of change of the current, as predicted by
classical electrodynamics. The induced electric field penetrates the
tissue and induces eddy currents on conductors, such as nerve fibers.
When a nerve fiber is aligned with the direction of the electric field,
a large current is induced in the axon which causes its membrane to
depolarize, effectively causing the transmission of an action potential
[41–43]. After depolarization of the axonal membrane, the sodium–
potassium pumps rebuild the membrane potential and the nerve
fibers return to their original state within seconds.
The effects of these induced discharges are complex, depend-
ing upon the magnitude and timing of the TMS pulse affecting
inhibitory and/or excitatory neuronal populations (for recent
reviews, see [44–46]). If a single TMS pulse is applied in the hand
 Introduction to Functional MRI Hardware 63

region of the motor cortex, for example, motor neurons depolarize

and the hand will twitch ('motor evoked potential', or MEP).
Subthreshold stimulation, followed by supra-threshold stimula-
tion, can inhibit or facilitate the MEP, which varies with the dis-
tance and location of the subthreshold stimulus [47].
By selectively applying a TMS pulse during the performance of
a task, neural circuits are effectively jammed, and performance
interruptions can be observed. In effect, one creates a controlled,
completely reversible “lesion,” enabling the study of brain function
through perturbation of neuronal activity [44, 48]. Although these
studies have rapidly become a popular investigative tool for cogni-
tive neuroscientists, one important limitation stems from inade-
quate knowledge about the shape and magnitude of the induced
current fields that introduce the perturbation. Hence, TMS and
fMRI can be complementary for the study of brain function. TMS
can interfere, or modulate the cognitive process under scrutiny by
locally altering the responsiveness of the tissue, while fMRI can
allow the investigators to precisely map out these effects. While
TMS and fMRI experimental data can be coregistered and inte-
grated after each experiment has been carried out separately [49], it
is desirable to be able to observe the BOLD responses to TMS.
However, there are some clear challenges to carrying out joint
TMS and fMRI experiments. Such experiments require TMS coils
that contain no ferromagnetic parts and extra long cabling so that
the amplifier/capacitor bank can be kept outside of the 5 G line.
Specialized holders must be constructed to hold the TMS coil in
the appropriate position during the duration of the scanning ses-
sion (Fig. 25). Like all electronic equipment, the TMS hardware

Fig. 25 MRI compatible TMS coil and holder apparatus (image courtesy of Dr.
A. Thielscher of the Max Plank Institute for Biological Cybernetics http://www.
kyb.mpg.de/)
64 Luis Hernandez-Garcia et al.

must be shielded in a faraday cage to prevent RF contamination of

the MRI signal [50]. Commercial TMS coils currently have these
characteristics as optional features.
Another important aspect is the synchronization of TMS pulses
and image acquisition. It is very important that the scanner’s RF
receiver chain be switched off while the stimulator is pulsing in
order to protect it from the large signals that may severely damage
it. At the same time the scanner’s pulses may induce currents in the
TMS hardware (although these are reportedly not harmful to the
hardware). Isolation between the two is achieved by synchronizing
the TMS pulses with the MRI scanner via TTL pulses. The imag-
ing pulse sequence is typically designed with long gaps for the
TMS pulses. These gaps include about 0.1 s to allow TMS induced
eddy currents inside the bore to decay. There is also the challenge
of the large torques that a large, sudden dipole exerts when in the
presence of a large magnetic field. However, in the case of figure-­
eight coils, those torques cancel since the two “wings” of the coil
are torqued in opposite directions. Furthermore, rapid bi-phasic
pulses also cancel those torques and the subject does not perceive
any such effects. The coil, however, experiences internal stresses
from the magnetic forces [50–52].
Recently, transcranial direct current stimulation (TDCS) has
been adopted by neuroscientists as a method to manipulate the
excitability of neurons [53–56]. The technique is very simple and
can be easily combined with functional MRI with minor safety
concerns. In a nutshell, TDCS works by running a small direct cur-
rent (1–2 mA) from an anode to a cathode placed across brain.
While the mechanism is not entirely understood, the neurons
beneath the anode experience enhanced excitability because of a
shift in the transmembrane potential, and the opposite is true at
the cathode [57]. At the time of writing, TDCS and its mechanism
are a very exciting and active area of research. Indeed it is often
combined with BOLD and ASL FMRI to study its effects [58, 59].
The main issues that must be addressed when conducting
TDCS-FMRI experiments are the heating of the electrodes and
wires by currents induced by the scanner’s oscillating magnetic
fields (RF and gradients). A solution to this problem is to increase
the impedance of the electrodes and wires by placing resistors
(>5 kOhm) in line. One must also be careful of the presence of the
ancillary equipment, such as the power supply to the TDCS device,
in the magnet room. In general it is good to keep that part outside
the scanner room and feed the cabling through a penetration
panel. In that case, the wires are likely to act as an antenna and
bring in RF noise from outside the magnet room, which is detri-
mental for image quality. A solution to this problem is to also place
a low-pass filter in line (between the power supply and the elec-
trodes) to eliminate the RF noise in the system. One example of
this approach can be seen in [58], where the investigators placed a
 Introduction to Functional MRI Hardware 65

5.6 kOhm resistor on MR compatible rubber electrodes, and filters

in line with the power supply that were rated to attenuate 60 dB
between 20 and 200 MHz.

7 Conclusions

The hardware used in magnetic resonance imaging is quite extensive

and we hope to have provided an adequate overview of the subsys-
tems involved in the generation of MRI images. Functional MRI
requires additional hardware for collection of behavioral data and
stimulation of the subject while collecting the functional images.
The greatest challenge is perhaps to coordinate all these devices
while being mindful of the interactions between the devices and the
MRI scanner. Failure to do so often results in severe artifacts in the
desired measurements, or worse, the subject could be severely
injured. In this chapter we have also explored the hardware require-
ments for multimodal imaging, such as EEG-fMRI or TMS-fMRI.

References
1. Shellock FG (2002) Reference manual for mag- 10. Tropp J (2004) Image brightening in samples
netic resonance safety, implants, and devices. of high dielectric constant. J Magn Reson
Saunders, Oxford, UK 167:12–24
2. Shellock FG, Crues JV 3rd (2002) MR safety 11. Schneider E, Glover G (1991) Rapid
and the American College of Radiology white in vivo proton shimming. Magn Reson Med
paper. AJR Am J Roentgenol 178:1349–1352 18:335–347
3. Train JJ (2003) Magnetic resonance compat- 12. Dylan Tisdall M, Witzel T, Tountcheva V,
ible equipment. Anaesthesia 58:387, Author McNab JA, Adad JC, Kimmlingen R, Hoecht
reply 387 P, Eberlein E, Heberlein K, Schmitt F, Thein
4. Durand E, van de Moortele PF, Pachot-­Clouard H, Wedeen Van J, Rosen BR, Wald LL (2012)
M, Le Bihan D (2001) Artifact due to B0 fluc- Improving SNR in high b-value diffusion
tuations in fMRI: correction using the k-space imaging using Gmax = 300 mT/m human gra-
central line. Magn Reson Med 46:198–201 dients, Proc ISMRM 2012
5. Tinkham M (2004) Introduction to supercon- 13. Gach HM, Lowe IJ, Madio DP et al (1998)
ductivity, 2nd edn, Dover Books on Physics. A programmable pre-emphasis system. Magn
Dover Publications, Mineola, NY Reson Med 40:427–431
6. Radebaugh R (2009) Cryocoolers: the state 14. Wysong RE, Madio DP, Lowe IJ (1994) A
of the art and recent developments. J Phys novel eddy current compensation scheme for
Condens Matter 21:164219 pulsed gradient systems. Magn Reson Med
7. Williams DS, Detre JA, Leigh JS, Koretsky AP 31:572–575
(1992) Magnetic resonance imaging of perfu- 15. Mansfield P, Chapman B (1986) Active mag-
sion using spin inversion of arterial water. Proc netic screening of coils for static and time-­
Natl Acad Sci U S A 89:212–216 dependent magnetic field generation in NMR
8. Yang QX, Wang J, Zhang X et al (2002) Analysis imaging. J Phys E Sci Instrum 19:540–545
of wave behavior in lossy dielectric samples at 16. Edelstein WA, Kidane TK, Taracila V et al
high field. Magn Reson Med 47:982–989 (2005) Active-passive gradient shielding for
9. Collins CM, Liu W, Schreiber W, Yang QX, MRI acoustic noise reduction. Magn Reson
Smith MB (2005) Central brightening due to Med 53:1013–1017
constructive interference with, without, and 17. Pruessmann KP et al (1999) SENSE: sensitivity
despite dielectric resonance. J Magn Reson encoding for fast MRI. Magn Reson Med
Imaging 21:192–196 42(5):952–962
66 Luis Hernandez-Garcia et al.

18. Blaimer M, Breuer F, Mueller M et al (2004) of physiological fluctuation in functional

SMASH, SENSE, PILS, GRAPPA: how to MRI. Magn Reson Med 34:201–212
choose the optimal method. Top Magn Reson 32. Pfeuffer J, Van de Moortele PF, Ugurbil K,
Imaging 15:223–236 Hu X, Glover GH (2002) Correction of physi-
19. Hoult DI, Chen CN, Sank VJ (1984) ologically induced global off-resonance effects
Quadrature detection in the laboratory frame. in dynamic echo-planar and spiral functional
Magn Reson Med 1:339–353 imaging. Magn Reson Med 47:344–353
20. Roemer PB, Edelstein WA, Hayes CE, Souza 33. Tremblay M, Tam F, Graham SJ (2005)
SP, Mueller OM (1990) The NMR phased Retrospective coregistration of functional mag-
array. Magn Reson Med 16:192–225 netic resonance imaging data using external
21. Griswold MA et al (2002) Generalized auto- monitoring. Magn Reson Med 53:141–149
calibrating partially parallel acquisitions 34. Zaitsev M, Dold C, Sakas G, Hennig J, Speck
(GRAPPA). Magn Reson Med 47:1202–1210 O (2006) Magnetic resonance imaging of freely
22. Larkman D, Hajnal J, Herlihy A, Coutts G, moving objects: prospective real-time motion
Young I, Ehnholm G (2001) Use of multicoil correction using an external optical motion
arrays for separation of signal from multiple tracking system. Neuroimage 31:1038–1050
slices simultaneously excited. J Magn Reson 35. Thesen S, Heid O, Mueller E, Schad LR (2000)
Imaging 13(2):313–317 Prospective acquisition correction for head
23. Setsompop K, Gagoski BA, Polimeni JR, motion with image-based tracking for real-time
Witzel T, Wedeen VJ, Wald LL (2012) fMRI. Magn Reson Med 44:457–465
Blipped-­controlled aliasing in parallel imaging 36. Chen W, Zhu XH (1997) Suppression of phys-
for simultaneous multislice echo planar imag- iological eye movement artifacts in functional
ing with reduced g-factor penalty. Magn Reson MRI using slab presaturation. Magn Reson
Med 67:1210–1224 Med 38:546–550
24. Feinberg D, Moeller S, Smith S, Auerbach E, 37. Harrivel AR et al (2009) Toward improved
Ramanna S, Glasser M, Miller K, Ugurbil K, headgear for monitoring with functional near
Yacoub E (2010) Multiplexed echo planar imag- infrared spectroscopy. NeuroImage 47:S141
ing for sub-second whole brain fmri and fast dif- 38. Barker AT (1991) An introduction to the basic
fusion imaging. PLoS One 5(12), e15710 principles of magnetic nerve stimulation. J Clin
25. Zhang Z, Yip CY, Grissom W, Noll DC, Neurophysiol 8:26–37
Boada FE, Stenger VA (2007) Reduction of 39. Barker AT (1999) The history and basic principles
transmitter B1 inhomogeneity with transmit of magnetic nerve stimulation. Electroencephalogr
SENSE slice-select pulses. Magn Reson Med Clin Neurophysiol Suppl 51:3–21
57(5):842–847 40. Jalinous R (1991) Technical and practical
26. Stenger VA, Boada FE, Noll DC (2000) aspects of magnetic nerve stimulation. J Clin
Three-­dimensional tailored RF pulses for the Neurophysiol 8:10–25
reduction of susceptibility artifacts in T2*- 41. Ruohonen J, Ravazzani P, Tognola G, Grandori
weighted functional MRI. Magn Reson Med F (1997) Modeling peripheral nerve stimula-
44(4):525–531 tion using magnetic fields. J Peripher Nerv Syst
27. Yip CY, Fessler JA, Noll DC (2006) Advanced 2:17–29
three-dimensional tailored RF pulse for signal 42. Ilmoniemi RJ et al (1997) Neuronal responses
recovery in T2*-weighted functional mag- to magnetic stimulation reveal cortical reactivity
netic resonance imaging. Magn Reson Med and connectivity. Neuroreport 8:3537–3540
56(5):1050–1059
43. Berne RM, Levy MN (1993) Physiology,
28. Jakob PM et al (1998) Functional burst imag- Mosby year book. Mosby, St. Louis
ing. Magn Reson Med 40:614–621
44. George MS et al (2003) Transcranial mag-
29. Edmister WB, Talavage TM, Ledden PJ, netic stimulation. Neurosurg Clin N Am
Weisskoff RM (1999) Improved auditory cor- 14:283–301
tex imaging using clustered volume acquisi-
tions. Hum Brain Mapp 7:89–97 45. Paus T (2005) Inferring causality in brain
images: a perturbation approach. Philos Trans
30. Noll DC, Schneider W (1994) Theory, simu- R Soc Lond B Biol Sci 360:1109–1114
lation, and compensation of physiological
motion artifacts in functional MRI. Image pro- 46. Pascual-Leone A, Walsh V, Rothwell J (2000)
cessing, 1994. Proceedings ICIP-94. IEEE Int Transcranial magnetic stimulation in cognitive
Conf 3:40–44 neuroscience–virtual lesion, chronometry, and
functional connectivity. Curr Opin Neurobiol
31. Hu X, Le TH, Parrish T, Erhard P (1995) 10:232–237
Retrospective estimation and correction
 Introduction to Functional MRI Hardware 67

47. Rothwell JC (1999) Paired-pulse investiga- SP, Silva MT, Paulus W, Pascual-Leone A
tions of short-latency intracortical facilitation (2005) Anodal transcranial direct current stim-
using TMS in humans. Electroencephalogr ulation of prefrontal cortex enhances working
Clin Neurophysiol Suppl 51:113–119 memory. Exp Brain Res 166(1):23–30
48. Ilmoniemi RJ, Ruohonen J, Karhu J (1999) 55. Dieckhöfer A, Waberski TD, Nitsche
Transcranial magnetic stimulation–a new tool M, Paulus W, Buchner H, Gobbelé R
for functional imaging of the brain. Crit Rev (2006) Transcranial direct current stimu-
Biomed Eng 27:241–284 lation applied over the somatosensory
49. Bastings EP et al (1998) Co-registration of corti- cortex – differential effect on low and
cal magnetic stimulation and functional magnetic high frequency SEPs. Clin Neurophysiol
resonance imaging. Neuroreport 9:1941–1946 117(10):2221–2227
50. Bohning DE et al (1998) Echoplanar BOLD 56. Wagner T, Valero-Cabre A, Pascual-Leone A
fMRI of brain activation induced by concur- (2007) Noninvasive human brain stimulation.
rent transcranial magnetic stimulation. Invest Annu Rev Biomed Eng 9:527–565
Radiol 33:336–340 57. Radman T, Ramos RL, Brumberg JC, Bikson
51. Bohning DE et al (1999) A combined TMS/ M (2009) Role of cortical cell type and mor-
fMRI study of intensity-dependent TMS over phology in subthreshold and suprathreshold
motor cortex. Biol Psychiatry 45:385–394 uniform electric field stimulation in vitro. Brain
52. Bohning DE et al (2000) BOLD-f MRI response Stimul 2:215–228
to single-pulse transcranial magnetic stimulation 58. Antal A et al (2011) Transcranial direct current
(TMS). J Magn Reson Imaging 11:569–574 stimulation over the primary motor cortex dur-
53. Nitsche MA, Paulus W (2000) Excitability ing fMRI. Neuroimage 55(2):590–596
changes induced in the human motor cortex 59. Weber MJ et al (2014) Prefrontal transcranial
by weak transcranial direct current stimulation. direct current stimulation alters activation and
J Physiol 527(Pt 3):633–639 connectivity in cortical and subcortical reward
54. Fregni F, Boggio PS, Nitsche M, Bermpohl F, systems: A tDCS‐fMRI study. Hum Brain
Antal A, Feredoes E, Marcolin MA, Rigonatti Mapp 35(8):3673–3686
 Chapter 3

Selection of Optimal Pulse Sequences for fMRI

Mark J. Lowe and Erik B. Beall

Abstract
In this chapter, we discuss technical considerations regarding pulse sequence selection and sequence
parameter selection that can affect fMRI studies. The major focus is on optimizing MRI data acquisitions
for blood oxygen level-dependent signal detection. Specific recommendations are made for generic 1.5, 3,
and 7 T MRI scanners.

Key words MRI, fMRI, Pulse sequences, Blood oxygen level-dependent (BOLD), Echoplanar
i­maging, Spiral imaging, Multiband imaging, Motion

1 Introduction

NMR or MRI signals are generated by exposing nuclei placed in a

static magnetic field to radiofrequency (RF) pulses in the presence of
rapidly switching magnetic field gradients. These patterns of RF pulses
and magnetic field gradients are referred to as pulse sequences. Pulse
sequences dictate the contrast that will be present in MR images.
The issue of optimal pulse sequences, or more generally, optimal
data acquisition strategies, for functional neuroimaging is a complex
one. One cannot categorically say that a particular approach is supe-
rior to any other in all cases. In this chapter, we will examine the issues
that affect the detection sensitivity of neuronal activation in MRI and
discuss relevant data acquisition strategies that can be optimal in each
situation. For those readers not interested in the technical details
involved in fMRI pulse sequence optimization and wish to simply
read a summary of recommended pulse sequence strategies for blood
oxygen level-dependent (BOLD) fMRI, it is recommended that they
skip to Sect. 5, which summarizes the issues and presents recommen-
dations and caveats for each relevant sequence parameter.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_3, © Springer Science+Business Media New York 2016

69
70 Mark J. Lowe and Erik B. Beall

This chapter is organized in the following way:

1. Physics of functional contrast in MRI
(a) Nuclear Magnetic Resonance Relaxometry
(b) Exogenous Contrast
(c) Endogenous Contrast

2. Ultrafast Spatial encoding

(a) Echoplanar imaging
(b) Spiral imaging
(c) Parallel imaging
(d) Partial Fourier imaging
(e) Multiband echoplanar imaging
3. Artifacts
(a) Nonphysiologic
(b) Physiologic—head motion and cardiorespiratory noise
4. Optimization of sequence parameters
(a) Relaxation parameters and functional contrast
Field strength, intravascular, extravascular signal
(b) Experimental design
●● Block design fMRI
●● Event-related fMRI
5. Summary
Sequence recommendations

Many of these issues are covered in detail in other chapters. We

introduce them here in the context of pulse sequence selection and
optimization.

2 Physics of Functional Contrast in MRI

With a few notable exceptions, such as diffusion-weighted MRI,

MRI contrast stems from taking advantage of the different NMR
relaxation rates in different tissues and in the presence of pathol-
ogy. Felix Bloch phenomenologically characterized the dynamic
evolution of spin magnetization with two time constants, referred
to as T1 and T2. An in-depth discussion of the Bloch equations is
beyond the scope of this chapter, but for the purposes of under-
standing the interaction of pulse sequence parameters and func-
tional contrast in MRI, it is useful to briefly describe the processes
associated with these relaxation time constants.
 Optimal fMRI Pulse Sequences 71

T1 relaxation is taken to be the time constant of the return of

an excited ensemble of nuclei to the equilibrium state of the “lat-
tice” or surroundings. So, before excitation, the ensemble will
generally be in equilibrium with its surroundings. After excitation,
T1 governs the time for it to return to the state of equilibrium with
the lattice. This is sometimes referred to as spin-lattice relaxation.
T2 relaxation, which is technically an enhancement of T1 relax-
ation (i.e., the upper limit of T2 is T1), is the time constant for an
excited ensemble of nuclei to lose phase coherence through interac-
tions with each other. This is sometimes referred to as spin-spin relax-
ation. T2 relaxation in solids and tissue is typically much faster than T1.
For functional imaging, another important parameter govern-
ing relaxation is T2*. T2* is an enhancement of T2 caused by mag-
netic field gradients inhomogeneities. T2* is defined as:

1 1 1
*
= + ¢ (1)
T2 T2 T2

where T2′ is the additional relaxation contribution from field

inhomogeneities.
These relaxation processes are sensitive to the chemical environ-
ment of the nuclei. MRI utilizes this fact to produce images whose
contrast is based on the different relaxation rates in different tissues.

2.1 Nuclear Exposing nuclei in a static magnetic field to RF radiation at the

Magnetic Resonance Larmor frequency, given by:
Relaxometry
n L = gB (2)

will result in the absorption of energy by the nuclei. In Eq. (2), γ is

the gyromagnetic ratio and is a property of the nucleus. Since B is the
static field strength, we see from Eq. (2) that the Larmor frequency
will rise linearly with field strength. For protons, γ = 42.58 MHz/T,
so the Larmor frequency at 1.5 T is approximately 64 MHz.
When the RF radiation is stopped, the nuclei will gradually
release the energy into the surrounding material until they return
to the pre-excited state of equilibrium with their surroundings.
An MR pulse sequence is characterized mainly by two param-
eters that control the contrast of the acquired data. The first is
called the repetition time, or TR, which dictates how frequently
the nuclei in a particular location are excited. If they are excited
much more rapidly than the T1 relaxation rate of the tissue, the
protons will not recover to equilibrium between excitations. After
a few excitations, the nuclei in a given location will approach a
steady-state. Figure 1 shows an example of the signal evolution in
tissue with different T1’s as a function of TR.
The other important parameter that is used to control contrast is
the echo time, or TE. This is the time after excitation that the observed
72 Mark J. Lowe and Erik B. Beall

Fig 1 Steady-state MR signal as a function of repetition time for tissue with three
different T1 relaxation times

signal is spatially encoded. The amount of signal that can be spatially

encoded is dictated by both TR and TE. Two of the most common
methods for refocusing MR signal to allow spatial encoding are the
spin echo (SE) and the gradient recalled echo (GRE) methods.

2.1.1 The Spin Echo The time evolution of MR signal immediately after excitation is
referred to as free induction decay (FID). During the FID, the
processes governing the loss of signal coherence are a combination
of T1, T2, and T2′-related processes. In tissue, T2′ will have a large
effect on the loss of signal. T2′ effects are what is referred to as
reversible processes. The loss of phase coherence from these effects
can be reversed by applying a refocusing RF pulse. Figure 2 illus-
trates the sequence timing, using a pulse sequence timing diagram.
Application of a refocusing RF pulse at a time t after the initial
excitation pulse, will result in a complete refocusing of the revers-
ible dephasing effects at a time 2 t. This is referred to as a Spin
Echo. The time 2 t is usually called TE. The MR signal from a
given pulse sequence can be derived from the Bloch equations. For
a SE acquisition, the MR signal will be given by:

ì æ TE ö ü
æ -TE ö ïï ç TR - 2 ÷ æ -TR ö ïï
SSE µ exp ç ÷ í1 - 2 exp ç - ÷ + exp ç ÷ý (3)
è T2 ø ï ç T1 ÷ è T1 ø ï
ïî è ø ïþ

As is clear from Eq. (3), the MR signal from a SE acquisition is

moderated by T1 and T2. From this we can see that the T2 will
affect the encoded signal if the TE is comparable to, or longer than
T2. Figure 3 shows an example of the signal evolution for different
TE’s and T2’s.
 Optimal fMRI Pulse Sequences 73

Fig 2 Pulse sequence diagram of a spin echo acquisition. The envelope of the
signal indicates the FID, while the peak of the echo is modulated by T2 according
to Eq. (3)

Fig 3 Steady-state MR signal as a function of echo time for tissue with three
different T2 relaxation times

2.1.2 The Gradient It is possible to perform the spatial encoding for MRI during the
Recalled Echo FID. An echo can be created by increasing the dephasing through
application of a brief field gradient along a particular direction and
then reversing it while acquiring the signal data. This is called a
gradient recalled echo (GRE), or a field echo. The sequence
74 Mark J. Lowe and Erik B. Beall

Fig 4 Pulse sequence diagram of a gradient recalled echo acquisition

diagram for this technique is shown in Fig. 4. The signal obtained

from a GRE acquisition is given by:
æ -TR ö
1 - exp ç ÷
æ -TE ö è T1 ø
SGRE µ exp ç * ÷ sin a (4)
è T2 ø æ -TR ö
1 - cos a exp ç ÷
è T1 ø

where α is the flip angle, which is a sequence parameter that is a

function of the amount of transmitted power1.
It can be shown from Eqs. (3) and (4) that the TE that will
maximize the difference in signal between two tissues with differ-
ent T2 or T2* is approximately the average of the T2’s or T2* of the
two tissue types. Table 1 lists the T1 and T2 of gray matter and
white matter in the human brain at 1.5 T and 3.0 T.
Historically, evidence of regionally specific functional contrast
using MRI was first observed using an exogenous contrast agent
[1]. However, this observation was very quickly followed by several
groups, employing the phenomenon of BOLD contrast observed
by Ogawa and colleagues [2], utilizing endogenous contrast to
observe brain activation in several different brain regions [3–6].

2.2 Exogenous It is possible to generate dynamic MR images with functional con-

Functional Contrast trast by utilizing the fact that regional blood flow increases proxi-
mal to activated neurons. This is typically done by using methods
similar to those used to measure regional blood perfusion with
1
The flip angle can also affect the contrast of the generated images, but for
simplicity, we focus here on the more intuitive parameters TE and TR.
 Optimal fMRI Pulse Sequences 75

Table 1
Approximate relaxation times for gray and white matter at 1.5 T and 3 T

1.5 T 3T

T1(ms) T2(ms) T2*(ms) T1(ms) T2(ms) T2*(ms)

White Matter 600 80 70 800 70 60
Gray Matter 900 100 60 1100 90 50

Fig 5 Changes in MR brain signal intensity during the first-pass transit of intrave-
nously administered paramagnetic contrast agent. Triangle (Δ) symbols represent
the time course of signal during photic stimulation and circle (o) symbols represent
the time course during rest (darkness). Reproduced with permission from Ref. [1]

MRI. Gadolinium chelates will not cross the blood brain barrier.
Thus, a bolus injection of such a paramagnetic material will cause
a transient change in the T2 relaxation near arterial blood vessels
that are perfusing brain tissue. If this is done while rapidly acquir-
ing T2*-weighted MR images of the brain region that is active, one
will observe a decrease in the measured signal intensity that is
monotonically related to the volume of gadolinium passing
through. One can infer directly the volume of blood perfusing this
region from the signal decrease.
Functional contrast can thus be obtained in MRI by comparing
the regional perfusion, measured with bolus contrast injection, while
performing a task to that measured while at rest. Figure 5 is an
example of the difference in the MRI signal evolution from the same
brain region in visual cortex while undergoing photic stimulation
and in darkness. One can see that the area under the curve for photic
stimulation is larger than that for rest, indicating that the volume of
blood perfusing the tissue was increased during stimulation.
76 Mark J. Lowe and Erik B. Beall

In this manner, one can produce voxel level comparisons of the

area under the bolus passage curve in the MR time courses and
determine those whose measured volume change was statistically
significant.
Due to the invasive nature of the necessary bolus injection, the
(albeit low level) risk of adverse reaction to contrast agents, the
lower signal-to-noise ratio (SNR) of perfusion measurement tech-
niques, and to a lesser extent, the limited volume coverage of per-
fusion measuring techniques, exogenous contrast-enhanced fMRI
is only rarely performed and usually for reasons specific to a par-
ticular experimental design.

2.3 Endogenous There are two principal mechanisms for generating contrast in MR
Functional Contrast images using endogenous features related to neuronal activation.
Both of these mechanisms are related to the hemodynamic response
to an increase in neuronal activation. One of these is a regional increase
in blood flow and the other is a concomitant increase in the oxygen-
ation content of the blood perfusing tissue near activated neurons.
Far and away the most commonly employed fMRI acquisitions
utilize the fact that regional brain activation results in a local
increase in blood oxygenation. This is called BOLD contrast. The
contrast in BOLD stems from the fact that oxygenated hemoglo-
bin is a weakly diamagnetic molecule, while deoxygenated hemo-
globin is a strongly paramagnetic molecule. The relative increase in
the concentration of oxygenated hemoglobin in the vessels perfus-
ing activated tissue results in an increase in the T2 and T2* relax-
ation times in the affected brain regions. Thus, methods utilizing
BOLD contrast for fMRI employ acquisition techniques that are
sensitive to changes in T2 and T2*. Because of the flexibility of T2
and T2* acquisition methods, this has become the contrast of
choice for the vast majority of fMRI experiments. For this reason,
the remainder of this chapter will focus on acquisition strategies to
acquire BOLD-weighted MRI data and we will discuss methods to
optimize these depending on experimental needs.

3 Ultrafast Spatial Encoding

It is possible to generate MR images that will demonstrate a change

in signal in brain regions that transition from the inactive to active
state. These transitions are typically very rapid and the advantage
of MRI over other imaging techniques is the ability to acquire even
whole brain images very rapidly. In this section, we introduce the
concept of spatial encoding in MRI and discuss the most common
ultrafast imaging pulse sequences used in fMRI. For simplicity,
throughout this section, we refer to the net magnetization within
a sample as “spin.”
 Optimal fMRI Pulse Sequences 77

3.1 The Pulse As stated above, the pulse sequence refers to the specific acquisi-
Sequence tion strategy in which spatial encoding and magnetization read-out
is performed, providing the basic structure of the RF pulses and
field gradients used. Because conventional MRI is based on Fourier
spatial encoding, it has become convention within MRI to discuss
pulse sequences in the context of k-space, another name for the
Fourier conjugate of coordinate space. K-space is essentially the
image in the spatial frequency domain, and most pulse sequences
acquire image data in this domain. There are a variety of advan-
tages to this; most importantly that a coordinate space image can
be produced simply by performing a 2-dimensional Fourier trans-
form (typically computed using the Fast Fourier Transform, or
FFT) on sequentially acquired MRI data.
A pulse sequence for reading one arbitrary line of k-space with
a gradient-recalled echo is shown in Fig. 6.
Starting with stage (1), waveforms are played out on the
z-direction gradient, GZ, and the RF transmit channel to excite a
slice of proton spins. During stage (2), the readout gradient (Gx)
prewind and phase-encode gradient selection (Gy) is performed
while rephasing spins across the slice/slab with GZ. During stage
(3), the readout gradient is switched on while the emitted RF sig-
nal from the sample is recorded, denoted by the block of dotted
lines. The diagrams shown are simplifications, where the timing
and form of the gradients are changed according to various design
considerations. The corresponding traversal of k-space for the
pulse sequence in Fig. 6 is shown in Fig. 7.
Typical conventional (i.e., not single-shot) sequences repeat
this process, for different lines of kY, or phase-encoding positions.
This is shown in Fig. 6, step 2 with the GY gradient at multiple

Fig 6 Pulse sequence diagram for reading one line of k-space, time increases
from left to right. The proton echo from the sample is shown here in light gray
during and under the readout window, which will be sampled by a receive coil
78 Mark J. Lowe and Erik B. Beall

Fig 7 k-space diagram showing 16x16 matrix of data sampling points and trajectory of pulse sequence dia-
grammed in Fig. 6. k-space is first pre-wound in step 2 (GX moves position in kX from 0 to −8, GY moves posi-
tion in kY from 0 down to line −3), then kX is traversed in step 3 while sampling from –8 to +8

possible values containing the variation in the repeated lines of

k-space sampling. It should be noted that the distance traveled in
k-space is proportional to the time integral of the gradient strength
in space, so it is possible to use a larger amplitude to shorten the
time taken to traverse k-space2.
The sequence described above pertains to a GRE, but without
loss of generality, the same sequence applies to a SE sequence. A
slice selective RF excitation pulse for both GRE and SE is typically
a sinc function-shaped pulse, with amplitude set to rotate the slice
magnetization 90° from the longitudinal magnetization direction
into the plane transverse to the static field. A SE sequence is very
similar, but with the addition of a refocusing pulse set at 180°,
timed to play out midway between the centers of the RF excitation
pulse in step 1 and the readout window in step 3 in Fig. 6. The
differences between SE and GRE can be seen in Fig. 8. The timing
of the inversion pulse after the excitation pulse dictates the TE, so
a short TE can preclude the SE method. SE can be advantageous
2
up to the limits of the gradient hardware and not without various drawbacks.
 Optimal fMRI Pulse Sequences 79

Fig 8 Generalized diagram for either gradient-echo (GE) or spin-echo (SE) ultrafast pulse sequences specific to
ultrafast imaging. A fat saturation pulse (described later), followed by excitation, then a refocus pulse (if spin-
echo only), prewind gradients to set position in k-space, then the readout and spatial encoding gradients. Finally
there may be postwind spoiler/crusher gradients (RF may also be used at the end) to dephase residual signal

because it can cancel dephasing due to local field inhomogeneities,

leading to an improved SNR, but single shot sequences get less of
this benefit due to an effective spread of TEs which will be dis-
cussed later.
Strategies for what is referred to as “single-shot” imaging are
critical to the high sampling rates necessary for dynamic imaging
techniques such as fMRI. Echo-planar imaging (EPI) and spiral
imaging are the most widely used of these. In addition, in-plane
and multiband parallel imaging techniques, combined with the
ubiquitous use of multichannel coil technology, will play an increas-
ing role in fMRI.
Single-shot sequences differ from conventional pulse
sequences in that the data for an entire slice are acquired in one
readout window after one excitation. This has been made possi-
ble by fast gradient switching technologies, and single-shot
sequences are available on all modern MRI scanners. Common
to all fast imaging sequences are higher demands on the hard-
ware, which increase vibration and heating of the scanner, lead-
ing to increasing inhomogeneity and field drift over time during
long scans [7, 8]. Parallel imaging is a recent development,
which reduces the readout time by acquiring data from multiple
coils. These imaging strategies have various artifacts and trad-
eoffs, which will be discussed below following an introduction to
the most common strategies.
80 Mark J. Lowe and Erik B. Beall

3.2 Echo-Planar EPI follows the basic strategy of excitation of a slice or slab fol-
Imaging lowed by readout of one line (in the read-out direction, or kx) in
k-space. The GRE sequence was shown in the pulse sequence tim-
ing diagram in Fig. 8 without the SE refocusing pulse, which was
also shown in parts in Fig. 6. With the fast gradient switching
speeds available in recent years, it has become possible to spatially
encode an entire slice in one echo by performing multiple readouts
and phase-encoding steps after a single excitation. The most com-
mon implementation, known as “Blipped EPI”, involves excitation
of a slice followed by readout of kx line like the GRE sequence. The
sequence continues however, after an increment, or “blip”, of the
position in k-space in the other dimension using a short duration
gradient pulse (in the phase-encode direction, or ky). Readout con-
tinues when the read-out gradient is reversed to read another kx
line in k-space immediately adjacent to the first line sampled but in
the opposite direction. This is shown in Fig. 9.
These reversals and blips are repeated to adequately sample
k-space and the resulting data can be treated in the same manner as
multi-shot imaging, with the full readout of the slice or slab cen-
tered on the TE. The trajectory in k-space is shown in Fig. 10.
Three-dimensional acquisitions can be performed using an
additional increment in the perpendicular dimension, or kz,
although most blipped-EPI sequences are two dimensional only
due to the constraint of a shorter TE required. There exist many
modifications to this basic structure, but all EPI strategies contain

Fig 9 Blipped-EPI pulse sequence. Readout gradients are reversed following

readout of each kx line, along with a small increment of k-space in ky direction,
or a “blip” in Gy. 16 kY lines are read out, corresponding to the k-space diagram
in Fig 7. Gradient-stimulated echo train is shown in light gray, which becomes
stronger closer to center of k-space, and at center of each kx-readout
 Optimal fMRI Pulse Sequences 81

Fig 10 Cartesian trajectory in k-space for one-shot blipped-EPI sequence shown in Fig. 5. One-shot means full
coverage of k-space is accomplished during echo of one excitation

a fast back-and-forth cycling of the gradients to produce a GRE

train. The blipped-EPI strategy is commonly also referred to as
Cartesian imaging, due to the rectangular trajectory of read-out in
k-space. We will not discuss other non-Cartesian strategies that are
no longer common such as constant-phase encode EPI or square-­
spiral EPI. The signal generation stage before the spatial encoding
can include a refocusing pulse or not, depending on whether T2
(SE-EPI) or T2* (GRE-EPI) weighting is desired.

3.3 Spiral Imaging Another common strategy for single shot imaging is spiral imaging
[9]. In this scheme, k-space is sampled in a spiral or circular man-
ner, such as in Fig. 11, with less asymmetry between the rate of
sampling in kx- and ky-space.
By applying sinusoidal gradients 90° out of phase to the read-
and phase-encode gradients, k-space can be traversed in a circular
manner by increasing the amplitude of the sinusoidal gradients. A
typical sequence for spiral acquisitions is shown in Fig. 12.
82 Mark J. Lowe and Erik B. Beall

Fig 11 k-space sampling trajectory for a one-shot spiral imaging sequence

shown over a rectangular grid. Central k-space is sampled first. Prior to FFT
reconstruction, the data must be resampled from spiral grid to Cartesian grid

Fig 12 Pulse sequence timing diagram for spiral acquisition. Gradients during
readout window are 90° phase-offset ramped sinusoids

This process continues until k-space is adequately sampled.

There are many trajectory modifications to this scheme, but all
have the basic property that the sampling of k-space is not Cartesian,
but instead approximately radial-symmetric. Reconstruction of
image space from k-space may be done using a Fourier transform
after resampling the k-space data to a Cartesian grid, but there are
implementations that reconstruct the image data using the discrete
Fourier transform [10, 11].
 Optimal fMRI Pulse Sequences 83

Spiral imaging has advantages compared with blipped EPI,

mostly related to the lower overall demand on the gradients. These
include reduced gradient noise, improved SNR, lower induced
eddy currents3, and different geometric distortion artifacts [12].
Increased SNR is due to the earlier sampling of the center of
k-space, but the correspondingly later acquisition of the outer
regions of k-space mean the higher spatial frequencies have lower
specificity than the readout direction of a blipped-EPI image. The
typically sinusoidal gradient play-out means the gradients are
switched at a lower rate of change, which reduces the induced eddy
currents and gradient noise. Since most of the magnetization sig-
nal naturally lies near the center of k-space, which is sampled early,
it is preferable to start the readout window at the TE. This modi-
fies the timing from the Cartesian EPI sequence where the readout
is centered on the TE, although newer spiral sequences (such as
spiral-in/out) are available which also center the TE in the readout
window [13]. A spiral-in/out sequence is shown in Fig. 13.
A Spiral-in/out sequence reduces the effects of the echo-­shifting
by centering the readout at the TE as in EPI. Readout begins prior
to the TE, starting near the edges of k-space and spiraling in to the
center, which is reached at the TE, before spiraling back out over
new data points. After resampling the grid, every point in k-space
now has two samplings symmetrically spaced about the TE, which
are passed through FFT to give two images. These two images can
be combined and the result is a reduced sensitivity to susceptibility
signal loss and image data with an effective TE closer to that speci-
fied [14]. This is more demanding on the gradient hardware than
spiral imaging and there can be drawbacks in image quality.

3.4 Parallel Imaging Multiple receive coils have become a popular and widely available
means to increase image SNR by providing multiple samples of a
k-space trajectory. Because the coils cannot be located in the same
place, they have varying spatial sensitivities to the tissue, which is
maximal at the tissue nearest each coil element. This provides an
alternative spatial encoding mechanism, where one sample of sev-
eral parallel coil elements provides information about the magneti-
zation density over several regions of tissue. Parallel imaging
combines this spatial encoding with the gradient-mediated spatial
encoding to skip some gradient-encoded lines in k-space and
replace those gaps with information derived from the parallel coil
elements [15]. The skipped lines in k-space reduce the field of view
(FOV) seen by the coils by a reduction factor. The individual coils,
if reconstructed with only the data acquired, would see the nearest
portion of tissue inside that coil’s FOV, but with aliased image
overlap with other portions of tissue further away from the coil.

3
Eddy currents are currents induced in gradient coils and other scanner com-
ponents from the rapidly changing fields generated by the gradient coils.
84 Mark J. Lowe and Erik B. Beall

Fig 13 Spiral-in/out sequence acquires full k-space data prior to echo time and
a second acquisition of k-space after echo time. The data from these echoes are
combined in reconstruction

The methods used to un-alias the data can be separated into two
strategies: image-space unfolding and k-space interpolation. While
there are many methods, and more than a few hybrids, the most
common implementations of each (sensitivity encoding, SENSE
[16] and generalized autocalibrating partially parallel acquisitions,
GRAPPA [17]) will be discussed, along with benefits/drawbacks.

3.4.1 SENSE SENSE performs the reconstruction of parallel images in image

space in an iterative manner using a seeded coil sensitivity matrix.
Prior to the parallelized scan, the sensitivity of each coil in the full
FOV is measured. These sensitivity maps are used as an initial guess
for the “unfolding” matrix.
The under sampling of k-space shown in Fig. 14 leads to image
aliasing when reconstructed. However, if the multiple coils are sen-
sitive to spins from different aliased regions, then the portion of
signal aliased or unaliased in each image can be differentiated using
the sensitivity of the multiple coils to the different regions. The
matrix inversion is performed iteratively after pre-processing the
data to handle the problem of nonideal coil geometry.

3.4.2 GRAPPA GRAPPA is a regenerative k-space method, using measured data to

calculate missing phase-encoding lines. Outer regions of k-space
have reduced sampling. The acceleration factor defines the number
of lines skipped per line acquired. The central k-space lines, or
autocalibration signal (ACS) lines, are fully sampled, which is
shown in Fig. 15.
The ACS lines are used to interpolate the nonacquired lines of
k-space by fitting the acquired lines to the ACS data. This is per-
formed separately for each coil used, leading to weights specific to
 Optimal fMRI Pulse Sequences 85

Fig 14 SENSE k-space traversal for acceleration factor 2. Odd lines of k-space are missed, even lines acquired.
Acceleration factor equals acquired plus nonacquired number of lines, divided by acquired lines, in this case
the full k-space matrix would have twice the number of lines as were actually acquired

each ACS line, for each coil. So for N coils, there will be N2 weights
resulting from the fitting procedure to use in interpolating the non-
acquired lines. A particular coil’s matrix is based on all coil signals,
but masks out, or de-weights, signal from other regions outside the
FOV of that coil, in k-space. The matrix weighting removes the alias-
ing seen in the original, undersampled images. The final, unaliased
image data for each coil is combined by sum-­of-­square. The greatest
advantage of GRAPPA over image-space methods is the determina-
tion of sensitivity from the k-space data itself, which is useful in images
containing regions with poor homogeneity or low signal, both of
which are the case with ultrafast imaging [18].

3.4.3 Tradeoffs The primary benefit of parallel imaging is a reduction of the time
spent spatially encoding (the readout window), but at a cost of
SNR compared to the same sequence with a fully gradient-based
spatially encoded image using the average signal from the parallel
86 Mark J. Lowe and Erik B. Beall

Fig 15 GRAPPA k-space traversal. Central k-space is fully sampled to provide ACS lines. Outer k-space is
undersampled in phase-encoding direction by acceleration factor. Acceleration factor here is 2, so every other
line is acquired. Same sampling is acquired for other coils

coils [19, 20]. The reduction in SNR is due to reduced coverage of

k-space, or the square root of the acceleration factor. An additional
cost for any parallel imaging is the coil geometry coverage, or
g-factor. In SENSE imaging, the g-factor directly relates to the
invertibility of the sensitivity matrix [21]. Since most ultrafast par-
allel blipped-EPI sequences are acquired in 2D only, the coil cover-
age should be optimized in the phase encoding direction [22].
Spiral-EPI with parallel imaging is more complicated than blipped-­
EPI and much more time consuming but recent advances have
reduced the reconstruction time for parallel spiral-EPI [23–25].

3.4.4 Artifacts Residual alias in the image is a common artifact seen with parallel
imaging. The SENSE method requires the full image FOV to be
greater than the object of interest in any accelerated directions; other-
wise reconstruction will fail resulting in considerably aliased images.
The only current solution with SENSE is to expand the FOV so there
is no image wrapping [26]. This is not an issue with GRAPPA, because
 Optimal fMRI Pulse Sequences 87

the spacing of k-space lines determines the FOV, but not the signal of
a particular line. Therefore, k-space fitting under GRAPPA is not
compromised by a smaller FOV, while image space fitting would be
compromised by the aliasing image. With an ideal sensitivity map, it
was believed SENSE could theoretically give better results than
GRAPPA, however the accuracy of the maps are highly dependent on
local field homogeneity and subject motion can invalidate them.
There are now several sensitivity map methods, including ones based
in part on GRAPPA autocalibrating methods that derive the maps
from the data to get around these problems [27]. Finally, there is the
issue of fitting the systems of equations in the presence of incomplete
coil coverage. The SENSE method requires the solution of an inverse
problem, but if there are regions of tissue that no coil has adequate
sensitivity to, this is an ill-conditioned problem that cannot be exactly
solved. All implementations of SENSE regularize or condition the
data to work around this, but it means that the final reconstructed
image may have local noise enhancements [28–30]. GRAPPA is also
sensitive to this problem, but because fitting is done with k-space data,
the noise enhancement is global rather than local [21].

3.4.5 Limitations There is a practical limit to the number of coils and the acceleration
factor, because adjacent coils will overlap spatially in their sensitivity
and coverage of magnetized spins, reducing the ability to separate
aliased signals. In typical applications of ultrafast imaging, the use of
2D EPI sequences leads to a limit on the acceleration factor of
between 4 and 5 [19]. While not every scanner has multiple chan-
nels, it is becoming the standard for vendors to offer such capability.
Many sites do not use parallel imaging for BOLD at 1.5 and 3 T due
to higher than expected SNR loss at even the lowest acceleration
factor [31], but at 7 T and higher field strengths parallel imaging is
necessary in most BOLD acquisitions to reduce echo time and image
warping. Future implementations of parallel imaging promise higher
acceleration factors with less loss of SNR using hybrids of k-space
and image-space methods with dynamically changing undersam-
pling strategies, such as k–t SENSE [32] or k-t GRAPPA [33].

3.5 Partial Fourier Ideal k-space data has complex conjugate symmetry, which can be
Imaging exploited to reduce the acquisition time. Up to half of k-space can
be interpolated from symmetry with the other half. This is referred
to as partial Fourier imaging. The symmetry is only approximately
true in real data due to scanner and tissue nonidealities, so algo-
rithms to take advantage of this fact must use low-resolution
approximations to account for nonzero phases in regions breaking
this symmetry [34, 35]. With the use of partial Fourier acquisition,
a higher spatial resolution can be acquired with less signal loss and
blurring, with the result that the SNR does not drop along with
the reduced acquisition time [36, 37].
88 Mark J. Lowe and Erik B. Beall

3.6 Multiband Multiband EPI refers to the simultaneous excitation of multiple

Echoplanar Imaging 2D slices (multiband excitation), and the reconstruction of indi-
vidual slices using parallel imaging reconstruction in the slice-­
encoding direction (slice-GRAPPA) [38, 39]. Multiband EPI is a
recent addition to the arsenal of accelerated techniques to increase
spatial and temporal resolution. Most importantly, multiband is
not hindered by the SNR reduction associated with reduced sam-
pling in traditional parallel imaging, because there is no reduced
sampling of the signal with multiband. There is however a g-factor
penalty from nonoptimal coil sensitivity profile. Similar to in-plane
parallel imaging, the signal from N simultaneous slices can only be
resolved if there are N coil measurements with varying sensitivity
at each slice. This dependence is reduced considerably by using
balanced blipped phase encoding in the slice direction, termed
blipped-Controlled Aliasing in Parallel Imaging, or blipped-CAIPI,
to improve signal separation in the slice direction [39]. Blipped-­
CAIPI consists of applying an alternating “balanced” blip on the
slice-encoding gradient simultaneously with each phase-encoding
blip on the phase-encoding gradient. By balanced blip, we mean
that the sum of blips cancels out over the readout train. Using
these blips, the signal from simultaneous slices can be shifted in the
phase-encoding direction by a fraction of the FOV depending on
where the slice is located and on the balancing of the slice blips,
while sustaining minimal signal blurring from those slice-direction
blips. The train of slice blips is designed to cancel out, either with
alternating positive and negative polarity blips or a series of alter-
nating fractions. This results in improved separation of the signals
in k-space, leading directly to a reduced effective g-factor penalty
and improved reconstruction. When using blipped-CAIPI multi-
band, the slice acceleration factor typically ranges from 2 to 8.
Although higher acceleration factors have been reported [38], the
reconstruction quality declines and artifacts increase as a nonlinear
function of the acceleration factor. These methods have been used
to acquire 2D EPI (BOLD, DTI, and ASL) data with a corre-
sponding increase in the number of slices acquired per unit time.
This can be used in either direction, e.g., either to obtain a lower
TR or higher slice count in a given TR than previously possible.
Multiband EPI is becoming more practical and is widely used at
both 3 and 7 T, but it is critically dependent upon the use of a coil
with a sensitivity profile that varies across channels in the slice-­
encoding direction. Fortunately, most new coils being designed or
sold for functional brain imaging are suitable for multiband imaging.
Furthermore, multiband with blipped-CAIPI has been validated
and shown to have minimal artifacts in optimized protocols [40,
41]. Nevertheless, there is one artifact most associated with multi-
band EPI: interslice leakage artifact. Interslice leakage refers to
incomplete separation of slice signals over time, leading to spatial
aliasing of BOLD effects from a given location across the other slices
 Optimal fMRI Pulse Sequences 89

that had been excited at the same time as that given location. The
level of artifact has been assessed [40–42] using simulation or recon-
struction details not usually available to the average investigator. It
may be possible to determine the level of artifact on time series
image data alone, such as using seed voxel correlation [43] and com-
puting the difference between correlation patterns to aliased slices
and non-aliased slices, but there is at present no established method.
The flexibility afforded by these new techniques increases the
likelihood that investigators will customize the protocol according
to their specific needs. For example, some investigators are focusing
efforts on whole brain BOLD acquisitions with short TR, while oth-
ers focus on whole brain with thinner slices and are less concerned
with TR. At present it is too early to recommend a specific protocol,
but we will here describe two protocols with different goals that
have been used for connectivity and fMRI studies. The first protocol
was designed for a short TR and whole-brain coverage, the second
protocol designed for high spatial resolution and whole-brain cover-
age. Both protocols use a 32 channel head coil.
3 T Short TR protocol: acceleration factor = 8, 2 mm isotropic
voxel size, TR = 720 milliseconds. The short TR was intended both
to directly sample physiologic noise artifacts from the cardiorespi-
ratory cycles and to enhance statistical power using more BOLD
samples in a given scan time [42]. Figure 16 shows an example
functional connectivity study with this acquisition.
7 T high-resolution protocol: acceleration factor = 3, FOV/3
blipped-CAIPI FOV shifting, voxel size = 1.2 × 1.2 × 1.5 mm, 81
1.5 mm thick slices, TR = 2.8 s. The high resolution protocol was
intended to reduce slice thickness and increase resolution.
Figure 17 shows an example functional connectivity study done
using this protocol.

4 Artifacts

4.1 Non Physiologic There are several potential artifacts from single-shot imaging tech-
niques due to hardware realities, such as chemical shift (fat) artifact,
eddy current artifacts induced by the fast gradient switching, imperfec-
tions in gradient ramping waveforms, and both blurring and signal loss
due to nonuniform TEs combined with static field inhomogeneity.

4.1.1 Water-Fat Shift Water-fat shift image artifact is a consequence of the off-resonance
frequency of body fat that shifts the fat signal mostly in the phase-­
encode direction, misplacing it across the image. Fat suppression
with an RF pulse at the resonance frequency of fat, often called
chemical saturation, followed by a strong dephasing gradient is the
standard countermeasure on EPI sequences. This RF pulse is done
immediately before the initial excitation pulse and, due to the fact
that the longitudinal signal of the fat is saturated, water protons in
fat will experience no excitation. One immediate consequence of
90 Mark J. Lowe and Erik B. Beall

Fig 16 (a) Single-slice correlation map to seed located in left primary motor cortex, data consists of 132 vol-
umes acquired with 2.8 s repetition time, (b) shows power spectrum of seed timeseries in Hz, (c) correlation
map to same seed location in separate data consisting of 416 volumes acquired with 0.72 s repetition time
and (d) shows power spectrum of seed timeseries of fast sampled data

this approach is an increase in the time taken by the sequence, as

this off-resonance pulse must be performed once before every exci-
tation pulse. Because the resonance frequency of fat protons is only
3.35 parts per million (ppm) in frequency away from water pro-
tons, the homogeneity of the static field must be very good to help
ensure the suppression pulse acts only on fat protons and the exci-
tation pulse acts only on water protons.
An alternative strategy to chemical saturation is the use of spatial-­
spectral RF excitation pulses [44]. These are patterned RF and gra-
dient pulses played out over many milliseconds. Properly designed,
the aggregate affect of the ensemble of pulses is to create discrete
regions of excitation in space and frequency. It is possible to design
these pulses such that the excited regions are separated by more than
 Optimal fMRI Pulse Sequences 91

Fig 17 Correlation map to seed in left primary motor cortex in 7 T data acquired
with multiband EPI acquisition with 1.2 × 1.2 mm × 1.5 mm voxels

3.35 ppm in frequency, such that water protons in tissue in a given

slice will be excited and water protons in fat will not. This strategy
requires less field homogeneity than chemical saturation, but they
tend to have a poor slice profile and can take up to twice as long as a
good chemical saturation pulse to achieve the same result.

4.1.2 Gradient Time-varying magnetic gradients induce eddy currents in nearby

Nonidealities electrical conductors, such as the magnet cryostat. These eddy cur-
rents create magnetic fields that partially cancel the effect of the
applied gradients. Fast gradient ramping is limited in hardware by
the reactance of the gradient coils, creating effective upper limits
on gradient switching that perturb the intended ramping wave-
form that the gradient coil is driven with. These waveform pertur-
bations increase as gradient switching time decreases. The induced
eddy currents and gradient ramping imperfections create phase
errors in k-space magnetization read-out, which produces different
artifacts depending on the k-space trajectory.
In blipped-EPI, artifact is magnified in the phase-encode direc-
tion. This creates what is known as a “phase ghost” or “N/2
ghost”, an identical image at 2–5 % of the original image signal
level but offset by 90° in the phase-encode direction. The ghost
can have some overlap with the image of interest. In spiral EPI, the
artifact is not as simple, but will result in an increase in noise level.
92 Mark J. Lowe and Erik B. Beall

Fig 18 Gradient waveform calibration. GT shows the theoretical, intended gradi-

ent, while GR shows the real waveform due to eddy current damping the intended
waveform. GC shows a calibrated waveform to be played out on the gradient coils
to produce the intended gradient despite the presence of eddy current

The corrective methods used vary by scanner manufacturer,

but there are some commonalities. The first line of defense is in the
screening of the gradient coils to reduce the field change and con-
comitant eddy currents. A second method employed is calibration
of the gradient waveforms. To an extent, eddy currents can be
predicted and compensated for by pre-emphasis of the gradient
waveforms. This is shown in Fig. 18.
Generally, the effect of coil reactance is to dampen the intended
gradient waveform by providing a resistance to it, so the waveform
to be played out on each gradient coil is modified by a predeter-
mined calibration. Changing the configuration of conductive
objects in the scanner room can make this calibration obsolete, if
they are near and large enough to be affected by the gradient fields.
This could show up as a sudden increase in N/2 ghosting in
blipped-EPI images, requiring a recalibration of the gradient wave-
form perturbation. Further anti-ghost calibrations to account for
system timing offsets and residual eddy current effects may be per-
formed, such as phase line correction. A calibration is typically
taken during just prior to the readout window in the blipped-EPI
sequence by sampling forwards and back across the middle of
k-space. Eddy current and timing offsets result in a nonideal
k-space trajectory that can be approximated as a simple shifting of
each line of k-space forwards and back depending on direction of
traversal. The calibration lines are used to resample every readout
kY line to center the received echo [45, 46]. A failure of this online
phase ghost correction algorithm would show up as a dramatic
increase in phase ghost signal level, to a level comparable to the
image of interest. An example of a failure of online phase ghost
correction is shown in Fig. 19. In this case, signal changes seen as
a result of phase ghost correction overwhelm the BOLD effect.
 Optimal fMRI Pulse Sequences 93

Fig 19 Phase ghost typical level on left. Phase ghost correction algorithm failure on right. All measured signal
values normalized to first brain tissue measurement on top left

The effect of eddy current on spiral imaging is smaller since

dB/dt is lower, but if uncompensated will result in warping,
because it warps the k-space trajectory in both dimensions.
Measuring the actual trajectory taken in k-space can be used in the
resampling portion of reconstruction to correct this image warping
similar to the blipped-EPI phase line correction [47].

4.1.3 Echo Shifting The long readout time employed leaves single-shot images highly
sensitive to static field homogeneity, leading to image distortion
artifact in regions of inhomogeneity. The off-resonance frequency
in these regions causes an accumulation of phase errors in those
regions over the readout time. Phase errors specific to a region
result in spatial encoding errors, which manifest as signal misplace-
ment from that region. For blipped-EPI images, this is insignifi-
cant in the read-out, or kx direction because it is sampled so quickly,
but the phase-encode direction is sampled more slowly, resulting in
spatial distortion, or blurring in the phase-encode direction in
those regions. Spiral imaging samples the kx and ky dimensions at
approximately the same rate, but the radial dimension is sampled
more slowly, akin to the phase-encode direction in EPI. Spiral
images are therefore blurred across both dimensions [48]. The
geometric distortion can be “unwarped” from the images using
the calculated pixel shifts from an acquired field map for both
blipped-EPI [7, 49] and spiral imaging [35] (Fig. 20).

4.1.4 Signal Loss Signal loss, or slice dropout, is caused by through-slice dephasing
after the RF excitation. This signal loss cannot be recovered with-
out modifying the pulse sequence. Strategies for overcoming this
include: use of SE to refocus the dephasing effects, reducing the
TE, reducing slice thickness and/or in-plane voxel size, and chang-
ing the scan plane. If hardware permits, the use of high order gra-
dient shims and image-based shimming can help [50–52].
94 Mark J. Lowe and Erik B. Beall

Fig 20 (a) Sagittal, (b) axial views of fieldmap. Other pictures show (c) and (d) blurred blipped-EPI signal and
(e) and (f) unwarped images of the same using pixelshifts calculated from the fieldmap. Anterior regions
shifted roughly 1–2 pixels, but signal loss near the frontal sinuses cannot be recovered, leaving a signal void

Another common method is z-shimming to unwrap some of

the dephasing, but this has the disadvantage of reducing the SNR
in unaffected regions. Z-shim methods rely on acquiring a field-
map to estimate the gradient across a slice, and then applying an
opposing (negative) slice gradient that is equal to the fieldmap
measured gradient in the high susceptibility region [53, 54]. This
reduces the dephasing in the high susceptibility region, but at the
cost of increasing dephasing in other less-affected regions. To deal
with this problem, typically two images are taken; one with the
z-shim and one without and then these are added together to
restore all signal loss, but at the cost of nearly doubled imaging
time. This method is approximately equivalent to an older tech-
nique of tailoring RF pulses to apply a set dephase across the slice
at time of excitation [55]. Alternatively, several images with a range
of z-shim gradients linearly spaced between zero and the maxi-
mum measured gradient are taken, and these are averaged, how-
ever, this is more time consuming with little benefit over the more
common method with two images.

4.1.5 Spiral Regridding Apart from lower spatial specificity and increased acquisition
time, problems typically associated with spiral imaging lie in the
regridding of the spiral trajectory to a Cartesian coordinate sys-
tem prior to Fourier transform. Regridding introduces subtle
artifacts and reduces SNR, and is computationally intensive com-
pared with the reconstruction methods used in blipped EPI,
although computational improvements have been made [56, 57].
 Optimal fMRI Pulse Sequences 95

Until recently, the reconstruction had to be performed offline on

a separate image-­processing computer after the scan, which made
it difficult to validate data online or use prospective motion cor-
rection. There are now online versions of spiral reconstruction,
such that prospective motion implementations for spiral imaging
have been used to monitor subject motion [58].

4.2 Physiologic There is another class of image artifacts present in functional neuro-
Artifacts imaging that have physiologic origins. Head motion [59–61] and
physiologic noise from the heart and breathing cycles [62–64] are
unavoidable non-neuronal sources of variance, changing underly-
ing statistical distributions and introducing possible systematic
effects in population studies. Accounting for these artifacts requires
care in the acquisition of data and several stages of postprocessing
of data (retrospective correction techniques will not be discussed
here) after collection is complete. Preventive measures include head
restraints or navigator echoes to reduce the effects of head motion,
and routine scanner quality assurance measures to track the stability
of the scanning hardware [65]. In addition, the collection of paral-
lel measures of state during the image acquisition may be useful for
artifact removal during postprocessing. These parallel measure-
ments can include online motion detection parameters from naviga-
tor echo or prospective motion correction along with signals
representing physiologic cardiac and respiratory cycles. Parallel
measurements in general must be acquired as fast or faster than the
slice acquisition sampling rate. It should be noted that there are
post-acquisition methods for obtaining slice-sampling rate motion
parameters [66] and physiologic cardiac and respiratory cycles [67].

4.2.1 Head Motion Due to the fact that it is desired to maintain a high temporal resolu-
tion, the sampling rate in most fMRI acquisitions is fast compared
to the T1 of brain tissue. The consequence of this is that, after equi-
librium is achieved after acquisition of a few volumes, the tissue is in
a saturated state. This means that the magnetization is not com-
pletely recovered between excitations of a given slice. If a subject
moves such that tissue from one slice moves into an adjacent slice,
the tissue will, for the first excitation after the motion, be in a dif-
ferent state of saturation than the rest of the tissue in that slice. This
leads to a signal change that is correlated with the motion, but will
not be corrected by the traditional technique of retrospective
realignment of the images. Prospective motion correction tech-
niques are intended to deal with this problem in real-time.
Navigator-echoes can be used to obtain motion information
during the acquisition of data [68, 69]. This technique uses the
fact that the phase of MR data is sensitive to motion. Typically,
low-power RF pulses are interspersed with the fMRI data acquisi-
tion and the phase information from the readout of the signal from
these pulses is used to infer motion along a given direction. The
96 Mark J. Lowe and Erik B. Beall

drawbacks of the navigator echo approach is that, unless the power

is very low, in which there is a limited ability to determine phase
changes, the excitation pulses will affect the spin history of the
fMRI data. Nevertheless, these approaches have been used with
some success in fMRI.
An alternative approach that has been applied is to use real-­
time co-registration of a reference volume to the current volume to
determine motion. The motion parameters are determined from
the co-registration and are applied to the imaging system prior to
acquiring the next volume [58, 70]. The drawback of this approach
is that it is more computationally intensive that the navigator echo
approach and it does not update the slice locations until after the
motion occurs. Thus, registration-based prospective motion cor-
rection must be coupled with postprocessing motion correction. It
should be noted that it is now possible to acquire motion param-
eters at the slice level using the image data itself [66], and this
could be used in the future to further ameliorate head motion arti-
facts at the point of acquisition.

4.2.2 Physiologic Noise Ongoing physiologic processes in living subjects present an addi-
tional potential artifact. Effects due to the cardiac and respiratory
cycles have been identified as being significantly coupled to BOLD-­
weighted MR signal in voxels in the brain and spinal cord. The
primary effect of the respiratory cycle on blipped-EPI fMRI data is
an apparent shift in image position in the phase-encode direction.
This is due to shifting of the resonant water frequency as the main
field drifts due to chest expansion and contraction [71, 72]. The
primary effect of the cardiac cycle is pulsatility artifact with each
heartbeat, although the structure and timing of the artifact may
vary across the brain due to the range of vessel sizes, stage in the
vessel network, and location in the brain [73, 74]. The cardiac
effects are more pronounced in certain regions such as the insula
and brainstem, while respiratory effects are more global (Fig. 21).

Fig 21 Averaged physiologic coupling maps in blipped-EPI with phase-encode direction in Anterior-Posterior
axis, determined by temporal ICA. Cardiac coupling overlain on anatomy is shown on top row, respiratory
coupling is shown on bottom row
 Optimal fMRI Pulse Sequences 97

Correcting for the effect of respiration can be accomplished

using navigator echoes or off-resonance detection to follow the
field shift during the scan, in the same way as described above for
gross head motion. Gating acquisitions on either cardiac or
­respiratory rates is another method that has been used to reduce
the variability in fMRI data [75]. This approach necessitates a cor-
rection for T1 effects introduced by the variable TR.
Use of parallel measures can be used to effectively remove
physiologic noise sufficient for most purposes in fMRI [76, 77].
Therefore, acquiring a pair of signals representing the cardiac cycle
and respiratory cycle during the scan is desirable, although recent
methods have been developed to estimate equivalent signals from
the echo-planar data itself [67].

5 Optimization of Sequence Parameters

In order to generate MRI data with functional contrast, there are

a number of experimental issues that need to be considered.
Generally speaking, if the issue were simply rapidly generating
images with BOLD contrast then the procedure would be to sim-
ply select sequence parameters such that the TR is a short as pos-
sible and the TE such that the expected changes in capillary or
venous oxygenation result in a maximal signal change between rest
and active neuronal state. However, the choice of optimal acquisi-
tion encompasses many other experimental issues and these should
all be considered.

5.1 Relaxation As described above, BOLD contrast in MRI is produced either

Parameters through changes in T2 or T2*. We are interested in optimizing MR
and Functional signal differences between two states, rest and active. In this sec-
Contrast tion, we discuss scanner and sequence issues that can affect the
detection of these two states.

5.1.1 Field Strength The effect of static field strength on T2 relaxation in brain tissue, as
and Relaxation Parameters evidenced from examination of Table 1, is generally that it is
reduced. The effect of field strength on the BOLD signal is com-
plex, and depends on the nature of the proton transport mecha-
nism in the presence of the field defects introduced by the
deoxygenated hemoglobin. Recent studies suggest that this mech-
anism is largely diffusive in nature at clinical field strengths, which
would suggest a linear dependence of the BOLD signal on field
strength. Experimental data bear out the linearity of the depen-
dence of BOLD contrast on field strength [78]. Thus, BOLD con-
trast from extravascular protons can be taken to increase
approximately linearly in the regime used by most commonly avail-
able MRI scanners (i.e., 0.3 T to 3.0 T).
The intravascular contribution to BOLD signal stems from the
impact of the change in oxygenated hemoglobin concentration
98 Mark J. Lowe and Erik B. Beall

within the vessels and the consequent change in T2 of the blood.

The effect of this at the voxel level has been more difficult to describe
than the extravascular effect due to the dependence on many factors
such as blood volume, vessel size, and volume fraction.

5.2 Experimental The goal in experimental design of fMRI studies is to take advan-
Design tage of the fact that regional changes in blood flow and oxygen-
ation result proximal to regions of increased neuronal activation.
Historically, the basic methodology has been to acquire properly
weighted MRI data of the brain regions of interest while a subject
repeatedly performs tasks related to the brain function of interest.
Initial methodology took advantage of the observation that con-
tinuously repeating a task during short intervals leads to an accu-
mulation of signal from the overlapping of events in a time short
compared to the hemodynamic response. This is a feature of the
general linear model (GLM) of functional imaging [79].
Blocking activation in bursts of extended activity over many
seconds, interleaved with long period of rest, leads to up to a much
higher increase in hemodynamic response than short, isolated
events. This fact makes it desirable, when possible, to use what is
typically referred to as a block design.
In 1997, Josephs and colleagues proposed an alternative
experimental design, intended to more specifically detect the MRI
signal associated with neuronal events [80]. This experimental
design takes advantage of the fact that, through synchronization of
the time of stimuli and measurement of behavioral responses, func-
tional imaging data can be analyzed for signal fluctuations corre-
lated with brief, temporally separated neuronal events. This type of
experimental design is referred to as event-related fMRI. Due to its
suitability to address more complex neuroscience questions regard-
ing brain activation and interactions, this has become a preferred
experimental design among neuroscience researchers.
Since these two experimental approaches have different analy-
sis strategies, the issues with regard to optimizing pulse sequences
are different between them. In the sections below, we separately
discuss these issues.

5.2.1 Block Design fMRI As stated above, block design fMRI experiments are designed to
create a large aggregate signal from activated neurons extended in
time, interspersed with long periods of rest, or alternate task per-
formance. Analysis of this type of data is typically performed with
what is referred to as a reference function. The simplest method for
analyzing these data is simply to calculate the cross correlation of
the experimental reference function with the timeseries at each
voxel [81]. Although more sophisticated methods have been
developed that allow more complex analyses, accounting for
­nuisance effects and systematic effects of no-interest, for purposes
of pulse sequence optimization, a correlation approach is sufficient
to illustrate the issues.
 Optimal fMRI Pulse Sequences 99

Fig 22 Example of fMRI timecourse from a voxel for a block-design experiment

Figure 22 shows an example of the signal time course from a

voxel in response to a block design paradigm. The issues with
regard to pulse sequence optimization are contrast-to-noise ratio
(CNR), sampling rate, and number of samples. In principle, the
TE and TR will control the CNR for a given pulse sequence (i.e.,
EPI, spiral, etc.). The sampling rate is the inverse of the TR. The
detection efficiency of a pulse sequence will be determined by these
and the number of samples. As an illustration of this, Fig. 23 shows
the probability of getting a type II error at a false positive rate of
0.01 as a function of the number of samples.

5.2.2 Event-Related fMRI Event-related fMRI relies on a different analysis strategy than block
design fMRI. The principal difference as it relates to choice of pulse
sequence is temporal resolution. A typical method for analyzing
event-related fMRI is deconvolution. Deconvolution is an analysis
method where rapidly repeated, although temporally separated,
events can be extracted if the timing of the onset of the signal and
either the duration or the hemodynamic response function is
known. A detailed discussion of deconvolution techniques is beyond
the scope of this chapter. The reader is referred to the chapter on
statistical analysis in this volume for a more complete treatment.
Figure 24 shows a typical timing and signal response for a rap-
idly presented event-related fMRI experiment. Studies on the abil-
ity of deconvolution techniques to resolve neuronal timing shifts
indicate that volume sampling rates (i.e., TR) of up to 3 s permit
identification of neuronal timing shifts of order 100 ms [82].
100 Mark J. Lowe and Erik B. Beall

Fig 23 Probability of rejecting a true event (type II error) as a function of the number of samples at false positive
rate of 0.01 for a two-cycle block design fMRI experiment. Result is from simulating image SNR = 50 with a
2 % BOLD signal change

a
1

0.5

0
0 20 40 60 80 100 120 140 160 180 200
Seconds

0.5 b
MR signal (a.u.)

-0.5

0 20 40 60 80 100 120 140 160 180 200

Seconds
Hemodynamic Response

1 c

0.5

0 5 10 15 20 25 30
Seconds

Fig 24 Example of event-related fMRI experiment. Rapid presented stimuli (a) results in single-voxel time-
series shown in (b). hemodynamic response (c) is deconvolved from signal average over 100 voxels with
temporal resolution 3 s. is shown in (c)
 Optimal fMRI Pulse Sequences 101

Therefore, if a goal of an experiment is to study relative timing of

events, TR’s of up to 3 s should be sufficient.
Another pulse sequence issue that should be of concern to
researchers employing event-related experimental designs is
SNR. As stated above, the CNR of event-related design is much
lower than block design experiments. Therefore, it is recom-
mended that researchers choose their acquisition strategy with this
in mind. For instance, if a local cortical region is of interest, a sur-
face coil array could be adopted to significantly increase SNR. Pulse
sequence choices should be made carefully to avoid loss of SNR
(shortest TE permissible for BOLD contrast, for example). In the
next section of this chapter, issues with regard to SNR for the com-
mon pulse sequence parameters will be discussed in detail.

6 Summary Recommendations for Optimal BOLD fMRI

In this section, we will summarize the issues with regard to pulse

sequence optimization in the performance of fMRI experiments. As
stated above, currently, BOLD-weighted fMRI is the overwhelming
method chosen for fMRI. For that reason, the recommendations
presented in this summary will focus on BOLD-weighted acquisi-
tions. In most cases, the technical discussions will be relevant for
other types of image weighting.
In the context of the information presented previously in this
chapter, we will present recommended acquisition strategies for a
given field strength and we will then expand on the optimization
issues with regard to each of the sequence parameters.
Table 2 lists basic pulse sequence recommendations for two field
strengths and two experimental design strategies. The recommenda-
tions in Table 2 should be possible in almost any recently installed
clinical MR scanner of the indicated field strength. Further recom-
mendations made below may require special modifications to sup-
plied clinical pulse sequences and it is strongly recommended that
researchers seek the input of an MRI physicist experienced in fMRI.

6.1 Pulse Sequence Various pulse sequences that have been proposed for fMRI acquisi-
tion were outlined in Sect. 2 of this chapter. Issues with regard to
optimization of fMRI experiments are discussed here.

6.1.1 Echoplanar This is now the most commonly available single shot imaging
Imaging sequence available on MRI scanners. Data can be acquired in GRE
mode and SE mode4. The issues with regard to optimization are

4
In addition, a mixed mode EPI has been used in the literature known as ASE,
or asymmetric spin echo. This is a SE EPI with the acquisition window shifted to
be centered on a time point early on in the SE evolution. The result is an acquisi-
tion that has, in a sense, adjustable sensitivity to capillary and venous signal.
102 Mark J. Lowe and Erik B. Beall

Table 2
Basic sequence parameters for BOLD fMRI acquisition on most clinical
MRI scanners

Field strength

Sequence parameter 1.5 T 3.0 T

Sequence GRE-EPI GE-EPI
Scan plane Axial Axial
FOV 256 mm × 256 mm 256 × 256
Matrix 64 × 64 128 × 128
TE 50 ms 30 ms
TR 2000 ms 2800 ms
Flip angle 77° 80°
Receiver bandwidth (total) 125 kHz 250 kHz
Slice thickness 7 mm 4 mm
Slices (for whole-brain coverage) 18 32

that SE EPI employs, by necessity, a longer TE that will result in a

smaller intravascular contribution, particularly at 3.0 T and higher,
and refocuses dephasing effects from the static dephasing compo-
nent of the extravascular signal that is specific to larger, distal ves-
sels. The result is that SE EPI can be more spatially specific to the
localization of neuronal activation. However, the SNR is much
lower than GRE EPI. SE EPI is not recommended for field
strengths below 3.0 T because the T2 of blood is not short enough
to be of benefit with regard to the intravascular BOLD signal and
the SNR is too low to employ high enough spatial resolution for
the spatial specificity to have a significant impact.

6.1.2 Spiral Imaging Spiral imaging is becoming more common and is available as a prod-
uct sequence on some clinical MRI scanners. As outlined above, the
major advantage of spiral imaging with respect to EPI is that it is less
demanding on the imaging gradients. Thus, there will be reduced
image warping from eddy current effects. In addition, the nonuni-
form sampling of the Fourier domain image will lead to a different,
possibly lower, sensitivity to motion effects and even some types of
physiologic noise. Variants of the spiral technique have been pro-
posed that are specifically designed to be more sensitive to the char-
acteristics of the BOLD signal. This sequence is recommended for
researchers employing systems with underpowered gradient systems
and for situations where motion and/or physiologic noise or other
 Optimal fMRI Pulse Sequences 103

types of image artifact, as discussed above, are a concern and alter-

nate methods of addressing these are not available.

6.1.3 Parallel Imaging MRI scanner manufacturers are increasingly moving to the use of
head array RF coils in lieu of circularly polarized quadrature head
RF coils for MRI. The cost, particularly at 1.5 T, is uniformity of
SNR, and thus fMRI signal detection efficiency across the brain.
This is less of an issue at high field strength, since dielectric effects
in this frequency regime reduce the uniformity of the quadrature
coil anyway. The advantage of head arrays is that parallel imaging
methods can be used to accelerate spatial encoding of images. The
result is dramatically increased image quality in regions where sin-
gle shot imaging methods have historically been very poor in qual-
ity (e.g., orbitofrontal regions, mesial temporal lobe, brainstem).
Some of these brain regions have important and interesting func-
tions. Parallel imaging techniques with the head arrays available to
most researchers will typically result in lower SNR throughout
most of the brain, but these can still be effectively employed in
situations where image artifact severely limits experimental options.

6.2 Scan Plane Issues with regard to scan plane are largely esthetic. However,
there are some technical issues that are worth discussing here.
Perhaps the most important issue is brain tissue coverage. The
scan plane of choice can affect the coverage of brain tissue. Given
a TE, receiver bandwidth, and TR, the number of slices available to
be acquired in one TR is fixed on a given scanner.5 The most effi-
cient scan plane for acquiring most human brains is the sagittal
plane. The brain in most adults is shortest in the right/left dimen-
sion, and so fewer slices will be required to cover the entire brain.
An axial acquisition plane is recommended in Table 2 due to
the fact that it is a more intuitive scan plane to work in, both ana-
tomically and from a physics perspective. Eddy current and higher
order artifacts stemming from gradient and shim coil interactions,
that are essentially related to coil geometry, are more easily under-
stood in the axial plane. With that said, it is a simple extension to
understand these effects in other scan planes. Axial imaging has the
added advantage over sagittal and coronal imaging planes in that
lateral, frontal, prefrontal, and posterior regions of the brain can be
imaged entirely within a relatively few slices (i.e., the very top and
very bottom of the brain are considered by many to be more
“expendable” than these other regions). The axial plane is a very
common imaging choice in fMRI and thus is listed in Table 2.

6.3 Field-of-View FOV has an impact on fMRI signal optimization in three ways: (1)
together with image matrix, it determines the in-plane voxel size
and there are a number of issues with regard to this that will be
5
There are, of course, other parameters that can affect this, such as gradient
slew rate, partial fourier and/or field-of-view acquisition, etc.
104 Mark J. Lowe and Erik B. Beall

outlined below, (2) image artifact reduction, particularly in the

phase direction, with volume RF coils, and (3) brain coverage.

6.3.1 In-Plane Voxel Size In-plane voxel size affects fMRI acquisition SNR (and thus fMRI
signal detection efficiency) and image quality. At lower field
strengths, the SNR issue will dominate and thus it is recommended
to use a larger voxel size at the expense of spatial resolution in
order to enhance signal detection efficiency. Further signal
enhancement can be attained with minimal loss of spatial resolu-
tion at 1.5 T through special spatial filtering techniques [83, 84].
At field strengths of 3.0 T and higher, voxel size has an inter-
action with physiologic noise from cardiac and respiratory sources
that can be detrimental to fMRI signal detection [64]. It is rec-
ommended that smaller voxels be employed at 3 T and higher to
limit the impact on BOLD CNR from physiologic noise. If spatial
resolution is not a concern, it is still recommended that data be
acquired at a higher spatial resolution (i.e., smaller voxel size)
and retrospective spatial filtering be employed to further increase
the CNR [84].

6.3.2 Image Artifact Due to the fact that the spatial encoding in the phase direction
Reduction (i.e., the direction encoded using, for instance, the phase blip-
ping described in Sect. 2.2 above) is not bandwidth limited, tis-
sue outside of the FOV in the phase direction that experiences
RF excitation, will be appear wrapped into the FOV with a signal
intensity related to the leakage RF experienced by that tissue.
For that reason, it is important for most acquisitions that the
field of view in the phase direction is adequate to contain the
entire brain volume and is oriented such that other tissue is not
proximal to the FOV. An example would be a coronal plane
acquisition with the phase encode direction in the inferior/supe-
rior direction. In this acquisition, even if the entire brain volume
is within the FOV, tissue from the neck and trunk of the body
that is within the sensitive volume of the transmit and receive RF
coils will appear aliased into the top of the FOV. A more com-
mon problem is that the field of view is chosen too small and one
side of the brain is wrapped into the other side of the brain (see
Fig. 25). This is avoided most simply by adopting a large enough
FOV in the phase direction. The recommendation in Table 2 is
sufficient for most adults.

6.4 Image Matrix As stated above, the principal impact of image matrix is on voxel
size and these issues are outlined above. However, it will also affect
the duration of the data acquistion for a single slice in single shot
imaging. This duration, as discussed in Sect. 3.2 above, can have a
detrimental effect on image quality in ultrafast imaging. Generally,
the total readout time should be much less than T2 or T2*. Typical
methods of decreasing the scan duration while maintaining good
 Optimal fMRI Pulse Sequences 105

Fig 25 Sagittal EPI image with phase direction too small for the brain dimension
in the anterior–posterior direction. The anterior portion is phase-wrapped into
the tissue at the posterior part of the brain

spatial resolution are partial Fourier and partial field-of-view tech-

niques, discussed briefly in Sect. 3.5. These are very commonly
employed and can result in acceptable SNR tradeoffs that allow
good image quality. For the 3.0 T acquisition recommended in
Table 2, it is typical to use a partial Fourier, or partial echo,
­acquisition strategy. These are recommended such that reasonable
SNR is maintained.

6.5 Echo Time (TE) The TE is probably the most important consideration in optimiz-
ing a pulse sequence for BOLD contrast (or any T2 or T2* contrast
for that matter). As discussed in Sect. 3.1, optimal BOLD contrast
is obtained by selecting a TE that is the average of the T2 or T2* of
the tissue in the active and inactive states. This will necessarily
depend on the tissue characteristics and the field strength. The TEs
recommended in Table 2 are typical echo times for a GRE-EPI
acquisition that have a reasonable balance between tissue sensitiv-
ity and specificity. Adopting a longer TE can result in less sensitiv-
ity to intravascular signal, especially at 3.0 T and higher, while a
shorter TE can result in higher SNR. Ranges of TE for T2* BOLD
imaging at 1.5 T include 40–65 ms, while ranges used experimen-
tally at 3.0 T can range from 25 to 40 ms. One should be careful
when adopting TEs outside of these ranges as BOLD contrast can
be severely attenuated.
106 Mark J. Lowe and Erik B. Beall

6.6 Repetition Time With regard to BOLD contrast, TR has the fairly simple effect of
(TR) increasing or lowering SNR based on the T1 of the tissue. For a TR
short with respect to the T1 of the tissue of interest, the MR signal
will be saturated. This will be discussed in some detail in the sec-
tion regarding the flip angle. Here, we will simply point out that
very short TRs can result a significant reduction in SNR, and can
subsequently also result in a significant contribution from flow
contrast, depending on the saturated state and whether a slice gap
is included in the prescription.
Increased blood flow results in an apparent shortening of the
T1 relaxation time due to the effect of infusing blood on the net
saturated state of the protons in a given voxel. Generally, the effect
of increased flow on the T1 of a given voxel can be expressed as:

1 1 f
= + (5.1)
T1eff T1 l

where T1eff is the observed T1 in the presence of flow. λ is the tissue

blood volume fraction and f is the rate of flow. Thus, an increase in
blood flow will result in an apparent shortening of the observed T1
in the affected brain region.
With regard to fMRI, the TR normally determines the sam-
pling rate. This, combined with experimental design (stimulus pre-
sentation, duration, number of samples, etc.) determines the
detection efficiency for BOLD signals (see also Sect. 4.2.1 above).
A note with regard to TR and fMRI is that sensitivity to out-­
of-­plane motion, discussed above in Sect. 3.2.1, is a consequence
of spin saturation in 2-dimensional single shot MRI. Longer TRs
will lead to lower saturation, and thus lower sensitivity to out-of-­
plane motion6.

6.7 Flip Angle Technically, the flip angle relates to the amount of RF power applied
at the excitation stage of a pulse sequence. For a given tissue type
(i.e., T1) and TE, MR signal is optimized at a flip angle referred to
as the Ernst angle. The formula for the Ernst angle is given by:

ìï æ -TR ö üï
a E = cos -1 íexp ç ÷ý (5.2)
îï è T1 ø þï

Since the flip angle controls the amount of RF power

transmitted to the tissue, reducing the flip angle in acquisitions
with a short TR can reduce the amount of flow contribution
observed in a BOLD-weighted acquisition, possibly increasing spa-
tial specificity of detected neuronal activation.

6
More accurately, retrospective motion correction techniques will be more
effective.
 Optimal fMRI Pulse Sequences 107

6.8 Receiver Receiver bandwidth (RBW), in a conventional MRI sequence,

Bandwidth has an easily interpreted impact on images: since the receiver
bandwidth is essentially the speed at which the MR signal is digi-
tized, lower bandwidth results in higher SNR, but the resulting
longer data acquisition can impact image quality. With single shot
imaging techniques, the effect of receiver bandwidth is not as
straightforward. At low RBW, the readout window length can be
long enough such that SNR is reduced. At higher RBW, the
reduced artifact from shorter readout time lessens or even com-
pletely eliminates the expected reduction in SNR. It is difficult to
recommend an exact RBW for these types of acquisitions, since
the optimal operating point will depend on scanner hardware
characteristics such as slew rate that is highly variable between
scanners. The parameters indicated in Table 2 should give reason-
able results in most clinical MRI scanners. Increasing RBW can
help to reduce susceptibility artifacts, such as image warping, in
orbitofrontal or other regions in much the same way as discussed
above under parallel imaging.

6.9 Slice Thickness Choice of slice thickness has generally the same effect as spatial res-
olution mentioned above. Principal effects are brain coverage, SNR,
and image quality. The recommended slice thickness in Table 2
should give nearly whole-brain coverage in most adults, with accept-
able SNR and image artifact given field strength limitations.
As a further note, there is no mention of a slice gap in Table 2.
A slice gap is not recommended in fMRI studies where the entire
brain is desired. The RF excitation for a given slice will not be
perfect, so if a small gap (<10 % of slice thickness) between slices
were permitted, the tissue in this gap would still be sampled,
although less than if the slices were simply made thicker.
Historically, slice gaps were included to improve SNR in 2D
acquisition by reducing RF crosstalk between adjacent slices. The
longer TR recommended in Table 2, along with an interleaved
style pattern of slice excitation, should be sufficient to make this a
negligible effect in most clinical scanners.

6.10 Number The desired number of slices in an fMRI acquisition will affect
of Slices temporal resolution and brain coverage. The recommended num-
ber of slices in Table 2 should permit whole brain coverage in most
situations. Issues with regard to TR are discussed above.

7 Concluding Remarks

As stated at the beginning, and illustrated throughout this chapter,

there are many experimental design issues in fMRI that will affect
the exact pulse sequence prescription adopted for a particular study.
The intent of this chapter is to give an overview of these issues and
108 Mark J. Lowe and Erik B. Beall

recommend a starting point for an sequence prescription that will be

generally feasible on most modern MRI scanners, along with a sense
of the impact of each of the sequence features. There are two caveats
with regard to the content of this chapter: (1) it is strongly recom-
mended that fMRI researchers work closely with an MR physicist
experienced in fMRI when there are specific issues that may affect
acquisition choices and (2) ideally, pulse sequence prescription and
paradigm design should both be considered together when design-
ing an fMRI experiment. Informing the data acquisition design
based on the needs with regard to the experimental hypothesis or
analysis methods is critical to a successful fMRI experiment.

References

1. Belliveau JW, Kennedy DN Jr, McKinstry RC 11. Pipe JG, Duerk JL (1995) Analytical resolution
et al (1991) Functional mapping of the human and noise characteristics of linearly reconstructed
visual cortex by magnetic resonance imaging. magnetic resonance data with arbitrary k-space
Science 254:716–719 sampling. Magn Reson Med 34:170–178
2. Ogawa S, Lee TM, Nayak AS, Glynn P (1990) 12. Bornert P, Schomberg H, Aldefeld B, Groen
Oxygenation-sensitive contrast in magnetic J (1999) Improvements in spiral MR imaging.
resonance image of rodent brain at high Magma 9:29–41
magnetic fields. Magn Reson Med 14:68–78 13. Glover GH, Law CS (2001) Spiral-in/out
3. Bandettini PA, Wong EC, Hinks RS, Tikofsky BOLD fMRI for increased SNR and reduced
RS, Hyde JS (1992) Time course EPI of susceptibility artifacts. Magn Reson Med
human brain function during task activation. 46:515–522
Magn Reson Med 25:390–397 14. Preston AR, Thomason ME, Ochsner KN,
4. Kwong KK, Belliveau JW, Chesler DA et al (1992) Cooper JC, Glover GH (2004) Comparison of
Dynamic magnetic resonance imaging of human spiral-in/out and spiral-out BOLD fMRI at
brain activity during primary sensory stimulation. 1.5 and 3 T. Neuroimage 21:291–301
Proc Natl Acad Sci U S A 89:5675–5679 15. Sodickson DK, Manning WJ (1997)
5. Ogawa S, Tank DW, Menon R et al (1992) Simultaneous acquisition of spatial harmonics
Intrinsic signal changes accompanying sensory (SMASH): fast imaging with radiofrequency
stimulation: functional brain mapping with coil arrays. Magn Reson Med 38:591–603
magnetic resonance imaging. Proc Natl Acad 16. Pruessmann KP, Weiger M, Scheidegger MB,
Sci U S A 89:5951–5955 Boesiger P (1999) SENSE: sensitivity encoding
6. Frahm J, Bruhn H, Merboldt KD, Hanicke W for fast MRI. Magn Reson Med 42:952–962
(1992) Dynamic MR imaging of human brain 17. Griswold MA, Jakob PM, Heidemann RM
oxygenation during rest and photic stimula- et al (2002) Generalized autocalibrating par-
tion. J Magn Reson Imaging 2:501–505 tially parallel acquisitions (GRAPPA). Magn
7. Weisskoff RM, Davis TL (1992) Correcting Reson Med 47:1202–1210
gross distortion on echo planar images, Society 18. Heidemann RM, Griswold MA, Kiefer B
of Magnetic Resonance in Medicine 11th et al (2003) Resolution enhancement in lung
annual meeting, Berlin 1H imaging using parallel imaging methods.
8. Foerster BU, Tomasi D, Caparelli EC (2005) Magn Reson Med 49:391–394
Magnetic field shift due to mechanical vibra- 19. Ohliger MA, Grant AK, Sodickson DK (2003)
tion in functional magnetic resonance imag- Ultimate intrinsic signal-to-noise ratio for
ing. Magn Reson Med 54:1261–1267 parallel MRI: electromagnetic field consider-
9. Ahn CB, Kim JH, Cho ZH (1986) High- ations. Magn Reson Med 50:1018–1030
speed spiral-scan echo planar NMR imaging- 20. Wiesinger F, Boesiger P, Pruessmann KP
I. IEEE Trans Med Imaging 5:2–7 (2004) Electrodynamics and ultimate SNR
10. Bruder H, Fischer H, Reinfelder HE, Schmitt in parallel MR imaging. Magn Reson Med
F (1992) Image reconstruction for echo planar 52:376–390
imaging with nonequidistant k-space sam- 21. Blaimer M, Breuer F, Mueller M, Heidemann
pling. Magn Reson Med 23:311–323 RM, Griswold MA, Jakob PM (2004) SMASH,
 Optimal fMRI Pulse Sequences 109

SENSE, PILS, GRAPPA: how to choose the 36. Jesmanowicz A, Bandettini PA, Hyde JS (1998)
optimal method. Top Magn Reson Imaging Single-shot half k-space high-­resolution gradi-
15:223–236 ent-recalled EPI for fMRI at 3 Tesla. Magn
22. Ohliger MA, Sodickson DK (2006) An Reson Med 40:754–762
introduction to coil array design for parallel 37. Hyde JS, Biswal BB, Jesmanowicz A (2001)
MRI. NMR Biomed 19:300–315 High-resolution fMRI using multislice par-
23. Pruessmann KP, Weiger M, Bornert P, Boesiger tial k-space GR-EPI with cubic voxels. Magn
P (2001) Advances in sensitivity encoding with Reson Med 46:114–125
arbitrary k-space trajectories. Magn Reson 38. Moeller S, Yacoub E, Olman CA et al (2010)
Med 46:638–651 Multiband multislice GE-EPI at 7 tesla, with
24. Weiger M, Pruessmann KP, Osterbauer R, 16-fold acceleration using partial parallel
Bornert P, Boesiger P, Jezzard P (2002) imaging with application to high spatial and
Sensitivity-encoded single-shot spiral imaging temporal whole-brain fMRI. Magn Reson
for reduced susceptibility artifacts in BOLD Med 63:1144–1153
fMRI. Magn Reson Med 48:860–866 39. Setsompop K, Gagoski BA, Polimeni JR,
25. Heidemann RM, Griswold MA, Seiberlich Witzel T, Wedeen VJ, Wald LL (2012)
N et al (2006) Direct parallel image recon- Blipped-­controlled aliasing in parallel imag-
structions for spiral trajectories using ing for simultaneous multislice echo planar
GRAPPA. Magn Reson Med 56:317–326 imaging with reduced g-factor penalty. Magn
26. Griswold MA, Kannengiesser S, Heidemann Reson Med 67:1210–1224
RM, Wang J, Jakob PM (2004) Field-of-view 40. Cauley SF, Polimeni JR, Bhat H, Wald LL,
limitations in parallel imaging. Magn Reson (2014) Setsompop K Interslice leakage arti-
Med 52:1118–1126 fact reduction technique for simultaneous
27. Griswold MA, Breuer F, Blaimer M et al multislice acquisitions. Magn Reson Med
(2006) Autocalibrated coil sensitivity esti- 72:93–102
mation for parallel imaging. NMR Biomed 41. Xu J, Moeller S, Auerbach EJ, et al. (2013)
19:316–324 Evaluation of slice accelerations using multi-
28. Sodickson DK (2000) Tailored SMASH image band echo planar imaging at 3 T. Neuroimage
reconstructions for robust in vivo parallel MR 83:991–1001
imaging. Magn Reson Med 44:243–251 42. Ugurbil K, Xu J, Auerbach EJ, et al. (2013)
29. Sanchez-Gonzalez J, Tsao J, Dydak U, Desco Pushing spatial and temporal resolution for
M, Boesiger P, Paul PK (2006) Minimum-­ functional and diffusion MRI in the Human
norm reconstruction for sensitivity-encoded Connectome Project. Neuroimage 80:80–104
magnetic resonance spectroscopic imaging. 43. Jo HJ, Saad ZS, Simmons WK, Milbury LA,
Magn Reson Med 55:287–295 Cox RW (2010) Mapping sources of correla-
30. Lin FH, Kwong KK, Belliveau JW, Wald LL (2004) tion in resting state FMRI, with artifact detec-
Parallel imaging reconstruction using automatic tion and removal. Neuroimage 52: 571–582
regularization. Magn Reson Med 51:559–567 44. Meyer CH, Pauly JM, Macovski A, Nishimura
31. Block KT, Frahm J (2005) Spiral imaging: DG (1990) Simultaneous spatial and spec-
a critical appraisal. J Magn Reson Imaging tral selective excitation. Magn Reson Med
21:657–668 15:287–304
32. Tsao J, Boesiger P, Pruessmann KP (2003) k-t 45. Zhou XJ, Du YP, Bernstein MA, Reynolds
BLAST and k-t SENSE: dynamic MRI with HG, Maier JK, Polzin JA (1998) Concomitant
high frame rate exploiting spatiotemporal cor- magnetic-field-induced artifacts in axial
relations. Magn Reson Med 50:1031–1042 echo planar imaging. Magn Reson Med
39:596–605
33. Huang F, Akao J, Vijayakumar S, Duensing
GR, Limkeman M (2005) k-t GRAPPA: a 46. Reeder SB, Atalar E, Faranesh AZ, McVeigh
k-space implementation for dynamic MRI ER (1999) Referenceless interleaved echo-­
with high reduction factor. Magn Reson Med planar imaging. Magn Reson Med 41:87–94
54:1172–1184 47. Duyn JH, Yang Y, Frank JA, van der Veen JW
34. Cuppen JJ, Groen JP, Konijn J (1986) (1998) Simple correction method for k-space
Magnetic resonance fast Fourier imaging. Med trajectory deviations in MRI. J Magn Reson
Phys 13:248–253 132:150–153
35. Noll DC, Nishimura DG, Macovski A (1991) 48. Yudilevich E, Stark H (1987) Spiral sampling
Homodyne detection in magnetic reso- in magnetic resonance imaging-the effect of
nance imaging. IEEE Trans Med Imaging inhomogeneities. IEEE Trans Med Imaging
10:154–163 6:337–345
110 Mark J. Lowe and Erik B. Beall

49. Jezzard P, Balaban RS (1995) Correction for 62. Jezzard P, LeBihan D, Cuenod D, Pannier L,
geometric distortion in echo planar images Prinster A, Turner R (1992) An investigation
from B0 field variations. Magn Reson Med of the contribution of physiological noise in
34:65–73 human functional MRI studies at 1.5 Tesla
50. Blamire AM, Rothman DL, Nixon T (1996) and 4 Tesla, Society of Magnetic Resonance in
Dynamic shim updating: a new approach Medicine 12th annual meeting, New York
towards optimized whole brain shimming. 63. Lowe MJ, Mock BJ, Sorenson JA (1998)
Magn Reson Med 36:159–165 Functional connectivity in single and multislice
51. Wilson JL, Jenkinson M, de Araujo I, echoplanar imaging using resting-state fluctua-
Kringelbach ML, Rolls ET, Jezzard P (2002) tions. Neuroimage 7:119–132
Fast, fully automated global and local mag- 64. Triantafyllou C, Hoge RD, Krueger G et al
netic field optimization for fMRI of the human (2005) Comparison of physiological noise at
brain. Neuroimage 17:967–976 1.5 T, 3 T and 7 T and optimization of fMRI
52. Ward HA, Riederer SJ, Jack CR Jr (2002) Real- acquisition parameters. Neuroimage 26:243–250
time autoshimming for echo planar timecourse 65. Friedman L, Glover GH (2006) Report on a
imaging. Magn Reson Med 48:771–780 multicenter fMRI quality assurance protocol.
53. Yang QX, Williams GD, Demeure RJ, Mosher J Magn Reson Imaging 23:827–839
TJ, Smith MB (1998) Removal of local field 66. Beall EB, Lowe MJ (2014) SimPACE: gener-
gradient artifacts in T2*-weighted images ating simulated motion corrupted BOLD data
at high fields by gradient-echo slice exci- with synthetic-navigated acquisition for the
tation profile imaging. Magn Reson Med development and evaluation of SLOMOCO:
39:402–409 a new, highly effective slicewise motion correc-
54. Constable RT, Spencer DD (1999) Composite tion. Neuroimage 101:21–34
image formation in z-shimmed functional MR 67. Beall EB, Lowe MJ (2007) Isolating physi-
imaging. Magn Reson Med 42:110–117 ologic noise sources with independently
55. Chen N, Wyrwicz AM (1999) Removal of determined spatial measures. Neuroimage
intravoxel dephasing artifact in gradient-echo 37:1286–1300
images using a field-map based RF refocusing 68. Fu ZW, Wang Y, Grimm RC et al (1995)
technique. Magn Reson Med 42:807–812 Orbital navigator echoes for motion measure-
56. Oesterle C, Markl M, Strecker R, Kraemer ments in magnetic resonance imaging. Magn
FM, Hennig J (1999) Spiral reconstruction Reson Med 34:746–753
by regridding to a large rectilinear matrix: a 69. Lee CC, Jack CR Jr, Grimm RC et al (1996)
practical solution for routine systems. J Magn Real-time adaptive motion correction in func-
Reson Imaging 10:84–92 tional MRI. Magn Reson Med 36:436–444
57. Moriguchi H, Duerk JL (2001) Modified 70. Thesen S, Heid O, Mueller E, Schad LR (2000)
block uniform resampling (BURS) algorithm Prospective acquisition correction for head
using truncated singular value decomposition: motion with image-based tracking for real-
fast accurate gridding with noise and artifact time fMRI. Magn Reson Med 44:457–465
reduction. Magn Reson Med 46:1189–1201 71. Zhao X, Bodurka J, Jesmanowicz A, Li SJ
58. Nehrke K, Bornert P (2005) Prospective (2000) B(0)-fluctuation-induced temporal
correction of affine motion for arbitrary MR variation in EPI image series due to the dis-
sequences on a clinical scanner. Magn Reson turbance of steady-state free precession. Magn
Med 54:1130–1138 Reson Med 44:758–765
59. Hajnal JV, Myers R, Oatridge A, Schwieso 72. Raj D, Anderson AW, Gore JC (2001)
JE, Young IR, Bydder GM (1994) Artifacts Respiratory effects in human functional mag-
due to stimulus correlated motion in func- netic resonance imaging due to bulk suscepti-
tional imaging of the brain. Magn Reson Med bility changes. Phys Med Biol 46:3331–3340
31:283–291 73. Dagli MS, Ingeholm JE, Haxby JV (1999)
60. Friston KJ, Williams S, Howard R, Frackowiak Localization of cardiac-induced signal change
RS, Turner R (1996) Movement-related in fMRI. Neuroimage 9:407–415
effects in fMRI time-series. Magn Reson Med 74. Bhattacharyya PK, Lowe MJ (2004) Cardiac-­
35:346–355 induced physiologic noise in tissue is a direct
61. Bullmore ET, Brammer MJ, Rabe-Hesketh S observation of cardiac-induced fluctuations.
et al (1999) Methods for diagnosis and treat- Magn Reson Imaging 22:9–13
ment of stimulus-correlated motion in generic 75. Guimaraes AR, Melcher JR, Talavage TM et al
brain activation studies using fMRI. Hum (1998) Imaging subcortical auditory activity
Brain Mapp 7:38–48 in humans. Hum Brain Mapp 6:33–41
 Optimal fMRI Pulse Sequences 111

76. Hu X, Le TH, Parrish T, Erhard P (1995) 81. Bandettini PA, Jesmanowicz A, Wong EC,
Retrospective estimation and correction of Hyde JS (1993) Processing strategies for
physiological fluctuation in functional time-­course data sets in functional MRI of
MRI. Magn Reson Med 34:201–212 the human brain. Magn Reson Med
77. Glover GH, Li TQ, Ress D (2000) Image-­ 30:161–173
based method for retrospective correction of 82. Miezin FM, Maccotta L, Ollinger JM, Petersen
physiological motion effects in fMRI: SE, Buckner RL (2000) Characterizing the
RETROICOR. Magn Reson Med 44:162–167 hemodynamic response: effects of presentation
78. Stefanovic B, Pike GB (2004) Human whole-­ rate, sampling procedure, and the possibility of
blood relaxometry at 1.5 T: assessment of dif- ordering brain activity based on relative tim-
fusion and exchange models. Magn Reson ing. Neuroimage 11:735–759
Med 52:716–723 83. Lowe MJ, Sorenson JA (1997) Spatially filter-
79. Friston KJ, Holmes AP, Worsley KJ, Poline J-B, ing functional magnetic resonance imaging
Frith CD, Frackowiak R (1995) Statistical para- data. Magn Reson Med 37:723–729
metric mapping in functional imaging: a general 84. Triantafyllou C, Hoge RD, Wald LL (2006)
linear approach. Hum Brain Mapp 2:189–210 Effect of spatial smoothing on physiological
80. Josephs O, Turner R, Friston KJ (1997) Event-­ noise in high-resolution fMRI. Neuroimage
related fMRI. Hum Brain Mapp 5:243–248 32:551–557
 Chapter 4

High-Field fMRI
Alayar Kangarlu

Abstract
Magnetic resonance imaging (MRI) allows detection of signal from constituent of biological tissues.
Hydrogen (1H) is the most widely used element from which spectra and images are detected due to its
abundance and high sensitivity manifested in its gyromagnetic ratio. The high contrast for soft tissue have
afforded scientists invaluable information about brain structure and function. Among many parameters
determining quality of MRI images, field strength is the most decisive one as it determines signal strength
in fMRI images. Considering the low inherent sensitivity of fMRI, high magnetic field are the only way
that activation contrast of neurofunctional studies could be increased. This is why there has been a relent-
less drive towards higher field strength in human imaging raising it up to 11.7 T to date. Technology of
7-T has become more widely available in scanners with fMRI capability. Development of many technolo-
gies such as multichannel RF coils, strong and fast gradients, simultaneous slice excitation, and brain-
stimulation protocols have contributed to the expansion of fMRI as the method of choice for study of
whole brain function. In this chapter, challenges of high-field fMRI in human studies are discussed among
which signal to noise, susceptibility artifacts, multichannel RF coil designs are highlighted.

Key words High field, fMRI, Neuroimaging, Magnetic field, High resolution

1 Introduction

The high water content of biological tissues makes acquisition of

anatomically accurate images of biological tissues possible [1, 2].
Imaging of structures are hinged on the contrast based on relax-
ation rates that are sensitive to the composition of tissues. The
same mechanism is also used to visualize changes in signal as a
function of physiology [3]. These facts have made fMRI a power-
ful tool for the study of neuroscience as it detects the changes in
signal during the brain activities in response to specific stimulation
designed to activate specific regions of the brain [4–10]. The
mechanism based on which brain function is detected by fMRI
depends on changes in the brain of magnetization caused by exter-
nal stimulations of neurons. This process is successful only if
responses to external cue stimulate enough neurons in the same

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_4, © Springer Science+Business Media New York 2016

113
114 Alayar Kangarlu

region. Field strength of fMRI determines its sensitivity to the

response of neuronal clusters. This makes differential response to
the external stimulus a complex entity that can better visualize
brain function. MR signal of brain activation is valid if it stems
from a change caused by neuronal activity. Simultaneous detection
of direct neuronal currents in the human brain has not yet been
reported due to the sheer number of neurons involved and a lack
of sensitivity to the neuron currents. However, neuronal activity
produces a change in magnetic susceptibility of hemodynamics
that modifies the magnetic field in the brain that can be detected as
change in MRI signal [3]. Although, the mechanism that connects
neuronal activity to hemodynamic response is still not well under-
stood [11], the correlation of stimulus pattern with hemodynamic
signals is well established in fMRI. This correlation has been
observed for sensory, motor, and cognitive paradigms. These fea-
tures have turned fMRI into a reliable tool for neuroscience and
neurology research and with the improvement in MRI hardware
and pulse sequences it is rapidly penetrating into psychiatry, neuro-
surgery, and psychology too.
Brain function can be noninvasively detected if the changes
caused by its activity produce electromagnetic signals strong enough
for high spatial localization and temporal resolution. Unlike inva-
sive techniques that operate at the cellular or single neuronal level,
whole brain access visualization with sensitivity for functional
response is required to simultaneously detect all activated regions.
The distinct advantage of fMRI is in its ability to acquire functional
information from regions with vastly different anatomical geometry
such as cortical regions and skull based brain tissues. Presently, only
MRI can noninvasively access brain function and can be repeatedly
applied on the same subject in multiple studies. The capability of
fMRI to access different brain regions responding to a specific stim-
ulation while simultaneously imaging the location of the functional
regions makes it indispensible for the studies of brain normal func-
tion and dysfunction. But, we must keep in mind that fMRI signal
is not a direct measure of neuronal activity. Computations for neu-
ronal currents (nc) MRI has predicted a few part per billion
(2–5 ppb) disturbance in MRI signal that is below noise floor. Such
estimations demonstrate challenges involved in making of ncMRI a
reality. Sensitivity of MRI to paramagnetic entities, however, brings
hemodynamic and its coupling to neuronal activity to rescue [3,
11]. Diamagnetic nature of biological tissues makes blood with its
rich iron content an ideal medium for the detection of physiological
changes. The blood oxygen level dependent (BOLD) is an effect
that measures changes in MR signal from deoxygenated hemoglo-
bin (dHb) to oxygenated hemoglobin (O2Hb) required by neuro-
nal activation which modifies the magnetic field around the regions
of oxygenation to the extent that changes in MR signal-to-noise
ratio (SNR) can be measured in a comparative measurement. In
 High-Field fMRI 115

fMRI studies, this change in signal (∆R/R) is taken as accurately

representing neuronal activity. Higher magnetic susceptibility (χ) of
dHb compared to that of O2Hb is enough at fields above 1 Tesla
(T) to raise ∆R/R to about 1 %/T which with modern instrumen-
tation is detectable. Dependence of BOLD strength on the static
magnetic field (B0) is a valuable feature of fMRI, which presently is
benefiting from availability of 7.0 T whole body magnets for the
study on human subjects.
The BOLD effect, however, depends on a number of physiologi-
cal factors. The primary ones being cerebral blood flow (CBF), cere-
bral blood volume (CBV), and cerebral metabolic rate of oxygen
(CMRO2). BOLD-based fMRI studies consider CBF, CBV, and
CMRO2 mechanisms as being induced by changes in neuronal activ-
ities. To more effectively use fMRI in neuroscience research, the neu-
rovascular coupling that relates neuronal activities to hemodynamics
(BOLD) must be understood. As high magnetic field enhances
BOLD signal it will play a vital role in determining the nature of MR
signature of neuronal activities. The use of BOLD in study of diseases
such as multiple sclerosis will become more widespread as evidence
for involvement of gray matter in this disease becomes more avail-
able. Assessing the specifics of the cortical damage with fMRI depends
on the ability to establish reliable correlation with specific physical
and cognitive disability that needs high sensitivity and specificity to
brain physiology. High-field fMRI could help with establishing an
association of cortical activity with clinical relapses.

2 MR Signal

Strong magnets exert a torque on small magnets like protons. Such

torque puts the spinning proton into a precession with a specific
frequency called Larmor frequency. If an electromagnetic waves
with the exact frequency as proton’s Larmor frequency, usually in
radio frequency (RF) range, is aimed at such proton, its energy will
be absorbed to excite the proton from its ground state to an excited
state [12, 13]. Magnets used in MRI scanners induce a resonance
frequency in about 100 MHz range (108 Hz). At 3 T, for example,
where proton Larmor frequency is 128 MHz, an RF wave with this
exact frequency will be able to transfer its energy into the protons
causing them to deflect from alignment with B0. Following this
disturbance, proton magnetic dipole moment (µ) will return to its
equilibrium position emitting an RF wave that will be picked up by
the RF coil [14, 15]. The RF magnetic field (B1) that is induced
into the coil circuit is mixed with signal from other events in the
body that produce similar signals. The collective effect at a protons
of µ magnetic moment at a frequency of ω is detected by the RF
coil. The RF coil is trusted with the task of detecting the narrow
frequency bandwidth that is created by the resonance condition.
116 Alayar Kangarlu

The design of the RF coils, hence, is a rather critical matter and it is

amply covered in the literature. While RF coils must operate at a
narrow frequency range, they should be capable of operating in a
very wide power range since kW RF power is required to excite the
sample while a few milliseconds after excitation the coil should be
able to detect a signal 1000’s times weaker than the transmitted
power. This remarkable resilience is constructed into a device that
provides coverage to the entire head and accurately records the
response of every cell that will get unraveled into images by com-
puter programs of reconstruction routines.
Relaxation values govern the time course of the signal decay.
So while the population of the protons (µ) aligned with B0 is a
function of B0, the signal will only persist within a time course
comparable to spin–lattice (T1) relaxation and spin–spin (T2) relax-
ation [16]. Images can be produced where the tissue intensity rep-
resents relative T1 values, the T1-weighted (T1W) contrast during
which the T2 relaxation must be kept at minimum. T2 relaxation is
a process of loss of coherence among aligned protons, while T1 is
the time during which excited dipoles return to their original ori-
entation where they are unable to contribute to the signal.
Considering a typical T1 value of 1 s for brain tissues, a whole head
image with 256 × 256 matrix takes about 5 min to acquire with no
acceleration factors applied. The relatively long T1 values set the
acquisition time, as realignment of protons occur with that time
scale. Acceleration of image acquisition is possible by simultaneous
acquisition of multiple k-space lines for each excitation. In addition
to T1, spin–spin relaxation or T2 decay, is also a mechanism that
slows down MR image acquisition. Due to the inherent insensitiv-
ity of MRI, the population of magnetic moment required to pro-
duce detectable signal within a voxel is the difference (ΔN) between
the parallel protons (N+) and anti-parallel protons (N-) which is
relatively large. The sum of µ’s (ΔNµ) within a voxel, i.e., magne-
tization vector or M determines the size of the signal. High mag-
netic field increases ΔN and through that the MR signal.
Consequently, high fields can produce detectable signal from
smaller voxels producing higher resolution images. As fMRI uses a
fast imaging sequence, echo planar imaging (EPI), with T2* con-
trast it has high sensitivity to magnetic susceptibility. T2* depends
on the sum of two mechanisms causing signal decay, i.e., spin–spin
and local B0 inhomogeneities that accelerate the loss of coherent
precession of M over time [16, 17]. The contrast-to-noise ratio
(CNR) of EPI-based BOLD signal used in fMRI depends on T2*
changes caused by the difference in magnetic susceptibility of oxy-
and deoxyhemoglobin. Since T2* is much shorter at high fields,
high-field fMRI more accurately represents local magnetic field
inhomogeneities caused by BOLD. As smaller voxels can be imaged
with high-field fMRI, faster and stronger gradients are being
designed and used to also increase temporal resolution to produce
information more directly related to neuronal activity [8, 9].
 High-Field fMRI 117

3 B0 Effects

Signal strength in MRI depends on B0, RF coil design, and relaxation

values. B0, however, determines excess proton population that sets
the limit of MR signal detectable by any magnet. In addition, the high
magnetic susceptibility endows high field with dual advantages of high
SNR and high susceptibility contrast. This fact is best at display with
images acquired from 7 T scanners (Fig. 1). Among the parameters
affecting image quality, B0 is the single parameter whose effect on
SNR and BOLD will expand the use of fMRI in assessment of physi-
ological signatures of neurological disorders. It should be mentioned
that dependence of MR signal on relaxations, susceptibility, CNR, and
hardware creates both advantages and disadvantages at high field.
Susceptibility artifacts in the regions with large cavities make the
choice of premium field strength for fMRI studies a nontrivial matter.
Susceptibility-based contrast can be used to image brain microstruc-
ture and to detect high brain iron as it has been suspected to play a
role in many neurodegenerative diseases. However, the challenges

Fig. 1 A 7 T FLAIR* image of an MS patient vs. a vascular patient, in which a

central vessel running through the MS lesion is clearly visible while absent
through the vascular lesion. Courtesy of Prof. I.D.. Kilsdonk, VUMC, the Netherlands
118 Alayar Kangarlu

involved in dealing the susceptibility artifacts of high-field fMRI has

prevented its widespread use. Clever techniques such as high-order
shimming, parallel transmit and receive (PTX), and short TE sequences
could reduce the magnetic susceptibility artifacts at high field. These
developments hold the promise to suppress the negative effects of
high field and turn blights into blessings. Success in this front has far
reaching implications on fMRI and its application in neurodegenera-
tive diseases.

4 Relaxation Effects

In addition to high SNR, relaxation effects make 7.0 T an attractive

field strength compared to 3.0 T and 1.5 T for fMRI studies (Fig. 2).
For anatomical images, typical voxel size at 1.5 T is about 5 mm3,
high SNR at 7.0 T allows image from biological tissues with 1-mm3
resolution in less than 10 min. The MR signal, however, is a function
of the relaxation values, which will reduce the time that the magne-
tization vectors are available for sampling in transverse plane. High-
field effects of susceptibility artifacts and BOLD effect make fMRI
resolutions and its CNR a complex factor for researchers to opti-
mize. While fMRI requires phase encoding an entire volume in one

1.08 7T
3T
normalised signal intensity

1.06 1.5 T

1.04

1.02

0.98
0 50 100 150
time (s)

Fig. 2 fMRI study at 1.5, 3.0, and 7.0 T performed on Philips Achieva. Courtesy of University of Nottingham, UK
 High-Field fMRI 119

shot as in EPI, the artifacts of phase encoding gradients cannot be

sufficiently refocused at high field. The high temporal resolution of
single-shot EPI might have to be forsaken at high field because the
spatial resolution and image quality suffer in single-shot EPI-at
high-field fMRI. The geometric distortions caused by the phase
encoding of single-shot EPI makes accurate coregistration of func-
tional maps with anatomical images more complex than in lower
fields. Furthermore, the point-spread functions (PSF) are broadened
due to the long acquisition window during which higher signal
decay occurs. One parameter that plays a critical role in determining
many of these quantities is TE and its selection is critical in high
field. But, the optimal TE is not always possible to find when large
in-plane matrices are chosen to reach higher resolutions in fMRI
studies. Variations of T1 values with field strength are also important.
T1 values reported for gray matter (GM) at 7.0 T are about 2 s and
for white matter values around 1.3 s [17–20]. Similar measurements
at 3.0 T, have found T1 values of about 1.5 s for GM and about 0.8 s
for white matter [18–20]. T1 values at 1.5 T reduce to about 1.2 s
for GM and about 0.6 s for white matter [21]. Due to a need for fast
encoding of the entire images in a time comparable to T1 value, its
absolute value does not have a large effect on fMRI images. However,
T2* relaxation has a direct effect on the BOLD activation signal. This
is especially true due to drastic change in T2* as a function of field
strength. A multi-field measurement has reported [17] T2* for GM,
WM, and putamen, respectively, to be 84.0, 66.2, and 55.5 ms at
1.5 T; 66.0, 53.2, and 31.5 ms at 3.0 T; and 33.2, 26.8., and
16.1 ms at 7.0 T. This shows that the difference between T2* of
GM/WM has reduced from 18 ms at 1.5 T to 7 ms at 7.0 T. These
results show a drop by a factor of about 2.5 in T2* of these tissues for
a field increase from 1.5 to 7.0 T. Such changes have great implica-
tions on the outcomes of fast sequences at high field. For example,
as T2* of some tissues of putamen drops to a value close to 10 ms, it
will reduce the possibility of phase encoding the structure in a single-
shot image for high-resolution images. This means that role of high
field in producing high-resolution functional images must be further
investigated.
Another challenge for high-field fMRI to exploit its advantages
is the need for stronger gradients. Higher gradient amplitudes and
faster switching rates can produce effects that can better be utilized
at high fields. The new strong gradients of 80 mT/m that have
become available on some 3 T and 7 T scanners will reduce the
encoding time for a resolution of 1 mm to be 1 ms/line. This makes
the total encoding time for 192 phase encoding steps to be 192 ms.
The T2* of the human brain at 7 T is about 30 ms while T2 of GM
and WM are 93 ms and 76 ms, respectively. Thus, readouts of about
100 ms might be needed for SE EPI or partial k-space filling in GE
EPI. Stronger gradients are going to be useful in reading a signal
that lasts 30 ms. The fast decay induced by high magnetic
120 Alayar Kangarlu

susceptibility during readout causes strong blurring of images that

must be suppressed before the advantages of EPI at high field could
be exploited. More robust gradients are becoming available that are
improving sensitivity and specificity of BOLD fMRI at high field.
This will enable the high-field BOLD fMRI to more accurately
localize and coregister with the high-resolution anatomical images
at 7 T. BOLD sensitivity does increase at higher fields, which can
only be fully utilized when their high-resolution maps are produced
from the entire brain including tissues in the near proximity of the
skullbase and brain regions near air/tissue boundaries.

5 Imaging the Brain Function

The best way to image brain function would be to directly detect neu-
ronal activities. Absence of such a technique has provided an opportu-
nity for indirect observation of brain function through BOLD, which
can measure changes in hemodynamics as a result of controlled neu-
ronal activation. The coupling of neuronal activity and vascular hemo-
dynamics makes BOLD dependents on the details of communication
between neurons, glia, and blood vessels. Furthermore, BOLD is an
indication of the existence of a tight level of vascular reactivity between
neuronal network and vascular system.
Accurate fMRI representation of the brain function depends
on the understanding of mechanism of neurovascular coupling,
i.e., how neuronal activity affects hemodynamic response and its
MR signal. Such relationship will enable us to account for pharma-
cological or disease-induced modulations of neurovascular cou-
pling that could use BOLD signal changes, or perfusion, for the
assessment of drug efficacy and pathological modification of physi-
ology. In addition, fMRI has become an important tool in studies
in psychology, psychiatry research, and basic cognitive neurosci-
ence research [11]. This is primarily due to the fact that fMRI has
proven to be able to provide a veritable readout of mental
contents.
Organization of the brain allows the functional studies to iden-
tify the neuronal basis of behavior, or at least the hemodynamic
manifestation of that. The measure of neuronal activity is obtained
by constructing activity maps from the functional units involved in
various brain networks [4, 22, 23]. The functional units, commonly
called cortical columns are made up of neuronal networks involved
in the implementation of a specific function. They form an orga-
nized structure that interacts with other units of the system that
repeatedly occur in the cortex. This organized structure and its
columnar activation contribute to elucidate the function of specific
cortical areas [24]. Imaging with a resolution allowing to detect the
simultaneous activation of these units, i.e., the collective response of
all columns involved in a specific external stimulation, would greatly
 High-Field fMRI 121

enhance the credibility of fMRI studies. This is due to the fact that
such resolution can establish the spatial localization of functional
units. Resting-state fMRI, which can detect spatially dispersed but
functionally connected regions that share information with each
other, offers information on functional connectivity (FC) of the
brain. Such FC maps are best utilized at high field, when they are
capable of offering the temporal dependency of neuronal activation
patterns of anatomically separate brain regions with high temporal
resolution. High-field FC offers a measure of interaction between
isolated clusters of columns involved in implementation of a func-
tion that will elucidate functional specialization of units and local
networks at columnar level, as well as new insights in the overall
organization of functional communication of brain networks. High-
field fMRI is capable of whole brain imaging at both high temporal
and spatial resolution, which together offer valuable information
about the core aspects of the human brain, providing an overview of
these novel imaging techniques and their implication to neurosci-
ence. High-field fMRI offers the opportunity of the (1) use of spon-
taneous resting-state fMRI in determining functional connectivity,
(2) to investigate the origin of these signals, (3) how functional con-
nections are related to structural connections in the brain network
and (4) how functional brain communication relate to cognitive
performance. Analysis of functional connectivity patterns using
graph theory, focusing on the overall organization of the functional
brain network, is also a promising technique that takes advantage of
these new functional connectivity tools in examining connectivity
diseases, like multiple sclerosis, dementia, schizophrenia, and
Alzheimer’s disease. The potential to further empower FC fMRI
with high-resolution maps based on functional units of the brain
[5–8, 22] is another reason that makes high field an exciting tech-
nology for brain studies.
The primary advantage of high-field fMRI, however, remains
the possibility of studying the brain physiology noninvasively with
a high spatial and temporal resolution at the same time. FC fMRI
reveals brain networks in resting state or based on experiments that
measure brain activation due to the execution of a specific task.
This is a unique capability that avails the entire brain for investiga-
tion at once. As such, it is critical that this capability is not compro-
mised as field strength increases. Diagnoses based on functional
mapping require high spatiotemporal resolution over the whole
brain as field strength increases. The heterogeneity of the brain
causes susceptibility-induced signal dropouts that worsen as the
field strength increases. Unfortunately, this is the same mechanism
that makes functional measurement of hemodynamics possible. So,
we must develop reliable techniques to suppress signal dropouts
while keeping susceptibility based CNR high. These conflicting
needs may provide new incentives to move fMRI toward direct
detection of neuronal correlates rather than the present
122 Alayar Kangarlu

mechanisms of BOLD or perfusion. This is where high field can

change the paradigm rather than just improving resolution. In the
meantime, efforts to boost both spatial and temporal resolution of
functional brain studies are focusing on susceptibility suppression.
The potential of 7 T fMRI (Fig. 2) has been shown in its ability
to take advantage of the magnetic susceptibility and the BOLD
effect to obtain [25] high-resolution images and functional maps.
Such advantage could increase linearly or even more than linearly
with B0 as more technology in hardware and software is developed
for high field. For example, the high SNR of SE BOLD could
eliminate contributions to the signal from large draining veins at
fields of 3.0 T and higher. This way, BOLD from microvascular
networks directly on or near the site of neuronal activity could be
detected [26]. Such tool is capable of dealing with more funda-
mental question in quantification of the signal. An fMRI signal
with resolution high enough to consistently quantify blood flow
and energy consumption provides a valuable insight into the rela-
tionship between neuroenergetics and neuronal activity. Such rela-
tionship has not received attention in fMRI studies. Most of fMRI
studies, instead, have concentrated on experimentally proving cog-
nitive neuroscience theories. High field can provide more powerful
tools and quantifiable measures for such endeavors.
Besides BOLD, arterial spin labeling (ASL) sequences have also
provided reliable measurements of CBF. This feature makes perfu-
sion-based fMRI a complement to BOLD. A version of ASL called
continuous, or CASL, has shown particular potential to take advan-
tages of high-field strengths to obtain high SNR and CNR. ASL is
implemented by tagging (normally with inversion RF pulse) the
blood flowing to the brain in the neck. After a delay time, the slice
select RF pulse is followed by an acquisition sequence. The blood
with its water magnetically labeled flows into the brain and has its
transverse magnetization decaying at the rate of T1. So, T1 duration
is important in detection of tissue perfusion. The signal in perfusion
imaging is a function of regional blood flow and the longitudinal
relaxation time T1. The T1-dependent part of perfusion signal makes
perfusion SNR a function of magnetic field. At high field, perfusion
will benefit from increase in T1 as it provides more transverse mag-
netization in the image slice. At high field, perfusion can provide
quantitative measures of absolute CBF, a more direct representa-
tion of neuronal activity than the BOLD signal.

5.1 Fast Imaging Image acquisition in MRI is slower than in other techniques such as
computed tomography and positron emission tomography. This is
mostly due to the relaxation phenomenon. Fast imaging techniques
are not widely popular for structural imaging due to the poor image
qualities and technical limitations. Relaxations and dephasing
requires refocusing of signals in the intervals of the order of TE and
realignment of spins with the main magnetic field every TR seconds,
 High-Field fMRI 123

where TR is called the repetition time. Refocusing can be achieved

by gradient reversal or RF pulses. Depending on the acceleration
rate and safety concerns, one or the other method can be used. For
detection of physiological signals, however, the image acquisition
rate should match the rate of physiological event. For brain func-
tional imaging, this rate is of the order of a second. So, there is a
need for imaging the entire brain within that timescale. For resolu-
tions of the order of 5 × 5 × 5 mm, the whole brain coverage requires
30–40 slices. For an image with 64 phase-encoding steps there is
only 300 ms for refocusing and readout. These facts leave very few
sequences for imaging at such rate. EPI is one such sequence. Its
sequence details and the implications of its execution at high fields
need close scrutiny in order to fully exploit its potentials in high-field
functional imaging studies.

5.1.1 Echo-Planar Fast imaging techniques achieve their speed by multiple refocusing
Imaging of the spin ensemble during one TR. EPI as a GE-based technique
is the fastest sequence and has a very low RF power content [27].
This aspect of EPI makes it suitable for high-field applications as
RF absorption increases at high fields increasing the RF require-
ments. On the other hand, other aspects of EPI such as geometric
distortion, blurring artifacts, and T2* signal loss are aggravated at
higher fields [28, 29]. For instance, the geometric distortion that
is caused by off-resonance effects will be further aggravated by
long readout train of EPI. A phase offset that increases with TE
will be created that will establish a linear phase gradient over
k-space in the phase-encoding direction [30]. The image signal
from these spins will get shifted as image is reconstructed. At high
fields, this effect is proportionally stronger resulting in larger
frequency shifts. However, the effect of long readouts can be dras-
tically reduced by using parallel imaging. This will reduce geomet-
rical distortions, but the T2* signal decay and blurring on the
images will still remain. Other techniques have been introduced to
deal with T2* relaxation causing distortion in images due to the
decay in the signal along the k-space trajectory. Minimization of
magnetic field inhomogeneities and susceptibility-induced effects
requires the choice of TE close to T2*. As B0 increases, T2* decreases
and hardware and safety considerations often makes the minimum
TE of single-shot EPI longer compared to T2*, which causes signal
loss due to the phase dispersion caused by such choices of
TE. Higher bandwidth could alleviate this problem but possibility
of peripheral nerve stimulation will limit the use of much stronger
gradients to achieve this. Other techniques have been proposed
that will effectively restore T2* relaxation-induced signal loss and
blurring. GE slice excitation profile imaging (GESEPI) is one such
method that, combined with multichannel parallel receiver tech-
nology, such as sensitivity encoding (SENSE), will significantly
enhance high-field EPI image qualities [31, 32].
124 Alayar Kangarlu

Other EPI artifacts such as Nyquist ghost are independent of

field strength and are inherent to the sequence k-space trajectory
with various solutions applicable to their minimization at all field
strengths [33]. Nyquist artifact is due to the time-reversal asym-
metry of even and odd echoes and its ghosts overlap with the image
causing a reduction in EPI SNR.
Next to its speed, the most important characteristic of EPI is
the high magnetic susceptibility weighting it casts on images (Figs. 2
and 3). In fact, fMRI as the most important application of EPI takes
advantage of EPI sensitivity to susceptibility change due to blood
oxygenation. Unlike fMRI applications in which susceptibility
enhances BOLD contrast, susceptibility weighting of EPI is not
considered an advantage in applications such as diffusion-weighted
imaging. As such, understanding of susceptibility is essential in
enhancing its role where it helps fMRI and suppressing its undesir-
able aspects where it hurts data quality. A brief account of magnetic
susceptibility of biological tissues is presented here to help appreci-
ate the role of susceptibility in EPI-based BOLD signal changes.

5.2 Magnetic Magnetic susceptibility, χ, is at the core of BOLD-based fMRI stud-

Susceptibility ies. When matter is exposed to strong magnetic field it will be mag-
netized [34]. In formation of χ, magnetic field (H), magnetic
induction (B), and magnetization (M) play roles. H is the entity that
exists in vacuum and its penetration through space, i.e., free space of
permeability μ0 = 4π × 10–7 H/m, is given by B = μ0H. The magneti-
zation, M, represents the total magnetic moments per unit volume
M = ∑µ/v. M is caused by H according to M = χH. B and H in SI
unit system have units of Tesla and Ampere/meter, respectively.
Inside a body placed in a magnetic field a magnetization M is gener-
ated that will produce a magnetic field of B = μ0(M + H). Replacing
M in this expression will yield B = μH where μ = μ0(1 + χ) will be the
magnetic permeability of matter. As such, susceptibility of an object
is a measure of enhancement of the magnetic field within its volume.
This is important as it will determine how uniform a magnetic field
(B0) can be established inside the body in MRI. In μ, the character-
istics of the free space and how its magnetic properties are modu-
lated by matter through χ are hidden. B0 in turn, changes locally by
χ causing the so-called susceptibility artifacts in MRI particularly in
the areas of air/tissue interface [34, 35]. This effect causes a change
in magnetic field as it is sensed inside a tissue and for heterogeneous
tissues a contrast is generated between tissues, which are B0 depen-
dent. Difference in susceptibility, ∆χ, between adjacent tissues are
small at low fields. If susceptibility-based inhomogeneity is smaller
than inherent B0 inhomogeneity it could be used for generating
contrast for better visualization of tissues such as GM. High ∆χ as
exists at the air/tissue interfaces causes large variation in B0 that is
responsible for signal dropouts interfering with studies focused on
these regions [29]. fMRI studies of regions near the ear canal, nasal
cavity, and inferior frontal lobe suffer from this phenomenon.
 High-Field fMRI 125

Fig. 3 An axial image showing cortical GM MS lesion acquired by 7 T FLAIR vs.

3 T FLAIR. (Courtesy of I.D. Klisdonk, VUMC, the Netherlands)

The most distinct role of susceptibility effect in MRI is in

fMRI. It is based on the fact that M within a voxel is linearly pro-
portional to B0 determining the role of high field in susceptibility-
based enhancements of CNR. Specifically, T2* values decrease
allowing paramagnetic molecules such as dHb to generate more
dephasing in collective proton precession at high magnetic fields.
Figures 2 and 3 show how high activation induces high functional
CNR on the images taken at 7.0 T compared to lower fields. The
short T2* values due to paramagnetic properties of dHb causes the
veins and structures with high-density vasculatures to have their
dimensions exaggerated as shown in Fig. 1. This mechanism affects
B0 through χ making a larger variation in susceptibility, ∆χ, in
brain tissues around activated neurons at high field subsequent to
a perturbation. In brain activation studies, the stimulus causes
change in volume and flow of oxygenated blood in the near prox-
imity of activated brain regions. For the same activation, high field
will use higher ∆χ for better visualization of vasculature network
which is coupled into the neuronal system in the brain.
Furthermore, high-field SNR allows the use of in vivo vascular
imaging in establishing a relationship between brain tissue vascular
density and functional imaging results. Independent information
from vascular density could be attained from MR angiography to
help better analysis of the fMRI data. In addition, such vascular
density information could be used for the study of various topics
from brain development and brain tumor staging to multiple scle-
rosis (MS). High-field fMRI in MS could better assess the effect of
any changes in cortical activation during a particular task such as
attention, memory, motor, etc. As high field enables better spatial-
ized maps of the response to stimuli, fMRI could help assess the
extent of neuropsychological problems. As fMRI becomes faster,
126 Alayar Kangarlu

more detailed brain activation in MS patients could be used to

assess their normal motor function as is done clinically today. As the
strength of response signal varies depending on the activated region
of the brain and the accessibility to the detector, i.e., RF coil, high
field could allow a wide range of regions and paradigms to be
designed to compare performance of MS patients with healthy con-
trols. Such quantitative assessment of functional performance of the
brain will provide a valuable tool in enhancing the disease manage-
ment. Structural MRI, however, has had great success in visualizing
lesions of demyelination [36–38]. But, lack of specificity has pre-
vented MRI from being established as a reliable one-shop stop for
diagnosis of MS. A reliable fMRI technique with resolution to
reveal accurate cerebral functional response to controlled stimula-
tions will complement the existing structural MRI tools in better
understanding of MS. Such potential is entirely due to inherent
sensitivity of fMRI to hemodynamic changes induced by cognitive
perturbation and will provide information independent of struc-
tural changes of the disease. In this regard, a unique aspect of high
field, i.e., high susceptibility and SNR, offers a tool that, although
is MR in nature, its attributes are not equally available at lower
fields. The B0-dependent susceptibility contrast, furthermore, pro-
vides potential for depiction of microvascular structures that will
further enrich the tool box of high-field magnets [39].
Magnetic susceptibility of blood is governed by the same
effects as discussed above. It is high enough to generate the BOLD
effect just due to the change in its oxygenation state. A dHb mol-
ecule contains four paramagnetic iron ions. During oxygenation,
dHb combines with four oxygen molecules which results in an
O2Hb molecule, which normally has no net paramagnetic moment.
O2Hb is in fact slightly diamagnetic. This will cause the magnetic
susceptibility of blood to change by about 10–6 if the blood is fully
oxygenated. Taking the susceptibility of O2Hb as zero, then change
in blood magnetic susceptibility with oxygenation constitutes the
basis of BOLD contrast in fMRI. A detailed account of the effect
of B0 on BOLD through the susceptibility mechanism will further
elucidate the impact of high field in fMRI.

5.3 Blood Oxygen A change in magnetic susceptibility of ∆χ = 10–6 (SI system) in

Level Dependent blood as a result of oxygenation is possible and forms the basis of
fMRI. Through μ = μ0(1 + χ), magnetic dipole strength of a voxel
changes by ∆χ and results in change in magnetization which con-
stitute the basis of NMR signal. The maximum possible change in
susceptibility due to blood oxygenation change is about one unit
in SI. Assuming that ∆χ = 10–6 is achieved during the activation, a
corresponding 1.0 × 10–6 or 1 ppm change will result in magnetic
field inhomogeneity. While at 1.5 T, 1 ppm inhomogeneity corre-
sponds to about 63 Hz, at 7.0 T it could produce frequency shift
of about 300 Hz. Such B0 inhomogeneity will induce dephasing
 High-Field fMRI 127

of spin coherence which will reduce the signal causing dark regions
on T2*-weighted EPI images. Even spin-echo sequence will bear
reminiscences of such susceptibility-induced signal loss near the
veins. While for stationary tissues RF does refocus the resulted
dephasing of spins, for moving water molecules in veins protons
rephrasing is not complete, making BOLD effective as a T2 as well
as T2* effect.
fMRI signal is believed to largely originate from BOLD effect
around small vessels, i.e., arterioles, capillaries, and venules [25].
The extravascular areas surrounding the small vessels represent
loci of neuronal activity. But, there are contributions from large
vessels to the BOLD signal as well. Such contribution must be
quantified to ensure an accurate account of the role of small ves-
sels vs. large vessels in fMRI. High magnetic fields provide a pow-
erful tool in this regard. A known magnetic field at any position
puts spins in a well-defined precession whose frequency provides
knowledge of its location to produce a map of proton density.
Spatial homogeneity and temporal stability of the field are impor-
tant requirements for creating images faithful to the structures
being studied. B0 field homogeneity of high-field magnets is
around 0.5 ppm that using high-order shimming could improve it
to about 0.1 ppm over the head. Beyond this, as it was discussed,
dHb produces high magnetic susceptibilities leading into compa-
rable local inhomogeneities in the static field within the brain. At
high field, regions in the brain, such as temporal lobes and basal
ganglia, demonstrate high magnetic susceptibility providing a
high contrast from the surrounding tissues [40]. Different sce-
narios for change in T2* are possible depending on the occupation
of the voxel by capillaries, large vessels, and extravascular and
intravascular BOLD [8]. In general, it can be stated that T2* signal
differential between activation and rest period from these regions
increases as a function of magnetic field. For example, if typical
acquisition parameters for fMRI studies are receiver bandwidth of
2 kHz/pixel; TR 4000 ms; TE 40 ms; FOV 190 × 190 mm2;
30–40 slices; slice thickness, 5 mm; then implications of these
parameters at 7.0 T can be contrasted to 1.5 T through a simple
frequency shift. A typical BOLD effect of 0.5 ppm or 150 Hz
frequency shift at 7.0 T could result in as high as 7 % change in
signal. Considering that BOLD has typically produced SNR,
∆R/R, of the order of 1 % at 1.5 T, this fact indicates that a linear
increase in ∆R/R with B0 is possible.
BOLD contrast acts as a change in T2* rate, ΔR2*. What are the
factors affecting ΔR2*? First, ΔR2* is directly influenced by the
change in concentration of dHb. In fact, the volume susceptibility
is directly proportional to volume of dHb and as such on ΔR2*
[34]. Assuming that dHb is proportional to blood volume, the
fraction of the blood volume fdHb occupied by dHb will have
direct effect on the signal. Models have been proposed that assign
128 Alayar Kangarlu

dependence of magnetic susceptibility difference between blood

O2Hb and dHb, ∆χ, raised to a power between 1 and 2. For a
venous oxygen saturation increasing from 60 to 95 %, Davis et al.
found a power of 1.5 fitting the simulated ΔR2* vs. ∆χ curve best
[41]. Such studies measure the oxygen consumption increase vs.
blood flow increase as a result of functional stimulation of the
brain, visual cortex in this case [41]. So, while there could be con-
siderable differences between the increase in blood flow and oxy-
gen consumption, there is a unanimous consent on the role of
oxidative metabolism as a significant component of the metabolic
response of the brain to externally induced neuronal activation.
A key role for oxidative metabolism during neuronal activation
makes the role of high field more momentous in both settling such
issues and enhancing the fMRI SNR. One needs to determine with
certainty where the changes of the blood activation come from.
They could come from the brain tissue or from the draining veins
near the activated region. Many fMRI studies do not make any
distinctions between these two contributions. This is partially due
to the challenges involved in addressing the issue. As it was
mentioned earlier, spin echo and diffusion weighting are used to
differentiate contributions from different-sized vessels. Considering
the small BOLD effect at low field, about 1 % change in signal, an
increase in fMRI signal is essential to enable suppression of BOLD
signal through SE or diffusion in order to accurately investigate the
source of activation. This is owing to the fact that SE EPI has more
T2 than T2* weighting reducing sensitivity to local susceptibility-
based changes. The GE readout is responsible for the T2* contrast.
The extent of T2* overlay on T2 contrast of SE EPI is field depen-
dent and drastic difference between T2 and T2* at high field makes
EPI readout in BOLD fMRI a good tool to investigate the exact
location of the activated region.
Furthermore, changes in oxygenation induced by neuronal
activation are complex. In the early stage of response, within the
first 2–3 s, an increase in dHb is observed, which is called the “ini-
tial dip” [42]. At the end of this stage a decrease in dHb and an
increase in O2Hb are observed. High field can refine this hemody-
namic behavior. The initial dip has not been so conspicuous at
1.5 T and as such not well documented. The strength of the initial
dip has been reported to be more than five times stronger at 7.0 T
compared to 1.5 T. Furthermore, the nature of the initial dip pro-
vides insight into the mechanism of oxygen utilization vs. cerebral
blood flow. In this regard, the initial dip could be used as another
tool at high field to study the correlation between hemodynamics
and neuronal activities.

5.4 Physiological In the absence of physiological noise, fMRI at high field could
Noise produce functional maps of the brain with even higher resolution
[43]. Scanners that already acquire submillimeter images at 7.0 T
 High-Field fMRI 129

will approach microscopic resolution in the absence of the limiting

noise. Unlike thermal noise, which is temperature dependent, flat
in frequency, not encoded by gradients, and hence constant at
room temperature, physiological noise is a function of biological
activities with relatively strong MR implications. As acquisition
time of individual slices is around 10 ms, physiological noise dur-
ing that time is not as debilitating as it is in the time series. Intensity
of this time series noise in fMRI has shown variations which cor-
relate with the respiratory and cardiac cycles, indicating physiologi-
cal modulation of BOLD by lung and cardiac function. In fact,
these signals have variations independent of stimulation paradigm
and thermal noise. Physiological noises and their correlation with
various physiological functions are independent of field strength.
Nevertheless, there are some indications that physiological noise
might have some components in brain activation as well [43, 44].
Nevertheless, physiological noises have BOLD-like signal with
low-frequency and TE-dependent variations [45]. It has also been
shown that physiological noise could be dependent on the signal
strength and its brain regional dependence. In this regard, it has
been shown to have greater magnitude in cortical GM than in
white matter [43]. The possibility of physiological noise depen-
dence on the signal strength could not be related to magnetic field
strength. However, conversion of brain metabolism into MR sig-
nal might produce a “resting-state” signal that will not correlate
with external stimulations and consequently degrades the fMRI
SNR. It has been proposed that relative strength of physiological
noise could also be due to the choice of imaging resolution. This
could be caused by a large voxel size which results in an increase in
physiological noise which in turn degrades the activation signal
[46]. These optimum voxels become smaller as field strength
increases. However, if there is any vascular cause of physiological
noise the inverse relation between the field strength and optimum
voxel size will be limited.

6 High-Resolution fMRI

High spatial resolution (submillimeter voxels) is an expected out-

come of imaging at high magnetic fields. The information content
of fMRI data can best be extracted by using an accurate account of
the effect of neural activity on fMRI signals. In order to make
fMRI images directly depicting cortical information, it is crucial to
image at the scale of functional units of cortical structure, i.e., cor-
tical columns [47]. Details of structures of cortical columns are the
most prominent features of the architecture of the cortex. The cor-
tex is organized in layers parallel to its outer surface (horizontal
layers). Layers are specialized in the cell types they contain. Both
the cell types and their connections with other neurons are unique
130 Alayar Kangarlu

in each layer. Nevertheless, there are distinct units connecting neu-

rons in the vertical direction (perpendicular to the outer surface).
From the outer surface of the cortex inward, these neuronal units
are piled up deep into the cortex and participate in producing
response to the same external stimulations. The fact that these ver-
tical structures penetrate though the entire cortical thickness gives
them the attribute of cortical columns. The cortex is made up of
about 20 billion neurons and contains progenitor cells and glial
cells and their structure is organized in units of minicolumn each
constituted of about 100 neurons. These minicolumns are tied to
each other to form cortical columns [47]. fMRI at high field is
capable of visualizing these columnar units.
To date, high-field fMRI of columnar organization has been
concentrated mostly on the visual systems. Since neurons involved
in the specific functions are incorporated in the same columns with
average dimension of 0.5 mm along cortex surface, fMRI resolutions
comparable to this dimension are essential for their observation. At
lower fields, fMRI has shown to be able to detect site of the BOLD
signal to within 5 mm at 1.5 T down to <1 mm at 7.0 T. But the
point spread functions make the relationship between susceptibil-
ity-based BOLD and loci of neuronal activity a function of correla-
tion between hemodynamics and neuronal response which is not
known with certainty [48]. It has been reported that submillimeter
in-plane resolution and the negative bold response (NBR) or “ini-
tial dip” can be used to locate the site of neural activation in the
visual cortex (V2) of anesthetized cats at 7.0 T [49, 50]. Such find-
ings at the columnar level will bestow fMRI a new capability in
functional mapping of the brain. Also, it is clear that low-field fMRI
cannot achieve similar results due to lower susceptibility and poor
SNR and CNR in reduced voxel volumes. The spatial resolutions
required for positive identification of sites of neuronal activities
require resolutions in hundreds of microns range which are only
possible at high magnetic field, i.e., >4.0 T. The neurophysiology of
neuronal columns has to be reflected in BOLD response in a way to
increase specificity and spatial resolution of fMRI. This places the
spotlight on high magnetic fields. One major requirement of an
imaging technique that is to elucidate the neurophysiology of the
central nervous system using the BOLD dynamics is to reach high
spatial and temporal resolution at the same time. High-field fMRI
has shown to have such potential.

6.1 RF and Gradient Gradients and RF coils are the two components of MRI scanners at
Coil Technology the forefronts of signal generation and detection. As such, their less
than ideal performance is the source of great many nuisances col-
lectively referred to as artifacts [51]. To eliminate the high-field dis-
tortions of fMRI images, a variety of solutions are available [52].
Postprocessing techniques are proposed to correct for some of the
distortions with known origins. Strong gradients also help reduce
 High-Field fMRI 131

distortions as they increase the receiver bandwidth which in com-

parison, susceptibility induced changes will reduce. High-field works
have also shown that multishot EPI has been able to reduce distor-
tions causing an increase in acquisition time. For those measure-
ments in which temporal resolution and spatial resolution do not
have to be at maximum multiecho EPI is a viable approach. However,
due to the low frequency nature of physiological noise, longer acqui-
sition will increase the signal variations of physiological functions.
RF coil technology appropriate for high field has many design
aspects in common with coils used in lower fields [15]. However,
due to the nature of RF distribution at higher field, RF engineering
needs many advances for adaptation to high field [53–55]. The
popular bird-cage designs will be unable to take full advantage of
high field. In particular, lumped element technology in which
capacitors and inductors are used is a design based on circuit analy-
sis using quasistatic field approximations [15]. But this analysis is
only valid at low fields since the RF wavelength required for spin
excitation decreases as the field strength increases. Specifically, RF
wavelength in air at 1.5, 3.0, and 7.0 T are about 5, 2.5, and 1 m
long. Taking into account the dielectric constant of biological tis-
sues which are around 80, the wavelength inside the body reduces
by a factor of inverse of square root of dielectric constant to around
50, 25, and 10 cm, respectively. Comparing the typical dimension
of an RF coil, say 20 cm diameter, with these numbers makes it
clear that quasistatic approximations are only valid for fields below
1.5 T where the RF coil dimension is much smaller than the wave-
length of the RF field. At 7.0 T, the resonance frequency of
300 MHz makes the in-tissue wavelength to be about 10 cm. Since
typical dimensions of the human head are comparable to this wave-
length, the wave nature of the RF pulse becomes dominant within
the head. Consequently, full wave Maxwell equation solutions are
required to estimate the magnetic field (B1) and electric field (E1)
of the RF as it penetrates into the body during the spin excitations
[54]. Such solutions are only possible through the use of sophisti-
cated numerical computations, such as finite difference time
domain (FDTD). This approach treats the RF coil interaction with
the human body as a full wave electromagnetic modeling that not
only provides an accurate map of distribution of B1 field over the
subject but also offers a precise measure of specific absorption rate
(SAR), which is an important indication of RF heating.
Inhomogeneous images acquired at high field (Fig. 1) demon-
strate the effectiveness of the techniques developed for alleviation of
inherent inhomogeneities of high-field images. These images point to
a need for change in paradigm in the use of RF in high-field MRI. Use
of computational tools for coil design is one pillar of the new para-
digm. In addition, potential for excessive heat deposition predicted
early in the history of MRI due to RF power constitutes a major safety
issue that high field will have for a long time. Another issue that is
132 Alayar Kangarlu

highlighted at high field is dielectric effects that have revealed their

presence in high-field images due to focusing of RF power at the cen-
tral regions of the imaged body [56]. In studies at high field which are
mostly done on the human heads, this effect shows strong inhomoge-
neous spread of RF reducing the power required in the peripheral
regions for spin excitation. Dielectric effects or dielectric resonance
problems at high field is an issue of concern for coil designers and
must be taken into account in the use of high-field scanners and analy-
sis of the data acquired by these systems.
Another pillar of the new paradigm is parallel imaging. Recent
techniques for acceleration of image acquisition based on parallel
imaging, SMASH-like methods and SENSE-like methods [57,
58], have shown promise in alleviating RF inhomogeneities at high
field. Both methods use surface like coil element which have an RF
profile stronger in the proximal regions than in the deep regions of
the body. While the immediate use of multichannel coil technology
(parallel imaging) is in the receiver mode to accelerate signal recep-
tion, parallel transmit will also play an important role by restoring
RF distribution over the whole head [59]. Possibility of the use of
multichannel receive and transmit technology will allow high-field
fMRI to further accelerate and enhance image qualities with poten-
tial to achieve microscopic resolution BOLD and perfusion-based
images with high temporal and spatial resolution.
Need for more powerful gradients is another necessity of high
field that has been highlighted recently as the receiver bandwidth
increases in high-field scanners. Although modern scanners are
equipped with more robust gradients, the increase in gradient
strength and slew rate continues. During the 1980s, clinical scan-
ners were equipped with gradients of 20 mT/m strength and
50 T/m/s slew rate. Today, 40-mT/m gradients with 150 T/m/s
slew rate are available in most clinical scanners. Such hardware has
helped many fMRI studies at 3.0 T and has helped research in the
development and use of more powerful gradients.
Gradients also are the source of many image artifacts. At high
field, artifacts due to EPI are aggravated and research has achieved
many successes in minimizing image artifacts. Advances have been
also achieved in gradient coil design and gradient amplifiers.
Technologies such as active shielding (AS) of gradient have been
realized. Considerable reduction in eddy current and its artifacts
are reported by the use of AS gradients. Improved technology in
pre-emphasis also has contributed in making modern gradients
capable of higher performance even compared to the recent gen-
erations. Manufacturers of specialty high-field gradients offer
products with strengths of 50–100 mT/m with capability of 150–
300 μs switch time. Such gradients can clock slew rate up to 200–
500 T/m/s. High-field fMRI is the primary beneficiary of this
technology, as strong gradients capable of faster switching rates can
be used to recover signal losses due to inhomogeneities through
 High-Field fMRI 133

suppression of T2* artifacts. There are, however, disadvantages such

as dB/dt which emerge as switching time reduces in gradients.
Faster switching increases dB/dt, which induces stronger electric
fields in conducting tissues of the body causing nerve stimulation
in the subject. Both designers and users of MRI scanners are made
aware of the potential hazards of high gradient-induced electric
fields and their use is governed by software and hardware safety
supervisors to prevent incidents, such as ventricular fibrillation.
Fortunately, fMRI uses sequences such as EPI which is very similar
to the conventional gradient recalled echo sequence and it acquires
the entire image in a single shot. The artifacts due to susceptibility
and fast switching have been addressed in solutions, such as inter-
leaved EPI and discontinuity in k-space, which have been dealt
with by flip angle adjustments. In all, solutions in cleaning up EPI
and other fast imaging techniques are making strong gradients
more useful for application in high-field fMRI studies.
Another approach for using strong gradients without their
undesirable side effects is through asymmetric designs, where the
gradient field is produced only over the intended body part. For
fMRI of the brain, this is particularly useful as it allows the estab-
lishment of stronger and faster gradients while at the same time
keeping the heart isolated from induced electric fields. As the field
strength increases, head-only scanners are gaining more attention.
While high-field advantage of SNR is independent of higher gradi-
ent strength or slew rate, the additional in-plane resolution and
slice thickness that can be achieved by using powerful gradients
will help achieve isotropic voxels and ultimately microscopic map
of brain function.

7 Conclusion

Functional imaging has achieved much success due to the MRI

inherent soft-tissue contrast and its capability of detecting
paramagnetic-based brain activation signals. The proportionality
of SNR with field strength is an opportunity that has the potential
of achieving microscopic brain mapping. High-field fMRI uses the
SNR currency to enhance sensitivity and specificity in probing neu-
rophysiology. Many high-field advantages can be utilized through
realizable improved ancillary technologies such as RF coils, new
excitation/detection schemes, artifact reduction, gradient technol-
ogy, and parallel imaging. Low-field fMRI has already produced
data from brain function that allows much insight into cognitive
neuroscience. High field, in turn, has shown potential of further
unraveling brain mysteries by detecting activation caused by con-
trolled external stimulations with resolution that is approaching
dimensions of functioning units of the brain. Such is the true
potential of high-field fMRI.
134 Alayar Kangarlu

References

1. Lauterbur PC (1973) Image formation by 16. Bloembergen PEM, Pound RV (1948)

induced local interactions: example employing Relaxation effects in nuclear magnetic reso-
nuclear magnetic resonance. Nature nance absorption. Phys Rev 73:679–746
242:190–191 17. Peters AM, Brookes MJ, Hoogenraad FG,
2. Hoult DI, Lauterbur PC (1979) The sensitiv- Gowland PA, Francis ST, Morris PG, Bowtell
ity of the zeumatographic experiment involv- R (2007) T2* measurements in human brain
ing human samples. J Magn Reson at 1.5, 3 and 7 T. Magn Reson Imaging
34:425–433 25:748–753
3. Ogawa S, Tank DW, Menon R, Ellermann JM, 18. Wansapura JP, Holland SK, Dunn RS, Ball WS
Kim SG, Merkle H, Ugurbil K (1992) Intrinsic Jr (1999) NMR relaxation times in the human
signal changes accompanying sensory stimula- brain at 3.0 Tesla. J Magn Reson Imaging
tion: functional brain mapping with magnetic 9:531–538
resonance imaging. Proc Natl Acad Sci U S A 19. Vymazal J, Righini A, Brooks RA, Canesi M,
89:5951–5955 Mariani C, Leonardi M, Pezzoli G (1999) T1
4. Sadek JR, Hammeke TA (2002) Functional and T2 in the brain of healthy subjects, patients
neuroimaging in neurology and psychiatry. with Parkinson disease, and patients with mul-
CNS Spectr 7:286–290, 295–299 tiple system atrophy: relation to iron content.
5. Yacoub E, Van De Moortele PF, Shmuel A, Radiology 211:489–495
Uğurbil K (2005) Signal and noise characteris- 20. Liu F, Garland M, Duan Y, Stark RI, Xu D,
tics of Hahn SE and GE BOLD fMRI at 7 T in Dong Z, Bansal R, Peterson BS, Kangarlu A
humans. Neuroimage 24:738–750 (2008) Study of the development of fetal
6. Duong TQ, Yacoub E, Adriany G, Hu X, baboon brain using magnetic resonance imag-
Ugurbil K, Vaughan JT, Merkle H, Kim SG ing at 3 Tesla. Neuroimage 40:148–159
(2002) High-resolution, spin-echo BOLD, and 21. Wright PJ, Mougin OE, Totman JJ, Peters
CBF fMRI at 4 and 7 T. Magn Reson Med AM, Brookes MJ, Coxon R, Morris PE,
48:589–593 Clemence M, Francis ST, Bowtell RW,
7. Pfeuffer J, Adriany G, Shmuel A, Yacoub E, Gowland PA (2008) Water proton T (1) mea-
Van De Moortele PF, Hu X, Ugurbil K (2002) surements in brain tissue at 7, 3, and 1.5T
Perfusion-based high-resolution functional using IR-EPI, IR-TSE, and MPRAGE: results
imaging in the human brain at 7 Tesla. Magn and optimization. MAGMA 21:121–130
Reson Med 47:903–911 22. Kim SG, Ugurbil K (2003) High-resolution
8. Uğurbil K, Hu X, Chen W, Zhu XH, Kim SG, functional magnetic resonance imaging of the
Georgopoulos A (1999) Functional mapping animal brain. Methods 30:28–41
in the human brain using high magnetic fields. 23. Meltzer HY, McGurk SR (1999) The effects
Philos Trans R Soc Lond B Biol Sci of clozapine, risperidone, and olanzapine on
354:1195–1213 cognitive function in schizophrenia. Schizophr
9. Logothetis NK (2008) What we can do and Bull 25:233–255
what we cannot do with fMRI. Nature 24. Kim SG, Fukuda M (2008) Lessons from
12(453):869–878 fMRI about mapping cortical columns.
10. Goense JB, Zappe AC, Logothetis NK (2007) Neuroscientist 14:287–299
High-resolution fMRI of macaque V1. Magn 25. Yacoub E, Shmuel A, Logothetis N, Uğurbil K
Reson Imaging 25:740–747 (2007) Robust detection of ocular dominance
11. Shulman RD (2001) Functional imaging stud- columns in humans using Hahn Spin Echo
ies: linking mind and basic neuroscience. Am BOLD functional MRI at 7 Tesla. Neuroimage
J Psychiatry 158:11–20 37:1161–1177
12. Bloch F (1946) Nuclear induction. Phys Rev 26. Yacoub E, Shmuel A, Pfeuffer J, Van De
7:460–473 Moortele PF, Adriany G, Andersen P, Vaughan
13. Pourcell EM, Torrey HC, Pound RV JT, Merkle H, Ugurbil K, Hu X (2001)
(1946) Resonance absorption by nuclear Imaging brain function in humans at 7 Tesla.
magnetic moments in a solid. Phys Rev Magn Reson Med 45:588–594
69:37–38 27. Mansfield P, Pykett IL, Morris PG (1978)
14. Hoult DI, Richards RE (1976) The signal-to- Human whole body line-scan imaging by
noise ratio of nuclear magnetic resonance NMR. Br J Radiol 51:921–922
experiment. J Magn Reson 24:71–85 28. Goense JB, Logothetis NK (2008)
15. Tropp J (1989) The theory of the bird-cage Neurophysiology of the BOLD fMRI signal in
resonator. J Magn Reson 82:51–62 awake monkeys. Curr Biol 18:631–640
 High-Field fMRI 135

29. Goense JB, Ku SP, Merkle H, Tolias AS, 41. Davis TL, Kwong KK, Weisskopff RM, Rosen
Logothetis NK (2008) fMRI of the temporal BR (1998) Calibrated functional MRI: map-
lobe of the awake monkey at 7 T. Neuroimage ping the dynamics of oxidative metabolism
39:1081–1093 (hypercapniaycerebrovascular reactivity). Proc
30. Farzaneh F, Riederer SJ, Pelc NJ (1990) Natl Acad Sci U S A 95:1834–1839
Analysis of T2 limitations and off-resonance 42. Yacoub E, Shmuel A, Pfeuffer J, Van De
effects on spatial resolution and artifacts in Moortele PF, Adriany G, Ugurbil K, Hu X
echo-planar imaging. Magn Reson Med (2001) Investigation of the initial dip in fMRI
14:123–139 at 7 Tesla. NMR Biomed 14:408–412
31. Yang QX, Smith MB, Briggs RW, Rycyna RE 43. Krüger G, Glover GH (2001) Physiological
(1999) Microimaging at 14 Tesla using noise in oxygenation-sensitive magnetic reso-
GESEPI for removal of magnetic susceptibility nance imaging. Magn Reson Med 46:631–637
artifacts in T(2)(*)-weighted image contrast. 44. Wang SJ, Luo LM, Liang XY, Gui ZG, Chen
J Magn Reson 141:1–6 CX (2005) Estimation and removal of physio-
32. Yang QX, Wang J, Smith MB, Meadowcroft logical noise from undersampled multi-slice
M, Sun X, Eslinger PJ, Golay X (2004) fMRI data in image space. IEEE EMBS
Reduction of magnetic field inhomogeneity 27:1371–1373
artifacts in echo planar imaging with SENSE 45. Hyde JS, Biswal BB, Jesmanowicz A (2001)
and GESEPI at high field. Magn Reson Med High-resolution fMRI using multislice partial
52:1418–1423 k-space GR-EPI with cubic voxels. Magn
33. Chen NK, Wyrwicz AM (2004) Removal of Reson Med 46:114–125
EPI Nyquist ghost artifacts with two- 46. Glover GH, Krüger G (2002) Optimum voxel
dimensional phase correction. Magn Reson size in BOLD fMRI. Proc Int Soc Magn Reson
Med 51:1247–1253 Med 10:1395
34. Schenck JF (1996) The role of magnetic sus- 47. Mountscale VB (1997) The columnar organi-
ceptibility in magnetic resonance imaging: zation of the neocortex. Brain 120:701–722
MRI magnetic compatibility of the first and 48. Triantafylloua C, Hogea RD, Wald LL (2006)
second kinds. Med Phys 23:815–850 Effect of spatial smoothing on physiological
35. Callaghan PT (1990) Susceptibility-limited noise in high-resolution fMRI. Neuroimage
resolution in nuclear magnetic resonance 32:551–557
microscopy. J Magn Reson 87:304–318 49. Duong TQ, Kim DS, Ugurbil K, Kim SG
36. Kangarlu A, Bourekas EC, Ray-Chaudhury A, (2001) Localized cerebral blood flow response
Rammohan KW (2007) Cerebral cortical at submillimeter columnar resolution. Proc
lesions in multiple sclerosis detected by MR Natl Acad Sci U S A 98:10904–10909
imaging at 8 Tesla. AJNR Am J Neuroradiol 50. Kim DS, Duong TQ, Kim SG (2000) High-
28:262–266 resolution mapping of isoorientation columns
37. Filippi M, Rocca MA (2007) Conventional by fMRI. Nat Neurosci 3:164–169
MRI in multiple sclerosis. J Neuroimaging 51. Jezzard P, Clare S (1999) Sources of distortion
17(Suppl 1):3S–9S in functional MRI data. Hum Brain Mapp
38. Fazekas F, Soelberg-Sorensen P, Comi G, 8:80–85
Filippi M (2007) MRI to monitor treatment 52. Speck O, Stadler J, Zaitsev M (2008) High
efficacy in multiple sclerosis. J Neuroimaging resolution single-shot EPI at 7T. MAGMA
17(Suppl 1):50S–55S Magn Reson Mater in Phys Biol Med
39. Christoforidis GA, Bourekas EC, Baujan M, 21:73–86
Abduljalil AM, Kangarlu A, Spigos DG, 53. Baertlein BA, Ozbay O, Ibrahim T, Lee R, Yu
Chakeres DW, Robitaille PM (1999) High Y, Kangarlu A, Robitaille PM (2000)
resolution MRI of the deep brain vascular Theoretical model for an MRI radio frequency
anatomy at 8 Tesla: susceptibility-based resonator. IEEE Trans Biomed Eng
enhancement of the venous structures. 47:535–546
J Comput Assist Tomogr 23:857–866
54. Ibrahim TS, Lee R, Baertlein BA, Kangarlu A,
40. Bourekas EC, Christoforidis GA, Abduljalil Robitaille PL (2000) Application of finite dif-
AM, Kangarlu A, Chakeres DW, Spigos DG, ference time domain method for the design of
Robitaille PM (1999) High resolution MRI of birdcage RF head coils using multi-port excita-
the deep gray nuclei at 8 Tesla. J Comput tions. Magn Reson Imaging 18:733–742
Assist Tomogr 23:867–874
136 Alayar Kangarlu

55. Ibrahim TS, Kangarlu A, Chakeress DW 57. Pruessmann KP, Weiger M, Scheidegger MB,
(2005) Design and performance issues of RF Boesiger P (1999) SENSE: sensitivity encoding
coils utilized in ultra high field MRI: experi- for fast MRI. Magn Reson Med 42:952–962
mental and numerical evaluations. IEEE Trans 58. Sodickson DK, Manning WJ (1997)
Biomed Eng 52:1278–1284 Simultaneous acquisition of spatial harmonics
56. Kangarlu A, Baertlein BA, Lee R, Ibrahim T, (SMASH): fast imaging with radiofrequency
Yang L, Abduljalil AM, Robitaille PM (1999) coil arrays. Magn Reson Med 38:591–603
Dielectric resonance phenomena in ultra high 59. Katscher U, Börnert P, Leussler C, van den
field MRI. J Comput Assist Tomogr Brink JS (2003) Transmit SENSE. Magn
23:821–831 Reson Med 49:144–150
 Chapter 5

Experimental Design
Hugh Garavan and Kevin Murphy

Abstract
This chapter addresses issues particular to the optimal design of fMRI experiments. It describes procedures
for isolating the psychological process of interest and gives an overview of block, event-related and
participant-­response-dependent designs. An additional focus is placed on data analysis with emphasis on
optimizing and isolating the neuroimaging signal in activated brain regions. Finally, the chapter addresses
a number of practical matters including optimal sample sizes and trial durations that confront all research-
ers when designing their experiments.

Key words Task design, Sample size, Scan durations, Analysis, Regression, Efficiency, Frequency

1 Overview

Noninvasive functional neuroimaging techniques enable

researchers to study the neurobiological substrates of psycho-
logical processes. The large body of neuroimaging research has
two fundamental purposes. The first is to identify the brain
regions that underlie a particular psychological process while the
second seeks to identify differential responses of these regions to
various stimuli or task challenges. The latter focus yields insights
into both how the brain accommodates varying task demands
and how differences between individuals or between clinical and
healthy comparison groups might be explained by differences in
neurobiological functioning. To achieve these goals, it is essen-
tial that one be able to isolate the psychological process of inter-
est and how best to do so, with particular regard to experimental
design, is the focus of this chapter. Part II will describe issues
particular to psychological experimental design, that is, experi-
mental control over the cognitive or emotional process of inter-
est. Part III focuses on data analysis with emphasis on optimizing
and isolating the neuroimaging signal in the activated brain
regions. Part III also addresses a number of practical matters
that confront all researchers when designing their experiments.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_5, © Springer Science+Business Media New York 2016

137
138 Hugh Garavan and Kevin Murphy

The distinction between isolating the psychological process of

interest and isolating the signal associated with that process is made
for pedagogical purposes. In practice, the two considerations are
closely intertwined in that the experiment must be designed with a
view to how the data are to be analyzed. In brief, a typical analysis
decomposes the time-series data into their contributing sources of
variance. These sources of variance, which are generally assumed to
be linearly additive, can include signals of interest such as task-­
induced brain activity as well as nuisance signals such as those cre-
ated by head-movement, scanner signal drift, physiological
processes, or the intrusion of extraneous psychological processes.
The most common method for analyzing these data is a linear
decomposition of the various signal sources using for example, a
multiple regression in which separate regressors and planned con-
trasts between regressors capture both the unwanted variance and
the variance of interest. Clearly, the design of the experiment needs
to take into consideration what regressors and what contrasts of
interest will be included in the analyses in order to ensure that the
final brain activation map can be attributed to the psychological
process of interest.

2 Task Design

As fMRI data are inherently noisy, it is important to induce as

strong a signal as possible. This serves to maximize the contrast
between the active task state and a comparison state (e.g., between
a cognitive task and a visuomotor control task). In addition to
maximizing contrast within an individual, it is also important to
maximize contrast between individuals (e.g., between a clinical
group and healthy controls) or between two times of testing (e.g.,
a pre-post comparison of treatment effects). To maximize the con-
trast between groups, it is advisable to isolate the psychological
process that best discriminates the two groups. In this regard, neu-
roimaging researchers would be well-served by grounding their
experimental methods in the relevant psychological literature that
identifies the key functions that distinguish the clinical and control
groups and that provides a wealth of research methods detailing
how to isolate those functions experimentally.
Experimental designs in fMRI can be categorized into block,
event-related, and a third, broader category, labeled participant-­
response dependent, in which a continuous measure obtained
from the participant provides a regressor for probing brain activ-
ity. The block design averages brain activation over a sustained
period of time (20–30 s would be typical durations) and con-
trasts this with similar periods of either a resting state or a com-
parison task which is typically chosen to contain all task demands
except the ­psychological process of interest. Brain regions that
 Experimental Design 139

differ between these conditions may then be attributed to the

psychological process. In an effort to exclude signals associated
with confounding physiological processes (described in detail
below), aperiodic block durations, in which the alternating ON
and OFF periods vary in durations, may be advisable.
This standard block design can be supplemented by including
gradations of task challenge. This type of parametric manipulation
can be quite advantageous: whereas the standard two-condition
comparison (e.g., task A vs. task B or task A vs. rest) is open to the
criticism of pure insertion (i.e., whether it is possible to selectively
include and exclude a psychological function without affecting
other task-related processes), the parametric manipulation assumes
that the process is always present but to varying degrees in accor-
dance with the demands placed on that process. Examples would
include presenting various intensity levels of a sensory stimulus [1]
or manipulating the number of memoranda in a working memory
task [2]. Block designs can also be enhanced by a sort of psycho-
logical triangulation in which the conjunction of distinct block
design contrasts allows one to isolate a psychological process that
can be separated from irrelevant surface features of the tasks [3].
For example, if one wishes to isolate the neuroanatomy of the
mental rehearsal component of verbal working memory, one might
design an experiment using quite distinct classes of stimuli with
each class accompanied by its own control comparison. One task
might require participants to store a list of common nouns over a
rehearsal period and recall the words after that rehearsal period. A
reasonable control condition for this task might be one in which
the word list remains on-screen for the duration of the rehearsal
period and participants read, rather than recall, the words after the
rehearsal period. The second task might present a list of nonsense
syllables through earphones. At the end of the rehearsal period a
single nonsense syllable is presented and participants report, using
a button box, if the single item was one of the rehearsed items. A
control condition for this second task might simply prompt the
participant to make a predetermined button press response at the
end of a delay period that was of similar duration to the rehearsal
period. The conjunction between the two activation maps, in
which activation for each task is first subtracted from its control
condition, may be argued to represent core regions responsible for
verbal working memory for which the influence of extraneous task
features (e.g., linguistic stimulus properties, response modalities,
recall vs. recognition) are minimized. This strength of the conjunc-
tion approach, however, may often need to be balanced against the
time costs involved in testing all the required conditions.
In circumstances in which a psychological process can be iso-
lated temporally then event-related designs are particularly useful.
Here, brain activation time-locked to the events of interest can be
selectively averaged enabling the researcher to embed trials of
140 Hugh Garavan and Kevin Murphy

interest amidst other control trials and to categorize the trials after
the participant has completed the experiment. Error trials can be
excluded (or averaged separately) and events can be coded by
whether a participant detected a target or not, responded relatively
fast or not, produced a subsequent behavior or not and so on [4].
This affords the researcher increased flexibility in probing the data-
set and has obvious advantages over the block design in circum-
stances in which the psychological process cannot be presented in
blocks as in, for example, an oddball paradigm in which the nature
of the phenomenon mandates that events are infrequent and
unpredictable. The block and event-related designs can also be
combined such that events of interest during an active task period
can be isolated while the task period itself can be simultaneously
contrasted against a control period [5, 6]. This type of mixed
design provides additional information in that one can determine
the inter-relationships between tonic activity levels (e.g., sustained
attention or an induced emotional state) and the processing of a
discrete trial (e.g., detection of a fearful face).
A final category of experimental design is what we have labeled
participant-response dependent. Here, the participant provides a
continuous measure that can, for example, be used to generate a
regressor to correlate against brain activity measures. Despite a loss
of experimental control over the participant’s behavior, this cate-
gory of design affords much flexibility when the phenomenon of
interest is either not strictly task-dependent or is difficult to experi-
mentally induce. Examples include resting state acquisitions (in
which correlated patterns of brain activity can be detected while
the participant simply rests) [7], biofeedback (in which, for exam-
ple, a participant learns to control their level of brain activity) [8],
passive viewing of a movie clip (in which there may be multiple
sources of stimulation with each varying with a different time-­
course) [9] or in which performance varies in an unpredictable
manner [10]. Performance modulations for which one could assess
brain activation changes can be quite wide-ranging including
response times or response time variability on a continuous perfor-
mance task [11], frequently sampled self-report measures of mood
[12] and physiological measures such as heart rate or pupil-­
diameter [13]. In these examples the discrete measurements can be
interpolated to provide a continuous time-series that can be cor-
related with the brain activation time-series data.
A related approach, albeit one that is manipulated to a degree
by the experimenter, is one in which the researcher derives a com-
putational model of a subject’s performance. Here, a formal model
of the processes hypothesized to underlie task performance is
developed by the experimenter. For example, on a forced-choice
reward task in which the subject attempts to maximize their win-
nings, one might hypothesize that subjects develop expectations of
reward based on previous trial outcomes, experience prediction
 Experimental Design 141

errors if those expectations are violated, learn at different rates and

so on. The parameterization in a formal model of these psychologi-
cal processes allows novel regressors to be created that track those
processes over time. Importantly, identifying brain regions where
activity covaries with those regressors serves to validate the theo-
retical computational model and its underlying mechanisms and, in
this regard, is a valuable advance beyond the more commonplace
localization of functions [14].
An important consideration permeating all experimental
designs is the choice of baseline against which activation is con-
trasted. These baselines can be explicit as in the block design in
which specific blocks are chosen for comparison or implicit as in the
event-related and participant-response dependent designs in which
the baseline is all task-related activity that is not accommodated by
regressors in the data analysis. The choice of baseline determines
the interpretation of what processes are captured in an activation
map and requires very careful consideration by the experimenter.

2.1 Choosing The choice of which design to employ will be dictated by the par-
an Experimental ticulars of the psychological process to be investigated and how easy
Design it is to isolate. Block designs can be employed if the psychological
function is easy to isolate or if it is of particular interest to compare
two tasks. A simple example would be a contrast between unilateral
and bilateral finger movements. Here, blocks of finger movement in
just one hand could be alternated with blocks of finger movements
in two hands. Rest periods might also be included in order to pro-
vide a low-level baseline against which any task-­ related activity
could be assessed. The inclusion of a resting state baseline is gener-
ally advantageous as contrasts between two task-­active periods can
often be ambiguous in that greater activation in condition A relative
to condition B could result from either more positive activation in
A or a greater deactivation in B. A resting state baseline allows one
to resolve this ambiguity by showing if activation increases or
decreases in any one condition relative to the resting baseline.
However, care must be taken when comparing task to rest as in the
last few years a large body of literature has demonstrated that rest
itself is not the absence of neural activity. For example, it has been
shown that the default mode network is more active during “rest”
than during a task (see further description of resting-state phenom-
ena and analyses in the chapter by Fornito—Chap. 10).
If the psychological function is not easily isolated then a conjunc-
tion analysis may be useful. As can be seen in the verbal working
memory example given above, the conjunction design enables the
researcher to identify the core functional neuroanatomy that is com-
mon across different operationalizations of a psychological process.
In addition, it can also reveal task-specific activations enabling, for
example, one to determine how verbal working memory rehearsal for
linguistic information differs to that of nonlinguistic information. An
142 Hugh Garavan and Kevin Murphy

alternative approach may parametrically manipulate verbal working

memory demands by asking participants to rehearse items of varying
set sizes. The presumption here is that more items will engage verbal
working memory rehearsal to a greater extent resulting in changes in
activation corresponding to the increased memory loads.
Although block designs suffer from an inability to isolate cog-
nitive events that are temporally proximal by virtue of averaging
over a prolonged duration, and may provide activation measures
contaminated by extraneous tonic processes or isolated events
(e.g., errors), they nonetheless have some advantages. For exam-
ple, if the psychological process of interest by its very nature exists
over a prolonged duration (e.g., sustained attention) or does not
exist as a temporally discrete event (e.g., an emotional reaction),
then it may be assayed best by a block design.
Conversely, the event-related design is particularly useful if
one’s goal is to isolate distinct cognitive events. In between-group
comparisons (or time 1 vs. time 2 comparisons) the event-related
design has the added advantage of being able to equate perfor-
mance levels by comparing the groups on correct trials only. That
is, one can compare correct performance trials of one group against
the correct performance trials of the second group even if the abso-
lute numbers of correct trials differ between the groups. In this
regard, contamination from activity specific to error-related pro-
cesses will not confound the between-group comparison [15]. In a
similar manner, selective averaging of trials may make it possible to
eliminate other group differences (e.g., response speed) assuming
that there are sufficient numbers of trials for this type of a matched-­
trial analysis. This is a particularly welcome feature as activation
differences between groups that one may wish to attribute to a
psychological difference can be confounded by secondary behav-
ioral or performance differences [16]. Indeed, the relationship
between performance and activation levels is not straightforward.
Often, researchers wish to ensure that the task produces perfor-
mance differences between groups (or within a group following
some experimental manipulation) in order to justify that choice of
task or the focus on the psychological process engaged by the task;
why study the neurobiology of attention between healthy controls
and children with attention deficit hyperactivity disorder (ADHD)
if the latter are not shown to be worse on the attention task?
However, this can be a double-edged sword in that performance
differences and knock-on effects such as differences in frustration
or anxiety levels can confound interpretation of activation levels.
One proposed solution is to administer a task that is within the
level of competence of all participants and which, therefore, may
not produce group differences in performance. Such a task can be
considered a probe of the neurocognitive functioning of the groups
and substantial empirical evidence shows that brain activation dif-
ferences are often observed in the absence of performance
 Experimental Design 143

differences. The typical interpretation of activation differences

when there are no performance differences is that reduced activa-
tion reflects better neural efficiency and less “effort.” This interpre-
tation is supported by studies showing greater levels of activity as
task difficulty increases or those that show reduced activation fol-
lowing practice of a psychological process [17].
Finally, the participant-dependent response design may be a
sensible choice when one can obtain a continuous measurement
from the participant but cannot exercise full experimental control
over behavior. For example, although emotional states are difficult
to induce (and extinguish) experimentally, a physiological, self-­
report, or task-induced measure can provide a time-course of that
emotional state which can be used to detect correlated brain regions.

3 Optimizing Experimental Task Designs

The key issue when optimizing experimental task design in fMRI is

statistical power. There are many important basic variables to be
chosen which, if selected wisely, will lead to high power and thus
robust and reliable results. Too often these variables are chosen
arbitrarily leading to poor experimental designs that fail to yield
the expected outcomes.
When designing a task one needs to consider practical issues
such as the number of participants or events required to give reli-
able results along with more analytic issues such as the estimation
efficiency of the task design. These pragmatics are often dictated by
feasibility constraints such as the availability of participants (e.g.,
how much access to the clinical population under study does one
have?), the cost of scan time or the amount of available time in
which the participant will remain comfortable and compliant.
Despite the ubiquity of these concerns, surprisingly few studies
have addressed them and, instead, more emphasis has been placed
on the analytic issues of presentation rate, duty cycles, sampling
procedures, detectability of activation and efficiency of response
estimation. These analytic issues relate the task that will be
­performed to the analysis methods that will be employed and pro-
vide guidance on the design details of an experiment. It is impor-
tant, however, that analytic considerations are not allowed to
dictate the design of a task such that it is no longer appropriate for
measuring/engaging the psychological process under study.

3.1 Practical Issues There has been increased concern in recent years about the power
of many (most?) fMRI studies and a number of cogent critiques
suggest that low sample sizes have generated a disconcerting num-
ber of false positive results [18, 19]. However, only a handful of
studies have addressed how many participants are required to yield
stable activation maps. The first paper addressing this issue showed
144 Hugh Garavan and Kevin Murphy

that conjunction analysis with a fixed-effect model is sufficient to

make inferences about population characteristics thus reducing the
number of participants required to infer differences between popu-
lations [20]. Although quite useful, this conclusion does not give
a clear indication of the number of participants required. By esti-
mating the mean differences and variability between two block
conditions, Desmond and Glover were able to perform simulation
experiments generating power curves from which they could calcu-
late the required number of participants [21]. They found that for
a liberal threshold of p=0.05, 12 participants were required to yield
80 % power in a single voxel for typical block design activation lev-
els. However, in fMRI the multiple comparisons problem and the
associated potential for high levels of false positives requires us to
go to stricter thresholds where they demonstrated that twice the
number of participants would be needed to maintain the same level
of statistical power. This recommended number of participants is
higher than the majority of fMRI studies but is similar to indepen-
dent assessments based on empirical data from a visual/audio/
motor task [22] and from an event-related cognitive task [23].
The Murphy and Garavan study [23] found that statistical
power is surprisingly low at typical sample sizes (n < 20) but that
voxels that were significantly active from these smaller sample sizes
tended to be true positives. Although voxelwise overlap may be
poor in tests of reproducibility, the locations of activated areas pro-
vide some optimism for studies with typical sample sizes. It was
found that the similarity between centers-of-mass for activated
regions does not increase after more than 20 participants are
included in the statistics. The conclusion can be drawn from this
paper that a study with fewer numbers of participants than Desmond
and Glover propose is not necessarily inaccurate but it is incom-
plete: activated areas are likely to be true positives but there will be
a sizable number of false negatives. Arriving at a similar conclusion,
Thirion and colleagues argue that the reliability of group analyses is
strongly affected by inter-subject variability and recommended that
20 subjects or more should be included in fMRI studies [22].
Needless to say, the required number of participants is influenced
by the effect size which, in turn, is affected by the sensitivity of the
experiment (e.g., the strength of the experimental manipulation,
the quality of the data acquisition and the accuracy of the data anal-
yses) and the contrast-to-noise of the signal that scales with field
strength of the MRI scanner. These considerations may be even
more important if one’s intention is to detect what is likely to be an
even smaller effect size of a between-­group comparison. However,
with the push to higher field strengths (e.g., 7 T), smaller effect
sizes should be detectable with similar numbers of participants.
Little research has addressed the optimal number of scans/
events needed for a successful fMRI study. A simple reason for this
is that there is no standard metric for determining the required
 Experimental Design 145

number of scans/events and no gold standard for determining

when the optimal number of events has been reached. One metric
that has been utilized is the spatial extent of activation under the
assumption that as more scans/events are included in the analysis,
the spatial extent of activation will increase until all activated cortex
is deemed above significance. When this occurs the spatial extent
should asymptote providing an estimate of the required number of
scans/events. Using this approach in a block design experiment,
Saad and colleagues demonstrated that the spatial extent of activa-
tion increased monotonically with the number of scans included in
the analysis and failed to asymptote after twenty-two 200 s long
scans [24]. Similarly, Huettel and McCarthy found that the spatial
extent of activation failed to asymptote even after 150 events in an
event-related design [25]. However, this failure to asymptote may
be a consequence of the analysis method employed [26]. The cor-
relation method does not asymptote because the goodness-of-fit to
the regressor continues to rise with increasing degrees-of-freedom
(df), which implies that the correlation measure will never plateau
by adding more time points. The Huettel and McCarthy result
[25] was replicated by Murphy and Garavan [26] but they also
demonstrated that when using a standard general linear modeling
(GLM) analysis rather than a correlation, the spatial extent of acti-
vation asymptotes after roughly 25 events in a properly jittered
event-related design. This is certainly a more attainable number of
events in the available scan time of standard fMRI studies. It can be
assumed that at least 25 of each type of event are needed if there is
more than one psychological process under study. Also, these
results have been derived from primary sensorimotor processes in
the brain so it is unclear whether they will still hold for more subtle
cognitive activations. Again, differences in activation have not been
addressed either: it is quite possible that many more events would
be required to distinguish two processes with slightly varying acti-
vation levels since these differences could be dominated by noise.
A related concern is the optimal duration of a scan. How long
a scan should last is obviously dependent on how densely the
required number of events can be distributed. For example, a GO/
NOGO task must sparsely distribute NOGO events due to the
need to build up a prepotency to respond while a simple motor
response task can present the events more frequently. Other issues
that limit how long a scan can last include participant comfort and
ability to stay engaged in the task along with technical concerns
such as throughput of data and image reconstruction times. For
these reasons and more, it is common to split a scanning session
into separate scans lasting 5–10 min each after which they can be
concatenated into one single time-series and treated as a single
scan in the analyses. However, breaks in scanning reduce the effi-
ciency of any temporal filtering that is used and can also introduce
unwanted session effects. If the goal is to detect activation then a
146 Hugh Garavan and Kevin Murphy

block design is the most efficient approach (see below). In this

case, the length of the scan is dependent on the amount of noise in
the time series (which can be measured by calculating the temporal
signal-to-noise ratio (TSNR) defined as the mean of the time series
divided by its standard deviation), the size of the effect to be mea-
sured (eff) and the significance level (P) at which the activation is
to be detected [27]. These authors derive an equation that deter-
mines how long one needs to scan to detect activation with a block
design for volumes with high spatial resolution and suggest how
this can be extended to an event-related design:
2
é æ + log10 P 2 ö æ erfc ( P ) ö ù
-1

N G = 8 ê1.5 ç 1 e ÷çç ÷ú (1)

êë è ø è ( TSNR )( eff ) ÷ø úû

where NG is the number of time points required for activation

detection. Estimates of the size of the effect can be obtained from
previous studies and TSNR measurements can be made using a
short resting scan. Since these variables differ widely across types of
task, brain regions, and scanners, it is impractical to suggest an
optimal scan duration here.

3.2 Analytic Issues The purpose of an experimental design is to alter neural activity,
and hence the blood oxygen level dependent (BOLD) signal, as
effectively as possible and in a predicted way thereby enabling the
researcher to detect the resulting brain changes. Using this predic-
tion, one looks for corresponding patterns in the fMRI time series
to determine which voxels were engaged in the task. A simple ref-
erence time series can be produced by convolving the stimulus
­timing function (which is equal to 0 when no stimulus is applied
and 1 when a stimulus is presented) with a hemodynamic response
function (HRF) that accurately represents the shape of the BOLD
response after a single event. The gamma variate function, y(t) = t r
e−t/b, has been shown to effectively model the hemodynamic
response to brief stimuli [28], with parameters r = 8.6 and c = 0.51,
and is a popular choice for modelling the hemodynamic shape. The
difference between two gamma-variates is also used in order to
model the post-stimulus undershoot. It is important that the cho-
sen HRF model accurately reflects the true shape of the response.
If, for some reason, the hemodynamic shape of a participant is
atypical (e.g., following treatment with a substance that directly
affects the vasculature or a patient group with vascular damage),
then the results of the analysis could be confounded by this differ-
ence in shape. (It should be noted that there are more advanced
approaches to reference time-series formation, such as ones that
use basis functions rather than a predetermined HRF shape and
these are addressed in a later chapter).
 Experimental Design 147

The simplest type of analysis is a linear least squares regression

of the equation:

y (t ) = b × x (t ) + a + e (t ) (2)

where y(t) is the voxel time-series data, x(t) is the reference func-
tion (i.e., the expected BOLD response to the stimulus) with β its
scaling factor, α is a constant, and ε(t) is a random Gaussian white-­
noise term. Both β and α are unknown parameters that are fit by
the linear regression method. This equation can be extended to
include extra regressors to remove unwanted trends in the data,
such as baseline drift and physiological nuisance regressors, whilst
simultaneously computing the scaling factor. This scaling factor, β,
can then be used as an activation measure for each voxel.
Multiple reference waveforms can easily be included in this
type of analysis, denoted by the term multiple linear regression. In
an experiment with two active conditions [1, 2] the equation:

y ( t ) = b1 × x1 ( t ) + b 2 × x2 ( t ) + a + d × t + e ( t ) (3)

is fitted to the data. For this model, four parameters are estimated,
the two scaling factors β1 and β2 and the baseline α and also a base-
line drift rate δ which accommodates for linear changes in the base-
line over time. It is possible to investigate whether β1 or β2 are
nonzero and whether β1 is different from β2 with statistical signifi-
cance calculated using F-tests. This method allows one to identify
active areas in the brain and calculate if an area is more active in one
condition than another, thereby satisfying the two primary pur-
poses of fMRI. This equation can be further generalized to Y = X
β + ε, where Y is a column vector of the voxel’s time-series data, X
is the design matrix, β is a column vector of scaling factors and ε is
a column vector of Gaussian white noise terms [29, 30]. This equa-
tion is called the General Linear Model (GLM) and is the basis for
most fMRI analytic techniques. The columns of the design matrix
X model the effects of interest and also confounding variables and
are, in essence, the reference waveforms mentioned above.
When designing an experimental task, one is essentially specify-
ing these reference waveforms/regressors. To maximize statistical
power, these regressors must be chosen wisely. For example, F-tests
are used to determine if there are significant differences between
the regressors. To increase statistical power, one can increase the df
by lengthening the task (for long TRs, each additional timepoint
adds a new df). It might seem like a good idea to use extremely
short TRs to increase the number of time points and hence the
statistical power. However, to gain an extra df for each additional
timepoint, each timepoint must be statistically independent from
every other. Unfortunately, due to autocorrelations introduced into
148 Hugh Garavan and Kevin Murphy

the fMRI data by physiological noise and scanner drifts, this is not
the case. This example demonstrates that knowledge of the under-
lying mechanisms of fMRI along with the analysis methods is
required when choosing even the simplest variables (such as the
number of time points and the TR) for experimental design.
Efficiency of a task design is a measure of how accurately the
GLM can estimate the β weights for each of the regressors, that is,
how small the predicted variance of the β estimates will be. For
example, assume a block design task that induces exactly a 2 % signal
change in hundreds of voxels, all with different noise properties but
with the same noise variance. If a GLM analysis is performed, the β
estimate for every voxel will be approximately 2 % for all voxels with
very little deviation. Since the variance of the estimates does not
differ widely with noise distributions, this would be considered an
efficient task. However, matters become more complicated when
there is more than one regressor. Assume that there are two block
conditions, A and B, where A and B are identical with the exception
that B is delayed with respect to A by one TR. These two regressors
are highly correlated so if a voxel responds only to condition A, it
will be extremely difficult for the GLM to distinguish this from a
voxel that responds only to B. For this reason, the β estimates for
each of the conditions will vary substantially and this would be con-
sidered an inefficient design. If the conditions are designed so that
they have zero correlation (this is achieved by delaying B by half a
block length relative to A), it would be very easy to distinguish
voxels that respond to each of the conditions individually or both of
the conditions together. Therefore, the variance of the β estimates
would be quite small and so the design is efficient. These simple
examples show that efficient task designs come from regressors that
are not correlated with each other. This can be slightly complicated
by the contrasts of interest. For example, say we have a jittered
event-related design where conditions A and B are randomly pre-
sented. If we want to find voxels that respond only to A (i.e., a
contrast matrix of C = [1 0]), only to B (i.e., C = [0 1]) or differ in
their response from A to B (i.e., C = [1 −1]), this design is very effi-
cient. However, if we want to determine voxels that respond equally
to both A and B (i.e., C = [1 1]), then the design is very inefficient
because such a voxel will always have an elevated activation level
and therefore will be indistinguishable from a voxel that does not
respond to either task. The simple idea that regressors must be min-
imally correlated becomes more complicated when multiple condi-
tions, nuisance regressors, and contrasts are placed into a GLM
analysis. The efficiency of a task is related to the covariance of the
design matrix X (i.e., all regressors expressed as columns of a matrix)
and is given the formula:

( )
-1
e = trace C ¢ ´ ( X ¢X ) ´ C
-1
(4)
 Experimental Design 149

where C is the matrix of contrast weights and ' denotes the

transpose of a matrix. Efficiency calculations should be carried
out on all experimental designs before scanning to check that
the regressors are sufficiently independent. A paper by Smith
and colleagues argues against this efficiency calculation since it
relates to computational precision rather than image noise [31].
This paper formulates the standard efficiency equations in terms
of the required BOLD effect which takes into account the
strength and smoothness of the time-series noise.
The question “how do we design a good fMRI task?” is really
asking “what experimental timing will produce the most efficient
design?”. There are two variables under our control, the stimulus
duration (SD: defined as the length of time the stimulus is dis-
played) and the interstimulus interval (ISI: defined as the length of
time between the offset of one event and the onset of another).
Another common term is stimulus onset asynchrony (SOA) defined
as SOA = SD + ISI. (Sometimes, ISI is used to mean SOA so care
must be taken to understand the true meaning when reading the
literature.) To maximize efficiency (i.e., minimize correlations
between regressors by ensuring a clear temporal separation between
the event types) one can use either a fixed ISI but vary the order of
events from different conditions or one can fix the order of the
conditions and vary the ISI. For example, if an event from either
condition A or condition B is to be presented every TR, it is very
inefficient to present the events in an alternating fashion A,B,A, …
However, efficiency is increased if the order is randomized. On the
other hand, if B must follow A (e.g., A is a picture of an object and
the participant must respond to B, a word, deciding whether it
matches the object or not), then randomizing the order is not pos-
sible. Therefore, we must vary the ISI between successive As and
Bs to increase the efficiency of the design.
The issue of experimental timing is very important in fMRI
tasks due to the relatively poor temporal resolution of the tech-
nique. Bandettini and Cox have shown that with a 2 s SD the opti-
mal ISI is 12 to 14 s when the ISI is kept constant [32]. At this
optimal ISI, the experimentally determined functional contrast
(i.e., the ability to detect activation) of an event-related task is only
35 % lower than that of a block-design (which, as explained below,
is the most efficient design). Simulations assuming a linear system
showed that this should be 65 % lower suggesting the HRF is a
nonlinear system. Most techniques in event-related fMRI analysis
assume that the hemodynamic shape of the BOLD signal is linearly
additive. It has also been shown that when the ISI is allowed to
vary, the hemodynamic response shows a 17–25 % reduction in
amplitude when trial onsets are spaced (on average) 5 s apart com-
pared to those spaced 20 s apart [33]. However, power analysis
indicated that the increased number of trials at fast rates outweighs
this decrease in amplitude if statistically reliable response detection
150 Hugh Garavan and Kevin Murphy

is the goal. So, despite the HRF being nonlinear at fast presentation
rates, the mismatch with the regressor is compensated by the
increase in trial numbers. Dale also demonstrated that if the ISI var-
ies, the statistical efficiency improves monotonically with decreasing
mean ISI and that the efficiency can be up to ten times greater than
that of a fixed ISI design [34]. These lessons on stimulus timing
suggest that even though the HRF is nonlinear at short ISIs, closely
packed, randomly presented events produce highly efficient designs.
There are two fundamentally different goals when analyzing
event-related fMRI tasks: detection of signal change (which has
been the focus thus far) and estimation of the HRF. Detection of the
signal change involves determining one variable: the amplitude of
the hemodynamic response. More information can be gleaned by
estimating the HRF (e.g., time to onset, rise time, fall time, area
under the curve) which can be used to determine subtle differences
between groups or conditions that may not show up in an amplitude
measure. However, this information comes at a cost: the experimen-
tal task can be optimized for either detection or estimation but not
both. Birn and colleagues showed that the estimation of the HRF is
optimized when stimuli are frequently alternated between task and
control states, whereas detection of activated areas is optimized by
block designs [35]. Liu and colleagues have developed a method
that can simultaneously achieve the estimation efficiency of random-
ized designs and the detection power of block designs at a cost of
increasing the length of the experiment by less than a factor of two
[36]. There are many programs that allow one to randomly (or not
so randomly) generate thousands of task designs in order to choose
the most efficient for the task at hand, be it detection or estimation.
Genetic algorithms (optimization algorithms that code different
designs like chromosomes and allow them to “crossover” and “point
mutate” as they “replicate”) that can produce designs that outper-
form random designs on estimation efficiency, detection efficiency,
and design counterbalancing have also been developed [37]. Further
work has also shown that using advanced mathematical techniques,
block designs, rapid event-related designs, m-sequence designs (ref-
erence time series with an autocorrelation of zero) and mixed designs
can nearly achieve their theoretically predicted efficiency and can be
used in practice to obtain advantageous trade-offs between efficiency
and detection power [38]. It is important when using programs to
design experiments to realize that they may converge on a structure
that may be problematic for the psychological process under investi-
gation (e.g., the most efficient task for detecting activation is a block
design, however, as noted above, if we want to design an oddball
study the oddball events of interest should not occur in a block).
When designing a task, one must also consider the frequencies
at which the events of interest are presented. Analysis packages
often perform high pass filtering to remove low-frequency drifts
from the data. If all frequencies below the limit of, say, 0.01 Hz are
 Experimental Design 151

removed then the activation to a task with a block lasting greater

than 100 s will also be removed. Similarly, this would be true for
event-related tasks if the events were presented at the same low
frequency. Other frequencies exist in the data that one must con-
sider. It is possible to remove the influence of physiological noise
from fMRI data using techniques such as RETROICOR [39].
These physiological noise sources are known to produce fluctua-
tions in the data at the cardiac frequency ~1.1 Hz, at the respira-
tion frequency ~0.3 Hz and also at the respiration volume variation
frequency ~0.03 Hz [40]. If these techniques are to be used and
the task predominantly displays power at one of these frequencies
(e.g., blocks lasting 33 s have a frequency of 0.03 Hz), then the
correction techniques may remove the activations of interest and
not just the fluctuations due to unwanted physiological processes.
Conversely, if these corrections are not used, then the GLM may
denote these physiological fluctuations as activations (if the phase
of the fluctuations matches the phase of the task). Indeed, it has
been demonstrated that task-related breathing fluctuations cause
changes in fMRI signals across the whole brain that are time-locked
to the task but are unrelated to neural activity [41]. One must also
bear in mind that when using a long TR, all frequencies will be
aliased into a narrow frequency band (e.g., with a TR of 2 s all
frequencies above 0.25 Hz will be aliased into the range of
0–0.25 Hz). This means that although the frequencies may seem
far apart, the task and the physiological noise may alias to the same
frequency (e.g., for a TR of 2 s, the respiratory frequency 0.3 Hz
will be aliased to 0.2 Hz as will a task frequency of 0.7 Hz, that is,
one event every 1.4 s). To avoid this problem it is best not to have
the events regularly spaced so they reside at one frequency but to
have random ISIs thus spreading the power to different frequen-
cies. The most efficient tasks are ones whose power is spread widely
across the whole available frequency spectrum.

4 Conclusions

Designing fMRI tasks can be difficult with logistical constraints

(e.g., how many participants and how much time per participant
can one afford) obliging the experimenter to optimize the study
design. The emphasis here has been on the experimental and ana-
lytic means of isolating a psychological process and its associated
fMRI signal. Both considerations are central: optimal efficiency is
of little comfort if one measures the wrong thing but there is little
to be gained from an inaccurate measurement of a robust psycho-
logical phenomenon. General recommendations include the
importance of grounding one’s experiment in the appropriate the-
oretical framework and using appropriate experimental methods,
generating designs that are tested for their efficiency prior to data
152 Hugh Garavan and Kevin Murphy

collection, ensuring that a sufficiently large sample is tested and

being clear on whether one’s goal is the detection of a response or
the estimation of that response.

References

1. Helmchen C, Mohr C, Erdmann C, Binkofski cocaine self-administration using BOLD

F, Buchel C (2006) Neural activity related to fMRI. Neuroimage 26:1097–1108
self- versus externally generated painful stimuli 13. Kampe KK, Frith CD, Frith U (2003) “Hey
reveals distinct differences in the lateral pain John”: signals conveying communicative inten-
system in a parametric fMRI study. Hum Brain tion toward the self activate brain regions asso-
Mapp 27:755–765 ciated with “mentalizing,” regardless of
2. Braver TS, Cohen JD, Nystrom LE, Jonides J, modality. J Neurosci 23:5258–5263
Smith EE, Noll DC (1997) A parametric study 14. O'Doherty JP, Dayan P, Friston K, Critchley
of prefrontal cortex involvement in human H, Dolan RJ (2003) Temporal difference mod-
working memory. Neuroimage 5:49–62 els and reward-related learning in the human
3. Price CJ, Friston KJ (1997) Cognitive con- brain. Neuron 38(2):329–337, PMID:
junction: a new approach to brain activation 12718865
experiments. Neuroimage 5:261–270 15. Murphy K, Garavan H (2004) Artifactual
4. Garavan H, Ross TJ, Murphy K, Roche RA, fMRI group and condition differences driven
Stein EA (2002) Dissociable executive func- by performance confounds. Neuroimage
tions in the dynamic control of behavior: inhi- 21:219–228
bition, error detection, and correction. 16. Poldrack RA (2000) Imaging brain plasticity:
Neuroimage 17:1820–1829 conceptual and methodological issues--a theo-
5. Donaldson DI, Petersen SE, Ollinger JM, retical review. Neuroimage 12:1–13
Buckner RL (2001) Dissociating state and item 17. Kelly AM, Garavan H (2005) Human func-
components of recognition memory using tional neuroimaging of brain changes associ-
fMRI. Neuroimage 13:129–142 ated with practice. Cereb Cortex
6. Simões-Franklin C, Hester R, Shpaner M, Foxe 15:1089–1102
JJ, Garavan H (2010) Executive function and 18. Ioannidis JP (2005) Why most published
error detection: the effect of motivation on research findings are false. PLoS Med 2:e124
cingulate and ventral striatum activity. Hum 19. Button KS, Ioannidis JP, Mokrysz C, Nosek BA,
Brain Mapp 31:458–469 Flint J, Robinson ES, Munafò MR (2013) MR.
7. Margulies DS, Kelly AM, Uddin LQ, Biswal Power failure: why small sample size undermines
BB, Castellanos FX, Milham MP (2007) the reliability of neuroscience. Nat Rev Neurosci
Mapping the functional connectivity of ­anterior 5:365–76. doi:10.1038/nrn3475, Epub 2013
cingulate cortex. Neuroimage 37:579–588 Apr 10. Erratum in: Nat Rev Neurosci. 6, 451
8. Weiskopf N, Veit R, Erb M et al (2003) 20. Friston KJ, Holmes AP, Worsley KJ (1999)
Physiological self-regulation of regional brain How many subjects constitute a study?
activity using real-time functional magnetic Neuroimage 10:1–5
resonance imaging (fMRI): methodology and 21. Desmond JE, Glover GH (2002) Estimating
exemplary data. Neuroimage 19:577–586 sample size in functional MRI (fMRI) neuro-
9. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach imaging studies: statistical power analyses.
R (2004) Intersubject synchronization of cor- J Neurosci Methods 118:115–128
tical activity during natural vision. Science 22. Thirion B, Pinel P, Meriaux S, Roche A,
303:1634–1640 Dehaene S, Poline JB (2007) Analysis of a large
10. Slotnick SD, Yantis S (2005) Common neural fMRI cohort: statistical and methodological
substrates for the control and effects of visual issues for group analyses. Neuroimage
attention and perceptual bistability. Brain Res 35:105–120
Cogn Brain Res 24:97–108 23. Murphy K, Garavan H (2004) An empirical
11. Hahn B, Ross TJ, Stein EA (2007) Cingulate investigation into the number of subjects
activation increases dynamically with response required for an event-related fMRI study.
speed under stimulus unpredictability. Cereb Neuroimage 22:879–885
Cortex 17:1664–1671 24. Saad ZS, Ropella KM, DeYoe EA, Bandettini
12. Risinger RC, Salmeron BJ, Ross TJ et al (2005) PA (2003) The spatial extent of the BOLD
Neural correlates of high and craving during response. Neuroimage 19:132–144
 Experimental Design 153

25. Huettel SA, McCarthy G (2001) The effects ordering brain activity based on relative timing.
of single-trial averaging upon the spatial extent Neuroimage 11:735–759
of fMRI activation. Neuroreport 34. Dale AM (1999) Optimal experimental design
12:2411–2416 for event-related fMRI. Hum Brain Mapp
26. Murphy K, Garavan H (2005) Deriving the 8:109–114
optimal number of events for an event-related 35. Birn RM, Cox RW, Bandettini PA (2002)
fMRI study based on the spatial extent of acti- Detection versus estimation in event-related
vation. Neuroimage 27:771–777 fMRI: choosing the optimal stimulus timing.
27. Murphy K, Bodurka J, Bandettini PA (2007) Neuroimage 15:252–264
How long to scan? The relationship between 36. Liu TT, Frank LR, Wong EC, Buxton RB
fMRI temporal signal to noise ratio and neces- (2001) Detection power, estimation efficiency,
sary scan duration. Neuroimage 34:565–574 and predictability in event-related
28. Cohen MS (1997) Parametric analysis of fMRI fMRI. Neuroimage 13:759–773
data using linear systems methods. Neuroimage 37. Wager TD, Nichols TE (2003) Optimization
6:93–103 of experimental design in fMRI: a general
29. Friston KJ, Holmes AP, Poline JB et al (1995) framework using a genetic algorithm.
Analysis of fMRI time-series revisited. Neuroimage 18:293–309
Neuroimage 2:45–53 38. Liu TT (2004) Efficiency, power, and entropy
30. Worsley KJ, Friston KJ (1995) Analysis of in event-related fMRI with multiple trial types.
fMRI time-series revisited–again. Neuroimage Part II: design of experiments. Neuroimage
2:173–181 21:401–413
31. Smith S, Jenkinson M, Beckmann C, Miller K, 39. Glover GH, Li TQ, Ress D (2000) Image-­based
Woolrich M (2007) Meaningful design and method for retrospective correction of physio-
contrast estimability in FMRI. Neuroimage logical motion effects in fMRI:
34:127–136 RETROICOR. Magn Reson Med 44:162–167
32. Bandettini PA, Cox RW (2000) Event-related 40. Birn RM, Diamond JB, Smith MA, Bandettini PA
fMRI contrast when using constant interstimu- (2006) Separating respiratory-variation-­ related
lus interval: theory and experiment. Magn fluctuations from neuronal-activity-­related fluctu-
Reson Med 43:540–548 ations in fMRI. Neuroimage 31:1536–1548
33. Miezin FM, Maccotta L, Ollinger JM, Petersen 41. Birn RM, Murphy K, Handwerker DA,
SE, Buckner RL (2000) Characterizing the Bandettini PA (2009) fMRI in the presence of
hemodynamic response: effects of presentation task-correlated breathing variations.
rate, sampling procedure, and the possibility of NeuroImage 47(3):1092–1104
 Chapter 6

Preparing fMRI Data for Statistical Analysis

John Ashburner

Abstract
This chapter describes the procedures applied to fMRI data prior to their statistical analysis. This usually
begins with converting the data from original MR format to a form that can be used by the analysis soft-
ware. The data are then motion corrected. If an anatomical scan is collected for the subject, then it would
be coregistered with the fMRI, and may serve to estimate the warps needed to spatially normalize the
fMRI to some standard space. The final processing step is usually to smooth the data.

Key words Generative model, fMRI, Registration, Artifact correction, Spatial normalization,
Smoothing

1 Introduction

This chapter provides a brief overview of the image processing steps

currently used for transforming fMRI data into a form suitable for
analysis using some form of statistical parametric mapping. Processing
strategies for fMRI data are not fixed, and the particular procedures
used depend on the data and the aims of the analysis. Pragmatic
motivations, such as software availability and ease of use, also play a
major role in determining how fMRI data are processed. This chap-
ter focuses on the main steps that are usually applied to the data,
which mostly involve various forms of image registration. The first
step is usually to convert from DICOM format, to a file format that
is more manageable. This is followed by motion correcting the data,
which may include a distortion correction procedure. Often, there is
also an anatomical scan collected for each subject, and this would be
brought into alignment with the fMRI by a coregistration step. The
anatomical image is sometimes useful in order to “spatially normal-
ize” the fMRI data. The warps, needed to deform the fMRI to some
standard space, can be estimated using the anatomical image. Once
these warps have been estimated, they can be applied to the motion
corrected fMRI data, to spatially normalize them. The final step,
before statistical analysis, is usually to smooth the data.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_6, © Springer Science+Business Media New York 2016

155
156 John Ashburner

There are many variations on this sequence of operations. For

example, a two-level approach for multi-subject analysis (random
effects model) may be performed by generating parameter-estimate
images from fMRI data that have not been spatially normalized.
These parameter images could then be warped to the standard
space and the statistical analysis performed on them. The smooth-
ing step would be omitted if the statistical analysis included a
model of spatial smoothness. This chapter says nothing about
“slice-time correction”, and assumes that the model used in the
subsequent statistical analysis accounts for the fact that slices of
fMRI data are not acquired simultaneously. Surface based
approaches, in which the fMRI data is projected on to a represen-
tation of the cortical surface, are not covered.

2 File Format Conversion

Most MRI scanners produce image data in a format that conforms

to the DICOM Standard. This stands for “Digital Imaging and
Communications in Medicine”, and it is the standard used in virtu-
ally all hospitals worldwide. To keep up with technological
advances, the DICOM Standard is re-published approximately
every year or two. The standard is also extensible, and scanner
manufacturers customize file formats to suit their own particular
needs. Several hundred pages of documentation describing the
basic file format are available from http://dicom.nema.org/.
Most fMRI analysis is currently performed within an academic
setting, where the complexity of DICOM is un-necessary.
Neuroimaging analysis tools are written by scientists and engineers
who wish to avoid working with complex and difficult formats. As
a result, several different file formats for fMRI data arose, many of
which were variants of the ANALYZE™ 7.5 format, which consists
of an “.img” file containing the image data itself, plus a “.hdr” file
containing various pieces of descriptive information. For a number
of years, various fMRI analysis software developers used the
ANALYZE format in slightly different ways, or had their own file
formats. This made inter-operability among packages very difficult,
which precluded the use of tools developed at one site with tools
developed at another. One classic example of such problems was
the different ways in which the voxels1 of an image are ordered,
which often caused uncertainty about the laterality of the brain.
The NIfTI-1 data format was developed in order to facilitate
inter-operability among fMRI data analysis packages2, and it was
recently extended to allow much larger images to be stored using

1
A voxel is a three dimensional pixel, and can be thought of as a “volume ele-
ment”, as opposed to a “picture element”.
2
See http://nifti.nimh.nih.gov/.
 Processing fMRI Data 157

the NIfTI-2 format. Standards have been agreed on how the data
format should be used, with the aim of making it easier to mix dif-
ferent software packages. Providing that only NIfTI compliant
software is used, there should no longer be any confusion about
the orientation of the brains within the images.
Images are usually treated as an array of voxels. For example,
an anatomical image is generally treated as a 3D array, and most
packages require this volume to be stored in a single file. DICOM
usually stores each slice separately (as a series of 2D arrays), but
most file format conversion routines will stack these slices together
into a 3D volume. A run of fMRI data is usually considered as a 4D
array, although for many procedures, it is often convenient to treat
it as a time-series of 3D arrays. Some packages assume that the
entire run is saved in a single file (e.g., FSL), whereas others treat
the data as a series of files containing 3D volumes (e.g., SPM).
The NIfTI format allows storage on disk to be in either a left- or
right-handed coordinate system. However, the format includes an
implicit spatial transformation into a right-handed coordinate system.
This transform maps from data coordinates (e.g., column i, row j,
slice k), into some real world (x,y,z) positions in space. These posi-
tions could relate to Talairach & Tournoux (T&T) space [1],
Montreal Neurological Institute (MNI) space [2, 3], or patient-
based scanner coordinates. For T&T and MNI coordinates, x
increases from left to right, y increases from posterior to anterior, and
z increases in the inferior to superior direction. MRI data are usually
exported from the scanner as DICOM format, which encodes the
positions and orientations of the slices. When data are converted
from DICOM to NIfTI format, the relevant position and orientation
information can be determined from the “Pixel Spacing”, “Image
Orientation” and “Image Position” fields of the DICOM files.
Terms such as “neurological” and “radiological convention”
relate only to visualization of axial images. They are unrelated to
how the data are stored on disk, or even how the real-world coor-
dinates are represented. It is more appropriate to consider whether
the real-world coordinates system is left- or right-handed. T&T
use a right-handed system, whereas the storage convention of
ANALYZE files is usually considered as left-handed (x increases
from right to left). These coordinate systems are mirror images of
each other, so transforming between left- and right-handed sys-
tems involves flipping, and cannot be done by rotations alone.

3 Corrections to fMRI Data

Most processing of fMRI data involves some form of spatial regis-

tration. The head of a single individual is generally considered to be
fairly rigid, so the initial aim is to bring all the image volumes of
each individual subject into alignment (intra-subject registration),
158 John Ashburner

prior to registering data from all the subjects together (inter-subject

registration). Various artifact corrections may be incorporated
within the intra-subject registration procedures.

3.1 Artifact There are a number of image artifacts that result from the very fast
Correction acquisition times required for fMRI. Many of these have detrimental
effects if not properly modelled. Some groups have developed in-
house software to improve on the algorithms supplied by scanner
manufacturers for reconstructing images from the original K-space
data. The objectives of these custom reconstruction algorithms
include reducing Nyquist ghosting artifacts and ensuring that the
model uses a better trajectory through K-space. Sites that perform
their own image reconstruction require the original complex K-space
data, for which there is no clearly defined DICOM standard.
The introduction of a subject into the scanner causes distortions
of the magnetic field [4]. The field may be uniform when there is no
subject present; but with a subject in the scanner, the field is influ-
enced (through Maxwell’s equations) by the varying magnetic suscep-
tibilities of different tissues. These effects are especially prominent at
the interface between tissue and air, resulting in (for example) drop-
out and distortion in the frontal lobe in regions close to the nasal
sinuses. For echo-planar images (EPI), the main effects of magnetic
field inhomogeneity are spatial distortions in the phase-encoding
direction of the images, and dropouts (signal loss) that arise because
of through-plane de-phasing. Some of the distortions can be reduced
by active shimming (changing the field of the scanner via the shim
coils), or by passive shimming (introducing diamagnetic material into
the orifices of the subject), but these measures only reduce the effects
of distortions and dropouts, and cannot completely counteract them.
The models used for intra-subject registration of the head
often assume rigid-body movement. Obtaining accurate alignment
of a relatively distortion-free anatomical image with highly dis-
torted fMRI data is not possible, unless the geometric distortions
are corrected. Therefore, one of the first steps is often a correction
for these distortions. Retrieving signal that is lost in dropouts is
not possible, but there are a number of possible post-hoc approaches
for correcting geometric distortions in the images.
● It is possible to compute field maps from additional scans that
are normally collected just prior to the fMRI runs [4, 5] (see
Fig. 1). This involves processing complex data (i.e., real and
imaginary, or phase and magnitude) from these measurements
in order to compute an unwrapped version of the phase. Phase
measurements are in the range of −π to π radians, or are from
0 to 2π radians. Phase unwrapping [6, 7] involves trying to add
or subtract multiples of 2π to the values, such that the result is
as spatially smooth as possible. With appropriate rescaling, this
unwrapped phase map is converted into a voxel-displacement
map for correcting the fMRI.
 Processing fMRI Data 159

Fig. 1 Distortions in EPI can be corrected by field maps

● If the air, bone and other tissue can be segmented from the ana-
tomical images, then it becomes possible to simulate field maps
[8] by solving Maxwell’s equations. Separating air from bone is
quite difficult from MRI scans, as both generally appear dark.
Air appears dark because of its low proton density, whereas hard
tissue such as bone has a very short T2 relaxation time, so most
of the signal has decayed before it is detected. Additional prior
information generated from computed tomography (CT) scans
is generally needed in order to attempt such segmentation [9,
10]. For this reason, the approach has not been widely adopted.
● Image registration procedures can also be used to estimate the
warps that align a distorted fMRI scan with a (relatively)
distortion-free anatomical image [11, 12]. Contrast differ-
ences between the images mean that some form of
information-theoretic objective function is usually required,
and the effects of signal dropout in the fMRI should also be
taken into consideration [13].
The field map approach is generally the most accurate way to
correct geometric distortions, although a correction that combines
all of the above strategies into a single model is likely to be the
most accurate.
160 John Ashburner

3.2 Motion The most common application of within-modality registration in

Correction functional imaging is to reduce motion artifacts by realigning the
volumes in the image time-series. The objective of realignment is to
determine the rigid-body transformations that best align the series
of functional image volumes to the same space. Blood oxygenation-
level-dependent (BOLD) signal changes elicited by the haemody-
namic response tend to be small compared to apparent signal
differences that can result from subject movement [14]. Subject
head movement in the scanner cannot be completely eliminated, so
retrospective motion correction is usually performed as a processing
step. This is especially important for experiments where subjects
may move in a way that is correlated with the different experimental
conditions. Even tiny systematic differences can result in a signifi-
cant signal accumulating over numerous scans. Without suitable
corrections, artifacts arising from subject movement, which corre-
late with the experiment, may appear as activations. A second rea-
son why motion correction is important is that it increases sensitivity.
The t-test is based on the signal change relative to the residual vari-
ance. The residual variance is computed from the sum of squared
differences between the data and the linear model to which it is
fitted; movement artifacts add to this residual variance, and so
reduce the sensitivity of the test to true activations.
At its simplest, image registration involves estimating a mapping
between a pair of images. One image is assumed to remain stationary
(the reference image), whereas the other (the moved image) is spa-
tially transformed to match it. In order to transform the moved
image to match the reference, it is necessary to determine a mapping
from the location of each voxel in the reference to a corresponding
location in the moved image. The moved image is then re-sampled
at the new locations. The mapping can be thought of as a function
of a set of estimated transformation parameters. The shape of a
human brain changes very little with head movement, so rigid-body
transformations can be used to model different head positions and
orientations of the same subject. Matching of two images is per-
formed by finding the spatial transformation (mapping) that opti-
mizes some mutual function of the images. For the case of a
rigid-body transformation in three dimensions, the mapping is
defined by six parameters: three translations and three rotations.
There are two steps involved in registering a pair of images
together. There is the registration itself, whereby the set of param-
eters describing the mapping is estimated. Then there is the trans-
formation, where one of the images is transformed according to
the estimated parameters. For rigid registration, this step is often
referred to as “reslicing”.
Registration involves estimating the parameters of a spatial
transformation that “best” match the images. The quality of the
match is based on an objective function, which is maximized or mini-
mized using some optimization algorithm. It is not computationally
 Processing fMRI Data 161

feasible to search over all possible parameter settings to find the

global optimum of the objective function. Instead, registration is
usually performed as a local optimization, which involves assigning
an initial guess to the parameters, and then trying to improve the
estimates in an iterative way. This involves iteratively transforming
the moved image many times, using different parameter values, until
the objective function can no longer be improved upon. Whether or
not the final estimate is globally optimal will depend on how far the
optimal solution is from the starting estimate. For this reason, pack-
ages such as SPM sometimes require images to be manually reposi-
tioned prior to performing any image registration.
Alignment of fMRI data is usually achieved by minimizing the
mean squared difference between each of the images and a refer-
ence image, where the reference image could be one of the images
in the series. For slightly better results, this procedure could be
repeated, but instead of matching to one of the images from the
series, the images would be registered to their mean (after a first-
pass alignment). In general, the best objective function to use for
image registration depends on what assumptions can be made
about the data. In the case of the mean squared difference objec-
tive function, the assumption is that the image noise is approxi-
mately Gaussian, and does not vary over the image.
Even after rigid realignment, there may still be some motion-
related artifacts remaining in the data. There are many sources of
such residual artifacts, and the most obvious ones are:
● Interpolation error from the re-sampling algorithm [15] used to
transform the images can be a source of motion-related artifacts.
For this reason, and also for speed and efficiency, some registra-
tion algorithms use a Fourier interpolation method [16, 17].
● When MR images are reconstructed, the final images are usually
the modulus of the initially complex data. This results in voxels
that should be negative being rendered positive. This has impli-
cations when the images are re-sampled, because it leads to errors
at the edge of the brain that cannot be corrected, irrespective of
how accurate the interpolation method is. Possible ways to cir-
cumvent this problem are to work with complex data, or apply a
low-pass filter to the complex data before taking the modulus.
● The sensitivity (slice-selection) profile of each slice also plays a
role in introducing artifacts [18]. Gaps between slices are dif-
ficult to deal with as it is not possible to recover information
that was not actually acquired.
● fMRI images are spatially distorted, and the amount of distor-
tion depends partly upon the position of the subject’s head
within the magnetic field. Interactions between image distortion
and the orientation of a subject’s head in the scanner can also
cause other problems because purely rigid alignment does not
take this into account. Relatively large subject movements result
162 John Ashburner

in the brain images changing shape, and these shape changes

cannot be corrected by a rigid-body transformation alone. The
interaction between image distortion and head orientation illus-
trates a limitation of conceptualizing processing as the applica-
tion of a series of tools to the data. These issues are better
resolved by a generative model that combines both a model for
image distortions, and a model of subject motion [19].
● Each volume of a series of fMRI data is currently acquired a
plane at a time over a period of about a second. Subject move-
ment between acquiring the first and last plane of any volume is
another reason why the image volumes may not strictly obey the
rules of rigid-body motion [20]. A better model would allow
each slice to move separately—but it may lead to technical prob-
lems if there was too much movement. For example, the model
would allow some points in the brain to be scanned more than
once during the acquisition of a volume, and some points not to
be scanned at all. Filling in the appropriate values in the cor-
rected images is difficult if there is no actual data to sample.
● After a slice is magnetized, the excited tissue takes time to
recover to its original state, and the amount of recovery that has
taken place will influence the intensity of the tissue in the image.
This effect can be seen in the first few scans of an fMRI time
series, and is the reason why a few “dummy scans” are collected
at the start of an fMRI run in order for the intensities to stabi-
lize. Out of plane movement will result in a slightly different
part of the brain being excited during each repeat. This means
that the spin-excitation will vary in a way that is related to head
motion, and so leads to more movement related artifacts [21].
● Nyquist ghost artifacts in MR images do not obey the same rigid-
body rules as the head, so a rigid rotation to align the head will
not mean that the ghosts are aligned. The same also applies to
other image artifacts, such as those arising due to chemical shifts.
● The accuracy of the estimated registration parameters is normally
in the region of tens of μm. This is dependent upon many factors,
including the effects just mentioned. Even the signal changes
elicited by the experiment can have a slight effect (a few μm) on
the estimated parameters [22], so this in turn may have conse-
quences in terms of how significant differences are interpreted
These problems cannot be corrected by simple rigid-body
realignment, and so may be sources of stimulus correlated motion
artifacts. Systematic movement artifacts resulting in a signal change
of only one or two percent can lead to highly significant false posi-
tives3 over an experiment with hundreds of scans. This is especially

3
These would actually be Type III errors, rather than Type I, because the null
hypothesis would be correctly rejected (there is a statistically significant effect
in the data) but for the wrong reason (the effect is due to motion, rather than
BOLD signal).
 Processing fMRI Data 163

important for experiments where some conditions may cause slight

head movements (such as motor tasks, or speech), because these
movements are likely to be highly correlated with the experimental
design. In cases like this, it is difficult to separate true activations
from stimulus correlated motion artifacts. All that can be con-
cluded is whether or not there is a difference among the data. The
specific causes of any difference remain unknown, but it is gener-
ally hoped that they relate to BOLD signal changes. Providing
there are enough images in the series and the movements are small,
some of these artifacts can be removed during the subsequent sta-
tistical analysis by regressing out any signal that is correlated with
functions of the estimated movement parameters [21]. However,
when the estimates of the movement are related to the experimen-
tal design, it is likely that much of the interesting BOLD signal will
also be regressed out of the data. For retrospective motion correc-
tion, these issues remain unresolved, and will remain unresolved
until interactions among processing steps (including the statistical
analysis) are properly modeled.
Prospective motion correction approaches are now becoming
more practical, and their application in fMRI studies is increasingly
widespread. These methods involve tracking the motion of the
subject in the scanner, and adjusting the data acquisition accord-
ingly. A recent review of this subject area can be found in [23].

4 Inter-Modality Registration

For studies of a single subject, sites of activation can be localized

more clearly by superimposing them on a high-resolution anatomi-
cal (structural) image of the subject (typically a T1-weighted MRI).
This requires registration of the functional images with the ana-
tomical image. A further use for this registration is that a more
precise spatial normalization can be achieved by estimating the
requisite warps from a more detailed anatomical image. If the func-
tional and anatomical images are in register, then a warp estimated
from the anatomical image can also be applied to the functional
images. In practice, this requires the usual geometric distortions
found in fMRI to have been corrected.
As in the case of movement correction, this registration is nor-
mally performed by optimizing a set of parameters describing a
rigid-body transformation, but the matching criterion needs to be
more complex because the anatomical and functional images nor-
mally have very different patterns of intensity. A simple mean-
squared difference model will no longer be effective, so alternative
objective functions (similarity measures) are needed. Inter-modal
registration approaches initially involved the use of landmarks, which
were manually defined on the images. The images were registered by
164 John Ashburner

bringing the landmarks into alignment. An early automated approach

based on a similarity measure between images was the AIR (auto-
mated image registration) algorithm [24]. The objective function
was obtained by dividing the intensities of one image into a number
of bins. For the voxels associated with each bin, the idea was to mini-
mize the variance of the corresponding voxel intensities of the other
image. The algorithm was originally intended for registering posi-
tron emission tomography (PET) and anatomical MRI, and worked
well—providing the MRI had non-brain tissue removed.
Many of the more recent similarity measures used for inter-modal
(as well as intra-modal [25]) registration are based on information
theory. These measures are based on joint probability distributions of
intensities in the images, usually discretely represented in the form of
2D joint histograms, which are normalized to sum to one. The most
commonly used measure of image alignment is mutual information
(MI) [26, 27] (also known as Shannon information). MI is a measure
of dependence of one image on the other, and can be considered as a
distance (Kullback-Leibler divergence) between the joint distribution
and the equivalent distribution assuming complete independence.
Another perspective is that MI is a measure of the reduction of uncer-
tainty about one image given the other. Registration algorithms work
under the assumption that the MI between the images is maximized
when they are in register (Fig. 2). A number of other information
theoretic measures have since been devised [28, 29], and a more
complete review of information theoretic image registration
approaches is given by [30]. A number of inter-modality registration
algorithms have been thoroughly evaluated on the same data [31],
and alignment accuracy is generally found not to be as high as that
obtained by within modality registration.
Artifacts in MRI can lead to problems when attempting to
align images using information theoretic approaches. In particular,
intensity nonuniformity artifacts (also known as “bias” or “inho-
mogeneity”) can severely degrade the accuracy of image registra-
tion procedures [32]. There are a number of intensity inhomogeneity
correction methods, which are typically based on information the-
oretic measures [33, 34]. It would seem natural to see image
registration and inhomogeneity correction combined into a com-
mon information theoretic framework.

5 Spatial Normalization

Currently, the main application for deformable image registration

within imaging neuroscience is the procedure known as spatial nor-
malization. This involves warping the brain images from different
subjects in a study into roughly the same standard space to allow
signal averaging across subjects. In functional imaging studies, spatial
normalization is useful for determining what happens generically
 Processing fMRI Data 165

Fig. 2 The top row shows orthogonal sections of two MR images of different contrasts. Below this are joint
intensity histograms of the image pair, both before and after image registration (note that the pictures show
log(1 + N), where N is the count in each histogram bin)

over individuals. A further advantage is that activation sites can be

reported according to their Euclidean coordinates within a standard
coordinate system [35]. The most commonly adopted coordinate
system within the brain imaging community is that described by
Talairach and Tournoux [1], although new standards are emerging,
which are based on digital atlases [2, 3, 36].
This section provides a brief overview of the ideas underlying
deformable image registration. This is a large area of research, but
a more comprehensive review may be found in [37, 38]. The pre-
vious sections described rigid-body approaches for registering
brain images of the same subject, where it was assumed that there
are no differences among the shapes of the brains. This is often a
reasonable assumption to make for intra-subject registration, but
166 John Ashburner

is not appropriate for aligning brain images of different subjects.

In addition to estimating an unknown pose and position, inter-
subject registration approaches also need to model the different
sizes and shapes of the subjects’ heads and brains.
Methods of spatially normalizing images can be divided broadly
into label based and intensity based. Label based techniques involve
identifying features (labels) in a subject’s image, and then bringing
these into alignment with the appropriate location in some atlas.
The original strategy proposed by Talairach and Tournoux involved
matching discrete points, but other forms of label could also be
used, such as lines or surfaces. Homologous features are often
identified manually, but this process is time consuming and subjec-
tive. Another disadvantage of using points as landmarks is that
there are very few readily identifiable discrete points in the brain,
so the registration accuracy in regions away from those points is
likely to be limited. The required transformation at the defined
features is known, but the deforming behavior in regions distant
from the features can only be estimated, so it is usually forced to be
as smooth as possible. There are a number of interpolation meth-
ods that ensure smooth spatial transforms, but the most commonly
used approaches generally involve some form of radially symmetric
basis functions, which are centered at the landmarks.
Intensity based approaches operate by identifying a spatial
transformation that optimizes some voxel-similarity measure
between template data and the subject’s image. The template
defines the standard space to which all the subjects’ data are warped
during spatial normalization. Typically, the spatial transformation
that best matches the template to a subject’s anatomical image is
estimated using an iterative optimization procedure.
Image registration uses a mathematical model to explain the
data. Such a model will contain a number of unknown parameters
that describe how an image is deformed, or warped. The objective
is usually to determine the best possible values for these parameters
by optimizing some objective function; in other words, to find the
single most probable deformation, given the data. In such cases,
the objective function can be considered as a measure of this prob-
ability. A key element of probability theory is Bayes’ theorem:

p ( q | D) = p ( D | q) p ( q) / p ( D)

This posterior probability of the parameters, given the image data

(p(θ|D)) is proportional to the probability of the image data given
the parameters (p(D|θ)—the likelihood), times the prior probability
of the parameters (p(θ)). The probability of the data (p(D)) is
treated as a constant because the data are fixed and known. The
objective is usually to find the most probable parameter values, and
not the actual probability density, so this factor can be ignored.
The most probable set of values for the parameters is known as the
 Processing fMRI Data 167

maximum a posteriori (MAP) estimate. In practice, the objective

function is normally the logarithm of the posterior probability (in
which case it is maximized) or the negative logarithm (which is
minimized). The objective function can therefore be considered as
the sum of two terms: a likelihood term, and a prior term:

- log p ( q,D ) = - log p ( D | q ) - log p ( q )

The likelihood term is a measure of the probability of observing an

image given some set of model parameters. A simple example
would be where an image is modeled as a warped version of a tem-
plate image, but with Gaussian random noise added. Such a model
reduces to minimizing the sum of squared differences between the
image and warped template. It is possible that parts of the image
correspond to a region that falls outside the field of view of the
template, so it is usual to simply use the mean-squared difference
in the overlapping region.
The prior term reflects the prior probability of a deformation
occurring—effectively biasing the deformations to be realistic. If
one considers a model whereby each voxel can move independently
in three dimensions, then there would be three times as many
parameters to estimate as there are observations. This would sim-
ply not be achievable without regularizing the parameter estima-
tion by modeling a prior probability [39].
Registration is usually considered as an optimization proce-
dure, which involves searching for the model parameters that maxi-
mize or minimize the objective function. If the registration is based
on matching landmarks together, then it is often possible to regis-
ter the images in a single step, because the problem can be solved
by a single matrix inversion. In contrast, if image registration is
based on matching intensities, then some form of iterative scheme
is required. These procedures are usually very susceptible to poor
starting estimates, so a number of hybrid approaches have emerged
[40, 41] that combine intensity based methods with feature match-
ing (typically sulci). Registration methods usually attempt to find
the single most probable realization of all possible transformations.
Robust methods that almost always find the global optimum would
take an extremely long time to run with a model that uses millions
of parameters, so these methods are simply not feasible for prob-
lems of this scale. However, if sulci and gyri can be labeled easily
from the brain images, then these features can be used to bias the
registration, therefore increasing the likelihood of obtaining a
more globally optimal solution.
In practice, the parameters describing the spatial transforma-
tions, which map between a subject’s images and a standard coor-
dinate system, are usually estimated by matching a template with
an anatomical (typically T1-weighted) image. Providing the fMRI
data are in accurate alignment with the anatomical image, then
168 John Ashburner

the spatial transformation that would warp them to the standard

coordinate system is also known. Spatially normalized versions of
the fMRI data can simply be created using the same set of esti-
mated parameters.

5.1 Matching Criteria The matching criterion is often based upon minimizing the
mean-squared differences or maximizing the correlation between
the image and template. For this criterion to be successful, it
requires the individual’s image to have the visual appearance of a
warped version of the template. In other words, there must be
correspondence in the gray levels of the different tissue types
between the images. The mean-squared difference objective
function makes a number of assumptions. If the image data do
not meet these assumptions, then the objective function may not
accurately reflect the goodness of fit, and the estimated deforma-
tions will be suboptimal. Under some circumstances, it may be
better to weight different regions to a greater or lesser extent.
For example, when spatially normalizing a brain image contain-
ing a lesion, the mean squared difference around the lesion
should contribute little or nothing to the objective function [42].
This is currently achieved by assigned lower weights for the
matching criterion in these regions, so that they have much less
influence on the final solution.
In addition to modeling geometric deformations of the tem-
plate, there may also be extra parameters within the model that
describe intensity variability. A very simple example would be the
inclusion of an additional intensity scaling parameter, but the mod-
els can be much more complicated. There are many possible objec-
tive functions, each making a different assumption about the data
and requiring different parameterizations of the template intensity
distribution. For example, matching can be based on feature vec-
tors derived from the images [43], or can rely on some information
theoretic model [44]. There is no single universally best criterion
to use for all data.
Often, it is necessary to process the anatomical images prior to
any attempt to align them with a template. This may involve strip-
ping off non-brain tissue from the image, which can improve the
accuracy with which the brains themselves are registered. Because
the interesting signal predominantly arises in gray matter, another
strategy for increasing spatial normalization accuracy is to simply
spatially normalize the data by aligning gray matter with a gray
matter template image. A slightly better approach would involve
simultaneously matching gray with gray and white with white.
There are a number of readily available tissue segmentation algo-
rithms that can be used for identifying gray matter in brain MRI.
Another strategy that can be of great benefit for increasing
registration accuracy is the correction of intensity nonunifor-
mity artifact [33, 34], which would otherwise prevent accurate
 Processing fMRI Data 169

alignment. Such correction algorithms may be standalone [45],

or they may be incorporated within tissue segmentation proce-
dures [46, 47]. Some deformable registration procedures explic-
itly incorporate bias correction [48], but recent developments
combine bias correction, warping and tissue segmentation into
the same model [49–53]. Within such unified generative mod-
els, the registration and bias correction inform the tissue seg-
mentation, and the tissue segmentation informs the registration
and bias correction.
Currently, most spatial normalization algorithms use only a
single image from each subject, which is typically T1-weighted.
Such images only really delineate different tissue types. Further
information that may help the registration could be obtained from
other data such as diffusion weighted images [54]. These provide
anatomical information more directly related to connectivity and
implicitly function, possibly leading to improved registration of
functionally specialized areas. Matching diffusion images of a pair
of subjects together is likely to give different deformation estimates
than would be obtained through matching T1 weighted images of
the same subjects. The only way to achieve an internally consistent
match is through performing the registrations simultaneously,
within the same model. Similarly, the patterns of bold signal across
subjects could, in principle, be used to drive the registration—
although a naïve implementation of such an approach may cause
problems for interpreting group results. We are now beginning to
see an interest in driving deformable brain registration using
resting-state data (e.g., [55]).
The choice of template data used for spatial normalization is
important. It is sometimes tempting to base a template on the
brain of a single individual, but such a procedure would produce
different results depending upon the choice of whose brain was
used. One could consider an optimal template being some form of
average [56–58]. On average, registering such a template with a
brain image generally requires smaller (and therefore less error
prone) deformations. Such averages generally lack some of the
detail present in the individual subjects. Structures that are more
difficult to match are generally slightly blurred in the average,
whereas the structures that can be more reliably matched are
sharper. Such an average generated from a large population of sub-
jects would be ideal for use as a general purpose template. Another
reason for using a template that better represents the study popula-
tion is that it does not bias the results more towards some brain
regions than others. During spatial normalization of a brain image,
some regions need to be expanded and other regions need to con-
tract in order to match. If some brain structure is especially small
in the template, then this region will be contracted in the brains in
the study, leading to a systematic reduction in the amount of
BOLD signal detected from it.
170 John Ashburner

5.2 Deformation At its simplest, deformable image registration involves estimating a

Models smooth, continuous mapping between the points in one image and
those in another. This mapping allows one image to be re-sampled
so that it is warped (deformed) to match another (see Fig. 3).
There are many ways of modeling such mappings, but these fit into
two broad categories [59].

Horizontal component Vertical component Original image

of displacement of displacement with deformed grid

Horizontal component Vertical component Warped image

of deformation of deformation with regular grid

Fig. 3 2D displacements generated from two scalar fields. The first two panels pf the top row show displace-
ments represented as images. Below these are different representations of these components. The deforma-
tion field resulting from combining the components is overlaid on the top-right image, in order to deform the
image as shown at the bottom-right
 Processing fMRI Data 171

● The small-deformation framework does not necessarily preserve

topology4—although if the deformations are relatively small,
then there may still be a one-to-one mapping between the
images. This framework usually models deformations by a
smooth displacement field.
● The diffeomorphic framework generates deformations that
have a number of useful properties, such as enforcing the pres-
ervation of topology [60]. Within this framework, deforma-
tions are parameterized in terms of smooth velocity fields.
Images can be treated as continuous functions of space.
Reading the value at some arbitrary point involves interpolating
between the original voxels. For many interpolation methods, the
functions are parameterized by linear combinations of basis func-
tions, such as B-spline bases, centered at each original voxel.
Similarly, the deformations themselves can also be parameterized
by linear combinations of smooth, continuous basis functions.
A potentially enormous number of parameters are required to
describe the deforming transformations that align two images (i.e.,
the problem can be very high-dimensional). However, much of the
spatial variability can be captured using just a few parameters.
Sometimes only an affine transformation is used to approximately reg-
ister images of different subjects. This accounts for differences in posi-
tion, orientation and overall brain dimensions, and often provides a
good starting point for higher-dimensional registration models.
Some of the more primitive deformable registration algorithms
encode displacements via a linear combination of a relatively small
numbers of basis functions. One such approach is part of the AIR
package [61, 62], which uses polynomials (see Fig. 4) to model
shape variability. Other models parameterize a displacement field,
which is added to an identity transform. Families of basis functions
for such models include Fourier bases [63], sine and cosine trans-
form basis functions, which were used by early versions of the
Statistical Parametric Mapping (SPM) software [64] (see Fig. 4).
These models involve in the order of about 1000 parameters, and
only permit the global head or brain shape to be modeled.
Radial basis functions are another family of parameterizations,
which are often used in conjunction with an affine transformation.
Each radial basis function is centered at some point and the ampli-
tude is then a function of the distance from that point. Thin-plate
splines are one of the most widely used radial basis functions for
image warping and are especially suited to manual landmark match-
ing [65, 66]. The landmarks may be known, but interpolation is

4
The word “topology” is used in the same sense as in “Topological Properties
of Smooth Anatomical Maps” [15]. If spatial transformations are not one-to-
one and continuous, then the topological properties of different structures
can change.
172 John Ashburner

Fig. 4 Polynomial basis functions (left) and Cosine transform basis functions (right). Horizontal and vertical (and
through-plane for 3D) displacement fields may be modeled by linear combinations of such functions (see Fig. 3)

needed in order to define the mapping between these known points.

By modeling it with thin-plate splines, the mapping function has
the smallest bending energy. Other choices of basis function reduce
other energy measures, and these functions relate to the convolu-
tion filters that are sometimes used for image matching [67, 68].
B-spline bases are also used for parameterizing displacements
[11, 69, 70] (see Fig. 5). They are related to the radial basis func-
tions in that they are centered at discrete points, but the amplitude
is the product of functions of distance in the three orthogonal
directions (i.e., they are separable). The separability and local sup-
port of these functions confers certain advantages in terms of being
able to rapidly generate displacement fields through a convolution-
like procedure. Very detailed displacement fields can be generated
by modeling an individual displacement at each voxel. This may not
appear to be a basis function approach, but the assumptions within
such models are often that the fields are tri-linearly interpolated.
This is the same as a first degree B-spline basis function model.
Regularization is generally based on some measure of defor-
mation smoothness. Smoother deformations are deemed to be
more probable—a priori—than deformations containing a great
deal of detailed information. The regularization term (prior term)
of the objective function is often thought of as an “energy density”.
Commonly used forms for this are the membrane energy, bending
energy or linear-elastic energy. The form of the prior used by the
registration will influence the estimated deformations (see Fig. 6).
Small deformation approaches do not guarantee that the esti-
mated warps will be one-to-one and invertible. It is easy to
 Processing fMRI Data 173

Fig. 5 B-spline basis functions allow more detailed warps to be estimated than polynomial or cosine transform
basis functions

introduce folding within this simple setting, where a point in one

image may appear to align with two or more points in the other. For
more biologically plausible results, it is useful to constrain the warps
to be one-to-one by working within a diffeomorphic5 setting. The
key concept of this framework is that the deformations are generated
by the composition of a series of much smaller deformations (i.e.,
warped warps). For deformations, the composition operation is

5
A diffeomorphism is a globally one-to-one (bijective) smooth and continu-
ous mapping with derivatives that are invertible (i.e., non-zero Jacobian
determinant).
174 John Ashburner

Fig. 6 This figure illustrates the effect of different types of regularization. The top row on the left shows simu-
lated 2D images of a circle and a square. Below these is the circle after it has been warped to match the
square, using both membrane and bending energy priors. These warped images are almost visually indistin-
guishable, but the resulting deformation fields using these different priors are quite different. These are shown
on the right, with the deformation generated with the membrane energy prior shown above the deformation
that used the bending energy prior

achieved by re-sampling one deformation field by another. Providing

the constituent deformations are small enough, then they are likely
to be one-to-one. A composition of a pair of one-to-one mappings
will produce a new mapping that is also one-to-one. Multiple nest-
ing can be achieved, so that large one-to-one deformations can be
obtained from the composition of many very small ones. From a
mathematical perspective, diffeomorphic deformations are the result
of integrating differential equations over a unit of time, in which the
deformations are a function of smooth continuous velocity fields.
The composition of a series of small deformations can be viewed as
an Euler integration of these differential equations.
Early diffeomorphic registration approaches were based on the
greedy “viscous fluid” registration method of Christensen and Miller
[71, 72], which models one image as it “flows” to match the shape
of the other. These methods can account for large deformations
while ensuring that the topology of the warped image is preserved.
Their disadvantage is that they are not formulated to find the
smoothest deformation. More recent algorithms for large deforma-
tion registration do aim to find the smoothest solution. For exam-
ple, the LDDMM (Large Deformation Diffeomorphic Metric
Mapping) algorithm [73] does not fix the deformation parameters
 Processing fMRI Data 175

once they have been estimated. It continues to update them such

that the objective function is properly optimized. Such approaches
essentially parameterize the model by velocities, and compute the
deformation as the medium warps over unit time. A number of other
variations on the diffeomorphic framework have emerged in recent
years. These include the very popular ANTS/Syn software [74], a
number of algorithms that compute deformations by applying a
“scaling and squaring” procedure to a single velocity field [75–78],
as well as some approaches that try to replicate the deformations of
LDDMM using a “geodesic shooting” approach [79–81].
Optimization problems for complex nonlinear models, such as
those used for image registration, can easily get caught in local
optima; so there is no guarantee that the estimate determined by
the algorithm is globally optimal. If the starting estimates are suf-
ficiently close to the global optimum, then a local optimization
algorithm is more likely to find the globally optimal solution.
Therefore, the choice of starting parameters can influence the
accuracy of the final registration result. One method of increasing
the likelihood of achieving a good solution is to gradually reduce
the amount of regularization. Registration is first performed using
heavy regularization. Once this solution is found, then the proce-
dure is repeated using less regularization, and so on. This has the
effect of making the registration estimate the more global defor-
mations before estimating more detailed ones. The images could
also be smoother for the earlier iterations in order to reduce the
amount of confounding information and the number of local
optima. A review of such approaches can be found in [82].
The accuracy of a number of deformable registration methods
has been assessed using T1-weighted brain MRI, using manually
labelled brain structures as a ground truth with which to compare
[83]. The general trend was that those registration methods with
most flexibility (i.e., lots of parameters) tended to outperform
those using relatively few parameters. Figure 7 shows the average
of 550 T1-weighted MRI scans, which have been spatially normal-
ized using a relatively flexible registration approach.

6 Smoothing

Usually, the final step of the processing pipeline is to smooth the

images, which involves convolving the data with a three dimen-
sional Gaussian kernel (see Fig. 8). The amount of smoothing is
defined by the full width at half maximum (FWHM) of the
smoothing kernel. A broader FWHM produces smoother results,
and the choice of FWHM is determined by many factors.
More smoothing is usually used prior to statistical analyses of
group studies, than would be used for studies of single individuals.
Inter-subject registration is generally less accurate than the rigid-body
176 John Ashburner

Fig. 7 An average of 550 spatially normalized T1-weighted images

Fig. 8 This figure shows the effect of convolving the image on the left with different kernels. In the center, the
image has been convolved with a circular kernel. The result is an image in which each pixel is the average of
the values from the original image within the radius of the kernel. At the right is a result from convolving with
a Gaussian kernel. Pixels in this image are weighted averages, where the weights depend on the distance from
the center of the kernel
 Processing fMRI Data 177

registration that is used within subject. If homologous functional

regions in different subjects are not well aligned, then the activations
will appear in different places in different subjects. Smoothing is used
in order to increase the amount of overlap across subjects, and so
increase the significance of the results. The optimal amount of
smoothing therefore depends upon the alignment accuracy.
Less smoothing would typically be used for single-subject anal-
yses. There are various reasons for smoothing such data, but the
main one relates to the spatial frequencies of the interesting signal,
compared to the noise. If the noise contains proportionally more
high frequency than the signal, then it makes sense to remove some
of the high frequencies by smoothing. Another reason for smooth-
ing is that it reduces the effective number of independent statistical
tests that are performed. This can lead to greater sensitivity in
results that are corrected for multiple comparisons. The disadvan-
tage would be that localization of activations is less precise.
Some of the more recent approaches for the analysis of fMRI
data involve spatiotemporal models of activation. Such analyses do
not require the data to be smoothed, as the models themselves deal
with this issue.

7 Summary and Conclusions

The current fMRI data analysis paradigm involves applying a pipe-

line of tools to the data. One procedure is applied to the images,
to produce some output. Then another procedure is applied to
the output of that, and so on. The end result is a version of the
data that has been massaged into a form suitable for applying sim-
ple statistical tests to. A number of centers have developed pipe-
line environments [84–86] to facilitate such processing streams.
An alternative and more principled approach is to consider a full
model of how the data could have arisen. Such a generative model
would involve components for modeling the physics of the scanner,
the motion of the subjects, the brain-shape variability among the pop-
ulation, and models for how the experiment elicits changes in the data.
It would then be used to model the raw data, in order to make the
kinds of inferences in which neuroscientists are interested. A full model
for everything is a long way off, but scientists are making progress in
terms of simulating fMRI data in individual subjects [87]. Once a sim-
ulation model can be made, then it is simply a matter of inverting it, to
make the necessary inferences. Such models usually include a number
of parameters that influence the data, but are of no interest in them-
selves. Accurate model inversion would require the effects of these
uninteresting variables to be “integrated out”, which is not a straight-
forward procedure and is an area that occupies much of current meth-
odological research. Choosing an optimal model for a data-set would
essentially be a form of model selection, and could be done empirically
by determining which has the greatest supporting evidence.
178 John Ashburner

Many of the issues that concern users are related to the

un-modeled interactions that arise through the sequential applica-
tion of tools to the data. For example, when motion correction is
applied to an fMRI time series, the interesting signal elicited by the
experiment will influence how the algorithm estimates subject
motion. Similarly, interactions between image artifacts and subject
movement can also lead to problems, which can only be reduced
by including artifact correction within the motion correction.
There are many other examples of where combining procedures
could produce similar benefits.

References
1. Talairach J, Tournoux P (1988) Coplanar 11. Studholme C, Constable RT, Duncan JS
stereotaxic atlas of the human brain. Thieme (2000) Accurate alignment of functional EPI
Medical, New York data to anatomical MRI using a physics-based
2. Evans AC, Collins DL, Milner B (1992) An distortion model. IEEE Trans Med Imaging
MRI-based stereotactic atlas from 250 young 19(11):1115–1127
normal subjects. Soc Neurosci Abstr 18:408 12. Kybic J, Thévenaz P, Nirkko A, Unser M (2000)
3. Evans AC, Collins DL, Mills SR, Brown ED, Unwarping of unidirectionally distorted EPI
Kelly RL, Peters TM. 3D statistical neuroana- images. IEEE Trans Med Imaging 19(2):80–93
tomical models from 305 MRI volumes. In: Proc 13. Li Y, Xu N, Fitzpatrick JM, Morgan
IEEE Nuclear Science Symposium and Medical VL, Pickens DR, Dawant BM (2007)
Imaging Conference. 1993. pp 1813–1817 Accounting for signal loss due to dephas-
4. Jezzard P, Clare S (1999) Sources of distor- ing in the correction of distortions in
tion in functional MRI data. Hum Brain Mapp gradient-echo EPI via nonrigid reg-
8(2):80–85 istration. IEEE Trans Med Imaging
5. Hutton C, Bork A, Josephs O, Deichmann R, 26(12):1698–1707
Ashburner J, Turner R (2002) Image distor- 14. Hajnal JV, Mayers R, Oatridge A, Schwieso JE,
tion correction in fMRI: a quantitative evalua- Young JR, Bydder GM (1994) Artifacts due to
tion. Neuroimage 16(1):217–240 stimulus correlated motion in functional imag-
6. Cusack R, Papadakis N (2002) New robust 3-D ing of the brain. Magn Reson Med 31:289–291
phase unwrapping algorithms: application to 15. Thévenaz P, Blu T, Unser M (2000)
magnetic field mapping and undistorting echo- Interpolation revisited. IEEE Trans Med
planar images. Neuroimage 16(3):754–764 Imaging 19(7):739–758
7. Jenkinson M (2003) Fast, automated, N‐ 16. Eddy WF, Fitzgerald M, Noll DC (1996)
dimensional phase‐unwrapping algorithm. Improved image registration by using Fourier
Magn Reson Med 49(1):193–197 interpolation. Magn Reson Med 36:923–931
8. Jenkinson M, Wilson J, Jezzard P (2004) A 17. Cox RW, Jesmanowicz A (1999) Real-time 3D
perturbation method for magnetic field calcu- image registration for functional MRI. Magn
lations of non-conductive objects. Magn Reson Reson Med 42:1014–1018
Med 52(3):471–477 18. Noll DC, Boada FE, Eddy WF (1997) A
9. Poynton C, Jenkinson M, Whalen S, Golby AJ, spectral approach to analyzing slice selection
Wells W III (2008) Fieldmap-free retrospective in planar imaging: optimization for through-
registration and distortion correction for EPI- plane interpolation. Magn Reson Med
based functional imaging. In: Proc Medical 38:151–160
Image Computing and Computer-Assisted 19. Andersson JLR, Hutton C, Ashburner J,
Intervention (MICCAI). Springer, Berlin- Turner R, Friston KJ (2001) Modeling geo-
Heidelberg, pp 271–279 metric deformations in EPI time series.
10. Poynton C, Jenkinson M, Wells W III (2009) NeuroImage 13:903–919
Atlas-based improved prediction of magnetic 20. Bannister PR, Brady JM, Jenkinson M (2007)
field inhomogeneity for distortion correction of Integrating temporal information with a non-
EPI data. In: Proc Medical Image Computing rigid method of motion correction for func-
and Computer-Assisted Intervention (MICCAI). tional magnetic resonance images. Image Vis
Springer, Berlin, pp 951–959 Comput 25(3):311–320
 Processing fMRI Data 179

21. Friston KJ, Williams S, Howard R, Frackowiak image contrast on functional MRI image regis-
RSJ, Turner R (1996) Movement-related tration. NeuroImage 67:163–174
effects in fMRI time-series. Magn Reson Med 33. Hou Z (2006) A review on MR image inten-
35:346–355 sity inhomogeneity correction. Int J Biomed
22. Freire L, Mangin JF (2001) Motion correc- Imag 2006, Article ID 49515, 11 pages.
tion algorithms of the brain mapping commu- doi:10.1155/IJBI/2006/49515
nity create spurious functional activations. In: 34. Vovk U, Pernus F, Likar B (2007) A review of
Insana MF, Leahy RM (eds) Proc Information methods for correction of intensity in homo-
Processing in Medical Imaging (IPMI), vol geneity in MRI. IEEE Trans Med Imaging
2082, Lecture Notes in Computer Science. 26(3):405–421
Springer, Berlin, pp 246–258 35. Fox PT (1995) Spatial normalization: origins,
23. Maclaren J, Herbst M, Speck O, Zaitsev M (2013) objectives, applications, and alternatives. Hum
Prospective motion correction in brain imaging: Brain Mapp 3:161–164
a review. Magn Reson Med 69(3):621–636 36. Mazziotta JC, Toga AW, Evans A, Fox P,
24. Woods RP, Mazziotta JC, Cherry SR (1993) Lancaster J (1995) A probablistic atlas of the
MRI-PET registration with automated algo- human brain: theory and rationale for its devel-
rithm. J Comput Assist Tomogr 17:536–546 opment. NeuroImage 2:89–101
25. Holden M, Hill DLG, Denton ERE, Jarosz 37. Sotiras A, Davatzikos C, Paragios N
JM, Cox TCS, Rohlfing T, Goodey J, Hawkes (2013) Deformable medical image registra-
DJ (2000) Voxel similarity measures for 3D tion: a survey. IEEE Trans Med Imaging
serial MR brain image registration. IEEE Trans 32(7):1153–1190
Med Imaging 19(2):94–102 38. Oliveira FP, Tavares JMR (2014) Medical
26. Collignon A, Maes F, Delaere D, Vandermeulen image registration: a review. Comput Methods
D, Suetens P, Marchal G (1995) Automated Biomech Biomed Engin 17(2):73–93
multi-modality image registration based on 39. Rohlfing T (2012) Image similarity and tissue
information theory. In: Bizais Y, Barillot C, Di overlaps as surrogates for image registration
Paola R (eds) Proc Information Processing in accuracy: widely used but unreliable. IEEE
Medical Imaging (IPMI). Kluwer Academic, Trans Med Imaging 31(2):153–163
Dordrecht, pp 263–274
40. Auzias G, Colliot O, Glaunes JA, Perrot
27. Wells WM III, Viola P, Atsumi H, Nakajima S, M, Mangin JF, Trouvé A, Baillet S (2011)
Kikinis R (1996) Multi-modal volume registra- Diffeomorphic brain registration under
tion by maximisation of mutual information. exhaustive sulcal constraints. IEEE Trans Med
Med Image Anal 1(1):35–51 Imaging 30(6):1214–1227
28. Maes F, Collignon A, Vandermeulen D, Marchal 41. Du J, Younes L, Qiu A (2011) Whole brain
G, Seutens P (1997) Multimodality image reg- diffeomorphic metric mapping via integration
istration by maximisation of mutual informa- of sulcal and gyral curves, cortical surfaces, and
tion. IEEE Trans Image Process 16:187–197 images. NeuroImage 56(1):162–173
29. Studholme C, Hill DLG, Hawkes DJ (1999) An 42. Brett M, Leff AP, Rorden C, Ashburner
overlap invariant entropy measure of 3D medi- J (2001) Spatial normalization of brain images
cal image alignment. Pattern Recogn 32:71–86 with focal lesions using cost function masking.
30. Pluim JPW, Maintz JBA, Viergever MA (2003) NeuroImage 14(2):486–500
Mutual-information-based registration of med- 43. Shen D, Davatzikos C (2002) HAMMER:
ical images: a survey. IEEE Trans Med Imaging hierarchical attribute matching mechanism for
22(8):986–1004 elastic registration. IEEE Trans Image Process
31. West J, Fitzpatrick JM, Wang MY, Dawant BM, 21(11):1421–1439
Maurer CR, Kessler RM, Maciunas RJ, Barillot 44. D’Agostino E, Maes F, Vandermeulen D, Suetens
C, Lemoine D, Collignon A, Maes F, Suetens P (2004) Non-rigid atlas-to-image registra-
P, Vandermeulen D, van den Elsen PA, Napel S, tion by minimization of class-conditional image
Sumanaweera TS, Harkness B, Hemler PF, Hill entropy. In: Barillot C, Haynor DR, Hellier
DLG, Hawkes DJ, Studholme C, Maintz JBA, P (eds) Proc Medical Image Computing and
Viergever MA, Malandain G, Pennec X, Noz Computer-Assisted Intervention (MICCAI),
ME, Maguire GQ, Pollack M, Pelizzari CA, vol 3216, Lecture notes in computer science.
Robb RA, Hanson D, Woods RP. Comparison Springer, Berlin-Heidelberg, pp 745–753
and evaluation of retrospective intermodality
brain image registration techniques. J Comput 45. Sled JG, Zijdenbos AP, Evans AC (1998) A
Assist Tomo 1997;21:554–566. non-parametric method for automatic correc-
tion of intensity non-uniformity in MRI data.
32. Gonzalez-Castillo J, Duthie KN, Saad ZS, Chu IEEE Trans Med Imaging 17(1):87–97
C, Bandettini PA, Luh WM (2013) Effects of
180 John Ashburner

46. Wells WM III, Grimson WEL, Kikinis R, Jolesz template estimation for computational anat-
FA (1996) Adaptive segmentation of MRI data. omy. In: Proc IEEE International Symposium
IEEE Trans Med Imaging 15(4):429–442 on Biomedical Imaging (ISBI). pp 173–176
47. van Leemput K, Maes F, Vandermeulen D, 58. Lorenzen P, Davis B, Gerig G, Bullitt E, Joshi
Suetens P (1999) Automated model-based S (2004) Multi-class posterior atlas formation
bias field correction of MR images of the brain. via unbiased Kullback-Leibler template esti-
IEEE Trans Med Imaging 18(10):885–896 mation. In: Barillot C, Haynor DR, Hellier
48. Studholme C, Cardenas V, Song E, Ezekiel P (eds) Proc Medical Image Computing and
F, Maudsley A, Weiner M (2004) Accurate Computer-Assisted Intervention (MICCAI),
template-based correction of brain MRI inten- vol 3216, Lecture notes in computer science.
sity distortion with application to dementia and Springer, Berlin, pp 95–102
aging. IEEE Trans Med Imaging 23(1):99–110 59. Miller MI (2004) Computational anatomy:
49. Fischl B, Salat DH, Busa E, Albert M, Dieterich shape, growth, and atrophy comparison via dif-
M, Haselgrove C, van der Kouwe A, Killiany R, feomorphisms. NeuroImage 23:S19–S33
Kennedy D, Klaveness S, Montillo A, Makris 60. Christensen GE, Rabbitt RD, Miller MI, Joshi
N, Rosen B, Dale AM (2002) Whole brain seg- SC, Grenander U, Coogan TA, Van Essen
mentation: automated labeling of neuroana- DC (1995) Topological properties of smooth
tomical structures in the human brain. Neuron anatomic maps. In: Bizais Y, Barillot C, Di
33:341–355 Paola R (eds) Proc Information Processing in
50. Fischl B, Salat DH, van der Kouwe AJW, Medical Imaging (IPMI). Kluwer Academic,
Makris N, Ségonne F, Quinn BT, Dale AM Dordrecht, pp 101–112
(2004) Sequence-independent segmentation 61. Woods RP, Grafton ST, Holmes CJ, Cherry
of magnetic resonance images. NeuroImage SR, Mazziotta JC (1998) Automated image
23:S69–S84 registration: I. General methods and intra-
51. Ashburner J, Friston KJ (2005) Unified seg- subject, intramodality validation. J Comput
mentation. NeuroImage 26:839–851 Assist Tomogr 22(1):139–152
52. Pohl KM, Fisher J, Levitt JJ, Shenton ME, 62. Woods RP, Grafton ST, Watson JDG, Sicotte
Kikinis R, Grimson WEL, Wells WM II (2005) NL, Mazziotta JC (1998) Automated image
A unifying approach to registration, segmenta- registration: II. Intersubject validation of lin-
tion, and intensity correction. In: Medical Image ear and nonlinear models. J Comput Assist
Computing and Computer-Assisted Intervention Tomogr 22(1):153–165
(MICCAI). Springer, Berlin, pp 310–318 63. Christensen GE (1999) Consistent linear elastic
53. D’Agostino E, Maes F, Vandermeulen D, transformations for image matching. In: Kuba
Suetens P (2006) A unified framework for A, Sámal M, Todd-Pokropek A (eds) Proc
atlas based brain image segmentation and reg- Information Processing in Medical Imaging
istration. In: Pluim JPW, Likar B, Gerritsen (IPMI), vol 1613, Lecture notes in computer
FA (eds) Proc Third International Workshop science. Springer, Berlin, pp 224–237
on Biomedical Image Registration (WBIR), 64. Ashburner J, Friston KJ (1999) Nonlinear spa-
vol 4057, Lecture notes in computer science. tial normalization using basis functions. Hum
Springer, Berlin-Heidelberg, pp 136–143 Brain Mapp 7:254–266
54. Zhang H, Yushkevich PA, Gee JC (2004) 65. Bookstein FL (1989) Principal warps: thin-
Registration of diffusion tensor images. In: plate splines and the decomposition of defor-
Proc IEEE Computer Society conference on mations. IEEE Trans Pattern Anal Mach Intell
Computer Vision and Pattern Recognition 11(6):567–585
(CVPR).pp 842–847. 66. Bookstein FL (1997) Quadratic variation of
55. Khullar S, Michael AM, Cahill ND, Kiehl KA, deformations. In: Duncan J, Gindi G (eds)
Pearlson G, Baum SA, Calhoun VD (2011) Proc Information Processing in Medical
ICA-fNORM: Spatial normalization of fMRI Imaging (IPMI), vol 1230, Lecture notes in
data using intrinsic group-ICA networks. computer science. Springer, Berlin-Heidelberg,
Front Sys Neurosci 5(93):1–18. pp 15–28
56. Joshi S, Davis B, Jomier M, Gerig G (2004) 67. Bro-Nielsen M, Gramkow G (1996) Fast fluid
Unbiased diffeomorphic atlas construction registration of medical images. In: Höhne
for computational anatomy. NeuroImage KH, Kikinis R (eds) Proc Visualization in
23:S151–S160 Biomedical Computing (VBC), vol 1131,
57. Davis B, Lorenzen P, Joshi S (2004) Large Lecture notes in computer science. Springer,
deformation minimum mean squared error Berlin, pp 267–276
 Processing fMRI Data 181

68. Thirion JP (1995) Fast non-rigid match- non-rigid registration using a stationary
ing of 3D medical images. Technical report velocity field. In: Proc IEEE Workshop on
no 2547. Institut National de Recherche en Mathematical Methods in Biomedical Image
Informatique et en Automatique Analysis (MMBIA). IEEE. pp 145–150
69. Rueckert D, Sonoda LI, Hayes C, Hill DLG, 79. Miller MI, Trouvé A, Younes L (2006)
Leachand MO, Hawkes DJ (1999) Nonrigid Geodesic shooting for computational anatomy.
registration using free-form deformations: J Math Imag Vis 24(2):209–228
application to breast MR images. IEEE Trans 80. Ashburner J, Friston KJ (2011) Diffeomorphic
Image Process 18(8):712–721 registration using geodesic shooting and
70. Thévenaz P, Unser M (2000) Optimization Gauss–Newton optimisation. NeuroImage
of mutual information for multiresolution 55(3):954–967
image registration. IEEE Trans Image Process 81. Vialard FX, Risser L, Rueckert D, Cotter CJ
9(12):2083–2099 (2012) Diffeomorphic 3D image registration
71. Christensen GE (1994) Deformable shape mod- via geodesic shooting using an efficient adjoint
els for anatomy. Doctoral Thesis. Washington calculation. Int J Comput Vis 97(2):229–241
University, Sever Institute of Technology 82. Lester H, Arridge SR (1999) A survey of hier-
72. Christensen GE, Rabbitt RD, Miller MI (1996) archical non-linear medical image registration.
Deformable templates using large deforma- Pattern Recogn 32:129–149
tion kinematics. IEEE Trans Image Process 83. Klein A, Andersson J, Ardekani BA, Ashburner
5:1435–1447 J, Avants B, Chiang MC, Christensen
73. Beg MF, Miller MI, Trouvé A, Younes L GE, Collins DL, Gee J, Hellier P, Song
(2005) Computing large deformation metric JH, Jenkinson M, Lepage C, Rueckert D,
mappings via geodesic flows of diffeomor- Thompson P, Vercauteren T, Woods RP,
phisms. Int J Comput Vis 61(2):139–157 Mann JJ, Parsey RV (2009) Evaluation of 14
74. Avants B, Gee JC (2004) Geodesic estima- nonlinear deformation algorithms applied to
tion for large deformation anatomical shape human brain MRI registration. Neuroimage
averaging and interpolation. NeuroImage 46(3):786–802
23:S139–S150 84. Fissell K, Tseytlin E, Cunningham D, Carter
75. Ashburner J (2007) A fast diffeomorphic CS, Schneider W, Cohen JD (2003) Fiswidgets:
image registration algorithm. Neuroimage a graphical computing environment for
38(1):95–113 neuroimaging analysis. Neuroinformatics
76. Hernandez M, Bossa MN, Olmos S (2007) 1(1):111–125
Registration of anatomical images using geo- 85. Rex DE, Maa JQ, Toga AW (2003) The LONI
desic paths of diffeomorphisms parameterized pipeline processing environment. NeuroImage
with stationary vector fields. In: Proc IEEE 19(3):1033–1048
11th International Conference on Computer 86. Zijdenbos AP, Forghani R, Evans AC (2002)
Vision (ICCV). IEEE. pp. 1–8. Automatic ‘pipeline’ analysis of 3-D MRI
77. Vercauteren T, Pennec X, Perchant A, Ayache data for clinical trials: application to mul-
N (2008) Symmetric log-domain diffeomor- tiple sclerosis. IEEE Trans Med Imaging
phic registration: a demons-based approach. 21(10):1280–1291
In: Proc Medical Image Computing and 87. Drobnjak I, Gavaghan D, Suli E, Pitt-Francis
Computer-Assisted Intervention (MICCAI). J, Jenkinson M (2006) Development of a
Springer, Berlin, pp 754–761 fMRI simulator for modelling realistic rigid-
78. Modat M, Daga P, Cardoso MJ, Ourselin S, body motion artifacts. Magn Reson Med
Ridgway GR, Ashburner J (2012) Parametric 56(2):364–380
 Chapter 7

Statistical Analysis of fMRI Data

Mark W. Woolrich, Christian F. Beckmann, Thomas E. Nichols,
and Stephen M. Smith

Abstract
fMRI is a powerful tool used in the study of brain function. It can noninvasively detect signal changes in
areas of the brain where neuronal activity is varying. This chapter is a comprehensive description of the
various steps in the statistical analysis of fMRI data. This will cover topics such as the general linear model
(including orthogonality, hemodynamic variability, noise modeling, and the use of contrasts), multisubject
statistics, and statistical thresholding (including random field theory and permutation methods).

Key words fMRI, Analysis, Statistics, General linear model, Multisubject statistics, Statistical
thresholding

1 Introduction

fMRI is a powerful tool used in the study of brain function. It can

noninvasively detect signal changes in areas of the brain where neu-
ronal activity is varying. fMRI can therefore give high-quality visual-
ization of the location of activity in the brain resulting from sensory
stimulation or cognitive function. It allows, for example, the study of
how the healthy brain functions, how different diseases affect the
brain, or how drugs can modulate activity or post-­damage recovery.
After an fMRI experiment has been designed and carried out,
the resulting data must be passed through various analysis steps
before the experimenter can get answers to questions about experi-
mentally related activations at the individual or multisubject level.
This chapter focuses on the statistical aspects of such analysis.
We need a statistical approach for two reasons. First, fMRI data
is very noisy. The noise is often of the same order of magnitude as
the fMRI signal changes we are trying to detect, and as such we
can only approximately estimate the signal changes. Statistics are
therefore needed to ask if the estimated signal changes are signifi-
cant, given the quality of the approximation. Second, many fMRI
studies are carried out with the intention of answering some

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_7, © Springer Science+Business Media New York 2016

183
184 Mark W. Woolrich et al.

­ uestion about a population of individuals. For example, we might

q
want to know what the difference in neural activation is between a
patient and control group. Such population differences in neural
activation can only ever be approximated, because not only is the
fMRI data noisy but also we only ever have a sample of subjects
from the populations. This issue is somewhat exacerbated in fMRI
studies as the number of subjects sampled is typically quite small!
Statistics are needed to see if the approximated population differ-
ences are significant given the quality of those approximations.
Figure 1 shows an illustration of the main analysis steps carried
out in a typical fMRI study. There are three main components in
the process. First, the individual subjects’ fMRI data must be pro-
cessed. Second, the information gleaned from this about the effect
sizes (the size of the fMRI signal change in response to the experi-
mental task) for each subject is then combined in a group analysis.
Finally, the group effect sizes are statistically thresholded to ask
questions such as “Where is there significant activity in response to
the experimental task for the population?” or “Where are there
significant differences between populations (e.g., controls versus
patients)?.” This final thresholding is carried out on statistic images
as it takes into account the spatial characteristics of the data. Note
that one could also perform thresholding on the effect size statis-
tics from a single-subject’s analysis, allowing one to ask questions
such as “Where is there significant activity in response to the exper-
imental task for this subject?” The various steps in the analysis will
be described in detail throughout the chapter.

2 Statistical Analysis of a Single fMRI Dataset

Later in the chapter, we will see how we go about asking statistical

questions about populations of subjects. However, before this can
be done, the individual subject fMRI data must be analyzed. The
most common way of statistically analyzing fMRI data is by using
a general linear model (GLM). As we shall see, this is a powerful
framework that allows a wide range of different statistical questions
to be asked about the data.

2.1 fMRI Data In a typical fMRI imaging session, a low-resolution functional vol-
ume is acquired every few seconds (MR volumes are often also
referred to as “images” or “scans”). Over the course of the experi-
ment, 100 volumes or more are typically recorded. In the simplest
possible experiment, some images will be taken while stimulation1 is
applied, and some will be taken with the subject at rest. Because the
images are taken using an MR sequence which is sensitive to changes

1
For the remainder of this chapter, reference to “stimulation” should be taken
to include also the carrying out of physical or cognitive activity.
 Statistical Analysis of fMRI Data 185

Fig. 1 Illustration of the analysis steps carried out in a typical fMRI group study. There are three main compo-
nents in the process. First, the individual subjects’ fMRI data must be processed (top). The information gleaned
from this about the effect sizes (the size of the fMRI signal change in response to the experimental task) for
each subject are then combined in a group analysis (middle). The group effect sizes statistic images are then
statistically thresholded to find significant brain areas (bottom)
186 Mark W. Woolrich et al.

Fig. 2 What are voxels? Shown here are surface renderings of 3D brain images. On the left is a high-resolution
image, with small (0.5 × 0.5 × 0.5 mm) voxels; the voxels are too small to see. On the right is a low-resolution
image of the same brain, with large (5 × 5 × 5 mm) voxels, clearly showing the voxels making up the image

in local blood oxygenation level, parts of the images taken during

stimulation should show increased intensity, compared with those
taken while at rest. The parts of these images that show increased
intensity should correspond to the brain areas that are activated by
the stimulation. The goal of fMRI analysis is to detect, in a robust,
sensitive, and valid way, those parts of the brain that show changes
in intensity at the points in time that the stimulation was applied.
A single volume is made up of individual cuboid elements
called voxels (Fig. 2). An fMRI dataset from a single session can be
thought of either as t volumes, one taken every few seconds, or as
v voxels, each with an associated time series of t time points. It is
important to be able to conceptualize both of these representa-
tions, as some analysis step make more sense when thinking of the
data in one way and others make more sense the other way.
An example time series from a single voxel is shown in Fig. 3.
Image intensity is shown on the y-axis and time (in scans) on the
x-axis. As described above, for some of the time points, stimulation
was applied (the higher intensity periods), and at some time points,
the subject was at rest. As well as the effect of the stimulation being
clear, the high-frequency noise is also apparent. The aim of fMRI
analysis is to identify in which voxels’ time series the signal of inter-
est is significantly greater than the noise level.

2.2 Preparing fMRI Initially, a 4D dataset is pre-processed. This pre-processing is aimed

Data for Statistical at not only removing artifacts and reducing noise but also
Analysis ­conditioning the data so that it is more amenable to the statistical
analysis that is to follow.
The most basic required steps that will be typically carried out
are as follows. Once data has been acquired by the MR scanner, the
pre-processing starts by reconstructing the raw “k-space” data into
 Statistical Analysis of fMRI Data 187

Fig. 3 An example time series at a strongly activated voxel from a visual stimulation experiment. Here the
signal is significantly larger than the noise level. Periods of stimulation are alternated with periods of rest—a
complete stimulation—rest cycle lasts 20 scans

images that actually look like brains. The data is then motion cor-
rected, where each volume is transformed (using rotation and
translation) so that the image of the brain within each volume is
aligned with that in every other volume. Spatial smoothing is then
carried out, principally to reduce noise, hopefully without signifi-
cantly affecting the activation signal. Finally, each voxel’s time
series is temporally high-pass filtered with a filter designed to remove
the large amount of low-frequency temporal noise found in FMRI
data, without removing the signal of interest.
Chapter 6 has already covered fMRI pre-processing in much
more detail, including other optional steps that have not been
mentioned here.

2.3 Predicting The first step in the statistical analysis is to come up with a good
the Response prediction, or model, of what we think the measured fMRI signal
response will look like in voxels that are active. In the simplest type
of fMRI experiment, we alternate periods of stimulation with peri-
ods of rest, in what we will refer to as a square-wave block design,
as shown in Fig. 4 (left). We expect that a voxel which is active in
response to the stimulus will contain an fMRI signal that generally
fluctuates up and down with a time course that is similar to the
stimulus time course (Fig. 4 left), whereas an inactive voxel will not.

2.4 Hemodynamic However, can we come up with a better prediction of the fMRI
Response Function signal than Fig. 4 (left)? In particular, we know, from experiment,
that the response to a very short stimulus looks like the curve
shown in Fig. 5. We refer to this response to an impulse of stimulus
188 Mark W. Woolrich et al.

Fig. 4 Predicting the response using the known stimulus timings. This example is a square-wave block design
where blocks of stimulation are alternated with blocks of rest. The square-waveform (left) describes the input
stimulus timing; the predicted response (middle) results from convolving the stimulus time course with the
hemodynamic response function and then sampling it at the temporal resolution of the experiment. This
experiment has a repetition time (TR) of 2 s. This process produces a model, or predicted response, that looks
much more like the data measured in voxels that are responding to the stimulus (right)

Fig. 5 The hemodynamic response function. A brief impulse of stimulation at

t = 0 s causes a blood oxygen level dependent (BOLD) signal that is delayed and
blurred. Here it is modeled as a double-gamma function

as the hemodynamic response function (HRF), and it is basically a

delayed and blurred version of the short stimulus burst. This is
because the variations that we can detect in blood oxygen level
dependent (BOLD) fMRI signal are due to processes taking place
in the vasculature: things such as the amount of blood oxygen-
ation, blood flow, and blood volume change when neural activa-
tion increases or decreases. Unsurprisingly, these vascular changes
occur on a slower timescale than the neural activity. Note that there
are also more subtle characteristics of the HRF. For example, as
 Statistical Analysis of fMRI Data 189

can be seen in Fig. 5, there can be a post-stimulus undershoot as

the HRF temporarily drops below baseline before rising back to
zero. The HRF in Fig. 5 is a commonly used HRF, and is a double-­
gamma function of the form:
G ( m1 ,s 12 ) - G ( m2 ,s 22 )
h (t ) = (1)
r

where h(t) is the HRF as a function of time, G(μ, σ2) is a Gamma

distribution parameterized by its mean, μ, and variance, σ2 (note
that we can convert these to the traditional Gamma distribution
parameters using α = μ2/σ2 and β = σ2/μ), and ρ is the ratio of the
height of the positive Gamma to the negative Gamma.2
The most straightforward way of incorporating the HRF into
our predicted response is to apply its delaying and blurring effect to
the raw stimulus time course that we have in our fMRI experiment.
This is achieved by the mathematical operation of convolution:
¥
x ( t ) = òh (t ) s ( t - t ) dt (2)
0

This essentially assumes that the effects of the different impulses that
make up the stimulus time course add together in an additive, linear
fashion [1–4]. Figure 4 (middle) shows the result of convolving the
HRF in Fig. 5 with the square-wave stimulus in Fig. 4 (left) to form
our new improved predicted response. Strictly speaking, making the
assumption of linearity is incorrect. We will see later how we can
address this issue, and also discuss how to tell if that is necessary.
For now, using the HRF and convolving it with the stimulus
time course provides us with a way in which we can come up with a
reasonable prediction of the response for any general stimulus type.
For example, Fig. 6 shows the predicted response for a sparse single-
event design and a dense randomized single-event design. Armed
with our predicted response, we can then look to find those voxels
that have fMRI time courses that match the predicted response well
and, if they pass a statistical test, label them as being active voxels.

3 General Linear Modeling

We have so far discussed how to come up with a prediction of the

fMRI signal in response to an experimental stimulus. However,
what do we do when we have more than one stimulus switching on
or off throughout the experiment? The answer is to use a GLM. This
assumes that each of the stimuli have their own predicted response,
and that these predicted responses then add together linearly in

2
The particular HRF in Fig. 5 has parameter values μ1 = 6 s, σ1 = 2.45 s,
μ2 = 16 s, σ2 = 4 s, and ρ = 6.
190 Mark W. Woolrich et al.

Fig. 6 Using the hemodynamic response function (HRF) and convolving it with the stimulus time course pro-
vides a way in which we can predict the response for any general stimulus type. Here we can see the predicted
response for a sparse single-event design, where short bursts of 0.1 s stimulation are 25 s apart (top), and for
a randomized single-event design, where short bursts of 0.1 s stimulation are separated with inter-stimulus
intervals (ISIs) sampled from a Poisson distribution with mean of 7 s (bottom). Randomized single-event
designs are an excellent way of working with stimuli that by their nature need to be single events, as they
generally have better sensitivity than sparse single-event designs [5]

some combination unique to each voxel, to explain the data mea-

sured in that voxel. For example, consider that we have two experi-
mental stimuli: one auditory and one visual. Both are square-wave
block designs, but they switch on and off at different times. The
overall predicted response is given by a linear combination of the
predicted responses:

y ( t ) = b1 x1 ( t ) + b 2 x2 ( t ) + c + e ( t ) (3)

where y(t) is the data in one voxel, and is a 1D vector (time course)
of intensity values with one value for each time point. x1(t) and
x2(t) are the predicted responses for our auditory and visual experi-
mental stimuli, respectively, and both are also 1D time courses
with one value for each time point. c is a constant and would cor-
respond to the mean intensity value in the data. The linear combi-
nation of the predicted responses needed to explain the data in a
particular voxel is described by the parameters β1 and β2. e(t) mod-
els the noise that is present in fMRI data.
Model fitting involves adjusting the mean level, c, and the
parameters β1 and β2, to best fit the data. For example, if a particu-
lar voxel responds strongly to model x1, the model-fitting will find
a large value for β1; if the data instead looks more like the second
 Statistical Analysis of fMRI Data 191

model time course, x2, then the model-fitting will give β2 a large
value. The GLM is used to analyze each voxel’s time series inde-
pendently. This is often referred to as a mass univariate analysis,
and outputs statistics independently at each voxel.
There are a number of different of names that get used to
describe the different components of the GLM. The predicted
responses within a GLM are often referred to as explanatory vari-
ables (EVs), as they explain different processes in the data. They can
also be referred to as regressors, and the βs as regression parameters,
as we are performing what is also known as a multiple regression.
The regression parameters, β, are also sometimes referred to as
effect sizes, as they describe the size of the response to the corre-
sponding underlying experimental stimuli.3

3.1 Design Matrix The GLM is often formulated in matrix notation. All of the param-
eters are grouped together into a P × 1 vector β (where P is the
number of EVs) and all of the EVs are grouped together into an
N × P matrix X, often referred to as the design matrix, where N is
the number of time points in the experiment. This gives us the
GLM in the following form:

¡ = Xb +e (4)

where ϒ is the N × 1 vector of intensity values in the data, and e is the

N × 1 noise vector. You may wonder what has happened to the mean
parameter, c, in this new equation. There are two common ways in
which this is handled. The first is to remove the mean, or de-mean,
the data ϒ, and to also separately de-mean all of the EVs in the design
matrix. This is appropriate as there is no information in the mean
signal intensity of the fMRI data that can aid us in our statistical
analysis. The second is to leave the mean as part of the GLM by creat-
ing an EV that has the value of 1 at every time point. The β or regres-
sion parameter for this EV will be determined when we fit the GLM
to the data and will relate to the estimate of the mean parameter.
Figure 7 shows a design matrix for our example experiment
with two stimuli (auditory and visual). Each column in the design
matrix is a different part of the model. The left column (x1 or EV1)
models the auditory stimulation, and the right column (x2 or EV2)
models the visual stimulation.

3.2 Fitting the GLM The GLM is fit to the data at each voxel separately. This is achieved
to the Data by adjusting the estimates of the regression parameters to find the
best fit of the model to the data. Typically, it is assumed that the
fMRI noise, e, is well modeled by a Gaussian distribution with a

3
Note that this common usage is slightly different from the definition some-
times used in the statistics literature, where effect size means β divided by the
noise level.
192 Mark W. Woolrich et al.

Fig. 7 Example of the general linear model (GLM) for an experiment containing
auditory and visual stimuli which have different stimulus timings. The design
matrix, X, contains two predicted responses (also known as regressors or
explanatory variables): x1 for the auditory stimulus and x2 for the visual stimu-
lus—see Eq. (4). In this, visualization time is running downwards. It can be seen
that the particular voxel data shown, Y, is from a voxel that is strongly activating
in response to the visual stimulus modeled by x2, but not to the auditory stimulus
modeled by x1

standard deviation, σ, unique to each voxel. When this assumption

is made, the best fit of the model to the data is equivalent to mini-
mizing the sum (over all time points) of the squared difference
between the data, ϒ, and the signal model, Xβ. That is we choose
the β values that minimize:

å(¡ - Xt b )
2
t (5)
t

Mathematically, it can be shown that this is minimized when we

estimate the βs as:

b = ( X T X ) X T ¡
-1
(6)

where b  is our regression parameter estimate. In the example

visual/auditory experiment in Fig. 7, it can be seen that the particu-
lar voxel data shown, ϒ, is from a voxel that is strongly activating in
response to the visual stimulus modeled by x2, but not to the audi-
tory stimulus modeled by x1. This would result in a large value for β2
and a low value for β1 when the GLM is fit to this data, suggesting
that there is visual activation but no auditory activation in this voxel.

3.3 Temporal Until now we have considered a rather simple approach to dealing
Autocorrelation with the noise that is present in fMRI, by assuming that it is well
 Statistical Analysis of fMRI Data 193

modeled as coming from a Gaussian distribution. Unfortunately, in

practice, this is not the whole story. This is because fMRI noise, that
is, the signal we record in the absence of any stimulation, is tempo-
rally autocorrelated. In particular, in the gray matter, this corre-
sponds to the fMRI noise being temporally smooth. This is because
the nature of the many artifacts that make up fMRI noise, for exam-
ple, thermal noise, cardiac and respiratory rhythms, autoregulatory
oscillations, and networks of spontaneous neural activity, tend to
occur more at low frequency than at high frequency. We can see this
imbalance between low and high frequency if we look at a plot of
the power spectrum (the absolute value of a Fourier transform) of
fMRI noise, an example of which is shown in Fig. 8a.
The presence of temporal autocorrelation is a concern because
it affects the choice of the optimal estimation (model fitting)
method, and perhaps more important, the accuracy of the

Fig. 8 A summary of the process of pre-whitening. (a) Plot of the power spectrum of the residuals from an
initial general linear model (GLM) fit to the data from one voxel. (b) Estimated power spectrum (representing
the temporal autocorrelation estimate) obtained from fitting an autocorrelation model. (c) This spectrum is
inverted to create the frequency characteristics of a temporal filter designed to “undo” the autocorrelation. (d)
The pre-­whitening temporal filter is applied to both the data and the explanatory variables (EVs) in the design
matrix, and then this pre-whitened GLM is refit to the pre-whitened data. The residuals that result from this
refit of the GLM should now be approximately white, that is have a flat power spectrum
194 Mark W. Woolrich et al.

subsequent statistical tests. For example, if we ignore the fact that

there is an increased amount of noise at low frequency, then we can
underestimate the variability in the data and produce under-­
conservative statistical tests.
A number of strategies have been proposed for dealing with the
problem of temporally autocorrelated noise. Traditionally, the most
commonly used approach is pre-whitening [6–8]. The process of
pre-whitening is summarized in Fig. 8. The first step is to estimate
 , from an
the temporal autocorrelation on the residuals, r = ¡ - X b
initial GLM fit. The temporal autocorrelation estimate is then used
to construct a pre-whitening temporal filter designed to “undo”
the autocorrelation. In other words, the filter is designed to re-dress
the imbalance between high and low frequency in the fMRI noise
power spectrum, so that the new power spectrum has equal power
at all frequencies. By analogy, since white light is a result of there
being equal amounts of light at all frequencies/colors, we refer to
noise with equal power at all frequencies as white noise—and there-
fore to the whole approach as pre-­whitening. The pre-whitening
temporal filter is applied to both the data and the EVs in the design
matrix, and then this pre-whitened GLM is refitted.
A crucial step in pre-whitening is the estimation of the tempo-
ral autocorrelation. A wide range of approaches have been pro-
posed, including the use of auto-regressive (AR) models [6, 9], AR
plus white noise models [10], spectral smoothing [8, 11], and spa-
tial regularization of autocorrelation estimates [8, 12, 13].
The pre-whitening approach described above is one that was
designed to work in a classical statistical (“frequentist”) frame-
work. It is possible that inferring the autocorrelation on the residu-
als from an initial GLM fit can introduce inaccuracies, since the
residuals, r, only serve as an approximation to the true error, e.
More recently, alternative Bayesian strategies have been developed
to deal with this problem [14, 15]. These have the advantage of
inferring the autocorrelation characteristics at the same time as the
GLM regression parameters, and to take into account the uncer-
tainty in the temporal autocorrelation estimation. However, these
issues aside, they are essentially performing the same pre-­whitening
approach we have already described. Although computationally
more demanding these techniques are being increasingly used.

3.4 Inferring Neural When we fit a particular GLM to a particular voxel’s data, we get
Activity regression parameter estimates that indicate how much of each EV is
needed to explain what we see in the data. If the parameter estimate
of β for any particular EV is nonzero, then it might seem reasonable
to assume that the voxel in question is neuronally responding to the
stimulus that the EV represents. However, we only have estimates/
approximations of the true β obtained from a limited amount of
noisy fMRI data. So, given the amount of noise and the estimate of β
obtained, how much can we trust that any particular β is nonzero?
 Statistical Analysis of fMRI Data 195

It is only those voxels where we can satisfy ourselves in a statistical

manner that this is the case, that we label as being active. The first
step towards this is to convert the parameter estimates of β into a use-
ful statistic. Most commonly, we use a T-statistic, given by:

b
t= (7)
( )

std b

where the denominator is the standard deviation (uncertainty) of our

parameter estimate. If the parameter estimate is low relative to its
estimated uncertainty, the T-statistic, t, will be low, implying that β is
unlikely to be significantly nonzero (and vice versa). We will see later
what the standard deviation of our parameter estimate depends upon.
The question remains as to how we determine that a T-statistic
is significantly nonzero. This is achieved by comparing the calcu-
lated T-statistic to the distribution of T-statistics we would expect
to get if the true β value was zero. This is a null hypothesis test (the
null hypothesis is that β is zero).
As mentioned earlier, we typically assume that the noise in
fMRI is Gaussian distributed. This means that our expected
­distribution of T-statistics under the null hypothesis is T-distributed.
This is a standard statistical distribution for which the probability,
or P-value, of getting a T-value greater than the one we have cal-
culated if the null hypothesis were true, can easily be calculated. As
illustrated in Fig. 9, a low probability (low P-value) of the null

P-Value=0.98
P-Value=0.02
P-Value=0.2

−4 −2 0 t 4 −4 −2 0 t 2 4 −4 t 0 2 4
T T T

Fig. 9 Performing a T-test. The T-statistic we calculate for each regression parameter estimate is compared
with the distribution of T-statistics we would expect if the true regression parameter were zero. This “null
distribution” is a standard T-distribution. Here we are using a T-distribution with 100 DOF (this would corre-
spond to about 100 time points in the data). We calculate a probability, or P-value, as the proportion of the area
under the curve in the positive tail of the distribution defined by our T-statistic, t. A low probability, or P-value,
(left plot) of the null hypothesis being true means that we can more confidently reject the null hypothesis and
label the voxel as having a nonzero β, and therefore as being neuronally activated by the stimulus that the β
in question represents. T-statistics that have larger P-values (middle plot) or are deep into the negative tail of
the distribution (right plot) have high P-values. In the latter case, this might seem a bit counterintuitive as we
have a T-statistic that is in the extremities of the null distribution. However, this is because the test is direc-
tional and so we calculate the P-value by looking in the positive tail of the distribution only
196 Mark W. Woolrich et al.

hypothesis being true means that we can more confidently reject

the null hypothesis and label the voxel as having a nonzero β. We
will see in Sect. 7 how we choose a threshold for the P-values.

3.5 Contrasts So far, we have addressed how we might go about producing

T-statistics and P-values, which describe how strongly each voxel is
related to each EV in our design matrix. Contrasts are a framework
whereby we can ask not just questions about each EVs parameter
estimate (PE, or β) in isolation, but also a wider range of questions
that compare the different parameter estimates with each other.
In general, a contrast is defined by a P × 1 vector, c, (recall that
P is the number of EVs in the design matrix). This is multiplied by
the P × 1 vector of parameter estimates, b  , to give what is known
T
as a contrast of parameter estimates (COPEs), c b . The COPE is
therefore just a linear combination of the parameters estimates; it
is equal to the sum of each PE multiplied by the relevant number
in the contrast vector.
For example, it may be desirable to compare two different PEs
to test directly whether one EV is more “relevant” to the data than
another EV. In our combined auditory and visual experiment, this
would be asking “Where does the brain respond more strongly to
the auditory stimulus compared with the visual stimulus?”. In this
example, we have two EVs, one for the auditory stimulus and one
for the visual (recall Fig. 7). So our contrast to answer this question
would be cT = [1 − 1] (c is a column vector, hence the transpose
here), resulting in a COPE, cTβ = 1β1 − 1β2.
The COPE is then simply treated as if it were itself an individ-
ual regression parameter estimate. In other words, in the same
manner as Eq. (7), we calculate a T-statistic by dividing the COPE
by its standard deviation:


cT b
t= (8)
( )

var c T b

where the T-distribution in question has degrees of freedom

(DOF) of N − P (recall that N is the number of time points in the
experiment), and:

( )  = s c T ( X T X ) c
-1
2
var c T b (9)

where σ2 is the estimate of the variance of the fMRI noise:

rTr
s =
2
(10)
( N - P)
where r is the residual, that is, an estimate of the error, e, and is what
.
is left over after the model is fit to the data, and is given by r = ¡ - X b
 Statistical Analysis of fMRI Data 197

As with the T-test on a single PE, we can calculate a probability,

or P-value, of getting the calculated T-statistic under the null
hypothesis in the same manner as in Fig. 9. The null hypothesis is
that the COPE = 0, so if the calculated P-value is low then this sug-
gests that the it is unlikely that the COPE = 0. If we apply a cT = [1 − 1]
contrast in the auditory/visual experiment, then this is equivalent
to saying “the voxel is responding more strongly to the auditory
stimulus compared with the visual stimulus.” We can infer that it is
the auditory that is stronger than the visual, and not vice versa,
because these T-tests on these contrasts are directional. Technically,
this is because we are doing the null-hypothesis test in Fig. 9 on one
tail (in particular, the right-hand tail) of the distribution only. As
shown in Table 1, if we want to ask “Where does the brain respond
more strongly to the visual stimulus compared with the auditory
stimulus?,” then we would use cT = [−1 1]. All that remains to com-
plete the null hypothesis test is a choice of threshold, such that if the
calculated P-value drops below that threshold, we reject the null
hypothesis and say that the contrast is significant. We shall see later
in Sect. 7 how we go about doing this while also taking into account
the spatial nature of the data.
Even in the relatively simple auditory/visual experiment, there
are a number of different questions that can be answered. See
Table 1 for some other examples.
Even in this relatively simple experiment, there are a number of
different questions that can be answered. In particular, it is impor-
tant to remember that the directionality of the T-test is important.
Note that:
EV1 is the auditory predicted response and EV2 is the visual pre-
dicted response.
COPE stands for contrast of parameter estimate and EV stands for
explanatory variable

3.5.1 F-Tests The last example contrast in Table 1 was a [1 1] contrast. It is a

common misconception that such a contrast asks “Where is there
significant activation due to either the visual or the auditory stimu-
lation?” In fact, this contrast calculates COPE = b  +b  , which is
1 2
proportional to the average value of the two regression parameters,
( )
b 1 + b 2 / 2 , and hence is actually asking “Where is there signifi-
cant activation averaged across both conditions?”
So how do we go about asking the question “I want to find
where there is significant activity due to either the visual or the
auditory stimulation”? The answer is to use F-tests. An F-test is
defined by specifying a set of contrasts that we want to test simul-
taneously. This then tests the null hypothesis that all of the COPEs
that are in the F-test are equal to zero. Therefore, we can find
significance (reject the null hypothesis) if any of the COPEs is non-
zero. Another perspective is that the F-test will find significance if
198 Mark W. Woolrich et al.

Table 1
Examples of contrasts that might be used in the two stimulus auditory/visual experiment

Contrast,
cT COPE, cTβ Meaning
[1 0] β1 Where is there significant auditory activation?
[0 1] β2 Where is there significant visual activation?
[−1 0] −β1 Where is there significant negative auditory activation?
[0 −1] −β2 Where is there significant negative visual activation?
[1 −1] β1 − β2 Where is there auditory activation significantly greater than visual activation?
[−1 1] β2 − β1 Where is there visual activation significantly greater than auditory activation?
[1 1] β1 + β2 Where is there significant activation averaged across both conditions?

there is any linear combination of the COPEs that explains a sig-

nificant amount of variance in the data.
So to ask “Where is there significant activity due to either the
visual or the auditory stimulation?,” we simply need to include in
an F-test the contrast that asks where there is significant activity
due to the auditory stimulation, [1 0], along with the contrast that
asks where there is significant activity to the visual stimulation,
[0 1]. Formally, this is done with an F-test contrast matrix, c, that
contains both of these contrasts:

æ1 0ö
cT = ç ÷.
è0 1ø

Note that in the T-tests, our contrasts were P × 1 vectors. Now,

F-tests are generally described as P × K contrast matrices, c, where
K is the number of contrasts in the F-test, and recall that P is the
number of regression parameters in the GLM. Using this contrast
matrix, we can then calculate an F-statistic:

( )
b c var c T b c T b
T

f = . (11)
K

Analagous to how the T-statistics were T-distributed under the

null hypothesis that the COPE is zero, this F-statistic is F-distributed
(with DOF K and N − P) under the null hypothesis that all of the
T
contrasts in the F-test are zero c b ( )
= 0 . As such, we can proceed
with a null-hypothesis test in exactly the same manner as we did
with the T-test.
 Statistical Analysis of fMRI Data 199

An important characteristic of F-tests is that they are blind to

the directionality of the contrasts that make up the test. In other
words, in our auditory/visual experiment example, the following
F-tests are all equivalent:

æ 1 0 ö æ -1 0 ö æ -1 0 ö æ 1 0 ö
ç ÷,ç ÷,ç ÷,ç ÷.
è 0 1 ø è 0 -1ø è 0 1 ø è 0 -1ø

This is because F-tests are testing if there is a linear combination of

the COPEs that explain a significant amount of variance in the
data, and when we consider this variance, we are ignoring the sign
of the COPEs.
It is instructive to consider how F-tests relate to T-tests by
considering an F-test that consists of just one contrast. As illus-
trated in Fig. 10, such an F-test is equivalent to a two-tailed T-test
on the contrast in question, with the relationship f = t2. Hence,
F-contrasts containing single contrasts can be used to mimic two-­
tailed T-tests. The F-test’s blindness to the directionality of the
contrasts is readily apparent when we consider the equivalent
­two-­tailed T-test. It is because we can get significance with either a
significantly positive or negative COPE in either tail. As a result of
calculating the P-value under both tails, a two-tailed T-test (or
equivalently the F-test) is more conservative (with respect to posi-
tive activation) than the one-tailed T-test on the same contrast.

3.6 Interaction It is possible that the response to two different stimuli, when
Example applied simultaneously, is greater than that predicted by adding up
the responses to the stimuli when applied separately. If this is the
case, then such “nonlinear interactions” may need to be allowed
for in the model. The simplest way of doing this is to set up the

P-Value=0.2

P-Value=0.2

0 1 f 2 3 4 5 6 −4 −2 -t 0 t 2 4
F T

Fig. 10 Here we are considering an F -test that consists of just one contrast. In this case, the F-test (left) is
equivalent to a two-tailed T-test (right) on the same contrast, with the relationship f = t2
200 Mark W. Woolrich et al.

Fig. 11 Example of modeling a nonlinear interaction between stimuli. The first

two explanatory variables (EVs) model the separate stimuli and the third models
the interaction, that is, accounts for the “extra” response when both stimuli are
applied together

two originals EVs, and then add an interaction EV, which will only
be “up” when both of the original EVs are “up” and “down” oth-
erwise. In Fig. 11, EV1 could represent the application of a drug
and EV2 could represent visual stimulation. EV3 will model the
extent to which the response to drug + visual is greater than the
sum of drug-only and visual-only. A contrast of [0 0 1] will show
this measure, whereas a contrast of [0 0 − 1] shows where negative
interaction is occurring. An F-contrast of

æ1 0 0 ö
ç ÷
è0 1 0ø

will ask where is there significant activity to either drug-only or

visual-only.

3.7 Converting T- and F-statistics can be converted to Z-statistics, that is, statistics
T- and F-Statistics that are distributed as a standardized Normal (Gaussian) d ­ istribution.
into Z-Statistics This is simply achieved by ensuring that the P-value is the same
regardless of which statistic is used, so to convert from a T- to
Z-statistic, we calculate the P-value for the given T-statistic and then
 Statistical Analysis of fMRI Data 201

determine the Z-statistic as being the one that gives the same P-value.
One reason for doing this is so that we can compare statistics using
a common currency. Another reason is so that we can perform
generic thresholding techniques (such as described in Sect. 7), using
Z-statistic maps, regardless of whether we have done T- or F-tests.

3.8 Percent Signal It is useful to be able to convert regression parameter estimates

Changes into percent BOLD changes. This is because percent BOLD
change can be a common currency across different experiments
(though BOLD is not a quantitative measure and depends on
many experimental factors, and so should not be treated as compa-
rable without a good deal of care). It does not depend on things
like the arbitrary scaling of intensity values output from the scan-
ner or arbitrary scaling of the EVs. As illustrated in Fig. 12, this is
simple to do and just requires that we have an estimate of the
baseline signal intensity, C, from the voxel in question, and that we
know the peak-to-peak height, H, of the relevant EV. The percent
BOLD change is then calculated as:


Hb
%change = 100 . (12)
C

Since the signal fluctuation b  ´ H is always small with respect to

the baseline C, it is common to approximate C as more simply the
mean of the time series, C′.
It is possible to do this for contrasts as well. However, it is not
immediately obvious what the peak-to-peak height is in the con-
text of a general contrast. This can be dealt with by determining

Fig. 12 Illustration of the calculation of percent blood oxygen level dependent

(BOLD) signal change from a general linear model (GLM) fit. %
change = 100H b  / C where b  is the regression parameter estimate (effect
size) and H is the peak-to-peak height for the relevant explanatory variable (EV).
C is the baseline intensity of the fMRI data, but is common to approximate this
with the mean of the time series, C′
202 Mark W. Woolrich et al.

the effective regressor for a contrast. The effective regressor is the

regressor in a new version of the design matrix whose regression
parameter estimate (and its variance) is equal to the original
COPE. This is given by [16]:

X eff = XQc ( c T Qc ) , where Q = ( X T X )

-1 -1
(13)

The peak-to-peak height of this effective regressor can then be

used in the percent signal change calculation above.

3.9 Issues We described in Sect. 3.2 how we obtain regression parameter esti-
with Orthogonality mates by finding the best fit of the GLM to the data (by using Eq.
and Estimating (6). The regression parameter estimates describe how much we
Contrasts need of each EV to explain what we see in the data. However, con-
sider what would happen in a poorly designed experiment, where
two different stimuli are switched on and off at very similar times.
The resulting predicted responses (EVs) are highly correlated.
Note that we use the terms correlated and non-orthogonal (simi-
larly uncorrelated and orthogonal) interchangeably. The top of
Fig.13 shows a design matrix containing our two very similar EVs.
The model fitting will determine the regression parameter esti-
mates b  and b  and describe how much we need of each EV to
1 2

explain what we see in the data. However, because EV1 and EV2
are so similar, we can equally well use either EV1 or EV2 to explain
what we see in the data. The result is that we cannot estimate either
b 1 or b 2 , separately from each other, very well. Mathematically,
we say that the design matrix is not of “full rank,” or that it is “rank
deficient.”
To understand this, consider solving two simultaneous equa-
tions. If we have two unknowns to solve for, then we need two
equations to solve for them. However, if the two equations are the
same, then we really have only one equation and we cannot solve
for the two unknowns.
The good news is that our statistical tests (T- and F-tests) take
this all into account. When two EVs are highly correlated, the
appropriate variances of the regression parameter estimates (see Eq.
(9)) are automatically increased—acknowledging the fact that we
cannot determine which EV should explain what in the data. So,
even though the statistics accounts for orthogonality, it is clear that
when we design our experiments, we want to avoid this being an
issue whenever possible. This can be achieved by using approaches
that assess the efficiency of experimental designs such as [16–18].
In Fig. 14, we can see three different design matrices. The first
contains two EVs that are highly correlated, the second shows two
EVs that are partly correlated, and the third shows two EVs that
are completely uncorrelated. As discussed, the first design matrix is
“rank deficient.” The third design matrix is the ideal scenario in
 Statistical Analysis of fMRI Data 203

Fig. 13 Rank-deficient design matrices. At the top, we can see a general linear model (GLM) with a design
matrix containing two identical explanatory variables (EVs). Since EV1 and EV2 are so similar, we can equally
well use either EV1 or EV2 to explain what we see in the data. At the bottom of the figure, we can see examples
 +b
of identically good model fits with any linear combination of the EVs as long as b  = 0.9. The result is
1 2
 
that we cannot estimate either b 1 or b 2 with any certainty (corresponding to [1 0] or [0 1] contrasts), but we
 +b
can estimate b  (corresponding to a [1 1] contrast)
1 2

that the EVs are uncorrelated and there is no ambiguity in how to

determine which EV explains what in the data. We refer to this as
a “well-conditioned” design matrix. But what happens in the case
of the second design matrix where the EVs are partially correlated?
It is useful to think of the EVs as having uncorrelated (orthogonal)
and correlated (non-orthogonal) components. The correlated (or
non-orthogonal) components are of no use, as they are, by defini-
tion, identical and cannot be used to disambiguate which EV
explains what in the data. Hence, the model fitting can only be
driven by the uncorrelated, or orthogonal, components of the
EVs. As long as there is a substantial orthogonal component, then
there is sufficient information to get an efficient estimate of the
regression parameters, and we can successfully infer on such GLMs.
The idea that the model fitting can only be driven by the
uncorrelated, or orthogonal, components of the EVs is a crucial
204 Mark W. Woolrich et al.

Fig. 14 Examples of different design matrices. Design matrix with two explana-
tory variables (EVs) that are highly correlated (left). Design matrix with two EVs
that are partially correlated (middle). Design matrix with two EVs that are uncor-
related (right)

one, and is well illustrated by the following example. What hap-

pens to the regression parameter estimates when we have two par-
tially correlated EVs, and orthogonalize one with respect to the
other? Figure 15 illustrates such a case. On the right, EV2 has been
“orthogonalized with respect to” EV1, which just means that the
part of EV2 which is correlated with EV1 has been subtracted from
it. The counterintuitive result is that, even though it is EV2 that
has been changed and EV1 has remained the same, it is β1 that has
changed and β2 that remains the same. To understand this, remem-
ber that the model fitting can only be driven by the orthogonal
components of the EVs. Although EV2 has changed, it has changed
to be equal to the original orthogonal component and hence its
orthogonal component is unchanged; subsequently, its regression
parameter estimate is still the same. In contrast, although EV1 has
not changed, because EV2 has been orthogonalized with respect
to EV1, the orthogonal component of EV1 has changed; subse-
quently, its regression parameter estimate is different.
Everything we have considered up to now has been within the
context of considering problems where we have two (or more) EVs
that are correlated with one another. In fact, the problem is more
general than this. We can see this by extending the analogy of solv-
ing simultaneous equations. In general, one encounters the same
problems whenever it is possible to find a linear combination of the
equations that is equal to another of the equations. For example,
consider that we have three equations to solve for three unknowns,
 Statistical Analysis of fMRI Data 205

Fig. 15 Effects of orthogonalization. We, first, fit the design matrix (containing two
partially correlated explanatory variables, EVs) on the left to the data from a voxel,
and obtain regression parameter estimates b  and b  . We then construct a new
1 2
design matrix, shown on the right, where EV1 is unchanged and EV2 is the old EV2
orthogonalized with respect to EV1. We then fit this new design matrix to the same
data and obtain new regression parameter estimates b  and b . The counterin-
1 2

tuitive result is that even though it is EV2 that has been changed and EV1 that has
remained the same, it is b  that has changed and b  that remains the same. The
1 2
underlying reason for this is that the model fitting can only ever be driven by the
orthogonal components of the EVs

but it turns out that if we take two times Eq. (1) and subtract Eq.
(2), then we get exactly Eq. (3). In that case, we actually only really
have two equations to solve for our three unknowns, and so we are
in trouble. In the GLM, the EVs are analogous to the equations, and
the regression parameters are the unknowns. And so we have a prob-
lem if any EV is the same (or close to being the same) as a weighted
sum of the other EVs in the design matrix. Again, we describe such
a design matrix as being (or close to being) “rank deficient.”
Even if we do have a design matrix that is close to being rank
deficient, and therefore, there are some regression parameters that
cannot be very well estimated, there may well be other regression
parameters that can be estimated. There may even be contrasts that
actually include the hard-to-estimate regression parameters, but
that can still be estimated [16]. At first glance, this may seem a
little counterintuitive. However, things should become clear if we
consider a simple example of this. This occurs when we have the
situation shown in Fig. 13 where we had a design matrix with two
very similar EVs. As already discussed, we cannot estimate at all
well the individual parameters β1 and β2 with [1 0] and [0 1] con-
trasts. However, we can estimate a [1 1] contrast since this does
206 Mark W. Woolrich et al.

not require us to separate out which EV explains what in the data.

Again the simultaneous the first equation analogy is useful.
Consider that we have two equations, 2x + 2y = 2 and 4x + 4y = 4.
The second equation is simply two times of the first equation; and
so we have a problem, and we cannot solve for x and y individually.
However, we can solve for x + y; it is equal to one. Solving for x + y
is analogous to estimating the [1 1] contrast in our GLM.

4 Modeling Hemodynamic Variability

Up to this point, we have been working under the assumption that

the HRF is known. However, it is well established that the HRF
varies between brain regions and subjects [19] and so in practice,
it is necessary to incorporate into our modeling of the fMRI data
some flexibility in the HRF. One option is to have a parameterized
model of the HRF and then estimate the HRF shape parameters at
the same time as we estimate the GLM regression parameters that
represent the size of the response. For example, we could use the
double gamma HRF illustrated in Fig. 5, but instead of fixing the
five parameters that describe the shape, we now look to estimate
those parameters from the fMRI data. The problem with this
approach is that it is not straightforward to estimate these HRF
shape parameters within the GLM framework, as they generally
require nonlinear estimation approaches. A number of these
approaches have been proposed, predominantly using Bayesian
techniques [20–24]. However, these approaches are computation-
ally demanding and are not yet in common use.

4.1 HRF Basis Sets A popular alternative is to use the approach of basis functions.
These allow HRF modeling flexibility but within the computation-
ally undemanding GLM framework [19]. Figure 16a shows just
one example of an HRF basis set that contains three basis func-
tions. The choice of basis set is clearly important and we will come
to that later. Whatever basis set is used, the principle is the same:
different linear combinations of the basis sets can be used to give
different HRF shapes. This is illustrated in Fig. 16b.
But how do we use these HRF basis functions in combination
with our known stimulus timings to create predicted responses
that can be used in the GLM? The answer is to separately convolve
each of the HRF basis functions with the stimulus function to cre-
ate an EV for each of the basis functions, as shown in Fig. 17.
When the resulting design matrix is fit to the fMRI data, the
required linear combination of these EVs is determined. If desired,
the same linear combination can then be applied to the HRF basis
functions to show the implied HRF shape.
The question remains as to how we set up a statistical test to
ask, for example, “Where is there significant activation due to
 Statistical Analysis of fMRI Data 207

Fig. 16 (a) Example of a hemodynamic response function (HRF) basis set con-
taining three basis functions. (b) Different linear combinations of the basis func-
tions in the basis set can be used to obtain different of HRF shapes

condition A?” when we model the response to condition A using

HRF basis functions. One answer is to use an F-test. Recall that one
way to think about an F-test is that it will find significance if there is
any linear combination of the contrasts (in the F-test) that explain a
significant amount of variance in the data. So if we simply create an
F-test made from the contrasts that pick out each of the regression
parameter estimates for each of our basis function EVs for condition
A, then we will find where there are any linear combinations of the
basis set EVs that can be used to give HRF shapes that explain sig-
nificant amounts of variation in the data. In other words, if we have
an experiment with just condition A, and we model it using three
basis functions (Fig. 17), then we can ask “Where is there signifi-
cant activation to condition A?” with the F-test contrast matrix:

æ1 0 0ö
ç ÷
c = ç0 1 0÷
T

ç0 0 1÷
è ø.

An important point to remember is that, as we are using an F-test,

we lose directionality of the test. So we cannot tell if we are finding
significance with a positive or negative response. However, it is
possible to recover this post hoc if we are using a basis set that
contains a canonical HRF, by looking at the sign of the regression
parameter estimate for the corresponding canonical HRF EV. We
208 Mark W. Woolrich et al.

Fig. 17 Setting up design matrix explanatory variables (EVs) using a hemodynamic response function (HRF)
basis set. Each HRF basis function is separately convolved with the stimulus function to create an EV for each
of the basis functions

can also make comparisons between two different conditions by

pairing up the corresponding HRF EVs for the two conditions and
setting up an F-test that looks for linear combinations of differ-
ences between corresponding HRF EV regression parameters. In
other words, with three basis functions in our basis set, EVs 1–3
model the HRF EVs for condition A, and EVs 4–6 model the HRF
EVs for condition B, then we can ask “Where is there significantly
different activation between condition A and condition B” with
the F-test contrast matrix:

æ1 0 0 -1 0 0 ö
ç ÷
c = ç0 1 0
T
0 -1 0 ÷
ç0 0 1 0 0 -1÷ø.
è

More precisely, this is looking for where there is a significant

amount of variance being explained by any linear combination of
differences between condition A and condition B for correspond-
ing basis function EVs. This means that we can find a significant
difference due to either a “shape” or “size” change. Note that for
this approach to be sensible, we need to use the same basis set for
both conditions.
 Statistical Analysis of fMRI Data 209

4.1.1 Choosing Figure 16a showed an example basis set that had been derived
a Basis Set from a parameterized HRF that was made up of a series of half-­
cosine functions [15]. This was obtained by sampling thousands of
example HRFs from within a range of plausible parameter values
for the HRF model (Fig. 18), and then a principal component
analysis was carried out on these samples to determine the principal
modes/components of variation in the HRF shape. The three
highest principle components were then used as the basis set. Note
that this can equally well be done on any form of parameterized
HRF, for example, a double gamma HRF or a biophysical model
such as the balloon model [25].
When this approach is taken with any plausible HRF model, it
is typical for the first basis function to turn out to be the mean HRF
shape, or a “canonical” HRF, for the second to approximate the
temporal derivative (i.e., linear combinations of the first and second
basis functions result in versions of the canonical HRF shifted in
time), and for the third to approximate a dispersion derivative (i.e.,
linear combinations of the first and third basis functions result in
versions of the HRF with different widths of the main positive
response). Although other basis sets that have been proposed (e.g.,
sets of Gamma functions and finite impulse response functions),
this kind of basis set is highly recommended since it parsimoniously
captures shape variations. It also has the advantage that the first
(canonical) basis function will tend to dominate the fit to the data
and can then be used to determine the positivity or negativity of the
HRF. Note that it is quite common for people to neglect the dis-
persion derivative and use just a temporal derivative since temporal
shifts represent the most important variation in the HRF shape,
particularly when working with boxcar stimuli.
Thus far, we have considered basis sets made up of two or
three basis functions. But why not use more? The reason for this,

a b c
m2

m4

m1 m3 0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30
seconds seconds

Fig. 18 (a) Parameterized hemodynamic response function (HRF) model. (b) Example HRFs sampled from this
parameterized model of the HRF for plausible parameter values. (c) Samples of the HRF obtained from random
linear combinations of the basis set shown in Fig. 16a
210 Mark W. Woolrich et al.

and in general the reason why basis sets with large numbers of basis
functions (e.g., finite impulse response basis sets) are suboptimal,
is that the GLM becomes unrealistically flexible. This problem is
evident even with just three basis functions. Figure 18c shows pos-
sible HRFs that result from random linear combinations of the
basis set in Fig. 16a. Clearly, many of these HRFs are nonsensical.
The problem is that random fluctuations in the fMRI noise can, by
chance, look like these nonsensical HRFs and so we “over-fit” the
model to the noise. The statistical inference (e.g., via F-tests across
the basis functions) is still valid, but we lose sensitivity; it becomes
harder to detect genuine activations. On the contrary, if we use too
few basis functions then we can fail to estimate the true HRF and
our model is then a poor match to the data and again we suffer a
reduction in sensitivity. So there is a trade-off between providing
enough basis functions to provide enough HRF variability, while
not having so many that over-fitting becomes a problem. A general
rule of thumb is that three basis functions are good for single-event
designs and two basis functions are good for boxcar designs.
An increasingly used approach to overcome this problem is to
infer on models that incorporate HRF variability using a Bayesian
framework. One advantage of a Bayesian approach is that prior
information can be included. Priors can be used that prohibit non-
sensical HRFs. Subsequently, more flexibility can be allowed while
protecting against over-fitting [15, 22–24].

4.1.2 Basis Functions In Sect. 6, we will discuss how we model multisession/subject fMRI
and Group Inference data. However, it is worth mentioning how basis functions are best
used when we are ultimately doing a group analysis. In particular, we
consider this in the context of the simple case of inferring a popula-
tion group mean. One option might be to pass up the regression
parameter estimates for all basis functions into the higher-level group
analysis, obtain the group average for each basis function separately,
and then perform an F-test across them at the group level (in the
same manner as we would do in a single-­session analysis). However,
it is not clear what benefit there would be of doing this. When our
basis set contains a “canonical” HRF, the other basis functions, such
as the temporal and dispersion derivatives, tend to average out to
zero at the group level due to the different subject HRF shape varia-
tions. Subsequently, an often-­recommended approach is to only pass
up to the group level the canonical HRF regression parameter esti-
mates. This makes for a simple group analysis, and the benefits of
including the basis function at the first level are still felt in terms of
accounting for HRF variability that would otherwise cause increased
noise in the first-­level analysis.
Another option that can be taken is to calculate a size summary
statistic from the single-session analyses (e.g., the root mean square
of the basis function regression parameter estimates), and pass that
up to the group level. However, it is important to note that this
 Statistical Analysis of fMRI Data 211

would then require different inference methods at the group level

(e.g., see Sect. 7.4.2) than is generally used, as the population dis-
tribution of such a summary statistic is likely to be non-Gaussian.

4.2 Nonlinearities So far we have assumed linearity of the HRF. That is, we have
assumed that the response to a stimulus is well modeled by (linear)
convolution of the stimulus with the HRF. Typically, this is the
approach that people take in the majority of fMRI analyses.
However, it has been shown that this assumption is poor in certain
situations. For example, it can be shown that the response to a
prolonged stimulus is not as large as the one we would predict
from extrapolating results from applying a short stimulus [26, 27],
and nonlinearities are predominant when there are short separa-
tions (less than ∼3 s) between stimuli [28]. Normally, these situa-
tions are intentionally avoided by designing experiments
appropriately. For example, we avoid experiments where single
events are occurring less than approximately 3 s apart, or experi-
ments that require comparisons between a mix of short (e.g.,
single-­event) and prolonged (e.g., boxcar) stimuli. However, if
these situations are unavoidable, then it becomes necessary to
model the nonlinearities.
Such nonlinearities are predicted by nonlinear biophysical
models, for example, the balloon model [25]. Hence, one solution
is to model fMRI data using these nonlinear biophysical models
[22]. Another approach that can be used in the GLM setting is to
extend the idea of convolution to include second-order nonlinear
terms using Volterra kernels [28].

4.2.1 Volterra Kernels Volterra kernels are a generalization of convolution to include

higher order nonlinear terms. In fMRI, we need only to add the
second-order terms to the first-order convolution terms to get the
most important nonlinear behavior. A second-order Volterra ker-
nel model is given by:
¥
x ( t ) = òh1 (t ) s ( t - t ) dt +
0
¥¥
(15)
òòh (t ,t ) s ( t - t ) s ( t - t ) dt dt
00
2 1 2 1 2 1 2

where s(t) is the stimulus, the first term contains the traditional
linear HRF first-order kernel, h1(τ), and the second term includes
the second order kernel, h2(τ1, τ2).
Many of the issues surrounding the use of second-order
Volterra kernel basis functions are the same as they are for linear
basis functions. For example, Volterra kernels can be determined
empirically [25, 28], or derived from nonlinear biophysical models
[22]. Either way, as with linear basis functions, there is variability
212 Mark W. Woolrich et al.

in the response between different subjects and brain regions, and

this variability can be parsimoniously captured within the GLM by
using basis functions. We can obtain parsimonious basis sets for
first- and second-order kernels by using principle component anal-
ysis on samples of the response from empirical data or from param-
eterized models. Furthermore, we can infer on the Volterra kernels
in a Bayesian framework with priors that prohibit nonsensical
responses, so that more flexibility can be allowed while protecting
against over-fitting [22].

5 De-Noising fMRI Data

fMRI data are inherently noisy and contain a variety of fluctuations

induced by processes beyond the control of the experimenter.
Examples of such effects include artifacts related to the MR physics
(such as slice-dependent signal dropout due to imperfect switching
of the slice-select gradients, EPI “ghosting,” and thermal noise),
subjects’ head motion effects, fluctuations induced by the cardiac
and respiratory cycles, and spontaneous low-frequency fluctuations
of the baseline signal.
Under the assumptions of the GLM, any fluctuation in the
measured BOLD signal that is not modeled by the EVs in the
design matrix is deemed to be noise. In Sect. 3.3, we discussed
how such artifacts are more likely to occur at low frequencies and
how we can use pre-whitening to deal with this. However, this
assumed that all these fluctuations are stochastic, and that within
the framework of the GLM, this stochastic noise is well described
by a Gaussian distribution.
In practice, however, some of the underlying random noise
fluctuations will have very distinct spatial and/or temporal struc-
ture. As an example, Fig. 19 shows a variety of such structured
noise components identified from a single fMRI dataset using an
independent component analysis (ICA) decomposition [29]. This
suggests that such effects are structured rather than random sto-
chastic noise and the challenge is to account for their existence in
order to obtain optimal estimates of the GLM model parameters.

5.1 Structured Noise The main problem with such structured noise artifacts is that they
and the GLM can severely impact our GLM-based analysis. The part of the arti-
fact that is orthogonal (uncorrelated) with all of the EVs will sim-
ply not be modeled by the design matrix regressors, and therefore,
the presence of the structured noise effect will be reflected by an
increase in the residual GLM noise variance. This, in turn, will
decrease any T- or F-statistics value, making it harder for us to
detect true activations. The non-orthogonal (correlated) part of
such an artifact, however, will result in wrong parameter estimates
for those EVs that correlate with the artifact. If the correlation is
 Statistical Analysis of fMRI Data 213

a b c

0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180

d e f

0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180

Fig. 19 Examples of “structured noise” identified in a single fMRI dataset using independent component analy-
sis [29]: (a) residual head motion, (b) signal fluctuations in the ventricles, (c) spontaneous fluctuations in the
bilateral sensory motor cortex, (d) fluctuations close to the sinuses (likely due to interactions between B0 field
inhomogeneities and head motion), (e) high-frequency image ghosting, and (f) more spontaneous low-­
frequency fluctuations

positive, we will overestimate the parameter for the EV, making it

more likely that we wrongly detect activations where there are
none. If, on the other hand, the correlation is negative, then esti-
mated effect sizes will be underestimated, making it harder to
detect true activations.
In short, the impact on the GLM estimates and the statistics
values can be profound. Therefore, if we can characterize the spa-
tial and/or temporal structure of such artifacts, it is desirable to
incorporate this knowledge into the data analysis and to explicitly
account for the presence of these effects in the data.

5.2 Nuisance One possible way of correcting for the negative impact on GLM
Regressors in the GLM statistics is to introduce additional “nuisance” or “confound”
regressors in the GLM design matrix. Remember from Sect. 3.9
that in the case of multiple regressors, the parameter estimates for
each of the EVs can only be driven by the uncorrelated (orthogo-
nal) component of an EV. If we can find a suitable characterization
of the temporal structure of an artifact, we can add this as a new
regressor to the design matrix in order to use this to “explain” some
of the measured variation in the data. The parameter estimate for
EVs of interest will then only reflect the amount of ­variation that
the EV can explain over and above what can already be explained by
214 Mark W. Woolrich et al.

Fig. 20 Example of the utility of nuisance regressors in the general linear model (GLM): (a) data without artifact
regressed against a single explanatory variables (EVs), (b) data with confound analyzed in a GLM without
nuisance regressor, and (c) confounded data analyzed using both the EV of interest and a nuisance regressor

nuisance variables. Note, however, that we do need to pay a price

for the use of nuisance regressors as part of the design matrix: every
new regressor does reduce the number of DOF for our final statisti-
cal comparison. As such, it is desirable to keep the number of nui-
sance regressors to a minimum while trying to maximize the amount
of structured noise variance captured by these regressors.
Figure 20 illustrates the use of a nuisance regressor. In this
example, an fMRI time series exhibits an intensity jump due to the
presence of a scanner-induced image artifact such that during one
of the TRs, the measured image intensity for this time point is
about 10 % above the mean intensity level. In Fig. 20a, the voxel’s
time series without the artifact is analyzed using a simple GLM
with one single EV. The analysis represents the data as a linear
combination of the EV of interest and residual noise. The level of
activation in this voxel is high, resulting in a very significant
T-statistic. In Fig. 20b, the same GLM design matrix is now fitted
to the voxel’s time series with the artifact present. The timing of
the artifact is almost entirely uncorrelated with the primary EV and
the presence of the artifact will therefore result in an inflated resid-
ual variance, causing a significant drop in the T-statistics value of
more than 20 %. If information about the temporal characteristics
of the artifact is available, then we can model the intensity variation
at the specific time of the artifact by introducing a nuisance variable
into the GLM design.

5.2.1 Deriving Nuisance There are various ways of deriving useful nuisance variables. In
Regressors general, these additional EVs should reflect the temporal charac-
teristics of structured noise that is thought to exist in the data.
Motion of the subject in the scanner is a typical problem in
fMRI, and there are often intensity fluctuations related to head
motion still present in the data even after alignment-based motion
 Statistical Analysis of fMRI Data 215

correction. A common set of nuisance regressors (used to help

model out such residual effects of motion) is the set of six time series
obtained as the parameters of the head motion correction proce-
dure. In this case, the intensity variations in the data are expected to
correlate with the size of the three translations and three rotations.
When included, these regressors can jointly “explain” any signal
variation in the data correlated with head motion.
Other sources of structured noise effects are the subjects’ car-
diac and respiratory cycles. A popular approach is to use retrospec-
tive image correction (RETROICOR [30]) in order to correct for
these effects. Using additional measurements of the heart and res-
piration cycles, one can derive nuisance regressors that permit one
to remove all signal variation in the fMRI data that temporally
correlates with the relative phase of these physiological cycles. The
regressors are based on these additional measurements as low-­
order Fourier terms, and can significantly reduce the amount of
structured noise induced by physiological fluctuations.

5.2.2 ICA-Based The ability to correct for additive structured noise depends on the
De-noising ability to characterize these noise components in terms of their
temporal evolution. In the previous two examples, this was
obtained by accurately estimating rigid-body motion or by using
secondary measurements of physiological processes. For other
types of noise, it is often not easy to predict such nuisance regres-
sors based on the understanding of the biophysics and of the imag-
ing process. One possibility is to use a model-free data analysis
approach, such as ICA, in order to identify structured noise effects
in the data prior to the GLM analysis. ICA and related techniques
decompose the fMRI data into modes of variation that define the
spatial and temporal extent of underlying fluctuations [29]. The
estimated time courses of a component can then be used as nui-
sance regressors as part of a GLM analysis. An alternative is to
explicitly regress out such effects prior to a GLM-based analysis,
effectively running the model-based analysis on the residuals of a
prior linear regression model designed to de-noise the data.
Currently, as no well-established techniques exist for automatically
identifying such noise components, such an approach relies on the
experimenter to visually inspect all components. Further research
is required to integrate such a model-free identification (e.g., using
ICA) into the standard GLM in an unbiased objective way.

5.3 Example Figure 21 demonstrates the impact of structured noise on the

GLM and highlights the utility of such a de-noising approach.
Subjects were requested to perform simple finger tapping using
either the left or right hand. All subjects were right-handed and
one of the contrasts of interest involved the left-right comparison
“Where is the activity larger when using the left hand when com-
pared with the right hand?” Prior to the noise removal, the
216 Mark W. Woolrich et al.

a b c
Z-stats Z-stats
350 350
300 300
250 250
200 200
150 150
100 100
50 50
0 0
–10 –5 0 5 10 –10 –5 0 5 10

Fig. 21 Example of the effect of fMRI de-noising in a simple finger tapping experiment: (a) histogram of the
Z-statistic image for the differential left- versus right-hand finger tapping contrast. Because of the presence of
structured noise, the histogram is far from being Gaussian distributed; (b) after regressing out a variety of
structured noise effects, the histogram of Z-statistic values becomes unimodal and much closer to a Gaussian
distribution; and (c) map of significant voxels after regressing nuisance effects out of the data

histogram of the Z-statistic values for this contrast is highly non-

Gaussian. The contrast map itself did not reveal any significant dif-
ferences when using standard thresholding. After de-noising, the
Z-statistic image of this contrast identifies significant differences in
the BOLD, particularly in right motor cortical areas.

6 Multisubject Statistics

We have so far only focused on ways of modeling and fitting the

(time series) signal and residual noise at the individual single-­
session level, in order to derive effect size estimates from a single
fMRI dataset. The majority of fMRI studies, however, are used to
address questions about activation effects in populations of sub-
jects. This generally involves a multisubject and/or multisession
approach where data are analyzed in such a way as to allow for
hypothesis tests at the group level [12, 31], for example, in order
to assess whether the observed effects are common and stable
across or between groups of interest.
Figure 22 illustrates an example scenario where the question of
interest involves estimating the difference in activation between
two groups of subjects. This question is addressed by having differ-
ent GLMs at the session, subject, and group level in a hierarchical
fashion. At the lowest level of the analysis, the single-session time
series data is modeled in the way described previously. At the sub-
ject and group level, there are GLMs that, for example, model the
subject (cross-session) mean and group (cross-subject) mean effect
sizes, respectively. At the top-level of the hierarchy, a set of statistic
images is created that can be used for final statistical inference.
 Statistical Analysis of fMRI Data 217

Fig. 22 Hierarchical general linear model (GLM) for the analysis of group fMRI data. Within a summary statistics
approach, the GLMs are estimated one level at a time and summary statistics are passed up to the next level
of the hierarchy

6.1 Brain Atlases Registration (aligning different brain images) is typically used
when combining fMRI data from different sessions or subjects in a
multisubject analysis. This allows us to assume that the data we are
comparing across sessions or subjects come from approximately
corresponding areas of the brain. In doing this, it is typical to
transform the data into a common “standard brain space,” for
example, the co-ordinate system specified by Talairach and
Tournoux [32]. These standard spaces can be either what are
known as templates or atlases.
A template is typically an average of many brains, all registered
into a common co-ordinate system. An example is the MNI 305
average [33]. An atlas is also based in a common co-ordinate sys-
tem, but contains richer information about the brain at each voxel,
for example, information about tissue type, local brain structure,
or functional area. Atlases can inform interpretation of fMRI
experiments in a variety of ways, helping the experimenter gain the
maximum value from the data.

6.2 Fixed- An important question is that of how to model and estimate effects
Versus Mixed-­Effects at the intermediate and higher level of the hierarchy. If we were
Models only concerned about the particular set of subjects in our study,
then we would use a fixed-effects model. More typically, however,
we would want to generate results that extend beyond the particu-
lar population of subjects scanned as part of the study, into the
wider population. In this case, we also need to account for the fact
that the individual subjects themselves are sampled from the wider
population and thus are random quantities with associated vari-
ances. It is exactly this step that marks the transition from a simple
fixed-effects model to a mixed-effects model and it is imperative to
formulate a model at the group level that allows for the explicit
modeling and estimation of these additional variance terms.
As an example, consider the simplest case of estimating the
effect size of a group of M subjects, where for each subject k, the
pre-processed fMRI data is ϒk, the design matrix is Xk and the
218 Mark W. Woolrich et al.

parameter estimates are βk (for k = 1,…, M). The individual first-­

level GLMs relate first-level regression parameters to the M indi-
vidual datasets: ϒk = Xkβk + εk, where εk specifies the single-subject
residuals. If we are only concerned about the exact population of
subjects scanned under our fMRI paradigm, then the estimate of
the group mean effect size is simply the average over all the lower-­
M
level parameter estimates: b g = (1 / M ) åb k , that is, our second-­
level GLM is simply k =1

bk = X g bg (16)

where Xg = [1/M, …, 1/M]T, and we have concatenated all of the

first-level regression parameters into one vector:

é b1 ù
êb ú
bk = ê 2 ú .
ê ú
ê ú
ëb2 û

In this case, we simply need to average the first-level regression

parameters and the only variance to consider is the average first-­
level variance.
If, however, we want to generalize our findings to the wider
population then the second-level analysis needs to account for the
sampling of the subjects, and we need to proceed by modeling the
group effect size of interest as

bk = X g b g + e g (17)

where εg accounts for the variation of the different subjects’ means

from the overall group mean. Both the within- and the between-­
subject variations contribute to the total mixed-effects variance
against which the mean effect size is tested during the statistical
inference procedure.
The difference between the two approaches is illustrated in
Fig. 23. In the fixed-effects model (a), only the first-level variances
need to be considered, whereas in the case of the mixed-effects
analysis (b), both the first-level fixed-effects variances and the
higher-level random-effects variance contribute to the total mixed-­
effects variance used for inference. The between-subject variance,
σg2, then accounts for the random sampling of the particular sub-
jects from the wider population.
In the following sections, we assume that a mixed-effects
analysis is being performed, as this is typically what is required.
Fixed-­ effect analyses are performed in a similar manner but
without the complication of needing to estimate the random-
effect variances.
 Statistical Analysis of fMRI Data 219

Fig. 23 Illustration of a simple group analysis using the fixed-effect and the mixed-effects models: (a) in the
fixed-effects analysis, the only variance contribution to consider is the lower-level within-subject variance; (b)
in the mixed-effects analysis, the between-subject random-effects variance, σ2, accounts for the random
sampling of the subjects themselves and contributes to the overall mixed-effects variance

6.3 Summary We now come to the question of how we infer on the multilevel hier-
Statistics Approach archy of a group study (an example of which was shown in Fig. 22),
in order to ask questions such as “Where is there significant activity in
response to the experimental task for the population?” or “Where is
there significant differences between populations (e.g., controls ver-
sus patients)?” Recall that each level in the hierarchy is represented by
its own GLM. Hence, one approach is to formulate a single complete
GLM that combines together the first-level and higher-level GLMs.
An example of such an approach is presented in Friston et al. [34]
where the group analysis is carried out “all-in-one” using the within-
session fMRI time series data as input. However, in fMRI, where the
human and computational costs involved in data analysis are rela-
tively high, it is desirable to be able to make group-level inferences
using the results of separate first-level analyses. This approach is com-
monly referred to as the “summary statistics” approach to fMRI anal-
ysis [31]. Within such an approach, group parameters of interest can
easily be refined as more data become available.
In Holmes and Friston [31], the regression parameter estimates
were used as summary statistics. The regression parameter estimates
from the lower level are used as the “data” at the next level up. For
example, the estimates of βk are used in place of βk in Eq. (17). This
approach was shown to be equivalent to inferring all-in-one under
certain conditions [31]. For example, it requires balanced designs,
that is, all lower-level design matrices need to be identical, preventing
the use of behavioral scores or subject-specific confound regressors.
However, top-level inference using the summary statistics
approach can be made equivalent to the all-in-one approach with-
out such restrictions [35, 36], if we pass up the correct summary
statistics. In particular, it is important to pass up information about
not only the effect sizes from the lower levels, but also their vari-
ances. We shall explore the benefits of doing this in Sect. 6.4.
220 Mark W. Woolrich et al.

6.4 Estimation When we use a summary statistic approach, we infer on each level
of the Mixed-­Effects of the hierarchy one at a time. The first-level inference is as we
Model described earlier in the chapter. At higher levels in the hierarchy,
however, estimating the regression parameters and variances of
group-level GLMs offers a different set of challenges. At the first
level, there typically exists a large number of observations (typically
more than 100), so that relevant parameters and variances can be
estimated with high DOF. In contrast, group-level variance com-
ponent estimation is typically troubled by having very few observa-
tions (i.e., low DOF).
A key issue in estimating mixed-effects models within the
“summary statistics” approach is whether the variance information
from the lower levels (e.g., first level) is used at the higher levels
(e.g., group level). Approaches that use the variance information
from the lower levels have a number of substantial advantages.
First, such approaches do not require balanced designs and so per-
mit the analysis of fMRI data where the first-level design matrices
have different structure from each other (e.g., contain behavioral
scores as regressors) or where the data contains different numbers
of observations (e.g., different numbers of sessions for each sub-
ject). Second, they provide more accurate variance estimation (and
therefore more accurate inference) by ensuring that at every level,
only positive estimates of the random-effects variances contribute
to the overall mixed-effects variance. Finally, such approaches
increase the ability to detect real activation, by weighting the dif-
ferent contributions from the lower levels by using the lower-level
variance information. For example, effect sizes from subjects with
high first-level variance get down-weighted compared with those
with low first-level variance, when inferring at the group level.
Estimating mixed-effects models when the lower-level variance
information is ignored can be carried out easily using ordinary least
squares [31]. Approaches that use the lower-level variance
­information and provide the advantages described above are a little
more involved. For example, Worsley et al. [12] used an expecta-
tion maximization approach. Woolrich et al. [36] used a fully
Bayesian framework using appropriate noninformative priors. This
approach had the added advantage that one can model different
variance components for different groups. For example, one can
contrast effect sizes in a population of patients relative to a popula-
tion of controls under the assumption that these two groups have
different within-group variance. This is important as, empirically,
patient populations exhibit larger within-group variability than a
carefully selected population of controls.

6.5 Handling Outlier The approaches described so far assume that the population distri-
Subjects butions of the effect sizes are well modeled using a Gaussian distri-
bution. However, in practice group studies can include “outlier”
subjects whose effect sizes are completely at odds with the general
 Statistical Analysis of fMRI Data 221

population for reasons that are not of experimental interest. For

example, it could be due to excessive subject motion or misunder-
standing by the subject of the experiment instructions. The esti-
mate of the population variance can be inflated by outlier subjects,
and the population mean estimates can be under-or over-estimated.
This is analogous to the presence of structured noise in the context
of single-session analysis, as discussed in Sect. 5.
A number of approaches have been proposed to deal with this
problem. One option is to visually inspect both the data and the
results of a group analysis to deduce outliers. These outlier subjects
can then be removed and the group study re-analyzed without
them. Although useful exploratory approaches have been proposed
that aid in this process [37–39], human intervention is often still
required. It is preferable to use approaches that are automatic, and
soft-assign outlier behavior in a spatially localized manner [40, 41].
The approach of Woolrich [41] has the added benefit that it uses
the lower-level variance information.
Another possibility to dealing with outliers is to use nonpara-
metric statistics (see Sect. 7.4.2). For example, Meriaux et al. [42]
and Roche et al. [43] use permutation tests that take advantage of
the lower-level variance information. These approaches protect the
validity of the statistics but can be less sensitive compared with tech-
niques that explicitly model the outliers [41]. However, they are
also potentially able to handle other deviations from Gaussian pop-
ulation distributions (e.g., populations with two sub-­populations),
and there is evidence that nonparametric statistics in general handle
the multiple comparison problem better than random field theory
(RFT) [44] (this is discussed further in Sect. 7.4.2).

6.6 Creating When creating higher-level GLM design matrices, it is important

Higher-­Level GLMs to ensure that the design matrix at least models the cross-subject
mean effect. This is different from a first-level analysis where the
overall time series mean is normally not of interest and might actu-
ally be removed prior to the first-level GLM. In the case of a
higher-level analysis, however, the mean lower-level effect often is
exactly what is of interest and therefore needs to be explicitly mod-
eled as part of the design matrix, either as a single EV or as a linear
combination of EVs.
Figure 24 gives a selection of typical fMRI higher-level designs.
In the simplest case (a), the higher-level design only models a sin-
gle group mean effect and a simple [1] contrast then tests if the
mean effect is greater than 0.
In some cases, additional subject-specific behavioral scores
need to be included as additional EVs (b). This can be either to
remove some higher-level variation of no interest by including
these EVs as nuisance regressors (e.g., by regressing out subjects’
age or gender or reaction time), or because these regressors are
part of a testable hypothesis and need to be included in a contrast
222 Mark W. Woolrich et al.

Fig. 24 Typical higher-level general linear model (GLM) design matrices: (a)
group mean effect size over eight subjects, (b) group mean and confounds over
eight subjects, (c) unpaired group difference over two groups of four subjects, (d)
paired group difference test over two conditions for four subjects. Note that for
the sake of space, the number of subjects assumed here is lower than what
would be typically expected in a group study

of interest (e.g., a researcher might be interested in effects which

correlate significantly with duration of treatment in a clinical popu-
lation). In both cases, the additional EVs would need to be orthog-
onalized relative to the EV that is modeling the group mean. This
is so that the regression parameter for the group mean EV can
indeed be interpreted as the overall group mean effect.
The simplest multiple-group design involves just two groups
where the question of interest involves assessing the between-­group
difference (c). In this case, each group’s mean effect is modeled using
a separate EV, a [1 − 1] contrast can then be used to assess A > B dif-
ferences; the negative [–1 1] contrast tests for B > A differences.
Another typical design involves testing for differences between a
set of data generated under different conditions A and B in the same
population (d), for example, where subjects get scanned before and
after a period of learning. This is often referred to as a paired T-test.
Every subject has two observations and we need to account for the
within-subject covariance by means of subject-­ specific confound
regressors. In this case, the first EV models the A − B differences for
the M subjects, while the M additional EVs account for the subject-
specific mean effects. For example, the second EV in Fig. 24d mod-
els the mean effect for the first subject. A [1 0 0 0 0] contrast can
then be used to assess the A − B paired difference.

7 Inference (“Thresholding”)

As we saw in Sect. 3.5, a result of fitting a GLM is a T-statistic

image for each contrast, where the intensity at each voxel assesses
the evidence for a nonzero effect. Ideally, the statistic image would
be zero where there is no effect and very large where there is an
effect. Of course, because of the noise, this is not the case, and we
 Statistical Analysis of fMRI Data 223

must make inference—a statistically calibrated decision—on which

voxels exhibit a signal and which voxels are just consistent with
noise. Here we are talking about inferring on T-statistic images;
however, the issues involved with F-statistic images generated from
F-tests, or Z-statistic images (see Sect. 3.7), are similar.
The simplest inference method is voxel-wise thresholding. If the
value of a T-statistic image is t at a given voxel, then we reject the
null hypothesis of no experimental effect if t > u (where u is a signifi-
cance threshold). However, one should first ask: Why threshold?

7.1 Inference The natural questions that any user of FMRI has of their data are:
with the Mass “What is the location of my signal?,” “What is the extent of my sig-
Univariate Model nal about that location?,” “What is the magnitude of my signal?.”
For each of these, our statistical model should provide an estimate, a
measure of uncertainty of the estimate (i.e., a standard error or a
confidence interval), and a significance measure, like a P-value (i.e.,
could our result be explained by chance alone). For example, if a
visual effect produced a cluster (a contiguous groups of supra-
threshold voxels, more on this in Sect. 7.2) with a peak at a certain
location, how certain can I be that the true center of activation is
near that location? Or, if a cluster had a volume of 500 voxels, what
is my confidence that the true signal extent is 500 voxels?
Surprisingly, such basic questions cannot be answered with
standard fMRI methods. In fMRI, we generally use a mass univari-
ate model, where a GLM is fit independently at each voxel. No
information is shared over space, and, specifically, no explicit spatial
model is used to express the extended signals that we expect. While
more advanced methods that address these issues exist [45], the
only inferential questions that a mass univariate model can answer
are (a) “What is the signal magnitude at each voxel (with standard
errors and P-values)?” and (b) “What is the signal extent for a given
cluster-defining threshold (P-values only)?.” However, standard
errors and P-values on locations (e.g., confidence intervals on local
maxima or center of mass of a cluster) are not available.
The remainder of Sect. 7.2 focuses on these two types of infer-
ences: voxel-wise and cluster-wise.

7.2 Voxel-Wise The result of applying a contrast to the GLM fit at each voxel is a
Versus Cluster-­Wise statistic image. This is anywhere from I = 20,000–100,000 brain
Inference voxels in a statistic image. The value in the image at each voxel is a
T-, F-, or Z-statistic that measures the evidence for an effect defined
by the contrast. The process of applying threshold u, and retaining
all voxels with statistic value greater than u is known as voxel-wise
inference. Precisely, we are performing I statistical tests of signifi-
cance, rejecting the null hypothesis at voxel i if ti ≥ u, where ti is
the statistic value at voxel i.
Alternatively, we can apply a cluster-forming threshold, uclus,
create a binary image of voxels ti ≥ uclus, and identify clusters, that is,
224 Mark W. Woolrich et al.

contiguous groups of supra-threshold voxels, with cluster having

size S. The process of retaining all clusters with size greater than k
is known as cluster-wise inference. Precisely, we are performing L
statistical tests, one for each of the L clusters in the image, rejecting
the cluster null hypothesis if S ≥ k. The cluster null hypothesis is that
all of the voxels in cluster have no signal, and the inference is clus-
ter-by-cluster. Hence, an unusually large cluster extent only tells us
that there exists one or more signal voxels within the cluster, but
not which voxels within the cluster have signal.
Figure 25 illustrates the difference between voxel- and cluster-­
wise inference. While voxel-wise inference rejects the null hypoth-
esis for individual voxels, cluster inference jointly rejects the null
hypothesis for a set of voxels within a cluster as significant. As such,
we say that cluster-wise inference has less spatial specificity than
voxel-wise inference. On the contrary, since there are many fewer
clusters than voxels, the multiple testing problem (see Sect. 7.3) is
more severe with voxel-wise inference.
What cluster-forming threshold should be used? Using a relatively
low threshold will allow clusters with small statistic values to be
formed, but the clusters may not be significant, as a low threshold will
also result in large clusters by chance alone. A high threshold ensures
that clusters due to chance noise are small, but may then miss true
signals that have relatively small magnitude. Below we will introduce
two methods for finding P-values for cluster size, one of which requires
relatively high thresholds: Random Field Theory requires relatively
high uclus values (uncorrected P-value of 0.001 or smaller) to give
accurate inferences [46], while permutation is valid with any chosen
uclus threshold. Regardless of which threshold is chosen, the threshold
should be picked before examining the data. Trying many thresholds
introduces a multiplicity that is not easily accounted for, and will
reduce the confidence of any significant findings. Later, in Sect. 7.5,
we will consider the approach of threshold-free cluster enhancement,
which provides cluster-like inference without the dependence on uclus.
Should we be using voxel-wise or cluster-wise inference, or
both? If one examines both cluster-wise and voxel-wise results yet
another multiple testing problem is introduced, and so one method
does need to be chosen a priori. Friston et al. [47] shows that when
the anticipated signals are broad or spatially extended, cluster size
inference is best, and when focal, intense signals are anticipated,
voxel-wise inference is best. Both methods are widely used, with
cluster-wise inference being slightly more common, probably due
to the large extent of effects typical after spatial smoothing.

7.3 Correction Statistical hypothesis testing provides a means to test a default, or

for Multiple Tests null, hypothesis with a pre-specified false positive rate. At a single
voxel, a test statistic can be compared to an α = 0.05 significance
threshold, denoted uα, where α is the allowed risk of false positives.
For example, a Z-statistic at one voxel will, with many repetitions of
a null experiment, exceed uα = 2 about 2 % of the time.
 Statistical Analysis of fMRI Data 225

Statistic Image

statistic value

space

Voxel-wise Inference
statistic value

uα

space

Significant Voxels No significant Voxels

Cluster-wise Inference
statistic value

uclus

space

Non-Significant kα kα Significant
Cluster Cluster

Fig. 25 Voxel-wise versus cluster-wise inference. Inferences on fMRI statistic images are made either through
voxel-wise or through cluster-wise methods. The top panel illustrates the values in a statistic image through
one line of space, where large values indicate evidence for an experimental effect. The middle panel illustrates
voxel-­wise inference, where a significance threshold uα is applied to the image, and voxels above that thresh-
old are labeled as significant. The advantage of voxel-wise inference is that individual voxels are marked as
significant, but the disadvantage is no spatial information is considered, and unusually expansive effects may
be missed. The bottom panel illustrates cluster-wise inference, where a cluster-forming threshold uclus is
applied to the image, and contiguous voxels are formed into clusters. Clusters that exceed a significance
threshold kα in size are marked as significant. The advantage of cluster-wise inference is that low, spatially
extended signals can be detected. The disadvantage is that clusters as a whole are marked as significant, and
individual voxels within a cluster cannot be marked individually as significant
226 Mark W. Woolrich et al.

For a particular observed statistic value, say t = 3.3, we measure

the evidence against the null hypothesis with a P-value, here
p = 0.0005, which is the chance of obtaining, over repeated null
experiments, a result greater or equal to t = 3.3. (Take care not to
confuse P-values with posterior probabilities: Bayesian methods
give the posterior probability that the null is true, conditional on
the data; classical P-values, in contrast, are the probability of the
data, conditional on the null hypothesis).
But if I = 10,000 voxels (or K = 100 clusters) are tested at level
α = 0.05, the false positive risk is only controlled at each voxel.
Then over all voxels, a total of I × α = 500 false positive voxels (or
five false positive clusters) would be expected. The problem is that
standard hypothesis testing only accounts for the risk of false posi-
tives for one test. If multiple tests are to be considered, some mea-
sure of the risk of false positives over multiple tests is required.

7.3.1 Measures To carefully define different types of false positive measures,

of Multiple False Positives: Table 2 shows a cross classification of each of the I voxels in an
FWE and FDR image. Voxels can be truly null, or truly have nonzero signal, and
additionally each voxel can be detected by some (imperfect) thresh-
olding method, or fail to be detected.
To fill in the values for true null and true signal voxels, we need
to somehow know the underlying truth. Voxels that are marked as
significant are “detected” and voxels that are not marked as signifi-
cant are “not detected”
The standard measure of false positives in multiple testing is
the family-wise error (FWE) rate, that is, the chance of one or more
false positive voxels (or clusters) anywhere in the image [FWE = P
(IN+ > 0)]. Bonferroni is the most widely known FWE method,
which produces critical thresholds uFWE that controls the FWE. This
method simply divides the FWE rate (e.g., 0.05) by the number of
tests (e.g., 10,000), to give a voxel-wise P-value threshold α
(0.000005) that, when applied voxel-wise, results in the originally
desired FWE control. Regardless of the method, if an αFWE = 0.05
threshold is used, one can be 95 % confident that there are no false
positives in the image at all.
A more recent measure of false positives is the false discovery
rate (FDR), the expected false discovery proportion (FDP), where
FDP is the proportion of false positives among all reported posi-
tives. Precisely, if I+ voxels are detected, FDP is the proportion of
these that are false “discoveries,” FDP = IN+/I+, where FDP is
defined to be 0 if no voxels are detected. FDP is a random quantity
that cannot be known for any particular real dataset, so the FDR is
defined as the expected value of FDP over many datasets,
FDR = E(FDP). Figure 26 shows the difference between FDR and
FWE inference. Imagine the ten images shown in the figure as the
next ten experiments you will analyze; of course, you only consider
a single dataset at a time, but this illustrates how the methods are
calibrated over many (idealized) repetitions of an experiment.
 Statistical Analysis of fMRI Data 227

Table 2
Cross-tabulation of the number voxels in difference inference categories

Voxels not detected Voxels detected

True null voxels IN- IN+ IN
True signal voxels IS- IS+ IS
I− I+ I

Fig. 26 Comparison of uncorrected versus family-wise error (FWE)-corrected versus false discovery rate (FDR)-
corrected inferences. The top rows shows ten realizations of a central, circular signal added to smooth noise, and
the next three rows show different possible voxel-wise thresholding methods applied to each realization. The
dashed circles indicate the extent of the signal. The second row shows the result of an α = 10 % uncorrected
threshold; most of the signal is correctly detected, but much of the background is also incorrectly detected. On
average, 10 % of the background consists of false positives, but notice that the exact proportion of false positives
varies from realization to realization. The third row shows the result of using an αFWE = 10 % threshold (e.g., a
threshold from Bonferroni or random field theory). Much less of the signal is detected, but there are many fewer
false positives, only 1-in-10 of the datasets considered had any false positives, this is a FWE. Of course in prac-
tice, we never know if the dataset in our hands is the 1-in-10 (or 20) that contains a family-wise error. The bottom
row shows the result of using an αFDR = 10 % threshold. While we are guaranteed that the percentage of detected
voxels that are false positive does not exceed 10 % on average, the actual percentage can vary considerably
228 Mark W. Woolrich et al.

FDR is a more lenient measure of false positives allowing some

false positives—as a fraction of the number of detections—while
FWE regards any false positives as an error. One special case is
notable when the definitions of FDR and FWE coincide. If there
are no signal voxels at all (IS = 0), then FDP is 1 whenever there is
an FWE, and the two methods give the same control of false posi-
tives. (In technical terms, it is said that FDR has weak control of
FWE). This hints at the adaptive nature of FDR: when there is no
signal, it behaves like FWE; as there are more and more signal vox-
els, it admits more and more false positives, yielding increased
power while still controlling false positives in proportion.
The voxel-wise P-values referred to in previous sections are
more accurately referred to as “uncorrected P-values,” as they do
not account for the multiple testing problem. “Corrected P-values”
refer to the FWE rate (or FDR if this is used instead).

7.4 Corrected So far we have only defined measure of false positives, but we have
Inference Methods not described how we obtain thresholds that control these false
positive measures. In addition to Bonferroni, there are two further
methods that are commonly used in fMRI for controlling FWE;
these are RFT and permutation, whereas there is generally just a
single method for FDR.

7.4.1 Controlling FWE The Bonferroni method for controlling FWE uses a significance
with RFT threshold corresponding to α = αFWE/I, the nominal FWE test level
divided by the number of tests. Bonferroni becomes quite conserva-
tive when the data is smooth, and has no way to adapt to the data in
anyway. For example, imagine an extreme case where FMRI data is
smoothed with a 1 m wide Gaussian smoothing kernel; such data will
produce a statistic image with essentially a single constant value,
meaning there is no multiple testing problem anymore; however, the
Bonferroni threshold will still prescribe dividing αFWE by, say, 10,000.
RFT uses the smoothness of the data to adjust the significance
threshold while still controlling FWE. The mathematics involved are
elegant yet quite involved, and in what follows, we only give the most
cursory review. For a more detailed review, see [48], or for a more
technical overview, see [49]. The original Gaussian RFT paper for
PET imaging remains a useful introduction [50], though also see [51]
for more up-to-date results including T- and F-statistic RFT results.
To use RFT results, we must know the smoothness of the data,
precisely the smoothness of the standardized noise images (e/σ in
the notation from Sect. 3). Smoothness is parameterized by the full
width at half maximum (FWHM) of the Gaussian kernel required
to simulate images with the same apparent spatial smoothness as
our data (Fig. 27). For example, if we say that our data has 6-mm
FWHM smoothness, it means that if we were to simulate our data,
we would generate noise data with no correlation and then con-
volve it with a Gaussian kernel with FHWM of 6 mm. The exact
 Statistical Analysis of fMRI Data 229

Full Width at Half Maximum

Full
Width

Half
Maximum

Fig. 27 Full width at half maximum (FWHM) is a generic way to describe the
spread of a distribution, and is the way that smoothness is measured for random
field theory

form of the spatial dependence of our data does not have to follow
a Gaussian kernel, but for convenience, we describe the strength of
the dependence in terms of the size a Gaussian kernel.
It may seem that if we take our fMRI data fresh from the scan-
ner and convolve it with a 6-mm Gaussian kernel, our FWHM for
RFT would be 6 mm. This is incorrect, however, as the noise
smoothness includes both intrinsic sources of smoothness (imper-
fect MRI resolution, physiological artifacts, etc.) and smoothness
induced by the applied smoothing. As a result, the smoothness for
RFT is not a user-specified parameter, but rather estimated from
(
the residuals of the GLM ¡ - X b  .
)
7.4.2 RESELs The definition of FWHM smoothness creates a notion of a
smoothness-­equivalent volume, a resolution element or RESEL. If
the smoothness of the data is FWHMx, FWHMy, FWHMz in each
of the principal directions, then a volume of space with dimensions
FWHMx × FWHMy × FWHMz is one RESEL. In very approximate
terms, the RESEL count captures the amount of independent
information in the image; fewer RESELs = smoother data = less
information = less severe multiple testing problem; more
RESELS = rougher data = more severe multiple testing problem.
The total RESEL count for a search volume is:

RESELcount = I / ( FWHM x ´ FWHM y ´ FWHM z ) (18)

where I is the total number of voxels in the brain and FHWM is

expressed in units of voxels. The RESEL count is important
because it is the summary measure that determines the RFT thresh-
old, as illustrated next.

7.4.3 RFT-Corrected RFT can be used with any type of statistic image a GLM can create,
P-Values including T-, F-, and Z-statistic images [51]. The formulas provide
FWE-corrected P-values for voxel-wise thresholds and cluster sizes
230 Mark W. Woolrich et al.

and include corrections to account for edge effects (i.e., when

blobs touch the edge of the search volume). The simplest result,
for Z-statistic images with no edge corrections, can be used to gain
some insight into the method.
For voxel-wise inference, a voxel with value z has a corrected
P-value of:

æ z2 ö
FWE
Pvox ( z ) = RESELcount ( 2p )
-2
(z 2
- 1) exp ç - ÷
è 2ø

This shows that as z grows, the corrected P-value shrinks (the expo-
nential term dominates), which of course makes sense, as larger sta-
tistic values should produce smaller P-values. As the RESEL count
grows, the corrected P-value grows. The RESEL count can increase
because the search volume increases, which again is sensible, as a
greater search volume demands a greater correction for multiple
testing, and hence a less significant P-value. The RESEL count can
also increase if the smoothness decreases, as per [18], which
increases the amount of information in the image, again demanding
a greater correction for multiple testing. For cluster-wise inference,
the equation for the FWE-corrected P-value is more involved, but
it also accounts for the image search space and smoothness.

7.4.4 Small-Volume The first RFT results published (and the equation shown above)
Correction assumed that the search region was large relative to the smooth-
ness of the image. This assumption was needed to avoid dealing
with the case when a cluster touches the boundary of the search
region. To see why this could be a problem, imagine two statistic
images with the same smoothness, one the size and shape of the
brain, the other with the same total volume but the shape of a
long, narrow sausage. In the latter case, it is more likely that clus-
ters will touch the edge of the image, and, relative to other clus-
ters, have smaller volume. The results in [51] can be used to
produce P-values that are accurate even with small search regions.
When these results are used, they are sometimes referred to as
“small volume correction.”

7.4.5 RFT Assumptions The use of RFT results is based on several assumptions and approx-
imations. The essential assumptions are:
1. Gaussian data. For any collection of voxels, the distribution of
the data is multivariate Gaussian.
2. Sufficient smoothness. The data must be sufficiently smooth to
approximate continuous random fields (upon which the the-
ory is based).
3. Known smoothness. The results assume that the FWHM
smoothness parameters are exact and contain at most negligi-
ble error.
 Statistical Analysis of fMRI Data 231

4. Constant smoothness for cluster-wise inference only. The standard

cluster-wise results assume that the smoothness is the same
everywhere in the image. If the data are “nonstationary,”
regions of the brain that are smoother than others will generate
large clusters just by chance. Updated methods are available
[52] which account for varying nonstationarity, but they have
reduced sensitivity unless the nonstationarity is severe; hence,
this is principally suitable for voxel-based morphometry data.
Generally, FMRI data does not exhibit severe nonstationarity.
In the light of these extensive assumptions, there can be good
reason to seek alternative methods that do not require as many
assumptions.

7.4.6 Controlling FWE Nonparametric methods are generally used when the standard
with Permutation parametric assumptions are known to be false or cannot be verified.
In the case of RFT, the assumptions are nearly impossible to verify,
but more importantly, it has been found that voxel-wise RFT
results are quite conservative for small group studies (e.g., when
the number of subjects is less than about 40; [48, 53]). Hence,
there has been interest in using alternative methods.
Instead of making assumptions about the distribution of the
data, permutation testing uses the distribution of the data itself to
find P-values and thresholds. Figure 28 illustrates the reasoning of
the permutation test in the two-group setting.
While nonparametric tests are sometimes referred to as
assumption-­ free, in fact they also have assumptions, just much
weaker ones than standard parametric methods. The essential
assumption for the permutation test is exchangeability under the
null hypothesis. Exchangeability means that the data can be per-
muted (relative to the model) without altering its joint distribution.
fMRI data presents a challenge for permutation testing. At the first
level, temporal autocorrelation renders the data nonexchangeable
and permutation methods cannot be directly applied (the data must
be de-correlated, then permuted, and then re-­correlated (see [6,
54] for more details). However, in second-level analyses, exchange-
ability is generally not a problem. In the example in the figure, we
assume that, under the null hypothesis, all six subjects are exchange-
able—this is very reasonable because if there is no group effect (this
is the null hypothesis), then there is nothing special about the first
three subjects versus the last three. For this example, there are 20
possible ways of permuting the groups (including the correct label-
ing). For an arbitrary dataset with group sizes n1 and n2, the num-
ber of possible permutations is (n1 + n2)!/(n1!n2!).
By permuting the data many times, and for each permutation,
assuming that the resulting test statistic (in this case, the
­group-­difference T-statistic) is an sample of what we would see if
there were no real effect present, we build up a histogram of test
232 Mark W. Woolrich et al.

Fig. 28 Permutation test applied to data from a single voxel for a hypothetical two-group fMRI second-level
analysis

statistic values that will serve as the null distribution of that test
statistic. We can then look to see how much “area under the tail”
lies to the right of the actual test statistic value that we originally
observed (under the correct labeling of the data), and hence esti-
mate our P-value; this is the same principle for relating the null
distribution of the test statistic to the P-value as we saw in Fig. 9,
but in this case, the null distribution has been generated via a com-
pletely different methodology.
The permutation test for the two group case can be generalized
to three or more groups. In that case, we are testing the null
 Statistical Analysis of fMRI Data 233

hypothesis that all groups are the same, and use an F-test to measure
the evidence of any difference. Under the null hypothesis, all sub-
jects can be freely permuted. Likewise for a simple correlation
model, the null hypothesis of no association justifies the free per-
mutation of all of the subjects. The permutation test for the one
group case, however, is problematic without further assumptions: If
all we have is a single group, what is there to permute? Shuffling the
order of subjects will not change the value of a one-sample T-test.
In a second-level single group mean, the COPE images are
always created as relative differences between baseline and active
data. Even if an event-related design is used, and a contrast selects a
single predictor, the effective predictor is a subtraction of event and
baseline data. This is the case due to the relative, nonquantitative
nature of the BOLD signal. As a result, we use here a slightly differ-
ent assumption to generate “permutations.” Under the null hypoth-
esis, we assume that each individual’s second-level COPE data are
mean zero and have a symmetric distribution. Assuming mean zero
data is reasonable, as, under the null, we expect no activation, posi-
tive or negative. Assuming a symmetric distribution is a weakened
form of normality, and is exactly satisfied for any balanced effect
(i.e., a COPE constructed as difference of two averages, where an
equal number of scans contributed to each average).
The one-sample, group-level fMRI permutation thus works as
follows. Assuming mean zero, symmetrically distributed COPE data,
we randomly multiply each subject’s data by 1 or −1, or, equivalently,
randomly flip the signs of each subject’s COPE image. Since the data
are symmetrically distributed about zero, multiplication by −1 does
not alter the distribution, and we generate a realization that is equiva-
lent to the original data. If there are n subjects in the analysis, there
are 2n possible ways to flip the signs of the group-level data.
The permutation methods described so far will create uncor-
rected P-values at each voxel. Control of the FWE rate is easily
obtained with permutation testing via the following observation: In
complete-null data, an FWE occurs whenever one or more voxels
exceed the threshold, which occurs exactly when the voxel with the
largest statistic exceeds the threshold. Hence, inference based on the
permutation distribution of the largest (maximum) statistic provides
valid FWE inferences. Specifically, at each permutation, the maxi-
mum statistic value (across all voxels in the brain) is noted, creating
a null distribution of the maximum-across-space test statistic. The
95th percentile of that distribution gives an FWE-­corrected thresh-
old (“p < 0.05, corrected”), and any particular statistic value can be
compared to the maximum permutation distribution to obtain an
FWE-corrected P-value. Similarly, the maximal cluster size distribu-
tion can be created to provide FWE cluster-wise inferences.
It is important to note that, while permutation may appear to
be a completely different approach than those we described earlier,
in fact all pre-processing and modeling are generally the same, and
234 Mark W. Woolrich et al.

it is only the P-value computation that differs. This is because the

standard statistical models used generally have good sensitivity and
robustness properties and should be used unaltered. For example,
above we only discussed one- and two-sample T-tests, and did not
mention other traditional nonparametric test statistics based on
ranks, like the Wilcoxon Mann-Whitney test, as they often have
much less power. There is an exception, however with small group
data, with 20 or fewer subjects. With such small group data, there
can be substantial sensitivity gains by using a nonstandard statistic,
namely the smoothed variance T-test. As the fMRI data is generally
smoothed before statistical modeling, we expect the variance image
to be smooth as well. However, when the number of subjects is
very small, the DOF available to estimate the variance is very low,
and this can result in a noisy sample variance image. Smoothing
regularizes the estimated variance image, effectively increasing the
DOF and increasing sensitivity. While the null distribution of the
smoothed estimated variance T-statistic image is not known, and
so parametric methods cannot be used, nonparametric permuta-
tion methods can easily be used to generate FWE inferences based
on the smoothed variance results.
Finally, if the number of possible permutations is very large, it
can be impossible to compute them all. For example, for a 20 sub-
ject one-group analysis, there are over one million possible sign-­
flips of the data. In fact, it is sufficient to run a random subset of all
possible permutations. If only k of a large number of possible per-
mutations is used, the margin of error on the P–values is approxi-
mately ±2p p(1 − p) / k, where p is the true P-value. For a nominal
p of 0.05, this suggests that 1000 permutations is nearly sufficient
(ME = ± 0.014, or 28 % of 0.05), while 10,000 is probably more
than enough (ME = ± 0.0044, or 8.7 % of 0.05).

7.4.7 Controlling FDR The method for finding a threshold that controls FDR is surpris-
ingly simple. It is based only on the uncorrected voxel-wise P-values
in the statistic image. Let Pi be the P-value at voxel i, and P(i) be
the ordered P-values, P(1) ≤ P(2) ≤ · · · ≤ P(I). Then the largest index i
that satisfies
i
P(i ) £ a FDR (19)
I
defines the FDR threshold as P(i) [55]. This method works even
when there is positive dependence between voxels [56, 57].

7.4.8 Controlling False So far we have considered techniques that control the rate of false
Positives and True positives. This depends on knowing the null distribution (or non-
Negatives: Mixture activation distribution) for relevant statistics under the null hypoth-
Modeling esis. In contrast, mixture modeling provides us with a way of
estimating the “activating” and “nonactivating” distributions from
 Statistical Analysis of fMRI Data 235

the data itself. For example, nonactivating voxel statistics may be

modeled as coming from a (zero, or close-to-zero, mean) Gaussian
distribution, activating voxels statistics as coming from a Gamma
distribution, and de-activating voxels statistics as coming from a
negative Gamma distribution [58–60]. The means and variances of
these distributions are estimated from the whole statistic image. An
example is shown in Fig. 29. Note that the mixture modeling
approach is similar to the permutation approach (discussed in
Sect. 7.4.2) in that both approaches extract information about the
null distribution (or nonactivation distribution) from the data
itself. However, permutation methods extract information about
only the null distribution without making strong distributional
assumptions, whereas mixture modeling extracts information
about both the nonactivating and activating distributions by mak-
ing strong distributional assumptions.
Mixture modeling can provide a number of advantages over
null hypothesis testing. First, there is a well-known problem in null
hypothesis testing of FMRI in that if enough observations (e.g.,
time points) are made, then every voxel in the brain will reject the
null hypothesis [34]. This is because in practice no voxels will show
completely zero response to the stimulus, if only due to modeling
inadequacies such as unmodeled stimulus-correlated motion or the
point spread function of the scanner. By doing mixture modeling,

Mixture model fit to the histogram of Z-statistics

Z-statistic image
8
6
4
2
0 non-activation
–2

Probability of being activation

0.8
0.6 activation
0.4
0.2
0
–2 0 2 4 6 8
Z-statistic

Fig. 29 Mixture modeling of a Z-statistic image. Top-left: Four example slices of a Z-statistic image obtained
from fitting a general linear model (GLM) to the fMRI data at each voxel from an individual subject. The experi-
ment was a pain stimulus applied using a sparse single-event design. Right: Mixture model fit to the histogram
of Z-statistics. Nonactivating voxel are modeled as coming from a close-to-zero mean Gaussian distribution,
activating voxels as coming from a Gamma distribution, and de-activating voxels as coming from a negative
Gamma distribution. However, note that there were found to be no de-activating voxels in this case. Bottom-
left: Image showing the probability that a voxel is activating—this information that can be extracted from the
mixture model fit and can be used in thresholding to approximately control the true positive rate (TPR) as an
alternative to null hypothesis testing
236 Mark W. Woolrich et al.

we can overcome this by instead of asking the question “Is the

activation zero or not?,” we ask the question “Is the activation big-
ger than the overall background level of ‘activation’?.”
Mixture modeling also provides inference flexibility. Because
we have both the “activating” and “nonactivating” distributions,
we can calculate the probability of a voxel being “activating” and
the probability of a voxel being “nonactivating.” This provides us
with far more inference flexibility compared with null hypothesis
testing. We can still look to control the FPR by thresholding using
the probability of a voxel being “nonactivating.” But now we could
also look to approximately control the true positive rate (TPR) by
thresholding using the probability of a voxel being “activating.”
Controlling the TPR may be of real importance when using fMRI
for pre-surgery planning [61].
The wider-spread use of mixture modeling is currently some-
what hampered by the violation of the strong distributional
assumptions that need to be made. In the future, this may be allevi-
ated by the improvement of techniques such as ICA de-noising (see
Sect. 5.2.2) rendering the distributional assumptions valid.

7.5 Enhancing Aside from thresholding the final statistic images (either voxel-wise
Statistic Images or cluster-wise), it is not advisable to make image-processing
adjustments to statistic images. For example, smoothing a T-statistic
image would be disastrous: While a T-statistic has approximately
unit variance and follows a particular null distribution, a smoothed
T-statistic image will have dramatically reduced variance with no
particular distribution. Two exceptions to this are wavelet de-­
noising methods and a recently proposed threshold-free cluster
enhancement (TFCE) method.
Wavelet methods transform the data in a scale-dependent fash-
ion, so that all of the large-scale information is segregated from the
fine-scale information. Since we generally expect the signals of
interest to be spatially extended, wavelet methods can be used to
“shrink” variation associated with the finest scales, “de-noising”
the image, while preserving the large-scale structure. For an over-
view of wavelet methods applied to fMRI, see [62].
Cluster-wise inference also tries to capture spatially extended
signals, but requires the specification of an arbitrary cluster-­forming
threshold uclus. TFCE removes this dependence by, in essence,
using all possible uclus values and then merging all the results into a
single image. Specifically, at each voxel, let ei(h) be the extent of
the cluster that voxel i belongs to with cluster-forming threshold h
åe ( h )
E
(or 0 if ti < h). Then, the TFCE image is defined by i hH ,
h>0
where the sum is computed for a discrete set of h values, from 0 to
the maximum statistic value, and E and H are tuning parameters.
In [63], E = 0.5 and H = 2 were found to give generally good per-
formance for a range of classes of signals. TFCE seems to succeed
 Statistical Analysis of fMRI Data 237

in matching or exceeding the sensitivity of optimized cluster-based

thresholding without the arbitrariness and instability of the
smoothing and initial thresholding. There is no known distribu-
tion for the TFCE image, and so permutation testing is used to
convert the TFCE image into P-values.

References
1. Friston K, Worsley K, Frackowiak R, Mazziotta 12. Worsley K, Liao C, Aston J et al (2002) A
J, Evans A (1994) Assessing the significance general statistical analysis for fMRI data.
of focal activations using their spatial extent. NeuroImage 15:1–15
Hum Brain Mapp 1:214–220 13. Gautama T, Van Hulle MM (2004) Optimal spa-
2. Hykin J, Bowtell R, Glover P, Coxon R, tial regularisation of autocorrelation estimates in
Blumhardt L, Mansfield P (1995) Investigation fMRI analysis. Neuroimage 23:1203–1216
of the linearity of functional activation signal 14. Penny W, Kiebel S, Friston K (2003)
changes in the brain using echo planar imag- Variational Bayesian inference for fMRI time
ing (EPI) at 3.0 T. In: Proc of the SMR and series. NeuroImage 19:1477–1491
ESMRB Joint Meeting. p 795 15. Woolrich M, Behrens T, Smith S (2004)
3. Cohen M (1997) Parametric analysis of fMRI Constrained linear basis sets for HRF mod-
data using linear systems methods. NeuroImage elling using Variational Bayes. NeuroImage
6:93–103 21:1748–1761
4. Dale A, Buckner R (1997) Selective averag- 16. Smith S, Jenkinson M, Beckmann C, Miller
ing of rapidly presented individual trials using K, Woolrich M (2007) Meaningful design and
fMRI. Hum Brain Mapp 5:329–340 contrast estimability in fMRI. NeuroImage
5. Burock MA, Buckner RL, Woldorff MG, 34:127–136
Rosen BR, Dale AM (1998) Randomized 17. Dale A, Greve D, Burock M (1999) Optimal
event-related experimental designs allow for stimulus sequences for event-related
extremely rapid presentation rates using func- fMRI. NeuroImage 9:S33
tional MRI. NeuroReport 9:3735–3739 18. Wager T, Nichols T (2003) Optimization of
6. Bullmore E, Brammer M, Williams S et al experimental design in fMRI: a general frame-
(1996) Statistical methods of estimation and work using a genetic algorithm. Neuroimage
inference for functional MR image analysis. 18:293–309
Magn Reson Med 35:261–277 19. Josephs O, Turner R, Friston K (1997) Event-­
7. Friston K, Josephs O, Zarahn E, Holmes A, related fMRI. Hum Brain Mapp 5:1–7
Rouquette S, Poline J-B (2000) To smooth or 20. Lange N, Zeger S (1997) Non-linear Fourier
not to smooth? NeuroImage 12:196–208 time series analysis for human brain mapping
8. Woolrich M, Ripley B, Brady J, Smith S (2001) by functional magnetic resonance imaging.
Temporal autocorrelation in univariate lin- Appl Stat 46:1–29
ear modelling of FMRI data. NeuroImage 21. Genovese C (2000) A Bayesian time-course model
14:1370–1386 for functional magnetic resonance imaging data
9. Locascio J, Jennings P, Moore C, Corkin S (with discussion). J Am Stat Assoc 95:691–703
(1997) Time series analysis in the time domain 22. Friston KJ (2002) Bayesian estimation
and resampling methods for studies of func- of dynamical systems: an application to
tional magnetic resonance brain imaging. Hum fMRI. NeuroImage 16:513–530
Brain Mapp 5:168–193
23. Marrelec G, Benali H, Ciuciu P, Pélégrini-Issac
10. Purdon P, Weisskoff R (1998) Effect of tem- M, Poline J-B (2003) Robust Bayesian estima-
poral autocorrelation due to physiological tion of the hemodynamic response function in
noise and stimulus paradigm on voxel-level event-related BOLD MRI using basic physio-
false-­positive rates in fMRI. Hum Brain Mapp logical information. Hum Brain Mapp 19:1–17
6:239–249
24. Woolrich M, Jenkinson M, Brady J, Smith S
11. Marchini J, Ripley B (2000) A new statistical (2004) Fully Bayesian spatio-temporal model-
approach to detecting significant activation in ling of FMRI data. IEEE Trans Med Imaging
functional MRI. NeuroImage 12:366–380 23:213–231
238 Mark W. Woolrich et al.

25. Buxton R, Uludag K, Dubowitz D, Liu T ses using robust regression. NeuroImage
(2004) Modeling the hemodynamic response to 26:99–113
brain activation. NeuroImage 23(S1):220–233 41. Woolrich M (2008) Robust group analysis using
26. Boynton G, Engel S, Glover G, Heeger D outlier inference. NeuroImage 41:286–301
(1996) Linear systems analysis of functional 42. Meriaux S, Roche A, Dehaene-Lambertz G,
magnetic resonance imaging in human V1. Thirion B, Poline J (2006) Combined permuta-
J Neurosci 16:4207–4221 tion test and mixed-effect model for group average
27. Glover G (1999) Deconvolution of analysis in fMRI. Hum Brain Mapp 27:402–410
impulse response in event-related BOLD 43. Roche A, Meriaux S, Keller M, Thirion B
fMRI. NeuroImage 9:416–429 (2007) Mixed-effect statistics for group analy-
28. Friston K, Josephs O, Rees G, Turner R sis in fMRI: a nonpara-metric maximum likeli-
(1998) Nonlinear event-related responses in hood approach. Neuroimage 38:501–510
fMRI. Magn Reson Med 39:41–52 44. Thirion B, Pinel P, Mriaux S, Roche A, Dehaene
29. Beckmann C, Smith S (2004) Probabilistic S, Poline J (2007) Analysis of a large fMRI
independent component analysis for functional cohort: statistical and methodological issues
magnetic resonance imaging. IEEE Trans Med for group analyses. Neuroimage 35:105–120
Imaging 23:137–152 45. Hartvig NV, Jensen JL (2000) Spatial mixture
30. Glover G, Li T, Ress D (2000) Image-based modeling of fMRI data. Hum Brain Mapp
method for retrospective correction of physi- 11:233–248
ological motion effects in fMRI: Retroicor. 46. Hayasaka S, Nichols TE (2003) Validating
Magn Reson Med 44:162–167 cluster size inference: random field and permu-
31. Holmes A, Friston K (1998) Generalisability, tation methods. NeuroImage 20:2343–2356
random effects & population inference. Fourth 47. Friston KJ, Holmes A, Poline J-B, Price CJ,
Int Conf on Functional Mapping of the Human Frith CD (1996) Detecting activations in
Brain. NeuroImage 7:S754 PET and fMRI: levels of inference and power.
32. Talairach J, Tournoux P (1988) Co-planar NeuroImage 4:223–235
stereotaxic atlas of the human brain. Thieme, 48. Nichols TE, Hayasaka S (2003) Controlling
New York the familywise error rate in functional neuro-
33. Collins D, Neelin P, Peters T, Evans A imaging: a comparative review. Stat Methods
(1994) Automatic 3D intersubject regis- Med Res 12:419–446
tration of MR volumetric data in standard- 49. Cao J, Worsley KJ (2001) Applications of ran-
ized Talairach space. J Comput Assist Tomo dom fields in human brain mapping. In: Moore
18:192–205 M, (ed) Spatial statistics: methodological
34. Friston KJ, Penny W, Phillips C, Kiebel S, aspects and applications, vol 159, Springer lec-
Hinton G, Ashburner J (2002) Classical and ture notes in statistics. Springer. pp 169–182
Bayesian inference in neuroimaging: theory. 50. Worsley KJ, Evans AC, Marrett S, Neelin P
NeuroImage 16:465–483 (1992) Three-dimensional statistical analy-
35. Beckmann C, Jenkinson M, Smith S (2003) sis for cbf activation studies in human brain.
General multi-level linear modelling for group J Cerebr Blood F Met 12:900–918
analysis in FMRI. NeuroImage 20:1052–1063 51. Worsley KJ, Marrett S, Neelin P, Vandal AC,
36. Woolrich M, Behrens T, Beckmann C, Jenkinson Friston KJ, Evans AC (1996) A unified statisti-
M, Smith S (2004) Multi-level linear modelling cal approach for determining significant signals
for FMRI group analysis using Bayesian infer- in images of cerebral activation. Hum Brain
ence. NeuroImage 21:1732–1747 Mapp 4:58–73
37. Kherif F, Poline J-B, Meriaux S, Benali H, Flandin 52. Hayasaka S, Luan Phan K, Liberzon I, Worsley
G, Brett M (2003) Group analysis in functional KJ, Nichols TE (2004) Nonstationary cluster-­
neuroimaging: selecting subjects using similarity size inference with random field and permuta-
measures. Neuroimage 20:2197–2208 tion methods. NeuroImage 22:676–687
38. Luo W-L, Nichols TE (2003) Diagnosis and 53. Nichols T, Holmes A (2001) Nonparametric
exploration of massively univariate neuroimag- permutation tests for functional neuroimaging: a
ing models. Neuroimage 19:1014–1032 primer with examples. Hum Brain Mapp 15:1–25
39. Seghier M, Friston K, Price C (2007) Detecting 54. Bullmore E, Long C, Suckling J et al (2001)
subject-specific activations using fuzzy cluster- Colored noise and computational inference in
ing. Neuroimage 36:594–605 neurophysiological (fMRI) time series analysis:
40. Wager T, Keller M, Lacey S, Jonides J (2005) resampling methods in time and wavelet
Increased sensitivity in neuroimaging analy- domains. Hum Brain Mapp 12:61–78
 Statistical Analysis of fMRI Data 239

55. Benjamini Y, Hochberg Y (1995) Controlling 60. Woolrich M, Behrens T (2006) Variational
the false discovery rate: a practical and power- Bayes inference of spatial mixture models for
ful approach to multiple testing. J R Stat Soc segmentation. IEEE Trans Med Imaging
Ser B Methodol 57:289–300 25:1380–1391
56. Genovese C, Lazar N, Nichols T (2002) 61. Bartsch A, Homola G, Biller A, Solymosi L,
Thresholding of statistical maps in functional Bendszus M (2006) Diagnostic functional
neuroimaging using the false discovery rate. MRI: illustrated clinical applications and
NeuroImage 15:870–878 decision-­
making. J Magn Reson Imaging
57. Benjamini Y, Yekutieli D (2001) The control of 23:921–932
the false discovery rate in multiple testing 62. Van De Ville D, Blu T, Unser M (2006) Surfing
under dependency. Ann Stat 29:1165–1188 the brain – an overview of wavelet-based tech-
58. Everitt B, Bullmore E (1999) Mixture model niques for fMRI data analysis. IEEE Eng Med
mapping of brain activation in functional mag- Biol 25:65–78
netic resonance images. Hum Brain Mapp 63. Smith SM, Nichols TE (2008) Threshold-free
7:1–14 cluster enhancement: addressing problems of
59. Hartvig N (2000) A stochastic geometry smoothing, threshold dependence and localisa-
model for fMRI data. Technical Report 410. tion in cluster inference. NeuroImage.
Department of Theoretical Statistics, University doi:10.1016/j.neuroimage.2008.03.061, In
of Aarhus press; Epub ahead of print April 11, 2008
 Chapter 8

Dynamic Causal Modeling of Brain Responses

Karl J. Friston

Abstract
This chapter is about modeling-distributed brain responses and, in particular, the functional integration
among neuronal systems. Inferences about the functional organization of the brain rest on models of how
measurements of evoked responses are caused. These models can be quite diverse, ranging from concep-
tual models of functional anatomy to mathematical models of neuronal and hemodynamics. The aim of
this chapter is to introduce dynamic causal models. These models can be regarded as generalizations of the
simple models employed in conventional analyses of regionally specific brain responses. In what follows,
we will start with anatomical models of functional brain architectures, which motivate some of the basic
principles of neuroimaging. We then review briefly statistical models (e.g., the general linear model) used
for making classical and Bayesian inferences about where neuronal responses are expressed. By incorporat-
ing biophysical constraints, these basic models can be finessed and, in a dynamic setting, rendered causal.
This allows us to infer how interactions among brain regions are mediated. This chapter focuses on causal
models for distributed responses measured with fMRI and electroencephalography. The latter is based on
neural-mass models and affords mechanistic inferences about how evoked responses are caused, at the level
of neuronal subpopulations and the coupling among them.

Key words Functional connectivity, Effective connectivity, Dynamic causal modeling, Causal,
Dynamic, Nonlinear

1 Introduction

Neuroscience depends on conceptual, anatomical, statistical, and

causal models that link ideas about how the brain works to observed
neuronal responses. Here, we highlight the relationships among the
sorts of models that are employed in imaging, with a special focus on
dynamic causal models of functional brain architectures. We will show
how simple statistical models used to identify where evoked brain
responses are expressed (cf, neo-phrenology) can be elaborated to
provide models of how neuronal responses are caused (e.g., dynamic
causal modeling—DCM). We will review a series of models that range
from conceptual models, motivating experimental design, to detailed
biophysical models of coupled neuronal ensembles that enable ques-
tions to be asked, at a physiological and computational level.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_8, © Springer Science+Business Media New York 2016

241
242 Karl J. Friston

Anatomical models of functional brain architectures motivate

the fundaments of neuroimaging. In the first section, we review
the distinction between functional specialization and integration
and how these principles serve as the basis for most models of neu-
roimaging data. The next section turns to simple statistical models
(e.g., the general linear model—GLM) used for making classical
and Bayesian inferences about functional specialization, in terms of
where neuronal responses are expressed. By incorporating biologi-
cal constraints, simple observation models can be made more real-
istic and, in a dynamic framework, causal. This section concludes
by considering the biophysical modeling of hemodynamic
responses. All the models considered in this section pertain to
regional responses. In the final section, we focus on models of dis-
tributed responses, where the interactions among cortical areas or
neuronal subpopulations are modeled explicitly. This section cov-
ers the distinction between functional and effective connectivity and
reviews DCM of functional integration, using fMRI and electroen-
cephalogram (EEG). We conclude with an example from event-­
related potential (ERP) research and show how the mismatch
negativity (MMN) can be explained by changes in coupling among
neuronal sources that may underlie perceptual learning.

2 Anatomical Models

2.1 Functional From a historical perspective, the distinction between functional

Specialization specialization and functional integration relates to the dialectic
and Integration between localizationism and connectionism that dominated thinking
about brain function in the nineteenth century. Since the formula-
tion of phrenology by Gall, who postulated fixed one-to-one rela-
tions between particular parts of the brain and specific mental
attributes, the identification of a particular brain region with a spe-
cific function has become a central theme in neuroscience. Somewhat
ironically, the notion that distinct brain functions could be localized
in the brain was strengthened by early scientific attempts to refute
the phrenologists’ claims. In 1808, a scientific committee of the
Athénée at Paris, chaired by Cuvier, declared that phrenology was
an unscientific and invalid theory [1]. This conclusion, which was
not based on experimental results, may have been enforced by
Napoleon Bonaparte (who, allegedly, was not amused after Gall’s
phrenological examination of his own skull did not give the flatter-
ing results expected). During the following decades, lesion and
electrical stimulation paradigms were developed to test whether
functions could indeed be localized in animal models. Initial lesion
experiments by Flourens on pigeons were i­ncompatible with phre-
nologist predictions, but later experiments, including stimulation
experiments in dogs and monkeys by Fritsch, Hitzig, and Ferrier,
supported the idea that there was a relation between distinct brain
 Dynamic Causal Modeling of Brain Responses 243

regions and certain cognitive or motor functions. Additionally, cli-

nicians like Broca and Wernicke showed that patients with focal
brain lesions in particular locations showed specific impairments.
However, it was realized early on that, in spite of these experimental
findings, it was generally difficult to attribute a specific function to
a cortical area, given the dependence of cerebral activity on the
anatomical connections between distant brain regions; for example,
a meeting that took place on 4 August 1881 addressed the difficul-
ties of attributing function to a cortical area, given the dependence
of cerebral activity on underlying connections [2]. This meeting
was entitled “localisation of function in the cortex cerebri.” Goltz
[3], although accepting the results of electrical stimulation in dog
and monkey cortex, considered that the excitation method was
inconclusive, in that movements elicited might have originated in
related pathways, or current could have spread to distant centers. In
short, the excitation method could not be used to infer functional
localization because localizationism discounted interactions or
functional integration among different brain areas. It was proposed
that lesion studies could supplement excitation experiments.
Ironically, it was the observations on patients with brain lesions
some years later (see Ref. [4]) that led to the concept of disconnec-
tion syndromes and the refutation of localizationism as a complete or
sufficient explanation of cortical organization. Functional localiza-
tion implies that a function can be localized in a cortical area,
whereas specialization suggests that a cortical area is specialized for
some aspects of perceptual or motor processing, and that this spe-
cialization is anatomically segregated within the cortex. The cortical
infrastructure supporting a single function may then involve many
specialized areas whose union is mediated by the functional integra-
tion among them. In this view, functional specialization is only
meaningful in the context of functional integration and vice versa.

2.2 Functional The functional role of any component (e.g., cortical area, sub-area,
Specialization or neuronal population) of the brain is defined largely by its con-
and Segregation nections. Certain patterns of cortical projections are so common
that they could amount to rules of cortical connectivity. “These
rules revolve around one, apparently, over-riding strategy that the
cerebral cortex uses—that of functional segregation” [5].
Functional segregation demands that cells with common func-
tional properties be grouped together. This architectural constraint
necessitates both convergence and divergence of cortical connec-
tions. Extrinsic connections among cortical regions are not
­continuous but occur in patches or clusters. This patchiness has, in
some instances, a clear relationship to functional segregation. For
example, when recordings are made in V2, directionally selective
(but not wavelength or color selective) cells are found exclusively
in its thick stripes. Retrograde (i.e., backward) labeling of cells in
V5 is limited to these thick stripes; all the available physiological
244 Karl J. Friston

evidence suggests that V5 is a functionally homogeneous area that

is specialized for visual motion. Evidence of this nature supports
the notion that patchy connectivity is the anatomical infrastructure
that mediates functional segregation and specialization. If it is the
case that neurons in a given cortical area share a common respon-
siveness, by virtue of their extrinsic connectivity, to some senso-
rimotor or cognitive attribute, then this functional segregation is
also an anatomical one.
In summary, functional specialization suggests that challeng-
ing a subject with the appropriate sensorimotor attribute or cogni-
tive process should lead to activity changes in, and only in, the
specialized areas. This is the anatomical and physiological model
upon which the search for regionally specific effects is based. We
will deal briefly with models of regionally specific responses and
return to models of functional integration.

3 Statistical Models

3.1 Statistical Functional mapping studies are usually analyzed with some form of
Parametric Mapping statistical parametric mapping (SPM). SPM entails the construc-
tion of continuous statistical maps (e.g., t-maps) to test hypotheses
about regionally specific effects [6]. SPM uses the GLM and ran-
dom field theory (RFT) to analyze and make classical inferences
about brain responses. Parameters of the GLM are estimated in
exactly the same way as in conventional analysis of discrete data.
RFT is used to resolve the multiple-comparisons problem induced
by making inferences over a volume of the brain. RFT provides a
method for adjusting p-values for the search volume of an SPM to
control false positive rates. It plays the same role for continuous
data (i.e., images or time series) as the Bonferroni correction for a
family of discontinuous or discrete statistical tests.
There is a Bayesian alternative to classical inference with SPMs.
This rests on conditional inferences about an effect, given the data,
as opposed to classical inferences about the data, given the effect is
zero. Bayesian inferences about effects that are continuous in space
use posterior probability maps (PPMs). Although less established
than SPMs, PPMs are potentially useful, not least because they do
not have to contend with the multiple-comparisons problem
induced by classical inference (see Ref. [7]). In contradistinction to
SPM, this means that inferences about a given regional response
do not depend on inferences about responses elsewhere. Bayesian
inference is particularly relevant to dynamic casual modeling
because the Bayesian formulation is an essential part of model
specification and inversion. Before looking at the models underly-
ing Bayesian inference, we briefly review estimation and classical
inference in the context of the GLM and show how this can be
generalized to give a Bayesian approach.
 Dynamic Causal Modeling of Brain Responses 245

3.2 General Linear The GLM is a simple equation

Model
y = Xb +e (1)

that expresses an observed response y in terms of a linear

combination of explanatory variables in the design matrix X, plus
a well-­behaved error term. The GLM is variously known as analy-
sis of variance or multiple-regression and subsumes simpler vari-
ants, like the t-test for a difference in means, to more elaborate
linear convolution models (see below). Each column of the design
matrix models a cause of the data. These are referred to as explan-
atory variables, covariates, or regressors. Sometimes the design
matrix contains covariates or indicator variables that take values of
zero or one, to indicate the presence of a particular level of an
experimental factor (cf, analysis of variance). The relative contri-
bution of each of these columns to the response is controlled by
the parameters, β. Inferences about the parameter estimates are
made using t or F-statistics, as in conventional statistics. Having
computed the statistic, RFT is used to assign adjusted p-values to
topological features of the SPM, such as the height of peaks or the
spatial extent of blobs. This p-value is a function of the search
volume and smoothness. The intuition behind RFT is that it con-
trols the false positive rate of peaks corresponding to regional
effects. A Bonferroni correction would control the false positive
rate of voxels, which is inexact and unnecessarily severe. The
p-value is the probability of getting a peak in the SPM, or higher,
by chance over the search volume. If sufficiently small (usually less
than 0.05), the regional effect is declared significant.

3.3 Classical Inference in neuroimaging is restricted largely to classical infer-

and Bayesian ences based upon SPMs. The statistics that comprise these SPMs
Inference are essentially functions of the data. The probability distribution of
the chosen statistic, under the null hypothesis (i.e., the null distri-
bution), is used to compute a p-value. This p-value is the probabil-
ity of obtaining the statistic, or the data, given that the null
hypothesis is true. If sufficiently small, the null hypothesis is
rejected and an inference is made. The alternative approach is to
use Bayesian or conditional inference based upon the posterior dis-
tribution of the activation given the data [8]. This necessitates the
specification of priors (i.e., the probability distribution of the acti-
vation or model parameter). Bayesian inference requires the condi-
tional or posterior distribution and therefore rests upon a posterior
density analysis. A useful way to summarize this posterior density is
to compute the probability that the activation exceeds some thresh-
old. This represents a Bayesian inference about the effect, in rela-
tion to the specified threshold. By computing posterior probability
for each voxel, we can construct PPMs that are a useful comple-
ment to classical SPMs.
246 Karl J. Friston

The motivation for using conditional or Bayesian inference is

that it has high face validity. This is because the inference is about
an effect, or activation, being greater than some specified size that
has some meaning in relation to underlying neurophysiology. This
contrasts with classical inference, in which the inference is about
the effect being significantly different than zero. The problem for
classical inference is that trivial departures from the null hypothesis
can be declared significant, with sufficient data or sensitivity. From
the point of view of neuroimaging, posterior inference is especially
useful because it eschews the multiple-comparisons problem. In
classical inference, one tries to ensure that the probability of reject-
ing the null hypothesis incorrectly is maintained at a small rate,
despite making inferences over large volumes of the brain. This
induces a multiple-comparisons problem that, for spatially contin-
uous data, requires an adjustment or correction to the p-value
using RFT as mentioned earlier. This correction means that classi-
cal inference becomes less sensitive or powerful with large search
volumes. In contradistinction, posterior inference does not have to
contend with the multiple-comparisons problem because there are
no false-positives. The probability that activation has occurred,
given the data, at any particular voxel is the same, irrespective of
whether one has analyzed that voxel or the entire brain. For this
reason, posterior inference using PPMs represents a relatively more
powerful approach than classical inference in neuroimaging.

3.3.1 Hierarchical PPMs require the posterior distribution or conditional distribution

Models and Empirical of the activation (a contrast of conditional parameter estimates),
Bayes given the data. This posterior density can be computed, under
Gaussian assumptions, using Bayes rule. Bayes rule requires the
specification of a likelihood function and the prior density of the
model parameters. The models used to form PPMs and the likeli-
hood functions are exactly the same as in classical SPM analyses,
namely, the GLM. The only extra information that is required is
the prior probability distribution of the parameters. Although it
would be possible to specify those using independent data or some
plausible physiological constraints, there is an alternative to this
fully Bayesian approach. The alternative is empirical Bayes in which
the prior distributions are estimated from the data. Empirical Bayes
requires a hierarchical observation model where the parameters and
hyper-parameters at any particular level can be treated as priors on
the level below. There are numerous examples of hierarchical
observation models in neuroimaging. For example, the distinction
between fixed- and mixed-effects analyses of multisubject studies
relies upon a two-level hierarchical model. However, in neuroim-
aging, there is a natural hierarchical observation model that is com-
mon to all brain mapping experiments. This is the hierarchy
induced by looking for the same effects at every voxel within the
brain (or gray matter). The first level of the hierarchy corresponds
 Dynamic Causal Modeling of Brain Responses 247

to the experimental effects at any particular voxel and the second

level comprises the effects over voxels. Put simply, the variation in
a contrast, over voxels, can be used as the prior variance of that
contrast at any particular voxel. Hierarchical linear models have the
following form:

y = X ( ) b ( )e ( )
1 1 1

b (1) = X ( 2) b ( 2)e ( 2) (2)

b ( 2) = ¼

This is exactly the same as Eq. (1) but now the parameters of the
first level are generated by a supra-ordinate linear model and so on
to any hierarchical depth required. These hierarchical observation
models are an important extension of the GLM and are usually
estimated using expectation maximization (EM) [9]. In the pres-
ent context, the response variables comprise the responses at all
voxels and β(1)s are the treatment effects we want to make an infer-
ence about. Because we have invoked a second level, the first-level
parameters embody random effects and are generated by a second-­
level linear model. At the second level, β(2) is the average effect over
voxels and ε(2) is its voxel-to-voxel variation. By estimating the vari-
ance of ε(2), one is implicitly estimating an empirical prior on the
first-level parameters at each voxel. This prior can then be used to
estimate the posterior probability of β(1) being greater than some
threshold at each voxel. An example of the ensuing PPM is pro-
vided in Fig. 1 along with the classical SPM.
In summary, we have seen how the GLM can be used to test
hypotheses about brain responses and how, in a hierarchical form,
it enables empirical Bayesian or conditional inference. Then, we
deal with the dynamic systems and how they can be formulated as
GLMs. These dynamic models take us closer to how brain responses
are actually caused by experimental manipulations and represent
the next step towards dynamic causal models of brain responses.

3.4 Dynamic Models In Friston et al. [10], the form of the impulsed hemodynamic
response function (HRF) was estimated using a least squares de-­
3.4.1 Convolution
convolution and a linear time invariant model, where evoked neu-
Models and Temporal
ronal responses are convolved or smoothed with an HRF to give the
Basis Functions
measured hemodynamic response (see also Ref. [11]). This simple
linear convolution model is the cornerstone for making statistical
inferences about activations in fMRI with the GLM. An impulse
response function is the response to a single impulse, measured at
a series of times after the input. It characterizes the input–output
behavior of the system (i.e., voxel) and places important constraints
on the sorts of inputs that will excite a response.
Knowing the form of the HRF is important for several reasons,
not least because it furnishes better statistical models of the data.
248 Karl J. Friston

contrast

100
SPM PPM
200

300

1 2 3 4
Design matrix
z = 3mm z = 3mm

Fig. 1 Statistical parametric mapping (SPM) and posterior probability map (PPM) for an fMRI study of attention
to visual motion. The display format (lower panel) uses an axial slice through extra-striate regions but the
thresholds are the same as employed the in maximum-intensity projections (upper panels). Upper right: The
activation threshold for the PPM was 0.7 au, meaning that all voxels shown had a 90 % chance of an activation
of 0.7 % or more. Upper left: The corresponding SPM using an adjusted threshold at p = 0.05. Note the bilateral
foci of motion-related responses in the PPM that are not seen in the SPM (gray arrows). As can be imputed
from the design matrix (upper-middle panel), the statistical model of evoked responses comprised boxcar
regressors convolved with a canonical hemodynamic response function. The middle column corresponds to
the presentation of moving dots and was the stimulus attribute tested by the contrast

The HRF may vary from voxel to voxel and this has to be accom-
modated in the GLM. To allow for different HRFs in different
brain regions, temporal basis functions were introduced [12] to
model evoked responses in fMRI and applied to event-related
responses in Josephs et al. [13] (see also Ref. [14]). The basic idea
behind temporal basis functions is that the hemodynamic response,
induced by any given trial type, can be expressed as the linear com-
bination of (basis) functions of peri-stimulus time. The convolution
model for fMRI responses takes a stimulus function encoding the
neuronal responses and convolves it with an HRF to give a regres-
sor that enters the design matrix. When using basis functions, the
stimulus function is convolved with each basis function to give a
series of regressors. Mathematically, we can express this model as
 Dynamic Causal Modeling of Brain Responses 249

y (t ) = X b + e y (t ) = (t ) Ä h (t )
Û (3)
X i = Ti ( t ) Ä u ( t ) h ( t ) = b1T1 ( t ) + b 2T2 ( t ) + ¼

where ⊗ means convolution. This equivalence shows how any convo-

lution model (right) can be converted into a GLM (left), using tem-
poral basis functions. The parameter estimates are the coefficients or
weights that determine the mixture of basis functions of time Ti(t)
that models h(t), the HRF for the trial type and voxel in question. We
find the most useful basis set to be a canonical HRF and its deriva-
tives with respect to the key parameters that determine its form (see
below). Temporal basis functions are important because they provide
a graceful transition between conventional multilinear regression
models with one stimulus function per condition and finite impulse
response (FIR) models with a parameter for each time point, follow-
ing the onset of a condition or trial type. Figure 2 illustrates this

Fig. 2 Temporal basis functions offer useful constraints on the form of the estimated response that retain the
flexibility of finite impulse response (FIR) models and the efficiency of single regressor models. The specifica-
tion of these constrained FIR models involves setting up stimulus functions u(t) that model expected neuronal
changes, for example, boxcar-functions of epoch-related responses or spike-(δ)-functions at the onset of
specific events or trials. These stimulus functions are then convolved with a set of basis functions Ti(t) of peri-­
stimulus time that, in some linear combination, model the HRF. The ensuing regressors are assembled into the
design matrix. The basis functions can be as simple as a single canonical HRF (middle), through to a series of
top-hat-functions δi(t) (bottom). The latter case corresponds to an FIR model and the coefficients constitute
estimates of the impulse response function at a finite number of discrete sampling times. Selective averaging
in event-related fMRI [39] is mathematically equivalent to this limiting case
250 Karl J. Friston

graphically. In short, temporal basis functions offer useful constraints

on the form of the estimated response that retain the flexibility of FIR
models and the efficiency of single regressor models.

3.5 Biophysical By adopting a convolution model for brain responses in fMRI, we

Models are implicitly positing a dynamic system that converts neuronal
3.5.1 Input-State-Output responses into observed hemodynamic responses. Our
Systems ­understanding of the biophysical and physiological mechanisms
that underpin the HRF has grown considerably in the last few years
(e.g., [15–17]). Figure 3 shows some simulations based on the
hemodynamic model described in Friston et al. [18]. Here, neuro-
nal activity induces some autoregulated vasoactive signal that
causes transient increases in regional cerebral blood flow (rCBF).
The resulting flow increases dilate a venous balloon, increasing its
volume, and diluting venous blood to decrease deoxyhemoglobin
content. The blood oxygen level dependent (BOLD) signal is
roughly proportional to the concentration of deoxyhemoglobin
and follows the rCBF response with about a second delay. The
model is framed in terms of differential equations, examples of
which are provided in left panel.

neuronal input
u(t)
induced signal v&q

normalized volume & dcoxyHb

0.5 1.15
hidden states
normalized flow signal

0.4
1.1
s, f, v, q 0.3
activity-dependent signal 0.2 1.05
0.1
s˙ = u − KS − ?(f −1) 1
0
0.95
f -0.1
-0.2 0.0
flow induction 0 10 20 30 0 10 20 30

f˙ = s
f hemodynamics

rCBF response BOLD response

1.5 1.5
changes in volume changes in dHb
v 1.4
1
normalized flow

tv˙ = f − v1/a tq˙ = f E(f. r)/ r − v1/a q/v

percent cahge

1.3

v q 1.2 0.5
1.1
0
1
0.9 0.5
0 10 20 30 0 10 20 30
time secs time secs
y = l(v.q)

Fig. 3 Right: Hemodynamics elicited by an impulse of neuronal activity as predicted by a dynamical biophysical
model (left). A burst of neuronal activity causes an increase in flow-inducing signal that decays with first-order
kinetics and is downregulated by local flow. This signal increases regional cerebral blood flow (rCBF), which
dilates the venous capillaries, increasing volume v. Concurrently, venous blood is expelled from the venous
pool decreasing deoxyhemoglobin content q. The resulting fall in deoxyhemoglobin concentration leads to a
transient increases in blood oxygen level dependent (BOLD) signal and a subsequent undershoot. Left:
Hemodynamic model; on which these simulations were based
 Dynamic Causal Modeling of Brain Responses 251

Note that we have introduced variables like volume and

deoxyhemoglobin concentrations that are not actually observed.
These are referred to as the hidden states of input-state-output models.
The state and output equations of any analytic dynamical system are

x ( t ) = f ( x,,u ,,q )
(4)
y ( t ) = g ( x,,u ,,q ) + e

The first line is an ordinary differential equation and expresses the

rate of change of the states as a parameterized function of the states
and inputs. Typically, the inputs u(t) correspond to designed
experimental effects (e.g., the stimulus function in fMRI). There is
a fundamental and causal relationship [19] between the outputs
and the history of the inputs in Eq. (4). This relationship conforms
to a Volterra series, which expresses the output as a generalized
convolution of the input, critically without reference to the hidden
states x(t). This series is simply a functional Taylor expansion of the
outputs with respect to the inputs [20]. The reason for it is a func-
tional expansion that the inputs are a function of time.1
t t
y ( t ) = å ò ¼ òk i (s 1 ,,¼,,s i ) u ( t - s 1 ) ,¼, u ( t - s i )
i 0 0
(5)
¶i y (t )
ds 1 ,¼, ds ik i (s 1 ,,¼,,s i ) =
¶u ( t - s 1 ) ,¼, ¶u ( t - s i )

where ki(σ1,…,σi) is the ith-order kernel. In Eq. (5), the integrals

are restricted to the past. This renders the system causal. The key
thing here is that Eq. (5) is simply a convolution and can be
expressed as a GLM as in Eq. (3). This means that we can take a
neurophysiologically realistic model of hemodynamic responses
and use it as an observation model to estimate parameters using
observed data. Here the model is parameterized in terms of kernels
that have a direct analytic relation to the original parameters θ of
the biophysical system. The first-order kernel is simply the conven-
tional HRF. High-order kernels correspond to high-order HRFs
and can be estimated using basis functions as described above. In
fact, by choosing basis functions according to

¶k (s )1
A ( s )i = , (6)
¶qi

one can estimate the biophysical parameters because βi = θi to a

first-order approximation. The critical step we have taken here is to
start with a dynamic causal model of how responses are generated
1
For simplicity, here and in Eq. (7), we deal with only one experimental input.
252 Karl J. Friston

and construct a general linear observation model that allows us to

estimate and infer things about the parameters of that model. This
is in contrast to the conventional use of the GLM with design
matrices that are not informed by a forward model of how data are
caused. This approach to modeling brain responses has a much
more direct connection with underlying physiology and rests upon
an understanding of the underlying system.

3.5.2 Nonlinear System Once a suitable causal model has been established (e.g., Fig. 3), we
Identification can estimate second-order kernels. These kernels represent a non-
linear characterization of the HRF that can model interactions
among stimuli in causing responses. One important manifestation
of the nonlinear effects, captured by the second-order kernels, is a
modulation of stimulus-specific responses by preceding stimuli
that are proximate in time. This means that responses at high-­
stimulus presentation rates saturate and, in some instances, show
an inverted U behavior. This behavior appears to be specific to
BOLD effects (as distinct from evoked changes in CBF) and may
represent a hemodynamic refractoriness. This effect has important
implications for event-related fMRI, where one may want to pres-
ent trials in quick succession.
In summary, we started with models of regionally specific
responses, framed in terms of the GLM, in which responses were
modeled as linear mixtures of designed changes in explanatory vari-
ables. Hierarchical extensions to linear observation models enable
random-effects analyses and, in particular, empirical Bayes. The
mechanistic utility of these models is realized though the use of for-
ward models that embody causal dynamics. Simple variants of these
are the linear convolution models used to construct explanatory
variables in conventional analyses of fMRI data. These are a special
case of generalized convolution models that are mathematically
equivalent to input-state-output systems comprising hidden states.
Estimation and inference with these dynamic models tells us some-
thing about how the response was caused, but only at the level of a
single voxel. Section 4 retains the same perspective on models, but
in the context of distributed responses and functional integration.

4 Models of Functional Integration

4.1 Functional Imaging neuroscience has established functional specialization as a

and Effective principle of brain organization in man. The integration of special-
Connectivity ized areas has proven more difficult to assess. Functional integration
is usually inferred on the basis of correlations among measurements
of neuronal activity. Functional connectivity is defined as statistical
dependencies or correlations among remote neurophysiological events.
However, correlations can arise in a variety of ways. For example, in
multiunit electrode recordings, they can result from stimulus-locked
 Dynamic Causal Modeling of Brain Responses 253

transients evoked by a common input or reflect stimulus-induced

oscillations mediated by synaptic connections [21]. Integration
within a distributed system is usually better understood in terms of
effective connectivity. Effective connectivity refers explicitly to the
influence that one neural system exerts over another, either at a synap-
tic (i.e., synaptic efficacy) or at a population level. It has been pro-
posed that “the (electrophysiological) notion of effective connectivity
should be understood as the experiment- and time-dependent, sim-
plest possible circuit diagram that would replicate the observed tim-
ing relationships between the recorded neurons” [22]. This speaks
about two important points: (a) effective connectivity is dynamic,
that is, activity-dependent and (b) it depends upon a model of the
interactions. The estimation procedures employed in functional
neuroimaging can be divided into linear nondynamic models (e.g.,
[23]) or nonlinear dynamic models.
There is a necessary link between functional integration and
multivariate analyses because the latter are necessary to model inter-
actions among brain regions. Multivariate approaches can be
divided into those that are inferential in nature and those that are
data-led or exploratory. We will first consider multivariate approaches
that look at functional connectivity or covariance patterns (and are
generally exploratory) and then turn to models of effective connec-
tivity (that allow for inference about their parameters).

4.1.1 Eigenimage In Friston et al. [24], we introduced voxel-based principal component

Analysis and Related analysis (PCA) of neuroimaging time series to characterize distributed
Approaches brain systems implicated in sensorimotor, perceptual, or cognitive
processes. These distributed systems are identified with principal com-
ponents or eigenimages that correspond to spatial modes of coherent
brain activity. This approach represents one of the simplest multivari-
ate characterizations of functional neuroimaging time series and falls
into the class of exploratory analyses. Principal component or eigen-
image analysis generally uses singular value decomposition to identify
a set of orthogonal spatial modes that capture the greatest amount of
variance expressed over time. As such, the ensuing modes embody the
most prominent aspects of the variance–covariance structure of a
given time series. Noting that covariance among brain regions is
equivalent to functional connectivity renders eigenimage analysis par-
ticularly interesting because it was among the first ways of addressing
functional integration (i.e., connectivity) with neuroimaging data.
Subsequently, eigenimage analysis has been elaborated in a number of
ways. Notable among these is canonical variate analysis (CVA) and
multidimensional scaling [25, 26]. CVA was introduced in the con-
text of multiple analysis of covariance and uses the generalized eigen-
vector solution to maximize the variance that can be explained by
some explanatory variables relative to error. CVA can be thought of as
an extension of eigenimage analysis that refers explicitly to some
explanatory variables and allows for statistical inference.
254 Karl J. Friston

In fMRI, eigenimage analysis (e.g., [27]) is generally used as

an exploratory device to characterize coherent brain activity. These
variance components may, or may not, be related to experimental
design. For example, endogenous coherent dynamics have been
observed in the motor system at very low frequencies [28]. Despite
its exploratory power, eigenimage analysis is limited for two rea-
sons. First, it offers only a linear decomposition of any set of neu-
rophysiological measurements and second, the particular set of
eigenimages or spatial modes obtained is determined by constraints
that are biologically implausible. These aspects of PCA confer
inherent limitations on the interpretability and usefulness of
­eigenimage analysis of biological time series and have motivated
the exploration of nonlinear PCA and neural network approaches.
Two other important approaches deserve to be mentioned here. The
first is independent component analysis (ICA). ICA uses entropy maximi-
zation to find, using iterative schemes, spatial modes or their dynamics
that are approximately independent. This is a stronger requirement than
orthogonality in PCA and involves removing high-order correlations
among the modes (or dynamics). It was initially introduced as spatial ICA
[29], in which the independence constraint was applied to the modes
(with no constraints on their temporal expression). More recent
approaches use, by analogy with magneto- and electrophysiological time
series analysis, temporal ICA where the dynamics are enforced to be inde-
pendent. This requires an initial dimension reduction (usually using con-
ventional eigenimage analysis). Finally, there has been an interest in cluster
analysis [30]. Conceptually, this can be related to eigenimage analysis
through multidimensional scaling and principal co-ordinate analysis.
All these approaches are interesting, but they are not used
very much. This is largely because they tell you nothing about
how the brain works or allow one to ask specific questions. Simply
demonstrating statistical dependencies among regional brain
responses or endogenous activity (i.e., demonstrating functional
connectivity) does not address how these responses were caused.
To address this, one needs explicit models of integration or more
precisely, effective connectivity.

4.2 Dynamic Causal This section is about modeling interactions among neuronal popu-
Modeling lations, at a cortical level, using neuroimaging time series and
with Bilinear Models dynamic causal models that are informed by the biophysics of the
system studied. The aim of DCM [31] is to estimate, and make
inferences about, the coupling among brain areas and how that
coupling is influenced by experimental changes (e.g., time or cog-
nitive set). The basic idea is to construct a reasonably realistic neu-
ronal model of interacting cortical regions or nodes. This model is
then supplemented with a forward model of how neuronal or syn-
aptic activity translates into a measured response (see previous sec-
tion). This enables the parameters of the neuronal model (i.e.,
effective connectivity) to be estimated from observed data.
 Dynamic Causal Modeling of Brain Responses 255

Intuitively, this approach regards an experiment as a designed

perturbation of neuronal dynamics that are promulgated and dis-
tributed throughout a system of coupled anatomical nodes to
change region-specific neuronal activity. These changes engender,
through a measurement-specific forward model, responses that are
used to identify the architecture and time constants of the system at
a neuronal level. This represents a departure from conventional
approaches (e.g., structural equation modeling and auto-­regression
models; [32, 33]), in which one assumes that the observed responses
are driven by endogenous or intrinsic noise (i.e., innovations). In
contrast, dynamic causal models assume that the responses are
driven by designed changes in inputs. An important conceptual
aspect of dynamic causal models pertains to how the experimental
inputs enter the model and cause neuronal responses. Experimental
variables can elicit responses in one of two ways. First, they can elicit
responses through direct influences on specific anatomical nodes.
This would be appropriate, for example, in modeling sensory-
evoked responses in early visual cortices. The second class of input
exerts its effect vicariously, through a modulation of the coupling
among nodes. This sort of experimental variable would normally be
more enduring, for example, attention to a particular attribute or
the maintenance of some perceptual set. These distinctions are seen
most clearly in relation to particular forms of causal models used for
estimation, for example, the bilinear approximation

x = f ( x,u ) = Ax + uBx + Cu
y = g ( x) + e (7)
¶f ( 0,0 ) ¶ f ( 0,0 )
2
¶f ( 0,0 )
A= B= C=
¶x ¶x¶u ¶u
where x = ¶x / ¶t . This is an approximation to any model of how
changes in neuronal activity in one region xi are caused by activity
in the other regions. Here the output function g(x) embodies a
hemodynamic convolution, linking neuronal activity to BOLD, for
each region (e.g., that in Fig. 3). The matrix A represents the cou-
pling among the regions in the absence of input u(t). This can be
thought of as the endogenous coupling in the absence of experi-
mental perturbations. The matrix B is effectively the change in
coupling induced by the input. It encodes the input-sensitive
changes in A or, equivalently, the modulation of coupling by
experimental manipulations. Because B is a second-order deriva-
tive, it is referred to as bilinear. Finally, the matrix C embodies the
exogenous influences of inputs on neuronal activity. The parame-
ters θ = A,B, and C are the connectivity or coupling matrices that
we wish to identify and define the functional architecture and
interactions among brain regions at a neuronal level. They play the
same role as rate constant in kinetic models and therefore have
units of Hertz or per second.
256 Karl J. Friston

Because Eq. (7) has exactly the same form as Eq. (4), we can
express it as a GLM and estimate the parameters using EM in the
usual way (see Ref. [31]). Generally, estimation in the context of
highly parameterized models like DCMs requires constraints in the
form of priors. These priors enable conditional inference about the
connectivity estimates. The sorts of questions that can be addressed
with DCMs are now illustrated by looking at how attentional mod-
ulation is mediated in sensory processing hierarchies in the brain.

4.2.1 DCM and It has been established that the superior parietal cortex (SPC) exerts
Attentional Modulation a modulatory role on V5 responses using Volterra-based regression
models [34] and that the inferior frontal gyrus (IFG) exerts a simi-
lar influence on SPC using structural equation modeling [32]. The
example here shows that DCM leads to the same conclusions but
starting from a completely different construct. The experimental
paradigm and data acquisition are described in the legend to Fig. 4.
This figure also shows the location of the regions that entered the
DCM. These regions were based on maxima from conventional
SPMs testing for the effects of photic stimulation, motion, and

Fig. 4 Results of a dynamic causal modeling (DCM) analysis of attention to visual motion with fMRI. Right
panel: Functional architecture based upon the conditional estimates shown alongside their connections, with
the percent confidence that they exceeded threshold in brackets. The most interesting aspects of this archi-
tecture involve the role of motion and attention in exerting bilinear effects. Critically, the influence of motion is
to enable connections from V1 to the motion-sensitive area V5. The influence of attention is to enable back-
ward connections from the inferior frontal gyrus (IFG) to the superior parietal cortex (SPC). Furthermore, atten-
tion increases the influence of SPC on V5. Dotted arrows connecting regions represent significant bilinear
effects in the absence of a significant intrinsic coupling. Left panel: Fitted responses based upon the condi-
tional estimates and the adjusted data are shown for each region in the DCM. The insert (upper left) shows the
location of the regions
 Dynamic Causal Modeling of Brain Responses 257

attention. Regional time courses were taken as the first eigenvariate

of 8-mm-spherical volumes of interest, centered on the maxima
shown in the figure. The exogenous inputs, in this example, com-
prise one sensory perturbation and two contextual inputs. The sen-
sory input was simply the presence of photic stimulation and the
first contextual one was presence of motion in the visual field. The
second contextual input, encoding attentional set, was one during
attention to speed changes and zero otherwise. The outputs corre-
sponded to the four regional eigenvariates in Fig. 4 (left panel). The
intrinsic connections were constrained to conform to a hierarchical
pattern in which each area was reciprocally connected to its supra-
ordinate area. Photic stimulation entered at, and only at, V1. The
effect of motion in the visual field was modeled as a bilinear modu-
lation of the V1 to V5 connectivity and attention was allowed to
modulate the backward connections from IFG and SPC.
Subjects were studied with fMRI under identical stimulus con-
ditions (visual motion subtended by radially moving dots) while
manipulating the attentional component of the task (detection of
velocity changes). The data were acquired from a normal subject at
2 T. Each subject had four consecutive 100-scan sessions compris-
ing a series of ten-scan blocks under five different conditions
D F A F N F A F N S. The first condition (D) was a dummy condi-
tion to allow for magnetic saturation effects. F (fixation) corresponds
to a low-level baseline where subjects viewed a fixation point at the
center of a screen. In condition A (attention), subjects viewed 250
dots moving radially from the center at 4.7°/s and were asked to
detect changes in radial velocity. In condition N (no attention), the
subjects were asked simply to view the moving dots. In condition S
(stationary), subjects viewed stationary dots. The order of A and N
was swapped for the last two sessions. In all conditions, subjects fix-
ated the center of the screen. During scanning, there were no speed
changes. No overt response was required in any condition.
The results of the DCM are shown in Fig. 4 (right panel). Of
primary interest here is the modulatory effect of attention that is
expressed in terms of the bilinear coupling parameters for this
input. As expected, we can be highly confident that attention mod-
ulates the backward connections from IFG to SPC and from SPC
to V5. Indeed, the influences of IFG on SPC are negligible in the
absence of attention (dotted connection). It is important to note
that the only way that attentional manipulation can affect brain
responses was through this bilinear effect. Attention-related
responses are seen throughout the system (attention epochs are
marked with arrows in the plot of IFG responses in the left panel).
This attentional modulation is accounted for, sufficiently, by chang-
ing just two connections. This change is, presumably, instantiated
by instructional set at the beginning of each epoch.
The second thing, this analysis illustrates, is how functional
segregation is modeled in DCM. Here one can regard V1 as
258 Karl J. Friston

“segregating” motion from other visual information and distribut-

ing it to the motion-sensitive area V5. This segregation is modeled
as a bilinear “enabling” of V1–V5 connections when, and only
when, motion is present. Note that in the absence of motion, the
intrinsic V1–V5 connection was trivially small (in fact the estimate
was −0.04 Hz). The key advantage of entering motion through a
bilinear effect, as opposed to a direct effect on V5, is that we can
finesse the inference that V5 shows motion-selective responses
with the assertion that these responses are mediated by afferents
from V1. The two bilinear effects above represent two important
aspects of functional integration that DCM is able to characterize.

4.2.2 Structural Equation The central idea, behind DCM, is to treat the brain as a determin-
Modeling as a Special istic nonlinear dynamic system that is subject to inputs and pro-
Case of DCM duces outputs. Effective connectivity is parameterized in terms of
coupling among unobserved brain states (e.g., neuronal activity in
different regions). The objective is to estimate these parameters by
perturbing the system and measuring the response. This is in con-
tradistinction to established methods for estimating effective con-
nectivity from neurophysiological time series, which include SEM
and models based on multivariate auto-regressive processes. In
these models, there is no designed perturbation and the inputs are
treated as unknown and stochastic. Furthermore, the inputs are
often assumed to express themselves instantaneously such that, at
the point of observation, the change in states is zero. From Eq.
(7), in the absence of bilinear effects, we have

x = 0 = Ax + Cu
(8)
x = - A-1Cu

This is the regression equation used in SEM where A = D–I and D

contains the off-diagonal connections among regions. The key
point here is that A is estimated by assuming that u(t) is some ran-
dom innovation with known covariance. This is not really tenable
for designed experiments when u(t) represent carefully structured
experimental inputs. Although SEM and related auto-regressive
techniques are useful for establishing dependence among regional
responses, they are not surrogates for informed causal models
based on the underlying dynamics of these responses.
In this section, we have covered multivariate techniques rang-
ing from eigenimage analysis that does not have an explicit forward
or causal model to DCM that does. The bilinear approximation to
any DCM has been illustrated though its use with fMRI to study
attentional modulation. The parameters of the bilinear approxima-
tion include first-order effective connectivity A and its experimen-
tally induced changes B. Although the bilinear approximation is
useful, it is possible to model coupling among neuronal subpopu-
lations explicitly. We conclude with a DCM that embraces a
 Dynamic Causal Modeling of Brain Responses 259

number of neurobiological facts and takes us much closer to a

mechanistic understanding of how brain responses are generated.
This example uses responses measured with EEG.

4.3 Dynamic Causal ERPs have been used for decades as electrophysiological correlates
Modeling with Neural of perceptual and cognitive operations. However, the exact neuro-
Mass Models biological mechanisms underlying their generation are largely
unknown. In this section, we use neuronally plausible models to
understand event-related responses. Our example shows that
changes in connectivity are sufficient to explain certain ERP com-
ponents. Specifically, we will look at the MMN, a component asso-
ciated with rare or unexpected events. If the unexpected nature of
rare stimuli depends on learning which stimuli are frequent, then
the MMN must be due to plastic changes in connectivity that
mediate perceptual learning. We conclude by showing that advances
in the modeling of evoked responses now afford measures of con-
nectivity among cortical sources that can be used to quantify the
effects of perceptual learning.

4.3.1 Neural Mass The minimal model we have developed [35] uses the connectivity
Models rules described by Felleman and Van Essen [36] to assemble a net-
work of coupled sources. These rules are based on a partitioning of
the cortical sheet into supra-, infra-granular, and granular layer
(layer 4). Bottom-up or forward connections originate in agranular
layers and terminate in layer 4. Top-down or backward connections
target agranular layers. Lateral connections originate in agranular
layers and target all layers. These long-range or extrinsic cortico-
cortical connections are excitatory and arise from pyramidal cells.
Each region or source is modeled using a neural mass model
described by David and Friston [35], based on the model of Jansen
and Rit [37]. This model emulates the activity of a cortical area
using three neuronal subpopulations, assigned to granular and
agranular layers. A population of excitatory pyramidal (output)
cells receives inputs from inhibitory and excitatory populations of
inter-neurons, via intrinsic connections (intrinsic connections are
confined to the cortical sheet). Within this model, excitatory inter-­
neurons can be regarded as spiny stellate cells found predominantly
in layer 4 and in receipt of forward connections. Excitatory pyra-
midal cells and inhibitory inter-neurons are considered to occupy
agranular layers and receive backward and lateral inputs (Fig. 5).
To model event-related responses, the network receives inputs
via input connections. These connections are exactly the same as
forward connections and deliver inputs to the spiny stellate cells in
layer 4. In the present context, inputs u(t) model subcortical audi-
tory inputs. The vector C controls the influence of the input on
each source. The lower, upper, and leading diagonal matrices
AF,AB, AL encode forward, backward, and lateral connections,
respectively. The DCM here is specified in terms of the state equa-
tions shown in Fig. 5 and a linear output equation
260 Karl J. Friston

Fig. 5 Schematic of the dynamic causal modeling (DCM) used to model electrical responses. This schematic shows
the state equations describing the dynamics of sources or regions. Each source is modeled with three subpopula-
tions (pyramidal, spiny stellate, and inhibitory inter-neurons) as described in the main text. These have been
assigned to granular and agranular cortical layers that receive forward and backward connections, respectively

x = f ( x,u )
(9)
y = Lx0 + e

where x0 represents the trans-membrane potential of pyramidal

cells and L is a lead field matrix coupling electrical sources to the
EEG channels. This should be compared with the DCM above for
hemodynamics; here, the equations governing the evolution of
neuronal states are much more complicated and realistic, as
opposed to the bilinear approximation in Eq. (7). Conversely, the
output equation is a simple linearity, as opposed to the nonlinear
observer used for fMRI. As an example, the state equation for the
inhibitory subpopulation is 2

2
Propagation delays on the extrinsic connections have been omitted for clarity
here and in Fig. 5.
 Dynamic Causal Modeling of Brain Responses 261

x7 = x8
H 2x x (10)
x8 = e éë( AB + AL + g 3 I ) S ( x0 ) ùû - 8 - 72
te te te

Within each subpopulation, the evolution of neuronal states rests

on two operators. The first transforms the average density of pre-­
synaptic inputs into the average postsynaptic membrane potential.
This is modeled by a linear transformation with excitatory and
inhibitory kernels parameterized by He,i and τe,j. He,j controls the
maximum post-synaptic potential and τe,j represents a lumped rate
constant. The second operator S transforms the average potential
of each subpopulation into an average firing rate. This is assumed
to be instantaneous and is a sigmoid function. Interactions, among
the subpopulations, depend on constants, γ1,2,3,4, which control the
strength of intrinsic connections and reflect the total number of
synapses expressed by each subpopulation. In Eq. (10), the top
line expresses the rate of change of voltage as a function of current.
The second line specifies how current changes as a function of volt-
age, current, and pre-synaptic input from extrinsic and intrinsic
sources. Having specified the DCM in terms of these equations,
one can estimate the coupling parameters from empirical data
using EM as described above. See Ref. [38] for more details.

4.3.2 Perceptual The example shown in Fig. 6 is an attempt to model the MMN in
Learning and the MMN terms of changes in backward and lateral connections among corti-
cal sources. In this example, two (averaged) channels of EEG data
were modeled with three cortical sources. Using this generative or
forward model, we estimated differences in the strength of these
connections for rare and frequent stimuli. As expected, we could
account for detailed differences in the ERPs (the MMN) by
changes in connectivity (see figure legend for details). Interestingly,
these differences were expressed selectively in the lateral connec-
tions. If this model is a sufficient approximation to the real sources,
these changes are a noninvasive measure of plasticity, mediating
perceptual learning, in the human brain.

5 Conclusion

In this chapter, we have reviewed some key models that underpin

image analysis and have touched briefly on ways of assessing
­specialization and integration in the brain. These models can be
regarded as a succession of modeling endeavors, that drawing more
and more on our understanding of how brain-imaging signals are
generated, both in terms of biophysics and the underlying neuro-
nal interactions. We have seen how hierarchical linear observation
models encode the treatment effects elicited by experimental
262 Karl J. Friston

Fig. 6 Summary of a dynamic causal modeling (DCM) analysis of event-related potentials (ERPs) elicited during
an auditory oddball paradigm, employing rare and frequent pure tones. Upper panel: Schematic showing the
architecture of the neuronal model used to explain the empirical data. Sources were coupled with extrinsic cor-
tico-cortical connections following the rules of Felleman and van Essen. The free parameters of this model
included intrinsic and extrinsic connection strengths that were adjusted to best explain the data. In this example,
the lead field was also estimated, with no spatial constraints. The parameters were estimated for ERPs recorded
during the presentation of rare and frequent tones and are reported beside their corresponding connection (fre-
quent/rare). The most notable finding was that the mismatch response could be explained by a selective increase
in lateral connection strength from 0.1 to 3.68 Hz (highlighted in bold). Lower panel: The channel positions (left)
and ERPs (right) averaged over two subsets of channels (circled on the left). Note the correspondence between
the measured ERPs and those generated by the model. Auditory stimuli, 1,000 or 2,000 Hz tones with 5 ms rise
and fall times and 80 ms duration, were presented binaurally. The tones were presented for 15 min, every 2 s in
a pseudo-random sequence with 2000-Hz tones occurring 20 % of the time and 1,000-Hz tones occurring 80 %
of the time. The subject was instructed to keep a mental record of the number of 2000-Hz tones (nonfrequent
target tones). Data were acquired using 128 EEG electrodes with 1,000 Hz sample frequency. Before averaging,
data were referenced to mean earlobe activity and band-pass filtered between 1 and 30 Hz. Trials showing ocular
artifacts and bad channels were removed from further analysis
 Dynamic Causal Modeling of Brain Responses 263

design. GLMs based on convolution models imply an underlying

dynamic input-state-output system. The form of these systems can
be used to constrain convolution models and explore some of their
simpler nonlinear properties. By creating observation models based
on explicit forward models of neuronal interactions, one can model
and assess interactions among distributed cortical areas and make
inferences about coupling at the neuronal level. The next years will
probably see an increasing realism in the dynamic causal models
introduced above (see Ref. [39]). These endeavors are likely to
encompass fMRI signals enabling the conjoint modeling, or fusion,
of different modalities and the marriage of computational neuro-
science with the modeling of brain responses.

References

1. Staum M (1995) Physiognomy and phrenol- magnetic resonance imaging in human V1.
ogy at the Paris Athénée. J Hist Ideas J Neurosci 16:4207–4221
6:443–462 12. Friston KJ, Frith CD, Turner R, Frackowiak
2. Phillips CG, Zeki S, Barlow HB (1984) RSJ (1995) Characterising evoked hemody-
Localisation of function in the cerebral cortex: namics with fMRI. NeuroImage 2:157–165
past present and future. Brain 107:327–361 13. Josephs O, Turner R, Friston KJ (1997) Event-­
3. Goltz F (1881) Transactions of the 7th interna- related fMRI Hum. Brain Mapp 5:243–248
tional medical congress (W. MacCormac, Ed.), 14. Lange N, Zeger SL (1997) Non-linear Fourier
Vol. I, JW Kolkmann: London, 1881:218–228 time series analysis for human brain mapping by
4. Absher JR, Benson DF (1993) Disconnection functional magnetic resonance imaging (with
syndromes: an overview of Geschwind’s contri- discussion). J Roy Stat Soc Ser C 46:1–29
butions. Neurology 43:862–867 15. Buxton RB, Frank LR (1997) A model for the
5. Zeki S (1990) The motion pathways of the coupling between cerebral blood flow and oxy-
visual cortex. Vision: coding and efficiency gen metabolism during neural stimulation.
(C. Blakemore, Ed.). Cambridge University J Cereb Blood Flow Metab 17:64–72
Press, pp 321–345 16. Mandeville JB, Marota JJ, Ayata C, Zararchuk
6. Friston KJ, Frith CD, Liddle PF, Frackowiak G, Moskowitz MA, Rosen B, Weisskoff RM
RSJ (1991) Comparing functional (PET) (1999) Evidence of a cerebrovascular postarte-
images: the assessment of significant change. riole Windkessel with delayed compliance.
J Cereb Blood Flow Metab 11:690–699 J Cereb Blood Flow Metab 19:679–689
7. Berry DA, Hochberg Y (1999) Bayesian per- 17. Hoge RD, Atkinson J, Gill B, Crelier GR,
spectives on multiple comparisons. J Stat Plan Marrett S, Pike GB (1999) Linear coupling
Infer 82:215–227 between cerebral blood flow and oxygen
8. Holmes A, Ford I (1993) A Bayesian approach ­consumption in activated human cortex. Proc
to significance testing for statistic images from Natl Acad Sci 96:9403–9408
PET. In: Uemura K, Lassen NA, Jones T, 18. Friston KJ, Mechelli A, Turner R, Price CJ
Kanno I (eds) Quantification of brain function, (2000) Nonlinear responses in fMRI: the
tracer kinetics and image analysis in brain Balloon model, Volterra kernels, and other
PET. Excerpta Medica, Int. Cong. Series No. hemodynamics. NeuroImage 12:466–477
1993. 1030:521–534 19. Fliess M, Lamnabhi M, Lamnabhi-Lagarrigue
9. Dempster AP, Laird NM, Rubin (1977) F (1983) An algebraic approach to nonlinear
Maximum likelihood from incomplete data via functional expansions. IEEE Trans Circuits
the EM algorithm. J Roy Stat Soc B 39:1–38 Syst 30:554–570
10. Friston KJ, Jezzard P, Turner R (1994) Analysis 20. Bendat JS (1990) Nonlinear system analysis
of functional MRI time series. Human Brain and identification from random data. John
Map 1:153–171 Wiley, New York
11. Boynton GM, Engel SA, Glover GH, Heeger 21. Gerstein GL, Perkel DH (1969) Simultaneously
DJ (1996) Linear systems analysis of functional recorded trains of action potentials: analysis
264 Karl J. Friston

and functional interpretation. Science 30. Baumgartner R, Scarth G, Teichtmeister C,

164:828–830 Somorjai R, Moser E (1997) Fuzzy clustering
22. Aertsen A, Preissl H (1991) Dynamics of activ- of gradient-echo functional MRI in the human
ity and connectivity in physiological neuronal visual cortex. Part 1: reproducibility. J Magn
Networks. In: Schuster HG (ed) Non linear Reson Imaging 7:1094–1101
dynamics and neuronal networks. VCH, 31. Friston KJ, Harrison L, Penny W (2003)
New York, pp 281–302 Dynamic causal modelling. NeuroImage
23. McIntosh AR, Gonzalez-Lima F (1994) 19:1273–1302
Structural equation modelling and its applica- 32. Büchel C, Friston KJ (1997) Modulation of con-
tion to network analysis in functional brain nectivity in visual pathways by attention: cortical
imaging. Hum Brain Mapp 2:2–22 interactions evaluated with structural equation
24. Friston KJ, Frith CD, Liddle PF, Frackowiak modelling and fMRI. Cereb Cortex 7:768–778
RSJ (1993) Functional connectivity: the princi- 33. Harrison LM, Penny W, Friston KJ (2003)
pal component analysis of large data sets. Multivariate autoregressive modelling of fMRI
J Cereb Blood Flow Metab 13:5–14 time series. NeuroImage 19:1477–1491
25. Friston KJ, Poline J-B, Holmes AP, Frith CD, 34. Friston KJ, Büchel C (2000) Attentional mod-
Frackowiak RSJ (1996) A multivariate analysis ulation of effective connectivity from V2 to
of PET activation studies. Hum Brain Mapp V5/MT in humans. Proc Natl Acad Sci U S A
4:140–151 97:7591–7596
26. Friston KJ, Frith CD, Fletcher P, Liddle PF, 35. David O, Friston KJ (2003) A neural mass
Frackowiak RSJ (1996) Functional topography: model for MEG/EEG: coupling and neuronal
multidimensional scaling and functional connec- dynamics. NeuroImage 20:1743–1755
tivity in the brain. Cereb Cortex 6:156–164 36. Felleman DJ, Van Essen DC (1992) Distributed
27. Sychra JJ, Bandettini PA, Bhattacharya N, Lin hierarchical processing in the primate cerebral
Q (1994) Synthetic images by subspace trans- cortex. Cereb Cortex 1:1–47
forms. I. Principal component images and 37. Jansen BH, Rit VG (1995) Electroencephalogram
related filters. Med Phys 21:193–201 and visual evoked potential generation in a math-
28. Biswal B, Yetkin FZ, Haughton VM, Hyde JS ematical model of coupled cortical columns. Biol
(1995) Functional connectivity in the motor Cybern 73:357–366
cortex of resting human brain using echo-­ 38. David O, Kiebel SJ, Harrison LM, Mattout J,
planar MRI. Magn Reson Med 34:537–541 Kilner JM, Friston KJ (2006) Dynamic causal
29. McKeown M, Jung T-P, Makeig S, Brown G, modelling of evoked responses in EEG and
Kinderman S, Lee T-W, Sejnowski T (1998) MEG. NeuroImage 30:1255–1272
Spatially independent activity patterns in func- 39. Horwitz B, Friston KJ, Taylor JG (2001)
tional MRI data during the Stroop colour nam- Neural modelling and functional brain imag-
ing task. Proc Natl Acad Sci 95:803–810 ing: an overview. Neural Netw 13:829–846
 Chapter 9

Brain Atlases: Their Development and Role in Functional

Inference
John Darrell Van Horn and Arthur W. Toga

Abstract
Imparting functional meaning to neuroanatomical location has been among the greatest challenges to
neuroscientists. The characterization of the brain architecture responsible in human cognition received a
boost in momentum with the emergence of in vivo functional and structural neuroimaging technology
over the past 30 years. Yet, individual variability in cortical gyrification as well as the patterns of blood flow-
related activity measured using fMRI and positron emission tomography complicated direct comparisons
across subjects without spatially accounting for overall brain size and shape. This realization resulted in
considerable effort now involving the collective efforts of neuroscientists, computer scientists, and math-
ematicians to develop common brain atlas spaces against which the regions of activity may be accurately
referenced. We examine recent developments in brain imaging and computational anatomy that have
greatly expanded our ability to analyze brain structure and function. The enormous diversity of brain maps
and imaging methods has spurred the development of population-based digital brain atlases. Atlases store
information on how the brain varies across age and gender, across time, in health and disease, and in large
human populations. We describe how brain atlases, and the computational tools that align new datasets
with them, facilitate comparison of brain data across experiments, laboratories, and from different imaging
devices. The major philosophies are presented that underlie the construction of probabilistic atlases, which
store information on anatomic and functional variability in a population. Algorithms which create compos-
ite brain maps and atlases based on multiple subjects are examined. We show that group patterns of cortical
organization, asymmetry, and disease-specific trends can be resolved that may not be apparent in individual
brain maps. Finally, we describe the development of four-dimensional maps that store information on the
dynamics of brain change in development and disease.

Key words Brain atlases, Neuroanatomy, Diffeomorphism, Warping, Functional activity, Inference

1 Introduction

Over a century ago, in a horrific accident, damage to the frontal

lobe of Phineas Gage produced profound changes in his personality
and cognitive function, the beginning of what may be considered as
the modern era of the localization of brain function [1]. Nearly a
decade later, the examination by Paul Broca of aphasic patients hav-
ing damage to the left inferior frontal areas (pars triangularis, pars

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_9, © Springer Science+Business Media New York 2016

265
266 John Darrell Van Horn and Arthur W. Toga

opercularis) solidified the notion that function could be linked to

specific areas of cortical tissue [2]. At the turn of the twentieth cen-
tury, Dr. Alois Alzheimer, a German psychiatrist, identified the first
case of what became known as Alzheimer’s Disease (AD) [3], now
known to severely affect the thickness of the cortical mantle as well
as the hippocampal area and its surrounding tissues. Since those
early reports, a principle goal in neuroscience has been to classify
the specific brain regions possessing unique functional components
of complex thought and how these functions might be altered in
response to injury or as a result of disease.
Over the past three decades, with the emergence of neuroim-
aging as the primary tool for the examination of the brain in vivo
during cognitive task performance, the mapping of mental func-
tion has given rise to an explosion of functional experimentation
and an ever-widening interest in understanding brain processes
from fields beyond traditional neuroscience (e.g., economics,
criminology, social science). This intense effort and expanse of data
has emphasized the realization that there is considerable individual
variation in brain size and shape that must be accounted for in the
processing of brain imaging data and the assignment of functional
significance. Evaluation and comparison of brain imaging data
with respect to and against well-defined anatomical references is
now a critical element in the localization of essential cognitive
functions in nearly all functional imaging investigations.
Brain atlases can now comprise imaging data describing mul-
tiple aspects of brain structure or function at different scales from
different subjects, yielding a truly integrative and comprehensive
description of this organ in health and disease [4, 5]. However,
such complexity and variability of brain structure, especially in the
gyral patterns of the human cortex, present challenges in creating
standardized brain atlases that reflect the anatomy of a population
[6]. This chapter discusses the concepts behind population-based,
age-, and disease-specific brain atlas construction that can be used
to reflect the specific anatomy and physiology of a particular clini-
cal subpopulation. Based on well-characterized subject groups,
age-specific atlases can potentially contain thousands of structure
models, composite maps, average templates, and visualizations of
structural variability, asymmetry, and group-specific differences.
They correlate the structural, metabolic, molecular, and histologic
hallmarks of the disease [7, 8]. Rather than simply fusing informa-
tion from multiple subjects and sources, new mathematical strate-
gies are being introduced to resolve group-specific features not
apparent in individual scans [9]. High-dimensional elastic map-
pings, based on covariant partial differential equations, are devel-
oped to encode patterns of cortical variation [10–12]. In the
resulting brain atlas, age-stratified features and regional asymmetries
emerge that are not apparent in individual anatomies. The conse-
quential probabilistic atlas spaces can be used to identify patterns
 Brain Atlases: Their Development and Role in Functional Inference 267

of altered structure and function, and can guide algorithms for

knowledge-based image analysis, automated image labeling, tissue
classification, data mining, and functional image analysis. These
integrative techniques have provided significant motivation for
human brain mapping initiatives and have important applications
in health and disease.

2 Methods for Brain Atlas Construction

Creating atlases relies on the accumulation and compilation of

many image sets along with appropriate registration and warping
strategies, indexing schemes, and nomenclature systems. The pro-
cessing of multimodal brain images in the context of an atlas
enables a more meaningful interpretation (Fig. 1). The complex-
ity and variability of human brain (as well as other species) across
subjects is so great that reliance on atlases is essential to effectively
manipulate, analyze, and interpret brain data. Central to these
tasks is the construction of averages, templates, and models to
describe how the brain and its component parts are organized.
Design of appropriate reference systems for human brain data
presents considerable challenges, since these systems must capture
how brain structure and function vary in large populations, across
age and gender, in different disease states, across imaging modali-
ties, and even across species.

Fig. 1 A variety of neuroimaging methods permit the acquisition of brain data over time and space having a
range of resolution granularity. Moreover, variation across individuals and how this changes over the lifespan
must be accounted for in statistical examination of the data. Mapping these data to known spatial coordinate
systems enables highly accurate inference concerning the brain’s structural change over time, between popu-
lations, or in terms of localizing functional change. Atlases denoting this variation after spatial warping, the
characterization of shape, three-dimensional (3D) distortion, etc. will be essential in describing structural and
functional alteration associated with normal aging as well as in disease
268 John Darrell Van Horn and Arthur W. Toga

2.1 Basic Image Image registration is an elemental step in many of the analytic
Registration strategies involving brain imaging today [13]. Initially developed
as an image processing technique to spatially align one image to
match another, image registration now has a vast range of applica-
tions, such as automated image labeling and for pathology detec-
tion in individuals or groups [14]. Registration algorithms can
encode patterns of anatomic variability in large human popula-
tions, and can use this information to create disease-specific,
population-based brain atlases [15]. They may also blend data
from multiple imaging devices to correlate different measures of
brain structure and function. Finally, modern registration algo-
rithms and workflows can serve as a basic measure for patterns of
structural change during brain development, tumor growth, or
degenerative disease processes [16].

2.2 Geodesic The objective in geodesic approaches has been to encourage varia-
Averaging of Brain tional methods for anatomical averaging that operate within the
Shape space of the underlying image registration problem [17]. This
approach is effective when using a large deformation viscous frame-
work, where linear averaging might not be appropriate. The theory
behind it is similar to registration-based techniques but with single
image force replaced by the average forces from multiple sources.
These group forces drive an average transport ordinary differential
equation allowing one to estimate the geodesic that moves an
image toward the mean shape configuration. This model provides
large deformation atlases that are optimal with respect to the shape
manifold as defined by the data and the image registration assump-
tions. These procedures generate refined average representations
of highly variable anatomy from distinct populations. For example,
the population statistics have been used to show a significant dou-
bling of the relative prefrontal lobe size in humans, as compared to
nonhuman primates [18].

2.3 Density-Based Initial approaches to population-based atlasing concentrated on

Atlases generating mean representations of anatomy through the “inten-
sity pooling” of multiple MRI scans. This involves large number of
MRI scans which are each linearly transformed into stereotaxic
space, intensity-normalized, and averaged on a voxel-by-voxel
basis, producing an average intensity MRI dataset. The average
brains that result have large areas, especially at the cortex, where
individual structures are blurred because of spatial variability in the
population. While this blurring limits their usefulness as a quantita-
tive tool, the templates can be used as targets for the automated
registration and mapping of MR and co-registered functional data
into stereotaxic space [19].

2.4 Label-Based In label-based approaches, large ensembles of brain data are labeled
Atlases or “segmented” by a human operator or algorithmically into
 Brain Atlases: Their Development and Role in Functional Inference 269

subvolumes after mapping individual datasets into stereotaxic

space. A probability map is then constructed for each segmented
structure, by determining the proportion of subjects assigned a
given anatomic label at each voxel position in stereotactic space.
The prior information which these probability maps provide on the
location of various tissue classes in stereotactic space has been use-
ful in designing automated tissue classifiers and approaches to cor-
rect radio-frequency and intensity inhomogeneities in MR scans.
Statistical data on anatomic labels and tissue types normally found
at given positions in stereotactic space provide a vital independent
source of information to guide and inform mathematical algo-
rithms, which analyze neuroanatomical data in stereotactic space.

2.5 Encoding Brain Measuring and accounting for the considerable variability in brain
Variation shape across human populations necessitates realistically complex
mathematical strategies to encode comprehensive information on
structural variability [20]. Particularly relevant is a three-
dimensional (3D) statistical information on group-specific patterns
of variation and how these patterns are altered in disease. This
information can be represented such that it can be exploited by
expert diagnostic systems, whose goal is to detect subtle or diffuse
structural alterations in disease [21]. Strategies for detecting struc-
tural anomalies can leverage information in anatomical databases
by invoking encoded knowledge on the variations in geometry and
location of neuroanatomic regions and critical functional inter-
faces, especially at the cortex.

2.6 Shape Of particular relevance in dealing with brain substructures are

and Pattern Theory methods used to define a mean shape in such a way that departures
from this mean shape can be treated as a linear process [22].
Linearization of the pathology detection problem, by constructing
various shape manifolds and their associated tangent spaces, allows
the use of conventional statistical procedures and linear decompo-
sition of departures from the mean to characterize shape change.
These approaches have been applied to detect structural anomalies
in schizophrenia by identification of statistical differences in mean
shape of brain structures [23–25] (Fig. 2).

2.7 Deformation When applied to two different 3D brain scans, a nonlinear registra-
Atlases tion or warping algorithm calculates a deformation map that
matches up brain structures in one scan with their counterparts in
the other. The deformation map indicates 3D patterns of anatomic
differences between the two subjects or populations [26]. In prob-
abilistic atlases based on deformation maps, statistical properties of
these deformation maps are encoded locally to determine the mag-
nitude and directional biases of anatomic variation [27]. Encoding
of local variation can then be used to assess the severity of struc-
tural variants outside of the normal range, which may be a sign of
270 John Darrell Van Horn and Arthur W. Toga

Normal Controls
Males Females

Schizophrenic Patients
Males Females

Millimeters of Asymmetry
0 16

Fig. 2 Variability in brain cortical architectural asymmetry leads directly to the

consideration of probabilistic atlases and the use of cortically derived landmarks,
for example, sulcal lines. In this figure, sulcal lines were manually determined for
male and female normal subjects and those diagnosed with schizophrenia.
Asymmetry maps were created in each group as defined by sex and diagnosis
(NC normal controls, SZ schizophrenic patients). Sulcal mesh averages for each
hemisphere were subtracted from a reflected version of the same structure in
the other hemisphere to create displacement vectors. Thus, the mappings mea-
sure the degree of lateralization in terms of millimeters of displacement for the
line under diffeomorphic and atlas space constraints. These maps, therefore,
represent the magnitude of average asymmetry in sulcal anatomy between the
two hemispheres between males and females and schizophrenic and normal
subjects (Figure adapted from Narr et al. [72])

disease. A major goal in designing this type of pathology detection

system is to recognize that both the magnitude and local direc-
tional biases of structural variability in the brain may be different at
every single anatomic point. Such atlases are not only cortically
based but can also be done, for instance, on substructures and cer-
ebellum [28].

2.8 Disease-Specific Disease-specific atlases are designed to reflect the unique anatomy
Atlases and physiology of a particular clinical subpopulation. Based on
well-characterized patient groups, these atlases contain thousands
of structure models, as well as composite maps, average templates,
and visualizations of structural variability, asymmetry, and group-
specific differences. They act as a quantitative framework that cor-
relates the structural, metabolic, molecular, and histologic hallmarks
 Brain Atlases: Their Development and Role in Functional Inference 271

of the disease. Because they retain information on group anatomical

variability, disease-specific atlases are a type of probabilistic atlas spe-
cialized to represent a particular clinical group. The resulting atlases
can identify patterns of altered structure or function, and can guide
algorithms for knowledge-based image analysis, automated image
labeling, tissue classification, and functional image analysis.

2.9 Genetic Atlases Inclusion of genetic data in an atlas makes it possible to go beyond
simply describing the effects of a disease on the brain to investigating
its fundamental causes. This not only allows the direct mapping of
genetic influences on brain structure, but also allows us to quantify
heritability for different features of the brain. Familial, twin, and
genetic linkage studies have recently begun to expand the atlas con-
cept to tie together genetic and imaging studies of disease [7, 29].
Atlases that contain genetic brain maps, and a means to analyze them,
can help screen relatives for inherited disease. They also offer a frame-
work to mine large imaging databases for risk genes and quantitative
trait loci, as well as genetic and environmental triggers of disease.

2.10 Age The brain changes remarkably in its size and complexity over the lifes-
and Developmental pan. There is considerable need to account for the age of particular
Stratification populations in the context of brain maturation and the development of
age-stratified normal brain atlas spaces [30]. People who are mildly
cognitively impaired, for instance, are at a fivefold increased risk of
imminent conversion to dementia, and present specific structural brain
changes that are predictive of imminent disease onset [31, 32].
Language impairment in AD patients is also correlated with cortical
atrophy in the left temporal and parietal lobes, bilateral frontal lobes,
and the right temporal pole [33]. However, characterizing such change
presents particular computational challenges. The fitting of brain anat-
omy to a single template of undetermined age specification may lead to
errors in inference about brain morphometry of function relative to an
inappropriate underlying template. Alternative approaches can also be
fruitful and metrics, such as shape [34], cortical thickness mapping,
tensor-based morphometry (TBM), may be better suited for shedding
light on the neuroscience of aging and brain degeneration in AD and
mild cognitive impairment (MCI) [35].

3 Openly Available Atlases of the Brain

An increasing number and variety of brain atlases for humans, as well

as other species, are being made openly available online for the neuro-
science community to use as authoritative references, for inclusion in
data processing workflows, or for the display of results. These include
probabilistic anatomical atlases [15, 36, 37], white matter fiber atlases
[38], and cortical surface atlases [39]. A brief listing of several from
human, nonhuman primate, and the mouse are provided in Table 1.
Table 1
Human, nonhuman primate, and mouse probabilistic anatomical atlases

Name URL(s) Comment

Adult C57BL/6 J Mouse https://www.bnl.gov/medical/RCIBI/ Anatomical atlas of the C57BL/6 J mouse brain
Brain mouse/Atlas_creation.asp
Atlases of the Brain https://msu.edu/~brains/brains/human/index.html Anatomical sections and MR image data of human brain
The Allen Brain Atlas http://www.alleninstitute.org/ Atlas of anatomy and gene expression in the mouse
http://www.brain-map.org/
Brainmaps.org http://brainmaps.org/ Digitally scanned images of serial sections of both primate and
nonprimate brains
Comparative Mammalian http://www.brainmuseum.org/ Digital photos of whole brain and serial sections from a range of
Brain Collections primate brains
Digital Anatomist http://da.biostr.washington.edu/da.html Interactive brain atlas surface models and neuroimaging data
The Human Brain Atlas http://www.thehumanbrain.info/ Interactive online atlas of anatomical preparations and
accompanying labeled sections
ICBM/LONI Probabilistic http://www.loni.usc.edu/atlases/ Human brain atlases based upon probabilistic metrics of
Atlas Series regional location
The Mouse Brain Library http://www.mbl.org/ High-resolution images and database of brains from many
genetically characterized strains of mice
The Mouse Connectome Atlas http://www.mouseconnectome.org/ Database of neuroanatomical connectivity in the mouse brain
Mouse Lemur Brain http://atlasserv.caltech.edu/Lemur/Start_lemur.html Downloadable MR image volumes of the mouse lemur
(Microcebus murinus) brain
Surface Data Management http://sumsdb.wustl.edu/sums/humanpalsmore. Standardized brain surface models with links to associated
System (SuMs) Atlases do;jsessionid=2gfybndwl1 functional data
White Matter Atlas http://www.dtiatlas.org/ Human brain white matter maps obtained from diffusion tensor
imaging (DTI)
The Whole Brain Atlas http://www.med.harvard.edu/AANLIB/home.html Human brain atlas including images from postmortem serial
sections and MRI. Includes aging brain images
 Brain Atlases: Their Development and Role in Functional Inference 273

4 Applications of Atlases for Regional Parcellation and Functional Inference

Without reference to known geometries or atlas spaces, precise

functional localization is not formally possible at the population
level. For instance, in positron emission tomography (PET)/fMRI
studies of human cognition analyzed using the statistical paramet-
ric mapping (SPM) software package relying on the Montreal
Neurological Institute (MNI) atlas as the basis for within group
and between group statistical comparisons, typically, each subject’s
high-resolution anatomical image is warped to the MNI multisub-
ject T1 whole brain template using nonlinear and affine methods
[40]. This transformation is then applied to the collection of lin-
early aligned fMRI time series or task-condition-specific PET
images [41]. Employing regional labeling based upon the chosen
atlas space, the process of localization analysis is enhanced by refer-
ence to known anatomical delineations. The process is typically
decomposed into a series of steps: data warping, feature extraction,
identification of loci, fitting of labels, and region activity value
extraction ([42], for discussion).
Obtaining reliable spatial registration with the chosen atlas
space is essential to accurate localization of functional activity. The
alignment accuracy and impact on functional maps of four spatial
normalization procedures have been compared using a set of high-
resolution brain MRIs and functional PET volumes [43], suggest-
ing that the functional variability is much larger than that comprised
anatomically and that precise alignment of anatomical features has
low influence on the resulting inter-subject functional maps. At
larger spatial resolutions, however, differences in localization of
activated areas appear to be a consequence of the particular spatial
normalization procedure employed. Despite these concerns, how-
ever, for typical sample sizes and numbers of observations per sub-
ject, reliable functional localization is achieved when performed for
each individual using data in atlas space [44].
The use of atlases provides more than just a space in which to
morph images for computing population averages and inferential
statistics on function, but is also useful for the precise labeling of
cortical regions. Figure 3 shows an example of a LONI Pipeline
(http://pipeline.loni.usc.edu) workflow for the process of brain
extraction using FSL’s BET followed by warping to the ICBM452
atlas using FSL FLIRT, and brain surface parcellation using the
Brain Parser algorithm [45]. Unlike alternative methods for detect-
ing the major cortical sulci, which use a set of predefined rules
based on properties of the cortical surface such as the mean curva-
ture, this approach learns a discriminative model using the proba-
bilistic boosting tree (PBT) algorithm, a supervised learning
approach which selects and combines hundreds of features at dif-
ferent scales, such as curvatures, gradients, and shape index [46].
Fig. 3 (a) A LONI Pipeline workflow diagram that reads in an MR structural image volume, performs skull stripping,
fits the data to the ICBM452 standardized atlas, performs Bayesian boost-tree region classification, and (b) returns
the regional delineation results to the original MR image space. Once obtained, the region labeling can be used to
mask functional and/or extract functionally related signal from blood oxygen level dependent data. Such automated
labeling of brain regions would be made considerably more challenging without the use of standard atlas spaces
 Brain Atlases: Their Development and Role in Functional Inference 275

Example output from this method can then be used as regions of

interest (ROIs) from which blood oxygen level dependent (BOLD)
values may then be obtained.
Historically, standardized atlases have demonstrated their
greatest utility in the fitting of experimental data from functional
imaging studies with PET [40] and fMRI [47]. Population-based
inference [48], the identification of individual differences [49],
meta-analytic comparisons [50], and other applications performed
across subjects depend upon accurate atlas-based normalization.
There is no doubt that this is the most scientifically beneficial jus-
tification of atlas construction and demonstrates their ability to
form the spatial basis for comparing subjects with respect to cogni-
tive operations or in comparisons between patient samples.
Continued improvement and enhancement of extant atlases,
such as the Talairach atlas [51], the several iterations of the MNI
atlas, cytoarchitectonic atlases [52], as well as those of the ICBM
probabilistic atlas [53], provide greater accuracy with respect to
functional data and, hence, localization power. Improvement and
enhancement of MNI or Talairach landmarks, for example, may
enable more rapid calculation of spatial transformations, thereby
providing flexibility for specific applications [54].

5 Brain Atlas Revision and Evolution

Spatial mapping of any form is an ongoing process of determining

accurate coordinate locations for content appropriate to that map-
ping’s purpose. As previous content in a map changes or as new
information is obtained, these maps need to be updated, corrected,
and/or modified to reflect these changes in knowledge and the
importance of what new knowledge is being conveyed. For instance,
the information contained in US aeronautical charts is republished
approximately every 3 months partly to reflect changes in the
Earth’s magnetic field isogonic declination lines that point toward
the magnetic North Pole. These field lines vary across North
America, from approximately −19° in the Western US through to
nearly +20° in the East, and must be accounted for to locate the
direction of the true North Pole when charting a navigational
course. However, the Earth’s magnetic field has been drifting slowly
westward since measurements began around 1850, shifting roughly
0.1° or 40 km/year. Failure to periodically update published maps
that incorporate this shift in magnetic declination, as well as other
information concerning changes in the Earth’s topographical fea-
tures, the construction of tall buildings in urban areas, alterations in
air traffic routing, errors in earlier revisions, etc., could result in
pilots or computers making substantial navigational errors due to
improper compass and course directional settings in the absence of
global positioning system (GPS) satellite information.
276 John Darrell Van Horn and Arthur W. Toga

Errors often appear in maps resulting from the information

that was used to create them. Landmarks of note may be mis-
located, their spatial extents distorted, and place names misspelled
or mistranslated. This would also be true for brain mapping atlases
where previous inaccuracies must be addressed, additional data
included, or data from other modalities considered.
Cytoarchitectonic maps from the classical period of describing
brain anatomy have been noted as failing to incorporate sulcal pat-
tern, variation in cell orientation, and being presented as idealized
versions of brain structure [15].
However, even modern approaches, using multimodal meth-
ods, large databases, and sophisticated computer methods, are not
immune from introducing errors. Data misaligned with respect to
a standard atlas space may result in gross inaccuracies and consider-
able problems when trying to make inferences between diagnostic
groups. For instance, the statistics of resulting voxel-based mor-
phometric comparisons may be uninformative about group differ-
ences wherever the spatial normalization algorithm has failed to
register on any robustly appearing image gradient [55]. This has
severe consequences for random-effects-based analyses of morpho-
metric changes due to disease or clinical outcome.
Electronic versions of the atlas of Talairach and Tournoux [56],
including the Talairach Daemon (http://www.talairach.org/) and
the official published versions by Thieme, have been found to con-
tain a discrepant region of the precentral gyrus on axial slice +35 mm
that extends far forward into the frontal lobe. This region has been
found to be anatomically incorrect and internally inconsistent
within the digital atlas software applications that employ multipla-
nar cross-referencing tools [57]. This may be a case of simple mis-
labeling but other forms of atlas warping are known to result in
distortions which must be predicated in context with the accurate
interpretation of location. As new data are included and novel tech-
niques are developed to inform atlases that are open to scrutiny by
researchers, with ongoing updates and corrections, will they become
most widely valuable.
Workers in our lab have pursued the construction of improved
digital brain atlases composed of data from manually delineated
high-resolution MRI [58]. A total of 56 structures were labeled on
the MR volumes of 40 normal healthy volunteers. The labeling was
performed according to a set of protocols developed specifically for
this project. In brief, pairs of raters were assigned to each structure
and trained on the protocol for delineating that structure. Each
rater pair was tested for concordance on 6 of the 40 brains; once
they had achieved reliability standards, they divided the task of
delineating the remaining 34 brains. The data were then spatially
normalized to well-known atlas-based templates using each of
three popular algorithms: AIR’s nonlinear warp [59] paired with
the ICBM452 Warp 5 atlas [60], FSL’s FLIRT [61] was paired
 Brain Atlases: Their Development and Role in Functional Inference 277

with its own template, a skull-stripped version of the ICBM152 T1

average; and SPM’s unified segmentation method [62] was paired
with its canonical brain, the whole head ICBM152 T1-weighted
average. In the end, these approaches produced three variants of a
resultant atlas, where each was constructed from 40 representative
samples of a data processing stream that one might use for analysis.
For each normalization algorithm, the individual structure delinea-
tions were then resampled according to the derived transforma-
tions and computed averages were obtained at each voxel location
to estimate the probability of that voxel belonging to each of the
56 structures. Each version of the atlas contains, for every voxel,
probability densities for each region, thereby providing a resource
for automated probabilistic labeling of external data types regis-
tered into standard spaces. Additionally, computed average inten-
sity images and tissue density maps based on the three methods
and target spaces were also obtained. These atlases are publicly
available on the LONI Web site (http://loni.usc.edu) and, we
believe, will serve as critical resources for diverse applications
including meta-analysis of functional and structural imaging data
and other bioinformatics applications where display of arbitrary
labels in probabilistically defined anatomic space will facilitate both
knowledge-based development and visualization of findings from
multiple disciplines. However, in time these, too, will be replaced
by still more accurate atlases of larger sample size, improved spatial
resolution, with finer anatomical detail.

6 Conclusions

The evolution of brain atlases has seen tremendous advances; they

can now accommodate observations from multiple modalities and
from populations of subjects collected at different laboratories.
The probabilistic systems described here show promise for identi-
fying patterns of structural, functional, and molecular variation in
large image databases for pathology detection in individuals and
groups and for determining the effects of age, gender, handedness,
and other demographic or genetic factors on brain structures in
space and time. Integrating these observations to enable statistical
comparison has already provided a deeper understanding of the
relationship between brain structure and function. Importantly,
the utility of an atlas depends on appropriate coordinate systems,
registration, and deformation methods to allow the statistical com-
bination of multiple observations in an agreed, but expandable,
digital reference framework. In this review, we highlighted two
sources of data that will have an increasingly important role in inte-
grative brain atlases: molecular architectonics and diffusion tensor
imaging (DTI). Once stored in a population-based atlas, informa-
tion from these techniques can help to interpret more conventional
278 John Darrell Van Horn and Arthur W. Toga

functional and structural brain maps by integrating them with data

on molecular content, physiology, and fiber connections – a devel-
opment that can help to formulate and test new types of neurosci-
entific models. A goal of systems neuroscience is to establish brain
systems that underlie cognitive processes and the factors that influ-
ence them. DTI data on fiber connectivity, stored in an atlas coor-
dinate system, can offer a rigorous computational basis to test how
identifiable anatomical systems (e.g., visual, limbic, or corticotha-
lamic pathways) interact. This atlas information can be invoked as
ROI that are incorporated into the statistical design of functional
brain mapping studies (e.g., with fMRI or electroencephalogra-
phy), even when underlying fiber connections are not evident in
the data being collected for a particular study. Molecular architec-
tonic mapping also provides a complementary perspective in which
known neurotransmitter and receptor pathways—the physiology
and molecular features of which are now well understood—can be
associated with functional subdivisions of the cortex, identified
with tomographic imaging. For example, an fMRI study of inhibi-
tory cognitive processes in drug abusers might be informed by
other modalities of data on limbic–prefrontal connectivity (from
DTI), or on cortical monoamine receptor distributions (from
architectonic mapping). In each of these contexts, the coordinate
system of the atlas, and the transformations that equate different
modality data in the same reference frame, provide the means to
build and test system-level models of cognition or disease, incorpo-
rating data from traditionally separate domains of neuroscience.
As brain atlases soon begin to incorporate data from thousands
of subjects, new questions in basic and clinical neuroscience can be
addressed that were previously out of reach. For example, quanti-
tative genetic studies are underway to link functional, structural,
and connectivity information with variations in candidate genetic
polymorphisms that could influence them. As polygenic disorders
involve the interaction of multiple genetic variations, each with a
small effect on the overall phenotype, digital atlases provide the
ideal setting to mine large numbers of images computationally
with hybrid techniques from computational anatomy and quantita-
tive genetics (such as linkage and association studies in which a
statistic is computed at each voxel location in the brain).
Should atlases be constructed specific to different age groups
or different age-related diseases? Several other authors [30, 63]
have come to the same conclusion and many population-based
atlases have emerged in response to this need [64]. But the same
logic can be carried to the next level by creating many different
population-based atlases, each specific to the group demographics,
disease, age, or other characteristics of the subjects being studied.
These provide, not only population statistics within the map, but
arguably better represent the morphological signature of that par-
ticular cohort. What must be included in all analyses are confidence
 Brain Atlases: Their Development and Role in Functional Inference 279

statistics on where the activity takes place. Whether this entails a

statistic on probability, percentile, or other metric may depend on
the experimental design and other factors. Adoption of a single
normal atlas, even a probabilistic version, for all subject studies
provides for the nominal capability for easier comparisons but in
doing so fails to adequately measure the nuances within or between
each group (Fig. 3). It, therefore, seems that it might be prudent
to avoid dependency on a single modality, single group representa-
tion for every study. Any given imaging experiment will be better
served by mapping to a population-based atlas that closely resem-
bles the cohort under study. We suggest that population atlases for
groups, such as AD [35, 65], schizophrenia [66, 67], pediatric
populations [68, 69], autism [70], even decades of life [71], should
be utilized, as appropriate, for that specific subject group.
The next generation of population-based atlases [73] will pro-
vide the necessary statistical power to identify demographic,
genetic, and environmental factors that influence therapeutic
response. These will be essential in the study of the normal and
abnormal human brain. Most important of all, brain atlases are
now being enriched with data from genetics, protein expression, as
well as reflecting associations with phenotypic behaviors. These
efforts can be expected to yield entirely new avenues of research
into the functional organization of the brain and how this is altered
in disease will be of interest not only just to specialists in neuroim-
aging, but also to all basic and clinical neuroscientists.

References
1. Haas LF (2001) Phineas Gage and the science 9. Davatzikos C (1996) Spatial normalization of
of brain localisation. J Neurol Neurosurg 3D brain images using deformable models.
Psychiatry 71:761 J Comput Assist Tomogr 20:656–665
2. Cowie SE (2000) A place in history: Paul Broca 10. Davatzikos C (1997) Spatial transformation
and cerebral localization. J Invest Surg and registration of brain images using elasti-
13:297–298 cally deformable models. Comput Vis Image
3. Goedert M, Ghetti B (2007) Alois Alzheimer: Underst 66:207–222
his life and times. Brain Pathol 17:57–62 11. Thompson PM, Woods RP, Mega MS, Toga
4. Roland PE, Zilles K (1994) Brain atlases – a AW (2000) Mathematical/computational
new research tool. Trends Neurosci challenges in creating deformable and probabi-
17:458–467 listic atlases of the human brain. Hum Brain
5. Toga AW, Thompson PM (2001) Maps of the Mapp 9:81–92
brain. Anat Rec 265:37–53 12. Weaver JB, Healy DM Jr, Periaswamy S,
6. Toga AW, Thompson PM (2002) New Kostelec PJ (1998) Elastic image registration
approaches in brain morphometry. Am using correlations. J Digit Imaging 11:59–65
J Geriatr Psychiatry 10:13–23 13. Barillot C, Lemoine D, Le Briquer L,
7. Thompson P, Cannon TD, Toga AW (2002) Lachmann F, Gibaud B (1993) Data fusion in
Mapping genetic influences on human brain medical imaging: merging multimodal and
structure. Ann Med 34:523–536 multipatient images, identification of struc-
tures and 3D display aspects. Eur J Radiol
8. Narr KL, Thompson PM, Sharma T, Moussai 17:22–27
J, Cannestra AF, Toga AW (2000) Mapping
morphology of the corpus callosum in schizo- 14. Woods RP, Grafton ST, Holmes CJ, Cherry
phrenia. Cereb Cortex 10:40–49 SR, Mazziotta JC (1998) Automated image
280 John Darrell Van Horn and Arthur W. Toga

registration. I. General methods and intrasu- 29. Toga AW, Thompson PM (2005) Genetics of
bject, intramodality validation. J Comput Assist brain structure and intelligence. Annu Rev
Tomogr 22:139–152 Neurosci 28:1–23
15. Toga AW, Thompson PM, Mori S, Amunts K, 30. Toga AW, Thompson PM, Sowell ER (2006)
Zilles K (2006) Towards multimodal atlases of Mapping brain maturation. Trends Neurosci
the human brain. Nat Rev Neurosci 7:952–966 29:148–159
16. Woods RP (2003) Characterizing volume and 31. Apostolova LG, Thompson PM (2007) Brain
surface deformations in an atlas framework: mapping as a tool to study neurodegeneration.
theory, applications, and implementation. Neurotherapeutics 4(3):387–400
Neuroimage 18:769–788 32. Apostolova LG, Akopyan GG, Partiali N et al
17. Avants B, Gee JC (2004) Geodesic estimation (2007) Structural correlates of apathy in
for large deformation anatomical shape averag- Alzheimer’s disease. Dement Geriatr Cogn
ing and interpolation. Neuroimage 23(Suppl Disord 24:91–97
1):S139–S150 33. Apostolova LG, Lu P, Rogers S et al (2008) 3D
18. Avants BB, Schoenemann PT, Gee JC (2006) mapping of language networks in clinical and
Lagrangian frame diffeomorphic image regis- pre-clinical Alzheimer’s disease. Brain Lang
tration: morphometric comparison of human 104:33–41
and chimpanzee cortex. Med Image Anal 34. Scher AI, Xu Y, Korf ES et al (2007)
10:397–412 Hippocampal shape analysis in Alzheimer’s dis-
19. Evans AC, Collins DL, Milner B (1992) An ease: a population-based study. Neuroimage
MRI-based stereotactic atlas from 250 young 36:8–18
normal subjects. J Neurosci Abstr 18:408 35. Thompson PM, Hayashi KM, Dutton RA et al
20. Durrleman S, Pennec X, Trouve A, Ayache N (2007) Tracking Alzheimer’s disease. Ann N Y
(2007) Measuring brain variability via sulcal Acad Sci 1097:183–214
lines registration: a diffeomorphic approach. 36. Mazziotta JC, Toga AW, Evans AC, Fox PT,
Med Image Comput Comput Assist Interv Lancaster JL (1995) Digital brain atlases.
10(Pt 1):675–682 Trends Neurosci 18:210–211
21. Alayon S, Robertson R, Warfield SK, Ruiz- 37. Toga AW, Thompson PM, Mega MS, Narr KL,
Alzola J (2007) A fuzzy system for helping Blanton RE (2001) Probabilistic approaches
medical diagnosis of malformations of cortical for atlasing normal and disease-specific brain
development. J Biomed Inform 40:221–235 variability. Anat Embryol (Berl) 204:267–282
22. Rohlfing T, Maurer CR Jr (2007) Shape-based 38. Wakana S, Jiang H, Nagae-Poetscher LM, van
averaging. IEEE Trans Image Process Zijl PC, Mori S (2004) Fiber tract-based atlas
16:153–161 of human white matter anatomy. Radiology
23. Narr KL, Bilder RM, Luders E et al (2007) 230:77–87
Asymmetries of cortical shape: effects of hand- 39. Van Essen DC (2005) A Population-Average,
edness, sex and schizophrenia. Neuroimage Landmark- and Surface-based (PALS) atlas of
34:939–948 human cerebral cortex. Neuroimage
24. Thompson PM, Giedd JN, Woods RP, 15:635–662
MacDonald D, Evans AC, Toga AW (2000) 40. Fox PT, Perlmutter JS, Raichle ME (1984)
Growth patterns in the developing brain Stereotactic method for determining anatomi-
detected by using continuum mechanical ten- cal localization in physiological brain images.
sor maps. Nature 404:190–193 J Cereb Blood Flow Metab 4:634
25. Corouge I, Dojat M, Barillot C (2004) 41. Evans AC, Marrett S, Neelin P et al (1992)
Statistical shape modeling of low level visual Anatomical mapping of functional activation in
area borders. Med Image Anal 8:353–360 stereotactic coordinate space. Neuroimage
26. Cardenas VA, Boxer AL, Chao LL et al (2007) 1:43–53
Deformation-based morphometry reveals brain 42. Nowinski WL, Thirunavuukarasuu A (2001)
atrophy in frontotemporal dementia. Arch Atlas-assisted localization analysis of functional
Neurol 64:873–877 images. Med Image Anal 5:207–220
27. Leow AD, Klunder AD, Jack CR Jr et al (2006) 43. Crivello F, Schormann T, Tzourio-Mazoyer N,
Longitudinal stability of MRI for mapping Roland PE, Zilles K, Mazoyer BM (2002)
brain change using tensor-based morphometry. Comparison of spatial normalization proce-
Neuroimage 31:627–640 dures and their impact on functional maps.
28. Diedrichsen J (2006) A spatially unbiased atlas Hum Brain Mapp 16:228–250
template of the human cerebellum. Neuroimage 44. Swallow KM, Braver TS, Snyder AZ, Speer
33:127–138 NK, Zacks JM (2003) Reliability of functional
 Brain Atlases: Their Development and Role in Functional Inference 281

localization using fMRI. Neuroimage and nonlinear models. J Comput Assist Tomogr
20:1561–1577 22:153–165
45. Tu Z, Zheng S, Yuille AL et al (2007) 60. Rex DE, Ma JQ, The TAW, LONI (2003)
Automated extraction of the cortical sulci Pipeline processing environment. Neuroimage
based on a supervised learning approach. IEEE 19:1033–1048
Trans Med Imaging 26:541–552 61. Smith SM, Jenkinson M, Woolrich MW et al
46. Luders E, Thompson PM, Narr KL, Toga AW, (2004) Advances in functional and structural
Jancke L, Gaser C (2006) A curvature-based MR image analysis and implementation as
approach to estimate local gyrification on the FSL. Neuroimage 23(Suppl 1):S208–S219
cortical surface. Neuroimage 29:1224–1230 62. Ashburner J, Friston KJ (2005) Unified seg-
47. Ashburner J, Friston KJ (1999) Nonlinear spa- mentation. Neuroimage 26:839–851
tial normalization using basis functions. Hum 63. Van Essen DC (2002) Windows on the brain:
Brain Mapp 7:254–266 the emerging role of atlases and databases in
48. Friston KJ, Stephan KE, Lund TE, Morcom A, neuroscience. Curr Opin Neurobiol
Kiebel S (2005) Mixed-effects and fMRI stud- 12:574–579
ies. Neuroimage 24:244–252 64. Mazziotta J, Toga A, Evans A et al (2001) A
49. Miller MB, Van Horn JD, Wolford GL et al four-dimensional probabilistic atlas of the
(2002) Extensive individual differences in human brain. J Am Med Inform Assoc
brain activations associated with episodic 8:401–430
retrieval are reliable over time. J Cogn Neurosci 65. Mega MS, Dinov ID, Mazziotta JC et al
14:1200–1214 (2005) Automated brain tissue assessment in
50. Fox PT, Parsons LM, Lancaster JL (1998) the elderly and demented population: con-
Beyond the single study: function/location struction and validation of a sub-volume prob-
metanalysis in cognitive neuroimaging. Curr abilistic brain atlas. Neuroimage
Opin Neurobiol 8:178–187 26:1009–1018
51. Nowinski WL (2005) The cerefy brain atlases: 66. Yoon U, Lee JM, Koo BB et al (2005)
continuous enhancement of the electronic Quantitative analysis of group-specific brain
talairach-tournoux brain atlas. tissue probability map for schizophrenic
Neuroinformatics 3:293–300 patients. Neuroimage 26:502–512
52. Amunts K, Schleicher A, Zilles K (2007) 67. Cannon TD, Thompson PM, van Erp TG et al
Cytoarchitecture of the cerebral cortex – more (2006) Mapping heritability and molecular
than localization. Neuroimage 37:1061–1065, genetic associations with cortical features using
discussion 6–8 probabilistic brain atlases: methods and applica-
53. Mazziotta J, Toga AW, Evans A et al (2001) A tions to schizophrenia. Neuroinformatics 4:5–19
probabilistic atlas and reference system for the 68. Wilke M, Schmithorst VJ, Holland SK (2002)
human brain: International Consortium for Assessment of spatial normalization of whole-
Brain Mapping (ICBM). Philos Trans R Soc brain magnetic resonance images in children.
Lond B Biol Sci 356:1293–1322 Hum Brain Mapp 17:48–60
54. Nowinski WL (2001) Modified Talairach land- 69. Jelacic S, de Regt D, Weinberger E (2006)
marks. Acta Neurochir (Wien) 143:1045–1057 Interactive digital MR atlas of the pediatric
55. Bookstein FL (2001) Voxel-based morphome- brain. Radiographics 26:497–501
try” should not be used with imperfectly regis- 70. Joshi S, Davis B, Jomier M, Gerig G (2004)
tered images. Neuroimage 14:1454–1462 Unbiased diffeomorphic atlas construction for
56. Talairach J, Tournoux P (1988) Co-planar stereo- computational anatomy. Neuroimage 23(Suppl
tactic atlas of the human brain. Tieme, New York 1):S151–S160
57. Maldjian JA, Laurienti PJ, Burdette JH (2004) 71. Mazziotta J, Toga A, Evans A et al (2001) A
Precentral gyrus discrepancy in electronic ver- four-dimensional probabilistic atlas of the
sions of the Talairach atlas. Neuroimage human brain. J Am Med Inform Assoc
21:450–455 8:401–430
58. Shattuck DW, Mirza M, Adisetiyo V et al 72. Narr K, Thompson P, Sharma T et al (2001)
(2008) Construction of a 3D probabilistic atlas Three-dimensional mapping of gyral shape and
of human cortical structures. Neuroimage cortical surface asymmetries in schizophrenia:
39:1064–1080 gender effects. Am J Psychiatry 158:244–255
59. Woods RP, Grafton ST, Watson JD, Sicotte 73. Amunts K, Hawrylycz MJ, Van Essen DC et al
NL, Mazziotta JC (1998) Automated image (2014) Interoperable atlases of the human
registration. II. Intersubject validation of linear brain. Neuroimage 99:525–532
 Chapter 10

Graph Theoretic Analysis of Human Brain Networks

Alex Fornito

Abstract
The human brain is a highly interconnected network. It is thus suitable for investigation with graph theory,
a branch of mathematics concerned with understanding systems of interacting elements. Graph theory has
become a popular tool for analyzing human MRI data. In this work, brain networks are modeled as graphs
of nodes connected by edges. The nodes represent distinct brain regions and the edges represent some
measure of structural or functional interaction between regions. This representation enables the computa-
tion of a broad range of metrics that quantify diverse aspects of network organization, thus offering a pow-
erful framework for understanding brain structure and function in both health and disease. This chapter
overviews the principles and methods involved in building and analyzing graph theoretic models of the
brain using MRI. It explains basic concepts, provides examples of how graph theory has shed new light on
brain organization, and considers some limitations of current applications.

Key words Connectome, Connectivity, Graph analysis, Network, Complexity, MRI, DTI, fMRI

1 Introduction

The human brain is a complex, interconnected network. At micro-

scopic resolutions, axons and dendrites sprout from neuronal soma
to enable communication with several thousand other neurons [1].
At macroscopic resolutions, the axons of populations of adjacent
neurons coalesce to form fiber bundles that project through the
white matter volume of the brain to connect distal areas. This
interconnectivity allows the integration of segregated and func-
tionally specialized neuronal systems distributed throughout the
brain. The network organization of the brain thus fundamentally
shapes its function, and generating comprehensive maps of brain
connectivity—so-called connectomes [2]—has become a major
goal of neuroscience [3–5]. The burgeoning field of neural con-
nectomics is thus generating rich data sets describing brain net-
work organization across multiple species and multiple scales of
resolution [6–12] using various microscopic, genetic, tract-­tracing,
informatic, and neuroimaging methods.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_10, © Springer Science+Business Media New York 2016

283
284 Alex Fornito

Graph theory provides an ideal framework for characterizing,

comparing, and integrating results across these diverse data. Graph
theory is a branch of mathematics concerned with studying systems
of interacting elements. The central assumption of the approach is
that any such system can be represented as a graph of nodes (also
called vertices) connected by edges (also called links, arcs, or con-
nections). Equivalently, the network can be represented as a matrix,
in which each i-th row and j-th column represents a distinct node
and each ij-th element encodes the type and strength of connectivity
between each node pair (Fig. 1). The simplicity of this approach is
matched by its versatility—nodes in brain graphs could represent
individual neurons, neuronal populations, or macroscopic regions,
while edges could represent axonal, dendritic, or synaptic contacts,
or large-scale fiber bundles. More broadly, graph theory has also
been used to model other networks found in nature, including social

Fig. 1 Matrix and graph-based representations of brain networks. In matrix form (top row), each row and col-
umn represents a different region and each element represents the connectivity between region pairs. Matrices
can be either binary, representing only the presence or absence of a connection (left and middle), or weighted
to represent variations in the strength of inter-regional connectivity (right). The matrices can also be symmet-
ric, representing an undirected network (left; note how the top-right and bottom left triangles of this matrix are
mirror images of each other), or asymmetric, to represent a directed network (middle and right). In graph form
(bottom row), brain regions are represented as nodes or circles and connectivity as edges. Arrowheads can be
used to represent the directionality of connectivity in directed networks (middle, right). Edge thickness can be
used to represent variations in edge weight (right). These graphs are used for illustrative purposes and do not
directly map onto the matrices shown in the top row. Weighted, undirected networks are not shown
 Graph Theoretic Analysis of Human Brain Networks 285

networks (e.g., nodes are people, edges are social or professional

ties), economic networks (e.g., nodes are companies or countries
and edges represent financial or trade transactions), technological
networks such the world wide web (e.g., nodes are websites and
edges are hyperlinks), transportation networks such as the global air
transportation network (e.g., nodes are airports and edges are con-
necting flights), ecological networks (e.g., nodes are species and
edges are predator-prey interactions) and biological networks other
than the brain (e.g., nodes can be proteins or genes and edges can
encode molecular interactions or coexpression patterns) (see [13,
14] for reviews). Graph theory thus provides a standardized approach
for representing and analyzing diverse types of network data.
In recent years, processing pipelines have been established to
allow the application of graph theoretic methods to human neuro-
imaging data (Fig. 2). This is an important advance, since in vivo

Fig. 2 Basic processing pipeline for graph theoretic analysis of MRI data. (a) Imaging data are first acquired.
Structural connectivity is often assessed using either diffusion MRI (top) or T1-weighted MRI (middle).
Functional and effective connectivity are typically investigated using fMRI (bottom). (b) Once the imaging data
have been acquired, the brain must be parcellated into distinct regions, which act as network nodes. Shown
here are examples of an anatomical parcellation (top), a random parcellation (middle), and a functional parcel-
lation (bottom). (c) The next step is to define some measure of connectivity between nodes, which will repre-
sent the edges of our brain graph. With diffusion MRI, inter-regional connectivity is measured using tractography
(top). With T1-weighted MRI, structural connectivity is indirectly measured using inter-subject covariations in
gray matter morphometry (middle). With fMRI, functional connectivity is measured as a statistical dependence
between regional time series (bottom). (d) The connectivity between all pairs of brain regions can be repre-
sented as a connectivity matrix. MRI analyses typically yield weighted and symmetric matrices (left). These
matrices can be thresholded to emphasize the strongest links in the network (right). (e) The connectivity matrix
can be used to generate a graph-based representation of the network (i.e., a brain graph), in which regions are
represented as nodes and connections as edges. See also Figs. 1 and 3. Parts of this figure have been repro-
duced from [109] with permission
286 Alex Fornito

imaging techniques such as magnetic resonance imaging (MRI)

currently represent the most cost-effective and tractable method
for generating connectomic maps in humans [2] (see also [15, 16]
for developments in the analysis of post-mortem tissue).
Accordingly, there now exists a large literature using graph theory
for the analysis of MRI data. Diffusion MRI is commonly used to
assess structural connectivity of the brain, while functional MRI is
used to characterize functional interactions between brain regions
[17, 18]. Either technique can be used to generate a graph-based
representation of brain network connectivity.
In this chapter, we consider some of the basic principles and
methods of graph theoretic analysis of human MRI data. We first
offer a brief history and rationale supporting the use of graph the-
ory in neuroscience, and in the analysis of human neuroimaging
data in particular. We then discuss how such analyses are performed
with respect to the two major steps involved: building a brain
graph and analyzing a brain graph. We close by considering some
emerging trends and areas for improvement in the field.

2 A Brief History of Graph Theory and MRI

Leonhard Euler is often credited as the founding father of graph

theory [19]. In 1736, he published a solution to a major unre-
solved mathematical problem of the time: whether it was possible
to find a route that traversed each of the seven bridges of the
Prussian city of Konigsberg (now Kaliningrad in Russia) without
crossing any single bridge more than once. Euler simplified the
problem by depicting the geography of the city as a graph, in which
the nodes were distinct landmasses and the edges were the bridges
that connected these masses. Using this abstraction, he was able to
show that no such path was possible.
Following Euler’s success, the application of graph theory was
largely restricted to mathematical studies of topology and certain
areas of theoretical chemistry. It was not until the middle of the
twentieth century that the broader applicability of graph theory
was realized first in the social sciences [20–23] and then in other
fields following the seminal work of Paul Erdős and Alfred Rényi
[24]. These two authors explored the mathematical properties of
probabilistic random graphs in which nodes were connected with
uniform probability. Such random graphs were used to model a
wide variety of real-world biological, technological, and social sys-
tems until two analyses, both published in 1998, demonstrated
that many such systems display a more complex pattern of connec-
tivity than implied by the Erdős-Rényi model.
One analysis, by Barabási and Albert [25], showed that net-
works as diverse as the world wide web, film actor collaborations,
 Graph Theoretic Analysis of Human Brain Networks 287

and the western US electricity power grid show a heterogeneous

distribution of connectivity across nodes. This distribution was
characterized by a large number of vertices with a small number of
connections and a small subset of vertices that were very highly
interconnected with the rest of the network. These highly con-
nected nodes represented putative network hubs. The hub domi-
nance of these networks departs from the expectations of the
Erdős-Rényi random graph model, in which each node has an equal
probability of being connected.
In parallel, Watts and Strogatz [26] published an analysis that
also suggested the Erdős-Rényi model was an insufficient model
for real-world systems. They contrasted the organization of the
Erdős-­Rényi model with a regular, lattice-like graph. The nodes of
this regular graph were arranged around a ring and were linked
only to their k nearest neighbors. Two properties of these networks
were examined: the clustering coefficient and the characteristic
path length. The clustering coefficient quantifies the probability
that two nodes connected to a third are also connected with each
other (Fig. 4c). This metric captures a phenomenon that is well
known in social networks, where two people are more likely to be
friends if they share a third friend in common. The characteristic
path length of a network is the average number of connections
required to travel from one node to any other node in the network
(Fig. 4b). A shorter average path length implies that the network is
integrated and that information is able to spread more rapidly
throughout the network. Watts and Strogatz found that the short-
range connectivity of the regular graph led to high clustering but
high path length, since many short-range links were required to
travel from one end of the network to the other. In contrast, the
Erdős-Rényi random graph had lower path length but also had
lower clustering, since all nodes had an equal probability of being
connected. Compared to the extreme cases presented by the ran-
dom and regular graphs, Watts and Strogatz found that many real
world networks, including the neuronal n ­ etwork of C elegans,
showed high clustering, much like a regular graph, coupled with a
short average path length comparable to a random network.
Indeed, they found that randomly rewiring just a small fraction of
edges in a regular graph created network “short-­cuts” that pro-
duced a dramatic reduction in the characteristic path length of the
network with negligible impact on clustering, leading to a regime
characterised by both high clustering and short average path
length. They termed this organization “small-world,” in reference
to the six degrees of separation phenomenon thought to character-
ize social networks.
It is easy to see how the findings of Barabási and Albert [25]
and Watts and Strogatz [26] can provide important insights into
brain network organization. For example, a heterogeneous
288 Alex Fornito

distribution of connectivity across nodes within the brain would

point to highly connected hub regions that play a particularly
prominent role in information-processing and integrating diverse
network elements. Such a role has been proposed, for example, for
association cortices [27]. Similarly, the small-world properties of
high clustering and short path length provide an ideal foundation
for functional specialization (tightly clustered connectivity) and
functional integration (low average path length)—two fundamen-
tal principles of brain function [28, 29]. Accordingly, graph the-
ory was employed in some of the earliest analyses of connectomes
inferred from the synthesis of published tract-tracing studies [9,
30]. This work paved the way for a substantial body of subsequent
work in these and other connectivity datasets [31–34] (reviewed
in Ref. [35]).
Graph theory was first applied to human electroencephalogra-
phy (EEG) and magnetoencephalography (MEG) data in 2004
[36], followed by several fMRI analyses published shortly thereaf-
ter [37–40]. Each of these studies focused on functional connec-
tivity networks. Connectivity was measured either between
individual voxels [37], electrodes/sensors [36], or large-scale
anatomical regions [38, 39]. This work presented evidence for a
heterogeneous distribution of connectivity across nodes, pointing
to the presence of highly connected hub regions, as well as the
high clustering and short average path length consistent with
small-world organization. Subsequent graph theoretic analysis of
human structural connectivity data measured with diffusion MRI
yielded the same conclusions [12, 41–43].
These developments paralleled an explosion in the graph theo-
retic characterization and modeling of diverse types of complex
systems, and the emergence of a formal science of complex net-
works [13, 14, 44]. As this field has matured, it has generated a
large repertoire of diverse measures and theoretical concepts for
understanding different aspects of network organization, many of
which have great appeal for neuroscience. Graph theory thus not
only offers a standardized, flexible, and scalable method for repre-
senting brain networks, it also provides a rich range of metrics for
making sense of brain network data.
In the following sections, we consider some of the basic
graph theoretic measures applied to human neuroimaging data
and discuss what they have taught us about brain network
organization. We focus principally on concepts and measures
that have been most commonly applied in neuroimaging con-
texts. We discuss more advanced topics in the final section. As
a first step, we consider issues associated with building a brain
graph, as these represent an important foundation for any sub-
sequent analysis.
 Graph Theoretic Analysis of Human Brain Networks 289

3 Building a Brain Graph

Nodes and edges are the fundamental building block of any net-
work graph. The basic unit of analysis in an MRI experiment is a
voxel, which is typically 1–3 mm3 in volume. Voxels thus represent
an aggregation of large populations of neural elements; on average
an estimated 20,000–30,000 neurons and billions of synapses
[45]. This coarse resolution creates ambiguities when attempting
to define appropriate nodes and edges for network analysis. This
problem is critical as invalid node definitions can alter, distort, or
bias results, and accurate mapping of connectivity (edges) is essen-
tial for any valid analysis of network organization [46–50].

4 Defining Nodes

The central problem for node definition in MRI concerns how vox-
els should be aggregated to define a valid parcellation of the brain.
Individual neurons and neuronal columns have both been proposed
as fundamental units for brain network organization [7, 51], but
cannot be resolved with typical human MRI acquisitions. Similarly,
cytoarchitectonic regions, such as those delineated in Brodmann’s
classic map, cannot be resolved with MRI and the boundaries of
these regions are often poorly correlated with macroscopic land-
marks (e.g., sulci and gyri) [52, 53]. Due to these limitations, several
different heuristic approaches have been used for node definition
with MRI.
To this end, three criteria for an ideal node for a brain graph
have been proposed [18]: (1) spatial embedding; (2) intrinsic
homogeneity; and (3) extrinsic heterogeneity. The first criterion
simply means that spatial relationships between nodes should be
taken into account. The brain is embedded within the
­three-­dimensional volume of the skull and this embedding places
important constraints on network wiring [54, 55]. Fortunately,
spatial relations between nodes are easily accounted for in MRI
analysis with standard stereotactic mapping techniques, since the
location of each region can be easily represented with reference to
the Montreal Neurological Institute (MNI) or Talairach and
Tourneoux coordinate systems.
The second and third criteria simply mean that each node
should represent a structurally or functionally homogeneous entity
(intrinsic homogeneity) and should be distinguishable from other
nodes (extrinsic heterogeneity). For example, a cytoarchitectonic
region is intrinsically homogeneous, to the extent that it defines a
population of neurons with shared histological properties. Different
regions are extrinsically heterogeneous, to the extent that they
serve different functional roles in the network (e.g., areas of visual
290 Alex Fornito

cortex processes visual information, regions of parietal cortex

processes spatial information, and so on).
In practice, the extrinsic heterogeneity criterion is difficult to
fulfill. The unique functional role of an individual node in a brain
graph is determined by its connectivity with other areas (i.e., its
“connectional fingerprint”) [56], as well as its own intrinsic circuitry,
cellular composition, gene expression patterns, physiological prop-
erties, and so on. Comprehensively understanding how the intrinsic
properties of each node vary, and how this variation defines the
functional role of that node, is an unresolved question in neurosci-
ence. Consequently, nearly all graph theoretic studies of brain net-
works treat nodes as uniform network elements, and the primary
distinctions between nodes are based on variations in their connec-
tivity profiles with other areas. (Note however, that some computa-
tional models do explicitly account for variations in the functional
roles of different networks nodes [57].)
Given that the spatial embedding criterion is easily accommo-
dated with standard imaging techniques and the extrinsic heteroge-
neity criterion is difficult to meet in practice, the primary emphasis
in developing methods for defining nodes for graph theoretic analy-
sis of MRI data has been on the intrinsic homogeneity criterion.
Five broad approaches have been used, which we refer to here as
voxel-based, anatomical, random, data-driven, and quantitative.

4.1 Voxel-Based A simple solution to the node identification problem in MRI is to

Parcellation use the best resolution possible and thus treat each individual voxel
as a distinct network node. This approach has been used in fMRI
analyses (e.g. [49, 58, 59],) but suffers from two major drawbacks.
First, there is no guarantee that voxel borders represent the
­appropriate boundaries for delineating homogeneous neuronal pop-
ulations in the brain [60]. In other words, functionally homoge-
neous populations could extend over spatial scales that are either
smaller (e.g., cortical columns) or larger (cytoarchitectonic divi-
sions) than the volume occupied by a single voxel. The second limi-
tation of a voxel-based approach is that it is computationally
intensive. Voxel-based networks typically comprise 104–105 nodes
and millions of connections. Such large networks can pose problems
for computational tractability.

4.2 Anatomical Anatomical atlases offer an alternative method for brain parcella-
atlases tion that minimizes computational burden. For example, one pop-
ular atlas, the Automated Anatomical Labelling (AAL) atlas [61],
parcellates the brain into 116 regions largely defined according to
the sulcal and gyral landmarks of an individual brain. The general-
izability of this template is thus questionable. Alternative atlases,
based on probabilistic maps of sulcal and gyral regions, such as the
Harvard-Oxford atlas (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/
 Graph Theoretic Analysis of Human Brain Networks 291

Atlases) and the Desikan-Killaney atlas [62](see Fig. 2b, top),

overcome this limitation. However, as previously stated, sulcal and
gyral landmarks often correspond poorly with the borders of actual
functional subdivisions of the brain. This poor correspondence
limits the validity of these anatomical atlases. Moreover, the size of
the regions in anatomical parcellations can vary considerably, and
these variations can affect connectivity estimates and thus bias sub-
sequent analyses [48, 63].

4.3 Random One way to ensure that a parcellation comprises regions with homo-
Parcellation geneous volume is to divide the brain into random parcels of similar
size (e.g., Fig. 2b, middle). These parcellations can be performed at
varying resolutions, typically yielding networks between 102 and 104
nodes [12, 48, 50]. This approach ensures homogeneity of regional
volume, but there is no guarantee that the random parcels accurately
capture true functional subdivisions of the brain. Replication of
results across several iterations of a random template may also be
necessary to ensure that any findings are not due to the specific char-
acteristics of any single instance of a random parcellation.

4.4 Functional A more hypothesis-driven method uses regions-of-interest defined

Parcellation according to some functional property of interest (e.g., Fig. 2b,
bottom). For example, one study used a meta-analysis of task-­
based fMRI activation studies across a range of cognitive processes
to identify 160 stereotactic coordinates of task-related activation
peaks [64]. Spherical regions-of-interest centered on these coordi-
nates were then created and used as nodes in a graph theoretic
analysis of developmental effects on human brain functional con-
nectivity (see also [65]). Other studies have used similar approaches
after defining spherical regions-of-interest centered on peak coor-
dinates derived either from resting-state functional connectivity
analyses [66] or task-related activation mapping [67, 68] (see also
[69]). This approach is well suited for testing hypotheses about
specific systems of interest with functional MRI. However, this
method is harder to use with diffusion imaging, where larger areas
of gray and white matter may need to be sampled to adequately
measure the tracts projecting into and out of a region.

4.5 Data-Driven In contrast to hypothesis-driven methods for defining regions-of-­

Parcellation interest, data-driven approaches attempt to define functionally
homogeneous collections of voxels using specific characteristics of
the imaging data. A classic example is spatial independent compo-
nent analysis (ICA), which decomposes fMRI data into a set of
components (networks) whose voxels are correlated with each
other, and maximally spatially independent of other components
[70, 71]. Lower-order decompositions typically recover canonical
neural networks such as the default mode network (DMN), fronto-­
parietal network, and so on [72]. Graph analysis between these
292 Alex Fornito

networks can be performed (called “functional network connectiv-

ity” analysis; e.g. [73],). However, as these networks commonly
involve multiple, spatially distributed brain regions, they do not
conform to a traditional conception of a brain network node as
consisting of an anatomically contiguous region. Higher order
decompositions can separate the regions comprising these distrib-
uted networks into separate components [74], although it is often
difficult to know, a priori, the dimensionality of the decomposition
required to obtain such a solution.
Other data-driven approaches examine the connectivity of
each voxel or small region to all other areas, and cluster voxels
or regions with similar connectivity profiles into single parcella-
tion units [59, 75–77]. When combined with spatial constraints
[78], these methods can yield parcellations of the brain that
guarantee spatial contiguity of all voxels within a region, while
also ensuring that such voxels fulfill the criterion of intrinsic
homogeneity (to the extent that they share similar inter-regional
connectivity profiles). Whole-brain parcellations using such
methods applied to task-free, resting-state fMRI are robust
across samples [59, 77]. Similar approaches have been applied
to diffusion MRI data to parcellate-­specific regions based on
structural connectivity profiles [79–81]. Diffusion MRI, being
a measure of brain anatomy rather than function, should pro-
vide a more stable parcellation than those based on fMRI, but
whole-brain parcellation of diffusion MRI has not yet been
extensively validated.

4.6 Quantitative A related data-driven approach involves using quantitative, bio-

Parcellations logical criteria to define nodes. Recently, it has been suggested
that the ratio of T1- to T2-weighted imaging contrast can be
used to generate myelin maps of the brain, and that gradient
detection algorithms can be used to identify boundaries where
there are sharp transitions in myeloarchitecture [82]. In some
areas, these boundaries correspond well with known functional
boundaries [82]. However, it is as yet unclear whether this
approach is scalable to whole-brain parcellations. Alternative
quantitative parcellations involve the projection of data from
postmortem analyses into stereotactic space, yielding probabilis-
tic cytoarchitectonic atlases [83] and maps of regional variations
in chemoarchitecture [84]. However, such data are presently
only available for limited regions of cortex.

4.7 Summary There is a variety of methods for delineating brain network nodes
in MRI data. Each approach has distinct strengths and weaknesses
and there is no gold standard. Ultimately, investigators must
choose an approach best suited to the specific hypothesis being
tested, and ensure that analyses are interpreted with respect to the
limitations of the specific method employed.
 Graph Theoretic Analysis of Human Brain Networks 293

5 Defining edges

The edges of a brain graph represent the connectivity between pairs

of brain regions. There are three broad classes of brain connectivity:
structural, functional, and effective. Both the specific class of con-
nectivity studied and the method used to measure it have a major
impact on the structure of the resulting brain graph and the types
of analyses that can be performed. This section discusses issues asso-
ciated with measuring each of these types of connectivity.

5.1 Structural Structural connectivity refers to the anatomical connections link-

Connectivity ing distinct neural elements. At resolutions accessible with MRI,
structural connectivity refers to axonal fiber bundles intersecting
macroscopic brain regions. Structural connectivity has been mea-
sured with MRI using two approaches. A relatively indirect method
involves analyzing correlations in the gray matter volume (or den-
sity or cortical thickness) of different regions across subjects (e.g.,
Fig. 2c, middle). Volumes of different brain regions vary across
individuals. If these inter-individual variations of volume are cor-
related between two regions, the regions are said to be “con-
nected.” It is generally assumed that these correlations reflect
anatomical connectivity or mutually trophic influences [85, 86].
A more direct approach uses diffusion MRI. Specifically, trac-
tographic analyses attempt to reconstruct the trajectories of major
fiber bundles based on the preferred direction of water diffusion in
each voxel. Axons present barriers to water diffusion. The direc-
tion of preferred water diffusion in the brain is thus constrained by
the trajectory of its axonal fibers. These trajectories are recon-
structed using streamlines that propagate through the white mat-
ter on a voxel-by-voxel basis according to specific algorithmic rules
(e.g., Fig. 2c, top). A wide range of tractography algorithms exists
and each is associated with distinct strengths and weaknesses [87]
(see also [18]). The accuracy of the algorithm critically determines
the validity of the resulting structural brain graph.
Once putative fiber tracts have been reconstructed, connectiv-
ity between regions is commonly measured using one of two
approaches. One method estimates the strength of connectivity as
the number of reconstructed trajectories intersecting each pair of
brain regions. The raw streamline count is also often normalized
by the size of the connected regions, since larger regions will gen-
erally intersect a larger number of streamlines [12]. Measuring
structural connectivity based on streamline counts assumes that
there is a correlation between the number of axons comprising a
fiber bundle and the number of streamlines required to reconstruct
that bundle. However, a streamline is not tantamount to an axon;
rather it is an abstract structure that is used to track a trajectory of
water diffusion through the brain. It therefore provides an indirect
294 Alex Fornito

measure of axonal structure. In reality, reconstructed bundles may

vary in the number of streamlines they posses because of differ-
ences in the axonal density of the actual fiber pathway, variations in
the orientation and organization of the fibers, or variations in the
signal-to-noise characteristics of the image [88].
An alternative approach is to compute some measure of
fiber integrity averaged over the extent of the reconstructed
fiber tract. One commonly used index is fractional anisotropy
(FA), which measures the degree to which water diffusion is
constrained within each voxel. Disorganized or damaged axons
offer reduced barriers to water diffusion. Lower FA values are
thus often used as a marker of impaired white matter integrity,
and the average FA of voxels within a tract can be used to index
of the integrity of that tract. Alternative diffusivity measures,
such as mean, radial, and axial measures of diffusivity, can also
be used for this purpose. This approach attempts to derive a
more biologically meaningful index of connectivity than stream-
line count, although diffusion-­based measures of white matter
integrity can also be affected by differences in white matter
organization and image signal-to-noise, making their interpre-
tation ambiguous [88]. An alternative approach involves com-
bining tractography results with magnetization transfer images,
which provide a more direct index of the myelin content of
brain voxels [89]. Another promising line of work is developing
novel diffusion imaging sequences for measuring axonal diam-
eter [90].
An important limitation of diffusion tractography is that it can-
not resolve the source and target of a fiber pathway. The resulting
connectivity measures are therefore undirected – they tell us
whether a connection between two regions exists, but they do not
tell us whether region i connects to j or vice-versa. As discussed
below, this simplification limits the types of graph theoretic analy-
ses that can be performed and precludes a consideration of the
directionality of information flow in the brain.

5.2 Functional Functional connectivity refers to a statistical dependence between

Connectivity neurophysiological recordings measured in distinct brain regions
[91] (e.g., Fig. 2c, bottom). It thus quantifies functional interac-
tions within brain networks. Most commonly, functional connec-
tivity is assessed via a simple Pearson correlation between time
courses extracted from two or more regions, although alternative
measures such as partial correlation, mutual information, coher-
ence, and so on can be used. Studies of simulated fMRI data have
shown that these measures, which result in undirected measures of
connectivity, vary in their ability to accurately reconstruct the true
edges of a graph, although simple measures such as the Pearson
correlation and partial correlation perform reasonably well [47].
 Graph Theoretic Analysis of Human Brain Networks 295

Nonetheless, these measures do have limitations. For example,

correlations do not distinguish between direct and indirect con-
nections, meaning that two regions lacking a structural connection
may still show highly correlated activity because they are indirectly
connected via a third area. Indeed, it is well known that correlation-­
based measures of functional connectivity are sensitive to polysyn-
aptic connections [92]; as a result, functional connectivity brain
graphs tend to be more densely connected than structural connec-
tivity graphs [93]. This sensitivity to indirect connections can
introduce non-trivial structure into the network [94]. Partial
­correlations partly correct this problem, but can also be associated
with bias in large-scale brain network analyses [94].
Functional connectivity is often assessed during task perfor-
mance or task-free “resting” states [95, 96]. During resting-state
designs, functional connectivity has traditionally been measured
using temporal correlation taken across the duration of the fMRI
acquisition protocol. In effect, this approach summarizes brain
activity occurring over that period with a single scalar value. It
therefore assumes stationarity of brain dynamics. Recent work has
shown that brain dynamics show significant non-stationarities [97,
98] (see [99] for a review). These non-stationarities can be assessed
using sliding window analyses [97] or multivariate decompositions
(e.g., ICA) in the temporal domain [98]. The result is a time series
of networks, which can then be analyzed to understand how brain
network organization evolves over time. Recent multiband fMRI
acquisition sequences that enable more rapid sampling of brain
activity increase the power of such analyses [100].
A parallel line of work investigates functional connectivity dur-
ing task performance. In such analyses, we are often interested in
isolating brain functional networks that are modulated by chang-
ing task conditions. Two approaches that are scalable to whole-­
brain networks have been developed. One method, termed beta
series correlation, attempts to model regional activation changes to
each and every task event. Events are then sorted by condition and
concatenated to generate condition-specific pseudo-time series
(termed beta series, because event-related activity is modeled using
a beta coefficient estimated with a general linear model) reflecting
trial-to-trial variations of evoked activity. These pseudo-time series
are then correlated between regions to quantify task-related func-
tional connectivity as covariations in trial-to-trial fluctuations of
brain activity evoked by each task condition [101, 102]. An alter-
native method is an adaptation of the psychophysiological interac-
tion (PPI) framework first proposed by Friston et al. [103]. PPI
analysis multiples a task regressor of interest by the time course in
a region-of-interest to generate a psychophysiological interaction
term representing task-related modulations of that region’s activ-
ity. These terms can be correlated between regions to estimate
task-related functional connectivity. Partialling out the raw regional
296 Alex Fornito

time courses and task regressors allows task-related functional con-

nectivity to be isolated from task-unrelated (intrinsic) dynamics
[69]. Several variants of this method that are scalable to whole-­
brain networks have been used [67–69, 104].

5.3 Effective Effective connectivity is the influence of one neural system over
Connectivity another [91]. It thus quantifies causal interactions amongst brain
regions and the resulting connectivity estimates are directed
(Fig. 1). Critically, the causal interactions that define effective con-
nectivity must be specified at the neuronal level. Because neuronal
dynamics are not directly observable with fMRI, estimating effec-
tive connectivity with this imaging technique requires a model that
maps the observed hemodynamic signal changes to the underlying
neuronal dynamics from which they were generated [105].
The most popular framework for effective connectivity analysis
of fMRI data is dynamical causal modeling (DCM) [106]. DCM
uses a model of neurovascular coupling to specify the mapping
between neuronal activity and hemodynamics. Different graph
models of causal interactions (i.e., directed graph configurations)
between neuronal systems are specified and compared to deter-
mine which model best accounts for the observed fMRI data. As
the number of possible graph models rapidly increases with net-
work size, DCM has traditionally only been applied to relatively
small subnetworks comprising a few regions. Work is under way to
scale these methods to larger systems [107]. DCM is applicable to
both task and resting-state fMRI data [108].

5.4 Summary MRI-based measures of structural and functional brain connectiv-

ity are indirect. As a result, connectivity analyses of MRI data must
be interpreted with regards to the limitations of the measurement
technique. In the context of graph theoretic analyses, a major limi-
tation is that most structural and functional connectivity measures
are undirected, limiting our capacity to resolve directions of infor-
mation flow in the brain and to uncover hierarchical organizational
features of brain networks (e.g., top-down vs. bottom-up connec-
tions). Nonetheless, many of the organizational properties of the
human brain identified with MRI have been replicated in analyses
of other species, where connectivity data have been acquired using
more precise and invasive techniques. We consider some of these
properties in Sect. 9.

6 From Connectivity Matrix to Brain Graph

Once structural, functional, or effective connectivity between every

pair of brain regions in a given parcellation has been measured, the
data can be succinctly represented as a connectivity matrix. In
 Graph Theoretic Analysis of Human Brain Networks 297

graph theory, this matrix is often called an adjacency matrix,

denoted A (Fig. 1). These matrices can be binary or weighted,
symmetric, or asymmetric. In a binary matrix, Aij = 1 if nodes i and
j are connected and Aij = 0 otherwise. All connections are treated
equally and no distinction is made between connections with dif-
ferent strength or weighting. Binary matrices thus represent the
presence or absence of a neural connection. By contrast, the ele-
ments of a weighted matrix can span a range of values that is deter-
mined by the method used to measure connectivity. For example,
if functional connectivity is estimated using a Pearson correlation,
the values will be bounded in the range [−1, 1]. If structural con-
nectivity is estimated using streamline counts, the values will
positive integers.
If a connectivity matrix is symmetric, the values in the upper
triangle of the matrix are the same as the values in the lower tri-
angle (Fig. 1). In other words, Aij = Aji . No distinction is made
concerning the source and target of a connection. Such matrices
are typical of diffusion MRI or correlation-based analyses.
Asymmetric matrices explicitly encode asymmetries (and thus,
directionality) in connectivity (i.e., Aij ¹ Aji ). Asymmetric matrices
are used to represent effective connectivity (Fig. 1). Both symmet-
ric and asymmetric matrices can be either weighted or unweighted.
MRI-based estimates of connectivity are inherently noisy and
it is often useful to threshold the connectivity matrix to distinguish
real or probable connections from spurious or improbable connec-
tions (Fig. 2d). Thresholding should be done with care, as net-
works should only be compared if they have the same number of
nodes and edges. Any thresholding procedure should thus try to
respect this condition. One common approach is to apply an adap-
tive threshold to different networks to achieve a pre-­specified con-
nection density, which represents the number of edges present in a
network relative to the total possible number of edges. For exam-
ple, we could apply a threshold that ensures only the top 10 % of
connections across a sample of individuals are retained. However,
systematic differences in connectivity strength between individuals
or groups can bias this approach. If one group has lower mean
connectivity than another group, a lower threshold will be required
to achieve the desired connection density. This lowered threshold
may result in the retention of a larger number of low-weight and
potentially spurious connections, altering network structure [109].
A variety of alternative thresholding methods are available, but no
method is completely free of bias [110]. It is therefore prudent to
repeat analyses across a range of thresholds and using alternative
strategies to ensure that any results obtained are robust to this
methodological parameter.
Once the final structure of the connectivity matrix has been deter-
mined, the network can be represented as a graph (Figs. 1 and 2d).
298 Alex Fornito

In graph form, the rows and columns of the matrix—that is, the brain
regions—are depicted as nodes (often circles), and the matrix ele-
ments Aij determine which pairs of nodes are linked by edges.
Variations in connectivity strength encoded in weighted matrices are
commonly represented as variations in edge thickness (Fig. 1). The
directionality of connectivity that is encoded in asymmetric matrices is
depicted by arrowheads attached to the edges (Fig. 1).
Brain graphs can be projected in different ways, in order to
highlight specific relations between nodes. Common displays in
neuroscience include anatomical projections, in which nodes are
positioned according to their stereotactic coordinates; topological
projections, in which node positions are determined based on
some topological relation between nodes; and circular projections
(also called connectograms [111]), which allow a simplified view
of the connectivity of the entire network. Examples of each of
these projections are presented in Fig. 3.

7 Analyzing Brain Graphs

Once the network has been mapped, it can be analysed with respect
to either its connectivity or topology. Connectivity analysis con-
centrates on variations in the type and strength of connectivity
between brain regions. Topological analysis is concerned with
understanding how connections are arranged with respect to each
other, and provides insight into key organizational principles of the
connectome.

Fig. 3 Different graph projections of a brain network. (a) A projection in anatomical space, where nodes are
positioned according to their stereotactic coordinates. Nodes and edges are colored according to the module
to which they have been assigned. (b) A topological projection, in which nodes are located more closely in
space if they have short path length between them. Nodes near the center of the graph have a short average
path length to other nodes, and thus represent central elements of the network. Nodes colors are the same as
in (a). (c) A ring projection, which is also called a connectogram. Nodes are clustered (and colored) by the
modules to which they belong. Within each grouping, the nodes have been ordered according to their averaged
connectivity strength with other nodes (yellow bars). The nodes have been labeled using arbitrary numbers
 Graph Theoretic Analysis of Human Brain Networks 299

8 Connectivity Analysis

Connectivity can be studied at the level of specific neural systems,

called candidate systems analysis, or across the entire brain, called
connectome-wide analysis [112]. Candidate system analyses do
not require a comprehensive map of connectivity between all pairs
of brain regions and typically focus on one or a few networks of
interest. Such analyses are exemplified by seed-based connectivity
approaches, in which the structural or functional connectivity of
specific seed regions to the rest of the brain is assessed [80, 113],
and studies using ICA to investigate brain network connectivity.
Connectome-wide analyses interrogate effects at each and
every element of the connectivity matrix. These analyses pose a
major multiple comparisons problem. In an undirected network
N ( N - 1)
with N nodes, there are possible connections. Thus, an
2
analysis of an undirected network with 103 nodes will require cor-
rection over 499, 500 comparisons. A simple Bonferroni correc-
tion in such circumstances will be too conservative. Fortunately,
more powerful correction procedures are available [114–116]. In
one such approach, called the network-based statistic (NBS) [116],
a test statistic of interest (e.g., t-test, correlation, etc.) is computed
at each and every edge, resulting in a statistic matrix with the same
dimensions as the connectivity matrix. A primary threshold is
applied to this matrix to define a pseudo-network of statistic val-
ues. The size of the connected components of this pseudo-network
are computed and evaluated with respect to an empirical null dis-
tribution, which is generated by repeating the analysis many times
after appropriate permutation of the data. In this context, a con-
nected component refers to a collection of nodes that can be linked
by a set of supra-threshold edges. The probability of observing
components as large as those seen in the data by chance is com-
puted with respect to the empirical null distribution of maximum
component sizes, ensuring that the resulting probability values are
corrected for multiple comparisons [117]. In this manner, the
NBS identifies sets of nodes and edges that show a common effect
of interest. Simulation studies have shown that, compared to tradi-
tional correction methods such as the False Discovery Rate [118],
the NBS can offer great gains in sensitivity when effects are distrib-
uted across multiple edges [116, 119].
An alternative to analyzing connectivity at each and every edge
is to examine the average connectivity of each node to all other
brain regions. This measure has been variously referred to as con-
nectivity strength, connectivity density, or global connectivity
[120–122]. Since the analysis is conducted across nodes rather
than edges, the multiple comparison correction required is much
smaller (i.e., on the order of N, rather than N ( N - 1) ). The analysis
300 Alex Fornito

is also readily scalable to voxel-wise networks and can be useful for

mapping areas in which connectivity varies with some variable of
interest (e.g., diagnosis or cognitive performance). However, as
the connectivity of each node is averaged across all other brain
regions, more subtle effects specific to particular pair-wise links or
circuits may be missed.

9 Topological Analysis

Topological analysis of brain networks draws most heavily on graph

theory. The goal of topological analysis is to characterize how con-
nections and nodes relate to each other, thereby shedding light on
principles of brain network organization. Studies of brain network
topology have revealed several non-trivial topological properties.
In this section, we overview some of the most commonly applied
metrics. Formal definitions are presented for binary, undirected
networks. Generalizations for weighted and undirected networks
are also available (see [123]).

9.1 Hub Dominance Barabási and Albert’s [25] discovery that many real-world net-
works have a heterogeneous distribution of connectivity across
nodes suggests that such networks possess highly connected hubs
that exert a disproportionate influence over the network. In net-
work parlance, the total number of connections attached to a node
is called its connectivity degree, denoted k, and the distribution of
degree values across nodes is the degree distribution of a network.
In the real-world networks they studied, Barabási and Albert found
that the probability of finding a node with increasing k decayed as
a power-law of the form P ( k ) ~ k -a , where α is the scaling expo-
nent that determines the rate of decay. This decay is generally much
slower than the decay observed in homogeneous random networks
such as those studied by Erdős and Rényi, where the degree distri-
bution approximates a Gaussian (more accurately, it conforms to a
binomial distribution); i.e., most nodes have a degree value close
to the mean, and the probability of finding large deviations from
this mean is very low. This clustering around a mean value endows
these networks with a single, characteristic scale. In contrast,
power-law degree distributions are heavily skewed with an extended
tail, pointing to a higher probability of finding nodes with very
high connectivity values despite most nodes having low degree.
There is no meaningful average degree or characteristic scale in
these networks, so they are sometimes called scale-free.
Early work suggested that brain networks, at least when con-
structed at high resolution, conform to a power-law degree distri-
bution [37, 49, 58]. Other studies of both structural and functional
connectivity networks have more commonly reported evidence
 Graph Theoretic Analysis of Human Brain Networks 301

that the degree distribution of the human brain follows an expo-

nentially truncated power law [12, 40, 48]. Truncated power-law
distributions show power-law scaling over a limited regime, point-
ing to the existence of highly connected hub nodes (e.g., Fig. 4a).
However, they show a rapid decay in the probability of finding
nodes with very high degree beyond a certain cut-off. For this rea-
son, networks with truncated power-law distributions are some-
times referred to as broad-scale [124]. This organization seems
plausible for the brain, as metabolic and spatial constraints place an
upper limit on the total number of connections that any single
region can possess.
The presence of highly connected hubs in the brain suggests that
certain brain regions play a critical role in integrative network func-
tion. Studies of both structural and functional connectivity converge
to suggest that brain network hubs are predominantly located in
multimodal association cortex, although hubs in the striatum and
thalamus have also been noted [125–127]. These findings are consis-
tent with the proposed role of association cortices and subcortical
nuclei in integrating information from diverse modalities. These
brain network hubs are more highly interconnected with each other
than expected by chance, forming a so-­called rich-club of connectiv-
ity: a densely connected core of high-degree nodes that acts as a cen-
tral information-processing backbone that absorbs a large bulk of
neuronal traffic [128, 129]. The rich-club organization of the brain
facilitates the rapid transfer and integration of information between
otherwise segregated neural systems [130].

9.2 Robustness The degree distribution of a network has important implications

to Damage for its robustness to damage. Network robustness can be assessed
by removing a node (or edge) and its incident connections accord-
ing to different rules. The properties of the remaining network can
be analyzed after removal of each node to uncover which elements
are most critical for network integrity. One commonly studied
property in this context is the size of the largest connected compo-
nent, S. In an intact network, S = N , where N is the number of
nodes in the network. The value S can be computed after each
node is removed to determine the number of nodes that is required
to fragment the network (determined by the point at which S < N ).
By removing nodes in different orders, we can simulate different
types of network disruption. Stochastic failures can be simulated by
random removal of nodes, whereas attacks can be simulated by tar-
geted deletion of nodes based on their connectivity degree or some
other property of interest. Compared to single-scale networks, scale-
free systems are more resilient to random failure but highly vulner-
able to targeted attack [131]. This vulnerability to attack arises
because the concentration of connectivity on the hub nodes of a
scale-free network means that only a few hubs need to be removed
302 Alex Fornito

Fig. 4 Example of key topological properties of brain networks. (a) illustration of a single network hub (red) with
high degree compared to other nodes. (b) example of the shortest path between two nodes at opposite ends
of the network (red). In this case, the shortest path traverses five edges, so the path length between these two
nodes is five. The characteristic path length of a network is the average path length between every pair of
nodes. (c) Example of a connected triangle of nodes (red). The clustering coefficient of a node is computed as
the number of such triangles attached to that node, relative to the total possible number of triangles. (d)
Illustration of a modular decomposition of the network. Three modules have been identified, as represented by
the different background colors. Nodes within a module are strongly connected with each other and sparsely
connected with nodes in other modules. Such a decomposition allows analysis of node roles and hub category.
Red highlights a provincial hub which is highly connected within its own module. Orange highlights a connec-
tor hub, which has connections distributed across all modules

to fragment the network. However, since hubs are ­relatively rare, the
probability that they will be affected by random failure is low; hence
the greater resistance to random node deletion relative to single-
scale systems. Compared to scale-free systems, networks with trun-
cated power-law degree distributions, such as those thought to
characterize human brain networks, show comparable resilience to
failure and better robustness in the face of targeted attack [40]. This
enhanced robustness occurs because the concentration of connectiv-
ity on hub nodes is less extreme than in scale-free systems.
Nonetheless, damage to hub nodes, or the links between them,
exerts a more severe impact on network function than damage to
 Graph Theoretic Analysis of Human Brain Networks 303

peripheral brain regions or connections [125]. The clinical impli-

cations of this conclusion should be obvious: we should expect that
brain disorders affecting hub regions will present with more severe
symptoms and/or a higher degree of impairment [132]. The high
connectivity of brain network hubs may also render them more
susceptible to disease processes originating elsewhere in the brain
[55, 132]. Consistent with this view, pathology of hub regions is
over-represented in a wide variety of brain disorders [133].

9.3 Small-Worldness The small-world class of networks discovered by Watts and Strogatz
[26] provides an appealing model for the brain. Clustered connectiv-
ity offers a substrate for functional specialization, whereas short aver-
age path length facilities functional integration. Clustering is formally
quantified using the clustering coefficient, which computes the prob-
ability that two nodes linked to an index node are also connected
with each other. In other words, it counts the number of closed tri-
angles attached to a node (Fig. 4c). The path length between two
nodes is simply the number of edges on the shortest path intersecting
those nodes (Fig. 4b). The characteristic path length of a network is
the average path length computed across all node pairs.
Empirically, the small-worldness of a network can be quanti-
fied by comparison to an ensemble of random graphs matched for
the number of nodes, edges, and degree distribution. Various algo-
rithms are available for constructing such graphs by rewiring the
connections of an observed network [122, 134]. Such surrogate
networks provide a useful baseline for evaluating the degree to
which a particular topological property is expressed in the brain.
Formally, a network is considered small-world if the scalar
quantity s > 1, where s = g / l [135]. The quantity γ is computed
as a ratio of the observed clustering coefficient to the average clus-
tering of an ensemble of matched randomized networks (i.e.,
g = Cobs / Crand ). The quantity λ is the ratio of the observed network
path length to average path length computed in the same ensemble
of randomized graphs (i.e., l = Lobs / Lrand ). Small-world networks
will have greater clustering and similar path length to randomized
surrogates. As such, g > 1 and l ~ 1, yielding s > 1. Since σ is a
ratio, variations in this value may be driven by changes in either γ
or λ. It is therefore often more useful to understand variations in
those parameters before considering σ.

9.4 Cost The characteristic path length of a network is closely related to its
and Efficiency topological efficiency. Communication in a network will be more
efficient when fewer connections are required to transfer informa-
tion between any two nodes; i.e., when the characteristic path
length is low. We can thus define a topological measure of the
304 Alex Fornito

global efficiency of a network as being inversely related to the char-

acteristic path length of the network [136]:

1 1
Eglob = å
N ( N - 1) i , j Lij
,

where Lij is the minimum path length between nodes i and j.

Similarly, a local measure of local communication efficiency can be
computed as the efficiency of the subgraph defined by an index
node’s neighbors (i.e., the nodes to which it directly connects), after
removal of that node [137]. It should be evident from the above
definition of Eglob that the efficiency of a network can be improved
simply adding more direct connections between nodes, since each
direct link between nodes reduces network path length. However, in
many real-world networks, there is often a cost associated with form-
ing and maintaining each connection. This is certainly true for the
brain, where axons, dendrites and synapses consume precious and
limited metabolic resources. Accordingly, minimization of wiring
costs is known to be an important pressure on brain organization,
although wiring costs in the brain are not absolutely minimized
[138]. If this were the case, brain connectivity would have a lattice-
like arrangement, in which links were only formed between spatially
adjacent nodes via short-range connections [55]. Instead the brain
forms certain, long-range and high-­cost connections that promote
integration and communication efficiency, and give rise to its small-
world organization [128, 137, 138]. Brain networks thus appear to
be configured, at least in part, to satisfy competitive pressures to
minimize cost and support efficient, integrated and complex func-
tion [54, 55, 139]. One fMRI study of healthy twins found evidence
that this trade-off between cost and efficiency is strongly heritable
[140], suggesting that it represents an important selection pressure
on brain network evolution.

9.5 Modularity The clustered connectivity of many real-world networks often

means that they can be (nearly) decomposed into subsets of nodes,
termed modules, which show higher connectivity with each other
than with other network elements (Fig. 4d). There exists a wide
variety of algorithms for decomposing networks into modules
(reviewed in [141]). These algorithms commonly attempt to find
a decomposition of the network that maximizes some quality
­function. The most commonly used function of this type is the
Newman-Girvan Q-statistic [142], defined as

1 æ ki k j ö
Q= å ç Aij -
2 m i , jÎ N è 2m
÷ d si , s j ,
ø
 Graph Theoretic Analysis of Human Brain Networks 305

where m is the total number of edges in the network, Aij = 1 if nodes

i and j are connected and zero otherwise, k is the node degree and
d si , s j = 1 if nodes i and j belong to the same module and 0 other-
ki k j
wise. The term represents the expected connectivity between
2m
nodes i and j if the positions of the edges in the graph were com-
pletely randomized. Thus, the modularity of a network is defined as
the mean difference between the actual and chance-­expected con-
nectivity between node pairs belonging to the same module.
The goal in modularity analysis is to find a decomposition that
maximizes the Q-statistic. Optimization via exhaustive search is
intractable for anything but the smallest networks, so various heu-
ristic algorithms have been proposed (reviewed in [141]; see [143]
for a comparative evaluation). Because such heuristics are used,
there may be many solutions that yield similar Q-values (i.e., the
solutions may be degenerate [144]). Consequently, multiple runs
of the algorithm should be performed to derive a consensus parti-
tion ( [145]; see also [67, 69, 122]). Furthermore, since even ran-
dom networks can show some modular structure [146], it is often
useful to compare the Q-statistic of the observed network to an
ensemble of random networks matched for the number of nodes,
edges, and degree distribution. Such an analysis allows statistical
inference on whether modularity is a significant characteristic of
the empirical network.
Modularity analysis offers a powerful tool for characterizing
topologically separable systems in the brain. Most such analyses of
MRI data typically yield 4-6 modules, in which spatially adjacent
nodes are often grouped together [125, 147, 148]. This spatial
clustering may occur because brain connectivity decays rapidly
with physical separation between nodes [149, 150]. This penalty
on long-distance connectivity is compounded by imaging tech-
niques such as diffusion MRI, which have a limited ability to recon-
struct long-range connections due to difficulty tracking trajectories
through voxels with crossing fibers [42]. Careful processing of
functional MRI data has been shown to uncover larger numbers of
spatially distributed networks that mirror those identified with
other decomposition techniques, such as ICA [59].
A common assumption is that the high interconnectivity of
nodes belonging to the same module implies some commonality of
function. Evidence in support of this hypothesis was provided by
meta-analytic work demonstrating that inter-regional coactivation
during specific types of tasks was preferentially expressed within
distinct modules [151]. This result links topological modules of
the brain to the psychological modules long thought to support
cognition [152]. Topological modularity also enables robustness
to damage while also promoting functional diversity and
306 Alex Fornito

adaptability [153, 154]: damage sustained in a modular system will

often be limited to the affected module, and the diverse function
of different modules facilitate adaptation to different environmen-
tal challenges.
A particular strength of modularity analysis is that it allows one
to characterize the role played by each node in the network with
respect to its degree of intra-modular and inter-modular connec-
tivity [155]. Intra-modular connectivity is commonly computed as
a z-score of each node’s intra-module connectivity relative to other
nodes in the same module [155]. Nodes with high intra-modular
connectivity are called “provincial hubs,” and are thought to play
an important role in functional specialization, acting as central
components of the module to which they belong (Fig. 4d). Inter-
module connectivity is commonly measured using the participa-
tion coefficient, which indexes how a node’s connectivity is
distributed across different modules [155]. Nodes with a relatively
even distribution of connectivity across different modules are called
“connector hubs” and play an important role in functional integra-
tion because they support communication between different mod-
ules (Fig. 4d). Node role analysis can be used to identify different
types of brain hubs [127] and to characterize the topological role
of brain nodes under different contexts [67, 69].
Most methods for modularity decomposition yield a hard seg-
mentation of the brain such that nodes can belong to only one
module. In reality, it is likely brain nodes can belong to more than
one functional system. Indeed, this is a defining feature of associa-
tion cortex. Algorithms for fuzzy or overlapping modular decom-
positions are available, although evaluations with respect to
benchmark networks have found their performance to be lacking
in many circumstances [156]. The modular organization of the
brain can also show a hierarchical structure, comprising modules
within modules across several scales of resolution [157], although
such multiscale organization has seldom been investigated in brain
networks (for an exception, see [158]).

9.6 Summary This section has presented a brief overview of some of the basic
graph theoretic concepts applied to neuroimaging data, and the
insights they have provided into brain network organization. Many
other metrics are available, enabling a diverse range of analyses (see
[159] for an introduction). It is important to bear in mind that
many such methods were developed in the physical or social sci-
ences with networks other than the brain in mind. As such, they
may not be directly portable to neuroscientific contexts. Indeed,
the first wave of graph theoretic studies of neuroimaging data have
concentrated on measuring canonical topological properties such
as those discussed here, but alternative measures may provide more
appropriate models of brain function. We consider some of these
issues in the next section.
 Graph Theoretic Analysis of Human Brain Networks 307

10 Issues for Consideration and Developing Trends

In this chapter, we have considered the basic methodologies

behind building and analyzing a graph theoretic model of the
human connectome with MRI data. We have also highlighted the
insights into brain organization that have been gained by this per-
spective and have drawn attention to the limitations of current
methodologies where appropriate. Some additional considerations
should be taken into account.
As already stated, many graph theoretic measures were devel-
oped for the analysis of networks other than the brain. Consequently,
not all typical graph theoretic metrics may be appropriate for the
characterization of brain networks. For example, topological mea-
sures based on shortest paths, such as the characteristic path length
and global efficiency, assume that information in the brain travels
along the shortest path between regions. In order to find the short-
est path in a network, one must have global knowledge of network
topology to find the optimal route. It is unlikely that any individual
neural element possesses such knowledge. This limitation was ele-
gantly shown in a recent study examining the relationship between
structural and functional connectivity as measured using diffusion
MRI and functional MRI, respectively [160]. Specifically, it was
shown that characteristics of the shortest structural path between
nodes strongly predicted the functional connectivity between
those regions. These characteristics indexed how easy it is to find
and/or remain on the shortest path. We should not expect such
properties to relate to functional connectivity if neuronal signaling
propagates exclusively along shortest paths (i.e., in such a scenario,
the ease with which such a path can be found should be irrelevant).
As such, alternative measures of brain network topology that
assume dynamics which more closely approximate information
transmission in the brain, such as those based on locally guided
diffusion processes, may offer useful alternative measures of net-
work communication processes [129, 160, 161]. Moreover, meth-
ods for characterizing the temporal evolution of topological
properties on non-stationary networks are being developed for
analyses of dynamic functional connectivity [162].
Another consideration in graph theoretic analysis of MRI data
specifically concerns functional connectivity networks. In such net-
works, edges are determined by some measure of statistical depen-
dence, such as the correlation coefficient. In this sense, the edges
are somewhat abstract quantities. A structural link measured with
diffusion MRI unambiguously indexes a physical connection
­
between nodes (notwithstanding the limitations of the measure-
ment technique; [88]). A functional link on the other hand, is a
statistical measure of covariation in some physiological process.
Due to the statistical nature of functional connectivity measures,
308 Alex Fornito

certain topological properties computed on brain functional net-

works may be difficult to interpret. For example, topological mea-
sures that consider indirect paths between nodes (e.g., those based
on path length) are unlikely to represent viable measures of infor-
mation exchanged between two regions, since the measured func-
tional connectivity of those regions provides a direct estimate of
their functional interaction [123]. Popular statistical measures
such as the correlation coefficient are also often signed (that is,
edge weights can be either positive or negative). Signed edges
imply a qualitatively distinct type of interaction between brain
regions [18]. This information is often ignored in imaging analy-
ses, where weights are either converted to absolute values or thres-
holded to focus only on positive weights. The adaptation of graph
theoretic measures to deal with signed weights will assist in over-
coming this problem [123].
Finally, graph theory is emerging as a useful tool for integrating
theory and experiment. Neural dynamics can be simulated on empiri-
cally derived network structures (e.g., a structural connectivity net-
work measured with diffusion MRI) to generate large-­scale models of
network functional connectivity [163]. These models allow analysis of
how simulated lesions impact network dynamics [164, 165].
Alternatively, models of disease processes can be simulated on the net-
work structure to determine whether they accurately predict empirical
patterns of disorder-related neuropathology [166, 167]. Similarly,
developmental processes can be investigated using network growth
models [168]. In these models, networks are grown by adding nodes
and edges according to specific rules. The properties of the resulting
network are compared to empirical data; a good match between
model and data implies that the growth rules represent an important
factor in brain network development. Growth models have been used
to uncover key organizational imperatives for brain networks, largely
involving ­trade-­offs between wiring costs and complex topological
properties [139, 150, 168, 169], and for modeling developmental
abnormalities in brain disorders [168] (see also [170, 171]).
Addressing the issues raised here and capitalizing on these
emerging trends will ensure more accurate modeling of brain
imaging data, while also establishing graph theory as a flexible and
powerful framework for the integration of theory and experiment
in neuroscience. This integration will be necessary to move beyond
the simple description of empirical findings to formulate and test
competing hypotheses about the underlying mechanisms that gen-
erated the observed data.
 Graph Theoretic Analysis of Human Brain Networks 309

References

1. Cherniak C (1990) The bounded brain: structural and functional systems. Nat Rev
toward quantitative neuroanatomy. J Cogn Neurosci 10:186–198
Neurosci 2:58–68 18. Fornito A, Zalesky A, Breakspear M (2013)
2. Sporns O, Tononi G, Kötter R (2005) The Graph analysis of the human connectome:
human connectome: a structural description promise, progress, and pitfalls. Neuroimage
of the human brain. PLoS Comput Biol 1:e42 80:426–444
3. Van Essen DC et al (2012) The Human 19. Euler L (1736) Solutio problematis ad geome-
Connectome Project: a data acquisition per- triam situs pertinentis. Commentarii Academiae
spective. Neuroimage 62:2222–2231 Scientiarum Imperialis Petropolitanae
4. Bohland JW et al (2009) A proposal for a 8:128–140
coordinated effort for the determination of 20. Luce RD, Perry AD (1949) A method
brainwide neuroanatomical connectivity in of matrix analysis of group structure.
model organisms at a mesoscopic scale. PLoS Psychometrika 14:95–116
Comput Biol 5:e1000334 21. Katz L (1947) On the matric analysis of
5. Kandel ER, Markram H, Matthews PM, sociometric data. Sociometry 10:233–241
Yuste R, Koch C (2013) Neuroscience thinks 22. Forsyth E, Katz L (1946) A matrix approach
big (and collaboratively). Nat Rev Neurosci to the analysis of sociometric data: prelimi-
14:659–664 nary report. Sociometry 9:340–347
6. White JG, Southgate E, Thomson JN, Brenner 23. Harary F, Norman RZ (1953) Graph theory
S (1986) The structure of the nervous sys- as a mathematical model in social science.
tem of the nematode Caenorhabditis elegans. University of Michigan Press
Philos Trans R Soc Lond B Biol Sci 314:1–340 24. Erdos P, Renyi A (1959) On random graphs.
7. Lichtman JW, Pfister H, Shavit N (2014) Publ Math Debrecen 6:290–297
The big data challenges of connectomics. Nat 25. Barabasi A, Albert R (1999) Emergence of scal-
Neurosci 17:1448–1454 ing in random networks. Science 286:509–512
8. Chiang A-S et al (2011) Three-dimensional 26. Watts DJ, Strogatz SH (1998) Collective
reconstructionof brain-wide wiring networks dynamics of ‘small-world’ networks. Nature
in Drosophila at single-cell resolution. Curr 393:440–442
Biol 21:1–11 27. Mesulam MM (1998) From sensation to cog-
9. Scannell JW, Young MP (1993) The connec- nition. Brain 121:1013–1052
tional organization of neural systems in the 28. Tononi G, Sporns O, Edelman GM (1994)
cat cerebral cortex. Curr Biol 3:191–200 A measure for brain complexity: relating
10. Shanahan M, Bingman VP, Shimizu T, functional segregation and integration in the
Gunturkun O (2013) Large-scale network nervous system. Proc Natl Acad Sci U S A
organization in the avian forebrain: a connec- 91:5033–5037
tivity matrix and theoretical analysis. Front 29. Friston KJ (2011) Functional and effec-
Comput Neurosci 7:1–17 tive connectivity: a review. Brain Connect
11. Stephan KE (2013) The history of CoCoMac. 1:13–36
NeuroImage 80:46–52 30. Felleman DJ, Van Essen DC (1991)
12. Hagmann P et al (2007) Mapping human Distributed hierarchical processing in the pri-
whole-brain structural networks with diffu- mate cerebral cortex. Cereb Cortex 1:1–47
sion MRI. PLoS One 2:e597 31. Scannell JW, Blakemore C, Young MP (1995)
13. Newman MJE (2003) The structure and Analysis of connectivity in the cat cerebral
function of complex networks. SIAM Rev cortex. J Neurosci 15:1463–1483
45:167–256 32. Hilgetag CC, Burns GA, O'Neill MA,
14. Boccaletti S, Latora V, Moreno Y, Chavez M, Scannell JW, Young MP (2000) Anatomical
Hwang DU (2006) Complex networks: struc- connectivity defines the organization of clus-
ture and dynamics. Phys Rep 424:175–308 ters of cortical areas in the macaque monkey
15. Axer M et al (2011) A novel approach to and the cat. Philos Trans R Soc Lond B Biol
the human connectome: ultra-high resolu- Sci 355:91–110
tion mapping of fiber tracts in the brain. 33. Sporns O, Tononi G, Edelman GM (2000)
Neuroimage 54:1091–1101 Theoretical neuroanatomy: relating anatomi-
16. Chung K, Deisseroth K (2013) CLARITY cal and functional connectivity in graphs and
for mapping the nervous system. Nat Meth cortical connection matrices. Cereb Cortex
10:508–513 10:127–141
17. Bullmore E, Sporns O (2009) Complex 34. Scannell JW, Burns GA, Hilgetag CC, O'Neil
brain networks: graph theoretical analysis of MA, Young MP (1999) The connectional
310 Alex Fornito

organization of the cortico-thalamic system 50. Zalesky A et al (2010) Whole-brain anatomi-

of the cat. Cereb Cortex 9:277–299 cal networks: does the choice of nodes mat-
35. Sporns O, Chialvo DR, Kaiser M, Hilgetag ter? Neuroimage 50:970–983
CC (2004) Organization, development and 51. Mountcastle VB (1997) The columnar orga-
function of complex brain networks. Trends nization of the neocortex. Brain 120(Pt
Cogn Sci 8:418–425 4):701–722
36. Stam CJ (2004) Functional connectivity pat- 52. Rademacher J, Caviness VS Jr, Steinmetz H,
terns of human magnetoencephalographic Galaburda AM (1993) Topographical varia-
recordings: a ‘small-world’ network? Neurosci tion of the human primary cortices: implica-
Lett 355:25–28 tions for neuroimaging, brain mapping, and
37. Eguiluz VM, Chialvo DR, Cecchi GA, neurobiology. Cereb Cortex 3:313–329
Baliki M, Apkarian AV (2005) Scale-free 53. Welker W (1990) 8b: Comparative structure
brain functional networks. Phys Rev Lett and evolution of cerebral cortex. In: Jones
94:018102 EG, Peters A (eds) Cerebral cortex. Plenum,
38. Salvador R, Suckling J, Schwarzbauer C, New York, pp 3–136
Bullmore E (2005) Undirected graphs of 54. Bassett DS et al (2010) Efficient physi-
frequency-­dependent functional connectivity cal embedding of topologically complex
in whole brain networks. Philos Trans R Soc information processing networks in brains
Lond B Biol Sci 360:937–946 and computer circuits. PLoS Comput Biol
39. Salvador R et al (2005) Neurophysiological 6:e1000748
architecture of functional magnetic reso- 55. Bullmore E, Sporns O (2012) The economy
nance images of human brain. Cereb Cortex of brain network organization. Nat Rev
15:1332–1342 Neurosci 13:336–349
40. Achard S, Salvador R, Whitcher B, Suckling J, 56. Passingham RE, Stephan KE, Kotter R
Bullmore E (2006) A resilient, low-frequency, (2002) The anatomical basis of functional
small-world human brain functional network ­localization in the cortex. Nat Rev Neurosci
with highly connected association cortical 3:606–616
hubs. J Neurosci 26:63–72 57. Eliasmith C et al (2012) A large-scale
41. Iturria-Medina Y et al (2007) Characterizing model of the functioning brain. Science
brain anatomical connections using diffusion 338:1202–1205
weighted MRI and graph theory. NeuroImage 58. van den Heuvel MP, Stam CJ, Boersma M,
36:645–660 Pol HEH (2008) Small-world and scale-free
42. Zalesky A, Fornito A (2009) A DTI-derived organization of voxel-based resting-state
measure of cortico-cortical connectivity. functional connectivity in the human brain.
IEEE Trans Med Imaging 28:1023–1036 Neuroimage 43:528–539
43. Skudlarski P et al (2008) Measuring brain 59. Power JD et al (2011) Functional network
connectivity: diffusion tensor imaging vali- organization of the human brain. Neuron
dates resting state temporal correlations. 72:665–678
Neuroimage 43:554–561 60. Wig GS, Schlaggar BL, Petersen SE (2011)
44. Albert R, Barabasi AL (2002) Statistical Concepts and principles in the analysis of brain
mechanics of complex networks. Rev Mod networks. Ann N Y Acad Sci 1224:126–146
Phys 74:47–97 61. Tzourio-Mazoyer N et al (2002) Automated
45. Logothetis NK (2008) What we can do anatomical labeling of activations in SPM
and what we cannot do with fMRI. Nature using a macroscopic anatomical parcella-
453:869–878 tion of the MNI MRI single-subject brain.
46. Butts CT (2009) Revisiting the foundations NeuroImage 15:273–289
of network analysis. Science 325:414–416 62. Desikan RS et al (2006) An automated label-
47. Smith SM et al (2011) Network modelling ing system for subdividing the human cere-
methods for FMRI. Neuroimage 54:875–891 bral cortex on MRI scans into gyral based
48. Fornito A, Zalesky A, Bullmore ET (2010) regions of interest. NeuroImage 31:968–980
Network scaling effects in graph analytic stud- 63. Salvador R et al (2008) A simple view of the
ies of human resting-state FMRI data. Front brain through a frequency-specific functional
Syst Neurosci 4:22 connectivity measure. NeuroImage
49. Hayasaka S, Laurienti PJ (2010) Comparison 39:279–289
of characteristics between region-and voxel-­ 64. Dosenbach NU et al (2010) Prediction of
based network analyses in resting-state fMRI individual brain maturity using fMRI. Science
data. Neuroimage 50:499–508 329:1358–1361
 Graph Theoretic Analysis of Human Brain Networks 311

65. Fair DA et al (2007) Development of distinct spectral clustering. Hum Brain Mapp
control networks through segregation and 33:1914–1928
integration. Proc Natl Acad Sci U S A 79. Johansen-Berg H et al (2004) Changes in
104:13507–13512 connectivity profiles define functionally dis-
66. Andrews-Hanna JR, Reidler JS, Sepulcre J, tinct regions in human medial frontal cortex.
Poulin R, Buckner RL (2010) Functional-­ Proc Natl Acad Sci U S A 101:13335–13340
anatomic fractionation of the brain’s default 80. Behrens TE et al (2003) Non-invasive map-
network. Neuron 65:550–562 ping of connections between human thalamus
67. Dwyer DB et al (2014) Large-scale brain net- and cortex using diffusion imaging. Nat
work dynamics supporting adolescent cogni- Neurosci 6:750–757
tive control. J Neurosci 34:14096–14107 81. Anwander A, Tittgemeyer M, von Cramon
68. Cocchi L et al (2014) Complexity in rela- DY, Friederici AD, Knosche TR (2007)
tional processing predicts changes in func- Connectivity-based parcellation of Broca's
tional brain network dynamics. Cereb Cortex area. Cereb Cortex 17:816–825
24:2283–2296 82. Glasser MF, Van Essen DC (2011) Mapping
69. Fornito A, Harrison BJ, Zalesky A, Simons JS human cortical areas in vivo based on myelin
(2012) Competitive and cooperative dynam- content as revealed by T1- and T2-weighted
ics of large-scale brain functional networks MRI. J Neurosci 31:11597–11616
supporting recollection. Proc Natl Acad Sci 83. Eickhoff SB et al (2005) A new SPM toolbox
U S A 109:12788–12793 for combining probabilistic cytoarchitectonic
70. Beckmann CF, DeLuca M, Devlin JT, Smith maps and functional imaging data.
SM (2005) Investigations into resting-state NeuroImage 25:1325–1335
connectivity using independent component 84. Zilles K et al (2002) Architectonics of the
analysis. Philos Trans R Soc Lond B Biol Sci human cerebral cortex and transmitter recep-
360:1001–1013 tor fingerprints: reconciling functional neuro-
71. Calhoun VD, Adali T, Pekar JJ (2004) A anatomy and neurochemistry. Eur
method for comparing group fMRI data Neuropsychopharmacol 12:587–599
using independent component analysis: appli- 85. Alexander-Bloch A, Giedd JN, Bullmore E
cation to visual, motor and visuomotor tasks. (2013) Imaging structural co-variance
Magn Reson Imaging 22:1181–1191 between human brain regions. Nat Rev
72. Smith SM et al (2009) Correspondence of the Neurosci 14:322–336
brain's functional architecture during activa- 86. Lerch JP et al (2006) Mapping anatomical
tion and rest. Proc Natl Acad Sci U S A correlations across cerebral cortex
106:13040–13045 (MACACC) using cortical thickness from
73. Yu Q et al (2011) Altered topological proper- MRI. NeuroImage 31:993–1003
ties of functional network connectivity in 87. Bastiani M, Shah NJ, Goebel R, Roebroeck A
schizophrenia during resting state: a small-­ (2012) Human cortical connectome recon-
world brain network study. PLoS One 6:e25423 struction from diffusion weighted MRI: the
74. Kiviniemi V et al (2009) Functional segmen- effect of tractography algorithm. Neuroimage
tation of the brain cortex using high model 62:1732–1749
order group PICA. Hum Brain Mapp 88. Jones DK, Knösche TR, Turner R (2013)
30:3865–3886 White matter integrity, fiber count, and other
75. Nelson SM et al (2010) A parcellation scheme fallacies: the do‘s and dont’s of diffusion
for human left lateral parietal cortex. Neuron MRI. Neuroimage 73:239–254
67:156–170 89. van den Heuvel MP, Mandl RCW, Stam CJ,
76. Cohen AL et al (2008) Defining functional Kahn RS, Hulshoff Pol HE (2010) Aberrant
areas in individual human brains using resting frontal and temporal complex network struc-
functional connectivity MRI. NeuroImage ture in schizophrenia: a graph theoretical
41:45–57 analysis. J Neurosci 30:15915–15926
77. Yeo BT et al (2011) The organization of the 90. Alexander DC et al (2010) Orientationally
human cerebral cortex estimated by intrinsic invariant indices of axon diameter and density
functional connectivity. J Neurophysiol from diffusion MRI. Neuroimage
106:1125–1165 52:1374–1389
78. Craddock RC, James GA, Holtzheimer PE, 91. Friston KJ (1994) Functional and effective
Hu XP, Mayberg HS (2012) A whole brain connectivity in neuroimaging: a synthesis.
fMRI atlas generated via spatially constrained Hum Brain Mapping 2:56–78
312 Alex Fornito

92. Vincent JL et al (2007) Intrinsic functional 109. Fornito A, Zalesky A, Pantelis C, Bullmore
architecture in the anaesthetized monkey ET (2012) Schizophrenia, neuroimaging and
brain. Nature 447:83–86 connectomics. Neuroimage 62:2296–2314
93. Honey CJ et al (2009) Predicting human 110. van Wijk BCM, Stam CJ, Daffertshofer A
resting-state functional connectivity from (2010) Comparing brain networks of differ-
structural connectivity. Proc Natl Acad Sci ent size and connectivity density using graph
U S A 106:2035–2040 theory. Plos One 5:e13701
94. Zalesky A, Fornito A, Bullmore E (2012) On 111. Irimia A, Chambers MC, Torgerson CM,
the use of correlation as a measure of network Van Horn JD (2012) Circular representation
connectivity. Neuroimage 60:2096–2106 of human cortical networks for subject and
95. Fox MD, Raichle ME (2007) Spontaneous population-­ level connectomic visualization.
fluctuations in brain activity observed with Neuroimage 60:1340–1351
functional magnetic resonance imaging. Nat 112. Fornito A, Bullmore ET (2015) Connectomics:
Rev Neurosci 8:700–711 a new paradigm for understanding brain
96. Fornito A, Bullmore ET (2010) What disease. Eur Neuropsychopharmacol. 25:
can spontaneous fluctuations of the blood 733–748
oxygenation-­ level-dependent signal tell 113. Fornito A et al (2013) Functional dyscon-
us about psychiatric disorders? Curr Opin nectivity of corticostriatal circuitry as a risk
Psychiatry 23:239–249 phenotype for psychosis. JAMA Psychiatry
97. Zalesky A, Fornito A, Cocchi L, Gollo LL, 70:1143–1151
Breakspear M (2014) Time-resolved resting-­ 114. Meskaldji DE et al (2011) Adaptive strategy
state brain networks. Proc Natl Acad Sci U S A for the statistical analysis of connectomes.
111:10341–10346 Plos One 6:e23009
98. Smith SM et al (2012) Temporally-­ 115. Ginestet CE, Simmons A (2011) Statistical
independent functional modes of spontane- parametric network analysis of functional
ous brain activity. Proc Natl Acad Sci U S A connectivity dynamics during a working
­
109:3131–3136 memory task. Neuroimage 55:688–704
99. Hutchison RM et al (2013) Dynamic func- 116. Zalesky A, Fornito A, Bullmore ET (2010)
tional connectivity: promise, issues, and inter- Network-based statistic: Identifying differences
pretations. NeuroImage 80:360–378 in brain networks. Neuroimage 53:1197–1207
100. Feinberg DA et al (2010) Multiplexed echo pla- 117. Nichols TE, Holmes AP (2002)
nar imaging for sub-second whole brain FMRI Nonparametric permutation tests for func-
and fast diffusion imaging. PLoS One 5:e15710 tional neuroimaging: a primer with examples.
101. Rissman J, Gazzaley A, D'Esposito M (2004) Hum Brain Mapp 15:1–25
Measuring functional connectivity during dis- 118. Benjamini Y, Hochberg Y (1995) Controlling
tinct stages of a cognitive task. NeuroImage the false discovery rate: a practical and power-
23:752–763 ful approach to multiple testing. J R Stat Soc
102. Fornito A, Yoon J, Zalesky A, Bullmore ET, Ser B 57:289–300
Carter CS (2011) General and specific func- 119. Zalesky A, Cocchi L, Fornito A, Murray MM,
tional connectivity disturbances in first-­episode Bullmore E (2012) Connectivity differences in
schizophrenia during cognitive control perfor- brain networks. Neuroimage 60:1055–1062
mance. Biol Psychiatry 70:64–72 120. Tomasi D, Volkow VD (2010) Functional
103. Friston KJ et al (1997) Psychophysiological connectivity density mapping. Proc Natl Acad
and modulatory interactions in neuroimag- Sci U S A, 107:9885–9890
ing. NeuroImage 6:218–229 121. Cole MW, Anticevic A, Repovs G, Barch D
104. Cole MW et al (2013) Multi-task connectivity (2011) Variable global dysconnectivity and
reveals flexible hubs for adaptive task control. individual differences in schizophrenia. Biol
Nat Neurosci 16:1348–1355 Psychiatry 70:43–50
105. Friston K, Moran R, Seth AK (2013) 122. Rubinov M, Sporns O (2011) Weight-­conserving
Analysing connectivity with Granger causal- characterization of complex functional brain
ity and dynamic causal modelling. Curr Opin networks. Neuroimage 56:2068–2079
Neurobiol 23:172–178 123. Rubinov M, Sporns O (2010) Complex net-
106. Friston KJ, Harrison L, Penny W (2003) Dynamic work measures of brain connectivity: uses and
causal modelling. NeuroImage 19:1273–1302 interpretations. Neuroimage 52:1059–1069
107. Seghier ML, Friston KJ (2013) Network 124. Amaral LA, Scala A, Barthelemy M, Stanley
discovery with large DCMs. Neuroimage HE (2000) Classes of small-world networks.
68:181–191 Proc Natl Acad Sci U S A 97:11149–11152
108. Friston KJ, Kahan J, Biswal B, Razi A (2014) 125. van den Heuvel MP, Sporns O (2011) Rich-­
A DCM for resting state fMRI. Neuroimage club organization of the human connectome.
94:396–407 J Neurosci 31:15775–15786
 Graph Theoretic Analysis of Human Brain Networks 313

126. Buckner RL et al (2009) Cortical hubs revealed 143. Lancichinetti A, Fortunato S (2009)
by intrinsic functional connectivity: map- Community detection algorithms: a compara-
ping, assessment of stability, and relation to tive analysis. Phys Rev E 80:056117
Alzheimer’s Disease. J Neurosci 29:1860–1873 144. Good BH, de Montjoye YA, Clauset A (2010)
127. Power JD, Schlaggar BL, Lessov-Schlaggar Performance of modularity maximization in
CN, Petersen SE (2013) Evidence for hubs practical contexts. Phys Rev E 81:046106
in human functional brain networks. Neuron 145. Lancichinetti A, Fortunato S (2012) Consensus
79:798–813 clustering in complex networks. Sci Rep 2:336
128. van den Heuvel MP, Kahn RS, Goni J, Sporns 146. Guimerà R, Sales-Pardo M, Amaral L (2004)
O (2012) High-cost, high-capacity backbone Modularity from fluctuations in random
for global brain communication. Proc Natl graphs and complex networks. Phys Rev E
Acad Sci U S A 109:11372–11377 70:025101
129. Mišić B, Sporns O, McIntosh AR (2014) 147. Hagmann P et al (2008) Mapping the struc-
Communication efficiency and congestion tural core of human cerebral cortex. PLoS
of signal traffic in large-scale brain networks. Biol 6:e159
PLoS Comput Biol 10:e1003427 148. Meunier D, Achard S, Morcom A, Bullmore
130. van den Heuvel MP, Sporns O (2013) An ana- E (2009) Age-related changes in modular
tomical substrate for integration among func- organization of human brain functional net-
tional networks in human cortex. J Neurosci works. NeuroImage 44:715–723
33:14489–14500 149. Buzsaki G, Geisler C, Henze DA, Wang
131. Albert R, Jeong H, Barabasi AL (2000) Error XJ (2004) Interneuron diversity series: cir-
and attack tolerance of complex networks. cuit complexity and axon wiring economy
Nature 406:378–382 of cortical interneurons. Trends Neurosci
132. Fornito A, Breakspear M, Zalesky A (2015) 27:186–193
The connectomics of brain disorders. Nat Rev 150. Ercsey-Ravasz M et al (2013) A predictive
Neurosci 16:159–172 network model of cerebral cortical con-
133. Crossley NA et al (2014) The hubs of the nectivity based on a distance rule. Neuron
human connectome are generally implicated 80:184–197
in the anatomy of brain disorders. Brain 151. Crossley NA, Mechelli A, Vertes PE (2013)
137:2382–2395 Cognitive relevance of the community struc-
134. Maslov S, Sneppen K (2002) Specificity and ture of the human brain functional coacti-
stability in topology of protein networks. vation network. Proc Natl Acad Sci U S A
Science 296:910–913 110:11583–11588
135. Humphries MD, Gurney K, Prescott TJ 152. Fodor JA (1983) Modularity of mind: an
(2006) The brainstem reticular formation is essay on faculty psychology. MIT Press
a small-world, not scale-free, network. Proc 153. Simon HA (1962) The architecture of com-
Biol Sci 273:503–511 plexity. Proc Am Philos Soc 106:467–482
136. Latora V, Marchiori M (2001) Efficient 154. Kitano H (2004) Biological robustness. Nat
behavior of small-world networks. Phys Rev Rev Genet 5:826–837
Lett 87:198701 155. Guimera R, Nunes Amaral LA (2005)
137. Latora V, Marchiori M (2003) Economic Functional cartography of complex metabolic
small-world behavior in weighted networks. networks. Nature 433:895–900
Eur Phys J B 32:249–263 156. Xie J, Kelley S, Szymanski BK (2013)
138. Kaiser M, Hilgetag CC (2006) Nonoptimal Overlapping community detection in net-
component placement, but short processing works. ACM Comput Surv 45:1–35
paths, due to long-distance projections in 157. Meunier D, Lambiotte R, Bullmore ET (2011)
neural systems. PLoS Comput Biol 2:e95 Modular and hierarchically modular organiza-
139. Chen Y, Wang S, Hilgetag CC, Zhou C tion of brain networks. Front Neurosci 4:200
(2013) Trade-off between multiple con- 158. Meunier D, Lambiotte R, Fornito A, Ersche
straints enables simultaneous formation of KD, Bullmore ET (2009) Hierarchical mod-
modules and hubs in neural systems. PLoS ularity in human brain functional networks.
Comput Biol 9:e1002937 Front Neuroinform 3:37
140. Fornito A et al (2011) Genetic influences on 159. Newman MEJ (2010) Networks. a Introduction.
cost-efficient organization of human cortical Oxford University Press
functional networks. J Neurosci 31:3261–3270 160. Goñi J et al (2014) Resting-brain functional
141. Fortunato S (2010) Community detection in connectivity predicted by analytic measures of
graphs. Phys Rep 486:75–174 network communication. Proc Natl Acad Sci
142. Newman M, Girvan M (2004) Finding and U S A 111:833–838
evaluating community structure in networks. 161. Betzel RF et al (2014) Multi-scale com-
Phys Rev 69:026113 munity organization of the human struc-
314 Alex Fornito

tural connectome and its relationship with 167. de Haan W, Mott K, van Straaten ECW,
resting-state functional connectivity. Net Sci Scheltens P, Stam CJ (2012) Activity depen-
1:353–373 dent degeneration explains hub vulnerability
162. Bassett DS et al (2011) Dynamic reconfigura- in Alzheimer's disease. PLoS Comput Biol
tion of human brain networks during learning. 8:e1002582
Proc Natl Acad Sci U S A 108:7641–7646 168. Vertes PE et al (2012) Simple models of
163. Deco G, Jirsa VK, McIntosh AR (2011) human brain functional networks. Proc Natl
Emerging concepts for the dynamical orga- Acad Sci U S A 109:5868–5873
nization of resting-state activity in the brain. 169. Song HF, Kennedy H, Wang X-J (2014)
Nat Publ Group 12:43–56 Spatial embedding of structural similarity in
164. Alstott J, Breakspear M, Hagmann P, the cerebral cortex. Proc Natl Acad Sci U S A
Cammoun L, Sporns O (2009) Modeling the 111:16580–16585
impact of lesions in the human brain. PLoS 170. Goni J et al (2013) Exploring the morpho-
Comput Biol 5:e1000408 space of communication efficiency in complex
165. Honey CJ, Sporns O (2008) Dynamical con- networks. PLoS One 8, e58070
sequences of lesions in cortical networks. 171. Avena-Koenigsberger A et al (2014) Using
Hum Brain Mapp 29:802–809 Pareto optimality to explore the topol-
166. Raj A, Kuceyeski A, Weiner M (2012) A net- ogy and dynamics of the human connec-
work diffusion model of disease progression tome. Philos Trans R Soc Lond B Biol Sci
in dementia. Neuron 73:1204–1215 369:20130530
 Part II

fMRI Application to Measure Brain Function

 Chapter 11

Functional MRI: Applications in Cognitive Neuroscience

Mark D’Esposito, Andrew Kayser, and Anthony Chen

Abstract
Neuroimaging, in many respects, revolutionized the study of cognitive neuroscience, the discipline that
attempts to determine the neural mechanisms underlying cognitive processes. Early studies of brain–
behavior relationships relied on a precise neurological exam as the basis for hypothesizing the site of brain
damage that was responsible for a given behavioral syndrome. The advent of structural brain imaging, first
with computerized tomography and later with magnetic resonance imaging, paved the way for more pre-
cise anatomical localization of the cognitive deficits that manifest after brain injury. Functional neuroimag-
ing, broadly defined as techniques that provide measures of brain activity, further increased our ability to
study the neural basis of behavior. Functional MRI (fMRI), in particular, is an extremely powerful tech-
nique that affords excellent spatial and temporal resolution. This chapter focuses on the principles underly-
ing fMRI as a cognitive neuroscience tool for exploring brain–behavior relationships.

Key words Functional MRI, Cognitive neuroscience, Experimental design, Statistics

1 Introduction

Cognitive neuroscience is a discipline that attempts to determine

the neural mechanisms underlying cognitive processes. Specifically,
cognitive neuroscientists test hypotheses about brain–behavior
relationships that can be organized along two conceptual domains:
functional specialization—the idea that functional modules exist
within the brain, that is, areas of the cerebral cortex that are spe-
cialized for a specific cognitive process, and functional integra-
tion—the idea that a cognitive process can be an emergent property
of interactions among a network of brain regions, which suggests
that a brain region can play a different role across many functions.
Early investigations of brain–behavior relationships consisted
of careful observation of individuals with neurological injury
resulting in focal brain damage. The idea of functional specializa-
tion evolved from hypotheses that damage to a particular brain
region was responsible for a given behavioral syndrome that was
characterized by a precise neurological examination. For instance,

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_11, © Springer Science+Business Media New York 2016

317
318 Mark D’Esposito et al.

the association of aphasia with right-sided limb weakness implicated

the left hemisphere as the site of language abilities. Moreover,
upon the death of a patient with a neurological disorder, clinico-
pathological correlations provided confirmatory information about
the site of damage causing a specific neurobehavioral syndrome
such as aphasia. For example, in 1861 Paul Broca’s observations of
nonfluent aphasia in the setting of a damaged left inferior frontal
gyrus (IFG) cemented the belief that this brain region was critical
for speech output [1]. The introduction of structural brain imag-
ing more than 100 years after Broca’s observations, first with com-
puterized tomography (CT) and later with magnetic resonance
imaging (MRI), paved the way for more precise anatomical local-
ization in the living patient of the cognitive deficits that develop
after brain injury. The superb spatial resolution of structural neu-
roimaging has reduced the reliance on the infrequently obtained
autopsy for making brain–behavior correlations.
Functional neuroimaging, broadly defined as techniques that
measure brain activity, expanded our ability to study the neural
basis of cognitive processes. One such method, fMRI is as an
extremely powerful technique that affords excellent spatial and
temporal resolution. Measuring regional brain activity in healthy
subjects while they perform cognitive tasks links localized brain
activity with specific behaviors. For example, functional neuroim-
aging studies have demonstrated that the left IFG is consistently
activated during the performance of speech production tasks in
healthy individuals [2]. Such findings from functional neuroimag-
ing are complementary to findings derived from observations of
patients with focal brain damage. This chapter focuses on the prin-
ciples underlying fMRI as a cognitive neuroscience tool for explor-
ing brain–behavior relationships.

2 Inference in Functional Neuroimaging Studies of Cognitive Processes

Insight regarding the link between brain and behavior can be gained
through a variety of approaches. It is unlikely that any single neuro-
science method is sufficient to fully investigate any particular ques-
tion regarding the mechanisms underlying cognitive function.
From a methodological point of view, each method will offer differ-
ent temporal and spatial resolution. From a conceptual point of
view, each method will provide data that will support different types
of inferences that can be drawn from it. Thus, data obtained address-
ing a single question but derived from multiple methods can pro-
vide more comprehensive and inferentially sound conclusions.
Functional neuroimaging studies support inferences about the
association of a particular brain system with a cognitive process.
However, it is difficult to prove in such a study that the observed
activity is necessary for an isolated cognitive process because perfect
 Functional MRI: Applications in Cognitive Neuroscience 319

control over a subject’s cognitive processes during a functional

neuroimaging experiment is never possible. Even if the task per-
formed by a subject is well designed, it is difficult to demonstrate
conclusively that he or she is differentially engaging a single, identi-
fied cognitive process. The subject may engage in unwanted cogni-
tive processes that either have no overt, measurable effects or are
perfectly confounded with the process of interest. Consequently, the
neural activity measured by the functional neuroimaging technique
may result from some confounding neural computation that is itself
not necessary for executing the cognitive process seemingly under
study. In other words, functional neuroimaging is an observational,
correlative method [3]. It is important to note that the inferences
that can be drawn from functional neuroimaging studies such as
fMRI apply to all methods of physiological measurement (e.g., elec-
troencephalography, EEG, or magnetoencephalography, MEG).
The inference of necessity cannot be made without showing
that a focal brain lesion disrupts the cognitive process in question.
However, unlike precise surgical or neurotoxic lesions in animal
models, lesions in patients are often extensive, damaging local neu-
rons and “fibers of passage.” For example, damage to prominent
white matter tracts can cause cognitive deficits similar to those pro-
duced by cortical lesions, such as the amnesia resulting from lesions
of the fornix, the main white matter pathway projecting from the
hippocampus [4]. In addition, connections from region “A” may
support the continued metabolic function of region “B,” but region
A may not be computationally involved in certain processes under-
taken by region B. Thus, damage to region A could impair the
function of region B via two possible mechanisms: (1) diaschisis [5,
6] and (2) retrograde trans-synaptic degeneration. Consequently,
studies of patients with focal lesions cannot conclusively demon-
strate that the neurons within a specific region are themselves criti-
cal to the computational support of an impaired cognitive process.
Empirical studies using lesion and electrophysiological meth-
ods demonstrate these issues regarding the types of inferences that
can be logically drawn from them. For example, in monkeys,
single-unit recording reveals neurons in the lateral prefrontal cor-
tex (PFC) that increase their firing during the delay between the
presentation of information to be remembered and a few seconds
later when that information must be recalled [7, 8]. These studies
are taken as evidence that persistent neural activity in the PFC is
involved in temporary storage of information, a cognitive process
known as working memory. The necessity of PFC for working
memory was demonstrated in other monkey studies showing that
PFC lesions impair performance on working memory tasks, but
not on tasks that do not require temporarily holding information
in memory [9]. Persistent neural activity during working memory
tasks is also found in the hippocampus [10, 11]. Hippocampal
lesions, however, do not impair performance on most working
320 Mark D’Esposito et al.

memory tasks [12], which suggests that the hippocampus is

involved in maintaining information over short periods of time, but
is not necessary for this cognitive operation. Observations in
humans support this notion. For example, the well-studied patient
H.M., with complete bilateral hippocampal damage and the severe
inability to learn new information, could nevertheless perform
normally on working memory tasks such as digit span [13]. The
hippocampus is implicated in long-term memory especially when
relations between multiple items and multiple features of a com-
plex, novel item must be retained. Thus, the hippocampus may
only be engaged during working memory tasks that require some-
one to subsequently remember novel information [14].
When the results from lesion and functional neuroimaging
studies are combined, a stronger level of inference emerges [15].
As in the examples of Broca’s aphasia or working memory, a lesion
of a specific brain region causes impairment of a given cognitive
process and when engaged by an intact individual, that cognitive
process evokes neural activity in the same brain region. Given these
findings, the inference that this brain region is computationally
necessary for the cognitive process is stronger than the data derived
from each study performed in isolation. Thus, lesion and func-
tional neuroimaging studies are complementary, each providing
inferential support that the other lacks.
Other types of inferential failure can occur in the interpretation
of functional neuroimaging studies when other common assump-
tions do not hold true. First, it is assumed that if a cognitive pro-
cess activates a particular brain region (evoked by a particular task),
the neural activity in that brain region must depend on engaging
that particular cognitive process. For example, a brain region show-
ing greater activation during the presentation of faces than to other
types of stimuli, such as photographs of cars or buildings, is consid-
ered to engage face perception processes. However, this region
may also support other higher-level cognitive processes such as
memory processes, in addition to lower level perceptual processes
[16]. See ref. [17] for a further discussion of this issue.
The opposite type of inference is made when it is assumed
that if a particular brain region is activated during the perfor-
mance of a cognitive task, the subject must have engaged the
cognitive process supported by that region during the task
(referred to as a “reverse inference”). For example, when activa-
tion of the frontal lobes was observed during a mental rotation
task, it was proposed that subjects engaged working memory
processes to recall the identity of the rotated target [18]. (They
derived this assumption from other imaging studies showing acti-
vation of the frontal lobes during working memory tasks.)
However, in this example, because some other cognitive process
supported by the frontal lobes could have activated this region
[19], one cannot be sure that working memory was engaged
 Functional MRI: Applications in Cognitive Neuroscience 321

leading to the activation of the frontal lobes. Unfortunately, this

potentially faulty logic is a fairly common practice in fMRI stud-
ies. See ref. [20] for a further discussion of this issue.
In summary, interpretation of the results of functional neuro-
imaging studies attempting to link brain and behavior rests on
numerous assumptions. Familiarity with the types of inferences
that can and cannot be drawn from these studies is helpful for
assessing the validity of the findings reported by such studies.

3 Functional MRI as a Cognitive Neuroscience Tool

Functional MRI has become the predominant functional neuroim-

aging method for studying the neural basis of cognitive processes
in humans. Compared to its predecessor, positron emission tomog-
raphy (PET) scanning, fMRI offers many advantages. For example,
MRI scanners are much more widely available, and imaging costs
are less expensive since MRI does not require a cyclotron to pro-
duce radioisotopes. MRI is also a noninvasive procedure since
there is no requirement for injection of a radioisotope into the
bloodstream. Also, given the half-life of available radioisotopes,
PET scanning is unable to provide comparable temporal resolution
to that of fMRI which can provide images of behavioral events
occurring on the order of seconds rather than the summation of
many behavioral events over tens of seconds.
In selected circumstances, however, PET scanning can provide
an advantage over fMRI for studying certain questions concerning
the neural basis of cognition. For example, a particular advantage
of PET scanning in the study of cognition that can nicely comple-
ment fMRI studies is its ability to assess neurochemical (neu-
rotransmitter and neuromodulator) systems. Radioactively labeled
ligands may be used to directly measure density and distribution of
particular receptors and even receptor subtypes, distribution of
presynaptic terminals or enzymes involved in the production or
breakdown of particular neurochemicals [21]. For example, one
study measured dopamine synthesis capacity in the striatum with
PET and used fMRI to measure brain activity during a working
memory task. It was found that activity in frontal cortex during the
working memory task was related to caudate dopamine levels as
well as task accuracy. Thus, combining PET and fMRI data in this
unique way allowed the investigators to test a question regarding
the neurochemical basis of cognition [22].
The MRI scanner, compared to a behavioral testing room, is
less than ideal for performing most cognitive neuroscience experi-
ments. Experiments are performed in the awkward position of lying
on one’s back, often requiring subjects to visualize the presentation
of stimuli through a mirror, in an acoustically noisy environment.
Moreover, most individuals develop some degree of claustrophobia
322 Mark D’Esposito et al.

due to the small bore of the MRI scanner and find it difficult to
remain completely motionless for a long duration of time that is
required for most experiments (e.g., usually 60–90 min). These
constraints of the MRI scanner make it especially difficult to scan
children or certain patient populations (e.g., Parkinson’s disease
patients), which has resulted in many fewer fMRI studies involving
children than adults and neurological patients in general. However,
mock scanners have been built in many imaging centers, with
motion devices that acclimate children to the scanner environment
before they participate in an fMRI study. This approach has led to
an increasing number of fMRI studies of children being reported in
the literature that are providing tremendous insight regarding the
mechanisms underlying the developing brain (for review, see [23]).
All sensory systems have been investigated with fMRI includ-
ing the visual, auditory, somatosensory, olfactory, and gustatory
systems. Each system requires different technologies for successful
presentation of relevant stimuli within an MRI environment. At
the time of this writing, there are now many off-the-shelf commer-
cial products that exist that are MRI-compatible. Acquiring ancil-
lary electrophysiological data such as electromyographic recordings
to measure muscle contraction or electrodermal responses to mea-
sure autonomic activity enhances many cognitive neuroscience
experiments. Devices have been developed that are MR compatible
for these types of measurements as well as other physiological mea-
sures such as heart rate, electrocardiography, oxygen saturation,
and respiratory rate. The recording of eye movements is common-
place in MRI scanners predominantly with the use of infrared video
cameras equipped with long range optics. Video images of the
pupil–corneal reflection can be sampled at 500–1000 Hz allowing
for the accurate (<0.5°) localization of gaze within 50 horizontal
and 40 vertical degrees of visual angle.
EEG recordings have also been successfully performed during
MRI scanning [24, 25]. Both measures of event-related potentials
(ERPs) and spectral EEG power in specific frequency bands and
have been successfully recorded and related to variations in under-
lying BOLD activity and behavior [26–29]. However, the record-
ing of low amplitude EEG events, such as ERPs and transient
changes in spectral EEG power, can be more difficult in a magnetic
field due to large artifacts induced by gradient switching and head
movement and voltage changes from cardiac pulsation. The
optimization of data acquisition methods and post-processing
algorithms to remove artifacts have allowed for reliable measure-
ments of ERPs and transient EEG events during fMRI scanning
[30–33]. In summary, most challenges facing cognitive experi-
ments and the study of spontaneous activity within the MRI envi-
ronment have been overcome, creating an environment that is
comparable to standard psychophysical testing labs outside of a
scanner. Recent work has focused on minimizing exacerbated EEG
 Functional MRI: Applications in Cognitive Neuroscience 323

artifacts present during high-field MRI scanning [34]. Although

individual laboratories have achieved most of these advancements,
MRI scanners originally designed for clinical use by manufacturers
are now being designed with consideration of many of these
research-related issues.
Another promising technique is the delivery of transcranial
magnetic stimulation (TMS) during MRI scanning [35, 36]. TMS
induces depolarization of neurons under the coil and, when com-
bined with functional MRI, can be used to reveal patterns of remote
connectivity, such as between the frontal eye field (FEF) and early
visual cortex [36], the lateral prefrontal cortex and face- and house-
selective regions in temporal cortex [35], and within and between
large-scale brain networks [37]. There are many challenges in com-
bining TMS and MRI such as the need for a large MRI head coil to
accommodate the presence of the TMS coil, the difficulty of precise
localization [38], and the increased subject discomfort. However,
perhaps the largest challenge of delivering TMS in a manner that
does not lead to artifacts in the MRI signal has been largely over-
come by new commercially available TMS coils

3.1 Temporal Two types of temporal resolution need to be considered for cogni-
Resolution tive neuroscience experiments. First, what is the briefest neural
event that can be detected as an fMRI signal? Second, how close
together can two neural events occur and be resolved as separable
fMRI signals?
The time scale on which neural changes occur is quite rapid. For
example, neural activity in the lateral intraparietal area of monkeys
increases within 100 ms of the visual presentation of a saccade target
[39]. In contrast, the fMRI signal gradually increases to its peak
magnitude within 4–6 s after an experimentally induced brief (<1 s)
change in neural activity, and then decays back to baseline after sev-
eral more seconds [40–42]. This slow time course of fMRI signal
change in response to such a brief increase in neural activity is infor-
mally referred to as the blood oxygen level-dependent (BOLD)
fMRI hemodynamic response or simply, the hemodynamic response
(Fig. 1). Thus, neural dynamics and neurally evoked hemodynamics,
as measured with fMRI, are on quite different time scales.
The sluggishness of the hemodynamic response limits the tem-
poral resolution of the fMRI signal to hundreds of milliseconds to
seconds as opposed to the millisecond temporal resolution of elec-
trophysiological recordings of neural activity, such as from single-
unit recording in monkeys and EEG or MEG in humans. However,
it has been clearly demonstrated that brief changes in neural activ-
ity can be detected with reasonable statistical power using
fMRI. For example, appreciable fMRI signal can be observed in
sensorimotor cortex in association with single finger movements
[43] and in visual cortex during very briefly presented (34 ms)
visual stimuli [44]. In contrast, the temporal resolution of fMRI
324 Mark D’Esposito et al.

proportion max evoked signal

0.75

0.5

0.25

-0.25
0 2 4 6 8 10 12 14
time (secs)

Fig. 1 A typical hemodynamic response (i.e., fMRI signal change in response to a

brief increase of neural activity) from the primary sensorimotor cortex. The fMRI
signal peaked approximately 5 s after the onset of the motor response (at time
zero)

limits the detection of sequential changes in neural activity that

occur rapidly with respect to the hemodynamic response. That is,
the ability to resolve the changes in the fMRI signal associated with
two neural events often requires the separation of those events by
a relatively long period of time compared with the width of the
hemodynamic response. This is because two neural events closely
spaced in time will produce a hemodynamic response that reflects
the accumulation from both neural events, making it difficult to
estimate the contribution of each individual neural event. In gen-
eral, evoked fMRI responses to discrete neural events separated by
at least 4 s appear to be within the range of resolution [45].
However, provided that the stimuli are presented randomly, sig-
nificant differential functional responses between two events (e.g.,
flashing visual stimuli) spaced as closely as 500 ms apart can be
detected [46–48]. The effect of fixed and randomized intertrial
intervals on the BOLD signal is illustrated in Fig. 2.
In some tasks, the order of individual trial events cannot be
randomized. For example, in certain types of working memory
tasks, the presentation of the information to be remembered dur-
ing the delay period, and the period when the subject must recall
the information, are individual trial events whose order cannot be
randomized. In these types of tasks, short time scales (<4 s) cannot
be temporally resolved. These temporal resolution issues in fMRI
have been extensively considered regarding their impact on experi-
mental design [49, 50].
 Functional MRI: Applications in Cognitive Neuroscience 325

Fig. 2 Effect of fixed vs. randomized intertrial intervals on the blood oxygen level-
dependent (BOLD) fMRI signal [46]

3.2 Spatial As approaches are sought that maximize both BOLD signal
Resolution strength and in-plane resolution, fMRI studies in humans have
recently been extended to higher magnetic field strengths (7.0 T
and 9.4 T) [51–53]. Such studies have the power to potentially
evaluate much finer cortical details, such as the representation of
individual fingertips within primary somatosensory cortex [51].
However, as the field strength increases, factors that are less conse-
quential at 3.0 T—including magnetic field inhomogeneities [52]
and the contribution of macrovascular structures to the typical
gradient-echo signal [53]—become significantly more problematic,
requiring further innovations in pulse sequence development.
Single-echo gradient-echo sequences using echo times (TE) that
exceed the repetition time (TR), for example, take advantage of
reduced distortion relative to single-shot gradient-echo sequences,
while also avoiding the prolonged acquisition times of typical
single-echo sequences. During functional MRI of a simple finger
tapping task at 9.4 T using such a sequence, researchers were able
to obtain 0.4 × 0.4 mm in-plane resolution within presumptive pri-
mary motor cortex [53]. Similarly, spin-echo sequences, which
have a reduced signal-to-noise ratio relative to gradient-echo
sequences but greater spatial specificity, become feasible for use at
9.4 T. Within a finger-tapping paradigm, a study taking this
approach reduced the influence of macrovascular contributions to
326 Mark D’Esposito et al.

the BOLD signal relative to a gradient-echo sequence, while

obtaining 1 mm isotropic resolution. As such techniques are vali-
dated and extended, they may someday allow for imaging of thou-
sands of neurons per voxel, as opposed to the hundreds of
thousands of neurons per voxel currently more typical for a human
cognitive neuroscience fMRI experiment.
Virtually all fMRI studies model the large BOLD signal
increase, which is due to a local low-deoxyhemoglobin state, in
order to detect changes correlating with a behavioral task. However,
optical imaging studies have demonstrated that preceding this
large positive response there is an initial negative response reflect-
ing a localized increase in oxygen consumption causing a high-
deoxyhemoglobin state [54]. This early hemodynamic response is
called the “initial dip” and is thought to be more tightly coupled
to the actual site of neural activity evoking the BOLD signal as
compared to the later positive portion of the BOLD response. For
example, Kim et al., scanning cats in a high field scanner, demon-
strated that the early negative BOLD response (e.g., initial dip)
produced activation maps that were consistent with orientation
columns within visual cortex. This finding is quite remarkable
given that the average spacing between two adjacent orientation
columns in cortex is approximately 1 mm. In contrast, the activa-
tion maps produced by the delayed positive BOLD response
appeared more diffuse and cortical columnar organization could
not be identified [55]. Thus, empirical evidence suggests that
deriving activation maps by correlating behavioral responses with
the initial dip may markedly improve spatial resolution.
Another unique method for improving spatial resolution has
been called functional magnetic resonance-adaptation (fMR-A),
which could provide a means for identifying and assessing the func-
tional attributes of sharply defined neuronal populations within a
given region of the brain [56]. Even if the spatial resolution of fMRI
evolves to the point of being able to resolve a population of a few
hundred neurons within a voxel, it is still likely that this small popu-
lation will contain neurons with very different functional properties
that will be averaged together. The adaptation method is based on
several basic principles. First, repeated presentation of the same type
of stimuli (i.e., a picture of the one object) causes neurons to adapt
to those stimuli (i.e., neuronal firing is reduced). Second, if these
neurons are then exposed to a different type of stimulus (i.e., a pic-
ture of another object) or a change in some property of the stimulus
(i.e., the same object in a different orientation), then recovery from
adaptation can be assessed (i.e., whether or not the BOLD signal
returns to its original state). If the signal remains adapted it implies
that the neurons are invariant to the attribute that was changed or if
the signal recovers from the adapted state it would imply that the
neurons are sensitive to that attribute. For example, Grill-Spector
et al. demonstrated that an area of lateral occipital cortex thought to
 Functional MRI: Applications in Cognitive Neuroscience 327

be important for object recognition was less sensitive to changes in

object size and position as compared to changes in illumination and
viewpoint [57]. Thus, with this method it is possible to investigate
the functional properties of neuronal populations with a level of spa-
tial resolution that is beyond that obtained from conventional fMRI
data analysis methods.
Considering all the neuroscientific methods available today for
studying human brain–behavior relationships, fMRI provides an
excellent balance of temporal and spatial resolution. Improvements
on both fronts will clearly add to type of basic and clinical neuro-
scientific questions that can be addressed with this method.

4 Issues in Functional MRI Experimental Design

Numerous options exist for designing experiments using fMRI. The

prototypical fMRI experimental design consists of two behavioral
tasks presented in blocks of trials alternating over the course of a
scanning session, and the fMRI signal between the two tasks is
compared. This is known as a blocked design. For example, a given
block might present a series of faces to be viewed passively, which
evokes a particular cognitive process, such as face perception. The
“experimental” block alternates with a “control” block that is
designed to evoke all of the cognitive processes present in the
experimental block except for the cognitive process of interest. In
this experiment the control block may comprise a series of objects.
In this way, the stimuli used in experimental and control tasks have
similar visual attributes, but differ in the attribute of interest (i.e.,
faces). The inferential framework of “cognitive subtraction” [58]
attributes differences in neural activity between the two tasks to
the specific cognitive process (i.e., face perception). Cognitive sub-
traction was originally conceived by Donders in the late 1800s for
studying the chronometric substrates of cognitive processes [59]
and was a major innovation in imaging [58, 60].
The assumptions required for cognitive subtraction may not
always hold and could produce erroneous interpretation of func-
tional neuroimaging data [45]. Cognitive subtraction relies on two
assumptions: “pure insertion” and linearity. Pure insertion implies
that a cognitive process can be added to a preexisting set of cogni-
tive processes without affecting them. This assumption is difficult
to prove because one needs an independent measure of the preex-
isting processes in the absence and presence of the new process
[59]. If pure insertion fails as an assumption, a difference in the
neuroimaging signal between the two tasks might be observed, not
because a specific cognitive process was engaged in one task and
not the other, but because the added cognitive process and the
preexisting cognitive processes interact.
328 Mark D’Esposito et al.

An example of this point is illustrated in working memory

studies using delayed-response tasks [61]. These tasks [62] typi-
cally present information that the subject must remember (engag-
ing an encoding process), followed by a delay period during which
the subject must hold the information in memory over a short
period of time (engaging a memory process), followed by a probe
that requires the subject to make a decision based on the stored
information (engaging a retrieval process). The brain regions
engaged by evoking the memory process theoretically are revealed
by subtracting the BOLD signal measured by fMRI during a block
of trials that the subject performs that do not have a delay period
(only engaging the encoding and retrieval processes) from a block
of trials with a delay period (engaging the encoding, memory, and
retrieval processes). In this example, if the addition or “insertion”
of a delay period between the encoding and retrieval processes
affects these other behavioral processes in the task, the result is
failure to meet the assumptions of cognitive subtraction. That is,
these “nonmemory” processes may differ in delay trials and no-
delay trials, resulting in a failure to cancel each other out in the two
types of trials that are being compared.
Empirical evidence of such failure exists [63]. For example,
Figure 3 demonstrates BOLD signal derived from the PFC from a
subject performing a delayed-response task similar to the tasks

Fig. 3 Data derived from the performance of a normal subject on a spatial delayed-response task [64]. This
task comprised both delay trials (circles) as well as trials without a delay period (no-delay trials; diamonds). (a)
Trial averaged fMRI signal from prefrontal cortex that displayed delay-correlated activity. The gray bar along
the x-axis denotes the 12 s delay period during delay trials. The delay trials display a level of fMRI signal
greater than baseline throughout the period of time corresponding to the retention delay (taking into account
the delay and dispersion of the fMRI signal). The peaks seen in the signal correspond to the encoding and
retrieval periods. (b) Trial averaged fMRI signal from a region in prefrontal cortex that did not display the char-
acteristics of delay-correlated activity. This region displays a significant functional change associated with the
no-delay trials, and a significant functional change associated with the encoding and retrieval periods of the
delay trials, but not one associated with the retention delay of delay trials
 Functional MRI: Applications in Cognitive Neuroscience 329

described above. The left side of the figure illustrates BOLD signal
consistent with delay period activity whereas the right side of the
figure illustrates BOLD signal from another region of PFC that did
not display sustained activity during the delay yet showed greater
activity in the delay trials as compared to the trials without a delay.
In any blocked functional neuroimaging study that compares delay
vs. no-delay trials with subtraction, such a region would be detected
and likely assumed to be a “memory” region. Thus, this result
provides empirical grounds for adopting a healthy doubt regarding
the inferences drawn from imaging studies that rely exclusively on
cognitive subtraction.
In functional neuroimaging, the transform between the neural
signal and the hemodynamic response (measured by fMRI) must
also be linear for the cognitive subtractive method to yield valid
results. In other words, it is assumed that the BOLD signal being
measured is approximately proportional to the local neural activity
that evokes it. Surprisingly, although thousands of empirical stud-
ies using fMRI to study brain–behavior relationships have been
published, only a handful exist that have explored the neurophysi-
ological basis of the BOLD signal (for reviews see refs. [64, 65]).
In several studies it has been demonstrated that linearity does not
strictly hold for the BOLD fMRI system but a linear transform
model is reasonably consistent with the data. For example, Boynton
et al. tested whether the BOLD signal in response to long duration
stimuli can be predicted by summing the responses to shorter
duration stimuli [42]. Using pulses of flickering checkerboard pat-
terns and measuring within human primary visual cortex, these
investigators found that the BOLD signal response to various
durations of stimulus presentation (6, 12, or 24 s) could be pre-
dicted from the responses they obtained from shorter stimulus pre-
sentations. For example, the BOLD signal response to a 6 s pulse
could be predicted from the summation of the BOLD signal
response to the 3 s pulse with a copy of the same response delayed
by 3 s. However, temporal summation did not always hold, and
there are clearly nonlinear effects in the transform of neural activity
to a hemodynamic response that must be considered [66–69]. If
these nonlinearities lead to saturation of the BOLD effect at a cer-
tain stimulus intensity, erroneous interpretation of particular results
of fMRI experiments may occur.
Another class of experimental designs, called event-related
fMRI, attempts to detect changes associated with individual trials,
as opposed to the larger unit of time comprising a block of trials
[70, 71]. Each individual trial may be composed of one behavioral
“event,” such as the presentation of a single stimulus (e.g., a face
or object to be perceived) or several behavioral events such as in
the delayed-response task described above (e.g., an item to be
remembered, a delay period, and a motor response in a delayed-
response task). For example, with an event-related design, activity
330 Mark D’Esposito et al.

within the PFC has consistently been shown to correlate with the
delay period, supporting the role of the PFC in temporarily main-
taining information [63]. This finding is consistent with single-
neuron recording studies in the PFC of monkeys [7]. An
event-related design offers numerous advantages. For example, it
allows for stimulus or trial randomization avoiding the behavioral
confounds of blocked trials. It also permits the separate analysis of
functional responses that are identified only in retrospect (i.e., tri-
als on which the subject made a correct or incorrect response). Of
course, an experiment does not have to be limited to either a block
or event-related designs—a mixed-type (both event-related and
blocked) design where particular trial types are randomized within
a block is perfectly feasible. In this type of design, both item-related
processes (e.g., transient responses to stimuli) as well as state-
related processes (processes sustained throughout a block of trials
or a task) are perfectly feasible [72, 73].
Overall, much flexibility exists in the type of experimental
design that can be utilized in fMRI experiments and continued
innovation in this area will greatly expand the types of neuroscien-
tific questions that can be addressed.

5 Issues in Interpretation of fMRI Data

5.1 Statistics Many statistical techniques are used for analyzing fMRI data, but
no single method has emerged as the ideal or “gold standard.” The
analysis of any fMRI experiment designed to contradict the null
hypothesis (i.e., there is no difference between experimental con-
ditions) requires inferential statistics. If the difference between two
experimental conditions is too large to be reasonably due to chance,
then the null hypothesis is rejected in favor of the alternative
hypothesis, which typically is the experimenter’s hypothesis (e.g.,
the fusiform gyrus is activated to a greater extent by viewing faces
than objects). Unfortunately, since errors can occur in any statisti-
cal test, experimenters will never know when an error is committed
and can only try to minimize them [74]. Knowledge of several
basic statistical issues provides a solid foundation for the correct
interpretation of the data derived from fMRI studies.
Two types of statistical errors can occur. A type I error is commit-
ted when the null hypothesis is falsely rejected, that is, a difference
between experimental conditions is found but a difference does not
truly exist. This type of error is also called a false-positive error. In an
fMRI study, a false-positive error would be finding a brain region
activated during a cognitive task, when actually it is not. A type II
error is committed when the null hypothesis is accepted when it is
false, that is, no difference between experimental conditions exists
when a difference does exist. This type of error is also called a false-
negative error. A false-negative error in an fMRI study would be
 Functional MRI: Applications in Cognitive Neuroscience 331

failing to find a brain region activated during the performance of a

cognitive task when actually it is. The concept of type II error is
closely related to the idea of statistical power. If the false-negative rate
for a given study design is 20 %, for instance, then the “power” of that
design to detect an activation is 100 − 20 % or 80 %.
In cognitive neuroscience studies, much emphasis has been
placed on avoiding type I errors. The negative effects of incorrectly
identifying a brain region as task-active include the expenditures of
time, money, and effort spent in replicating and/or expanding
upon a false positive result. Type II error, on the other hand, is seen
as less damning; failure to detect brain activity in a research study
has fewer implications for future research, provided that one is care-
ful to interpret so-called null results correctly. For example, cogni-
tive neuroscience studies (due to factors such as the expense and the
difficulty of finding research participants, for example) tend to
employ a small number of subjects—15 would typical—and there-
fore frequently lack power to detect significant brain activations.
One must consequently be careful to avoid interpreting a lack of
activation in one part of the brain as true inactivity during the task.
In a clinical research study, on the contrary, the emphasis may
be different, especially when fMRI studies are being used diagnos-
tically in individual patients. A type II error—failing to detect
active brain regions related to movement or language in the vicin-
ity of a brain tumor, for example—may lead to a larger surgical
resection that leaves the patient with avoidable residual deficits. On
the contrary, a type I error—for example, identifying motor activity
adjacent to a tumor when in fact none exists—may erroneously
lead to a more cautious surgical resection, or to use of a different
treatment modality. Which error is deemed more tolerable may
depend on the clinical situation.
In fMRI experiments, like all experiments, a tolerable probabil-
ity for type I error, typically less than 5 %, is chosen for adequate
control of specificity, that is, control of false-positive rates. Two fea-
tures of fMRI data can cause unacceptable false-positive rates, even
with traditional parametric statistical tests. First, there is the problem
of multiple comparisons. For the typical resolution of images
acquired during fMRI scans, the full extent of the human brain
could comprise as many as 15,000 voxels. Thus, with any given sta-
tistical comparison of two experimental conditions, there are actu-
ally 15,000 statistical comparisons being performed. With such a
large number of statistical tests, the probability of finding a false-
positive activation, that is, committing a type I error, somewhere in
the brain increases. Several methods exist to deal with this problem.
One method, a Bonferroni correction, assumes that each statistical
test is independent and calculates the probability of type I error by
dividing the chosen probability (p = 0.05) by the number of statisti-
cal tests performed. Another method is based on Gaussian field the-
ory [75], and calculates the probability of type I error when imaging
332 Mark D’Esposito et al.

data are spatially smoothed. Many other methods for determining

thresholds of statistical maps are proposed and utilized [76, 77] but
unfortunately, no single method has been universally accepted.
Nevertheless, all fMRI studies must apply some type of correction
for multiple comparisons to control the false-positive rate.
The second feature that might increase the false-positive rate is
the “noise” in fMRI data. Data from BOLD fMRI are temporally
autocorrelated, with more noise at some frequencies than at oth-
ers. The shape of this noise distribution is characterized by a 1/
frequency function with increasing noise at lower frequencies [78].
Traditional parametric and nonparametric statistical tests assume
that the noise is not temporally autocorrelated, that is, each obser-
vation is independent. Therefore, any statistical test used in fMRI
studies must account for the noise structure of fMRI data. If not,
the false-positive rates will inflate [78, 79].
Type II error is rarely considered in functional neuroimaging
studies. When a brain map from an fMRI experiment is presented,
several areas of activation are typically attributed to some experimen-
tal manipulation. The focus of most fMRI studies is on brain activa-
tion whereas it is often implicitly assumed that all of the other areas
(typically most of the brain) were not activated during the experi-
ment. Power as a statistical concept refers to the probability of cor-
rectly rejecting the null hypothesis [74]. As the power of an fMRI
study to detect changes in brain activity increases, the false-negative
rate decreases. Unfortunately, power calculations for particular fMRI
experiments are rarely performed, although methods exist to address
this issue [80–82]. Reports that specific brain areas were not active
during an experimental manipulation should provide an estimate of
the power required for detection of a change in the region. All
experiments should be designed to maximize power. Relatively sim-
ple strategies can increase power in an fMRI experiment in certain
circumstances, such as increasing the amount of imaging data col-
lected or increasing the number of subjects studied. It is also impor-
tant to note that task designs can affect sensitivity [83]. For example,
since BOLD fMRI data are temporally autocorrelated, experiments
with fundamental frequencies in the lower range (e.g., a boxcar
design with 60 s epochs) will have reduced sensitivity, due to the
presence of greater noise at these lower frequencies. Finally, in a
study that simultaneously measured neural signal via intracortical
recording and BOLD signal in a monkey, it was observed that the
SNR of the neural signal was on average at least one order of mag-
nitude higher than that of the BOLD signal. The investigators of
this study concluded that “the statistical and thresholding methods
applied to the hemodynamic responses probably underestimate a
great deal of actual neural activity related to a stimulus or task” [84].
Thus, the magnitude of type II error in BOLD fMRI may currently
be underestimated and warrants further consideration in the inter-
pretation of almost any cognitive neuroscience experiment.
 Functional MRI: Applications in Cognitive Neuroscience 333

5.2 Altered When comparing changes in fMRI BOLD signal levels within the
Hemodynamic brain of an individual subject across different cognitive tasks and
Response making conclusions regarding changes in neural activity and the
pattern of activity, numerous assumptions are made regarding the
steps comprising neurovascular coupling (stimulus → neural activ-
ity → hemodynamic response → BOLD signal) and the regional
variability of the metabolic and vascular parameters influencing the
BOLD signal. It should be obvious that fMRI studies of cognition
of individuals with local vascular compromise or diffuse vascular
disease (e.g., patients with strokes or normal elderly) are poten-
tially problematic. For example, many fMRI studies have sought to
identify age-related changes in the neural substrates of cognitive
processes. Those studies that directly compare changes in fMRI
BOLD signal intensity across age groups rely upon the assumption
of age-equivalent coupling of neural activity to BOLD signal.
However, there is empirical evidence that suggests that this general
assumption may not hold true. Extensive research on the aging
neurovascular system has revealed that it undergoes significant
changes in multiple domains in a continuum throughout the
human lifespan, probably as early as the fourth decade (for review
see ref. [85]). These changes affect the vascular ultrastructure [86],
the resting cerebral blood flow [87, 88], the vascular responsive-
ness of the vessels [89], and the cerebral metabolic rate of oxygen
consumption [90, 91]. Aging is also frequently associated with
comorbidities such as diabetes, hypertension, and hyperlipidemia,
all of which may affect the fMRI BOLD signal by affecting cerebral
blood flow and neurovascular coupling [92]. Any one of these age-
related differences in the vascular system could conceivably pro-
duce age-related differences in BOLD fMRI signal responsiveness,
greatly affecting the interpretation of results from such studies.
Our laboratory compared the hemodynamic response function
(HRF) characteristics in the sensorimotor cortex of young and
older subjects in response to a simple motor reaction-time task
[70]. The provisional assumption was made that there was identi-
cal neural activity between the two populations based on physio-
logical findings of equivalent movement-related electrical potentials
in subjects under similar conditions [93]. Thus, we presumed that
any changes that we observed in BOLD fMRI signal between
young and older individuals in motor cortex would be due to vas-
cular, and not neural activity changes in normal aging. Several
important similarities and differences were observed between age
groups. Although there was no significant difference in the shape
of the hemodynamic response curve or peak amplitude of the sig-
nal, we found a significantly decreased SNR in the fMRI BOLD
signal in older individuals as compared to young individuals. This
was attributed to a greater level of noise in the older individuals.
We also observed a decrease in the spatial extent of the BOLD
signal in older individuals compared to younger individuals in
334 Mark D’Esposito et al.

sensorimotor cortex (i.e., the median number of suprathreshold

voxels). Similar results have been replicated by other laboratories
(e.g., [94, 95]). These findings suggest that there is some property
of the coupling between neural activity and fMRI BOLD signal
that changes with age.
The notion that vascular differences among individuals may
affect BOLD signal is especially a concern when considering studies
of patient populations with known vascular changes such as stroke.
For example, in a fMRI study of patients with an isolated subcortical
lacunar stroke compared to a group of age-matched controls, a
decrease in the rate of rise and the maximal fMRI BOLD HRF to a
finger- or hand-tapping task in both the sensorimotor cortex of the
hemisphere affected by the stroke and the unaffected hemisphere
was found [96]. These investigators proposed that given the wide-
spread changes of these fMRI BOLD signal differences, the change
was unlikely a direct consequence of the subcortical lacunar stroke,
but rather a manifestation of preexisting diffuse vascular pathology.
In summary, comparing BOLD signal in two different groups
of individuals that may differ in their vascular system should be
done with caution [97]. For example, in one scenario, a compari-
son of activation of young and elderly individuals during a cogni-
tive task may show less activation by elderly (as compared to young
subjects) in some brain regions, but greater activation in other
regions. In this scenario, it is unlikely that regional variations in the
hemodynamic coupling of neural activity to fMRI signal would
account for such age-related differences in patterns of activation.
In another scenario, a comparison of young and elderly subjects
may show less activation by elderly (as compared to young sub-
jects) in some brain regions, but no evidence of greater activation
in any other region. In this case, it is possible that the observed
age-related differences are not due to differences in intensity of
neural activity, but rather to other nonneuronal contributions to
the imaging signal, i.e., neurovascular coupling.
In summary, fMRI BOLD contrast methods yield signal
changes that result from a complex mix of vascular effects and pro-
vide only relative, rather than absolute, measures. One approach to
accounting for the influence of purely vascular effects is to directly
measure regional and individual variability in vascular reactivity via
a breathholding task, which increases carbon dioxide concentration
in the blood and leads to vascular dilatation [98]. The task-related
BOLD signal in each subject can then be corrected for particular
region- and subject-specific vascular effects. One alternative func-
tional neuroimaging approach, based on more direct measurements
of cerebral blood flow to active brain areas, is known as arterial spin
labeling (ASL). In the various ASL techniques, the MRI scanner
selectively magnetizes blood flow with a particular range of loca-
tions and/or velocities, then waits for the appearance of the mag-
netic “tag” in downstream vessels. It thus becomes possible to
 Functional MRI: Applications in Cognitive Neuroscience 335

obtain absolute measures of cerebral perfusion [99], thereby open-

ing up the possibility of more quantitatively distinguishing between
the differential influence of a disease on blood flow, and its effect on
brain activity [100]. Additionally, relative to BOLD contrast these
absolute measurements appear to be more stable over long experi-
ments because of better signal-to-noise at very low frequencies
[101], to show less between-subject and between-session variability
[102], and to produce decreased susceptibility artifact in areas such
as medial temporal lobe [103]. A significant limitation is temporal
resolution: one must both wait for the generation of sufficient mag-
netic label, and also acquire two scans, a reference scan and a post-
labeling scan, to produce a single data point. However, a recently a
new MRI acquisition method has been developed that allows for
more slices for measuring perfusion in a larger region of the brain
than currently possible with previous methods [104]. Another
potential disadvantage somewhat related to the temporal issues is
the lower SNR of ASL relative to BOLD, but this decline may be
compensated in group studies by the observation that ASL meth-
ods appear to be less variable across subjects [99].

6 Types of Hypotheses Tested Using fMRI

Functional neuroimaging experiments test hypotheses regarding

the anatomical specificity for cognitive processes (functional spe-
cialization) or direct or indirect interactions among brain regions
(functional integration). The experimental design and statistical
analyses chosen will determine the types of questions that can be
addressed. Ultimately, the most powerful approach for the testing
of theories on brain–behavior relationships is the analysis of con-
verging data from multiple methods.

6.1 Functional The major focus of fMRI studies of cognition is testing theories on
Specialization functional specialization. The concept of functional specialization
is based on the premise that functional modules exist within the
brain, that is, areas of the cerebral cortex are specialized for a spe-
cific cognitive process. For example, facial recognition is a critical
primary function likely served by a functional module.
Prosopagnosia is the selective inability to recognize faces. Patients
with prosopagnosia, however, can recognize familiar faces, such as
those of relatives, by other means, such as the voice, dress, or
shape. Other types of visual recognition, such as identifying com-
mon objects, are normal. Prosopagnosia arises from lesions of the
inferomedial temporo-occipital lobe, which are usually due to a
stroke within the posterior cerebral artery circulation. No lesion
studies have precisely localized the area crucial for facial percep-
tion. However, they provide strong evidence that a brain area is
specialized for processing faces. Functional imaging studies have
336 Mark D’Esposito et al.

provided anatomical specificity for such a module. For example,

Kanwisher et al. [105] used fMRI to test a group of healthy indi-
viduals and found that the fusiform gyrus was significantly more
active when the subjects viewed faces than when they viewed
assorted common objects. The specificity of a “fusiform face area”
was further demonstrated by the finding that this area also
responded significantly more strongly to passive viewing of faces
than to scrambled two-tone faces, front-view photographs of
houses, and photographs of human hands. These elegant experi-
ments allowed the investigators to reject alternative functions of
the face area, such as visual attention, subordinate-level classifica-
tion, or general processing of any animate or human forms, dem-
onstrating that this region selectively perceives faces.
Of course, the existence of brain areas specialized for certain
functions does not exclude the strong possibility that those areas
show either finer, voxel-level structure or are part of larger networks.
Recent neuroimaging work has focused on pattern classification
methods—that is, on techniques to explore whether a distributed
spatial pattern of brain activity, both within a single region and across
larger brain areas, corresponds to object (or more abstract) represen-
tations. This area of research draws on results from physics, computer
science, and statistics, among other disciplines, to search for more
broadly distributed structure in neuroimaging data. As such, the
techniques themselves differ. For example, to distinguish between
voxel activity patterns across experimental conditions, various reports
have used correlations between the set of activations in visual
responses to faces and other objects [106]; neural network classifiers
to identify particular patterns correlated with particular memories
[107]; and variants of a matrix algebra transformation known as sin-
gular value decomposition to look for distributed spatial correlates of
memory storage and search [108]. By establishing sophisticated
models of the relationships between brain activity and visual stimuli
in visual cortex, representations of natural images may even be suc-
cessfully decoded [109]. A large number of other techniques—too
large to be reviewed here—are also continually being developed
[110, 111]. As such research demonstrates that task-relevant brain
activity can be detected even in the absence of classic univariate activ-
ity changes. However, it will remain important to control for poten-
tial confounds in brain activity data, with validation via comparison
with behavioral responses, in order to ensure that these patterns are
not epiphenomenal or a result of confounds such as reaction time
[112]. At a higher tier of analysis, information decoding techniques
are being used to examine mechanisms by which higher order cogni-
tion can modulate information representations. A step beyond simply
detecting the existence of a particular representational code, one can
now ask, for example, to what extent goal-direction (attention) might
change the tuning of neural network codes to better represent infor-
mation related to a goal [113, 114].
 Functional MRI: Applications in Cognitive Neuroscience 337

6.2 Functional Functional neuroimaging experiments can also test hypotheses

Integration about interactions between brain regions by focusing on covari-
ances of activation levels between regions [115, 116]. These
covariances reflect “functional connectivity,” a concept that was
originally developed in reference to temporal interactions among
individual neurons [117].
In addition to providing information about the specialization
of various brain regions, functional neuroimaging can also address
the interactions between brain regions that underlie cognitive
processing. Understanding the various techniques that permit
these types of analysis comprises a very active area of current
research [118]. However, most, if not all, of the techniques used
to test for regional interactions are ultimately based on the covari-
ance of activation levels in different brain regions across time—in
other words, on the way in which activity levels in different areas of
the brain rise or fall in relation to each other. Such statistical tech-
niques are commonly known as “multivariate,” both because they
rely on interactions between two or more brain areas, and to dis-
tinguish them from the “univariate” methods applied in most tests
of functional specialization.
The universe of multivariate techniques is further subdivided
into two types, determined by whether the method in question is
designed to assess connectivity in a model-free (“functional con-
nectivity”) or model-based (“effective connectivity”) fashion. The
former refers simply to methods that measure the temporal covari-
ance in activity between brain areas without a priori notions about
which brain areas are relevant or how they should interact. Examples
of model-free techniques would include correlation and its fre-
quency-based analogue, coherence, which can be applied irrespec-
tive of hypotheses about the neural events that produced them. On
the contrary, model-based, or effective connectivity, approaches
begin with hypotheses about the interactions between different
brain regions, and attempt to support/refute them by evaluating
the presence/absence of specific activity covariance patterns.
Examples of these techniques would include structural equation
modeling and dynamic causal modeling, both of which start by pos-
tulating the existence of influences (potentially complex, potentially
time-varying) between specific brain regions. Both types of statisti-
cal techniques have value, of course; their use is determined by the
problem at hand. Model-free approaches are more general, and
more easily deployed in exploratory analyses. However, they are not
as powerful as model-based methods that address specific hypoth-
eses about how regions interact—but which fail if the model is mis-
specified. Model-free methods, for example, may be more useful
when attempting to determine which networks of brain areas might
be involved in a task, whereas model-based methods may be most
appropriate when the nodes of the network are known, and specific
notions about how they interact need to be tested.
338 Mark D’Esposito et al.

In our own laboratory, we have developed and used functional

connectivity techniques to understand how brain interactions change
under different task conditions, and over time [119, 120]. For
example, we have shown that functional connectivity changes as
subjects learn a complex finger tapping task [121]. In the early
phases of learning, the data show that subjects not only activate wide
areas of primary sensorimotor cortex, premotor cortex, and the sup-
plementary motor area, but also that the coherence between these
areas is increased relative to later stages. Such changes were not
observed when subjects performed an already learned motor skill;
and more importantly, they were not found in the univariate
responses, whose means were unchanged despite the changes in the
subjects’ facility at the task. Similarly, in a working memory task for
faces [122], we have found an interesting dissociation between their
univariate and multivariate analyses in the networks that support so-
called delay period activity (see below). In our protocol, subjects
encoded a cue face, maintained the image across a delay of several
seconds, and then decided whether a subsequently presented probe
face matched the initial one. Interestingly, we found that despite a
general decrease in the univariate activity from the cue to the delay
period, there was a robust increase in the correlation between activ-
ity in the right fusiform face area (a brain region known to be sensi-
tive to face stimuli) and a diffuse set of brain regions including the
frontal and parietal cortices as well as the basal ganglia.
In such known networks, effective connectivity techniques can
be employed to more specifically evaluate the influence of the
nodes of the network on each other. McIntosh et al., for example,
were able to exploit their own functional neuroimaging research
on working memory networks to formulate a hypothesis about the
interactions of the PFC, cingulate cortex, and other brain regions
during task performance [116]. Using structural equation model-
ing, the authors found shifting prefrontal and limbic interactions in
a working memory task for faces as the retention delay increased
(Fig. 4). The different interactions between brain regions at short
and long delays were interpreted as a functional change. For exam-
ple, strong corticolimbic interactions were found at short delays,
but at longer delays, when the image of the face was more difficult
to maintain, strong fronto-cingulate-occipital interactions were
found. The investigators postulated that the former finding was
due to maintaining an iconic facial representation, and the latter
due to an expanded encoding strategy, resulting in more resilient
memory. As in our own previous studies, information that was not
seen in the univariate analysis was captured by an approach sensi-
tive to regional interactions. In addition to structural equation
modeling, other approaches have been applied to fMRI datasets to
capture information regarding the relative timing of activation
across brain regions such as Granger causality, information analysis,
and coherence (see [119, 120, 123]).
 Functional MRI: Applications in Cognitive Neuroscience 339

Fig. 4 Network analysis of fMRI data using structural equation modeling during performance of a working
memory task across three different delay periods [111]. Areas of correlated increases in activation (solid lines)
and areas of correlated decreases in activation (dotted lines) are shown. Note the different pattern of interac-
tions among brain regions at short and long delays

Mathematical tools based on graph theory have recently

emerged as a method to quantify large-scale network properties of
the brain as well as to identify the role of individual brain regions
within these large-scale networks. These tools, developed for ana-
lyzing a wide variety of networks (e.g., social networks, the inter-
net, protein associations), allow one to make quantitative
measurements of brain network structure. Typically, these meth-
ods are used to analyze the spontaneous coherent fluctuations in
BOLD signal measured by fMRI at rest, which consistently identi-
fies stable intrinsic functional networks, that, in a short fMRI
recording session, recapitulate a number of sub-networks normally
engaged by a variety of different tasks (see Fig. 5).

6.3 Cognitive Theory Experiments using fMRI can also test theories of the underlying
mechanisms of cognition. For example, an fMRI study [124]
attempted to answer the question, “To what extent does perception
depend on attention?” One hypothesis is that unattended stimuli in
the environment receive very little processing [125], but another
hypothesis is that the processing load in a relevant task determines
the extent to which irrelevant stimuli are processed [126]. These
alternative hypotheses were tested by asking normal individuals to
perform linguistic tasks of low or high load while ignoring irrele-
vant visual motion in the periphery of a display. Visual motion was
used as the distracting stimulus, because it activates a distinct region
of the brain (cortical area MT or V5, another functional module in
the visual system). Activation of area MT would indicate that irrel-
evant visual motion was processed. Although task and irrelevant
stimuli were unrelated, fMRI of motion-related activity in MT
showed a reduction in motion processing during the high-process-
ing load condition in the linguistic task. These findings supported
the hypothesis that perception of irrelevant environmental
340 Mark D’Esposito et al.

Fig. 5 A brain graph derived from resting state fMRI data collected from healthy young subjects illustrating
identified modules, represented as different shades of color. There are four distinct modules identified in this
graph

information depends on the information processing load that is

currently relevant and being attended to. Thus, by the finding that
perception depends on attention, this fMRI experiment provides
insight regarding underlying cognitive mechanism.

7 Integration of Multiple Methods

The most powerful approach toward understanding brain–behavior

relationships comes from analyzing converging data from multiple
methods. There are several ways in which different methods can
provide complementary data. For example, one method can pro-
vide superior spatial resolution (e.g., fMRI) whereas the other can
provide superior temporal resolution (e.g., ERP). Also, the data
from one method may allow for different conclusions to be drawn
from it such as whether a particular brain region is necessary to
implement a cognitive process (i.e., lesion methods) or whether it is
only involved during its implementation (i.e., physiological meth-
ods). The following sections describe examples of such approaches.

7.1 Combined fMRI/ The combined use of functional neuroimaging and lesions studies
Lesion Studies can be illustrated with studies of the neural basis of semantic mem-
ory, the cognitive system that represents our knowledge of the
 Functional MRI: Applications in Cognitive Neuroscience 341

world. Early studies of patients with focal lesions supported the

notion that the temporal lobes mediate the retrieval of semantic
knowledge [127]. For example, patients with temporal lobe lesions
may show a disproportionate impairment in the knowledge of liv-
ing things (e.g., animals) compared with nonliving things. Other
patients have a disproportionate deficit in the knowledge of nonliv-
ing things [128]. These observations led to the notion that the
semantic memory system is subdivided into different sensorimotor
modalities, that is, living things, compared with nonliving things,
are represented by their visual and other sensory attributes (e.g., a
banana is yellow), while nonliving things are represented by their
function (e.g., a hammer is a tool but comes in many different visual
forms). The small number of patients with these deficits, and often
large lesions, limits precise anatomical-behavioral relationships.
However, functional neuroimaging studies in normal subjects can
provide spatial resolution that the lesion method lacks [129].
These original observations regarding the neural basis of seman-
tic memory conflicted with functional neuroimaging studies consis-
tently showing activation of the left IFG during the retrieval of
semantic knowledge. For example, an early cognitive activation PET
study revealed IFG activation during a verb generation task com-
pared with a simple word repetition task [60]. A subsequent fMRI
study [130] offered a fundamentally different interpretation of the
apparent conflict between lesion and functional neuroimaging stud-
ies of semantic knowledge: left IFG activity is associated with the
need to select some relevant feature of semantic knowledge from
competing alternatives, not retrieval of semantic knowledge per se.
This interpretation was supported by an fMRI experiment in normal
individuals in which selection, but not retrieval, demands were var-
ied across three semantic tasks. In a verb generation task, in a high
selection condition, subjects generated verbs to nouns with many
appropriate associated responses without any clearly dominant
response (e.g., “wheel”), but in a low selection condition nouns
with few associated responses or with a clear dominant response
(e.g., “scissors”) were used. In this way, all tasks required semantic
retrieval, and differed only in the amount of selection required. The
fMRI signal within the left IFG increased as the selection demands
increased (Fig. 6). When the degree of semantic processing varied
independently of selection demands, there was no difference in left
IFG activity, suggesting that selection, not retrieval, of semantic
knowledge drives activity in the left IFG.
To determine if left IFG activity was correlated with but not
necessary for selecting information from semantic memory, the
same task used during the fMRI study was used to examine the
ability of patients with focal frontal lesions to generate verbs [131].
Supporting the earlier claim regarding left IFG function derived
from an fMRI study [130], the overlap of the lesions in patients
with deficits on this task corresponded to the site of maximum
342 Mark D’Esposito et al.

Fig. 6 Regions of overlap of fMRI activity in healthy human subjects (left side of
figure) during the performance of three semantic memory tasks, with the con-
vergence of activity within the left inferior frontal gyrus (white region) [125].
Regions of overlap of lesion location in patients with selection-related deficits on
a verb generation task (right side of figure) with maximal overlap within the left
inferior frontal gyrus [126]

fMRI activation in healthy young subjects during the verb genera-

tion task (Fig. 6). In this example, the approach of using converg-
ing evidence from lesion and fMRI studies differs in a subtle but
important way from the study described earlier that isolated the
face processing module. Patients with left IFG lesions do not pres-
ent with an identifiable neurobehavioral syndrome reflecting the
nature of the processing in this region. Guided by the fMRI results
from healthy young subjects, the investigators studied patients
with left IFG lesions to test a hypothesis regarding the necessity of
this region in a specific cognitive process. Coupled with the well-
established finding that lesions of the left temporal lobe impair
semantic knowledge, these studies further our understanding of
the neural network mediating semantic memory.

7.2 Combined fMRI/ Transcranial magnetic stimulation (TMS) is a noninvasive method

Transcranial Magnetic that can induce a reversible “virtual” lesion of the cerebral cortex in
Stimulation Studies a normal human subject [132]. Using both fMRI and TMS provides
another means of combining brain activation data with data derived
from the lesion method. There are several advantages for using TMS
as a lesion method. First, brain injury likely results in brain reorgani-
zation after the injury and studies of patients with lesions assume
that the nonlesioned brain areas have not been affected, whereas
TMS is performed on the normal brain. Another advantage for using
TMS is that it has excellent spatial resolution and can target specific
locations in the brain whereas lesions in patients with brain injury are
markedly variable in location and size across individuals. Such an
approach can be illustrated in an investigation of the role of the
 Functional MRI: Applications in Cognitive Neuroscience 343

medial frontal cortex in task switching [133]. In this study, subjects

first performed an fMRI study that identified the regions that were
active when they stayed on the current task vs. when they switched
to a new task. It was found that medial frontal cortex is activated
when switching between tasks. In order to determine if the medial
frontal cortex was necessary for the processes involved in task switch-
ing, the same paradigm was utilized during inactivation of the medial
frontal cortex with TMS. Guided by the locations of activation
observed in the fMRI study, and using an MRI guided frameless
stereotaxic procedure, it was found that applying a TMS pulse over
the medial frontal cortex disrupted performance only during trials
during which the subject was required to switch between tasks. TMS
over adjacent brain regions did not show this effect. Also, the excel-
lent temporal resolution of TMS allowed the investigators to stimu-
late during precise periods of the task, determining that the observed
effect was during the time when the subjects were presented a cue
indicating they must switch tasks prior to the actual performance of
the new task. Thus, combining the results from both fMRI and
TMS, it was concluded that medial PFC was essential for allowing
individuals to intentionally switch to a new task.
It is possible to perform TMS studies not only as an adjunct
to, but also concurrently with, fMRI. The advantage of this
approach is clear: applying TMS at various times during (rather
than after) fMRI scans permits it to be causally linked with func-
tional changes in the brain, even independently of behavior. In
an early study employing this technique, Ruff, Driver and col-
leagues [36, 134] examined the influence on early visual cortex
of a parietal region (the anterior intraparietal sulcus, or aIPS)
implicated in the generation of both covert spatial attention and
eye movements. They chose a range of TMS )stimulus intensi-
ties, all of which were thought to be in an effectively stimulatory
rather than inhibitory range, and applied them to the aIPS while
subjects fixated the center of a viewing screen. On some trials, a
randomly moving visual stimulus was present; subjects had no
other task than to maintain fixation. Using this approach, the
authors were able to demonstrate a parametric, so-called top-
down effect from aIPS following TMS—an increase in the BOLD
response in early visual cortex with increasing TMS intensity—
that could be found only when visual stimuli were absent, and
that did not vary with retinotopic eccentricity. In distinction,
their previous work (extended here) had shown that TMS of the
frontal eye field (FEF) led to a decrease in BOLD response in the
central visual field but to an increase in BOLD response in the
peripheral visual field, irrespective of the presence or absence of
a visual stimulus. The authors were consequently able to con-
clude that the aIPS and the FEF have distinct top-down effects
on visual cortex, a finding that would not have been possible
without concurrent TMS.
344 Mark D’Esposito et al.

7.3 Combined fMRI/ The strength of combining these two methods is coupling the
Event-Related superb spatial resolution of fMRI with the superb temporal resolu-
Potential Studies tion of ERP recording. An example of such a study was reported
by Dehaene et al. who asked the question “Does the human capac-
ity for mathematical intuition depend on linguistic competence or
on visuospatial representations?” [135]. In this study, subjects per-
formed two addition tasks—one in which they were instructed to
select the correct sum from two numerically close numbers (exact
condition) and one in which they were instructed to estimate the
result and select the closest number (approximate condition).
During fMRI scanning greater bilateral parietal lobe activation was
observed in the approximation condition as compared to the exact
condition. Since this activation was outside the perisylvian lan-
guage zone, it was taken as support that visuospatial processes were
engaged during the cognitive operations involved in approximate
calculation. Greater left lateralized frontal lobe activation was
observed to be greater in the exact condition as compared to the
approximate condition, which was taken as evidence for language
dependent coding of exact addition facts. In order to consider an
alternative explanation of the fMRI findings, the investigators also
performed an ERP study. The alternative explanation was that in
both the exact and approximate tasks, subjects would compute the
exact result using the same representation for numbers but later
processing, when they had to make a decision as to the correct
choice, was what led to the differences in brain activation. Since
fMRI does not offer adequate temporal resolution to resolve these
two behavioral events on such a brief time scale, ERP was the
appropriate method to test this hypothesis. In the ERP study it was
demonstrated that the evoked neural response during exact and
approximate trials already differed significantly during the first
400 ms of a trial before subjects had to make a decision.

7.4 Combined fMRI/ Combining pharmacological challenges during the performance of

Pharmacological cognitive tasks during fMRI scanning may yield significantly differ-
Studies ent information than either method alone. In isolation, fMRI cogni-
tive task paradigms provide little information with respect to the
underlying pharmacologic systems involved in cognition. On the
contrary, drug administration without a brain measure cannot deter-
mine underlying neural mechanisms of the effects of neuromodula-
tory systems on cognition. Combining the two approaches allows
the potential of probing the pharmacologic bases of behavior. One
may measure the interactive effects of drug (compared to placebo,
or a range of doses) with cognitive task-related modulation of brain
activity. It is fair to infer that drug × task interactions reflect modula-
tion of the underlying anatomical and chemical brain systems, and
do not simply reflect nonspecific vascular effects. For example, dopa-
minergic agonists have been shown to have task-specific effects
[136–138], and different component processes of working memory
 Functional MRI: Applications in Cognitive Neuroscience 345

are differentially affected by a dopaminergic drug, with effects that

may differ between individuals depending on their baseline state
[139]. This latter study demonstrated that a dopamine agonist
improved the flexible updating (switching) of relevant information
in working memory. However, the effect only occurred in individu-
als with low working memory capacity, but not in individuals with
higher working memory capacity. This behavioral effect was accom-
panied by dissociable effects of the dopaminergic agonist on fronto-
striatal activity. The dopamine agonist modulated the striatum
during switching but not during distraction from relevant informa-
tion in working memory, while the lateral frontal cortex was modu-
lated by the drug during distraction but not during switching.

8 Application of a Cognitive Neuroscience Approach Toward Clinical Studies

8.1 Use Cognitive neuroscience studies using fMRI may provide an impor-
of Biomarkers Derived tant foundation for clinical studies. A biomarker is an indicator that
from Cognitive reflects a process, event, or condition in a biological system.
Neuroscience Studies Biomarkers may be useful for providing a measure of exposure,
effect, or susceptibility. Reliable biomarkers of a neural system could
reliably quantify how such a neural system is affected by almost any
input. The input may be the effects of a drug, the effects of cogni-
tive therapy, or the effects of a disease process. For a measurement
to be useful as a biomarker in clinical studies, it needs to have well-
defined significance based on preclinical studies. That is, a change in
an fMRI measurement would ideally reflect a change in a well-
understood process, thus providing a clear a priori hypothesis and
interpretation of the findings. Once the processes are established,
fMRI biomarkers may then be useful for addressing a number of
clinical questions. For any neurophysiologic measurement to be a
surrogate marker, a stable, reliable relationship between the fMRI
measurement and a defined clinical outcome needs to be defined.
Only in that scenario would an fMRI measurement provide a suit-
able surrogate for other clinical outcomes. Cognitive neuroscience
studies provide the foundation for fMRI biomarkers, but the stud-
ies necessary for defining fMRI surrogate markers are rarely done.
Questions regarding the mechanisms of brain function dis-
rupted by pathologic states, processes affected by treatment inter-
ventions, or the nature of post-injury reorganization of function
are examples of clinical questions that can be tested with fMRI. For
example, attentional modulation of information processing-related
activity in visual cortex is a well-established phenomenon in cogni-
tive neuroscience studies, with effects measurable using fMRI. For
example, it has been shown that activity in category-selective
regions of inferior temporal cortex is modulated based on the tar-
get of attention, relatively up-modulated if the target is relevant to
the region and down-modulated if not relevant [140, 141]. This
346 Mark D’Esposito et al.

finding provides a biomarker of attentional control over visual pro-

cessing, and as noted below, could serve as a useful biomarker for
clinical interventions such as cognitive training in individuals with
attentional deficits.

8.2 Functional MRI Functional MRI may be useful not only in defining “static” brain–
for Measuring behavioral relationships, but also may be applied to defining the
the Effect of Clinical neural mechanisms that underlie learning, experience, or injury.
Interventions Two general categories of questions may be investigated. First,
fMRI can be used to examine factors that influence response to the
perturbations of training (learning), experience or injury. Second,
fMRI can be used to examine changes that underlie or are the
result of these various perturbations.
Investigation of baseline factors that may influence response to
training has particular clinical relevance. A better understanding of
pre-training neural characteristics that influence response to reha-
bilitation training could have major clinical value in guiding treat-
ment decisions. fMRI could provide a number of possible
measurements that could mark an important neural process. For
example, certain parameters of brain network organization may be
particularly important in supporting the potential for learning and
plasticity. For example, parameters of the functional organization
of whole brain networks have been shown to predict response to
training of attention regulation after injury [142]. In another
example, a simple measurement of the quantity of activation in
prefrontal cortex has been shown to predict response to training to
use a verbal memory strategy [143]. Such approaches may help
elucidate either personal factors or strategic approaches that under-
lie variations in learning or response to interventions.
Investigation of changes over time is particularly relevant for
understanding neural mechanisms of post-injury rehabilitation. In
order to assess changes with intervention, longitudinal or repeated
measurements are required. Because fMRI involves no
exposure-limiting factors such as radiation, it is suitable for repeated
measurements. However, multi-session studies are also signifi-
cantly more complicated to design, analyze, and interpret due to a
number of issues discussed below.
There are at least two distinct approaches relevant to assessing
changes within an individual. First, fMRI may be used for deter-
mining the after-effects of a learning intervention. Functional MRI
measures pre- and post-intervention may be used to address this
question. For example, after two pieces of information have been
strongly associated over repetitive exposures, one may find reduced
activation in response to presentation of that information, but
increased functional connectivity between regions of the brain that
process the two types of information [115]. Second, fMRI may be
used for determining the processes that occur during an interven-
tion, such as cognitive training. To do this one would need to
 Functional MRI: Applications in Cognitive Neuroscience 347

acquire fMRI data during the process of training. An alternative

approach is to use a cross-sectional approach to examine differ-
ences across individuals rather than within individuals [144]. For
example, brain activation differences between experts in a particu-
lar skill (e.g., long-term meditation practitioners, pianists) and
novices may be used to infer the neural effects of training to achieve
expertise. However, other confounding effects of differences
between cohorts are difficult to exclude (e.g., self-selection in per-
severing to achieve expertise), and a stronger inference for causa-
tion requires longitudinal, prospective studies.
The use of fMRI to define changes over time requires consider-
ation of certain additional methodological issues. Test–retest reliabil-
ity needs to be considered. Estimates of reliability depend on what is
being measured. For example, in statistical parametric mapping, the
question may be whether particular brain regions are stably labeled as
“active” or not in serial sessions. A handful of studies have addressed
this question. For example, one group showed that with a classifica-
tion learning task, scans 1 year apart resulted in highly concordant
results with defined regions of interest [145]. Another group showed
that maps obtained from a working memory task were similar across
time [146], but with a motor task, there appeared to be significant
variation over time in volume and spatial location of activation [147].
In longitudinal studies, sources of variability may be both phys-
iologic and nonphysiologic (e.g., MRI hardware). In some cases,
the magnitudes of activation in specific brain regions of interest are
themselves an outcome of interest. In these instances the stability of
BOLD signal measurements becomes an even more salient issue. It
may be worthwhile to utilize within-session indices that effectively
normalize parameters of interest. For example, rather than compar-
ing estimates of the magnitudes of activation, it may be worthwhile
utilizing an index of activity for one condition compared to a sec-
ond controlled condition with each session. An additional statistical
approach that could account for potential variability in SNR is to
combine data sets across sessions and then “whiten” the noise,
effectively normalizing noise contribution across sessions. Another
promising future direction is the use of quantitative techniques
such as arterial spin labeling (ASL), mentioned earlier in this chap-
ter, to help reduce nonphysiologic sources of variability. This type
of quantitative index may be particularly valuable in studies that
attempt to examine brain functioning longitudinally.
Other factors that concurrently change over time can produce
confounds to the interpretation of longitudinal studies. For exam-
ple, performance may change, resulting in changes in reaction time
or accuracy. All of these may alter measured responses making
determination of the neural bases of the process of interest, such as
a treatment intervention, more difficult. These and a number of
other theoretical issues are discussed by Poldrack in consideration
of learning-related (though not post-injury) changes [144].
348 Mark D’Esposito et al.

Other analytic approaches may be taken that are less sensitive

to nonphysiologic instabilities. For example, one could test for
changes in the spatial pattern of activation, which is not necessarily
affected by signal magnitude changes. For example, one could test
whether the patterns of activity are identical to within a scaling fac-
tor [108]. Furthermore, one could examine the more fundamental
measurement of the information coded within brain activity pat-
terns. These measurements may provide more informative indices
of particular neural process, while also being more robust for lon-
gitudinal studies.

9 Conclusions

Functional MRI is an extremely valuable tool for studying brain–

behavior relationships, as it is widely available, noninvasive, and has
superb temporal and spatial resolution. New approaches in fMRI
experimental design and data analysis continue to appear at an
almost exponential rate, leading to numerous options for testing
hypotheses on brain–behavior relationships. Combined with infor-
mation from complementary methods, such as the study of patients
with focal lesions, healthy individuals with TMS, pharmacological
interventions, or ERP, data from fMRI studies provide new insights
regarding the organization of the cerebral cortex as well as the
neural mechanisms underlying cognition. Moreover, cognitive
neuroscience approaches that have been developed for fMRI pro-
vide an excellent foundation for its use as a clinical tool.

References

1. Broca P (1861) Remarques sur le siege de la 7. Fuster JM, Alexander GE (1971) Neuron
faculte du langage articule suivies d’une activity related to short-term memory. Science
observation d’amphemie (perte de al parole). 173:652–654
Bull Mem Soc Anat Paris 36:330–357 8. Funahashi S, Bruce CJ, Goldman-Rakic PS
2. Buckner RL, Raichle ME, Petersen SE (1995) (1989) Mnemonic coding of visual space in
Dissociation of human prefrontal cortical the monkey’s dorsolateral prefrontal cortex.
areas across different speech production tasks J Neurophysiol 61:331–349
and gender groups. J Neurophysiol 9. Funahashi S, Bruce CJ, Goldman-Rakic PS
74(5):2163–2173 (1993) Dorsolateral prefrontal lesions and
3. Sarter M, Bernston G, Cacioppo J (1996) oculomotor delayed-response performance:
Brain imaging and cognitive neuroscience: evidence for mnemonic “scotomas”.
toward strong inference in attributing func- J Neurosci 13:1479–1497
tion to structure. Am Psychol 51:13–21 10. Watanabe T, Niki H (1985) Hippocampal
4. Gaffan D, Gaffan EA (1991) Amnesia in man unit activity and delayed response in the mon-
following transection of the fornix: a review. key. Brain Res 325(1–2):241–254
Brain 114:2611–2618 11. Cahusac PM, Miyashita Y, Rolls ET (1989)
5. Feeney DM, Baron JC (1986) Diaschisis. Responses of hippocampal formation neurons
Stroke 17(5):817–830 in the monkey related to delayed spatial
6. Carrera E, Tononi G (2014) Diaschisis: past, response and object-place memory tasks.
present, future. Brain 137:2408–2422 Behav Brain Res 33(3):229–240
 Functional MRI: Applications in Cognitive Neuroscience 349

12. Alvarez P, Zola-Morgan S, Squire LR (1994) ous electroencephalography/functional mag-

The animal model of human amnesia: long- netic resonance imaging study. J Neurosci
term memory impaired and short-term mem- 30(30):10243–10250
ory intact. Proc Natl Acad Sci U S A 27. Sadeh B, Podlipsky I, Zhdanov A, Yovel G
91(12):5637–5641 (2010) Event-related potential and functional
13. Corkin S (1984) Lasting consequences of MRI measures of face-selectivity are highly
bilateral medial temporal lobectomy: clinical correlated: a simultaneous ERP-fMRI investi-
course and experimental findings in gation. Hum Brain Mapp 31(10):1490–1501
H.M. Semin Neurol 4:249–259 28. Becker R, Reinacher M, Freyer F, Villringer
14. Ranganath C, D’Esposito M (2001) Medial A, Ritter P (2011) How ongoing neuronal
temporal lobe activity associated with active oscillations account for evoked fMRI variabil-
maintenance of novel information. Neuron ity. J Neurosci 31(30):11016–11027
31(5):865–873 29. Bergmann TO, Mölle M, Diedrichs J, Born J,
15. D’Esposito M (2010) Why methods matter in Siebner HR (2012) Sleep spindle-related
the study of the biological basis of the mind: a reactivation of category-specific cortical
behavioral neurologist’s perspective. In: regions after learning face-scene associations.
Reuter-Lorenz PA, Baynes K, Mangun GR, Neuroimage 59(3):2733–2742
Phelps EA (eds) The cognitive neursocience 30. Mantini D, Perrucci MG, Cugini S, Ferretti
of mind: a tribute to Michael Gazzaniga. MIT A, Romani GL, Del Gratta C (2007)
Press, Cambridge, MA Complete artifact removal for EEG recorded
16. Druzgal TJ, D’Esposito M (2001) Activity in during continuous fMRI using independent
fusiform face area modulated as a function of component analysis. Neuroimage
working memory load. Brain Res Cogn Brain 34:598–607
Res 10(3):355–364 31. Debener S, Strobel A, Sorger B et al (2007)
17. Henson R (2006) Forward inference using Improved quality of auditory event-related
functional neuroimaging: dissociations versus potentials recorded simultaneously with 3-T
associations. Trends Cogn Sci 10(2):64–69 fMRI: Removal of the ballistocardiogram
18. Cohen MS, Kosslyn SM, Breiter HC et al artefact. Neuroimage 34:587–597
(1996) Changes in cortical activity during 32. Moosmann M, Schönfelder VH, Specht K,
mental rotation: a mapping study using func- Scheeringa R, Nordby H, Hugdahl K (2009)
tional MRI. Brain 119:89–100 Realignment parameter-informed artefact
19. D’Esposito M, Ballard D, Aguirre GK, correction for simultaneous EEG-fMRI
Zarahn E (1998) Human prefrontal cortex is recordings. Neuroimage 45(4):1144–1150
not specific for working memory: a functional 33. Mullinger KJ, Yan WX, Bowtell R (2011)
MRI study. Neuroimage 8(3):274–282 Reducing the gradient artefact in simultane-
20. Poldrack RA (2006) Can cognitive processes ous EEG-fMRI by adjusting the subject’s
be inferred from neuroimaging data? Trends axial position. Neuroimage 54:1942–1950
Cogn Sci 10(2):59–63 34. Neuner I, Arrubla J, Felder J, Shah NJ (2014)
21. Grasby PM (2002) Imaging the neurochemi- Simultaneous EEG-fMRI acquisition at low,
cal brain in health and disease. Clin Med high and ultra-high magnetic fields up to
2(1):67–73 9.4T: perspectives and challenges.
22. Landau SM, Lal R, O'Neil JP, Baker S, Jagust Neuroimage 102(P1):71–79
WJ (2009) Striatal dopamine and working 35. Feredoes E, Heinen K, Weiskopf N, Ruff C,
memory. Cereb Cortex 19(2):445–454 Driver J (2011) Causal evidence for frontal
23. Blakemore SJ (2012) Imaging brain develop- involvement in memory target maintenance
ment: the adolescent brain. Neuroimage by posterior brain areas during distracter
61(2):397–406 interference of visual working memory. Proc
Natl Acad Sci U S A 108(42):17510–17515
24. Ritter P, Villringer A (2006) Simultaneous
EEG-fMRI. Neurosci Biobehav Rev 36. Ruff CC, Blankenburg F, Bjoertomt O,
30(6):823–838 Bestmann S, Freeman E, Haynes J, Driver
J (2006) Concurrent TMS-fMRI and psycho-
25. Jorge J, van der Zwaag W, Figueiredo P (2013) physics reveal frontal influences on human
EEG-fMRI integration for the study of human retinotopic visual cortex. Curr Biol
brain function. Neuroimage 102:24–34 16(15):1479–1488
26. Sadaghiani S, Scheeringa R, Lehongre K, 37. Chen AC, Oathes DJ, Chang C, Bradley T,
Morillon B, Giraud A-L, Kleinschmidt A Zhou Z-W, Williams LM, Etkin A (2013)
(2010) Intrinsic connectivity networks, alpha Causal interactions between fronto-parietal
oscillations, and tonic alertness: a simultane-
350 Mark D’Esposito et al.

central executive and default-mode networks 51. Besle J, Sanchez-Panchuelo R, Bowtell R, Francis
in humans. Proc Natl Acad Sci U S A S, Schluppeck D (2014) Event-related fMRI at 7
110(49):19944–19949 T reveals overlapping cortical representations for
38. Yau JM, Hua J, Liao DA, Desmond JE (2013) adjacent fingertips in S1 of individual subjects.
Efficient and robust identification of cortical Hum Brain Mapp 35:2027–2043
targets in concurrent TMS-fMRI experi- 52. Ehses P, Bause J, Shajan G, Scheffler
ments. NeuroImage 76:134–144 K. Efficient generation of T2*-weighted con-
39. Gnadt JW, Andersen RA (1988) Memory trast by interslice echo-shifting for human
related motor planning activity in posterior functional and anatomacil imaging at 9.4
parietal cortex of macaque. Exp Brain Res Tesla. Magn Reson Med (epub ahead of print)
70:216–220 53. Budde J, Shajan G, Zaitsev M, Scheffler K,
40. Aguirre GK, Zarahn E, D’Esposito M (1998) Functional PR, MRI (2014) Human subjects
The variability of human, BOLD hemody- with gradient-echo and spin-echo EPI at 9.4
namic responses. Neuroimage 8(4):360–369 T. Magn Reson Med 2014(71):209–218
41. Handwerker DA, Ollinger JM, D’Esposito M 54. Malonek D, Grinvald A (1996) Interactions
(2004) Variation of BOLD hemodynamic between electrical activity and cortical micro-
responses across brain regions and subjects circulation revealed by imaging spectroscopy:
and their effects on statistical analyses. implications for functional brain mapping.
NeuroImage 21:1639–1651 Science 272:551–554
42. Boynton GM, Engel SA, Glover GH, Heeger 55. Kim SG, Duong TQ (2002) Mapping cortical
DJ (1996) Linear systems analysis of func- columnar structures using fMRI. Physiol
tional magnetic resonance imaging in human Behav 77(4–5):641–644
V1. J Neurosci 16:4207–4221 56. Grill-Spector K, Malach R (2001) fMR-
43. Kim SG, Richter W, Ugurbil K (1997) adaptation: a tool for studying the functional
Limitations of temporal resolution in properties of human cortical neurons. Acta
fMRI. Magn Reson Med 37:631–636 Psychol (Amst) 107(1–3):293–321
44. Savoy RL, Bandettini PA, O’Craven KM et al 57. Grill-Spector K, Kushnir T, Edelman S,
(1995) Pushing the temporal resolution of Avidan G, Itzchak Y, Malach R (1999)
fMRI: studies of very brief stimuli, onset of Differential processing of objects under vari-
variability and asynchrony, and stimulu- ous viewing conditions in the human lateral
correlated changes in noise. Proc Soc Magn occipital complex. Neuron 24(1):187–203
Reson Med 3:450 58. Posner MI, Petersen SE, Fox PT, Raichle ME
45. Zarahn E, Aguirre GK, D’Esposito M (1997) (1988) Localization of cognitive operations
A trial-based experimental design for func- in the human brain. Science 240:1627–1631
tional MRI. NeuroImage 6:122–138 59. Sternberg S (1969) The discovery of process-
46. Burock MA, Buckner RL, Woldorff MG, ing stages: extensions of Donders’ method.
Rosen BR, Dale AM (1998) Randomized Acta Psychol 30:276–315
event-related experimental designs allow for 60. Petersen SE, Fox PT, Posner MI, Mintun M,
extremely rapid presentation rates using func- Raichle ME (1988) Positron emission tomo-
tional MRI. Neuroreport 9(16):3735–3739 graphic studies of the cortical anatomy of sin-
47. Clark VP, Maisog JM, Haxby JV (1997) gle word processing. Nature 331:585–589
fMRI studies of visual perception and recog- 61. Fuster J (1997) The prefrontal cortex: anat-
nition using a random stimulus design. Soc omy, physiology, and neuropsychology of the
Neurosci Abstr 23:301 frontal lobes, 3rd edn. Raven, New York
48. Dale AM, Buckner RL (1997) Selective aver- 62. Jonides J, Smith EE, Koeppe RA, Awh E,
aging of rapidly presented individual trials Minoshima S, Mintun MA (1993) Spatial
using fMRI. Hum Brain Mapp 5:1–12 working memory in humans as revealed by
49. Miezin FM, Maccotta L, Ollinger JM, PET. Nature 363:623–625
Petersen SE, Buckner RL (2000) 63. Sreenivasan KK, Curtis CE, D’Esposito M
Characterizing the hemodynamic response: (2014) Revising the role of persistent neural
effects of presentation rate, sampling proce- activity in working memory. Trends Cogn Sci
dure, and the possibility of ordering brain 18:82–89
activity based on relative timing. Neuroimage 64. Attwell D, Iadecola C (2002) The neural
11(6 Pt 1):735–759 basis of functional brain imaging signals.
50. D’Esposito M, Zarahn E, Aguirre GK (1999) Trends Neurosci 25(12):621–625
Event-related functional MRI: implications 65. Heeger DJ, Ress D (2002) What does fMRI
for cognitive psychology. Psychol Bull tell us about neuronal activity? Nat Rev
125:155–164 Neurosci 3(2):142–151
 Functional MRI: Applications in Cognitive Neuroscience 351

66. Friston KJ, Josephs O, Rees G, Turner R 81. Zarahn E, Slifstein M (2001) A reference
(1998) Nonlinear event-related responses in effect approach for power analysis in
fMRI. Magn Reson Med 39(1):41–52 fMRI. Neuroimage 14(3):768–779
67. Glover GH (1999) Deconvolution of impulse 82. Van Horn JD, Ellmore TM, Esposito G,
response in event-related BOLD Berman KF (1998) Mapping voxel-based sta-
fMRI. Neuroimage 9(4):416–429 tistical power on parametric images.
68. Miller KL, Luh WM, Liu TT et al (2001) Neuroimage 7(2):97–107
Nonlinear temporal dynamics of the cerebral 83. Aguirre GK, D’Esposito M (1999)
blood flow response. Hum Brain Mapp Experimental design for brain fMRI. In:
13(1):1–12 Moonen CTW, Bandettini PA (eds) Functional
69. Vazquez AL, Noll DC (1998) Nonlinear MRI. Springer, Berlin, pp 369–380
aspects of the BOLD response in functional 84. Logothetis NK, Pauls J, Augath M, Trinath
MRI. NeuroImage 7(2):108–118 T, Oeltermann A (2001) Neurophysiological
70. D’Esposito M, Zarahn E, Aguirre GK, Rypma investigation of the basis of the fMRI signal.
B (1999) The effect of normal aging on the Nature 412(6843):150–157
coupling of neural activity to the bold hemo- 85. Farkas E, Luiten PG (2001) Cerebral micro-
dynamic response. Neuroimage 10(1):6–14 vascular pathology in aging and Alzheimer’s
71. Rosen BR, Buckner RL, Dale AM (1998) disease. Prog Neurobiol 64(6):575–611
Event-related functional MRI: past, present, 86. Fang HCH (1976) Observations on aging
and future. Proc Natl Acad Sci U S A characteristics of cerebral blood vessels, mac-
95(3):773–780 roscopic and microscopic features. In: Gerson
72. Donaldson DI, Petersen SE, Ollinger JM, S, Terry RD (eds) Neurobiology of aging.
Buckner RL (2001) Dissociating state and Raven, New York
item components of recognition memory 87. Bentourkia M, Bol A, Ivanoiu A et al (2000)
using fMRI. Neuroimage 13(1):129–142 Comparison of regional cerebral blood flow
73. Mitchell KJ, Johnson MK, Raye CL, and glucose metabolism in the normal brain:
D’Esposito M (2000) fMRI evidence of age- effect of aging. J Neurol Sci 181(1–2):19–28
related hippocampal dysfunction in feature 88. Schultz SK, O’Leary DS, Boles Ponto LL,
binding in working memory. Brain Res Cogn Watkins GL, Hichwa RD, Andreasen NC
Brain Res 10(1–2):197–206 (1999) Age-related changes in regional cere-
74. Keppel G, Zedeck S (1989) Data analysis for bral blood flow among young to mid-life
research design. W.H. Freeman, New York adults. Neuroreport 10(12):2493–2496
75. Worsley KJ, Friston KJ (1995) Analysis of 89. Yamamoto M, Meyer JS, Sakai F, Yamaguchi
fMRI time-series revisited – again. F (1980) Aging and cerebral vasodilator
Neuroimage 2:173–182 responses to hypercarbia: responses in normal
76. Nichols T, Hayasaka S (2003) Controlling aging and in persons with risk factors for
the familywise error rate in functional neuro- stroke. Arch Neurol 37(8):489–496
imaging: a comparative review. Stat Methods 90. Yamaguchi T, Kanno I, Uemura K et al (1986)
Med Res 12(5):419–446 Reduction in regional cerebral rate of oxygen
77. Eklund A, Andersson M, Josephson C, during human aging. Stroke 17:1220–1228
Johannesson M, Knutsson H (2012) Does 91. Takada H, Nagata K, Hirata Y et al (1992)
parametric fMRI analysis with SPM yield valid Age-related decline of cerebral oxygen metab-
results? An empirical study of 1484 rest data- olism in normal population detected with
sets. Neuroimage 61(3):565–578 positron emission tomography. Neurol Res
78. Zarahn E, Aguirre GK, D’Esposito M (1997) 14(2 Suppl):128–131
Empirical analyses of BOLD fMRI statistics. I 92. Claus JJ, Breteler MM, Hasan D et al (1998)
Spatially unsmoothed data collected under Regional cerebral blood flow and cerebrovas-
null-hypothesis conditions. NeuroImage cular risk factors in the elderly population.
5:179–197 Neurobiol Aging 19(1):57–64
79. Aguirre GK, Zarahn E, D’Esposito M (1997) 93. Cunnington R, Iansek R, Bradshaw JL,
Empirical analyses of BOLD fMRI statistics. Phillips JG (1995) Movement-related poten-
II Spatially smoothed data collected under tials in Parkinson’s disease. Presence and pre-
null-hypothesis and experimental conditions. dictability of temporal and spatial cues. Brain
NeuroImage 5:199–212 118(Pt 4):935–950
80. D’Esposito M, Ballard D, Zarahn E, Aguirre 94. Buckner RL, Snyder AZ, Sanders AL, Raichle
GK (2000) The role of prefrontal cortex in ME, Morris JC (2000) Functional brain
sensory memory and motor preparation: an imaging of young, nondemented, and
event-related fMRI study. Neuroimage 11(5 demented older adults. J Cogn Neurosci
Pt 1):400–408 12(Suppl 2):24–34
352 Mark D’Esposito et al.

95. Huettel SA, Singerman JD, McCarthy G brain activity associated with memory storage
(2001) The effects of aging upon the hemo- and search. Neuroimage 33(2):794–804
dynamic response measured by functional 109. Kay KN, Naselaris T, Prenger RJ, Gallant JG
MRI. Neuroimage 13(1):161–175 (2008) Identifying natural images from
96. Pineiro R, Pendlebury S, Johansen-Berg H, human brain activity. Nature 452:352–355
Matthews PM (2002) Altered hemodynamic 110. Haxby JV, Connolly AC, Swaroop GJ (2014)
responses in patients after subcortical stroke Decoding neural representational spaces
measured by functional MRI. Stroke using multivariate pattern analysis. Annu Rev
33(1):103–109 Neurosci 37:435–456
97. D’Esposito M, Deouell L, Gazzaley A (2003) 111. Tong F, Pratte MS (2012) Decoding patterns
Alterations in the BOLD fMRI signal with of human brain activity. Annu Rev Psychol
ageing and disease: a challenge for neuroim- 63:483–509
aging. Nat Rev Neurosci 4:863–872 112. Todd MT, Nystrom LE, Cohen JE (2013)
98. Handwerker DA, Gazzaley A, Inglis BA, Confounds in multivariate pattern analysis:
D’Esposito M (2006) Reducing vascular vari- theory and rule representation case study.
ability of fMRI data across aging populations NeuroImage 77:157–165
using a breath holding task. Hum Brain Mapp 113. Chen AJW, Britton MS, Thompson TW,
28:846–859 Turner GR, Vytlacil J, D’Esposito M (2012)
99. Wolf RL, Detre JA (2007) Clinical neuroimag- Goal-directed attention alters the tuning of
ing using arterial spin-labeled perfusion mag- object-based representations in extrastriate
netic resonance imaging. Neurotherapeutics cortex. Front Neurosci 6:187
4(3):346–359 114. Çukur TNishimoto S, Huth A, Gallant
100. Brown GG, Clark C, Liu TT (2007) J (2013) Attention during natural vision
Measurement of cerebral perfusion with arte- warps semantic representation across
rial spin labeling. Part 2. Applications. J Int the human brain. Nat Neurosci
Neuropsychol Soc 13(3):526–538 16:763–770
101. Aguirre GK, Detre JA, Zarahn E, Alsop DC 115. Buchel C, Coull JT, Friston KJ (1999) The
(2002) Experimental design and the relative predictive value of changes in effective con-
sensitivity of BOLD and perfusion nectivity for human learning. Science
fMRI. Neuroimage 15(3):488–500 283(5407):1538–1541
102. Liu TT, Brown GG (2007) Measurement of 116. McIntosh AR, Grady CL, Haxby JV,
cerebral perfusion with arterial spin labeling. Ungerleider LG, Horwitz B (1996) Changes
Part 1. Methods. J Int Neuropsychol Soc in limbic and prefrontal functional interac-
13(3):517–525 tions in a working memory task for faces.
103. Fernandez-Seara MA, Wang J, Wang Z et al Cereb Cortex 6(4):571–584
(2007) Imaging mesial temporal lobe activa- 117. Gerstein GL, Perkel DH, Subramanian KN
tion during scene encoding: comparison of (1978) Identification of functionally related
fMRI using BOLD and arterial spin labeling. neural assemblies. Brain Res 140(1):43–62
Hum Brain Mapp 28(12):1391–1400 118. Penny WD, Stephan KE, Mechelli A, Friston
104. Feinberg DA, Beckett A, Chen L (2013) KJ (2004) Modelling functional integration:
Arterial spin labeling with simultaneous a comparison of structural equation and
multi-slice echo planar imaging. Magn Reson dynamic causal models. Neuroimage 23(Suppl
Med 70(6):1500–1506 1):S264–S274
105. Kanwisher N, McDermott J, Chun MM 119. Sun FT, Miller LM, D’Esposito M (2004)
(1997) The fusiform face area: a module in Measuring interregional functional con-
human extrastriate cortex specialized for face nectivity using coherence and partial coher-
perception. J Neurosci 17:4302–4311 ence analyses of fMRI data. Neuroimage
106. Haxby JV, Gobbini MI, Furey ML, Ishai A, 21(2):647–658
Schouten JL, Pietrini P (2001) Distributed 120. Sun FT, Miller LM, D’Esposito M (2005)
and overlapping representations of faces and Measuring temporal dynamics of functional
objects in ventral temporal cortex. Science networks using phase spectrum of fMRI data.
293(5539):2425–2430 Neuroimage 28(1):227–237
107. Polyn SM, Natu VS, Cohen JD, Norman KA 121. Sun FT, Miller LM, Rao AA, D’Esposito M
(2005) Category-specific cortical activity pre- (2007) Functional connectivity of cortical net-
cedes retrieval during memory search. Science works involved in bimanual motor sequence
310(5756):1963–1966 learning. Cereb Cortex 17(5):1227–1234
108. Zarahn E, Rakitin BC, Abela D, Flynn J, 122. Gazzaley A, Rissman J, D’Esposito M (2004)
Stern Y (2006) Distinct spatial patterns of Functional connectivity during working
 Functional MRI: Applications in Cognitive Neuroscience 353

memory maintenance. Cogn Affect Behav 136. Gibbs SE, D’Esposito M (2005) Individual
Neurosci 4(4):580–599 capacity differences predict working memory
123. Fuhrmann Alpert G, Sun FT, Handwerker performance and prefrontal activity following
D, D’Esposito M, Knight RT (2007) Spatio- dopamine receptor stimulation. Cogn Affect
temporal information analysis of event- Behav Neurosci 5(2):212–221
related BOLD responses. Neuroimage 137. Gibbs SE, D’Esposito M (2005) A func-
34(4):1545–1561 tional MRI study of the effects of bro-
124. Rees G, Frith CD, Lavie N (1997) mocriptine, a dopamine receptor agonist, on
Modulating irrelevant motion perception by component processes of working memory.
varying attentional load in an unrelated task. Psychopharmacology (Berl) 180(4):644–653
Science 278(5343):1616–1619 138. Gibbs SE, D’Esposito M (2006) A func-
125. Treisman AM (1969) Strategies and mod- tional magnetic resonance imaging study of
els of selective attention. Psychol Rev the effects of pergolide, a dopamine receptor
76(3):282–299 agonist, on component processes of working
126. Lavie N, Tsal Y (1994) Perceptual load as memory. Neuroscience, 139:359–71
a major determinant of the locus of selec- 139. Cools R, Sheridan M, Jacobs E, D’Esposito M
tion in visual attention. Percept Psychophys (2007) Impulsive personality predicts dopa-
56(2):183–197 mine-dependent changes in frontostriatal
127. McCarthy RA, Warrington EK (1994) activity during component processes of work-
Disorders of semantic memory. Philos Trans ing memory. J Neurosci 27(20):5506–5514
R Soc Lond B Biol Sci 346(1315):89–96 140. Kastner S, Pinsk MA (2004) Visual attention
128. Warrington EST (1984) Category specific as a multilevel selection process. Cogn Affect
semantic impairments. Brain 107:829–854 Behav Neurosci 4(4):483–500
129. Thompson-Schill SL (2003) Neuroimaging 141. Gazzaley A, Cooney JW, McEvoy K, Knight
studies of semantic memory: inferring RT, D’Esposito M (2005) Top-down
“how” from “where”. Neuropsychologia enhancement and suppression of the mag-
41(3):280–292 nitude and speed of neural activity. J Cogn
Neurosci 17(3):507–517
130. Thompson-Schill SL, D’Esposito M, Aguirre
GK, Farah MJ (1997) Role of left inferior pre- 142. Arnemann KL, Chen AJW, Novakovic-
frontal cortex in retrieval of semantic knowl- Agopian T, Gratton C, Nomura EM,
edge: a reevaluation. Proc Natl Acad Sci U S D'Esposito M (2015) Functional brain net-
A 94(26):14792–14797 work modularity predicts response to cogni-
tive training after brain injury. Neurology
131. Thompson-Schill SL, Swick D, Farah MJ, 84:1568–74
D’Esposito M, Kan IP, Knight RT (1998)
Verb generation in patients with focal frontal 143. Chen AJW, Novakovic-Agopian T, Nycum
lesions: a neuropsychological test of neuro- TJ, Song S, Turner G, Rome S, Abrams G,
imaging findings. Proc Natl Acad Sci U S A D’Esposito M (2011) Training of goal-
95(26):15855–15860 directed attention regulation enhances con-
trol over neural processing for individuals
132. Pascual-Leone A, Tarazona F, Keenan J, with brain injury. Brain 134(5):1541–1554
Tormos JM, Hamilton R, Catala MD (1999)
Transcranial magnetic stimulation and neuro- 144. Poldrack RA (2000) Imaging brain plasticity:
plasticity. Neuropsychologia 37(2):207–217 conceptual and methodological issues – a the-
oretical review. Neuroimage 12(1):1–13
133. Rushworth MF, Hadland KA, Paus T, Sipila
PK (2002) Role of the human medial fron- 145. Aron AR, Gluck MA, Poldrack RA (2006)
tal cortex in task switching: a combined Long-term test-retest reliability of functional
fMRI and TMS study. J Neurophysiol MRI in a classification learning task.
87(5):2577–2592 Neuroimage 29(3):1000–1006
134. Ruff CC, Bestmann S, Blankenburg F et al 146. Wei X, Yoo SS, Dickey CC, Zou KH,
(2008) Distinct causal influences of parietal Guttmann CR, Panych LP (2004) Functional
versus frontal areas on human visual cortex: MRI of auditory verbal working memory:
evidence from concurrent TMS fMRI. Cereb long-term reproducibility analysis.
Cortex 18(4):817–827 Neuroimage 21(3):1000–1008
135. Dehaene S, Spelke E, Pinel P, Stanescu R, 147. Yoo SS, Wei X, Dickey CC, Guttmann CR,
Tsivkin S (1999) Sources of mathematical Panych LP (2005) Long-term reproducibility
thinking: behavioral and brain-imaging evi- analysis of fMRI using hand motor task. Int
dence. Science 284(5416):970–974 J Neurosci 115(1):55–77
 Chapter 12

fMRI of Language Systems

Jeffrey R. Binder

Abstract
Language refers to the uniquely human capacity for communication through productive combination of
symbolic representations. Functional neuroimaging studies have in recent decades greatly expanded our
knowledge of the brain systems supporting language, producing a dramatic reawakening of interest in this
topic and a call to revise and extend the nineteenth century neuroanatomical model formulated by Broca,
Wernicke, and others. This chapter presents some theoretical issues regarding functional imaging of lan-
guage systems, a model of the functional neuroanatomy of language based on recent empirical results in
several selected processing domains, and a survey of language mapping paradigms in common clinical use.
A central theme is that interpretation of fMRI language studies depends on an informed analysis of the
cognitive processes engaged during scanning. This analytic approach can help avoid common pitfalls in
task design that limit the sensitivity and specificity of language mapping studies and should encourage the
development of a standardized methodological and conceptual framework for such studies.

Key words fMRI, Language, Semantics, Phonology, Orthography

1 Language and Language Processes

The central role of language in human culture and social interac-

tion is self-evident. In addition to providing a formal system for
overt communication, the symbolic structures of language enable
such uniquely human cognitive capacities as the ability to manipu-
late concepts, plan the future, and invent technology. Scientific
investigation of the neural basis of language began in earnest with
the work of Broca, Wernicke, and other nineteenth-century neu-
rologists, leading to the classical Wernicke-Lichtheim model of lan-
guage and aphasia that remains with us today [1–4]. Over the past
two decades, however, functional imaging methods, particularly
fMRI, have greatly expanded our knowledge of the brain systems
supporting language, producing a dramatic reawakening of inter-
est in this topic and a call to revise and extend the classical model
[5, 6]. This chapter provides a brief survey of some of this work,
together with a discussion of theoretical issues central to the design
and interpretation of language mapping studies. The goal is to

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_12, © Springer Science+Business Media New York 2016

355
356 Jeffrey R. Binder

provide a basic theoretical foundation and practical suggestions for

designing effective and interpretable clinical protocols.
What is language? One definition often cited is that language
processes are those that enable communication. In biological
terms, however, this definition is overly inclusive, in that many
bodily functions (e.g., cardiac, pulmonary, general arousal, and
sustained attention systems) provide critical support for communi-
cation but are not linguistic in nature. Communication typically
requires neural systems that process auditory or visual sensory
information, hold this information in a short-term store, direct
attention to specific features or aspects of the information, perform
comparisons and other general operations on the information,
select a response based on such operations, and carry out the
response. The extent to which any of these systems is specialized
for use in language behavior is a matter of debate. Careful consid-
eration of these domain-general systems is especially relevant for
interpreting and designing language mapping studies, which often
employ relatively complex tasks that engage motor, sensory, atten-
tion, memory, and “central executive” functions in addition to lan-
guage. Should these other components be considered part of the
language system because they are so critical for adequate task per-
formance, or should they be delineated from language processes
per se? In this chapter, I assume that the goal of language mapping
is to identify neural systems involved specifically in language pro-
cesses, i.e., to distinguish these brain networks from early sensory,
motor, and general executive systems.
A more precise definition of language is that it is a system of
communication based on the symbolic representation and manipu-
lation of information. Languages are also, by definition, generative,
in that the symbols of a language can be productively combined to
make a virtually limitless number of new expressions. In formulat-
ing a general definition of this kind, however, it is critical to keep
in mind that language is not a unitary process, but rather a collec-
tion of processes operating at distinct levels and on distinct types of
information. Clinicians working with aphasic patients historically
have focused on the dichotomy between “expressive” and “recep-
tive” language functions, but a more useful taxonomy of compo-
nent language processes is available from the field of linguistics.
For spoken languages, these processes include: (1) phoneme per-
ception, the processes serving recognition of speech sounds; (2)
phonology, the processes by which speech sounds are represented
and manipulated in abstract form; (3) speech articulation, the pro-
cesses by which speech movements are planned and executed; (4)
orthography, the processes by which written characters are repre-
sented and manipulated in abstract form; (5) semantics, the pro-
cessing of word meanings, names, and other declarative knowledge
about the world; and (6) syntax, the processes by which words are
combined to make sentences and sentences analyzed to reveal
underlying relationships between words. A basic assumption of
 fMRI of Language Systems 357

language mapping is that activation tasks can be designed to make

varying demands on these processing subsystems. For example, a
task requiring careful listening to word-like nonwords (called pseu-
dowords, e.g., “nurdle”) would make great demands on phoneme
perception (and on pre-phonetic auditory processing and attention)
but very little demand on semantic or syntactic processing, given
that the stimuli have no (or very little) meaning. In contrast, a task
requiring semantic categorization of printed words (e.g., Is it an
animal or not?) would make great demands on orthographic and
semantic processing but relatively little on phonetic, phonological,
or syntactic processing.
On the other hand, the processing subcomponents of language
often act together. The extent to which each component can be
examined in isolation remains a major methodological issue, as it is
not yet clear to what extent the systems responsible for these pro-
cesses become active “automatically” when presented with linguis-
tic stimuli [7]. One familiar example of this interaction is the
Stroop effect, in which orthographic and phonological processing
of printed words occurs even when subjects are instructed to attend
to the color of the print, and even when this processing interferes
with task performance [8]. Other examples include semantic prim-
ing effects during word recognition, picture–word interference
effects, lexical effects on phonetic perception, orthographic effects
on letter perception, and semantic–syntactic interactions during
sentence comprehension [9–17]. If linguistic stimuli such as words
and pictures evoke obligatory, automatic language processing,
these effects need to be considered in the design and interpretation
of language activation experiments. Use of such stimuli in a “base-
line” condition could result in undesirable subtraction (or partial
subtraction) of language-related activation. Because investigators
frequently try to match stimuli in control and language tasks very
closely, such inadvertent subtraction is relatively commonplace in
functional imaging studies of language processing.
A final theoretical issue is the extent to which language pro-
cesses occur during “resting” states or states with minimal task
requirements (e.g., visual fixation or “passive” stimulation).
Language involves interactive systems for manipulating internally
stored knowledge about words and word meanings. In examining
these systems we typically use familiar stimuli or cues to engage
processing, yet it seems likely that activity in these systems could
occur independently of external stimulation and task demands.
The idea that the conscious mind can be internally active indepen-
dent of external events has a long history in psychology and neuro-
science [18–22]. When asked, subjects in experimental studies
frequently report experiencing seemingly unprovoked thoughts
(including words and recognizable images) that are unrelated to
the task at hand [21, 23–26]. The extent to which such “thinking”
engages linguistic knowledge remains unclear [27, 28], but many
researchers have demonstrated close parallels between behavior
358 Jeffrey R. Binder

and language content, suggesting that at least some internal

thought processes make use of verbally encoded semantic knowl-
edge and other linguistic representations [27, 29, 30]. Many
authors have argued that “rest” and similar conditions are actually
active states in which subjects frequently are engaged in processing
linguistic and other information [25, 26, 31–38]. Use of such
states as control conditions for language imaging studies may thus
obscure similar processes that occur during the language task of
interest. This is a particularly difficult problem for language studies
because the internal processes in question cannot be directly mea-
sured or precisely controlled.

2 Functional Neuroanatomy of Component Language Systems

Considering that neuroimaging studies on language processing

now number in the thousands, the following review is inevitably
incomplete and somewhat cursory. Nor is it possible to cover every
topic that might be of interest. This review focuses on single word
and sublexical processes, with less attention to studies of sentence
comprehension and syntax. Several interesting topics, including
bilingualism and sign language processing, are not touched on
here; the interested reader is referred to reviews on these topics
[39–48]. A visual summary of the following discussion is provided
in Fig. 1.

2.1 Phoneme Traditional clinical models of aphasia often treat comprehension of

Perception speech as a single function [49], but it is important to distinguish at
least two processes engaged during speech comprehension. Spoken
words not only possess meanings, they are also composed of very
complex and rapidly changing auditory signals. Thus, useful models
of auditory word recognition include not only a semantic stage in
which words are mapped onto their meanings, but also a stage prior
to semantic access in which consonant and vowel sounds—known
collectively as phonemes—are identified. The distinction between
these stages becomes clearer if one considers the differences between
listening to a tone (for example, a note played on a piano), a pseu-
doword (such as “dap”), and a word (such as “tap”). The tone has
no phonemic value; it cannot be identified as any vowel or conso-
nant. In contrast, the pseudoword “dap” contains three phonemes—
/d/,/æ/,/p/—although it has no meaning. Finally, the word “tap”
conveys both phonemic and semantic information. The importance
of the phoneme perception stage is illustrated by the fact that “dap”
and “tap” differ at a physical level only in the presence of a brief
(typically 20–30 ms) noise at the beginning of “tap” but not “dap,”
produced by release of the tongue from the roof of the mouth
slightly prior to the onset of vocal cord vibration. These and many
other subtle acoustic cues must be rapidly and continuously
 fMRI of Language Systems 359

Fig. 1 A schematic model of some major language regions and networks. Yellow indicates a bilaterally repre-
sented phoneme (speech sound) perception system. Blue indicates the posterior perisylvian area (posterior
superior temporal and supramarginal gyri, roughly equivalent to the traditional Wernicke area), which supports
pre-articulatory phonological access. Red indicates the temporal and parietal components of a distributed sys-
tem that stores and processes word meaning (semantic memory) information. ORANGE indicates several pre-
frontal components of the semantic processing network, including the pars orbitalis of the inferior frontal gyrus
(IFGpo) and the dorsomedial prefrontal cortex (DMPFC), which are proposed to control the activation and selec-
tion of information in the posterior semantic memory store. GREEN indicates a more general language control
system, made up of the pars triangularis and opercularis of the IFG (roughly equivalent to the traditional Broca
area) and adjacent cortex in the inferior frontal sulcus, which is proposed to control the retrieval and mainte-
nance of phonological information, a process that is critical for word retrieval, verbal working memory, and
sentence production. Speech repetition requires the pathway designated A in the figure, linking phoneme per-
ception with phonological access systems, as well as more anterior sensorimotor regions (not shown) that
support articulatory preparation and execution. Spoken word comprehension involves pathway B in the figure,
which maps perceived phoneme sequences to word concepts. Communicative speech production, in which the
speaker retrieves words and formulates sentences to express concepts, requires control of the semantic system
by the pathways marked D, as well as pathway C, which maps concept representations onto phonological rep-
resentations, and pathway E, which controls and maintains the activation of phonological codes. Pathway F
indicates a direct mapping from visual word forms to phonological representations, required for reading aloud

processed in real time for accurate speech comprehension to occur.

The absence of this important linguistic process in traditional clinical
models of language can be attributed to at least two factors. First,
very little was known about the physical acoustic properties of pho-
nemes prior to the mid-twentieth century; the scientific study of
speech perception has developed only in the last 60–70 years.
Second, cases of isolated phoneme perception difficulty, known as
pure word deafness, are rare, resulting in a relative lack of familiarity
with this field of study on the part of many clinicians.
Over the past several decades, scientists using functional neu-
roimaging methods have identified a region in the superior tempo-
ral lobes that responds more strongly to speech than to nonspeech
sounds like tones and noise [50–57]. These speech-related
activations are found consistently in the middle portion of the
superior temporal sulcus (STS, see yellow region in Fig. 1), i.e., the
sulcus separating the superior temporal gyrus (STG) from the mid-
dle temporal gyrus (MTG). This activation is often found in both
360 Jeffrey R. Binder

the left and right STS, though usually with leftward lateralization.
These activations are identical whether the stimuli are words or
word-like pseudowords, thus they reflect processing of phonemes
and not word meaning [54]. Though some of the activation in this
region could be explained by the fact that speech sounds are more
acoustically complex than the tones and noises used as nonspeech
controls, more recent experiments using acoustically matched
speech and nonspeech sounds (e.g., rotated speech, sinewave
speech) have shown convincingly that at least some of the activa-
tion in this region is due specifically to activation of phoneme codes
[58–63].
These observations are fully consistent with localization data
from patients with pure word deafness, who typically have lesions
restricted to the STG and STS [64–70]. Most of these cases have
bilateral lesions, though rarely a large left temporal lobe lesion can
produce the syndrome [71, 72]. These patients show impairments
in recognizing speech phonemes and may have other deficits of
higher-order auditory perception, especially when the lesions are
bilateral, but they have no deficits in written comprehension, nam-
ing, or propositional speech production that would indicate any loss
of word concepts. Taken together, these functional imaging and
lesion data make it clear that the left STG plays a relatively specific
role in language processing, i.e., that it contains general auditory
systems and specialized networks for recognizing speech phonemes,
regardless of whether these phonemes form words or have meaning.
This conceptualization stands in stark contrast to the traditional clin-
ical model of aphasia, which identifies the left STG as “Wernicke’s
area,” the principal site for “language comprehension.”

2.2 Grapheme Traditional neuroanatomical models of written word recognition

Perception and derive mainly from the late 19th century descriptions by Déjerine of
Orthographic alexia with and without agraphia [73–75]. According to these
Processing models, visual perception of letters occurs in the primary visual
cortex of both hemispheres, which then transmit this information
to the left angular gyrus, where “memories” of written words are
activated. Lesions of the left angular gyrus destroy these visual
word codes, producing both inability to read and inability to write.
Alexia without agraphia (also known as pure alexia, peripheral
alexia, or letter-by-letter reading) results when an occipital lobe
lesion destroys both the left visual cortex and the decussating white
matter pathway from right visual cortex to left angular gyrus,
effectively disconnecting the angular gyrus from visual input with-
out destruction of the visual word codes themselves [76].
In recent decades it has become clear, however, that pure alexia
can result from ventral occipital–temporal lesions that damage nei-
ther the primary visual cortex nor the angular gyrus. Though initially
ascribed to involvement of white matter pathways projecting to the
angular gyrus [77–79], it is now clear that most of these cases are due
to focal damage to the left ventral occipital-temporal cortex,
 fMRI of Language Systems 361

particularly the mid-portion of the left fusiform gyrus [80–84]. Thus,

normal reading requires the participation of a left-lateralized visual
association area in or near the mid-fusiform gyrus, which receives
input from earlier visual processing stages in both hemispheres.
Functional neuroimaging research has strongly confirmed this
model. Numerous studies have demonstrated a focal region in the
lateral left fusiform gyrus that responds more strongly to words and
word-like pseudowords than to consonant letter strings or non-
sense characters [85–92]. This focal region of cortex has conse-
quently been named the “visual word form area.” Activation in this
area increases as a direct function of how frequently the letter com-
binations comprising the stimulus occur in the reader’s language,
and this activation is also correlated with efficiency of letter percep-
tion during tachistoscopic presentation [93]. It thus appears that
during the many hours spent learning to read fluently, neurons in
this region become “tuned” to detect familiar letter combinations,
resulting in a high degree of perceptual expertise that allows multi-
letter fragments and even whole words to be processed in parallel.
Destruction of these “expert” neurons prevents the patient from
recognizing multiletter fragments efficiently, forcing the adoption
of a much slower, letter-by-letter decoding process [94–96].

2.3 Phonological The term paraphasia refers to speech production that is fluent but
Access and contains errors, such as substitution of incorrect phonemes or words,
Phonological Working or rearrangement of the order of phonemes within a word. Paraphasia
Memory is characteristic of many forms of aphasia, particularly the Wernicke
and conduction syndromes, and typically affects both spoken and
written output. Paraphasia indicates an inability to retrieve or properly
use a mental representation of word sounds—what nineteenth cen-
tury theorists called “sound images” and what in modern parlance are
referred to as phonological representations. Patients who cannot access
(i.e., activate, compute) correct phonological representations show
paraphasic errors on all speech output tasks, including conversing,
naming objects, reading aloud, and repeating, as well as on a variety of
other tasks that require phonological access. For example, patients
with phonological impairments may be unable to determine whether
two printed words rhyme. Writing normally involves a mapping from
phonological (sound-based) to orthographic (grapheme-based) rep-
resentations, which is why patients with impaired phonology also typi-
cally show paraphasic errors in their writing.
The brain regions most strongly implicated in phonological access
(see blue region in Fig. 1) are in the left posterior perisylvian area,
especially the posterior STG, posterior STS, and supramarginal gyrus
(SMG). For example, patients with conduction aphasia—a relatively
isolated disorder of phonological access featuring phonemic parapha-
sia in naming, reading, and repetition tasks—have lesions confined to
this region [97–104], as do patients with phonological deficits in writ-
ten production [105–107]. A recent voxel-based lesion correlation
study linked damage in this posterior perisylvian region with inability
362 Jeffrey R. Binder

to silently judge whether written word pairs rhyme [108], indicating

that lesions in this region impair phonological processing prior to and
independent of any overt articulation processes. A number of func-
tional imaging results also support this model. For example, contrasts
between visual stimuli that can be named and those that cannot (e.g.,
pictures vs. nonsense shapes, pronounceable vs. unpronounceable let-
ter strings, letters vs. unfamiliar characters) reliably produce activation
in the left posterior STS, STG, and inferior SMG [109–116] as do
silent “word generation” tasks [117–119].
Temporary activation of phonological codes is also central to
the concept of verbal working memory. The standard model of ver-
bal working memory includes a “phonological loop” responsible
for maintaining phonological sequences in short-term memory
[120]. The phonological loop is further subdivided into a “phono-
logical store” that represents the phonological information itself
and an “articulatory rehearsal” mechanism that reactivates the
information before it fades from the store. A number of neuroim-
aging studies have linked the phonological store with the left SMG
[121] and with the posterior STG and STS [122–125].
These regions implicated in phonological access and tempo-
rary storage of phonological representations overlap partly with
those implicated in speech perception, though the evidence sug-
gests that the phoneme perception system is situated more anteri-
orly along the STG and STS [126], whereas the phonological
access system is more posterior and extends more dorsally, involv-
ing dorsal STG (planum temporale) and SMG. These systems can-
not be entirely overlapping, since most patients with conduction
aphasia (phonemic paraphasia) do not have speech perception defi-
cits. As noted above, the speech perception system is also bilaterally
represented, which may explain why it is more resistant to left STG
damage than is the phonological access system, which is more
strongly left-lateralized.

2.4 Semantic The human brain has an enormous capacity to acquire knowledge
Memory and Semantic from experience. The characteristic shapes, colors, textures, move-
Processing ments, sounds, smells, and actions associated with objects in the
environment, for example, must all be learned from experience. In
addition, consider the enormous variety of verb concepts (build, cel-
ebrate, discuss, throw, etc.), which depend on knowledge of how par-
ticular kinds of events happen, or social/emotional concepts (anger,
deceit, love, trust, etc.), which depend on knowledge of how human
beings behave and why. Much of this knowledge is represented sym-
bolically in language and underlies our understanding of word
meanings. These relationships between words and the stores of
knowledge they signify are known collectively as the semantics of a
language [127]. The term semantic processing refers to the cognitive
act of accessing stored knowledge about the world through words.
The stored knowledge itself is often called semantic memory.
 fMRI of Language Systems 363

Semantic properties of words are readily distinguished from

their structural properties. For example, words can have both spo-
ken (phonological) and written (orthographic) forms, but these
surface forms are typically related to word meanings only through
the arbitrary conventions of a particular language. There is nothing,
for example, about the letter sequences D-O-G or C-H-I-E-N that
inherently links these sequences to a particular concept. Conversely,
it is trivial to construct surface forms (e.g., CHOG) that possess all
of the phonological and orthographic properties of words in a par-
ticular language, but which have no meaning in that language. A
simple, operational distinction can thus be made between the pro-
cesses involved in analyzing the surface form (phonology, orthogra-
phy) of words, and semantic processes, which concern access to
knowledge that is not directly represented in the surface form.
Semantic processing is a defining feature of human behavior,
central not only to language, but also to our capacity to access
acquired knowledge in reasoning, planning, and problem solving.
Impairments of semantic processing figure in a variety of brain disor-
ders, such as Alzheimer disease, semantic dementia, fluent aphasia,
schizophrenia, and autism. The neural basis of semantic processing
has been studied extensively by analyzing patterns of brain damage in
such patients [128–135]. This topic has also been addressed in a
large number of functional neuroimaging studies on healthy volun-
teers (see [133, 136–140] for reviews). Of greatest interest for the
present review are studies that focused specifically on semantic pro-
cessing by incorporating control tasks that make comparable demands
on surface form (phonological or orthographic) processing and on
general executive processes such as attention, working memory, and
response production (see [25, 50, 141–151] for some examples).
Binder et al. [138] performed a quantitative meta-analysis of
120 of these well-controlled studies. The results revealed a widely
distributed, left-lateralized network underlying semantic memory
storage and retrieval (see red and orange regions in Fig. 1). Seven
major brain regions were implicated: (1) the angular gyrus; (2) the
middle and inferior temporal gyri, extending into the lateral ante-
rior temporal lobe; (3) the anterior fusiform and parahippocampal
gyri; (4) the anterior aspect (pars orbitalis) of the inferior frontal
gyrus (IFGpo); (5) dorsomedial prefrontal cortex, including the
superior frontal gyrus and the posterior aspect of the middle fron-
tal gyrus; (6) ventromedial prefrontal cortex; and (7) the posterior
cingulate gyrus. These are all regions considered to be supramodal
cortex, distant from primary sensory and motor areas, and there-
fore likely to be involved in processing highly abstracted (i.e., non-
perceptual) information. Activation in these regions tends to be
left-lateralized, though most studies show at least some activation
in homologous regions of the right hemisphere. Some evidence
suggests that the right hemisphere semantic system contributes to
processing concrete, imageable concepts, and much less to pro-
cessing abstract concepts [150, 152, 153].
364 Jeffrey R. Binder

These functional neuroimaging results are very consistent with

pathological data from patients with semantic disorders. For exam-
ple, lesion localization studies in patients with transcortical sensory
aphasia, a syndrome characterized by multimodal semantic impair-
ment with intact phonological processing, implicate widely distrib-
uted regions of the left ventral temporal lobe and angular gyrus
[128, 154–156]. Semantic dementia, a degenerative disorder char-
acterized by gradual loss of semantic knowledge, is associated with
progressive neuronal loss in the anterior and ventral temporal lobes
bilaterally [135, 157–161]. Other pathological conditions that
affect the ventral temporal lobes, such as herpes encephalitis and
Alzheimer disease, often produce focal semantic memory loss, par-
ticularly loss of knowledge about living things [132, 162–165],
while inferior parietal and posterior temporal lobe damage may
produce selective loss of knowledge about man-made objects, par-
ticularly tools [132, 137]. Whereas these temporal and parietal
lesions damage the semantic memory store itself (red regions in
Fig. 1), dorsal left prefrontal lesions seem to impair the ability to
retrieve information from the semantic store. These latter lesions
produce transcortical motor aphasia, a syndrome characterized by
inability to initiate spontaneous speech [155, 166].

2.5 Sentence The work reviewed so far focused on processing of single word
Comprehension structure and meaning, which can be thought of as the basic
and Syntax Processing building blocks of language. Natural language, however, consists
almost entirely of sentences. At least two phenomena distinguish
processing at the sentence level from processing of single words.
First, in sentence processing, the meanings of individual words are
combined to create more complex and context-specific meanings.
For example, consider:
1. The tigers lost their jungle habitat.
2. The tigers lost in extra innings.
It is the combination of words that specifies in each case the
meaning of “tigers” and “lost.” This process of conceptual combi-
nation is a fundamental phenomenon in language production and
comprehension. A second distinguishing feature of sentence pro-
cessing is the use of syntactic information—word order, grammati-
cal function words, and word inflections—to indicate the thematic
roles played by constituent content words. In the two example
sentences above, for example, “the” marks the beginning of a noun
phrase, which can be followed by either a noun or a modifier
phrase. The plural inflection of “tigers” then identifies this second
word as a noun, which because of its position is likely to be the
subject of the sentence, and so on.
One type of neuroimaging study used to examine these pro-
cesses compares processing of sentences with word lists that do not
form a sentence, the latter sometimes created simply by randomly
 fMRI of Language Systems 365

rearranging the order of words in sentences (“scrambled sen-

tences”). Common areas of activation in these contrasts (sentences
vs. scrambled sentences) include the left anterior superior temporal
lobe, left IFG, and left angular gyrus [167–173]. Debate has
ensued over whether these activations represent syntactic or seman-
tic processes, as sentences possess both more syntactic structure
and more complex meanings than lists of isolated words. This
question has focused particularly on the anterior temporal lobe, a
region often activated in studies using sentence materials but rarely
in studies using isolated words. Humphries et al. [172] showed
that activation in the anterior STS is modulated by the presence of
syntactic structure independently of the meaningfulness of con-
stituent content words, suggesting that this region may play a role
in early parsing processes (e.g., role assignment). These authors
also examined the processing of combinatorial semantic structure
in word lists and sentences by manipulating the degree to which
words in the stimuli were thematically related. Remarkably, this
contrast showed widespread regions in the ventral left temporal
lobe, angular gyrus, and inferior frontal lobe that were activated
when words were thematically related (and thus could be com-
bined to form more complex and specific meanings) compared to
when words were unrelated [172, 174]. This effect of combinatorial
semantic structure was largely unaffected by whether the stimuli
were syntactically correct sentences or word lists.
Many other fMRI and PET studies have focused on specific
syntactic operations, such as repair of syntactic and morphosyntac-
tic violations [175–180] and comprehension of object-extracted
relative clauses, passive voice, and other noncanonical or derived
syntactic structures [181–189]. While these studies have generally
implicated regions in the left IFG and left superior temporal lobe,
the precise localization of specific syntactic operations remains a
source of debate. Another ongoing discussion centers on whether
these activations reflect operations specific to syntax processing or
instead more domain-general working memory and executive pro-
cesses [187, 190–197]. The reader is referred to reviews that cover
this work in detail [198–202].

2.6 Retrieval, Using language depends on a variety of executive “control” pro-

Selection, cesses, including the ability to voluntarily activate phonological or
and Maintenance semantic information as needed for a given task, the ability to select
the correct name or concept when a number of competing alterna-
tives are activated, and the ability to maintain the selected item(s) in
short-term memory while the task is completed. For example, if the
task is to answer a question, such as, “What farm animal gives milk?”,
it is necessary to use the content words in the question (i.e., farm,
animal, give, milk) to activate a field of associated concepts, select
from among several activated alternatives (e.g., cow, goat, sheep),
use the selected concept to retrieve an associated name, and
366 Jeffrey R. Binder

maintain the concept and name in an activated state during produc-

tion of the response. These control processes depend mainly on the
left prefrontal cortex (green and orange regions in Fig. 1) [195,
203].
This modern view of the left prefrontal cortex contrasts with the
traditional concept of “Broca’s area” as a region involved only in
speech production. In fact, the same retrieval, selection, and mainte-
nance operations are required for many tasks in which no speech pro-
duction occurs, such as silently naming a picture, or comprehending
a sentence. Damage to the prefrontal cortex produces obvious
impairments on a range of language production tasks, but usually not
because speech articulation or motor sequencing is impaired. Rather,
frontal lesions impair the ability to voluntarily retrieve concepts and
verbal labels and to maintain these in short-term memory. The con-
tribution of these regions increases as the need for these control pro-
cesses increases, for example as sentences become more complex or
ambiguous, or items to be retrieved become less familiar.

3 An Analysis of Some Language Mapping Paradigms in Common Use

The variety of possible stimuli and tasks that could be used to

induce language processing is vast, and a coherent, concise discus-
sion is difficult. Table 1 lists some of the broad categories of stimuli
that have been used and some of the brain systems they engage.
“Auditory Nonspeech” refers to noises or tones that are not per-
ceived as speech. Such stimuli can be variably “complex” in their
temporal or spectral features, and possess to varying degrees the
acoustic properties of speech (see [54, 204–207] for some exam-
ples). “Auditory Phonemes” are speech sounds that do not com-
prise words in the listener’s language; these may be simple
consonant-vowel monosyllables or longer sequences (e.g., pseudo-
words). “Visual Nonletter” refers to any unfamiliar visual shape.
Examples include characters from unfamiliar alphabets, nonsense
shapes, and “false font.” Such stimuli can be variably complex and
possess to varying degrees the visual properties of familiar letters.
“Visual Letterstrings” are random strings of letters that do not
form familiar or easily pronounceable letter combinations (e.g.,
FCJVB). “Visual Pseudowords” are letterstrings that are not words
but possess the orthographic and phonological characteristics of
real words (e.g., SNADE).
The degree to which these stimuli engage the processes listed in
Table 1 depend partly on the task that the subject is asked to per-
form, though the processes in Table 1 are activated “automatically”
to some degree even when subjects are given no explicit task. This
is less true for the processing systems listed in Table 2, which seem
to be strongly task-dependent. The semantic system appears to be
partly active even during “rest” or when stimuli are presented
 fMRI of Language Systems 367

Table 1
Effects of auditory and visual stimuli on sensory and linguistic processing systems

Early Phoneme Visual Object

Stimuli sensory perception wordform recognition Syntax
Auditory nonspeech Aud − − − −
Auditory phonemes Aud + − − −
Auditory words Aud + − − −
Auditory sentences Aud + − − +
Visual nonletters Vis − − − −
Visual letterstrings Vis − +/− − −
Visual pseudowords Vis − + − −
Visual words Vis − + − −
Visual sentences Vis − + − +
Visual objects Vis − − + −

Table 2
Effects of task states on some linguistic processing systems

Phonological Speech Working

Tasks Semantics Access Articulation Memory Other language

Rest or “passive” + − − − −
Sensory discrimination − − − +/− −
Read or repeat covert + + − +/− −
Read or repeat overt + + + +/− −
Phonetic decision − + − + −
Phonological decision − + − + −
Orthographic decision − +/− − − −
Semantic decision + +/− − + Semantic search
Word generation covert + + − + Lexical search
Word generation overt + + + + Lexical search
Naming covert + + − − Lexical search
Naming overt + + + − Lexical search
368 Jeffrey R. Binder

“passively” to the subject [25, 33–35, 37, 208]. Other tasks sup-
press semantic processing by requiring a focusing of attention on
perceptual, orthographic, or phonological properties of stimuli [25,
26, 35, 208]. Examples include “Sensory Discrimination” tasks
(e.g., intensity, size, color, frequency, and other discriminations
based on physical features), “Phonetic Decision” tasks in which the
subject must detect a target phoneme or phonemes, “Phonological
Decision” tasks requiring a decision based on the phonological
properties of a stimulus (e.g., detection of rhymes, judgment of syl-
lable number), and “Orthographic Decision” tasks requiring a deci-
sion based on the letters in the stimulus (e.g., case matching, letter
identification). Other tasks, such as reading and repeating, make no
overt demands on semantic systems but may elicit automatic seman-
tic processing. The extent to which this occurs probably depends
on how meaningful the stimulus is: sentences likely elicit more
semantic processing than isolated words, which in turn elicit more
than pseudowords. Finally, many tasks make overt demands on
retrieval and use of semantic knowledge. These include “Semantic
Decision” tasks requiring a decision based on the meaning of the
stimulus (e.g., “Is it living or nonliving?”), “Word Generation”
tasks requiring retrieval of a word or series of words related in mean-
ing to a cue word, and “Naming” tasks requiring retrieval of a ver-
bal label for an object or object description.
As noted earlier, “Phonological Access” refers to the processes
engaged in retrieving a phonological (sound-based) representation
of a word (or pseudoword). In addition to speech output and pho-
nological tasks, any task using printed words, including ortho-
graphic and semantic tasks, will be accompanied to some degree by
obligatory phonological access [8, 15, 111]. In contrast, “Speech
Articulation” processes are engaged fully only when an overt spo-
ken response is produced [119]. “Verbal Working Memory” is
required whenever a written or spoken stimulus must be held in
memory. Some degree of short-term phonological memory is
needed for most language tasks, and particularly in cases where the
stimulus is relatively long (i.e., sentences more than single words)
or has multiple components, or must be held in memory while a
response is generated (e.g., word generation tasks involving mul-
tiple responses for each cue). Finally, semantic decision, word gen-
eration, and naming tasks make strong demands on frontal
mechanisms involved in searching for and retrieving information
associated with a stimulus [118, 195, 209, 210].
With these somewhat over-simplified stimulus and task charac-
terizations in mind, it is possible to make some general predictions
about the processing systems whose level of activation will differ
when two task conditions are contrasted, and thus the likely pat-
tern of brain activation that will be observed in a simple subtrac-
tion analysis. Some commonly encountered examples are listed
below and in Table 3.
 fMRI of Language Systems 369

Table 3
Some task contrasts used for language mapping and the regions in which robust activations are
typically observed

Ventrolateral Dorsal Superior Ventrolateral Ventral Angular

Prefrontal Prefrontal Temporal Temporal Occipital Gyrus

1. Hearing words vs.
Rest B
2. Hearing words vs.
Nonspeech sounds L>R
3. Word generation vs.
Rest L>R L>R B
4. Word generation vs.
Reading L
5. Object naming vs.
Rest B L>R B
6. Semantic decision vs.
Sensory discrimination L L L>R L L
7. Semantic decision vs.
Phonological decision L L L
8. Reading sentences vs.
Letterstrings L>R L>R L>R
L = left hemisphere, R = right hemisphere, B = bilateral

Paradigm 1
Language task: Passively Listening to Words or Sentences
Control task: Rest
As shown in Table 1, auditory words activate early auditory cor-
tices and phoneme perception areas. Since both rest and passive
stimulation are accompanied by spontaneous semantic processes and
make no other overt cognitive demands, no other language-related
activation should appear in this contrast. The resulting activation
pattern involves mainly auditory cortex in the superior temporal gyri
bilaterally (Fig. 2a) [54, 167, 208, 211, 212]. The magnitude and
extent of this activation increase with rate of word presentation
[213, 214]. This STG activation is relatively symmetrical and is not
correlated with language dominance as measured by Wada testing
[215]. Although some authors have equated this STG activation
with “Wernicke’s area for receptive language,” most of this
Fig. 2 Group average fMRI activation patterns in 26 neurologically normal, right-handed volunteers during five
fMRI language paradigms (see [208] for details). Auditory word and tone stimuli were equivalent in each of the
five paradigms. (a) Passive listening to words contrasted with resting. Superior temporal activation occurs
bilaterally. (b) Passive listening to words contrasted with passive listening to tones. A small region in the left
STS shows activation specifically related to speech processing. (c) Semantic decision on words contrasted
with resting. Activation occurs in bilateral auditory (STG) and attentional/working memory (dorsolateral pre-
frontal, anterior cingulate, anterior insula, IPS, and subcortical) networks, with left lateralization in the IFG.
 fMRI of Language Systems 371

activation represents early auditory processing rather than language-

specific processes per se.
Paradigm 2
Language task: Passively Listening to Words or Sentences
Control task: Passively Listening to Auditory Nonspeech
Because there are no differences in task requirements, and because
semantic processing occurs in all passive conditions, the activation pat-
tern associated with this contrast mainly reflects activation of the pho-
neme perception system (Table 1). As mentioned earlier, studies
employing such contrasts reliably show activation in the STS, with
leftward lateralization, and little or no activation elsewhere (Fig. 2b)
[51, 52, 54, 56, 204, 208]. When sentences are used, this STS activa-
tion extends more anteriorly into the dorsal temporal pole region,
possibly reflecting early syntactic parsing processes [167, 171, 172,
204, 216–219].
Paradigm 3
Language task: Word Generation
Control task: Rest
Because the rest state includes no control for sensory process-
ing, early auditory or visual cortices may be activated bilaterally
depending on the sensory modality of the cue stimulus (Table 1).
The strength of this sensory activation depends on the rate of stimu-
lus presentation: in some protocols, a single cue (e.g., a letter or a
semantic category) is provided only at the beginning of an activation
period; in others, a new cue is provided every few seconds. Unlike
rest, word generation makes demands on lexical search, phonologi-
cal access, and working memory systems (Table 2). Speech articula-
tion systems will also be activated if an overt spoken response is
required. These predictions are confirmed by many studies employ-
ing this contrast, which results primarily in activation of the left IFG
and left > right premotor cortex, systems thought to be involved in
phonological production, working memory, and lexical search [118,
211, 215, 220–225]. There may be activation of left posterior tem-
poral regions (posterior MTG and STG) due to engagement of the
phonological access system [117, 119, 226].

Fig. 2 (continued) (d) Semantic decision on words contrasted with a tone decision task. Activation is strongly
left-lateralized in prefrontal, lateral and ventral temporal, angular, and posterior cingulate cortices. (e) Semantic
decision on words contrasted with a phoneme decision task on pseudowords. Activation is strongly left-later-
alized in dorsal prefrontal, angular, ventral temporal, and posterior cingulate cortices. Data are displayed as
serial sagittal sections through the brain at 9-mm intervals. X-axis locations for each slice are given in the top
panel. Green lines indicate the stereotaxic Y and Z origin planes. Hot colors (red–yellow) indicate positive
activations and cold colors (blue–cyan) negative activations for each contrast. All maps are thresholded at a
whole-brain corrected P < 0.05 using voxel-wise P < 0.0001 and cluster extent >200 mm3. Adapted, with per-
mission, from [208]
372 Jeffrey R. Binder

Paradigm 4
Language task: Word Generation
Control task: Reading or Repeating
Here we assume that the same stimulus modality (auditory or
visual) is used for both tasks. The stimuli in both cases are single
words, thus no difference in activation of sensory, phoneme per-
ception, or visual word form systems is expected. Both tasks are
accompanied by semantic processing (automatic semantic access in
the case of the control task, effortful semantic retrieval in the case
of word generation) and by phonological access processes. The
word generation task makes greater demands on lexical search and
on working memory; consequently greater activation is expected
in left inferior frontal areas associated with these processes. These
predictions match findings in many studies using this contrast,
which show primarily left-lateralized activation in the IFG [118,
210, 226, 227].
Paradigm 5
Language task: Visual Object Naming
Control task: Rest
Compared to resting, visual object perception activates early visual
sensory cortices and higher-level object recognition systems bilaterally
(Table 1) [228–230]. There may be additional, left-lateralized activa-
tion in semantic systems of the ventrolateral posterior temporal lobe
[231–235]. Unlike resting, naming requires lexical search and phono-
logical access, and, when overt, speech articulation (Table 2). These
predictions match findings in several studies using this contrast, which
show extensive bilateral visual system activation and modest left later-
alized inferior frontal activation [223, 234–236].
Paradigm 6
Language task: Semantic Decision
Control task: Sensory Discrimination
We again assume that the same stimulus modality is used for
both tasks. If the stimuli in the sensory discrimination task are non-
linguistic (e.g., tones or nonsense shapes), then the semantic deci-
sion task will produce relatively greater activation in phoneme
perception or visual wordform systems, depending on the sensory
modality. In addition, there will be greater activation of semantic
memory and semantic search mechanisms in the semantic decision
task. Note that unlike the resting and passive control tasks used in
the protocols described so far, effortful sensory discrimination tasks
interrupt ongoing semantic processes, providing a control state that
is relatively free of conceptual or semantic processing [25, 34, 35,
37, 208]. Working memory systems may or may not be activated in
 fMRI of Language Systems 373

this contrast, depending on whether or not the control task also has
a working memory component. These predictions match findings
in studies using this contrast, which show left lateralized activation
of phoneme perception (middle and anterior superior temporal sul-
cus) or visual wordform (mid-fusiform gyrus) regions, and exten-
sive activation of left prefrontal, lateral and ventral left temporal,
and left posterior parietal systems involved in semantic memory and
semantic access (Fig. 2d) [5, 50, 208, 237–240].
Paradigm 7
Language task: Semantic Decision
Control task: Phonological Decision
These tasks can also be given in either the visual or auditory
modality. Stimuli in the phonological decision task can be either
words or pseudowords, and these can be matched to the words
used in the semantic task on all structural (physical, orthographic,
phonological) variables. Thus, there should be no activation of
sensory or wordform systems in this contrast. There will be greater
activation of semantic memory and semantic search systems in the
semantic decision task. These predictions match findings in many
studies using this contrast, which show activation of left prefrontal,
lateral and ventral left temporal, and left posterior parietal systems
believed to be involved in semantic processing (Fig. 2e) [25, 50,
142–144, 146, 148, 208, 241, 242].
Paradigm 8
Language task: Sentence or Word Reading
Control task: Passively Viewing Letterstrings
Compared to letterstrings, sentences engage visual word-form,
syntactic, and phonological access systems, and make variable
demands on working memory. Both reading and passive viewing
probably involve semantic processing. There should be left-lateralized
activation of the fusiform gyrus (visual word-form system), posterior
STG and STS (phonological access), and IFG (orthographic-phono-
logical mapping, working memory, syntax). These predictions are
consistent with several studies using this contrast [111, 243–246].
In many clinical settings, the main goal of language mapping is
simply to identify as many language-related areas as possible and to
assess hemispheric lateralization of language. A review of Table 3
suggests that the “Semantic Decision vs. Sensory Discrimination”
paradigm may offer advantages for this purpose in terms of the sheer
number of regions activated and leftward lateralization of activation.
Binder et al. put this prediction to a quantitative test by comparing
the extent and lateralization of activation produced by five language-
related task contrasts, conducted on the same 26 participants during
a single scanning session [208]. These contrasts included: (1)
374 Jeffrey R. Binder

Fig. 3 Group average activation volumes (top graph) and laterality indexes (bottom graph) for five fMRI lan-
guage paradigms [208]. Laterality indexes can vary from −1 (all activation in the right hemisphere) to +1 (all
activation in the left hemisphere). Error bars represent standard error. The Semantic Decision–Tone Decision
paradigm produces the greatest left hemisphere activation as well as a strongly left-lateralized pattern

passively listening to words vs. resting, (2) passively listening to

words vs. passively listening to tones, (3) performing a semantic
decision task with words vs. resting, (4) performing a semantic deci-
sion task with words vs. a sensory discrimination task with tones, and
(5) performing a semantic decision task with words vs. a phonologi-
cal task with pseudowords. As shown in Fig. 3, the Semantic
Decision—Tone Decision contrast produced by far the largest acti-
vation volume in the left hemisphere, as well as an optimal combina-
tion of extensive activation and strong left-lateralization.
The example paradigms discussed here cover only a small sam-
ple of all possible language activation protocols. There are also
numerous published studies employing designs that do not fit
neatly into the schema provided here. Many of these represent
attempts to further define or fractionate a particular language pro-
cess, or to define further the functional role of a specific brain
 fMRI of Language Systems 375

region. The reader should appreciate that the review given here is
merely a coarse outline of some of the most commonly used types
of stimuli and tasks. Above all, it is important to note that activa-
tions in a particular part of the language system are seldom “all or
none,” but vary in a graded way depending on the particular stim-
uli and tasks used.

4 Conclusions and Future Directions

Functional neuroimaging techniques have enhanced profoundly

our understanding of how language processes are implemented in
the human brain. This work has led to a number of new discover-
ies, such as a more precise localization of cortical networks under-
lying phoneme and grapheme perception, phonological access,
and semantic processing. Not all of the claims made here with
regard to these component language systems are uncontroversial.
In particular, there are ongoing debates and a number of unre-
solved issues concerning localization of semantic memory and
semantic retrieval systems, especially with regard to the role played
by the left IFG in semantic processes [146, 209, 210, 247–250].
The notion that semantic processes are actively engaged during the
resting state, though gaining traction in some quarters, is far from
universal acceptance or recognition. For example, many authors
continue to regard with suspicion any activation associated with
semantic tasks that is not also observed in comparison to a resting
baseline [147, 251, 252]. As in other areas of cognitive neurosci-
ence, neuroimaging research on language processing has been to
some extent clouded by an incomplete understanding of task
demands and inadequate recognition of potential confounding
factors. For example, many task contrasts are confounded by dif-
ferences in task difficulty, which are well known to cause differen-
tial activation of domain-general networks involved in arousal,
attention, working memory, decision, response selection, and error
monitoring [151, 253–259]. Despite these well-documented
effects, many researchers continue to advocate the use of covert
tasks and passive conditions that provide no information about
task performance, level of attention, or degree of difficulty. These
deficiencies are particularly troubling in clinical studies, where the
interpretation of brain activation (or lack of activation) can sub-
stantially influence clinical decision-making and patient outcome.
As the field of functional neuroimaging continues to mature, it
is likely that these potential pitfalls will eventually be universally
recognized and that an increasingly standardized methodological
and conceptual framework for language mapping studies will
emerge. FMRI practitioners, whether working in clinical or
research fields, should continue to strive toward these goals.
376 Jeffrey R. Binder

References
1. Broca P (1861) Remarques sur le siège de 17. Glaser WR (1992) Picture naming. Cognition
la faculté du langage articulé; suivies d’une 42:61–105
observation d’aphemie. Bulletin de la Société 18. James W (1890) Principles of psychology.
Anatomique de Paris 6:330–357 Dover, New York
2. Wernicke C (1874) Der aphasische 19. Hebb DO (1954) The problem of con-
Symptomenkomplex. Cohn & Weigert, sciousness and introspection. In: Adrian ED,
Breslau Bremer F, Jasper HH (eds) Brain mechanisms
3. Lichtheim L (1885) On aphasia. Brain 7:433–484 and consciousness. A symposium. Charles
4. Geschwind N (1971) Aphasia. N Engl J Med C. Thomas, Springfield, IL, pp 402–421
284(12):654–656 20. Miller GA, Galanter E, Pribram K (1960)
5. Binder JR, Frost JA, Hammeke TA, Cox RW, Plans and the structure of behavior. Holt,
Rao SM et al (1997) Human brain language New York
areas identified by functional MRI. J Neurosci 21. Pope KS, Singer JL (1976) Regulation of the
17(1):353–362 stream of consciousness: Toward a theory of
6. Démonet J-F, Thierry G, Cardebat D (2005) ongoing thought. In: Schwartz GE, Shapiro
Renewal of the neurophysiology of lan- D (eds) Consciousness and self-regulation.
guage: functional neuroimaging. Physiol Rev Plenum, New York, pp 101–135
85(1):49–95 22. Picton TW, Stuss DT (1994) Neurobiology of
7. Binder JR, Price CJ (2001) Functional imag- conscious experience. Curr Opin Neurobiol
ing of language. In: Cabeza R, Kingstone A 4:256–265
(eds) Handbook of functional neuroimaging 23. Antrobus JS, Singer JL, Greenberg S (1966)
of cognition. MIT Press, Cambridge, MA, Studies in the stream of consciousness:
pp 187–251 experimental enhancement and suppression
8. Macleod CM (1991) Half a century of of spontaneous cognitive processes. Percept
research on the Stroop effect: an integrative Mot Skills 23:399–417
review. Psychol Bull 109:163–203 24. Teasdale JD, Proctor L, Lloyd CA, Baddeley
9. Reicher GM (1969) Perceptual recognition AD (1993) Working memory and stimulus-
as a function of meaningfulness of stimulus independent thought: effects of memory load
material. J Exp Psychol 81:274–280 and presentation rate. Eur J Cogn Psychol
10. Warren RM, Obusek CJ (1971) Speech per- 5(4):417–433
ception and phonemic restorations. Percept 25. Binder JR, Frost JA, Hammeke TA, Bellgowan
Psychophys 9:358–362 PSF, Rao SM et al (1999) Conceptual pro-
11. Ganong WF (1980) Phonetic categorization cessing during the conscious resting state:
in auditory word perception. J Exp Psychol a functional MRI study. J Cogn Neurosci
Hum Percept Perform 6:110–115 11(1):80–93
12. Marslen-Wilson WD, Tyler LK (1981) 26. McKiernan KA, Kaufman JN, Kucera-
Central processes in speech understanding. Thompson J, Binder JR (2003) A parametric
Philos Trans R Soc Lond B 295:317–332 manipulation of factors affecting task-induced
deactivation in functional neuroimaging.
13. Carr TH, McCauley C, Sperber RD, Parmalee J Cogn Neurosci 15(3):394–408
CM (1982) Words, pictures, and priming: on
semantic activation, conscious identification, 27. Révész G (ed) (1954) Thinking and speak-
and the automaticity of information process- ing: a symposium. North Holland Publishing,
ing. J Exp Psychol Hum Percept Perform Amsterdam
8:757–777 28. Weiskrantz L (ed) (1988) Thought without
14. Marcel AJ (1983) Conscious and uncon- language. Clarendon, Oxford
scious perception: Experiments on visual 29. Vygotsky LS (1962) Thought and language.
masking and word recognition. Cogn Psychol Wiley, New York
15:197–237 30. Karmiloff-Smith A (1992) Beyond modular-
15. Van Orden GC (1987) A ROWS is a ROSE: ity: a developmental perspective on cognitive
spelling, sound, and reading. Mem Cogn science. MIT Press, Cambridge, MA
15(3):181–198 31. Andreasen NC, O’Leary DS, Cizadlo T,
16. Burton MW, Baum SR, Blumstein SE (1989) Arndt S, Rezai K et al (1995) Remembering
Lexical effects on phonetic categorization of the past: two facets of episodic memory
speech: the role of acoustic structure. J Exp explored with positron emission tomography.
Psychol Hum Percept Perform 15:567–575 Am J Psychiatry 152:1576–1585
 fMRI of Language Systems 377

32. Shulman GL, Fiez JA, Corbetta M, Buckner 47. Buchweitz A, Prat C (2013) The bilingual
RL, Meizin FM et al (1997) Common blood brain: flexibility and control in the human
flow changes across visual tasks: II. Decreases cortex. Phys Life Rev 10(4):428–443
in cerebral cortex. J Cogn Neurosci 48. Li P, Legault J, Litcofsky KA (2014)
9(5):648–663 Neuroplasticity as a function of second lan-
33. Mazoyer B, Zago L, Mellet E, Bricogne S, guage learning: anatomical changes in the
Etard O et al (2001) Cortical networks for human brain. Cortex 58:301–324
working memory and executive functions sus- 49. Bogen JE, Bogen GM (1976) Wernicke’s
tain the conscious resting state in man. Brain region – where is it? Ann NY Acad Sci
Res Bull 54(3):287–298 290:834–843
34. Stark CE, Squire LR (2001) When zero is 50. Démonet JF, Chollet F, Ramsay S, Cardebat
not zero: the problem of ambiguous baseline D, Nespoulous JL et al (1992) The anatomy
conditions in fMRI. Proc Natl Acad Sci U S A of phonological and semantic processing in
98(22):12760–12766 normal subjects. Brain 115:1753–1768
35. McKiernan KA, D’Angelo BR, Kaufman JN, 51. Zatorre RJ, Evans AC, Meyer E, Gjedde A
Binder JR (2006) Interrupting the “stream (1992) Lateralization of phonetic and pitch
of consciousness”: an fMRI investigation. discrimination in speech processing. Science
Neuroimage 29(4):1185–1191 256:846–849
36. Smallwood J, Schooler JW (2006) The rest- 52. Mummery CJ, Ashburner J, Scott SK, Wise
less mind. Psychol Bull 132(6):946–958 RJS (1999) Functional neuroimaging of
37. Mason MF, Norton MI, Van Horn JD, speech perception in six normal and two apha-
Wegner DM, Grafton ST et al (2007) sic subjects. J Acoust Soc Am 106:449–457
Wandering minds: the default network and 53. Belin P, Zatorre RJ, Lafaille P, Ahad P, Pike B
stimulus-independent thought. Science (2000) Voice-selective areas in human audi-
315(5810):393–395 tory cortex. Nature 403:309–312
38. Andrews-Hanna JR (2012) The brain’s 54. Binder JR, Frost JA, Hammeke TA, Bellgowan
default network and its adaptive role in inter- PSF, Springer JA et al (2000) Human tempo-
nal mentation. Neuroscientist 18(3):251–270 ral lobe activation by speech and nonspeech
39. Abutalebi J, Cappa S, Perani D (2005) What sounds. Cereb Cortex 10:512–528
can functional neuroimaging tell us about the 55. Blumstein SE, Myers EB, Rissman J (2005)
bilingual brain? In: Kroll J, de Groot AMB The perception of voice onset time: an fMRI
(eds) Handbook of bilingualism: psycholin- investigation of phonetic category structure.
guistic approaches. Oxford University Press, J Cogn Neurosci 17(9):1353–1366
New York, pp 497–515 56. Desai R, Liebenthal E, Possing ET, Waldron E,
40. Hernandez A, Li P, MacWhinney B (2005) Binder JR (2005) Volumetric vs surface-based
The emergence of competing modules in alignment for localization of auditory cortex
bilingualism. Trends Cogn Sci 9(5):220–225 activation. Neuroimage 26(4):1019–1029
41. Corina DP, Knapp H (2006) Sign language 57. Turkeltaub PE, Coslett HB (2010)
processing and the mirror neuron system. Localization of sublexical speech perception
Cortex 42(4):529–539 components. Brain Lang 114:1–15
42. MacSweeney M, Capek CM, Campbell R, 58. Dehaene-Lambertz G, Pallier C, Serniclaes W,
Woll B (2008) The signing brain: the neu- Sprenger-Charolles L, Jobert A et al (2005)
robiology of sign language. Trends Cogn Sci Neural correlates of switching from auditory
12(11):432–440 to speech perception. Neuroimage 24:21–33
43. Corina D, Singleton J (2009) Developmental 59. Liebenthal E, Binder JR, Spitzer SM, Possing
social cognitive neuroscience: insights from ET, Medler DA (2005) Neural substrates
deafness. Child Dev 80(4):952–967 of phonemic perception. Cerebral Cortex
44. Emmorey K, McCullough S (2009) The 15:1621–1631
bimodal bilingual brain: effects of sign lan- 60. Möttönen R, Calvert GA, Jaaskelainen IP,
guage experience. Brain Lang 109:124–132 Matthews PM, Thesen A et al (2006) Perceiving
45. Kotz SA (2009) A critical review of ERP and identical sounds as speech or non-speech modu-
fMRI evidence on L2 syntactic processing. lates activity in the left posterior superior tem-
Brain Lang 109(2):68–74 poral sulcus. Neuroimage 30:563–569
46. van Heuven WJ, Dijkstra T (2010) Language 61. Obleser J, Zimmerman J, Van Meter J,
comprehension in the bilingual brain: fMRI Rauschecker JP (2007) Multiple stages of
and ERP support for psycholinguistic models. auditory speech perception reflected in event-
Brain Res Rev 64(1):104–122 related fMRI. Cereb Cortex 17:2251–2257
378 Jeffrey R. Binder

62. Desai R, Liebenthal E, Waldron E, Binder JR 78. Vincent FM, Sadowsky CH et al (1977)
(2008) Left posterior temporal regions are Alexia without agraphia, hemianopia, or
sensitive to auditory categorization. J Cogn color-naming defect: a disconnection syn-
Neurosci 20:1174–1188 drome. Neurology 27:689–691
63. Liebenthal E, Desai R, Ellingson MM, 79. Henderson VW (1986) Anatomy of posterior
Ramachandran B, Desai A et al (2010) pathways in reading: a reassessment. Brain
Specialization along the left superior temporal Lang 29:119–133
sulcus for phonemic and non-phonemic cat- 80. Binder JR, Mohr JP (1992) The topography
egorization. Cerebral Cortex 20:2958–2970 of callosal reading pathways: a case-control
64. Barrett AM (1910) A case of pure word- analysis. Brain 115:1807–1826
deafness with autopsy. J Nerv Ment Dis 81. Beversdorf DQ, Ratcliffe NR, Rhodes CH,
37(2):73–92 Reeves AG (1997) Pure alexia: clinical-
65. Henschen SE (1918–1919) On the hearing pathological evidence for a lateralized visual
sphere. Acta Otolaryngol 1:423–486 language association cortex. Clin Neuropathol
66. Wohlfart G, Lindgren A, Jernelius B (1952) 16(6):328–331
Clinical picture and morbid anatomy in a case 82. Sakurai Y, Takeuchi S, Takada T, Horiuchi
of “pure word deafness”. J Nerv Ment Dis E, Nakase H et al (2000) Alexia caused by a
116:818–827 fusiform or posterior inferior temporal lesion.
67. Lhermitte F, Chain F, Escourolle R, Ducarne J Neuro Sci 178:42–51
B, Pillon A et al (1972) Etude des troubles 83. Leff AP, Crewes H, Plant GT, Scott SK,
perceptifs auditifs dans les lésions temporales Kennard C et al (2001) The functional anatomy
bilatérales. Revue Neurologique 24:327–351 of single-word reading in patients with hemi-
68. Kanshepolsky J, Kelley J, Waggener JD anopic and pure alexia. Brain 124:510–521
(1973) A cortical auditory disorder: clinical, 84. Cohen L, Martinaud O, Lemer C, Lehericy S,
audiologic and pathologic aspects. Neurology Samson Y et al (2003) Visual word recogni-
23:699–705 tion in the left and right hemispheres: ana-
69. Buchman AS, Garron DC, Trost-Cardamone tomical and functional correlates of peripheral
JE, Wichter MD, Schwartz D (1986) Word alexias. Cereb Cortex 13:1313–1333
deafness: one hundred years later. J Neurol 85. Tarkiainen A, Helenius P, Hansen PC,
Neurosurg Psychiatry 49:489–499 Cornelissen PL, Salmelin R (1999) Dynamics
70. Poeppel D (2001) Pure word deafness and of letter string perception in the human occipi-
the bilateral processing of the speech code. totemporal cortex. Brain 122:2119–2131
Cogn Sci 25:679–693 86. Cohen L, Dehaene S, Naccache L, Lehéricy
71. Liepmann H, Storch E (1902) Der mikros- S, Dehaene-Lambertz G et al (2000) The
kopische gehirnbefund bei dem fall gorstelle. visual word form area. Spatial and temporal
Monatsschrift fur Psychiatrie und Neurologie characterization of an initial stage of reading
11:115–120 in normal subjects and posterior split-brain
72. Stefanatos GA, Gershkoff A, Madigan S patients. Brain 123:291–307
(2005) On pure word deafness, temporal 87. Dehaene S, Naccache L, Cohen L, Bihan DL,
processing and the left hemisphere. J Int Mangin JF et al (2001) Cerebral mechanisms
Neuropsychol Soc 11(4):456–470 of word masking and unconscious repetition
73. Déjerine J (1891) Sur un cas de cécité verbal priming. Nat Neurosci 4(7):752–758
avec agraphie, suivi d’autopsie. C R Seances 88. Polk TA, Farah MJ (2002) Functional MRI
Soc Biol 3:197–201 evidence for an abstract, not perceptual, word-
74. Déjerine J (1892) Contribution à l’étude form area. J Exp Psychol Gen 131:65–72
anatomo-pathologique et clinique des différ- 89. Cohen L, Jobert A, Le Bihanc D, Dehaene
entes variétés de cécité verbale. C R Seances S (2004) Distinct unimodal and multimodal
Soc Biol 44(2):61–90 regions for word processing in the left tempo-
75. Déjerine J, Vialet N (1893) Contribution a ral cortex. Neuroimage 23:1256–1270
l’étude de la localisation anatomique de la 90. Vinckier F, Dehaene S, Jobert A, Dubus J,
cécité verbale pure. C R Seances Soc Biol Sigman M et al (2007) Hierarchical coding
45:790–793 of letter strings in the ventral stream: dissect-
76. Geschwind N (1965) Disconnection syn- ing the inner organization of the visual word-
dromes in animals and man. Brain 88:237– form system. Neuron 55(1):143–156
294, 585-644 91. Glezer LS, Jiang X, Riesenhuber M (2009)
77. Greenblatt SH (1976) Subangular alexia Evidence for highly selective neuronal tuning
without agraphia or hemianopsia. Brain Lang of whole words in the “Visual Word Form
3:229–245 Area”. Neuron 62:199–204
 fMRI of Language Systems 379

92. Mano QR, Humphries CJ, Desai R, 105. Roeltgen DP, Sevush S, Heilman KM (1983)
Seidenberg MS, Osmon DC et al (2013) Phonological agraphia: writing by the lexical-
The role of left occipitotemporal cor- semantic route. Neurology 33:755–765
tex in reading: reconciling stimulus, 106. Alexander MP, Friedman RB, Loverso F,
task, and lexicality effects. Cereb Cortex Fischer RS (1992) Lesion localization of pho-
23(4):988–1001 nological agraphia. Brain Lang 43:83–95
93. Binder JR, Medler DA, Westbury CF, 107. Rapcsak SZ, Beeson PM, Henry ML, Leyden
Liebenthal E, Buchanan L (2006) Tuning A, Kim E et al (2009) Phonological dyslexia
of the human left fusiform gyrus to sub- and dysgraphia: cognitive mechanisms and
lexical orthographic structure. Neuroimage neural substrates. Cortex 45(5):575–591
33:739–748 108. Pillay SB, Stengel BC, Humphries C, Book
94. Patterson KE, Kay J (1982) Letter-by-letter DS, Binder JR (2014) Cerebral localization of
reading: psychological descriptions of a impaired phonological retrieval during rhyme
neurological syndrome. Q J Exp Psychol judgment. Ann Neurol 76:738–746
34A:411–442 109. Howard D, Patterson K, Wise R, Brown WD,
95. Reuter-Lorenz PA, Brunn JL (1990) A prel- Friston K et al (1992) The cortical localiza-
exical basis for letter-by-letter reading: a case tion of the lexicons. Brain 115:1769–1782
study. Cognit Neuropsychol 7:1–20 110. Price CJ, Wise RJS, Watson JDG, Patterson
96. Behrmann M, Plaut DC, Nelson J (1998) K, Howard D et al (1994) Brain activity dur-
A literature review and new data supporting ing reading. The effects of exposure duration
an interactive activation account of letter- and task. Brain 117:1255–1269
by-letter reading. In: Coltheart M (ed) Pure 111. Price CJ, Wise RSJ, Frackowiak RSJ (1996)
alexia (letter-by-letter reading). Pscyhology, Demonstrating the implicit processing of
Hove, UK, pp 7–51 visually presented words and pseudowords.
97. Liepmann H, Pappenheim M (1914) Über Cereb Cortex 6:62–70
einem Fall von sogenannter Leitungsaphasie 112. Hickok G, Erhard P, Kassubek J, Helms-
mit anatomischer Befund. Z Gesamte Neurol Tillery AK, Naeve-Velguth S et al (2000) A
Psychiatr 27:1–41 functional magnetic resonance imaging study
98. Benson DF, Sheremata WA, Bouchard R, of the role of left posterior superior tempo-
Segarra JM, Price D et al (1973) Conduction ral gyrus in speech production: implications
aphasia. A clinicopathological study. Arch for the explanation of conduction aphasia.
Neurol 28:339–346 Neurosci Lett 287:156–160
99. Damasio H, Damasio AR (1980) The ana- 113. Booth JR, Burman DD, Meyer JR, Gitelman
tomical basis of conduction aphasia. Brain DR, Parrish TB et al (2002) Functional anat-
103:337–350 omy of intra- and cross-modal lexical tasks.
100. Anderson JM, Gilmore R, Roper S, Crosson Neuroimage 16:7–22
B, Bauer RM et al (1999) Conduction apha- 114. Indefrey P, Levelt WJM (2004) The spatial
sia and the arcuate fasciculus: a reexamination and temporal signatures of word production
of the Wernicke-Geschwind model. Brain components. Cognition 92(1-2):101–144
Lang 70:1–12 115. Burton MW, Locasto PC, Krebs-Noble D,
101. Quigg M, Fountain NB (1999) Conduction Gullapalli RP (2005) A systematic investi-
aphasia elicited by stimulation of the left gation of the functional neuroanatomy of
posterior superior temporal gyrus. J Neurol auditory and visual phonological processing.
Neurosurg Psychiatry 66:393–396 Neuroimage 26(3):647–661
102. Axer H, Keyserlingk AG, Berks G, Keyserlingk 116. Callan AM, Callan DE, Masaki S (2005)
DF (2001) Supra- and infrasylvian conduc- When meaningless symbols become letters:
tion aphasia. Brain Lang 76:317–331 neural activity change in learning new phono-
103. Fridriksson J, Kjartansson O, Morgan PS, grams. Neuroimage 28:553–562
Hjaltason H, Magnusdottir S et al (2010) 117. Fiez JA, Raichle ME, Balota DA, Tallal P,
Impaired speech repetition and left parietal Petersen SE (1996) PET activation of poste-
lobe damage. J Neurosci 30:11057–11061 rior temporal regions during auditory word
104. Buchsbaum BR, Baldo J, D’Esposito presentation and verb generation. Cereb
M, Dronkers N, Okada K et al (2011) Cortex 6:1–10
Conduction aphasia, sensory-motor integra- 118. Warburton E, Wise RJS, Price CJ, Weiller C,
tion, and phonological short-term memory: Hadar U et al (1996) Noun and verb retrieval
an aggregate analysis of lesion and fMRI data. by normal subjects. Studies with PET. Brain
Brain Lang 119:119–128 119:159–179
380 Jeffrey R. Binder

119. Wise RSJ, Scott SK, Blank SC, Mummery 134. Dronkers NF, Wilkins DP, Van Valin RD,
CJ, Murphy K et al (2001) Separate neural Redfern BB, Jaeger JJ (2004) Lesion analysis
subsystems within ‘Wernicke’s area’. Brain of the brain areas involved in language com-
124:83–95 prehension. Cognition 92:145–177
120. Baddeley AD (1986) Working memory. 135. Patterson K, Nestor PJ, Rogers TT (2007)
Oxford University Press, Oxford Where do you know what you know? The
121. Paulesu E, Frith CD, Frackowiak RSJ (1993) representation of semantic knowledge in the
The neural correlates of the verbal component human brain. Nat Rev Neurosci 8:976–987
of working memory. Nature 362:342–345 136. Thompson-Schill SL (2003) Neuroimaging
122. Hickok G, Buchsbaum B, Humphries C, studies of semantic memory: inferring
Muftuler T (2003) Auditory-motor inter- “how” from “where”. Neuropsychologia
action revealed by fMRI: speech, music, 41:280–292
and working memory in area Spt. J Cognit 137. Martin A (2007) The representation of object
Neurosci 15(5):673–682 concepts in the brain. Annu Rev Psychol
123. Buchsbaum BR, Olsen RK, Koch P, Berman 58:25–45
KF (2005) Human dorsal and ventral audi- 138. Binder JR, Desai R, Conant LL, Graves WW
tory streams subserve rehearsal-based and (2009) Where is the semantic system? A
echoic processes during verbal working mem- critical review and meta-analysis of 120 func-
ory. Neuron 48(4):687–697 tional neuroimaging studies. Cereb Cortex
124. Buchsbaum BR, D’Esposito M (2008) The 19:2767–2796
search for the phonological store: from 139. Kiefer M, Pulvermüller F (2012) Conceptual
loop to convolution. J Cogn Neurosci representations in mind and brain: theoretical
20(5):762–778 developments, current evidence and future
125. Acheson DJ, Hamidi M, Binder JR, Postle directions. Cortex 48:805–825
BR (2011) A common neural substrate for 140. Meteyard L, Rodriguez Cuadrado S, Bahrami
language production and verbal working B, Vigliocco G (2012) Coming of age: a
memory. J Cogn Neurosci 23:1358–1367 review of embodiment and the neuroscience
126. DeWitt I, Rauschecker JP (2012) Phoneme of semantics. Cortex 48:788–804
and word recognition in the auditory ven- 141. Mummery CJ, Patterson K, Hodges JR, Wise
tral stream. Proc Natl Acad Sci U S A RJS (1996) Generating ‘tiger’ as an animal
109:E505–514 name or a word beginning with T: differ-
127. Bréal M (1897) Essai de sémantique (science ences in brain activation. Proc R Soc Lond B
des significations). Librairie Hachette, Paris 263:989–995
128. Alexander MP, Hiltbrunner B, Fischer RS 142. Price CJ, Moore CJ, Humphreys GW, Wise
(1989) Distributed anatomy of transcortical RJS (1997) Segregating semantic from pho-
sensory aphasia. Arch Neurol 46:885–892 nological processes during reading. J Cognit
129. Hart J, Gordon B (1990) Delineation of Neurosci 9(6):727–733
single-word semantic comprehension deficits 143. Cappa SF, Perani D, Schnur T, Tettamanti
in aphasia, with anatomic correlation. Ann M, Fazio F (1998) The effects of seman-
Neurol 27(3):226–231 tic category and knowledge type on lexical-
130. Chertkow H, Bub D, Deaudon C, Whitehead semantic access: a PET study. Neuroimage
V (1997) On the status of object concepts in 8(4):350–359
aphasia. Brain Lang 58:203–232 144. Roskies AL, Fiez JA, Balota DA, Raichle ME,
131. Tranel D, Damasio H, Damasio AR (1997) Petersen SE (2001) Task-dependent modu-
A neural basis for the retrieval of con- lation of regions in the left inferior frontal
ceptual knowledge. Neuropsychologia cortex during semantic processing. J Cognit
35:1319–1327 Neurosci 13(6):829–843
132. Gainotti G (2000) What the locus of brain 145. Binder JR, McKiernan KA, Parsons M,
lesion tells us about the nature of the cogni- Westbury CF, Possing ET et al (2003)
tive defect underlying category-specific disor- Neural correlates of lexical access during
ders: a review. Cortex 36:539–559 visual word recognition. J Cognit Neurosci
15(3):372–393
133. Damasio H, Tranel D, Grabowski T, Adolphs
R, Damasio A (2004) Neural systems behind 146. Devlin JT, Matthews PM, Rushworth MFS
word and concept retrieval. Cognition (2003) Semantic processing in the left infe-
92:179–229 rior prefrontal cortex: a combined functional
magnetic resonance imaging and transcranial
 fMRI of Language Systems 381

magnetic stimulation study. J Cogn Neurosci 160. Rohrer JD, Warren JD, Modat M, Ridgway
15(1):71–84 GR, Douiri A et al (2009) Patterns of corti-
147. Rissman J, Eliassen JC, Blumstein SE (2003) cal thinning in the language variants of fron-
An event-related fMRI investigation of totemporal lobar degeneration. Neurology
implicit semantic priming. J Cogn Neurosci 72:1562–1569
15(8):1160–1175 161. Mion M, Patterson K, Acosta-Cabronero J,
148. Scott SK, Leff AP, Wise RJS (2003) Going Pengas G, Izquierdo-Garcia D et al (2010)
beyond the information given: a neural sys- What the left and right anterior fusiform
tem supporting semantic interpretation. gyri tell us about semantic memory. Brain
Neuroimage 19:870–876 133:3256–3268
149. Ischebeck A, Indefrey P, Usui N, Nose I, 162. Warrington EK, Shallice T (1984) Category
Hellwig F et al (2004) Reading in a regular specific semantic impairments. Brain
orthography: an fMRI study investigating the 107:829–854
role of visual familiarity. J Cognit Neurosci 163. Gonnerman LM, Andersen ES, Devlin JT,
16(5):727–741 Kempler D, Seidenberg MS (1997) Double
150. Binder JR, Westbury CF, Possing ET, dissociation of semantic categories in
McKiernan KA, Medler DA (2005) Distinct Alzheimer’s disease. Brain Lang 57:254–279
brain systems for processing concrete 164. Chan AS, Salmon DP, De La Pena J (2001)
and abstract concepts. J Cognit Neurosci Abnormal semantic network for “animals”
17(6):905–917 but not “tools” in patients with Alzheimer’s
151. Binder JR, Medler DA, Desai R, Conant LL, disease. Cortex 37:197–217
Liebenthal E (2005) Some neurophysiologi- 165. Fung TD, Chertkow H, Whatmough C,
cal constraints on models of word naming. Murtha S, Péloquin L et al (2001) The spec-
Neuroimage 27:677–693 trum of category effects in object and action
152. Sabsevitz DS, Medler DA, Seidenberg M, knowledge in dementia of the Alzheimer’s
Binder JR (2005) Modulation of the seman- type. Neuropsychology 15(3):371–379
tic system by word imageability. Neuroimage 166. Alexander MP, Benson DF, Stuss DT (1989)
27:188–200 Frontal lobes and language. Brain Lang
153. Vandenbulcke M, Peeters R, Fannes K, 37:656–691
Vandenberghe R (2006) Knowledge of 167. Mazoyer BM, Tzourio N, Frak V, Syrota A,
visual attributes in the right hemisphere. Nat Murayama N et al (1993) The cortical rep-
Neurosci 9(7):964–970 resentation of speech. J Cognit Neurosci
154. Damasio H (1989) Neuroimaging contri- 5(4):467–479
butions to the understanding of aphasia. In: 168. Stowe LA, Paans AMJ, Wijers AA, Zwarts F,
Boller F, Grafman J (eds) Handbook of neu- Mulder G et al (1999) Sentence comprehen-
ropsychology. Elsevier, Amsterdam, pp 3–46 sion and word repetition: a positron emission
155. Rapcsak SZ, Rubens AB (1994) Localization tomography investigation. Psychophysiology
of lesions in transcortical aphasia. In: Kertesz 36:786–801
A (ed) Localization and neuroimaging in 169. Friederici AD, Meyer M, von Cramon DY
neuropsychology. Academic, San Diego, (2000) Auditory language comprehension:
pp 297–329 an event-related fMRI study on the process-
156. Berthier ML (1999) Transcortical aphasias. ing of syntactic and lexical information. Brain
Psychology, Hove Lang 74:289–300
157. Mummery CJ, Patterson K, Price CJ, Ashburner 170. Vandenberghe R, Nobre AC, Price CJ (2002)
J, Frackowiak RS et al (2000) A voxel-based The response of left temporal cortex to sen-
morphometry study of semantic dementia: rela- tences. J Cogn Neurosci 14(4):550–560
tionship between temporal lobe atrophy and 171. Humphries C, Swinney D, Love T, Hickok G
semantic memory. Ann Neurol 47:36–45 (2005) Response of anterior temporal cortex
158. Rosen HJ, Gorno-Tempini ML, Goldman to syntactic and prosodic manipulations dur-
WP et al (2002) Patterns of brain atrophy ing sentence processing. Hum Brain Mapp
in frontotemporal dementia and semantic 26:128–138
dementia. Neurology 58:198–208 172. Humphries C, Binder JR, Medler DA,
159. Davies RR, Hodges JR, Krill JJ, Patterson Liebenthal E (2006) Syntactic and semantic
K, Halliday GM et al (2005) The patho- modulation of neural activity during auditory
logical basis of semantic dementia. Brain sentence comprehension. J Cognit Neurosci
128:1984–1995 18:665–679
382 Jeffrey R. Binder

173. Pallier C, Devauchelle AD, Devauchelle AD, 186. Wartenburger I, Heekeren HR, Burchert
Dehaene S (2011) Cortical representation of F, Heinemann S, De Bleser R et al (2004)
the constituent structure of sentences. Proc Neural correlates of syntactic transformations.
Natl Acad Sci U S A 108(6):2522–2527 Hum Brain Mapp 22:72–81
174. Humphries C, Binder JR, Medler DA, 187. Fiebach CJ, Schlesewsky M, Lohmann G
Liebenthal E (2007) Time course of semantic (2005) Revisiting the role of Broca’s area
processes during sentence comprehension: an in sentence processing: Syntactic integration
fMRI study. Neuroimage 36(3):924–932 versus syntactic working memory. Hum Brain
175. Kang AM, Constable RT, Gore JC, Avrutin Mapp 24:79–91
S (1999) An event-related fMRI study of 188. Chen E, West WC, Waters G, Caplan D
implicit phrase-level syntactic and semantic (2006) Determinants of BOLD signal cor-
processing. Neuroimage 10(5):98–110 relates of processing object-extracted relative
176. Embick D, Marantz A, Miyashita Y, O’Neil clauses. Cortex 42:591–604
W, Sakai KL (2000) A syntactic specializa- 189. Caplan D, Stanczak L, Waters G (2008)
tion for Broca’s area. Proc Natl Acad Sci USA Syntactic and thematic constraint effects
97:6150–6154 on blood oxygenation level dependent sig-
177. Meyer M, Friederici AD, von Cramon DY nal correlates of comprehension of relative
(2000) Neurocognition of auditory sentence clauses. J Cogn Neurosci 20(4):643–656
comprehension: event related fMRi reveals 190. Just MA, Carpenter PA, Keller TA, Eddy WF,
sensitivity to syntactic violations and task Thulborn KR (1996) Brain activation modu-
demands. Cogn Brain Res 9:19–33 lated by sentence comprehension. Science
178. Ni W, Constable RT, Mencl WE, Pugh KR, 274:114–116
Fullbright RK et al (2000) An event-related 191. Stowe LA, Broere CA, Paans AM, Wijers
neuroimaging study distinguishing form and AA, Mulder G et al (1998) Localizing com-
content in sentence processing. J Cognit ponents of a complex task: Sentence pro-
Neurosci 12:120–133 cessing and working memory. Neuroreport
179. Newman AJ, Pancheva R, Ozawa K, Neville 9:2995–2999
HJ, Ullman MT (2001) An event-related 192. Caplan D, Waters GS (1999) Verbal working
fMRI study of syntactic and semantic viola- memory and sentence comprehension. Behav
tions. J Psycholinguist Res 30:339–364 Brain Sci 22(1):77–94
180. Kuperberg GR, Holcomb PJ, Sitnikova T, 193. Keller TA, Carpenter PA, Just MA (2001)
Greve D, Dale AM et al (2003) Distinct pat- The neural bases of sentence comprehension:
terns of neural modulation during the pro- a fMRI examination of syntactic and lexical
cessing of conceptual and syntactic anomalies. processing. Cereb Cortex 11:223–237
J Cognit Neurosci 15(2):272–293 194. Cooke A, Zurif EB, DeVita C, Alsop D,
181. Caplan D, Alpert N, Waters GS (1998) Effects Koenig P et al (2002) Neural basis for sen-
of syntactic structure and prepositional num- tence comprehension: grammatical and short-
ber on patterns of regional cerebral blood term memory components. Hum Brain Mapp
flow. J Cogn Neurosci 10(4):541–552 15:80–94
182. Fiebach CJ, Schlesewsky M, Friederici 195. Novick JM, Trueswell JC, Thompson-Schill
AD (2001) Syntactic working memory SL (2005) Cognitive control and parsing:
and the establishment of filler-gap depen- reexamining the role of Broca’s area in sen-
dencies: insights from ERPs and fMRI. J tence comprehension. Cognit Affect Behav
Psycholinguist Res 30:321–338 Neurosci 5(3):263–281
183. Ben-Shachar M, Hendler T, Kahn I, Ben- 196. Rogalsky C, Hickok G (2011) The role of
Bashat D, Grodzinsky Y (2003) The neural Broca’s area in sentence comprehension.
reality of syntactic transformations: evidence J Cogn Neurosci 23(7):1664–1680
form fMRI. Psychol Sci 14:433–440 197. Caplan D, Waters G (2013) Memory mecha-
184. Friederici AD, Rüschemeyer SA, Hahne A, nisms supporting syntactic comprehension.
Fiebach CJ (2003) The role of left inferior fron- Psychon Bull Rev 20(2):243–268
tal gyrus and superior temporal cortex in sen- 198. Caplan D (2001) Functional neuroim-
tence comprehension: localizing syntactic and aging studies of syntactic processing.
semantic processes. Cerebr Cortex 13:170–177 J Psycholinguist Res 30(3):297–320
185. Ben-Shachar M, Palti D, Grodzinsky Y (2004) 199. Friederici AD, Kotz SA (2003) The brain basis
The neural correlates of syntactic movement: of syntactic processes: functional imaging and
converging evidence from two fMRI experi- lesion studies. Neuroimage 20:S8–S17
ments. Neuroimage 21:1320–1336
 fMRI of Language Systems 383

200. Martin RC (2003) Language processing: within the human auditory cortex when lis-
functional organization and neuroanatomical tening to words. Neurosci Lett 146:179–182
basis. Annu Rev Psychol 54:55–89 214. Binder JR, Rao SM, Hammeke TA, Frost JA,
201. Grodzinsky Y, Friederici AD (2006) Bandettini PA et al (1994) Effects of stimu-
Neuroimaging of syntax and syntactic pro- lus rate on signal response during functional
cessing. Curr Opin Neurobiol 16:240–246 magnetic resonance imaging of auditory cor-
202. Bornkessel-Schlesewsky I, Schlesewsky M tex. Cogn Brain Res 2:31–38
(2013) Reconciling time, space and function: 215. Lehéricy S, Cohen L, Bazin B, Samson S,
a new dorsal-ventral stream model of sentence Giacomini E et al (2000) Functional MR
comprehension. Brain Lang 125(1):60–76 evaluation of temporal and frontal language
203. Badre D, Poldrack RA, Pare-Blagoev EJ, dominance compared with the Wada test.
Insler RZ, Wagner AD (2005) Dissociable Neurology 54:1625–1633
controlled retrieval and generalized selection 216. Humphries C, Willard K, Buchsbaum B,
mechanisms in ventrolateral prefrontal cortex. Hickok G (2001) Role of anterior temporal
Neuron 47:907–918 cortex in auditory sentence comprehension:
204. Scott SK, Blank C, Rosen S, Wise RJS an fMRI study. Neuroreport 12:1749–1752
(2000) Identification of a pathway for intel- 217. Crinion JT, Lambon-Ralph MA, Warburton
ligible speech in the left temporal lobe. Brain EA, Howard D, Wise RJS (2003) Temporal
123:2400–2406 lobe regions engaged during normal speech
205. Davis MH, Johnsrude IS (2003) Hierarchical comprehension. Brain 126:1193–1201
processing in spoken language comprehen- 218. Spitsyna G, Warren JE, Scott SK, Turkheimer
sion. J Neurosci 23(8):3423–3431 FE, Wise RJS (2006) Converging lan-
206. Specht K, Reul J (2003) Functional segrega- guage streams in the human temporal lobe.
tion of the temporal lobes into highly differ- J Neurosci 26(28):7328–7336
entiated subsystems for auditory perception: 219. Awad M, Warren JE, Scott SK, Turkheimer
an auditory rapid event-related fMRI task. FE, Wise RJS (2007) A common system for
Neuroimage 20:1944–1954 the comprehension and production of narra-
207. Uppenkamp S, Johnsrude IS, Norris D, tive speech. J Neurosci 27(43):11455–11464
Marslen-Wilson W, Patterson RD (2006) 220. Eulitz C, Elbert T, Bartenstein P, Weiller C,
Locating the initial stages of speech-sound Müller SP et al (1994) Comparison of mag-
processing in human temporal cortex. netic and metabolic brain activity during a
Neuroimage 31:1284–1296 verb generation task. NeuroReport 6:97–100
208. Binder JR, Swanson SJ, Hammeke TA, 221. Ojemann JG, Buckner RL, Akbudak E, Snyder
Sabsevitz DS (2008) A comparison of five AZ, Ollinger JM et al (1998) Functional MRI
fMRI protocols for mapping speech compre- studies of word-stem completion: reliability
hension systems. Epilepsia 49(12):1980–1997 across laboratories and comparison to blood
209. Thompson-Schill SL, Aguirre GK, D’Esposito flow imaging with PET. Hum Brain Mapp
M, Farah MJ (1997) Role of left inferior pre- 6:203–215
frontal cortex in retrieval of semantic knowl- 222. Yetkin FZ, Swanson S, Fischer M, Akansel
edge: a reevaluation. Proc Natl Acad Sci U S G, Morris G et al (1998) Functional MR
A 94:14792–14797 of frontal lobe activation: comparison with
210. Thompson-Schill SL, D’Esposito M, Kan IP Wada language results. Am J Neuroradiol
(1999) Effects of repetition and competition 19:1095–1098
on activity in left prefrontal cortex during 223. Benson RR, FitzGerald DB, LeSeuer LL,
word generation. Neuron 23:513–522 Kennedy DN, Kwong KK et al (1999)
211. Wise R, Chollet F, Hadar U, Friston K, Language dominance determined by whole
Hoffner E et al (1991) Distribution of cor- brain functional MRI in patients with brain
tical neural networks involved in word lesions. Neurology 52:798–809
comprehension and word retrieval. Brain 224. Palmer ED, Rosen HJ, Ojemann JG, Buckner
114:1803–1817 RL, Kelley WM et al (2001) An event-related
212. Price CJ, Wise RJS, Warburton EA, Moore fMRI study of overt and covert word stem
CJ, Howard D et al (1996) Hearing and say- completion. Neuroimage 14:182–193
ing. The functional neuro-anatomy of audi- 225. Liégois F, Connelly A, Salmond CH, Gadian
tory word processing. Brain 119:919–931 DG, Vargha-Khadem F et al (2002) A direct
213. Price C, Wise R, Ramsay S, Friston K, Howard test for lateralization of language activation
D et al (1992) Regional response differences using fMRI: comparison with invasive assess-
384 Jeffrey R. Binder

ments in children with epilepsy. Neuroimage 239. Devlin JT, Russell RP, Davis MH, Price
17:1861–1867 CJ, Moss HE et al (2002) Is there an ana-
226. Raichle ME, Fiez JA, Videen TO, MacLeod AM, tomical basis for category-specificity?
Pardo JV et al (1994) Practice-related changes Semantic memory studies with PET and
in human brain functional anatomy during non- fMRI. Neuropsychologia 40:54–75
motor learning. Cereb Cortex 4(1):8–26 240. Xu B, Grafman J, Gaillard WD, Spanaki M,
227. Petersen SE, Fox PT, Posner MI, Mintun M, Ishii K et al (2002) Neuroimaging reveals
Raichle ME (1988) Positron emission tomo- automatic speech coding during percep-
graphic studies of the cortical anatomy of tion of written word meaning. Neuroimage
single-word processing. Nature 331:585–589 17(2):859–870
228. Malach R, Reppas JB, Benson RR, Kwong 241. Mummery CJ, Patterson K, Hodges JR, Price
KK, Jiang H et al (1995) Object-related activ- CJ (1998) Functional neuroanatomy of the
ity revealed by functional magnetic resonance semantic system: divisible by what? J Cogn
imaging in human occipital cortex. Proc Natl Neurosci 10(6):766–777
Acad Sci USA 92:8135–8139 242. Miceli G, Turriziani P, Caltagirone C, Capasso
229. Kanwisher N, Woods R, Iacoboni M, R, Tomaiuolo F et al (2002) The neural cor-
Mazziotta J (1996) A locus in human relates of grammatical gender: an fMRI inves-
extrastriate cortex for visual shape analysis. tigation. J Cognit Neurosci 14:618–628
J Cognit Neurosci 91:133–142 243. Bavelier D, Corina D, Jezzard P, Padmanabhan
230. Grill-Spector K, Kushnir T, Edelman S, S, Clark VP et al (1997) Sentence reading:
Avidian-Carmel G, Itzchak Y et al (1999) a functional MRI study at 4 tesla. J Cognit
Differential processing of objects under vari- Neurosci 9(5):664–686
ous viewing conditions in the human lateral 244. Herbster AN, Mintun MA, Nebes RD, Becker
occipital complex. Neuron 24:187–203 JT (1997) Regional cerebral blood flow dur-
231. Martin A, Wiggs CL, Ungerleider LG, ing word and nonword reading. Hum Brain
Haxby JV (1996) Neural correlates of Mapp 5:84–92
category-specific knowledge. Nature 245. Indefrey P, Kleinschmidt A, Merboldt KD,
379(6566):649–652 Krüger G, Brown C et al (1997) Equivalent
232. Price CJ, Moore CJ, Humphreys GW, responses to lexical and nonlexical visual
Frackowiak RSJ, Friston KJ (1996) The stimuli in occipital cortex: a functional mag-
neural regions sustaining object recogni- netic resonance imaging study. Neuroimage
tion and naming. Proc Roy Soc Lond B 5:78–81
263:1501–1507 246. Chee MW, Caplan D, Soon CS, Sriram
233. Zelkowicz BJ, Herbster AN, Nebes RD, N, Tan EWL et al (1993) Processing of
Mintun MA, Becker JT (1998) An examina- visually presented sentences in Mandarin
tion of regional cerebral blood flow during and English studied with fMRI. Neuron
object naming tasks. J Int Neuropsychol Soc 23:127–137
4:160–166 247. Démonet JF, Wise R, Frackowiak RSJ (1993)
234. Murtha S, Chertkow H, Beauregard M, Evans Language functions explored in normal sub-
A (1999) The neural substrate of picture jects by positron emission tomography: a crit-
naming. J Cognit Neurosci 11(4):399–423 ical review. Hum Brain Mapp 1(1):39–47
235. Price CJ, Devlin JT, Moore CJ, Morton C, Laird 248. Fiez JA (1997) Phonology, semantics and
AR (2005) Meta-analyses of object naming: the role of the left inferior prefrontal cortex.
effect of baseline. Hum Brain Mapp 25:70–82 Hum Brain Mapp 5:79–83
236. Kiasawa M, Inoue C, Kawasaki T, Tokoro T, 249. Poldrack RA, Wagner AD, Prull MW,
Ishii K et al (1996) Functional neuroanatomy Desmond JE, Glover GH et al (1999)
of object naming: a PET study. Graefes Arch Functional specialization for semantic and
Clin Exp Ophthalmol 234:110–115 phonological processing in the left inferior
237. Vandenberghe R, Price C, Wise R, Josephs O, prefrontal cortex. Neuroimage 10:15–35
Frackowiak RSJ (1996) Functional anatomy 250. Gold BT, Buckner RL (2002) Common
of a common semantic system for words and prefrontal regions coactivate with disso-
pictures. Nature 383:254–256 ciable posterior regions during controlled
238. Carpentier A, Pugh KR, Westerveld M, semantic and phonological tasks. Neuron
Studholme C, Skrinjar O et al (2001) 35:803–812
Functional MRI of language processing: 251. Henson RNA, Price CJ, Rugg MD, Turner
dependence on input modality and temporal R, Friston KJ (2002) Detecting latency dif-
lobe epilepsy. Epilepsia 42:1241–1254 ferences in event-related BOLD responses:
 fMRI of Language Systems 385

application to words versus nonwords and demand in healthy volunteers: An fMRI study.
initial versus repeated face presentations. Synapse 42:266–272
Neuroimage 15(1):83–97 256. Braver TS, Barch DM, Gray JR, Molfese DL,
252. Mechelli A, Gorno-Tempini ML, Price CJ Snyder A (2001) Anterior cingulate cortex and
(2003) Neuroimaging studies of word and response conflict: effects of frequency, inhibi-
pseudoword reading: consistencies, incon- tion and errors. Cereb Cortex 11:825–836
sistencies, and limitations. J Cogn Neurosci 257. Ullsperger M, Von Cramon DY (2001)
15(2):260–271 Subprocesses of performance monitoring: a
253. Braver TS, Cohen JD, Nystrom LE, Jonides dissociation of error processing and response
J, Smith EE et al (1997) A parametric study competition revealed by event-related fMRI
of prefrontal cortex involvement in human and ERPs. Neuroimage 14:1387–1401
working memory. Neuroimage 5:49–62 258. Binder JR, Liebenthal E, Possing ET,
254. Honey GD, Bullmore ET, Sharma T (2000) Medler DA, Ward BD (2004) Neural cor-
Prolonged reaction time to a verbal working relates of sensory and decision processes in
memory task predicts increased power of pos- auditory object identification. Nat Neurosci
terior parietal cortical activation. Neuroimage 7(3):295–301
12(5):495–503 259. Desai R, Conant LL, Waldron E, Binder JR
255. Adler CM, Sax KW, Holland SK, Schmithorst (2006) FMRI of past tense processing: the
V, Rosenberg L et al (2001) Changes in neu- effects of phonological complexity and task
ronal activation with increasing attention difficulty. J Cognit Neurosci 18(2):278–297
 Chapter 13

Neuroimaging Approaches to the Study of Visual Attention

George R. Mangun, Yuelu Liu, Jesse J. Bengson, Sean P. Fannon,
Nicholas E. DiQuattro, and Joy J. Geng

Abstract
Selective attention is a core cognitive ability that enables organisms to effectively process and act upon relevant
information while ignoring distracting events. Elucidating the neural bases of selective attention remains a key
challenge for neuroscience and represents an essential aim in translational efforts to ameliorate attentional deficits
in a wide variety of neurological and psychiatric disorders. Moreover, knowledge about the cognitive and neural
mechanisms of attention is essential for developing and refining brain–machine interfaces, and for advancing
methods for training and education. We will discuss how functional imaging methods have helped us to under-
stand fundamental aspects of attention: How attention is controlled, focused on relevant inputs, and reoriented,
and how this control results in the selection of relevant information. Work from our groups and from others will
be reviewed. We will focus on fMRI methods, but where appropriate will include related discussion of electro-
magnetic recording methods used in conjunction with fMRI, including simultaneous EEG/fMRI methods.

Key words Attention, Control, fMRI, EEG, ERPs, DCM, Human, Vision

1 Introduction

Attention is a key cognitive ability that supports our momentary

awareness, and affects how we analyze sensory inputs, retain informa-
tion in memory, process it for meaning, and, finally, act upon it. In
this review chapter we will consider the role of attention in sensory
processing and perception. We begin with a definition of attention as
it will be investigated in the studies to follow, starting with nineteenth
century insights provided by psychologist William James [1]:
“Everyone knows what attention is. It is the taking possession by the
mind, in clear and vivid form, of one out of what seem several simulta-
neously possible objects or trains of thought. … It implies withdrawal
from some things in order to deal effectively with others, …”
Relying on introspection, James noted key characteristics of atten-
tion that frame the theoretical landscape of attentional mechanisms.
He noted the voluntary aspects of attention and its selection of rele-
vant from irrelevant information, as well as its capacity limitations.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_13, © Springer Science+Business Media New York 2016

387
388 George R. Mangun et al.

Our review will focus on studies of visual attention, and

therefore, we also consider the work of James’ contemporary,
Hermann von Helmholtz [2], who in his studies of visual psycho-
physics made observations and speculations on the mechanisms of
visual attention. He wrote, in studies investigating the limits of
visual perception, the following:
“… by a voluntary kind of intention, even without eye movements,
and without changes of accommodation, one can concentrate atten-
tion on the sensation from a particular part of our peripheral nervous
system and at the same time exclude attention from all other parts.”
Helmholtz’s observations are in line with today’s knowledge
about attention mechanisms, which include detailed understand-
ing of how attention influences stimulus processing at early and
late stages of sensory analysis.

1.1 Varieties Attention is neither a single capability, nor is it supported by a single

of Attention mechanism or brain system. Theoretically, we can consider two main
forms of attention—voluntary attention and reflexive attention [3].
Voluntary attention is goal-directed and suggests a top-down influ-
ence that is under intentional control. In contrast, reflexive attention
is a stimulus-driven process involving bottom-up effects, for exam-
ple, as when a salient sensory signal grabs our attention. These two
general categories of attention differ in their properties and perhaps
their neural mechanisms. In this chapter, we will concentrate on
studies of voluntary attention to illustrate how functional imaging
has been used to elucidate brain attention mechanisms.
Another way of thinking about attention mechanisms is to
consider the domain of information processing on which attention
operates. For example, attention can operate within or between
sensory modalities. That is, we can attend to visual inputs at the
expense of auditory ones, or vice versa, or may attend to one aspect
of visual input (e.g., stimulus location) at the expense of other
stimulus attributes (e.g., color or motion). Thus, one may ask
whether the mechanisms of attentional control are similar or dif-
ferent for attention to different sensory modalities compared with
attention to different stimulus attributes [4]. One may also con-
sider whether the mechanisms supporting stimulus selection are
modality and attribute independent, being instead specialized for
the particular items to be attended [5]. In order to constrain the
discussion in this chapter, for the most part, we will focus on visual
attention, emphasizing attentional control and selection for atten-
tion based on stimulus location (spatial attention) and elementary
stimulus features (e.g., color and motion).

1.2 Early and Late One of the key questions in attention research has been where in
Selection Models information processing attention has its influence. If as Helmholtz
suggested, attention could select some information coming from
 Neuroimaging Approaches to the Study of Visual Attention 389

the peripheral nervous system, then it is essential to ask where

within the ascending sensory pathways attention can alter stimulus
processing to achieve selective processing. In the 1950s, Broadbent
[6] described the idea of an attentional gate that could be opened
for attended information and closed for ignored information. Like
Helmholtz, Broadbent suggested that information selection might
occur early in sensory processing. This idea has been termed early
selection, and it is the idea that a stimulus need not be completely
perceptually analyzed before it can be selected for further process-
ing or rejected by a gating mechanism.
In contrast, so-called late selection models hold that all
(attended and unattended) sensory inputs are processed equiva-
lently by the perceptual system to a very high level of coding before
they are selected by attention [7]. This high level is generally con-
sidered to be the level of categorical, or semantic, encoding where
the elementary feature codes (e.g., orientation, contrast, color,
form) are replaced by conceptual codes (e.g., that is a chair). Late
selection models posit that selection takes place on these higher-
level codes and thereafter representation in awareness may take
place. This view, then, holds that selective attention does not influ-
ence our perceptions of stimuli by changing the low-level sensory-
perceptual processing of the stimulus.
A long debate and many studies have addressed the early ver-
sus late selection controversy [8–11], and physiological approaches
including functional imaging have provided key information about
the stages of sensory processing that are influenced by selective
attention. We will thus begin our consideration of attention with
its role in stimulus selection processes and how functional imaging
has been used to investigate these mechanisms. But first, in the
next section, some design issues in the study of selective attention
deserve consideration as they are of paramount importance for
physiological studies of attention, including functional imaging.

1.3 Methodological The focus of the vast majority of studies of effects of attention on
Issues in Experimental perception involves selective attention, attention to one thing at
Studies of Selective the expense of another. This is to be contrasted with nonselective
Attention attention, which includes generalized behavioral arousal (e.g., the
classic orienting response) that does not necessarily involve attend-
ing one input while ignoring another. These latter nonselective
attention mechanisms are certainly interesting, but selective mech-
anisms have generated the greatest interest and we review only
these studies. Therefore, it is critical to understand the design
parameters that permit selective versus nonselective attention to be
isolated and studied, and to make note of some of the confounding
influences that might contaminate studies of selective attention
when nonselective factors (e.g., arousal) are not properly con-
trolled. We turn then to an example from the early history of the
physiology of attention.
390 George R. Mangun et al.

In the 1950s, the great Mexican neurophysiologist Raúl

Hernández Peón and his colleagues [12] studied the neuroanatomy
and neurophysiology of the ascending sensory pathways and how
top-down attention might modulate sensation. Their work in the
auditory system was motivated by the well-known neuroanatomical
substrate for top-down modulation, the olivocochlear bundle
(OCB), which involves centrifugal neural projections from higher
levels of the central nervous system downward to earlier processing
stages especially the peripheral nervous system out to the level of
the cochlea. The OCB provided a very strong neuroanatomical
mechanism by which top-down effects of mental processes like
attention might gate early auditory processing, in a fashion sug-
gested by Helmholtz and later Broadbent, among others.
Hernández Peón’s group recorded the activity in neurons in
the subcortical auditory pathway in cats while they were either pas-
sively listening to the sounds from a speaker or were not attending
the sounds. By showing the cats two live mice safely contained in a
closed bottle, the cats were induced to ignore the sounds while
they attended the mice. The researchers found that the amplitude
of activity recorded from electrodes implanted in the cochlear
nucleus was larger when the animals attended the sounds versus
ignored the sounds while attending the bottled mice. This was
interpreted as evidence that selective attention influences stimulus
processing as early as the subcortical sensory pathways via the influ-
ence of the top–down neural control inputs.
This work, published in the journal Science, would however,
later be shown to include a fatal flaw. The flaw was that the sounds
presented to the cats to evoke auditory activity were delivered by
speakers near the cats. When the cats attended the mice, they also
oriented their ears and heads toward the mice, and away from the
speakers, thereby altering the amplitudes of the auditory sounds at
the ears due to simple physical differences in the relation of the ears
to the sounds. Changes in the amplitudes of the sounds at the ear
lead, in and of themselves, to changes in the amplitudes of auditory
responses in the ascending pathways, and this tells us nothing
about attention, although it mimics the kind of effect one would
expect from an attentional mechanism. This problem can be elimi-
nated by controlling the amplitudes and qualities of the sensory
stimuli when attended and ignored so that an ensuing difference in
neural responses would be attributable to internal attentional
modulations of sensory processing, not differences in the physical
stimuli themselves across conditions. Similarly, the two conditions
of Hernández Peón and colleagues likely also differed in nonspe-
cific behavioral arousal that may also have influenced the neuronal
recordings since when cats see mice they are undoubtedly more
aroused than when they listen to Beethoven, although the oppo-
site would hopefully hold for at least some human listeners. In
either case, differences in behavioral arousal between comparison
 Neuroimaging Approaches to the Study of Visual Attention 391

conditions could confound the effects of selective attention, and

therefore, like physical stimulus differences, must be rigorously
controlled [13]. Failure to do so properly could lead to changes in,
for example, hemodynamic signals that would present a serious
confound in functional imaging studies of attention.

2 Imaging Attentional Selection Mechanisms

Studies using electrophysiological recordings have provided evidence

that voluntary selective attention can influence the processing of sen-
sory inputs at early stages of neural analysis. Hillyard and colleagues
[14] in what is considered the first properly controlled study of audi-
tory selective attention in humans, showed that scalp-recorded audi-
tory cortical event-related potentials (ERPs) were enhanced with
selective attention. In vision, the Hillyard group [15] similarly dem-
onstrated in humans that by 70-ms post stimulus onset, the electrical
brain response to a stimulus presented to an attended spatial location
was enhanced compared with an identical stimulus presented when
that location was ignored; similar findings reported previously could
not rule out nonspecific effects of arousal or differential preparation
for relevant targets based on learning the stimulus sequence [16, 17].
The timing of the ERP method (in the order of milliseconds)
provides strong evidence that early stages of information processing
were influenced by visual selective attention because the first inputs
to the human visual cortex arrive only at about 40 ms after stimulus
onset (based on intracranial recordings in patients). However, the
scalp recordings are limited in the spatial localization they can pro-
vide in humans because the ERPs are recorded outside the skull and
are, therefore, distant from the neuroelectric sources generating the
signals. The volume conduction of the electrical signal through the
brain, skull, and scalp is, on the one hand, an advantage that per-
mits scalp recordings in the first place, but it also means that the
electrical currents spread on the scalp (and are also filtered and spa-
tially blurred by the intervening tissues, especially the skull) and are
hard to track backward to their three-dimensional intracranial site
of generation [18]. Related studies in animals showed that area
V4 in extrastriate cortex in the macaque monkey showed increased
neuronal firing rates to attended-location stimuli as compared with
unattended stimuli [19], and subsequent work has extended this to
other extrastriate areas [20–22], as well as striate cortex [23, 24]
and the subcortical visual relays in the thalamus [25].
In humans, however, studies using functional imaging meth-
ods have provided the most detailed information about where in
the ascending pathways visual selective attention modulates stimu-
lus processing, because methods such as positron emission tomog-
raphy (PET) and functional magnetic resonance imaging (fMRI)
provide precise information about the neuroanatomical loci of
attention effects and attention processes in the brain.
392 George R. Mangun et al.

2.1 Functional Functional imaging methods provide neuroanatomical information

Imaging of Visual– about the mechanisms of human attention. That is, together with
Spatial Attention information from neurophysiological recordings, imaging approaches
help to locate the neuroanatomical stages of information processing
influenced by attention. For visual-spatial attention (see [26] for stud-
ies of nonspatial attention), we began using O15 PET to provide func-
tional anatomical information to complement our earlier ERP studies.
Indeed, in these early studies that began in 1991, we first combined
ERP and PET methods to provide a spatial–temporal approach to
studying human attention [27, 28]. In subsequent studies beginning
in the mid-1990s, we incorporated fMRI methods.
The goal of these functional imaging studies was to identify
where within the visual system visual–spatial selective attention first
influenced sensory analysis. We presented subjects with bilateral
stimulus arrays of nonsense symbols (two in each hemifield) flashed
at a rapid varying rate, averaging about two stimulus arrays per
second (Fig. 1). In order to control for physical stimulus differ-
ences between conditions, the subjects were required to maintain
fixation on a central fixation point and their compliance was
ensured using high-resolution infrared photometric monitoring

Fig. 1 Stimuli and task used in a functional imaging and event-related potential
(ERP) study of spatial attention [105]. Two blocked conditions of attention are
shown: Attend left condition (left column) and attend right condition (right col-
umn). Subjects viewed rapid sequences of arrays (about 2.5/s) of nonsense sym-
bols (flashed for 100 ms) while maintaining fixation of their eyes on a central
fixation spot (plus sign). There were always two symbols in the left and two in the
right visual hemifield in locations each demarcated by an outline rectangle. The
task was to detect and press a button to pairs of identical symbols at the attended
location and to ignore all stimuli in the opposite hemifield. In the figure (but not
in the actual experiment) the focus of covert spatial attention is indicated by a
dashed circle
 Neuroimaging Approaches to the Study of Visual Attention 393

and the horizontal electro-oculograms (EOGs). In order to con-

trol for differences in nonspecific behavioral arousal, we included
two main attention conditions that were equated in task difficulty
and arousal. The subjects had to covertly pay attention to the left
half of the array in one condition and ignore the right (attend left
condition), or attend the right half of the array while ignoring the
left (attend right condition). Hemodynamic responses (activations)
in the brain could be compared for attend-left versus attend-right
conditions. To be clear, the relevant design features were: (1) the
stimulus arrays were identical, striking the same regions of the ret-
ina in the two attention conditions, (2) the two attention condi-
tions (attend left and attend right) were also equated for nonspecific
arousal because the discrimination task was identical for both con-
ditions, and (3) the stimuli were presented randomly in a nonpre-
dictable sequence, so that the subjects could not adjust the global
level of attentiveness on a trial-by-trial basis. As a result, any
changes we observed in visual cortex could not be attributed to
either differences in visual stimulation or to nonspecific factors
such as arousal, but instead could be attributed to spatially-specific
modulations of visual processing with the direction of spatial
attention.
We observed that spatial attention led to activations in extrastri-
ate cortex in the cerebral hemisphere that was contralateral to the

Fig. 2 fMRI and event-related potential (ERP) data from a study of visual-spatial selective attention. (a) Coronal
structural scan of a single subject showing activations in contralateral visual cortex with spatial attention in
the extrastriate cortex (left hemisphere is on the left). The activations were focused in the lingual gyrus (LG)
and posterior fusiform gyrus (FG) and the middle occipital gyrus (MOG). (b) ERP attention effects shown as
difference waves from a single lateral occipital electrode site in the right hemisphere (attend left minus attend
right). The vertical scale is 2 μV per side (positive plotted downward). The onset of the array is indicated at time
zero (t = 0), and the tick marks are 100 ms. (c) Topographic voltage attention difference map (110-ms latency)
on the scalp surface viewed from the rear (left side of head on left side of figure) (adapted from [105])
394 George R. Mangun et al.

attended side of the stimulus arrays (Fig. 2a). These activations were
highly statistically significant in the posterior fusiform gyrus on the
ventral cortical surface and in lateral-ventral occipital regions. In
these early studies, no effects of visual-spatial attention were observed
in primary visual cortex (area V1) but as we shall see later, such
effects have since been observed using fMRI. The finding of local-
ized brain activations corresponding to the action of spatial atten-
tion alone is in accord with evidence from prior ERP studies [15, 29,
30]. Also in this study, in a separate session, testing the same sub-
jects, we recorded ERPs for the same stimuli and task. The scalp-
recorded ERPs showed the expected P1 attention effects (Fig. 2b,
c), and by using neuroelectric dipole modeling we investigated
whether intracranial neural generators at the loci of functional acti-
vations could produce activity in the model scalp that was similar to
what we actually recorded in the ERPs (not shown in figure). We
showed that the ERP and fMRI attention effects in the posterior
fusiform gyrus (i.e., extrastriate visual cortex) were strongly related.
This combined use of ERPs and functional imaging provided
evidence for short-latency (around 100 ms after stimulus onset)
changes in responses to visual stimuli as a function of spatial attention
that were generated early in extrastriate visual cortex. In several stud-
ies, we and others have followed up these effects [31, 32] and have
observed that these modulations with spatial attention affect multiple
stages of visual cortical processing, from V1 toward inferotemporal
cortex in the ventral visual stream. The next section reviews related
work that also combined structural, functional, and cognitive imag-
ing to detail the structure of spatial attention effects in visual cortex.

2.2 Mapping Spatial A beautiful illustration of how spatial attention influences sensory
Attention in Vision processing in human visual cortex comes from the work of Tootell
et al. [33]. They used fMRI to identify the borders of the first few
visual areas in humans (retinotopic mapping) and then conducted a
spatial attention study similar to what was described earlier. In the
Tootell study, subjects performed a simple spatial attention task that
required subjects to covertly and selectively attend stimuli located in
one visual field quadrant while ignoring those in the other quad-
rants; different quadrants were attended in different conditions.
Attentional activations were then mapped onto the flattened repre-
sentations of the visual cortex, permitting the attention effects to be
related directly to the multiple visual areas of human visual cortex,
showing that spatial attention led to robust modulations of activity
in striate cortex and multiple extrastriate visual areas (Fig. 3).
By combining different methods for recording electrical activ-
ity, imaging brain structure, defining functional anatomy (i.e., reti-
notopic maps), and combining this with functional imaging in
carefully controlled studies of selective attention, we can learn a
great deal about the effects exerted by attention on sensory pro-
cesses in humans. Specifically, for spatial attention we now
 Neuroimaging Approaches to the Study of Visual Attention 395

Fig. 3 Spatial attention effects in multiple visual cortical areas in humans as demonstrated by fMRI. Activations
with spatial attention to left-field stimuli are shown in the flattened right visual cortices of two subjects (one in
the left column and the other on the right). The white lines (dotted and solid) indicate the borders of the visual
areas as defined by representations of the horizontal and vertical meridians; each area is labeled from V1 (stri-
ate cortex) through V7, a retinotopic area adjacent to V3A. The solid black line is the representation of the hori-
zontal meridian in V3A. Panels (a) and (b) show the retinotopic mappings of the left visual field for each subject,
with colored activations corresponding to the polar angles shown at right (which represent the left visual field).
Panels (c) and (d) show the attention-related modulations (attended vs. unattended) of sensory responses to
a target in the upper left quadrant (the quadrant of the stimuli is shown at right). Panels (e) and (f) show the
same for stimuli in the lower left quadrant. In (c) through (f), the yellow to red colors indicate areas where
activity was greater when the stimulus was attended to than when it was ignored; the bluish colors represent
the opposite, where the activity was greater when the stimulus was ignored than when attended. The attention
effects in (c) through (f) can be compared to the pure sensory responses to the target bars when passively
viewed [(g) and (h)]. Note the retinotopic pattern of the attention effects in (c) through (f): The attention effects
to targets in the lower left quadrant produced activity in several lower field representations, which included the
appropriate half of V3A (inferior V3A labeled with an “i”) in both subjects, and V3 and V2 in one subject. In
contrast, attention to the upper left quadrant produced activity in the upper field representation of V3A (S) and
in the adjacent upper field representation of area V7 (from [33])

understand that it exerts powerful influences over the processing

of visual inputs: Attended stimuli produce greater neural responses
than do unattended stimuli, even when arousal and physical stimu-
lus differences are rigorously controlled, and this happens in mul-
tiple visual cortical areas beginning at short latencies after stimulus
396 George R. Mangun et al.

onset (as short as 70 ms in humans). In part, such effects may help

to bias competition between attended and ignored sensory inputs
at the level of neuronal receptive fields as well as higher order rep-
resentations of the stimuli [34].

3 Neuroimaging of Attentional Control Networks

3.1 Isolating A major goal in studies of selective attention is to understand how

Attentional Control attention is controlled in the brain. Given that activity in sensory-
Mechanisms specific cortex is modulated by attention, and therefore that the
neuronal response properties of sensory neurons have been tempo-
rarily altered, what mechanisms lead to these changes in informa-
tion processing? Most models posit that some brain systems form
executive control systems that, as a function of momentary behav-
ioral goals, are able to alter sensory-neural activity. Some models
have referred to attentional control networks as the “sources” of
attentional signals, and the sensory (and motor) systems that are
affected as the “sites” where attention is implemented [35].
This source versus site dichotomy is useful as it helps to distin-
guish between the role attention plays in modulating sensory pro-
cessing in the sensory systems and the neural mechanisms that
produce this effect (Fig. 4). Presumably, neuronal projections from
executive attentional control systems contact and influence neu-
rons in sensory-specific cortical areas in order to alter their excit-
ability. As a result, the response in sensory areas to a stimulus may
be either enhanced if the stimulus is given high priority (i.e., is
relevant to the behavioral goal) or attenuated if it is irrelevant to
the current goal. More generally, though, attentional control sys-
tems could be involved in modulating thoughts and actions, as
well as sensory processes.
Studies of patients with brain damage, animal recordings and
lesion studies, and functional imaging converge to show that a large
network of cortical and subcortical areas is activated during atten-
tional orienting and selection [36, 37]. How can we measure the
activity of the different brain regions during the execution of atten-
tional selection tasks in order to determine which networks involve
attention control (sources of attention) and which are the sites of
selection? In part, the answer is to implement tasks that, at least
theoretically, dissociate attentional control processes from selection
in time, as do trial-by-trial attentional cuing paradigms. In such par-
adigms, attention is cued at time 1, which is followed by a delay
period of several hundred milliseconds or seconds, and then by the
target stimuli at time 2 [38]. Such designs are different from that
described earlier (Fig. 1), which used blocked attention conditions
and rapid streams of stimuli to study attentional mechanisms,
because theoretically, the cue (e.g., an arrow) triggers the action of
the attentional control network at time 1, whereas selective stimulus
 Neuroimaging Approaches to the Study of Visual Attention 397

Fig. 4 Diagrammatic models of the influence of top-down attention control net-

works (top) on sensory processing (bottom). Sensory inputs are transduced at
left and processed in multiple stages of analysis (A, B, and C). Top-down influ-
ences are shown as vertical arrows from the attentional control network. Here,
the dashed line arrows indicated no influence on sensory processing stages A or
B, and the heavy solid line arrow indicates an influence of attention on process-
ing in sensory processing stage C, with the result being selection, which is rep-
resented by three arrows coming into C, but only one arrow leaving as output
(bottom right). If this were to refer to spatial attention, then the arrows in the
sensory processing stream could correspond to different parallel visual field
location inputs in a retinotopic fashion, and the single output arrow at C reflect-
ing that selective attention to one location was relatively facilitated (selected)
with respect to the other locations

processing would not take place until the target is presented later at
time 2. Hence, by recording brain responses to the cue and target
separately, one can identify the networks for control and selection,
which was first utilized in ERP studies by Russell Harter and his col-
leagues in the late 1980s [39]. Blocked-design fMRI attention stud-
ies make this difficult because the activations produced by attentional
control and selection processes cannot be easily distinguished given
that they are co-occurring in the images. Although functional brain
imaging, which taps hemodynamic changes, is a rather poor method
for tracking the time course of brain activity, when used in an event-
related design, and appropriate analytic strategies are employed to
separate overlapping hemodynamic responses, it is possible to con-
duct an experiment like that just described; we and others have done
so in several studies of selective attention.
In our initial studies [31], we used event-related fMRI to inves-
tigate attentional control mechanisms during visual-spatial
398 George R. Mangun et al.

attention. An arrow, presented at the center of the display, indicated

the side to which attention should be directed for that trial. Eight
seconds later, a bilateral target display (flickering black and white
checkerboards) appeared for 500 ms. The participants’ task was to
press a button if some of the checks on the cued sides only were
gray rather than white. The 8-s gap between the cue and the target
stimuli allowed us to extract the hemodynamic responses linked to
the attention-directing cues separately from those linked to the sub-
sequent targets. We were thus able to identify a top–down atten-
tional control network triggered by the presentation of the cue.
The attentional control network consisted of regions in the
superior frontal cortex, inferior parietal cortex, superior temporal
cortex, and portions of the posterior cingulate cortex and insula
(Fig. 5). These areas were not involved in the sensory analysis of
the cue that was reflected by activity in the visual cortex (identified
in control scans where the cue was passively viewed). Therefore,
this attentional network of frontal, parietal, and temporal brain
areas can be considered the sources of attention control signals in
the brain [40], and has come to be known as the frontoparietal
attention network or the dorsal attention network [41, 42]. The
result of attentional control on the activity of visual cortex before
and during target processing helps us to understand the relation-
ship between control signals and selective sensory effects of spatial
attention, as described next.
Figure 6 shows coronal sections through the visual cortex at
two time points: first in the cue-to-target period, prior to the
appearance of the lateral target arrays, and then second, during
target processing. Two contrasts are shown for each time point:
the case for activity when left cue was greater than right cue (attend
left > right), and the inverse (attend right > left). Following the
attention-directing cue and prior to target onset, one can see sig-
nificant contralateral activation in multiple regions of visual cortex.
These changes are spatially selective, being in the right visual cor-
tex for leftward attention, and in the left cortex for rightward
attention. Importantly, they occur prior to the onset of the targets,
which occurred more than 8 s later, and were not related to the
simple visual features of the cue, which stimulated different regions
of visual cortex. These contralateral activations to the cues repre-
sent a kind of attentional priming of sensory cortex, which is
thought to form the basis for later selective processing of target
inputs [43, 44]. Indeed, as also shown in the bottom half of Fig. 6,
the selective processing of subsequent target stimuli produced sim-
ilar contralateral activations in visual cortex. Another way to
describe these patterns of activity to cues and targets is to say that
regions of visual cortex that code the spatial locations of the
expected target stimuli showed increases in background activity
levels when attention was directed to those locations by the cues,
even before the targets appeared.
 Neuroimaging Approaches to the Study of Visual Attention 399

Fig. 5 Activations during attention control and target selection. Group (N = 6 subjects) average activations
time-locked to the cue (cues > targets contrast) isolating attentional control regions are shown on the left.
Those time locked to the targets (targets > cues contrast), indicating target processing and motor responses,
are on the right. The activated areas are shown in different colors to reflect the statistical contrasts that
revealed the activations: bluish for brain regions that were more activated to cues than targets, and reddish-
yellow to indicate regions that were more active to targets than cues. The top row shows a view of the dorsal
surface of the brain, the middle row shows a lateral view of the left hemisphere, and the bottom row shows its
medial surface; the activity was the same for the right hemisphere, which is not shown. Attentional control
involved the frontal-parietal attention network involving superior and middle frontal gyri (labeled 1–3), and the
regions in and around the IPS (labeled 4–7), the superior temporal cortex (labeled 8), and the posterior cingu-
late cortex (labeled 10). In contrast, during target selection, the main areas of activation were now the supple-
mentary motor area (labeled a), the motor and somatosensory cortex (labeled b–e), posterior superior parietal
lobule (labeled f, g), the ventral-lateral prefrontal cortex (labeled h) and the cuneus (labeled i) and the visual
cortex (labeled j) (adapted from [31])

Similar effects have been observed in neurophysiological stud-

ies in monkeys [22]. Computationally, such effects may well be a
mechanism for selective sensory processing by changing the base-
line gain of sensory neurons such that when later stimulated, they
produced enhanced responses [45]. This illustrates the mechanism
put forward at the beginning of this section that top–down spatial
attentional control may lead to selective changes in sensory pro-
cessing by changing the background firing rates of neurons, thereby
improving their sensitivity and/or selectivity.
400 George R. Mangun et al.

Fig. 6 Priming of visual cortex by spatial attention. The top row shows increased baseline activity in six sub-
jects to the cues in visual cortex. The bottom row shows the same effects to targets (adapted from [31])

3.2 Specializations One key question about top–down attentional control systems is
in Attentional Control the extent to which such mechanisms are generalized for the con-
trol of attention regardless of the modality or specific stimulus
attributes to which selective attention is directed [46]. For exam-
ple, does preparatory attention for spatial selective attention involve
the same or different control networks as would selective attention
for color or motion? We have addressed this in studies using similar
methods to those described in the foregoing.
In one study [32] we undertook a direct test of whether the neu-
ral mechanisms for the top-down control of both spatial and nonspa-
tial attention were the same. In this study, we compared spatial
selective attention to nonspatial color-selective attention. That is,
subjects were either cued to select the target stimulus based on loca-
tion or color. Our design was as follows. We randomly intermingled
trials in which either the location or the color of an upcoming target
was cued. The cues were letters (e.g., “L” = attend left and “B” = attend
blue) located at fixation in one task or the periphery above fixation in
another version of the task. The stimuli to be discriminated were
rectangles (outlines) that when following spatial cues were located in
the left or right hemifields or when following color cues were over-
lapping outline rectangles located at fixation in one task version or
above fixation in the second task version. The participants were
instructed to covertly direct attention to select an upcoming rectan-
gle stimulus based on the cued feature (location or color) with the
 Neuroimaging Approaches to the Study of Visual Attention 401

task of discriminating its orientation. That is, in the spatial cue condi-
tion, if the cue signified left, the subjects were to maintain fixation on
the central fixation point, but focus covert attention to the left and to
discriminate the orientation of the rectangle presented there (vertical
or horizontal), and to press the appropriate response key. When the
cue indicated blue, however, this meant that the subjects should
focus covert attention in preparation for discriminating the blue rect-
angle location in the same place as the cue (i.e., either at fixation or
in the periphery just above fixation).
As in our studies [31] described earlier, event-related fMRI
measures were obtained to the cues and to the targets, and sepa-
rately for spatial location and color attention trials. Because we
jittered the stimulus-onset-asynchrony between cues and targets
from 1000 to 8000 ms, it was possible to deconvolve the overlap-
ping hemodynamic responses [47–49]. This permitted us to utilize
a paradigm that was more similar to cued attention designs in the
cognitive psychology literature where the time between cues and
targets was not too long for subjects to maintain a strong atten-
tional set (i.e., to sustain the covert allocation of attention to the
cued color or location). In our prior work [31] and that of others,
the need to separate the sluggish hemodynamic responses to adja-
cent stimuli typically resulted in long stimulus-onset-asynchronies
in order to avoid overlap of the responses to cues and targets. By
using specially designed stimulus sequences and analytic strategies
it is possible to design cued attention studies that optimize the
design parameters for investigating attentional mechanisms.
Indeed, in cued attention designs, the speed of stimulus presenta-
tion can be even faster than the 1000–8000-ms lag between cues
and target described here [50, 51].
In this study, comparing preparatory attention for locations
and colors we found that large regions of the frontoparietal net-
work were commonly activated by the spatial and nonspatial cues
alike (Fig. 7). Such related patterns of activity reflect those aspects
of the task that the two attentional control conditions shared, such
as low-level sensory processing of the cues, decoding of linguistic
information in the cue letter and matching that to the task instruc-
tion, establishing the appropriate attentional set and holding this
information in working memory during the cue-to-target period,
and finally, preparing to respond.
To test whether any of the areas activated in response to the
cues were selective for spatial or nonspatial orienting, we directly
statistically compared activity in response to location and color
cues. The results of this direct comparison are shown in Fig. 8a for
the activations where locations cues produce more activity than
color cues. We found nonoverlapping regions of superior frontal
and parietal cortex that were activated during orienting to location
versus color. Orienting attention based on stimulus location acti-
vated regions of the dorsal frontal cortex [posterior middle frontal
402 George R. Mangun et al.

Fig. 7 Cue-related activity. Group-averaged data for brain regions significantly

activated to attention-directing cues, overlaid onto a brain rendered in 3D. Areas
activated in response to location cues are shown in blue, color cues in red, and
those areas activated by both cues are shown in green. Maps are displayed
using a height threshold of p < 0.005 (uncorrected) and an extent threshold of ten
contiguous voxels (modified from [32])

gyrus and frontal eye fields (FEF)], posterior parietal cortex [intra-
parietal sulcus (IPS) and precuneus], and supplementary motor
cortex. The inverse statistical contrast (color > location cues)
produced no significant activations in the superior frontal or pari-
etal regions, and only showed activity in the ventral occipital cortex
(OCC) (posterior fusiform, posterior middle temporal cortex, and
left insula) (not shown in the figure).
The pattern of selectivity in the frontoparietal attentional con-
trol network for location cuing suggests that neural specializations
exist for the control of orienting attention to locations or for cod-
ing some aspect of space tapped in spatial attention tasks [52]. But
a question that must be addressed is whether these superior frontal
and parietal regions are sensitive only for spatial orienting, and we
have tested this by investigating specializations in attentional con-
trol for other presumably nonspatial features.
We investigated the idea that specializations in superior frontal
and parietal cortex for preparatory spatial selective attention might
also be involved in other aspects of visual attentional processing. In
one study we compared preparatory attention for stimulus motion, a
dorsal stream process, to the same nonspatial ventral stream feature,
color used in Giesbrecht and colleagues [32]. In this study each trial
began with an auditory word cue that instructed subjects to attend to
a target of a particular stimulus feature (i.e., involving either color or
motion attention). If cued to a color, then they were to prepare for
and detect brief color flashes within a display of randomly moving
dots presented during a subsequent test period. If cued to motion,
then they were to prepare for and detect the brief coherent motion
stimulus in the display of randomly moving dots [53, 54].
 Neuroimaging Approaches to the Study of Visual Attention 403

In the study of preparatory attention for motion and color we

observed activity in the frontal-parietal attention network for
both motion and color cues that was similar to prior work [31,
32, 40, 42, 43, 50, 55–58]. As in our study comparing location
versus color processing (Fig. 8a) we conducted direct statistical
contrasts for motion versus color cues (Fig. 8b). Again, as with
the location versus color analyses, for motion versus color we
found subregions of the attentional control network that were
more active for motion than for color, and vice versa. Bilateral
posterior superior frontal cortex and the left superior parietal cor-
tex were selectively active to motion cues, but no superior corti-
cal areas were selectively activated for the inverse contrast of color
versus motion (color > motion). The activations for the motion
versus color attentional control contrast were very similar to three
regions of the attention control network we previously found to
be selectively activated for preparatory spatial attention. There
were also some differences, however, including failure to see
activity in the right superior parietal cortex for the motion versus
color contrast, suggesting that this right parietal region may also
be involved in directing attention to locations.

Fig. 8 Attentional control activations for location and motion cues versus color
cues. (Top) Results of the direct comparison between location and color cue
conditions overlaid on an axial slice. Greater activity to location cues than color
cues is seen in the superior frontal gyrus (SFG) and the superior parietal lobule
(SPL) bilaterally (modified from [32]). (Bottom) Results of the direct comparison
between motion and color cue conditions overlaid on an axial slice. Greater activ-
ity to motion cues than color cues is seen in the superior frontal gyrus (SFG)
bilaterally, but the superior parietal lobule (SPL) only on the left (modified from
[54])
404 George R. Mangun et al.

It is important to point out that in Fig. 8 we are comparing

results across different studies in different populations of volun-
teers, and indeed using slightly different imaging methods and
analyses (although conceptually the same). In order to know
whether preparatory attention for location and motion activated
identical subregions in the frontal-parietal attention network, one
would have to conduct the within subjects experiment contrasting
attentional cuing to location, motion, and color. Nonetheless, the
similar findings across these studies are highly suggestive and require
some interpretation. How can one conceptualize these fMRI results
of similar specializations in location and motion attention?
One view is that the dorsal visual stream projecting from visual
cortex to parietal cortex represents, in part, visual information
required for generating actions [59–62], rather than merely cod-
ing location per se as originally conceived by Ungerleider and
Mishkin [63]. In line with this view is the evidence that the dorsal
stream attentional control areas overlap with regions implicated in
the control of voluntary eye movements [52, 64–66]. This opens
the possibility that the close correspondence of superior frontal
and parietal activity present for location and motion selective atten-
tion more than for nonspatial (color) attention is related to the role
of both of these forms of attention in preparing actions, specifically
those involved in oculomotor output. In the case of attention to
locations the activity in oculomotor areas may represent prepara-
tion for unexecuted eye movements toward the attended location,
and in the case of attention to directions of motion the activity may
represent analogous preparation for ocular pursuit. Such a view is
supported by the study of [67] who investigated brain activations
in cued covert attention, overt saccade tasks, and pointing tasks.
They observed overlapping activity in the superior frontal and pari-
etal cortex for all three tasks. As noted, future studies will be
required to address the speculations we have provided here and
must include studies designed to compare and contrast different
forms of preparatory attention.

3.3 The Ventral So far we have focused our discussion of attentional control on the
Attention Network dorsal frontal and parietal cortex, the so-called dorsal attention
network. The idea is that top-down signals from this system exert
attentional control over perception based on behavioral goals and
strategies. But when attention must be reoriented from an attended
location to another location or object, a different network of brain
areas has been shown to be activated (Fig. 9). This network has
been called the ventral attention network, and involves the
temporal-parietal junction (TPJ), together with ventral frontal
regions including the insula, the portions of the inferior frontal
gyrus and middle frontal gyrus of the right hemisphere [41, 68,
69]. Evidence from various sources, such as resting-state fMRI
[70] indicates that the dorsal and ventral attention networks are
 Neuroimaging Approaches to the Study of Visual Attention 405

Fig. 9 Diagram of the nodes in the dorsal attention network (blue), ventral atten-
tion network (orange), and visual cortex (light blue) in the left and right hemi-
spheres of the human brain. The dorsal system is clearly bilateral in organization,
but the ventral system is believed to be more right lateralized (noted by the dif-
ference in saturation of the orange-colored nodes). Putative connections within
each hemisphere are shown by the arrows. FEF frontal eye fields, IPS intrapari-
etal sulcus, VFC ventral frontal cortex, TPJ temporoparietal junction, V visual
cortex [76]

largely independent from one another anatomically, interacting via

presumed frontal nodes (middle frontal gyrus) where the two net-
works show common activity in resting-state imaging.
In early models [40] activity of the ventral attention network
had been likened to a circuit breaker that upon detecting a poten-
tially task relevant item, sends a signal to the dorsal attention net-
work to disrupt its current attention focus and reorient attention
to the new event. This model would require that the ventral atten-
tion network or some portion of it respond with short latency to
stimulus events that might be task relevant. However, research
using magnetoencephalography (MEG) has shown that both the
parietal and frontal areas of the dorsal attention network have
shorter latency responses [71] than those in the ventral attention
network, such as in ERP recordings of the P300 that originate in
TPJ [72]. These data, therefore, raise some doubts that the ventral
attention network supports a fast circuit breaking mechanism.
Despite evidence against the idea that the ventral attention
network acts as a circuit breaker permitting the dorsal attention
network to reorient, it is nonetheless the case that the integrity of
the ventral network is important for successful reorienting of atten-
tion to relevant events. Patients with lesions in the right ventral
attention network, and who have contralateral neglect, have dis-
ruption in the interhemispheric coordination of the left and right
IPS as shown by resting-state fMRI [73]. Transcranial magnetic
stimulation (TMS) studies has shown that disruption of the ventral
406 George R. Mangun et al.

attention network in healthy volunteers leads to disordered orient-

ing and target detection [74]. Data such as these support the idea
that the dorsal and ventral attention networks work together in
attentional control, but this interaction may be different depend-
ing on the specific tasks demands [75] (for reviews see [76, 77]).
In order to evaluate the directionality of influences between the
dorsal and ventral attention networks (or nodes in the network) we
[78] used fMRI and dynamic causal modeling (DCM) to investigate
the interactions between TPJ and FEF during spatial attention [79,
80]. In the task, subjects received a spatial cue at fixation that
directed them to focus covert attention on a location in either the
left or right visual hemifield. Following the cue a few seconds later,
bilateral stimuli were flashed to the left and right visual field loca-
tions. The subjects’ task was to make a button press response (yes/
no) indicating whether a predesignated color target appeared at the
cued (attended) location. Three different stimulus conditions were
possible: a neutral condition where neither stimuli were the color
target (response = no); a target condition where the color target
appeared at the cued location (response = yes); and a target-colored
distractor condition where the predesignated color stimulus appeared
at the uncued location (response = no). The design, therefore,
included a condition where a distractor stimulus that was the same
color as the predesignated target color could appear at the uncued
location. We reasoned that if the ventral attention network regions
of the right hemisphere were involved only in reorienting spatial
attention towards task-relevant stimuli, they would only respond
with significant changes in activation when the target-colored dis-
tracter was present. This target-colored distractor condition is the
only condition where a relevant color stimulus appeared in the unat-
tended field; neither of the other two conditions were expected to
reorient spatial attention based on feature relevance because the tar-
get either appeared in the cued location or was absent.
We found that reaction times (RT) were about 50 ms slower in
the target-colored distractor condition, indicating that the relevant
color stimulus presented to the uncued location resulted in distrac-
tion, as would be expected based on the literature on contingent
attentional capture [81, 82]. The question then, is what brain
regions are sensitive to this attentional capture? Significant activa-
tions in regions of the dorsal attention network were obtained to
the target-colored distractor, including FEF, IPS, and the precu-
neus, and these activations were greater than for the target-present
or neutral distractor conditions. This is in line with the idea that the
dorsal attention network was more engaged during stimulus condi-
tions where attentional capture by distractors had to be overcome.
Regions within the dorsal network, including right FEF, were also
significantly activated by the cues, suggesting involvement during
shifts of spatial attention reorienting, in response to the cue and
target-colored objects in the uncued location.
 Neuroimaging Approaches to the Study of Visual Attention 407

In contrast, the right TPJ was significantly activated in all three

stimulus conditions, with activation being largest when the target-
colored distractor was presented. Interestingly, right TPJ was not
selectively activated in the condition where the relevant color stim-
ulus appeared at the uncued location, providing evidence that this
node of the ventral attention network is not only engaged to when
attention is reoriented to task-relevant stimuli at unattended loca-
tions [69]. Importantly, the right TPJ did not respond to merely
any stimulus: all responses to the 3 stimulus conditions were greater
than that evoked by the spatial cues, which were not different from
zero. From these patterns of activity, it appears the right FEF, but
not the right TPJ was most engaged by the need to reorient spatial
attention to potential or actual target stimuli; TPJ was activated by
all stimulus conditions that required a response.
In order to assess the dynamics of interactions between the
dorsal and ventral attention networks, we used DCM with time
series data extracted from the right FEF and TPJ. The hypothesis
that TPJ sends a signal to FEF to initiate reorienting of attention
by the dorsal attention network predicts that our model should
display an early TPJ response to stimulus driven inputs from the
colored distractor, and a positive modulatory parameter on the
connection from TPJ to FEF. However, if instead TPJ is involved
in post-perceptual attentional or decisional processes, we would
expect stimulus driven inputs to FEF, with modulation of the con-
nection from FEF to TPJ. First, none of the models that best fit the
data were consistent with the idea that right TPJ is activated rap-
idly by the potentially relevant (target-colored) distractor and
sends a reorienting signal to the dorsal attention network. Instead,
the model best supported by the DCM exercise was that right FEF
receives stimulus inputs first and then modulates activation in right
TPJ (Fig. 10). This supports the notion that right TPJ is involved
in post-perceptual processes rather than as a stimulus-driven circuit
breaker that interrupts the current focus of attention and reorients
attention by signaling the dorsal attention network [68, 77].

3.4 Attentional Models of attentional control, as laid out in the foregoing, hold
Control Mechanisms that activity in the sensory cortex is under top-down influences
Revealed from control areas in the frontoparietal cortex during voluntary
by Simultaneous attention [35, 41, 83]. Despite recent advancements in our under-
EEG-fMRI standing of visual selective attention, one unresolved issue about
the nature of top-down attentional control is whether attentional
biasing is achieved primarily by enhancing the sensitivity of task-
relevant sensory cortices or alternatively, via the inhibition of task-
irrelevant areas. Further, a related question is whether areas residing
outside of the sensory cortex but known to mediate other task-
independent processes, such as the default mode network [84], are
also “sites” that receive top–down modulation from the attentional
control system as part of goal-directed behavior.
408 George R. Mangun et al.

Fig. 10 Dynamic causal models of interactions between rTPJ and rFEF showing
driving inputs and intrinsic connections. (a) Model for the condition where the stimu-
lus included the target-colored distractor, which shows positive stimulus-driven
influences on rFEF (0.72) but not rTPJ (-0.22), and positive (.60) intrinsic connectivity
from FEF to TPJ: the only significant modulatory parameter (−0.19) was on the con-
nection from rFEF to rTPJ. (b) Model for the condition where there was a target
present (on the cued side). Here again the pattern is for a positive stimulus driven
input to rFEF, and a positive intrinsic connection from rFEF to rTPJ. These models are
consistent with FEF begin active first, followed by activity in TPJ

The concurrent recording of EEG and fMRI open up new

avenues to address the above questions related to attentional con-
trol mechanisms. Recording EEG simultaneously with fMRI
enables the examination of trial-by-trial functional relationship
between well-established attention-related neural markers in EEG/
ERP and BOLD activities in the attentional control system within
an individual, thereby largely alleviating the issue of inter-session
variability associated with multisession recordings.
In a recent study, we [85] utilized simultaneous EEG-fMRI to
examine the mechanisms of visuospatial attention. Participants
performed a standard visuospatial attention task during which they
were cued to attend either the left or right spatial location in their
peripheral vision. Following a random cue-target interval (2–8 s),
the target stimulus was flashed at one of the peripheral spatial loca-
tion for 100 ms. Only when the target stimulus appeared in the
attended hemifield were participants required to discriminate the
spatial frequency of the target stimulus (5.0 vs. 5.5 cycles per
degree of visual angle) and make a two-alternate forced choice
response via a button press. EEG and fMRI data from the cue-
target interval, when participants allocated their attention but
before the target onset, were analyzed to isolate processes specific
to attentional control.
 Neuroimaging Approaches to the Study of Visual Attention 409

EEG studies of visuospatial attention have shown modulations

of the posterior alpha rhythm (8–12 Hz) as a reliable signature of
selective sensory biasing during lateralized attention allocation
[86]. Following an attention-directing cue the alpha rhythm is
more strongly suppressed over the occipital scalp regions contralat-
eral to the direction of attention than over the hemisphere ipsilat-
eral to the attended direction [86–89] (Fig. 11a). This
topographically-specific modulation of alpha is thought to reflect
an increase in baseline cortical excitability in the task-relevant sen-
sory cortex and at the same time might also reflect an inhibition of
task-irrelevant sensory areas in anticipation of upcoming stimulus
events [90, 91]. To investigate the relationship between attention-
related alpha lateralization and top–down attentional control net-
works we used the post-cue alpha modulation as an index of
top-down sensory facilitation and correlated single-trial alpha
power on each hemisphere with concurrently recorded BOLD sig-
nals to study mechanisms of attentional control.

Fig. 11 Attentional modulation of EEG alpha and its coupling with attentional control structures. (a) Scalp topog-
raphy of alpha desynchronization showed that relative to a pre-cue baseline, alpha power was more strongly
suppressed over the hemisphere contralateral to the attended hemifield for both the attend-left (top left panel)
and attend-right (top-middle) conditions. The asymmetry in alpha desynchronization was further demonstrated
by the difference topography contrasting attend-left and attend-right (top-right). (b) BOLD activities in bilateral
intraparietal sulci (IPS) showed inverse coupling with alpha power measured on the hemispheres both contra-
lateral (bottom-left) and ipsilateral (bottom-right) to the attended hemifield (adapted from [85])
410 George R. Mangun et al.

We first established a direct link between attentional modulation

of alpha power (Fig. 11a) and BOLD activity in the dorsal attention
network (Fig. 11b). The bilateral intraparietal sulci (IPS) were
inversely correlated with single-trial alpha power measured in the
hemispheres both contralateral and ipsilateral to the attended hemi-
field. This indicates that on a trial-by-trial basis, when the IPS was
more active, alpha was reduced more over the occipital regions on
both hemispheres, providing direct evidence that the sensory cor-
tex excitability is modulated by the dorsal attention system. Further,
a ROI-based analysis within the IPS revealed that the inverse cou-
pling was stronger for alpha contralateral to the attended hemifield
than for alpha ipsilateral to the attended hemifield. Specifically, the
number of voxels in IPS showing inverse coupling with alpha con-
tralateral to the attended hemifield were significantly higher than
those coupled with alpha ipsilateral to the attended hemifield (lIPS-
contralateral alpha: 222 voxels vs. lIPS-ipsilateral alpha: 184 voxels;
rIPS-contralateral alpha: 262 voxels vs. rIPS-ipsilateral alpha:
55 voxels; p < 0.001, one-sided, paired t-test). In addition to the
differences in cluster size, the level of inverse coupling between
alpha and BOLD in lIPS was stronger (more negative) for alpha on
the hemisphere contralateral than ipsilateral to the attended hemi-
field (coupling strength for contralateral alpha: −1.26; ipsilateral
alpha: −0.96; p < 0.05, one-sided, paired t-test). In rIPS, we also
found a similar but insignificant trend toward stronger inverse cou-
pling for contralateral alpha compared with ipsilateral alpha (cou-
pling strength for contralateral alpha: −1.04; ipsilateral alpha: −0.96;
p > 0.05). Since post-cue reduction of alpha power was observed in
both hemispheres relative to a pre-cue baseline in our study, the
stronger inverse coupling between IPS and alpha contralateral to
the attended hemifield suggests that visuospatial attention is
enhancing neuronal activity within the task-relevant visual cortex,
instead of inhibiting the task-irrelevant visual cortex.
Next, we examined whether other task-irrelevant networks
outside of the visual system, or even outside of the sensory/motor
system, were also modulated by visuospatial attention. For this
purpose, we examined regions showing positive trial-by-trial
coupling with occipital alpha following the cue onset. This positive
BOLD-alpha coupling would indicate a decreased level of BOLD
activity when alpha power also decreased during attention deploy-
ment, and hence would identify regions potentially inhibited by
attentional mechanisms. We found that areas in the sensorimotor
cortices including the pre- and post-central gyri, as well as nodes in
the default mode network including the middle temporal gyrus
(MTG), and the medial prefrontal cortex (MPFC) showed this
positive BOLD-alpha coupling (Fig. 12a, b). This might suggest a
“push–pull” mechanism similar to that observed in studies involv-
ing attention to multiple sensory modalities [92, 93], where
increased visual alpha (decreased visual cortical excitability) was
 Neuroimaging Approaches to the Study of Visual Attention 411

Fig. 12 Regions showing positive coupling between BOLD and alpha power mea-
sured on hemispheres contralateral (a) and ipsilateral (b) to the attended hemi-
field. (c) A sagittal slice showing a region in the dorsal anterior cingulate cortex
(dACC) with BOLD being positively correlated with the alpha hemispheric lateral-
ization. MPFC medial prefrontal cortex, MTG middle temporal gyrus, postCG
post-central gyrus (adapted from [85])

seen when attention is directed either to the somatosensory [94,

95] or to the auditory domain [96–98]. Here we extend this
“push–pull” mechanism to incorporate the default mode network
and suggest that the level of task-independent self-referential pro-
cesses were also inversely coupled with attentional processes on a
trial-by-trial level.
Finally, as described in Sect. 1.1 of this chapter, voluntary
attention is a type of goal-directed behavior under intentional
control. One question related to this definition that is still not
well-understood is what brain structures are responsible of repre-
senting the goal and mediating the voluntary allocation of atten-
tional resources. More specifically, is the goal (or attentional-set)
represented intrinsically within the attentional control system, or
alternatively, are other executive structures also engaged in this
process? Past studies have proposed the degree of alpha lateraliza-
tion across two hemispheres as an index of voluntary attention
allocation during spatial attention [88, 99–101], with greater
alpha lateralization signifying more efficient allocation of atten-
tional resources according to the task rules. We thus correlated
the trial-by-trial alpha lateralization index with BOLD and found
412 George R. Mangun et al.

that activity in the dorsal anterior cingulate cortex (dACC) and

adjacent regions within the MPFC and the left dorsolateral pre-
frontal cortex (DLPFC) are positively correlated with this alpha
lateralization (Fig. 12c). This positive correlation indicates that
when alpha is more strongly lateralized, BOLD activity in these
regions are also higher, implicating these areas in the mainte-
nance of the attentional-set and the modulation of goal-directed
behavior [102–104]. Our results suggest that the voluntary allo-
cation of attentional resources also involves prefrontal executive
regions other than the dorsal attention system.
In summary, the simultaneous EEG-fMRI recording tech-
nique has emerged as a powerful tool that can contribute signifi-
cantly to our understanding of the top-down mechanisms of
attention. By mapping regions that were either positively or
inversely coupled with posterior alpha over the hemispheres con-
tralateral or ipsilateral to the attended hemifield, we revealed that
attentional mechanisms act as a combination of selective enhance-
ment of the task-relevant visual cortex and inhibition of task-
irrelevant sensory and cognitive modalities. The voluntary
allocation of attentional resources is further mediated by prefrontal
executive cortices outside of the dorsal attention system.

4 Conclusions

In this chapter we have reviewed the use of functional imaging in

the study of selective attention. We addressed this topic from four
perspectives: First, we laid out the theoretical and experimental
design issues that are critical for studying attention, especially
selective attention. Second, we illustrated that the attention system
can be conceptualized as consisting of different cognitive-neural
components (i.e., sources and sites of attention), and therefore,
functional imaging methods that permit these different compo-
nents to be isolated and studied are necessary. Third, we described
evidence that specializations in attention systems can be investi-
gated by experimental design approaches that capitalized on
comparisons between different forms of attention such as spatial
versus nonspatial, and top-down voluntary attention and stimulus
driven attention. In describing how specialized networks can be
revealed we presented an application of the analysis of functional
connectivity to probe the nodes within and between attention net-
works, in this case, using DCM as described in detail in a chapter
of this volume by Karl Friston. Fourth, we pointed to the use of
combined imaging and electrophysiological recording methods (in
parallel or simultaneously) to provide temporal information and
single-trial information (from electroencephalography) that is not
available using functional imaging based on hemodynamic or met-
abolic methods, as well as to shown how electrophysiological
information can be used to inform the analysis of imaging data.
 Neuroimaging Approaches to the Study of Visual Attention 413

We took our examples primarily from work in our laboratories

but also presented the work of others where useful. We focused on
visual attention, principally spatial attention, but the methodologi-
cal approaches are germane to all studies of attention, across
modalities, and including translational efforts in disorders of atten-
tion and other psychiatric and neurological disorders.
Methodological advances have helped to reveal the neural
mechanisms of attention. It is clear that attention mechanisms help
to refine the processing of sensory information beginning as early
as sensory-specific cortex, a concept that was hotly debated until
the 1990s. For voluntary attention, a frontal parietal attentional
control system provides biasing signals to sensory cortex that result
in selective sensory processing, and functional imaging has proven
crucial for understanding these relationships in humans. Other
networks, such as the ventral attention network and the default
mode network have also been identified by functional imaging in
humans. This work is leading to an increasingly rich understanding
of the principles, organization, and mechanisms of human
attention.

References
1. James W (1890) Principles of psychology. H 10. Luck SJ, Hillyard SA, Mouloua M, Woldorff
Holt, New York MG, Clark VP, Hawkins HL (1994) Effects
2. Hv H (1867) Handbuch der physiologischen of spatial cuing on luminance detectability:
optik. Voss, Leipzig psychophysical and electrophysiological evi-
3. Posner MI, Cohen Y (1984) Components of dence for early selection. J Exp Psychol Hum
visual orienting. In: Bouma H, Bouwhis D Percept Perform 20:887–904
(eds) Attention and performance. Erlbaum, 11. Palmer J, Ames CT, Lindsey DT (1993)
Hillsdale, NJ, pp 531–556 Measuring the effect of attention on simple
4. Harter MR, Aine CJ (1984) Brain mecha- visual search. J Exp Psychol Hum Percept
nisms of visual selective attention. In: Perform 19:108–130
Parasuraman R, Davies DR (eds) Varieties of 12. Hernández-Peón R, Scherrer R, Jouvet M
attention. Academic, Orlando, pp 293–321 (1956) Modification of electric activity in
5. Malcolm GL, Shomstein S (2015) Object- cochlear nucleus during “attention” in
based attention in real-world scenes. J Exp unanesthetized cats. Science 123:331–332
Psychol Gen 144:257–263 13. Naatanen R (1975) Selective attention and
6. Broadbent DE (1958) Perception and com- evoked potentials in humans–a critical review.
munication. Pergamon, New York Biol Psychol 2:237–307
7. Deutsch JA, Deutsch D (1963) Attention – 14. Hillyard SA, Hink RF, Schwent VL, Picton
some theoretical considerations. Psychol Rev TW (1973) Electrical signs of selective atten-
70:80–90 tion in the human brain. Science
182:177–180
8. Johnston WA, Heinz SP (1979) Depth of
nontarget processing in an attention task. 15. Van Voorhis S, Hillyard SA (1977) Visual
J Exp Psychol Hum Percept Perform evoked potentials and selective attention to
5:168–175 points in space. Percept Psychophys
22:54–62
9. Hawkins HL, Hillyard SA, Luck SJ, Mouloua
M, Downing CJ, Woodward DP (1990) 16. Eason R, Harter M, White C (1969) Effects
Visual attention modulates signal detectabil- of attention and arousal on visually evoked
ity. J Exp Psychol Hum Percept Perform cortical potentials and reaction time in man.
16:802–811 Physiol Behav 4:283–289
414 George R. Mangun et al.

17. Spong P, Haider M, Lindsley DB (1965) 31. Hopfinger JB, Buonocore MH, Mangun GR
Selective attentiveness and cortical evoked (2000) The neural mechanisms of top-down
responses to visual and auditory stimuli. attentional control. Nat Neurosci 3:284–291
Science 148:395–397 32. Giesbrecht B, Woldorff MG, Song AW,
18. Nunez PL, Srinivasan R (2006) Electric Mangun GR (2003) Neural mechanisms of
fields of the brain: the neurophysics of EEG, top-down control during spatial and feature
2nd edn. Oxford University Press, Oxford, attention. Neuroimage 19:496–512
New York 33. Tootell RBH, Hadjikhani N, Hall EK et al
19. Moran J, Desimone R (1985) Selective atten- (1998) The retinotopy of visual spatial atten-
tion gates visual processing in the extrastriate tion. Neuron 21:1409–1422
cortex. Science 229:782–784 34. Desimone R, Duncan J (1995) Neural mech-
20. Chelazzi L, Miller EK, Duncan J, Desimone R anisms of selective visual-attention. Annu Rev
(1993) A neural basis for visual search in infe- Neurosci 18:193–222
rior temporal cortex. Nature 363:345–347 35. Posner MI, Petersen SE (1990) The atten-
21. Chelazzi L, Miller EK, Duncan J, Desimone tion system of the human brain. Annu Rev
R (2001) Responses of neurons in macaque Neurosci 13:25–42
area V4 during memory-guided visual search. 36. Gitelman DR, Nobre AC, Parrish TB et al
Cereb Cortex 11:761–772 (1999) A large-scale distributed network for
22. Luck SJ, Chelazzi L, Hillyard SA, Desimone R covert spatial attention: further anatomi-
(1997) Neural mechanisms of spatial selective cal delineation based on stringent behav-
attention in areas V1, V2, and V4 of macaque ioural and cognitive controls. Brain 122(Pt
visual cortex. J Neurophysiol 77:24–42 6):1093–1106
23. McAdams CJ, Reid RC (2005) Attention 37. Mesulam MM (1981) A cortical network for
modulates the responses of simple cells in directed attention and unilateral neglect. Ann
monkey primary visual cortex. J Neurosci Neurol 10:309–325
25:11023–11033 38. Posner MI, Snyder CR, Davidson BJ (1980)
24. Briggs F, Mangun GR, Usrey WM (2013) Attention and the detection of signals. J Exp
Attention enhances synaptic efficacy and the Psychol 109:160–174
signal-to-noise ratio in neural circuits. Nature 39. Harter MR, Miller SL, Price NJ, Lalonde ME,
499:476–480 Keyes AL (1989) Neural processes involved
25. McAlonan K, Cavanaugh J, Wurtz RH (2008) in directing attention. J Cogn Neurosci
Guarding the gateway to cortex with atten- 1:223–237
tion in visual thalamus. Nature 456:391–394 40. Corbetta M, Kincade JM, Ollinger JM,
26. Corbetta M, Miezin FM, Dobmeyer S, McAvoy MP, Shulman GL (2000) Voluntary
Shulman GL, Petersen SE (1991) Selective orienting is dissociated from target detec-
and divided attention during visual discrimi- tion in human posterior parietal cortex. Nat
nations of shape, color, and speed: functional Neurosci 3:292–297
anatomy by positron emission tomography. 41. Corbetta M, Shulman GL (2002) Control of
J Neurosci 11:2383–2402 goal-directed and stimulus-driven attention in
27. Heinze HJ, Mangun GR, Burchert W et al the brain. Nat Rev Neurosci 3:201–215
(1994) Combined spatial and temporal imag- 42. Corbetta M, Shulman GL (2011) Spatial
ing of brain activity during visual selective neglect and attention networks. Annu Rev
attention in humans. Nature 372:543–546 Neurosci 34(34):569–599
28. Mangun GR, Hopfinger JB, Kussmaul CL, 43. McMains SA, Fehd HM, Emmanouil TA,
Fletcher EM, Heinze HJ (1997) Covariations Kastner S (2007) Mechanisms of feature-
in ERP and PET measures of spatial selective and space-based attention: response modu-
attention in human extrastriate visual cortex. lation and baseline increases. J Neurophysiol
Hum Brain Mapp 5:273–279 98:2110–2121
29. Hillyard SA, Munte TF (1984) Selective 44. Kastner S, Pinsk MA, De Weerd P, Desimone
attention to color and location: an analysis R, Ungerleider LG (1999) Increased activ-
with event-related brain potentials. Percept ity in human visual cortex during directed
Psychophys 36:185–198 attention in the absence of visual stimulation.
30. Mangun GR, Hillyard SA (1991) Modulations Neuron 22:751–761
of sensory-evoked brain potentials indicate 45. Chawla D, Rees G, Friston KJ (1999) The
changes in perceptual processing during physiological basis of attentional modula-
visual-spatial priming. J Exp Psychol Hum tion in extrastriate visual areas. Nat Neurosci
Percept Perform 17:1057–1074 2:671–676
 Neuroimaging Approaches to the Study of Visual Attention 415

46. Greenberg AS, Esterman M, Wilson D, 59. Cohen YE, Andersen RA (2002) A common
Serences JT, Yantis S (2010) Control of spa- reference frame for movement plans in the
tial and feature-based attention in frontopari- posterior parietal cortex. Nat Rev Neurosci
etal cortex. J Neurosci 30:14330–14339 3:553–562
47. Burock MA, Buckner RL, Woldorff MG, 60. Goodale MA, Milner AD (1992) Separate
Rosen BR, Dale AM (1998) Randomized visual pathways for perception and action.
event-related experimental designs allow for Trends Neurosci 15:20–25
extremely rapid presentation rates using func- 61. Goodale MA, Westwood DA (2004) An
tional MRI. Neuroreport 9:3735–3739 evolving view of duplex vision: separate
48. Ollinger JM, Corbetta M, Shulman GL but interacting cortical pathways for per-
(2001) Separating processes within a trial in ception and action. Curr Opin Neurobiol
event-related functional MRI – II analysis. 14:203–211
Neuroimage 13:218–229 62. Goodale MA (2014) How (and why) the
49. Ollinger JM, Shulman GL, Corbetta M visual control of action differs from visual per-
(2001) Separating processes within a trial ception. Proc Biol Sci 281:20140337
in event-related functional MRI - I. The 63. Ungerleider LG, Mishkin M (1982) Two cor-
method. Neuroimage 13:210–217 tical visual systems. In: Ingle DJ, Goodale MA,
50. Woldorff MG, Hazlett CJ, Fichtenholtz HM, Mansfield RJW (eds) Analysis of visual behav-
Weissman DH, Dale AM, Song AW (2004) ior. MIT Press, Cambridge, pp 549–586
Functional parcellation of attentional con- 64. Corbetta M, Tansy AP, Stanley CM, Astafiev
trol regions of the brain. J Cogn Neurosci SV, Snyder AZ, Shulman GL (2005) A func-
16:149–165 tional MRI study of preparatory signals for
51. Walsh BJ, Buonocore MH, Carter CS, spatial location and objects. Neuropsychologia
Mangun GR (2011) Integrating conflict 43:2041–2056
detection and attentional control mecha- 65. Moore T, Fallah M (2004) Microstimulation
nisms. J Cogn Neurosci 23:2211–2221 of the frontal eye field and its effects on covert
52. Corbetta M (1998) Frontoparietal cortical spatial attention. J Neurophysiol 91:152–162
networks for directing attention and the eye 66. Nobre AC, Sebestyen GN, Miniussi C
to visual locations: identical, independent, or (2000) The dynamics of shifting visuospatial
overlapping neural systems? Proc Natl Acad attention revealed by event-related potentials.
Sci U S A 95:831–838 Neuropsychologia 38:964–974
53. Fannon SP, Saron CD, Mangun GR (2007) 67. Astafiev SV, Shulman GL, Stanley CM, Snyder
Baseline shifts do not predict attentional modu- AZ, Van Essen DC, Corbetta M (2003)
lation of target processing during feature-based Functional organization of human intrapari-
visual attention. Front Hum Neurosci 1:7 etal and frontal cortex for attending, looking,
54. Mangun GR, Fannon SP (2007) Networks and pointing. J Neurosci 23:4689–4699
for attentional control and selection in spa- 68. Corbetta M, Patel G, Shulman GL (2008)
tial vision. In: Mast F, Jäncke L (eds) Spatial The reorienting system of the human brain:
processing in navigation, imagery and percep- from environment to theory of mind. Neuron
tion. Springer, New York, pp 411–432 58:306–324
55. Kastner S, Ungerleider LG (2000) 69. Shulman GL, Pope DLW, Astafiev SV,
Mechanisms of visual attention in the human McAvoy MP, Snyder AZ, Corbetta M (2010)
cortex. Annu Rev Neurosci 23:315–341 Right hemisphere dominance during spatial
56. Kincade JM, Abrams RA, Astafiev SV, Shulman selective attention and target detection occurs
GL, Corbetta M (2005) An event-related outside the dorsal frontoparietal network.
functional magnetic resonance imaging study J Neurosci 30:3640–3651
of voluntary and stimulus-driven orienting of 70. Fox MD, Corbetta M, Snyder AZ, Vincent
attention. J Neurosci 25:4593–4604 JL, Raichle ME (2006) Spontaneous neuro-
57. Wilson KD, Woldorff MG, Mangun GR nal activity distinguishes human dorsal and
(2005) Control networks and hemispheric ventral attention systems. Proc Natl Acad Sci
asymmetries in parietal cortex during atten- U S A 103:10046–10051
tional orienting in different spatial reference 71. Sestieri C, Pizzella V, Cianflone F, Romani
frames. Neuroimage 25:668–683 GL, Corbetta M (2008) Sequential activation
58. Geng JJ, Mangun GR (2009) Anterior of human oculomotor centers during planning
intraparietal sulcus is sensitive to bottom-up of visually guided eye movements: a combined
attention driven by stimulus salience. J Cogn fMRI-MEG study. Front Hum Neurosci
Neurosci 21:1584–1601 1:10.3389/neuro.3309/3001.2007
416 George R. Mangun et al.

72. Yamaguchi S, Knight RT (1991) Anterior 86. Worden MS, Foxe JJ, Wang N, Simpson GV
and posterior association cortex contribu- (2000) Anticipatory biasing of visuospatial
tions to the somatosensory P300. J Neurosci attention indexed by retinotopically specific
11:2039–2054 alpha-band electroencephalography increases
73. He BJ, Snyder AZ, Vincent JL, Epstein A, over occipital cortex. J Neurosci 20:RC63
Shulman GL, Corbetta M (2007) Breakdown 87. Sauseng P, Klimesch W, Stadler W et al (2005)
of functional connectivity in frontoparietal A shift of visual spatial attention is selectively
networks underlies behavioral deficits in spa- associated with human EEG alpha activity.
tial neglect. Neuron 53:905–918 Eur J Neurosci 22:2917–2926
74. Chica AB, Bartolomeo P, Valero-Cabré A 88. Thut G, Nietzel A, Brandt SA, Pascual-Leone
(2011) Dorsal and ventral parietal contribu- A (2006) Alpha-band electroencephalo-
tions to spatial orienting in the human brain. graphic activity over occipital cortex indexes
J Neurosci 31:8143–8149 visuospatial attention bias and predicts visual
75. Shulman GL, McAvoy MP, Cowan MC et al target detection. J Neurosci 26:9494–9502
(2003) Quantitative analysis of attention 89. Rajagovindan R, Ding M (2011) From pre-
and detection signals during visual search. stimulus alpha oscillation to visual-evoked
J Neurophysiol 90:3384–3397 response: an inverted-U function and its
76. Vossel S, Geng J, Fink GR (2014) Dorsal and attentional modulation. J Cogn Neurosci
ventral attention systems: distinct neural cir- 23:1379–1394
cuits but collaborative roles. Neuroscientist 90. Romei V, Brodbeck V, Michel C, Amedi
20:150–159 A, Pascual-Leone A, Thut G (2008)
77. Geng JJ, Vossel S (2013) Re-evaluating Spontaneous fluctuations in posterior alpha-
the role of TPJ in attentional control: con- band EEG activity reflect variability in excit-
textual updating? Neurosci Biobehav Rev ability of human visual areas. Cereb Cortex
37:2608–2620 18:2010–2018
78. DiQuattro NE, Sawaki R, Geng JJ (2014) 91. Romei V, Gross J, Thut G (2010) On the role
Effective connectivity during feature- of prestimulus alpha rhythms over occipito-
based attentional capture: evidence against parietal areas in visual input regulation: corre-
the attentional reorienting hypothesis of lation or causation? J Neurosci 30:8692–8697
TPJ. Cereb Cortex 24:3131–3141 92. Klimesch W, Sauseng P, Hanslmayr S (2007)
79. Friston KJ, Harrison L, Penny W (2003) EEG alpha oscillations: the inhibition-timing
Dynamic causal modelling. Neuroimage hypothesis. Brain Res Rev 53:63–88
19:1273–1302 93. Jensen O, Mazaheri A (2010) Shaping functional
80. Friston KJ, Li B, Daunizeau J, Stephan architecture by oscillatory alpha activity: gating
KE (2011) Network discovery with by inhibition. Front Hum Neurosci 4:186
DCM. Neuroimage 56:1202–1221 94. Haegens S, Osipova D, Oostenveld R, Jensen
81. Folk CL, Remington RW, Johnston JC (1992) O (2010) Somatosensory working memory
Involuntary covert orienting is contingent on performance in humans depends on both
attentional control settings. J Exp Psychol engagement and disengagement of regions
Hum Percept Perform 18:1030–1044 in a distributed network. Hum Brain Mapp
82. Serences JT, Shomstein S, Leber AB, Golay 31:26–35
X, Egeth HE, Yantis S (2005) Coordination 95. Anderson KL, Ding M (2011) Attentional
of voluntary and stimulus-driven atten- modulation of the somatosensory mu rhythm.
tional control in human cortex. Psychol Sci Neuroscience 180:165–180
16:114–122 96. Foxe JJ, Simpson GV, Ahlfors SP (1998)
83. Petersen SE, Posner MI (2012) The atten- Parieto-occipital approximately 10 Hz activ-
tion system of the human brain: 20 years after. ity reflects anticipatory state of visual atten-
Annu Rev Neurosci 35:73–89 tion mechanisms. Neuroreport 9:3929–3933
84. Buckner RL, Andrews-Hanna JR, Schacter 97. Fu KM, Foxe JJ, Murray MM, Higgins BA,
DL (2008) The brain’s default network: anat- Javitt DC, Schroeder CE (2001) Attention-
omy, function, and relevance to disease. Ann dependent suppression of distracter visual
N Y Acad Sci 1124:1–38 input can be cross-modally cued as indexed
85. Liu Y, Bengson J, Huang H, Mangun GR, by anticipatory parieto-occipital alpha-band
Ding M (2016) Top-down modulation of oscillations. Brain Res Cogn Brain Res
neural activity in anticipatory visual attention: 12:145–152
control mechanisms revealed by simultaneous 98. Bollimunta A, Chen Y, Schroeder CE, Ding
EEG-fMRI. Cereb Cortex 26:517–529 M (2008) Neuronal mechanisms of cortical
 Neuroimaging Approaches to the Study of Visual Attention 417

alpha oscillations in awake-behaving locations: a high-density EEG study. Eur

macaques. J Neurosci 28:9976–9988 J Neurosci 30:2224–2234
99. Handel BF, Haarmeier T, Jensen O (2011) 102. Dosenbach NU, Visscher KM, Palmer ED
Alpha oscillations correlate with the successful et al (2006) A core system for the implemen-
inhibition of unattended stimuli. J Cogn tation of task sets. Neuron 50:799–812
Neurosci 23:2494–2502 103. Dosenbach NU, Fair DA, Cohen AL,
100. Haegens S, Handel BF, Jensen O (2011) Schlaggar BL, Petersen SE (2008) A dual-
Top-down controlled alpha band activity in networks architecture of top-down control.
somatosensory areas determines behavioral Trends Cogn Sci 12:99–105
performance in a discrimination task. 104. Sakai K (2008) Task set and prefrontal cortex.
J Neurosci 31:5197–5204 Annu Rev Neurosci 31:219–245
101. Kelly SP, Gomez-Ramirez M, Foxe JJ (2009) 105. Mangun GR, Buonocore MH, Girelli M, Jha AP
The strength of anticipatory spatial biasing (1998) ERP and fMRI measures of visual spatial
predicts target discrimination at attended selective attention. Hum Brain Mapp 6:383–389
 Chapter 14

fMRI of Memory
Federica Agosta, Indre V. Viskontas, Maria Luisa Gorno-Tempini,
and Massimo Filippi

Abstract
Numerous fMRI studies have investigated the network of brain regions critical for memory. Whereas
neuropsychological techniques can delineate the brain regions that are necessary for intact memory func-
tion, neuroimaging techniques can be used to investigate which regions are recruited during healthy
memory formation, storage, and retrieval. For example, fMRI studies have shown that lateral prefrontal
cortex (PFC) supports some components of working memory function. However, working memory is not
localized to a single brain region but is likely a property of the functional interaction between the PFC and
posterior brain regions. The medial temporal lobe (MTL) and its connections with neocortical, prefrontal,
and limbic structures are implicated in episodic memory. Semantic memory is mediated by a network of
neocortical structures, including lateral and anterior temporal lobes, and inferior frontal cortex, possibly to
a greater extent in the left hemisphere. Memory for semantic information benefits from the MTL for only
a limited time, and can be acquired, albeit slowly and with difficulty, without it. To date, most of the
emphasis has been on exploring the unique aspects of these different types of memory. Some evidence,
however, of functional overlap in general retrieval processes does exist.

Key words Working memory, Encoding, Retrieval, Episodic memory, Semantic memory, Prefrontal
cortex, Medial temporal lobe

1 Introduction

Memory shapes our behavior by allowing us to store, retain, and

retrieve past experiences, and thus enables us to imagine the con-
sequences of our actions. These processes influence and are modi-
fied by the type of information that is to be remembered, the
duration of time over which it must be retained, and the way in
which the brain will use the information in the future. The neural
circuits underlying these processes are dynamic, reflecting the flex-
ibility of memory itself. fMRI enables detailed study of the neural
networks that support memory function. To delineate the neural
circuitry underlying memory, it is helpful to breakdown memory
into simpler components. In this chapter, we focus on memory
that is consciously accessible, and use the duration of retention to

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_14, © Springer Science+Business Media New York 2016

419
420 Federica Agosta et al.

dictate its parcellation: beginning with working memory, which

holds information in mind for seconds to minutes, and moving on
to long-term memory, which can be further divided into episodic
and semantic memory.

2 Working Memory

Working memory (WM) refers to the temporary storage and

manipulation of information that was most recently experienced or
retrieved from long-term memory but is no longer available in the
external environment [1–3]. Most models of WM separate two
components of WM: temporary stores of information, in the form
of “buffers” or “slave systems,” that are usually modality-specific,
and a central “executive” or set of processes that manipulate the
information [2, 4]. Items in WM are stored only as long as the
information is either being rehearsed (subvocally) or manipulated
in some other fashion (i.e., rotated or integrated with existing
information in semantic memory). The capacity of WM is limited
by attention to about seven (plus or minus two) meaningful “bits”
(or chunks) of information—these bits can be manipulated and
either discarded or associated and transferred into long-term mem-
ory [5]. WM is central to everyday functioning and contributes
significantly to other areas of cognition. Baddeley and Hitch [1]
proposed a model of memory that has influenced virtually all sub-
sequent research in the area. Their model is composed of a three-
component system: (1) the “phonological loop,” comprising a
limited capacity phonological store in which verbal information is
stored temporarily and maintained by subvocal rehearsal (e.g.,
repeated subvocal articulation when trying to keep a phone num-
ber in mind), (2) the “visuospatial sketch pad,” a storage buffer for
nonverbal material, such as the visual representations of objects,
and (3) the “central executive,” which is responsible for strategic
manipulation and execution of the aforementioned “slave” sys-
tems. The original model has subsequently been updated [2] to
include an “episodic buffer” that provides an interface between the
subsystems of WM and long-term memory.
The first evidence for a role of prefrontal cortex (PFC) in WM
came from lesion and electrophysiological studies in nonhuman pri-
mates [6, 7]. The first neurons discovered – showing persistent
activity during the delay period of WM—were found in the monkey
PFC using single-unit neuron recording techniques [8–10]. This
sustained activity is thought to provide a bridge between the stimu-
lus cue, for instance, the location of a flash of light, and its contin-
gent response, such as a saccade to the remembered location.
Persistent activity during blank memory intervals is a very powerful
observation and established a strong link implicating the PFC as a
critical node supporting WM [3]. Since then, physiological studies
 fMRI of Memory 421

in nonhuman primates have revealed active delay neurons in a large

number of brain regions, including the dorsolateral (DL) and ven-
trolateral (VL) PFC, the intraparietal sulcus (IPS), posterior percep-
tual areas, and subcortical structures, such as the caudate and the
thalamus (for a review, see Ref. [11]). In humans, however, the
advent of modern functional brain imaging techniques, such as pos-
itron emission tomography (PET) and fMRI, enabled a more
detailed study of the functional neuroanatomy of WM processes.
Many of the neuroimaging experiments designed to elucidate the
neural underpinnings of WM separate executive control processes
from storage. For example, a number of studies have been designed
such that WM maintenance is the task of interest (e.g., delayed
response, delayed recognition, delayed alternation, and delayed
match-to-sample tasks). In a typical delayed recognition trial, the
subject is first required to remember a stimulus presented during a
cue period and then to maintain this information for a brief delay
interval when the stimulus is absent. Then, the subject responds to
a probe stimulus to determine whether the information was success-
fully retained. Thus, the maintenance and recognition decision pro-
cesses are temporally segregated and can be investigated in relative
isolation. In this way, using fMRI, investigators can record neural
activity during these distinct stages of WM. Brain regions exhibiting
persistent activity above resting baseline during the delay period are
often interpreted as being involved in WM maintenance processes
[12–20]. Functional neuroimaging studies have revealed that the
same collection of regions that were shown to be involved in WM
in nonhuman primates displays significantly increased activity dur-
ing delay tasks in humans [21–29]. For instance, in one fMRI study,
subjects were scanned while they performed an oculomotor delayed
matching-to-sample task that required maintenance of the spatial
position of single dot of light over a delay period after which a
memory-guided saccade was generated [27]. Both frontal eye fields
and IPS showed activity that spanned the entire delay period.
Moreover, the magnitude of the activity correlated positively with
the accuracy of the memory-guided saccade that followed later
[27]. Despite the relative simplicity of the delay period, however,
multiple cognitive processes remain engaged concurrently, includ-
ing information maintenance, suppression of distraction, motor
response preparation, mental timing, expectancy, monitoring of
internal and external states, and preservation of alertness. As a
result, even the maintenance period of WM is likely to be mediated
by a distributed network of distinct brain regions, rather than to be
localized to a single brain region [20].

2.1 Organization The extensive reciprocal connections from the PFC to virtually all
of the WM Network cortical and subcortical structures place the PFC in an unique neu-
roanatomical position to monitor and manipulate diverse cognitive
processes [20]. Marklund et al. [28] employed a mixed block and
422 Federica Agosta et al.

event-related design in an fMRI study of episodic, semantic, and

working memories contrasted with sustained attention. This
approach identified transient activity, particularly in the left DL
PFC, that appears to reflect the operation of WM during retrieval
from long-term memory [28]. Furthermore, it has been found
that the PFC shows activity during retention interval of delay task
regardless of the type of information (e.g., spatial, faces, objects,
words) [22, 24, 29]. In fact, there is a critical mass of functional
neuroimaging studies emphasizing the stable persistent neural
activity in selective lateral PFC neurons during the delay period in
humans (“the fixed-selectivity model”) (for review, see Ref. [3]).
A controversial issue in the literature is the extent to which
WM can be segregated anatomically according to the type of to-
be-retained material (e.g., verbal, space, object, visual, and audi-
tory) or the component processes (e.g., maintenance vs.
manipulation of information) (for a review, see Refs. [3, 30–34]).

2.1.1 Organization First, let us evaluate the evidence supporting anatomical segregation
of WM by Material Type on the basis of stimulus category. Three types of material have been
most commonly studied: verbal, spatial, and object information. A
prevalent theory of material-type segregation in the frontal cortex
suggests that there are dorsal and ventral memory streams for spatial
and object information, respectively, similarly to the “where” and
“what” pathways of the visual system [35]. The dorsal stream proj-
ects from the extrastriate cortex to the inferior parietal lobule (IPL)
and the IPS and is involved in processing spatial information [35].
The ventral stream extends from the extrastriate cortex to the inferior
surface of the frontal pole and processes object information [35].
Within the frontal cortex, WM for spatial information involves the
superior DL PFC or the superior frontal sulcus, whereas object WM
relies upon several mid- and inferior frontal regions (VL PFC) (for
review, see Ref. [3]). Furthermore, there is a tendency for verbal and
object WM to recruit more left-hemisphere areas, and for spatial tasks
to recruit more right-hemisphere areas (for review, see Ref. [3]).
The processing of nonverbal spatial information is right lateral-
ized and associated with the activation of a fronto-parietal network
(e.g., see Ref. [36–41]). Using event-related fMRI, Courtney et al.
[37] demonstrated a neuroanatomical dissociation between delay
period activity during WM maintenance for either the identity
(object memory) or location (spatial memory) of a set of three face
stimuli. Greater activity during the delay period on face identity trials
was observed in the left inferior frontal gyrus (IFG), whereas greater
activity during the delay period of the location task was observed in
dorsal frontal cortex (bilateral superior frontal sulcus) [37] (Fig. 1).
WM for objects, mostly visually presented faces, houses, and line
drawings that are not easily verbalizable, seems to be right-lateralized
and activates the temporal-occipital regions [Brodmann area (BA)
37] (e.g., see Refs. [21, 26, 29, 36, 39, 42–44]).
 fMRI of Memory 423

a b Superior frontal
sulcus
L Superior
R frontal
Control stimulus set Memory set sulcus

Control Memory test

response and response
Precentral
sulcus
+52 m
2s2s 2s2s m
2s 2s Middle
Precentral
frontal
sulcus
gyrus
Control Memory
delay ITI delay ITI
1
9s 6s 9s 6s
2

3 Precentral
sulcus +32 m
4 m
Inferior
frontal
gyrus
5

6
1 unit
0 6 12 18 24 30 36 42 s

+7 mm

Spatial working memory

Face working memory

Fig. 1 An event-related fMRI study of delay period activity during working memory (WM) maintenance for
either the identity (object memory) or location (spatial memory) of a set of three face stimuli. (a) Schematic
depiction of the fMRI tasks: subjects saw a series of three faces, each presented for 2 s in a different location
on the screen, followed by a 9-s memory delay. Then a single test face appeared in some location on the
screen for 3 s, followed by a 6-s intertrial interval. Before each series, subjects were instructed to remember
the locations or the identities of the three faces in the memory set. For the spatial task, the subject indicated
with a left or right button whether the test location was the same as one of the three locations presented in the
memory set, regardless of the face that marked that location. For the face memory task, the subject indicated
whether the test face was the same as one of the three faces observed in the memory set, regardless of the
location where the face appeared. For the sensorimotor control task, scrambled faces appeared (control stimu-
lus set), and when the fourth scrambled picture appeared after the delay, subjects pressed both buttons
(control response). Contrasts between task components are shown below the task diagram: (1) visual stimula-
tion vs. no visual stimulation, (2) memory stimuli vs. control stimuli, (3) control stimulus set vs. control response,
(4) memory stimulus set vs. test stimulus and response, (5) delays during anticipation of response vs. intertrial
intervals, and (6) memory delays vs. control delays. (b) Areas with significant sustained recruitment in a single
subject during the WM delay for faces (blue outline) and for spatial locations (red outline) overlaid onto the
subject’s Talairach normalized anatomical MR image. Level above the bicommissural plane is indicated for
each axial section (from Courtney et al. [37])

In contrast to object, processing of verbal WM activates regions

in the left hemisphere. Broca’s area (BAs 44/45), premotor areas
(supplementary motor area and premotor cortex) [45–48], and the
cerebellum [24, 47, 49, 50] are critical for the articulatory subvocal
rehearsal. Phonological maintenance has been associated with
424 Federica Agosta et al.

activity in parietal areas, particularly the IPL (i.e., BAs 39 and 40)
[24, 45, 46, 49–53] but also in the superior parietal lobe [47, 49,
52, 53]. For instance, using an event-related design, Chein and Fiez
[45] designed a delayed serial recall task requiring subjects to encode,
maintain, and overtly recall sets of verbal items for which phonologi-
cal similarity, articulatory length, and lexical status were manipulated
and reported that Broca’s area and BA 40 showed patterns of sus-
tained activity during the delay period of a verbal WM task.
More recently, Raye et al. [48] provided evidence that left VL
PFC appears associated with subvocal rehearsal of single words,
whereas left DL PFC activation is associated with participants
“refreshing” (simply thinking about) the visual appearance of a
recently presented word [48].
Despite this growing evidence for segregation by material type,
there are also several human functional imaging studies that have
failed to find evidence for segregation in PFC WM activity (e.g., see
Refs. [16, 54, 55]). For instance, using an event-related design,
Postle and D’Esposito [16] evaluated the organization of WM for
the identity and location of visually presented stimuli (target stimuli
for all object trials were 16 abstract polygon stimuli, determined in
normative testing to be difficult to associate with real-world objects).
Although the task produced considerable delay-period activity in VL
PFC, DL PFC, and superior frontal cortex, in no subject, PFC activ-
ity was greater for one stimulus domain than for the other [16].
Moreover, in a large meta-analysis based on 60 neuroimaging (both
PET and fMRI) studies of WM [55], analyses of material type showed
the expected dorsal–ventral dissociation between spatial and nonspa-
tial storage in the posterior cortex, but not in the frontal lobe. Some
support was found for left frontal dominance in verbal WM, but only
for tasks with low executive demand. Executive demand increased
right lateralization in the frontal cortex for spatial WM [55].

2.1.2 Organization The other axis along which investigators have suggested that human
of WM by Process Type lateral PFC involvement in WM is segregated is according to the type
of operation performed upon the contents of WM, rather than the
type of information being maintained. In particular, several fMRI
studies have focused on the distinction between two fundamental
WM processes, namely the passive maintenance of information in
short-term memory and the active manipulation of this information,
within the PFC (for review, see Ref. [3]). This model received initial
support from a PET study by Owen et al. [56] in which dorsal PFC
activation was found during three spatial WM tasks thought to
require greater monitoring of remembered information (i.e., a mne-
monic variant of modified Tower of London planning task requiring
short-term retention and reproduction of problem solutions) than
two other WM tasks (i.e., the modified Tower of London planning
task, and a control condition that involved identical visual stimuli and
motor responses) that activated only ventral PFC.
 fMRI of Memory 425

This model was also tested using fMRI (for review, see Ref. [3]).
The VL PFC has been suggested to be primarily involved only in
WM mechanisms that support simple retrieval of information for
sensory-guided sequential behavior (maintenance) [57–60], whereas
DL PFC (particularly BA 46) has been found to serve mechanisms
of active monitoring and manipulation of information (or general-
ized executive processing) in WM [61–63]. For instance, in an
event-related fMRI study [61], subjects were presented with two
types of trials in random order in which they were required to either
maintain a sequence of letters across a delay period or manipulate
(alphabetize) this sequence during the delay in order to respond cor-
rectly to a probe. The authors found that dorsal PFC activity was
greater in trials during which actively maintained information was
manipulated, providing further support for a process-specific PFC
organization. However, this functional PFC division has not been
consistently replicated and seems to vary with materials and difficulty
[55]. Moreover, this distinction is not compatible with evidence of
continuous DL PFC activity in tasks without any manipulation [32,
64]. A large meta-analysis [55] showed that tasks requiring execu-
tive processing generally produce more dorsal frontal activations
than do storage-only tasks, but not all executive processes show this
pattern. For instance, superior frontal cortex (BAs 6, 8, and 9)
responded most when WM must be continuously updated and when
memory for temporal order has to be maintained [55]. Right ventral
frontal cortex (BAs 10 and 47) responded more frequently with
demand for manipulation (including dual-task requirements or
mental operations) [55]. Posterior parietal cortex (BA 7) was found
to be involved in all types of executive functions [55].

2.2 Medial Temporal fMRI studies of WM have also found that the fronto-parietal net-
Lobe Involvement work is not the only region that is active during the temporary
in WM retention of task-relevant information. PFC and parietal cortex do
not seem to be sufficient to perform WM for novel stimuli when
parahippocampal regions are lesioned [65], although they are suf-
ficient to maintain normal WM for familiar stimuli [66].
Surprisingly, early fMRI studies of WM did not report activity
within parahippocampal regions such as perirhinal (PrC) or ento-
rhinal cortex [67]. An fMRI study by Stern et al. [68] demon-
strated differential activation for novel vs. familiar stimuli during
performance of a 2-back WM task. This study showed that WM for
a highly familiar set of complex visual images primarily activated
prefrontal and parietal cortices, whereas the same task using novel
(trial-unique) visual images strongly activated parahippocampal
structures in addition to prefrontal and parietal cortices. Activation
of parahippocampal structures associated with WM for novel stim-
uli has also been shown in an event-related fMRI study using novel
face stimuli [25, 69].
426 Federica Agosta et al.

2.3 Connectivity By varying experimental design (e.g., parametric memory load

Analysis variation [23, 43]), attempts have been made to associate identi-
Within the WM fied brain regions with different processes occurring during the
Network delay. The first fMRI study aimed at characterizing regional inter-
actions in WM has used a block-design fMRI paradigm with a
graded n-back verbal WM task [70]. Changes of effective connec-
tivity within a fronto-parietal WM network related to different lev-
els of WM load were studied. The results revealed enhanced inferior
fronto-parietal connectivity and also increased interhemispheric
communication between DL PFC regions as correlates of increas-
ing WM load [70]. A following event-related fMRI study charac-
terized the neural network mediating the online maintenance of
faces [71] (Fig. 2). The fusiform face area (FFA) was defined as a
seed and was then used to generate whole-brain correlation maps.
A random effects analysis revealed a network of brain regions
exhibiting significant correlations with the FFA seed during the
WM delay period. This maintenance network included the DL and
VL PFC, the premotor cortex, the IPS, the caudate nucleus, the
thalamus, the hippocampus, and occipitotemporal regions [71].
These findings support the notion that the coordinated func-
tional interaction between nodes of a widely distributed network
underlies the active maintenance of a perceptual representation and

a b
Delay Network: Right FFA seed
–12 –6 0 6 12 18
18 Right FFA Correlation Data
Cue
16 Delay
Probe
14
% ROI suprathreshold

12 24 30 36 42 48 54

10

6 L R
4

0
Right MFG Left MFG Right IFG Left IFG R L
Region of Interest left hemisphere rostral right hemisphere
t-value
7.23 18.0

Fig. 2 Functional connectivity analysis between brain regions associated with the maintenance of a represen-
tation of a visual stimulus (face recognition task) over a short delay interval. The fusiform face area (FFA) was
used as the exploratory seed. (a) Quantification of right FFA correlation effects in the prefrontal cortex (PFC).
Bars indicate the number of significant voxels present during each stage (cue, delay, and probe) in four pre-
frontal regions of interest. MFG middle frontal gyrus, IFG inferior frontal gyrus, ROI region of interest. (b) Delay
period right FFA seed correlation map including bilateral regions in the dorsolateral and ventrolateral PFC,
premotor cortex, IPS, caudate nucleus, thalamus, hippocampus, and occipitotemporal regions. Activations are
thresholded at p < 0.05 (corrected) and are shown overlaid on both axial slices and a three-dimensionally
rendered Montreal Neurological Institute (MNI) template brain. The color scale indicates the magnitude of t
values (from Gazzaley et al. [71])
 fMRI of Memory 427

provides convergent evidence that fronto-parietal and

interhemispheric frontal connectivities are central in WM [70, 71].
More recently, it has been demonstrated that visual WM involves
top-down signals from the lateral PFC to parietal cortex [72, 73] as
well as communication from fronto-parietal regions to visual cortex
[74]. Additionally, interactions between basal ganglia and the lat-
eral PFC are thought to mediate the filtering of task-irrelevant
information and the updating of task-relevant information in WM
[75]. Future studies are warranted to understand further the func-
tions resulting from interactions between these regions to build a
complete picture of WM.

3 Long-Term Memory for Consciously Accessible Material: Episodic

and Semantic

Within the declarative or consciously accessible long-term memory

system, “episodic” and “semantic” memory can be distinguished.
Episodic memory (EM) allows the recollection of unique personal
experiences: rich, vivid reexperiencing of past events. Semantic
memory (SM), in contrast, refers to generic information that is
acquired across many different instances and accessed indepen-
dently of the details of the context in which the information was
first encountered [76]. This fractionation of declarative memory is
supported by evidence that episodic and semantic memory have
distinctive anatomical substrates [77–80]. Therefore, we will con-
sider each memory type individually. Of note, autobiographical
memory can be either semantic, as in one’s knowledge of the
names of all the schools that one attended, or episodic, as in one’s
memory for a particular birthday: what binds autobiographical
memories to each other is self-awareness.

4 Episodic Memory

EM enables us to access and reexperience the sights, sounds, smells,

and other details of a specific event [76]. Most EMs are available
for several minutes or hours but over time access to their details
degrades [81]. Others remain accessible with their details for a life-
time [82]. This temporal difference in storage highlights the com-
plexity of EM: episodic remembering is composed of several
component processes, including the retrieval of information from
across sensory domains and the reconstruction of an event from a
set of individual details.

4.1 Distinguishing One of the major advances in the study of EM has been the appli-
Encoding and Retrieval cation of neuroimaging techniques to distinguish the component
Processes processes of encoding and retrieval. In neuropsychological studies,
it is often difficult or even impossible to separate failures of EM
428 Federica Agosta et al.

due to encoding or retrieval processes. Functional imaging affords

researchers the opportunity to separate the neural underpinnings
of these processes and is particularly helpful in understanding the
distinct roles of medial temporal lobe (MTL) subregions, which
are often damaged equivocally in patients with lesions [83].
Furthermore, neuroimaging evidence has shown that prefrontal
and other cortical areas are engaged during episodic remembering,
a fact that had not been clear from patient data alone [84].
Neuroimaging studies have indicated that individual elements
of EMs may be permanently stored within the same neocortical
regions that are involved in initial processing and analyzing of the
information [85, 86]. Several studies have demonstrated the reac-
tivation of the visual cortex during retrieval of visual details [85,
87], activation in the auditory cortex during auditory memory
retrieval [88], and activation in the motor cortex during the
retrieval of memory for actions [89]. Insofar as episodic remem-
bering involves reexperiencing the details of an event, it is not sur-
prising that brain regions involved in the initial perception of these
details are reactivated during their retrieval.
According to several influential memory models, each differ-
ent cortical region makes a unique contribution to the storage of
a given memory and all regions participate together in the cre-
ation of a complete memory representation [90–92]. The MTL,
then, is saddled with the task of binding together these different
regional contributions into a coherent memory trace [90]. In the
MTL, the hippocampal formation receives processed sensory
information from association areas in the frontal, parietal, and
occipital lobes via the parahippocampal cortex [93]. Given its
anatomical placement and architecture, the hippocampus has the
unique ability to bind “what happened,” “when it happened,”
and “where it happened” together [90]. The architecture of the
hippocampus includes a circular pathway of neurons from the
entorhinal cortex to the dentate gyrus, CA3 and CA1 neurons of
the hippocampus to the subiculum, and back to the entorhinal
cortex [94]. The connections within the hippocampal formation
and between the MTL and neocortical regions are formed more
rapidly than are the connections between disparate cortical
regions [92]. Therefore, when a particular cue in the environ-
ment or the mental state of the person activates cells in the corti-
cal regions, the MTL network that is associated with that cue is
reactivated and the entire neocortical representation is strength-
ened. As multiple reactivations occur, the connections between
the relevant neocortical regions are slowly strengthened until the
memory trace no longer depends on the activity of the MTL, but
may be entirely represented in the neocortex [90]. Using both
fMRI and neuropsychological techniques, consistent evidence
suggests that the hippocampus remains involved in the retrieval
of EMs regardless of the age of the memory [95]: several authors,
 fMRI of Memory 429

then, have proposed that the hippocampus acts as a pointer or

index, recreating the activation in the neocortex that represents
the individual elements of the memory [96].

4.1.1 Encoding Simply perceiving and attending to information in the world is not
sufficient to create a lasting long-term memory trace. Some other
process or processes are engaged in order to bind elements of an
episode into a coherent memory trace. There is some disagreement
concerning whether all memories are initially episodic and later
become semantic, or whether some memories are semantic in
nature from the outset [97, 98]. One might argue that every learn-
ing episode is encoded as such, and as items from an episode get
integrated into the network of semantic information, those items
become disassociated from the episode itself and associated instead
with the information already in semantic memory [97]. In fact,
several memory models suggest that episodic and semantic infor-
mation is learned via different mechanisms: elements of an episode
are rapidly bound together by autoassociative processes in the den-
tate gyrus and CA3 field [99], while semantic information is grad-
ually acquired over many repetitions by reorganization of Hebbian
synapses in the neocortex [91, 100, 101].
Pioneering neuroimaging studies in the 1990s implicated the
PFC, particularly the left lateral PFC, in semantic or associative
encoding, by showing that this region, in addition to the MTL,
shows greater neural activity during semantic encoding than during
more superficial or perceptual encoding [102, 103]. Furthermore,
novel stimuli have been shown to elicit greater neural activity in the
MTL than familiar stimuli [104, 105], providing more evidence
for the involvement of the MTL in encoding processes.
A direct link to episodic encoding processes, however, required
the advent of event-related fMRI designs to investigate these cog-
nitive processes [106]. Encoding trials were binned according to
whether information presented on a given trial is subsequently
remembered or forgotten. Subsequent memory studies have shown
that activation in the left and right PFC, the parahippocampal
gyrus [107, 108], and hippocampus [109] during encoding pre-
dicts successful memory retrieval (Fig. 3). Since these original
studies, many more studies have replicated the finding that MTL
activity correlates with episodic encoding [110–113]. Furthermore,
fMRI studies have shown that hippocampal activity correlates with
the subsequent retrieval of contextual details presented during
encoding, while activity in the PrC correlates with successful
retrieval of an individual item, but not with retrieval of episodic
details [111–115]. Staresina and Davachi [116] have shown that
since the PrC receives inputs mainly from cortical areas devoted to
the processing of visual information, recruitment of this region
also correlates with the retrieval of visual features of items, such as
the color in which they were presented, but not with other
430 Federica Agosta et al.

a Right frontal b 1505 Left parahippocampal

1770
1500

Signal intensity

Signal intensity
1765
1495
1760
1490
1755 1485
1750 1480
1745 1475
Slice 1 Slice 2 0 5 10 15 20
0 5 10 15 20
Scan within trial Scan within trial
c 1440 Right parahippocampal
d 120 Remembered

Event-related response
Familiar
100 Forgotten

Signal intensity
1435
1430 80
60
1425
40
L Slice 3 Slice 4 R 1420
20
P< 0.01 P< 0.0005 1415 0
0 5 10 15 20 1 2 3 4 5 6
Scan within trial Subject

Fig. 3 Composite statistical activation maps displaying voxels with significant positive correlations between
event-related activations to pictures and subsequent memory for those pictures. Areas activated are right
dorsolateral prefrontal cortex (upper right in slice 1) and bilateral parahippocampal cortex (lower left in slices
1 and 3; left and right in slices 2 and 4). Examples of average signal magnitude during study from six subjects
in (a) right frontal (slice 1), (b) left parahippocampal (slice 4), and (c) right parahippocampal (slice 2) regions
for remembered (red), familiar (green), and forgotten (blue) pictures. Gray block depicts onset and offset of
picture presentation. (d) Mean voxel response in parahippocampal areas showing significant correlation with
subsequent memory in each subject for remembered, familiar, and forgotten pictures (from Brewer et al. [107])

contextual details. These data suggest that while the hippocampus

binds elements from all domains, the more specialized regions of
the MTL and surrounding cortices may play domain-specific roles
in EM encoding. Furthermore, the authors report that the activa-
tion in the hippocampus increased stepwise with increasing num-
bers of successfully encoded associations.
A number of studies, however, have also failed to find reliable
MTL activation related to successful episodic encoding [117–120].
While neuropsychological data demonstrate that the MTL is nec-
essary for new episodic encoding, processes following encoding
such as retrieval, consolidation, interference, and so on may have a
larger impact on subsequent remembering than encoding-related
MTL activity.

4.1.2 Retrieval Work on EM retrieval points to a set of highly flexible retrieval

operations that are differentially engaged depending on the par-
ticular demands of the situation. In general, the network that sup-
ports these retrieval operations includes PFC, MTL structures, and
posterior sensory cortices. Identifying the contributions of these
brain regions to episodic retrieval processes has demanded new
methodologies capable of distinguishing subtle dissociations across
similar retrieval tasks. This section explores three emerging themes:
the role of PFC and posterior sensory cortices, the role of MTL
subregions and the role of parietal cortex in retrieval success.
 fMRI of Memory 431

PFC and Posterior Sensory Early neuroimaging work contrasting encoding and retrieval
Cortices processes revealed a consistent asymmetry in the activations of pre-
frontal regions [121, 122]: specifically, greater activation of the left
PFC during encoding, and relatively greater activation of the right
PFC during episodic retrieval [121, 122]. This pattern, termed
“hemispheric encoding-retrieval asymmetry” (HERA), provided
early insights into the neural basis of EM retrieval and a framework
for the design of future studies [122].
Further exploration of this relationship, however, demon-
strated that the HERA model was insufficient. The current view is
that HERA reflects the maintenance of a retrieval mode, or a back-
ground cognitive state in which one is mentally attuned to the
retrieval process, is sensitive to incoming cues, and is capable of
becoming consciously aware of successful retrieval. Lepage et al.
[123] compared data across four PET studies and concluded that
six PFC regions were consistently recruited by episodic recogni-
tion. These included bilateral posterior ventrolateral areas, bilateral
frontopolar regions, right dorsal PFC, and midline cingulate area
near the supplementary motor area. How and when these regions
are recruited in EM retrieval depends on task demands. As pro-
posed in the source-monitoring framework [124], retrieval
attempts differ in the strategies used to access different cortical
representations depending on the nature and source of those represen-
tations. For example, visuospatial, semantic, or emotional cues are
often called upon to aid in episodic retrieval, and each of these cues
stimulates a slightly different set of cortical regions.
Using event-related fMRI, Dobbins and Wagner [125] showed
that the recollection of conceptual or perceptual details of an epi-
sode results in greater activation in the left frontopolar and poste-
rior PFC than the detection of novelty. The authors interpret this
finding as an evidence that a domain-general control network is
engaged during contextual remembering. In contrast, left anterior
VL PFC coactivated with a left middle temporal region associated
with semantic representation, during conceptual recollection,
while right VL PFC and bilateral occipito-temporal cortices were
coactivated during recollection of perceptual details. Therefore,
whereas left frontopolar and posterior PFC may be involved in
domain-general retrieval processes, the middle temporal, right VL
PFC, and occipito-temporal regions may be more
domain-specific.
Interestingly, emerging data suggest that PFC is not involved in
distinguishing correct from incorrect retrieval. Dobbins et al. [126]
found that activation in the left MTL was greater for correct than
for incorrect source attributions, or episodic retrieval, but that
numerous PFC regions that showed increased activation during
source memory retrieval did not distinguish between correct and
incorrect trials. Moreover, activation in some of these regions was
actually numerically greater for failed retrieval attempts. The authors
432 Federica Agosta et al.

interpret these findings as evidence that the PFC is involved in

elaborative or monitoring operations, which may be enhanced
when retrieval fails. Furthermore, Velanova et al. [127], using a
mixed block/event-related design to explore sustained and tran-
sient control processing during EM retrieval, found that left PFC
activation was associated trial-by-trial with retrieval when high con-
trol was required, whereas right PFC and several right posterior
regions were associated with sustained control processes, which the
authors viewed as a reflection of attentional set or retrieval mode. In
line with this idea, Kahn et al. [128] also failed to find differences in
prefrontal responses as a function of source retrieval outcome, and
PFC regions were more active during trials for old items than for
new ones. In addition, Wheeler and Buckner [129] found that acti-
vation in PFC correlates with retrieval processes and not with
retrieval success: they found that left posterior and mid-VL PFC
activity for items that were highly rehearsed and therefore easily
identified as old was not different than for new items. By contrast,
left PFC activity for items previously encountered only once was
greater than for both the old and new items, suggesting that left
PFC activity does not correspond to retrieval success. The authors
concluded that left VL PFC activity reflects a processing control
operation that is selectively engaged during demanding retrieval.

MTL Activation in Episodic Investigating MTL activation using fMRI can be difficult not only
Remembering because the region is susceptible to artifacts attributed to the ear
canal, but also because the region seems to be active during a wide
variety of tasks, including undirected “rest” [130]. Therefore,
many studies of episodic remembering do not report greater acti-
vation in the MTL, even though this region is known to be
involved. In order to observe MTL activity, a baseline task such as
an odd/even digit judgment may be used to deactivate the MTL
during the control trials [130].
Unlike data from the PFC, activation in the MTL has been
found to correlate with episodic retrieval success [131, 132]
(Fig. 4). Several neuroimaging studies investigating the role of
MTL subregions in EM have relied on the remember/know
procedure to distinguish episodic retrieval, or the processes of
“remembering or recollecting” (R), from retrieval based on famil-
iarity, called the “knowing or recalling” (K) [133]. This technique
was based on the finding that patients with MTL damage, espe-
cially those showing selective loss of hippocampal function, show
impairments in EM and a comparatively intact SM [80, 134]. In
addition, patients with degeneration of extrahippocampal temporal
cortex show the opposite pattern: impaired SM combined with a
relatively intact EM [78]. Using the remember/know procedure,
Eldridge et al. [131, 132] found that hippocampal activity is pri-
marily associated with “remembering” rather than “knowing”: the
hippocampus was more active during retrieval of “R” items than
 fMRI of Memory 433

b
Left hippocampus Right hippocampus
fMRI signal (percent change)

0.4
Correct R
0.3 Correct K
0.2 Correct Rejection
Miss
0.1
0
–0.1
–0.2
–0.3
–0.4
0 5 10 15 0 5 10 15
Time (s)

Fig. 4 Results from anatomically defined hippocampal regions of interest. (a) Sections from the anatomical
template with the left hippocampal region of interest outlined in red. (b) Averaged event-related responses in
the hippocampus from 11 subjects. Error bars represent one standard error (between subjects) of estimated
response amplitudes. Correct R correct remember (R) response (when subjects could recollect the moment the
item was studied), Correct K correct know (K) response (when the word is familiar but unaccompanied by the
recollection of the specific moment the word was presented), Correct rejection correct response for nonrecog-
nized items, Miss miss response (in which subjects did not recognize old items) (from Eldridge et al. [131])

during retrieval of “K” items. In a follow-up study, Eldridge et al.

[135] found that during encoding of items that were subsequently
correctly recognized, there was more activity in the dentate gyrus
and CA2/3 fields of the hippocampus than during encoding of
items that were subsequently forgotten. During retrieval, activity
in the subiculum correlated uniquely with “R” responses. These
data are particularly compelling because the dentate gyrus and
CA2/3 fields are located early in the hippocampal circuit, while
the subiculum is the major output region of the hippocampus.

Parietal Cortex The medial parietal cortex exhibits opposite levels of fMRI activity
during encoding and retrieval, a pattern dubbed the encoding/
retrieval (E/R) flip. These opposing effects were originally reported
434 Federica Agosta et al.

by Daselaar et al. [136] who observed this pattern within the same
participants for a variety of stimuli and memory paradigms. Since
then, the E/R-flip pattern has been replicated in several other
studies (for review, see Ref. [137]). It has been shown that this pat-
tern occurs regardless of the type of information (words, faces,
spatial scenes), stimulus modality (auditory or visual), and memory
test (item or relational memory) (Fig. 5). Several hypotheses have
been formulated to explain the E/R-flip pattern (for review, see
Ref. [137]). The internal orienting account asserts that medial
parietal cortex involvement in encoding and retrieval is dependent
on the internal versus external orientation of attentional. The self-
referential processing account explains medial parietal cortex activity
in terms of orienting toward self-relevant thoughts versus the
external environment. The reallocation account states that the acti-
vation differences in medial parietal cortex depend on task-demands
and follows response times. Finally, the bottom-up attention account
asserts that activity within medial parietal regions reflects bottom-
up orienting of attention towards information retrieved from
memory. Yet none of these cognitive accounts seems to provide a
full explanation for the E/R-flip pattern in the PMC [137].

4.2 Temporal One somewhat less-studied component of EM is the process of

Processing assigning a temporal order to a series of events. While this part of
the field is sparse in terms of neuroimaging data, there are a hand-
ful of studies that can shed some light on this process. Bilateral
middle prefrontal areas near BA 9, left inferior prefrontal (near BA
44/45), left anterior prefrontal (near BA 10/46), and bilateral
medial temporal areas show more activation during “high” tempo-
ral order retrieval trials (that is, when choosing between two words
that were close together on a list: i.e., which came first, word # 6
or word #3?) than during “low” trials (that is, when choosing
between words that were spaced far apart on a list: i.e., which came
first, word #1 or word # 9?) [138]. Activation in the middle frontal
gyrus (MFG) near BA 9 is especially interesting because there is
convergent evidence for the involvement of this region in the
human neuropsychological [139] and monkey literature [140].
There may also be a hemispheric specialization in temporal pro-
cessing. Suzuki et al. [141] asked participants to study pictures
during two separate sessions: one in the morning and another in
the afternoon. In the scanner, participants were asked to judge
whether an item was studied in the morning or in the afternoon, or
which of two items in the same list was studied earlier. They found
that right prefrontal activity was associated with temporal order
judgments of items between lists (morning vs. afternoon) while
left prefrontal activity was associated with the retrieval of temporal
order information within a list.
 fMRI of Memory 435

Fig. 5 Figure shows the encoding/retrieval flip in four different fMRI experi-
ments using (1) faces, (2) spatial scenes, (3) word pairs, and (4 and 5) single
words. During encoding, activity in the posterior midline region was greater for
misses (M) than for hits (H), whereas during retrieval, activity was greater for
hits than for misses. Bar graphs indicate mean cluster activity for the compari-
son between hits and misses during encoding and retrieval, respectively (from
Daselaar et al.[136])
436 Federica Agosta et al.

4.3 Summary Prior to the advent of fMRI, declarative memory, particularly EM,
of Episodic Memory was thought to be dependent almost exclusively on the MTL. Both
animal and human lesion studies provided the bulk of this evi-
dence. Neuroimaging techniques have demonstrated, however,
that PFC regions are recruited time and time again to support EM
retrieval. Interestingly, PFC does not seem to be sensitive to the
outcome of retrieval, but rather seems to support retrieval inten-
tion and strategy. The MTL, in contrast, seems to be driven largely
by “bottom-up” processes: being more data-driven and less volun-
tary [142]. Whereas details of an event are thought to be eventu-
ally stored in the same regions that were initially involved in their
perception, the hippocampus creates and stores an index of the
memory that can regenerate the original pattern of activation dur-
ing EM retrieval. Therefore, in healthy adults, EM involves a large
network of prefrontal, neocortical, and MTL regions acting in
concert to support this complex reconstructive process.

4.4 Future Directions The difference between EM and SM is the difference between the
active reconstruction of an event to extract information specific to
that occurrence and the abstraction of statistical regularities and
general properties about the world over multiple experiences. The
constructive nature of memory, therefore, is most easily observed
in EM: the reexperiencing of past events requires the reconstruc-
tion of a narrative sequence via the reactivation of stored sensory
information. A growing interest in the constructive aspects of EM
has led to the postulation of the constructive episodic simulation
hypothesis [143–145], which suggests that the EM system is built,
in part, to enable the simulation or imagination of future events.
Support for this view comes from several neuroimaging studies
demonstrating that the network of brain regions involved in EM
retrieval overlaps substantially with that supporting the imagina-
tion of future events [146–148]. Furthermore, as Hassabis and
Maguire [149, 150] have eloquently described, EM retrieval can
be thought of as relying heavily on scene construction as a key
component process. This approach may help explain why many
other cognitive tasks such as imagining a fictitious event seem to
employ many of the same regions involved in episodic
remembering.

5 Semantic Memory

In contrast to EM, SM corresponds to the general knowledge of

objects, words, facts, and people: declarative memories that are laid
down independently of the original encoding context [76]. This
information is often encountered over multiple repetitions, in a vari-
ety of contexts, and thus can be retrieved without regenerating details
of the original learning event. While initially limited to a memory
 fMRI of Memory 437

system for “words and other verbal symbols, their meanings and ref-
erents, about relations among them, and about rules, formulas and
algorithms” [76], SM currently refers to a broader knowledge set
that includes facts, concepts, and beliefs [151].
Reports of patients with focal deficits in SM that were category-
specific [152–155] led to the hypothesis that SM is organized by
taxonomic categories [156]. In addition, a highly influential the-
ory proposed by Allport [157] suggests that the same sensorimo-
tor areas that are involved in the initial processing of experience are
also used to represent abstractions related to the experience. Much
like findings from neuroimaging studies of EM, this theory pre-
dicts that modality-specific information is represented in the cortex
that processes that modality. Finally, studies of patients with pro-
gressive neural degeneration leading to deficits in SM suggest that
information in SM is also organized hierarchically, such that the
more specific a concept is, the more vulnerable it may be to brain
damage [158]. Therefore, we will consider evidence from neuro-
imaging studies that address these three components of SM:
category specificity, reactivation of sensorimotor areas, and hierar-
chical organization.

5.1 Organization Before the advent of functional brain imaging, our knowledge of
of SM Network the neural bases of SM was dependent on studies of patients with
brain injury. Investigations of semantic impairment arising from
brain disease suggested that the anterior temporal lobes (ATLs)
are critical for semantic abilities in humans, across all stimulus
modalities and for all types of conceptual knowledge [154, 155,
159–165]. As mentioned earlier, patients with semantic demen-
tia and progressive degeneration of the anterior temporal cortex
are impaired on all tasks requiring knowledge about the mean-
ings of words, objects, and people, although possibly to different
degrees for each category depending on the lateralization of
atrophy [162, 164, 166, 167]. Other brain diseases that can
affect the ATLs, such as Alzheimer’s disease [168] and herpes
simplex viral encephalitis [155], also often disrupt SM. Finally, it
is worth noting that patients with damage to the left PFC often
have difficulty in retrieving words in response to specific cues,
even in the absence of aphasia [169].
Although ATLs’ activation has been associated with a few
semantic tasks (i.e., sentence comprehension, and famous face
naming or identification, e.g., see Refs. [170, 171]), the vast major-
ity of the functional imaging experiments on SM have reported
posterior temporal, typically stronger in the left than in the right
hemisphere, and/or frontal activations in the left VL PFC, with no
mention to the ATLs (for review, see Refs. [151, 172–175]).
Intersubject variability and the occurrence of fMRI susceptibility
artifacts in the ATLs may be possible reasons for the lack of fMRI
activations detected in ATLs [176].
438 Federica Agosta et al.

Functional imaging results have also indicated that semantic

knowledge is encoded in a widely distributed cortical network,
with different regions specialized to represent particular types of
information [177], particular categories of objects [178], or both
[179], leading to the concept that no single region supports
semantic abilities for all modalities and categories. However, in
addition to these modality-specific regions and connections, the
various different surface representations (such as shape) connect
to, and communicate through, a shared, amodal “hub” in the
ATLs [175]. At the hub stage, associations between different pairs
of attributes (such as shape and name, shape and action, or shape
and color) are all processed by a common set of neurons and syn-
apses, regardless of the task. Damasio [180] was the first to argue
for unified conceptual representations that abstract away from
modality-specific attributes. He proposed the existence of “conver-
gence zones” that associate different aspects of knowledge clearly
articulating the importance of such zones for semantic processing.
The convergence-zone hypothesis proposes that there is no multi-
modal cortical area that would build an integrated and indepen-
dent semantic representation from its low-level sensory
representations. Instead, the representation takes place only in the
low-level cortices, with the different parts bound together by a
hierarchy of convergence zones. A semantic representation can be
recreated by activating its corresponding binding pattern in the
convergence zone.
Finally, unlike EM, remote SM is not dependent on the
involvement of the MTL [95]. The MTL is needed only temporar-
ily, until the knowledge itself is represented permanently by the
neocortical structures specialized in processing the acquired
information.

5.2 Category- Functional neuroimaging studies in which the cortical organiza-

Specific tion for semantic knowledge has been addressed have revealed dis-
Organization of SM sociations in the processing of different object categories (for
review, see Refs. [156, 172, 173, 175, 177, 181]). The most fre-
quently documented distinction is between “living” and “nonliv-
ing” items, body parts and numerals, and the most studied
categories have been human faces, houses, animals, and tools.
Various theoretical models have been proposed to explain the cog-
nitive mechanisms underlying category specificity. (1) The sensory
and functional/motor theory states that categories are defined by
the type of information needed to recognize items as belonging to
that category. “Living” items require object-related information
appreciable through perceptual channels (shape, color, sound,
etc.), whereas tools and body parts are more recognizable from
information concerning action, activity, or the motor scheme to
use them [154, 155, 160], (2) The “domain-specific theory” sug-
gests that evolutionary pressure has led to specific adaptations for
 fMRI of Memory 439

recognizing and responding to animals and plants, but not to

objects [156]. (3) The “correlated-structure principle theory”
proposes that conceptual organization reflects the statistical co-
occurrence of the properties of objects rather than an explicit divi-
sion into “living” and “nonliving” categories [182].
An impairment for processing items in the “living things” cat-
egory has mainly been described in patients with damage to the
anterior portions of the temporal lobe bilaterally [162, 163, 183,
184]. Nevertheless, the majority of functional neuroimaging stud-
ies (both PET and fMRI) failed to show consistent ATLs’ activa-
tions for “living item” stimuli (see earlier). PET studies in normal
subjects have shown that “living items” tend to activate predomi-
nantly posterior visual association cortices [185–188]. Only a
meta-analysis of seven individual PET studies [189] found
activations for “living” objects in the temporal poles bilaterally.
This large multistudy dataset provided sufficient sensitivity to
detect ATLs’ activations despite their inconsistency across subjects
and lack of significance in each study taken in isolation [189]. In
contrast, patients with deficits in the “nonliving” category show
damage in the left dorsolateral perisylvian regions [163, 183, 184]
Consistently, PET studies found activations specific to “nonliving”
stimuli in the left posterior middle and superior temporal gyri and
in the left inferior frontal cortex [185, 186, 189–191]. In an event-
related fMRI study in which words belonging to the categories
“living” and “nonliving” were presented visually, common areas of
activation during processing of both categories included the infe-
rior occipital gyri bilaterally, the left IFG, and the left IPL [192].
During processing of “living” minus “nonliving” items, signal
changes were present in the right inferior frontal, middle temporal,
and fusiform gyri.
Numerous fMRI studies have shown that different object cat-
egories elicit activity in different regions of the ventral temporal
cortex (for review, see Refs. [156, 172, 173, 175, 177, 181]).
Although it is more likely that these differences are due to percep-
tual, object recognition process rather than to semantic memory
function, we will briefly discuss them here. Perceiving animals
showed heightened, bilateral activity in the more lateral region of
the fusiform gyrus, whereas tools show heightened, bilateral activity
in the medial region of the fusiform gyrus and in the posterior mid-
dle temporal gyrus (MTG) [193]. A similar pattern of activations
was found for viewing faces (in the lateral fusiform) relative to view-
ing houses (the medial fusiform) [193]. The so-called FFA [194]
responds more strongly to faces than to other object categories, but
is not exclusive for faces [195, 196] (Fig. 6). House-related activity
was reported in more medial regions, including the fusiform and
lingual gyri [197], and parahippocampal cortex (the parahippocam-
pal place area [191], especially for landmarks [198]). More interest-
ingly, a meta-analysis by Joseph [195] revealed that the recognition
440 Federica Agosta et al.

Fig. 6 A PET study investigating brain responses to famous and nonfamous faces and buildings. The results
showed category-specific effects in the right fusiform and bilateral parahippocampal/lingual gyri for faces and
buildings, respectively, but no effect of fame. In contrast, the left anterior middle temporal gyrus showed an
effect of fame for both faces and buildings, but no effect of category. (a) Examples of the stimuli used in experi-
ments 1 and 2 for the face conditions and in experiment 2 for the building conditions. (b) From top to bottom,
this figure illustrates areas of activation and parameter estimates for regions that were more activated for (a)
faces than for buildings, (b) buildings than for faces, (c) famous than for nonfamous faces and buildings, and
(d) famous than for nonfamous faces only. In the left column, all activations are superimposed on axial slices
of the mean of the nine subjects’ normalized structural MRIs and thresholded at p < 0.001 (uncorrected). In the
right column, the plots indicate the value of the normalized regional cerebral blood flow at the indicated voxel
(y-axis) for each of the experimental conditions in experiments 1 and 2 (x-axis). EXP experiment, FF famous
faces, NFF nonfamous faces, FB famous buildings, NFB nonfamous buildings (from Gorno-Tempini et al. [196])

task used (i.e., viewing, matching, or naming) also predicted brain

activation patterns. Specifically, matching tasks recruit more inferior
occipital regions than either naming or viewing tasks do, whereas
naming tasks recruit more anterior ventral temporal sites than either
viewing or matching tasks do, thus indicating that the cognitive
demands of a particular recognition task are as predictive of cortical
activation patterns as is category membership.
 fMRI of Memory 441

5.3 Modality- Several of the early PET studies of semantic processes focused on
Specific the question of whether words (both auditory and visual) and pic-
Organization of SM tures are interpreted by a common semantic system [170, 188,
199, 200], or whether distinct systems are needed to support these
two domains. These studies reached similar conclusions providing
evidence for a distributed semantic system that is shared by visual/
auditory and verbal modalities [170, 188, 199, 200] and that it is
distributed throughout inferior temporal and frontal cortices with
a few areas uniquely activated by pictures only (left posterior
inferior temporal sulcus) or words (left anterior MTG and left infe-
rior frontal sulcus) [200]. More recently, fMRI studies have pro-
vided additional support for this view by demonstrating that
regions of the left posterior temporal cortex, known to be active
during conceptual processing of pictures and words (fusiform
gyrus and inferior and middle temporal gyri), are also active during
auditory sentence comprehension [201–203]: activity was modu-
lated by speech intelligibility [201, 202] and semantic ambiguity
[203]. A fMRI study used the phenomenon of semantic ambiguity
to identify regions within the fronto-temporal language network
that are involved in the processes of activating, selecting, and inte-
grating contextually appropriate word meanings [203]. Subjects
heard sentences containing ambiguous words (e.g., “The shell was
fired towards the tank”) and well-matched low-ambiguity sen-
tences (e.g., “Her secrets were written in her diary”). Although
these sentences had similar acoustic, phonological, syntactic, and
prosodic properties, the high-ambiguity sentences required addi-
tional processing by those brain regions involved in activating and
selecting contextually appropriate word meanings. The ambiguity
in these sentences went largely unnoticed, and yet high-ambiguity
sentences resulted in increased recruitment of the left posterior
inferior temporal cortex and the IFG bilaterally [203].

5.4 Representations While the neural systems involved in SM may be modulated both
of Object Properties by categories and modalities there is also evidence that the senso-
rimotor regions that are involved in the initial processing of par-
ticular information are recruited during SM retrieval (for review,
see Refs. [172, 173, 175, 177, 181, 204]). In particular, semantic
decisions involving object properties suggest a broad relationship
between perceptual knowledge retrieval and sensory brain mecha-
nisms, though recent work also suggests that this observation may
be itself category-specific [165].
Activation of the left or bilateral ventral temporal cortex (fusi-
form gyrus) when retrieving color information, relative to other prop-
erties, has been replicated by several fMRI studies [174, 205, 206].
Beauchamp et al. [205] showed that neural activity is limited to the
occipital lobes when color perception was tested by a passive viewing.
When the task was made more demanding by requiring subjects to
use color information to perform a color-sequencing task, several
442 Federica Agosta et al.

areas in the ventral cortex were identified: the most posterior, located
in the posterior fusiform gyrus, corresponded to an area activated by
passive viewing of colored stimuli, and more anterior and medial
color-selective areas located in the collateral sulcus and fusiform gyrus
[205]. These more anterior areas were also most active when visual
color information was behaviorally relevant, suggesting that atten-
tion influences activity in color-selective areas [205].
Parietal cortex appears to be involved in retrieval of size [174].
For instance, in Oliver and Thompson-Schill [174], seven subjects
made binary decisions about the shape, color, and size of named
objects during fMRI. Bilateral parietal activity was significantly greater
during retrieval of shape and size than during retrieval of color [174].
An area in the posterior superior temporal cortex, adjacent to
the auditory-association cortex, is activated when participants are
asked to judge the sound that an object makes [207]. Action knowl-
edge involves the left lateral temporal cortex, particularly the medial
and superior temporal (MT/MST) regions, anterior to an area
associated with motion perception [208–210]. For instance, in two
fMRI experiments Kourtzi and Kanwisher [209] found stronger
activation of the MT/MST regions during viewing of static photo-
graphs with implied motion compared with viewing of photographs
without it. Taken together, these data provide strong evidence that
information about a particular object property is stored in the same
neural system engaged when the property is perceived.

6 Conclusions

Working, episodic, and semantic memory systems engage multiple

brain regions and rely upon a number of cognitive processes. WM
is likely a property of the functional interaction between the PFC
and posterior brain regions. The MTL and its connections with
neocortical, prefrontal, and limbic structures are implicated in
EM. SM is mediated through a network of neocortical structures,
including the lateral and ATLs, and the inferior frontal cortex, pos-
sibly to a greater extent in the left hemisphere. Taken together,
these memory systems do have overlapping neuroanatomical
underpinnings, and even share some of the same component pro-
cesses, such as item and information retrieval. One emergent simi-
larity between the systems is the finding that those sensory and
association areas that are engaged during perceptual and senso-
rimotor processing of item information are tapped once again
when the information is required for memory processing. Future
work by researchers using neuroimaging to study memory will
require even further refinement in the definitions of memory sys-
tems and their components, as exemplified by the current work in
EM, where the retrieval component is being further reduced into
components such as scene construction.
 fMRI of Memory 443

References
1. Baddeley A, Hitch G (1974) Working mem- imaging of human prefrontal cortex activa-
ory. The psychology of learning and motiva- tion during a spatial working memory task.
tion. B. GH Academic, San Diego, pp 47–90 Proc Natl Acad Sci U S A 91:8690–8694
2. Baddeley A (2000) The episodic buffer: a 15. Awh E, Jonides J, Smith EE, Buxton RB, Frank
new component of working memory? Trends LR, Love T, Wong EC et al (1999) Rehearsal
Cogn Sci 4:417–423 in spatial working memory: evidence from neu-
3. Curtis CE, D’Esposito, M (2006) Working roimaging. Psych Sci 10:433–437
memory. In: Cabeza R, Kingstone A (eds) 16. Postle BR, D’Esposito M (1999) “What”-
Handbook of functional neuroimaging of Then-Where” in visual working memory: an
cognition. MIT Press, Cambridge, MA. event-related fMRI study. J Cogn Neurosci
269–306 11:585–597
4. Miyake A, Shah P (1999) Models of work- 17. Rowe JB, Passingham RE (2001) Working
ing memory. Cambridge University Press, memory for location and time: activity in
New York prefrontal area 46 relates to selection rather
5. Miller GA (1956) The magical number seven, than maintenance in memory. Neuroimage
plus or minus two: some limits on our capac- 14:77–86
ity for processing information. Psych Rev 18. Corbetta M, Kincade JM, Shulman GL
63:81–97 (2002) Neural systems for visual orienting
6. Goldman-Rakic PS (1994) Working and their relationships to spatial working
memory dysfunction in schizophrenia. memory. J Cogn Neurosci 14:508–523
J Neuropsychiatry Clin Neurosci 6:348–357 19. Davachi L, Maril A, Wagner AD (2001)
7. Muller NG, Machado L, Knight RT (2002) When keeping in mind supports later bring-
Contributions of subregions of the prefron- ing to mind: neural markers of phonological
tal cortex to working memory: evidence from rehearsal predict subsequent remembering.
brain lesions in humans. J Cogn Neurosci J Cogn Neurosci 13:1059–1070
14:673–686 20. D’Esposito M (2008). Working memory. In:
8. Fuster JM, Alexander GE (1971) Neuron Goldenberg G, Miller B (eds) Handbook
activity related to short-term memory. Science of clinical neurology. Neuropsychology and
173:652–654 Behavioral Neurology, Vol 88. Elsevier,
9. Kubota K, Niki H (1971) Prefrontal cor- Amsterdam, The Netherlands. 237–247
tical unit activity and delayed alternation 21. Courtney SM, Ungerleider LG, Keil K, Haxby
performance in monkeys. J Neurophysiol JV (1997) Transient and sustained activity in
34:337–347 a distributed neural system for human work-
10. Funahashi S, Bruce CJ, Goldman-Rakic PS ing memory. Nature 386:608–611
(1989) Mnemonic coding of visual space in 22. Courtney SM, Petit L, Haxby JV, Ungerleider
the monkey’s dorsolateral prefrontal cortex. LG (1998) The role of prefrontal cortex in
J Neurophysiol 61:331–349 working memory: examining the contents of
11. Owen AM, Herrod NJ, Menon DK, Clark consciousness. Philos Trans R Soc Lond B
JC, Downey SP, Carpenter TA, Minhas Biol Sci 353:1819–1828
PS et al (1999) Redefining the functional 23. Jha AP, McCarthy G (2000) The influence of
organization of working memory processes memory load upon delay-interval activity in a
within human lateral prefrontal cortex. Eur working-memory task: an event-related func-
J Neurosci 11:567–574 tional MRI study. J Cogn Neurosci 12(Suppl
12. Jonides J, Smith EE, Koeppe RA, Awh E, 2):90–105
Minoshima S, Mintun MA (1993) Spatial 24. Gruber O (2001) Effects of domain-specific
working memory in humans as revealed by interference on brain activation associated
PET. Nature 363:623–625 with verbal working memory task perfor-
13. Petrides M, Alivisatos B, Meyer E, Evans AC mance. Cereb Cortex 11:1047–1055
(1993) Functional activation of the human 25. Ranganath C, D’Esposito M (2001) Medial
frontal cortex during the performance of ver- temporal lobe activity associated with active
bal working memory tasks. Proc Natl Acad maintenance of novel information. Neuron
Sci U S A 90:878–882 31:865–873
14. McCarthy G, Blamire AM, Puce A, Nobre 26. Postle BR, Druzgal TJ, D’Esposito M (2003)
AC, Bloch G, Hyder F, Goldman-Rakic P Seeking the neural substrates of visual work-
et al (1994) Functional magnetic resonance ing memory storage. Cortex 39:927–946
444 Federica Agosta et al.

27. Curtis CE, Rao VY, D’Esposito M (2004) 40. Walter H, Wunderlich AP, Blankenhorn M,
Maintenance of spatial and motor codes Schafer S, Tomczak R, Spitzer M, Gron G
during oculomotor delayed response tasks. (2003) No hypofrontality, but absence of
J Neurosci 24:3944–3952 prefrontal lateralization comparing verbal and
28. Marklund P, Fransson P, Cabeza R, Petersson spatial working memory in schizophrenia.
KM, Ingvar M, Nyberg L (2007) Sustained Schizophr Res 61:175–184
and transient neural modulations in prefron- 41. Leung HC, Seelig D, Gore JC (2004) The
tal cortex related to declarative long-term effect of memory load on cortical activity in
memory, working memory, and attention. the spatial working memory circuit. Cogn
Cortex 43:22–37 Affect Behav Neurosci 4:553–563
29. Druzgal TJ, D’Esposito M (2003) Dissecting 42. Courtney SM, Ungerleider LG, Keil K,
contributions of prefrontal cortex and fusi- Haxby JV (1996) Object and spatial visual
form face area to face working memory. working memory activate separate neural sys-
J Cogn Neurosci 15:771–784 tems in human cortex. Cereb Cortex 6:39–49
30. D’Esposito M, Aguirre GK, Zarahn E, Ballard 43. Druzgal TJ, D'Esposito M (2001) Activity in
D, Shin RK, Lease J (1998) Functional MRI fusiform face area modulated as a function of
studies of spatial and nonspatial working working memory load. Brain Res Cogn Brain
memory. Brain Res Cogn Brain Res 7:1–13 Res 10:355–364
31. D’Esposito M, Postle BR, Rypma B (2000) 44. Rama P, Sala JB, Gillen JS, Pekar JJ, Courtney
Prefrontal cortical contributions to working SM (2001) Dissociation of the neural systems
memory: evidence from event-related fMRI for working memory maintenance of ver-
studies. Exp Brain Res 133:3–11 bal and nonspatial visual information. Cogn
32. Curtis CE, D’Esposito M (2003) Persistent Affect Behav Neurosci 1:161–171
activity in the prefrontal cortex during work- 45. Chein JM, Fiez JA (2001) Dissociation of
ing memory. Trends Cogn Sci 7:415–423 verbal working memory system components
33. Passingham D, Sakai K (2004) The prefron- using a delayed serial recall task. Cereb Cortex
tal cortex and working memory: physiology 11:1003–1014
and brain imaging. Curr Opin Neurobiol 46. Gruber O, von Cramon DY (2003) The
14:163–168 functional neuroanatomy of human working
34. Sreenivasan KK, Curtis CE, D'Esposito M memory revisited. Evidence from 3-T fMRI
(2014) Revisiting the role of persistent neu- studies using classical domain-specific inter-
ral activity during working memory. Trends ference tasks. Neuroimage 19:797–809
Cogn Sci 18:82–89 47. Ravizza SM, Delgado MR, Chein JM, Becker
35. Ungerleider LG, Haxby JV (1994) ‘What’ JT, Fiez JA (2004) Functional dissociations
and ‘where’ in the human brain. Curr Opin within the inferior parietal cortex in verbal
Neurobiol 4:157–165 working memory. Neuroimage 22:562–573
36. McCarthy G, Puce A, Constable RT, Krystal 48. Raye CL, Johnson MK, Mitchell KJ, Greene
JH, Gore JC, Goldman-Rakic P (1996) EJ, Johnson MR (2007) Refreshing: a mini-
Activation of human prefrontal cortex during mal executive function. Cortex 43:135–145
spatial and nonspatial working memory tasks 49. Chen SH, Desmond JE (2005)
measured by functional MRI. Cereb Cortex Cerebrocerebellar networks during articula-
6:600–611 tory rehearsal and verbal working memory
37. Courtney SM, Petit L, Maisog JM, Ungerleider tasks. Neuroimage 24:332–338
LG, Haxby JV (1998) An area specialized for 50. Kirschen MP, Chen SH, Schraedley-Desmond
spatial working memory in human frontal P, Desmond JE (2005) Load- and practice-
cortex. Science 279:1347–1351 dependent increases in cerebro-cerebellar
38. Munk MH, Linden DE, Muckli L, activation in verbal working memory: an
Lanfermann H, Zanella FE, Singer W, Goebel fMRI study. Neuroimage 24:462–472
R (2002) Distributed cortical systems in 51. Smith EE, Jonides J (1999) Storage and exec-
visual short-term memory revealed by event- utive processes in the frontal lobes. Science
related functional magnetic resonance imag- 283:1657–1661
ing. Cereb Cortex 12:866–876 52. Henson RN, Burgess N, Frith CD (2000)
39. Sala JB, Rama P, Courtney SM (2003) Recoding, storage, rehearsal and grouping in
Functional topography of a distributed neu- verbal short-term memory: an fMRI study.
ral system for spatial and nonspatial infor- Neuropsychologia 38:426–440
mation maintenance in working memory. 53. Crottaz-Herbette S, Anagnoson RT, Menon
Neuropsychologia 41:341–356 V (2004) Modality effects in verbal work-
 fMRI of Memory 445

ing memory: differential prefrontal and pari- cal course and experimental findings in
etal responses to auditory and visual stimuli. H.M. Semin Neurol 4:249–259
Neuroimage 21:340–351 67. Braver TS, Cohen JD, Nystrom LE, Jonides
54. Postle BR (2006) Working memory as an J, Smith EE, Noll DC (1997) A paramet-
emergent property of the mind and brain. ric study of prefrontal cortex involvement
Neuroscience 139:23–38 in human working memory. Neuroimage
55. Wager TD, Smith EE (2003) Neuroimaging 5:49–62
studies of working memory: a meta-analysis. 68. Stern CE, Sherman SJ, Kirchhoff BA,
Cogn Affect Behav Neurosci 3:255–274 Hasselmo ME (2001) Medial temporal and
56. Owen AM, Doyon J, Petrides M, Evans AC prefrontal contributions to working mem-
(1996) Planning and spatial working mem- ory tasks with novel and familiar stimuli.
ory: a positron emission tomography study in Hippocampus 11:337–346
humans. Eur J Neurosci 8:353–364 69. Ranganath C, Rainer G (2003) Neural mech-
57. Thompson-Schill SL, D'Esposito M, Aguirre anisms for detecting and remembering novel
GK, Farah MJ (1997) Role of left inferior pre- events. Nat Rev Neurosci 4:193–202
frontal cortex in retrieval of semantic knowl- 70. Honey GD, Fu CH, Kim J, Brammer MJ,
edge: a reevaluation. Proc Natl Acad Sci U S Croudace TJ, Suckling J, Pich EM et al (2002)
A 94:14792–14797 Effects of verbal working memory load on
58. Thompson-Schill SL, D’Esposito M, Kan IP corticocortical connectivity modeled by path
(1999) Effects of repetition and competition analysis of functional magnetic resonance
on activity in left prefrontal cortex during imaging data. Neuroimage 17:573–582
word generation. Neuron 23:513–522 71. Gazzaley A, Rissman J, D’Esposito M (2004)
59. D’Esposito M, Postle BR, Jonides J, Smith Functional connectivity during working
EE (1999) The neural substrate and tempo- memory maintenance. Cogn Affect Behav
ral dynamics of interference effects in work- Neurosci 4:580–599
ing memory as revealed by event-related 72. Crowe DA, Goodwin SJ, Blackman RK,
functional MRI. Proc Natl Acad Sci U S A Sakellaridi S, Sponheim SR, MacDonald AW
96:7514–7519 3rd, Chafee MV (2013) Prefrontal neurons
60. Jonides J, Smith EE, Marshuetz C, Koeppe transmit signals to parietal neurons that reflect
RA, Reuter-Lorenz PA (1998) Inhibition executive control of cognition. Nat Neurosci
in verbal working memory revealed by 16:1484–1491
brain activation. Proc Natl Acad Sci U S A 73. Edin F, Klingberg T, Johansson P, McNab F,
95:8410–8413 Tegner J, Compte A (2009) Mechanism for
61. D’Esposito M, Postle BR, Ballard D, Lease top-down control of working memory capac-
J (1999) Maintenance versus manipulation ity. Proc Natl Acad Sci U S A 106:6802–6807
of information held in working memory: 74. Chadick JZ, Gazzaley A (2011) Differential
an event-related fMRI study. Brain Cogn coupling of visual cortex with default or
41:66–86 frontal-parietal network based on goals. Nat
62. Postle BR, Berger JS, D'Esposito M (1999) Neurosci 14:830–832
Functional neuroanatomical double disso- 75. Hazy TE, Frank MJ, C O’Reilly R (2007)
ciation of mnemonic and executive control Towards an executive without a homunculus:
processes contributing to working memory computational models of the prefrontal cor-
performance. Proc Natl Acad Sci U S A tex/basal ganglia system. Philos Trans R Soc
96:12959–12964 Lond B Biol Sci 362:1601–1613
63. Bunge SA, Klingberg T, Jacobsen RB, 76. Tulving E (1972) Episodic and semantic
Gabrieli JD (2000) A resource model of the memory. Organisation of memory (Tulving E
neural basis of executive working memory. and Donaldson W (eds). Academic,
Proc Natl Acad Sci U S A 97:3573–3578 New York, pp 381–403
64. Cohen JD, Perlstein WM, Braver TS, 77. Vargha-Khadem F, Gadian DG, Watkins KE,
Nystrom LE, Noll DC, Jonides J, Smith EE Connelly A, Van Paesschen W, Mishkin M
(1997) Temporal dynamics of brain activa- (1997) Differential effects of early hippocam-
tion during a working memory task. Nature pal pathology on episodic and semantic mem-
386:604–608 ory. Science 277:376–380
65. Squire LR, Stark CE, Clark RE (2004) The 78. Hodges JR, Graham KS (2001) Episodic mem-
medial temporal lobe. Annu Rev Neurosci ory: insights from semantic dementia. Philos
27:279–306 Trans R Soc Lond B Biol Sci 356:1423–1434
66. Corkin S (1984) Lasting consequences of 79. Levine B, Black SE, Cabeza R, Sinden M,
bilateral medial temporal lobectomy: clini- McIntosh AR, Toth JP, Tulving E et al (1998)
446 Federica Agosta et al.

Episodic memory and the self in a case of iso- 94. Amaral DG, Insausti R, Cowan WM (1987)
lated retrograde amnesia. Brain 121(Pt The entorhinal cortex of the monkey:
10):1951–1973 I. Cytoarchitectonic organization. J Comp
80. Viskontas IV, McAndrews MP, Moscovitch M Neurol 264:326–355
(2000) Remote episodic memory deficits in 95. Moscovitch M, Rosenbaum RS, Gilboa A,
patients with unilateral temporal lobe epilepsy Addis DR, Westmacott R, Grady C,
and excisions. J Neurosci 20:5853–5857 McAndrews MP et al (2005) Functional neu-
81. Dudukovic NM, Knowlton BJ (2006) roanatomy of remote episodic, semantic and
Remember-know judgments and retrieval of spatial memory: a unified account based on
contextual details. Acta Psychol (Amst) multiple trace theory. J Anat 207:35–66
122:160–173 96. Teyler TJ, Rudy JW (2007) The hippocampal
82. Levine B, Svoboda E, Hay JF, Winocur G, indexing theory and episodic memory: updat-
Moscovitch M (2002) Aging and autobio- ing the index. Hippocampus 17:1158–1169
graphical memory: dissociating episodic from 97. Squire LR, Knowlton B, Musen G (1993)
semantic retrieval. Psychol Aging The structure and organization of memory.
17:677–689 Annu Rev Psychol 44:453–495
83. Bayley PJ, Hopkins RO, Squire LR (2006) 98. Baddeley A, Vargha-Khadem F, Mishkin M
The fate of old memories after medial tempo- (2001) Preserved recognition in a case of
ral lobe damage. J Neurosci developmental amnesia: implications for the
26:13311–13317 acquisition of semantic memory? J Cogn
84. Nolde SF, Johnson MK, D’Esposito M Neurosci 13:357–369
(1998) Left prefrontal activation during epi- 99. Rolls ET (1996) A theory of hippocampal
sodic remembering: an event-related fMRI function in memory. Hippocampus
study. Neuroreport 9:3509–3514 6:601–620
85. Wheeler ME, Petersen SE, Buckner RL 100. Gluck MA, Myers CE (1993) Hippocampal
(2000) Memory’s echo: vivid remembering mediation of stimulus representation: a com-
reactivates sensory-specific cortex. Proc Natl putational theory. Hippocampus 3:491–516
Acad Sci U S A 97:11125–11129 101. Norman KA, O'Reilly RC (2003) Modeling
86. Heil M, Rosler F, Hennighausen E (1996) hippocampal and neocortical contributions to
Topographically distinct cortical activation in recognition memory: a complementary-
episodic long-term memory: the retrieval of learning-systems approach. Psychol Rev
spatial versus verbal information. Mem 110:611–646
Cognit 24:777–795 102. Fletcher PC, Shallice T, Dolan RJ (1998) The
87. O’Craven KM, Kanwisher N (2000) Mental functional roles of prefrontal cortex in epi-
imagery of faces and places activates corre- sodic memory. I. Encoding. Brain 121(Pt
sponding stiimulus-specific brain regions. 7):1239–1248
J Cogn Neurosci 12:1013–1023 103. Shallice T, Fletcher P, Frith CD, Grasby P,
88. Halpern AR, Zatorre RJ (1999) When that Frackowiak RS, Dolan RJ (1994) Brain
tune runs through your head: a PET investi- regions associated with acquisition and
gation of auditory imagery for familiar melo- retrieval of verbal episodic memory. Nature
dies. Cereb Cortex 9:697–704 368:633–635
89. Nyberg L, Petersson KM, Nilsson LG, 104. Dolan RJ, Fletcher PC (1997) Dissociating
Sandblom J, Aberg C, Ingvar M (2001) prefrontal and hippocampal function in epi-
Reactivation of motor brain areas during explicit sodic memory encoding. Nature
memory for actions. Neuroimage 14:521–528 388:582–585
90. Marr D (1971) Simple memory: a theory for 105. Gabrieli JD, Brewer JB, Desmond JE, Glover
archicortex. Philos Trans R Soc Lond B Biol GH (1997) Separate neural bases of two fun-
Sci 262:23–81 damental memory processes in the human
91. Hasselmo ME, McClelland JL (1999) Neural medial temporal lobe. Science 276:264–266
models of memory. Curr Opin Neurobiol 106. Wagner AD, Schacter DL, Rotte M, Koutstaal
9:184–188 W, Maril A, Dale AM, Rosen BR et al (1998)
92. Aggleton JP, Brown MW (1999) Episodic Building memories: remembering and forget-
memory, amnesia, and the hippocampal- ting of verbal experiences as predicted by
anterior thalamic axis. Behav Brain Sci brain activity. Science 281:1188–1191
22:425–444, discussion 444-489 107. Brewer JB, Zhao Z, Desmond JE, Glover
93. Insausti R, Amaral DG, Cowan WM (1987) GH, Gabrieli JD (1998) Making memories:
The entorhinal cortex of the monkey: brain activity that predicts how well visual
II. Cortical afferents. J Comp Neurol experience will be remembered. Science
264:356–395 281:1185–1187
 fMRI of Memory 447

108. Wagner AD, Poldrack RA, Eldridge LL, 120. Otten LJ, Rugg MD (2001) Task-dependency
Desmond JE, Glover GH, Gabrieli JD (1998) of the neural correlates of episodic encoding
Material-specific lateralization of prefrontal as measured by fMRI. Cereb Cortex
activation during episodic encoding and 11:1150–1160
retrieval. Neuroreport 9:3711–3717 121. Nyberg L, McIntosh AR, Cabeza R, Habib R,
109. Fernandez G, Weyerts H, Schrader-Bolsche Houle S, Tulving E (1996) General and spe-
M, Tendolkar I, Smid HG, Tempelmann C, cific brain regions involved in encoding and
Hinrichs H et al (1998) Successful verbal retrieval of events: what, where, and when.
encoding into episodic memory engages the Proc Natl Acad Sci U S A 93:11280–11285
posterior hippocampus: a parametrically ana- 122. Tulving E, Kapur S, Craik FI, Moscovitch M,
lyzed functional magnetic resonance imaging Houle S (1994) Hemispheric encoding/
study. J Neurosci 18:1841–1847 retrieval asymmetry in episodic memory: pos-
110. Davachi L, Wagner AD (2002) Hippocampal itron emission tomography findings. Proc
contributions to episodic encoding: insights Natl Acad Sci U S A 91:2016–2020
from relational and item-based learning. 123. Lepage M, Ghaffar O, Nyberg L, Tulving E
J Neurophysiol 88:982–990 (2000) Prefrontal cortex and episodic mem-
111. Davachi L, Mitchell JP, Wagner AD (2003) ory retrieval mode. Proc Natl Acad Sci U S A
Multiple routes to memory: distinct medial 97:506–511
temporal lobe processes build item and source 124. Johnson MK, Hashtroudi S, Lindsay DS
memories. Proc Natl Acad Sci U S A (1993) Source monitoring. Psychol Bull
100:2157–2162 114:3–28
112. Kirwan CB, Stark CE (2004) Medial tempo- 125. Dobbins IG, Wagner AD (2005) Domain-
ral lobe activation during encoding and general and domain-sensitive prefrontal
retrieval of novel face-name pairs. mechanisms for recollecting events and
Hippocampus 14:919–930 detecting novelty. Cereb Cortex
113. Ranganath C, Yonelinas AP, Cohen MX, Dy 15:1768–1778
CJ, Tom SM, D'Esposito M (2004) 126. Dobbins IG, Rice HJ, Wagner AD, Schacter
Dissociable correlates of recollection and DL (2003) Memory orientation and success:
familiarity within the medial temporal lobes. separable neurocognitive components under-
Neuropsychologia 42:2–13 lying episodic recognition. Neuropsychologia
114. Kensinger EA, Schacter DL (2006) Amygdala 41:318–333
activity is associated with the successful 127. Velanova K, Jacoby LL, Wheeler ME, McAvoy
encoding of item, but not source, informa- MP, Petersen SE, Buckner RL (2003)
tion for positive and negative stimuli. Functional-anatomic correlates of sustained
J Neurosci 26:2564–2570 and transient processing components engaged
115. Uncapher MR, Otten LJ, Rugg MD (2006) during controlled retrieval. J Neurosci
Episodic encoding is more than the sum of its 23:8460–8470
parts: an fMRI investigation of multifeatural 128. Kahn I, Davachi L, Wagner AD (2004)
contextual encoding. Neuron 52:547–556 Functional-neuroanatomic correlates of rec-
116. Staresina BP, Davachi L (2008) Selective and ollection: implications for models of recogni-
shared contributions of the hippocampus and tion memory. J Neurosci 24:4172–4180
perirhinal cortex to episodic item and associa- 129. Wheeler ME, Buckner RL (2003) Functional
tive encoding. J Cogn Neurosci dissociation among components of remem-
20:1478–1489 bering: control, perceived oldness, and con-
117. Baker JT, Sanders AL, Maccotta L, Buckner tent. J Neurosci 23:3869–3880
RL (2001) Neural correlates of verbal mem- 130. Stark CE, Squire LR (2001) When zero is not
ory encoding during semantic and structural zero: the problem of ambiguous baseline con-
processing tasks. Neuroreport ditions in fMRI. Proc Natl Acad Sci U S A
12:1251–1256 98:12760–12766
118. Buckner RL, Wheeler ME, Sheridan MA 131. Eldridge LL, Knowlton BJ, Furmanski CS,
(2001) Encoding processes during retrieval Bookheimer SY, Engel SA (2000)
tasks. J Cogn Neurosci 13:406–415 Remembering episodes: a selective role for
119. Henson RN, Hornberger M, Rugg MD the hippocampus during retrieval. Nat
(2005) Further dissociating the processes Neurosci 3:1149–1152
involved in recognition memory: an FMRI 132. Wheeler ME, Buckner RL (2004) Functional-
study. J Cogn Neurosci 17:1058–1073 anatomic correlates of remembering and
knowing. Neuroimage 21:1337–1349
448 Federica Agosta et al.

133. Tulving E (1985) Memory and conscious- future: common and distinct neural substrates
ness. Can Psychol 1:1–12 during event construction and elaboration.
134. Tulving E, Schacter DL, McLachlan DR, Neuropsychologia 45:1363–1377
Moscovitch M (1988) Priming of semantic 148. Szpunar KK, Watson JM, McDermott KB
autobiographical knowledge: a case study of (2007) Neural substrates of envisioning the
retrograde amnesia. Brain Cogn 8:3–20 future. Proc Natl Acad Sci U S A
135. Eldridge LL, Engel SA, Zeineh MM, 104:642–647
Bookheimer SY, Knowlton BJ (2005) A dis- 149. Hassabis D, Maguire EA (2007)
sociation of encoding and retrieval processes Deconstructing episodic memory with con-
in the human hippocampus. J Neurosci struction. Trends Cogn Sci 11:299–306
25:3280–3286 150. Hassabis D, Maguire EA (2009) The con-
136. Daselaar SM, Prince SE, Dennis NA, Hayes struction system of the brain. Philos Trans R
SM, Kim H, Cabeza R (2009) Posterior mid- Soc Lond B Biol Sci 364:1263–1271
line and ventral parietal activity is associated 151. Martin A (2001) Functional neuroimaging of
with retrieval success and encoding failure. semantic memory, Handbook of functional
Front Hum Neurosci 3:13 neuroimaging of cognition (Cabeza R and
137. Huijbers W, Vannini P, Sperling RA, Pennartz Kingstone A (eds) ). The MIT Press,
CM, Cabeza R, Daselaar SM (2012) Cambridge, pp 153–186
Explaining the encoding/retrieval flip: 152. Damasio AR, McKee J, Damasio H (1979)
memory-related deactivations and activations Determinants of performance in color ano-
in the posteromedial cortex. Neuropsychologia mia. Brain Lang 7:74–85
50:3764–3774 153. McKenna P, Warrington EK (1980) Testing
138. Konishi S, Uchida I, Okuaki T, Machida T, for nominal dysphasia. J Neurol Neurosurg
Shirouzu I, Miyashita Y (2002) Neural cor- Psychiatry 43:781–788
relates of recency judgment. J Neurosci 154. Warrington EK, McCarthy R (1983)
22:9549–9555 Category specific access dysphasia. Brain
139. Milner B, Corsi P, Leonard G (1991) Frontal- 106(Pt 4):859–878
lobe contribution to recency judgements. 155. Warrington EK, Shallice T (1984) Category
Neuropsychologia 29:601–618 specific semantic impairments. Brain 107(Pt
140. Petrides M (1991) Functional specialization 3):829–854
within the dorsolateral frontal cortex for serial 156. Caramazza A, Shelton JR (1998) Domain-
order memory. Proc Biol Sci 246:299–306 specific knowledge systems in the brain the
141. Suzuki M, Fujii T, Tsukiura T, Okuda J, animate-inanimate distinction. J Cogn
Umetsu A, Nagasaka T, Mugikura S et al Neurosci 10:1–34
(2002) Neural basis of temporal context 157. Allport DA (1985) Distributed memory,
memory: a functional MRI study. Neuroimage modular systems and dysphagia, Current per-
17:1790–1796 spectives in dysphagia (Newman SK and
142. Moscovitch M (1995) Recovered conscious- Epstein R (eds)). Churchill Livingstone,
ness: a hypothesis concerning modularity and Edinburgh, pp 32–60
episodic memory. J Clin Exp Neuropsychol 158. Hodges JR, Graham N, Patterson K (1995)
17:276–290 Charting the progression in semantic demen-
143. Schacter DL, Addis DR (2007) The cognitive tia: implications for the organisation of
neuroscience of constructive memory: semantic memory. Memory 3:463–495
remembering the past and imagining the 159. Warrington EK (1975) The selective impair-
future. Philos Trans R Soc Lond B Biol Sci ment of semantic memory. Q J Exp Psychol
362:773–786 27:635–657
144. Dudai Y, Carruthers M (2005) The Janus face 160. Warrington EK, McCarthy RA (1987)
of Mnemosyne. Nature 434:567 Categories of knowledge. Further fraction-
145. Suddendorf T, Corballis MC (1997) Mental time ations and an attempted integration. Brain
travel and the evolution of the human mind. 110(Pt 5):1273–1296
Genet Soc Gen Psychol Monogr 123:133–167 161. Hart J Jr, Gordon B (1990) Delineation of
146. Okuda J, Fujii T, Ohtake H, Tsukiura T, Tanji single-word semantic comprehension deficits
K, Suzuki K, Kawashima R et al (2003) in aphasia, with anatomical correlation. Ann
Thinking of the future and past: the roles of Neurol 27:226–231
the frontal pole and the medial temporal 162. Hodges JRPK, Oxbury S, Funnell F (1992)
lobes. Neuroimage 19:1369–1380 Semantic dementia. Progressive fluent aphasia
147. Addis DR, Wong AT, Schacter DL (2007) with temporal lobe atrophy. Brain
Remembering the past and imagining the 115:1783–1806
 fMRI of Memory 449

163. Gainotti G (2000) What the locus of brain comparing PET and fMRI on a semantic task.
lesion tells us about the nature of the cogni- Neuroimage 11:589–600
tive defect underlying category-specific disor- 177. Martin A, Chao LL (2001) Semantic memory
ders: a review. Cortex 36:539–559 and the brain: structure and processes. Curr
164. Gorno-Tempini ML, Dronkers NF, Rankin Opin Neurobiol 11:194–201
KP, Ogar JM, Phengrasamy L, Rosen HJ, 178. Caramazza A, Mahon BZ (2003) The organi-
Johnson JK et al (2004) Cognition and anat- zation of conceptual knowledge: the evidence
omy in three variants of primary progressive from category-specific semantic deficits.
aphasia. Ann Neurol 55:335–346 Trends Cogn Sci 7:354–361
165. Brambati SM, Myers D, Wilson A, Rankin 179. Thompson-Schill SL, Aguirre GK, D'Esposito
KP, Allison SC, Rosen HJ, Miller BL et al M, Farah MJ (1999) A neural basis for cate-
(2006) The anatomy of category-specific gory and modality specificity of semantic
object naming in neurodegenerative diseases. knowledge. Neuropsychologia 37:671–676
J Cogn Neurosci 18:1644–1653 180. Damasio AR (1989) Time-locked multire-
166. Snowden JS, Thompson JC, Neary D (2004) gional retroactivation: a systems-level pro-
Knowledge of famous faces and names in posal for the neural substrates of recall and
semantic dementia. Brain 127:860–872 recognition. Cognition 33:25–62
167. Gorno-Tempini ML, Rankin KP, Woolley JD, 181. Thompson-Schill SL (2003) Neuroimaging
Rosen HJ, Phengrasamy L, Miller BL (2004) studies of semantic memory: inferring “how”
Cognitive and behavioral profile in a case of from “where”. Neuropsychologia 41:280–292
right anterior temporal lobe neurodegenera- 182. Tyler LK, Moss HE (2001) Towards a dis-
tion. Cortex 40:631–644 tributed account of conceptual knowledge.
168. Hodges JR, Patterson K (1995) Is semantic Trends Cogn Sci 5:244–252
memory consistently impaired early in the 183. Hillis AE, Caramazza A (1991) Category-
course of Alzheimer's disease? specific naming and comprehension impair-
Neuroanatomical and diagnostic implications. ment: a double dissociation. Brain 114(Pt
Neuropsychologia 33:441–459 5):2081–2094
169. Baldo JV, Shimamura AP (1998) Letter and 184. Tranel D, Damasio H, Damasio AR (1997) A
category fluency in patients with frontal lobe neural basis for the retrieval of conceptual
lesions. Neuropsychology 12:259–267 knowledge. Neuropsychologia 35:1319–1327
170. Gorno-Tempini ML, Price CJ, Josephs O, 185. Martin A, Wiggs CL, Ungerleider LG, Haxby
Vandenberghe R, Cappa SF, Kapur N, JV (1996) Neural correlates of category-
Frackowiak RS (1998) The neural systems specific knowledge. Nature 379:649–652
sustaining face and proper-name processing.
Brain 121(Pt 11):2103–2118 186. Mummery CJ, Patterson K, Hodges JR, Wise
RJ (1996) Generating ‘tiger’ as an animal name
171. Damasio H, Grabowski TJ, Tranel D, Hichwa or a word beginning with T: differences in
RD, Damasio AR (1996) A neural basis for brain activation. Proc Biol Sci 263:989–995
lexical retrieval. Nature 380:499–505
187. Perani D, Cappa SF, Bettinardi V, Bressi S,
172. Bookheimer S (2002) Functional MRI of lan- Gorno-Tempini M, Matarrese M, Fazio F
guage: new approaches to understanding the (1995) Different neural systems for the rec-
cortical organization of semantic processing. ognition of animals and man-made tools.
Annu Rev Neurosci 25:151–188 Neuroreport 6:1637–1641
173. Martin A (2007) The representation of object 188. Perani D, Schnur T, Tettamanti M, Gorno-
concepts in the brain. Annu Rev Psychol Tempini M, Cappa SF, Fazio F (1999) Word
58:25–45 and picture matching: a PET study of seman-
174. Oliver RT, Thompson-Schill SL (2003) tic category effects. Neuropsychologia
Dorsal stream activation during retrieval of 37:293–306
object size and shape. Cogn Affect Behav 189. Devlin JT, Moore CJ, Mummery CJ, Gorno-
Neurosci 3:309–322 Tempini ML, Phillips JA, Noppeney U,
175. Patterson K, Nestor PJ, Rogers TT (2007) Frackowiak RS et al (2002) Anatomic con-
Where do you know what you know? The straints on cognitive theories of category
representation of semantic knowledge in the specificity. Neuroimage 15:675–685
human brain. Nat Rev Neurosci 8:976–987 190. Gorno-Tempini ML, Cipolotti L, Price CJ
176. Devlin JT, Russell RP, Davis MH, Price CJ, (2000) Category differences in brain activa-
Wilson J, Moss HE, Matthews PM et al tion studies: where do they come from? Proc
(2000) Susceptibility-induced loss of signal: Biol Sci 267:1253–1258
450 Federica Agosta et al.

191. Cappa SF, Perani D, Schnur T, Tettamanti M, 201. Davis MH, Johnsrude IS (2003) Hierarchical
Fazio F (1998) The effects of semantic cate- processing in spoken language comprehen-
gory and knowledge type on lexical-semantic sion. J Neurosci 23:3423–3431
access: a PET study. Neuroimage 8:350–359 202. Giraud AL, Kell C, Thierfelder C, Sterzer P,
192. Leube DT, Erb M, Grodd W, Bartels M, Russ MO, Preibisch C, Kleinschmidt A
Kircher TT (2001) Activation of right fronto- (2004) Contributions of sensory input, audi-
temporal cortex characterizes the ‘living’ cat- tory search and verbal comprehension to cor-
egory in semantic processing. Brain Res Cogn tical activity during speech processing. Cereb
Brain Res 12:425–430 Cortex 14:247–255
193. Chao LL, Haxby JV, Martin A (1999) 203. Rodd JM, Davis MH, Johnsrude IS (2005)
Attribute-based neural substrates in temporal The neural mechanisms of speech compre-
cortex for perceiving and knowing about hension: fMRI studies of semantic ambiguity.
objects. Nat Neurosci 2:913–919 Cereb Cortex 15:1261–1269
194. Kanwisher N, McDermott J, Chun MM 204. Kartsounis LD, Shallice T (1996) Modality
(1997) The fusiform face area: a module in specific semantic knowledge loss for unique
human extrastriate cortex specialized for face items. Cortex 32:109–119
perception. J Neurosci 17:4302–4311 205. Beauchamp MS, Haxby JV, Jennings JE,
195. Joseph JE (2001) Functional neuroimaging DeYoe EA (1999) An fMRI version of the
studies of category specificity in object recog- Farnsworth-Munsell 100-Hue test reveals
nition: a critical review and meta-analysis. multiple color-selective areas in human ven-
Cogn Affect Behav Neurosci 1:119–136 tral occipitotemporal cortex. Cereb Cortex
196. Gorno-Tempini ML, Price CJ (2001) 9:257–263
Identification of famous faces and buildings: a 206. Goldberg RF, Perfetti CA, Schneider W
functional neuroimaging study of semanti- (2006) Perceptual knowledge retrieval acti-
cally unique items. Brain 124:2087–2097 vates sensory brain regions. J Neurosci
197. Aguirre GK, Zarahn E, D'Esposito M (1998) 26:4917–4921
An area within human ventral cortex sensitive 207. Kellenbach ML, Brett M, Patterson K (2001)
to “building” stimuli: evidence and implica- Large, colorful, or noisy? Attribute- and
tions. Neuron 21:373–383 modality-specific activations during retrieval
198. Epstein R, Harris A, Stanley D, Kanwisher N of perceptual attribute knowledge. Cogn
(1999) The parahippocampal place area: recog- Affect Behav Neurosci 1:207–221
nition, navigation, or encoding? Neuron 208. Puce A, Allison T, Bentin S, Gore JC,
23:115–125 McCarthy G (1998) Temporal cortex activa-
199. Petersen SE, Fox PT, Posner MI, Mintun M, tion in humans viewing eye and mouth move-
Raichle ME (1988) Positron emission tomo- ments. J Neurosci 18:2188–2199
graphic studies of the cortical anatomy of sin- 209. Kourtzi Z, Kanwisher N (2000) Activation in
gle-word processing. Nature 331:585–589 human MT/MST by static images with
200. Vandenberghe R, Price C, Wise R, Josephs O, implied motion. J Cogn Neurosci 12:48–55
Frackowiak RSJ (1996) Functional anatomy 210. Kable JW, Lease-Spellmeyer J, Chatterjee A
of a common semantic system for words and (2002) Neural substrates of action event
pictures. Nature 383:254–256 knowledge. J Cogn Neurosci 14:795–805
 Chapter 15

fMRI of Emotion
Simon Robinson, Ewald Moser, and Martin Peper

Abstract
Recent brain imaging work has expanded our understanding of the mechanisms of perceptual, cognitive, and
motor functions in human subjects, but research into the cerebral control of emotional and motivational
function is at a much earlier stage. Important concepts and theories of emotion are briefly introduced, as are
research designs and multimodal approaches to answering the central questions in the field. We provide a
detailed inspection of the methodological and technical challenges in assessing the cerebral correlates of
emotional activation, perception, learning, memory, and emotional regulation behavior in healthy humans.
fMRI is particularly challenging in structures such as the amygdala as it is affected by susceptibility-related
signal loss, image distortion, physiological and motion artifacts, and colocalized Resting State Networks
(RSNs). We review how these problems can be mitigated by using optimized echo-planar imaging (EPI)
parameters, alternative MR sequences, and correction schemes. High-quality data can be acquired rapidly in
these problematic regions with gradient-compensated multiecho EPI or high-resolution EPI with parallel
imaging and optimum gradient directions, combined with distortion correction. Although neuroimaging
studies of emotion encounter many difficulties regarding the limitations of measurement precision, research
design, and strategies of validating neuropsychological emotion constructs, considerable improvement in
data quality and sensitivity to subtle effects can be achieved. The methods outlined offer the prospect for
fMRI studies of emotion to provide more sensitive, reliable, and representative models of measurement that
systematically relate the dynamics of emotional regulation behavior with topographically distinct patterns of
activity in the brain. This will provide additional ­information as an aid to assessment, categorization, and
treatment of patients with emotional and personality disorders.

Key words Emotion, fMRI, Research design, Reliability, Validity, Amygdala, Signal loss, Distortion,
Resting state networks

1 Introduction

While recent brain imaging work has expanded our understanding

of the mechanisms of perceptual, cognitive, and motor functions in
human subjects, research into the cerebral control of emotional
and motivational functions has been less intense. For several years,
however, a growing body of fMRI and positron emission tomogra-
phy (PET) work has been assessing the cerebral correlates of emo-
tional activation, perception, learning and memory, and emotional
regulation behavior in healthy humans [1–6].

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_15, © Springer Science+Business Media New York 2016

451
452 Simon Robinson et al.

Current brain imaging work is based on the concepts and

hypotheses of the multidisciplinary field of “affective neurosci-
ence” [7–9]. The endeavors of the subdisciplines of affective neu-
roscience have not only complemented but also promoted each
other, stimulating a rapid growth of knowledge in the functional
neuroanatomy of emotions. It is increasingly recognized that these
areas also share similar methodological problems.
The expanding area of emotion neuroimaging has provided
new methods for validating neurocognitive models of emotion
processing that are crucial for many areas of research and clinical
application. Progress is being made in disentangling the cerebral
correlates of interindividual differences, personality, as well as of
abnormal conditions such as, for example, anxiety, depression, psy-
choses, and personality disorders [10–12]. It has been recognized
that psychological assessment, categorization procedures, and psy-
chotherapy treatment may profit from models that integrate func-
tional connectivity information. The relevance and usefulness of
valid neurocognitive models of emotion processing have recently
been recognized by many fields of applied research such as, for
example, psychotherapy research [13], criminology [14], as well as
areas such as “neuroeconomics” [15] and “neuromarketing” [16].
Several human lesion studies have pointed to the deficits of
neurological patients in recognizing emotions in faces, particularly
often for the decoding of fearful faces especially after bilateral
amygdala damage [17–20]. Other studies have reported impair-
ments not only for fear but also for other negative emotions such
as anger, disgust, and sadness [21, 22]. Recent functional imaging
studies have confirmed the importance of the amygdala in emotion
processing. Due to the multiple connections between the amyg-
dala and various cortical and subcortical areas, and the fact that the
amygdala receives processed input from all the sensory systems, its
participation is essential during the initial phase of stimulus evalu-
ation [23]. The appraisal function of the amygdala, combining
external cues with an internal reaction, reflects the starting point
for a differential emotional response and is hence the basis for
emotional learning. Involvement of the amygdala during classical
conditioning especially during the initial stages of learning [24,
25] as well as during processing signals of strong emotions has
been documented repeatedly with fMRI. However, a problem in
verifying amygdala activation with neuroimaging tools may be the
rapid habituation of its responses [10, 26].
Although the need for brain imaging data is not unequivocally
acknowledged by all researchers in their specialties, the increasing
body of neuroimaging data has value in challenging and constraining
existing theories. Followers of cognitive emotion theory must face the
fact that their results need to be compatible with or at least not con-
tradict with established neuroscience (neuroimaging) findings [27].
However, to appropriately evaluate and integrate this knowledge, it is
necessary to deal with the basic methodological problems of the field.
 fMRI of Emotion 453

Therefore, this chapter is organized around the two major issues

of the neuroimaging of emotional function. First, it addresses under-
lying conceptual issues and difficulties associated with operational-
izing and measuring emotion (for more detailed reviews, see Refs.
[28–31]). The problems and limitations of brain imaging work that
are associated with measurement precision, response scaling, repro-
ducibility, as well as validity and generalizability are discussed corre-
sponding to general principles of behavioral research [32].
Second, the complexities of neuroimaging methods are exam-
ined to supplement recent quantitative meta-analyses (for a sum-
mary of findings of the emotional neuroimaging literature, see
Refs. [2, 4, 5]). We raise here some grounds for reflection about
current measurement in neuroimaging of emotions, and to encour-
age the adoption of recent methodological advances of fMRI tech-
nology. In summary, it is suggested that additional interdisciplinary
efforts are needed to advance measurement quality and validity,
and to accomplish an integration of brain imaging technology and
neuropsychological assessment theory.

2 Psychological Methods

2.1 Emotion Theories Emotions have been defined as episodes of temporarily coupled,
and Constructs coordinated changes in component functions as a response of the
organism to external or internal events of major significance. These
2.1.1 Definitions
component functions entail subjective feelings, physiological acti-
vation processes, cognitive processes, motivational changes, motor
expression, and action tendencies [33, 34]. Emotions represent
functions of fast and flexible systems that provide basic response
tendencies for adaptive action [35].
Emotions can be differentiated from mood changes (extended
change in subjective feeling with low intensity), interpersonal
stances (affective positions during interpersonal exchange), atti-
tudes (enduring, affectively colored beliefs, preferences, and pre-
dispositions toward objects or persons), and personality traits
(stable dispositions and behavior tendencies) [29, 34].
The frequently used concept of “emotional activation” charac-
terizes a relatively broad class of physiological or mental phenom-
ena (e.g., strain, stress, physiological activation, arousal, etc.). It
can be specified with respect to a variety of dimensions such as
valence (quality of emotional experience), intensity or arousal
(global organismic change), directedness (motivational and orien-
tating functions), and selectivity (specific patterns of change) [36].
In contrast, the terms emotional reactivity or arousability, and psy-
chophysical reactivity refer to the dispositional variability of the
above activation processes under defined test conditions [37, 38].
Environmental objects possess a latent meaning structure of
emotional information, which is represented by a hierarchy of
454 Simon Robinson et al.

constructs with relatively fixed intra- and interclass relations [29,

39, 40]. Accordingly, physical stimulus properties or surface cues
serve as a basis for “universal” emotion categories such as happi-
ness, surprise, fear, anger, sadness, and disgust that originate at a
primary level [41]. On a secondary level, dimensions such as valence
and arousal arise from the preceding levels [42]. Table 1 suggests a
potential structure of emotional concepts or domains that inte-
grates both discrete (primary) and secondary emotions [29].
Emotional activation has also been characterized as a process
with a sequence of stages [29, 35]: following an initial evaluation
of novelty, familiarity, and self-relevance, a stimulus object or con-
text is fully encoded. This involves detection of physical stimulus
features, recognition of object identity, and identification of higher-
order emotional dimensions such as pleasantness or need signifi-
cance. During the subsequent stages, cognitive appraisal processes
are initiated to evaluate the significance of the event. These evalu-
ation checks include an appraisal of whether the stimulus is relevant
for personal needs or achieving certain goals. Finally, the potential
to overcome or cope with the event and the compatibility of behav-
ior with the self or social norms is evaluated [35].

2.1.2 Operationalization The measurement of emotions crucially depends on an appropriate

operationalization of the construct of interest and definition of
response parameters. Such considerations have typically been elab-
orated in the context of psychological assessment theory [30, 38,

Table 1
Hierarchical organization of emotion concepts (modified from [29])

Emotion concepts or
domains Example constructs Basis for higher-order grouping
Dimensional concepts Valence (positive/negative Conceptual or meaning space for
emotions), approach/withdrawal, subjective experience and verbal
activity (active/passive), control, labels
etc.
Basic, fundamental, Anger, fear, sadness, joy, etc. Similarity of appraisal, motivational
discrete, modal emotions consequences, and response
or emotion families patterns; convenient label for
appropriate description and
communication
Specific appraisal/response Righteous anger, jealousy, mirth, Temporal coordination of different
configurations for fright, etc. response systems for a limited
recurring events/ period of time as produced by a
situations specific appraisal pattern
Continuous adaptational Orienting reflex, defense reflex, Automatic activations and
changes startle, sympathetic arousal, etc. coordination of basic
biobehavioral units
 fMRI of Emotion 455

43]. The latter explains how psychological and physiological mea-

sures can be empirically assessed, decomposed, and used as indica-
tors of the psychological constructs of interest. It organizes the
assumptions concerning measurement, segmentation, and aggre-
gation of activation measures, and evaluates the distribution char-
acteristics and reliability of the data. It also determines the range of
the construct of interest by localizing it according to variables, sub-
jects or settings/situations, or combinations of these sources of
variation. Since most current operationalizations are confined to
one of these aspects, the range of conclusions to be drawn from the
findings is also limited.
A particular problem associated with measuring emotional
reactions is a certain lack of covariation of response measures. A
frequent finding is that the expected synchronization of verbal,
motor, and physiological response systems during an emotional
episode is the exception rather than the rule. Although emotional
episodes supposedly give rise to a synchronization of central, auto-
nomic, motor, and behavioral variables [44], most emotional
response measures only show imperfect coupling [45]. This
response incoherence may be attributed to a temporary decoupling
or dissociation of function [46]. This has led authors to suggest a
triple response measurement strategy that suggests a multimodal
assessment of emotion including responses in the verbal, gross
motor, and physiological (autonomic, cortical, neuromuscular)
response systems [47].
Research on human emotion has illustrated how the broad
concept of emotion is subdivided into several component func-
tions that dynamically interact during an emotional episode.
Diverse operationalizations have been suggested to assess these
subconstructs, many of which are highly correlated and form clus-
ters or families of similar functions. Emotional activation processes
are embedded in a multicomponential system of situational and
personal determinants. Factors that shape the level and pattern of
the emotional activation process are the following [29, 48]: the
functional context of the task (e.g., cognitive processing, motor
responses, autonomic functions, etc.); the direction and extension
of effects (e.g., global versus selective activation); the intensity and
the degree of emotional strain (e.g., low, middle, or traumatic
intensity; degree of threat; intensity of physical/mental load; stim-
ulus intensities below or above threshold); the time characteristics
(e.g., duration, structure, and variability of a stimulus; effects of
stimulus repetition or pre-exposure); the informational content
(e.g., the degree of information and dimensions inherent in the
experimental stimuli such as emotional valence or arousal, pre-
paredness, novelty, safety, predictability, contingency information,
etc.); the implications for action (conduciveness, implications for
instrumental reactions; artificial vs. realistic nature of the proce-
dure); the coping potential (e.g., active coping vs. passive
456 Simon Robinson et al.

enduring, degree of controllability, helplessness, social support,

specific coping strategies); and, compatibility with self or social
norms (e.g., personal relevance).
These different aspects have led to a large number of opera-
tionalizations. These include procedures to elicit orienting or star-
tle reactions, basic emotions or “stress,” as well as stimulus-response
paradigms and conditioning procedures. For example, one such
standard procedure is to elicit orienting reactions (OR) by emo-
tionally meaningful stimuli. The OR is a nonassociative process
being modulated by excitatory (sensitization) and inhibitory
(habituation) mechanisms. Pavlovian (classical) or instrumental
conditioning of excitatory or inhibitory reactions has traditionally
been investigated in autonomic reactions (cardiovascular, vasomo-
tor, and electrodermal conditioning), motor responses (eye blink),
and endocrine or immune system reactions [1].
Emotional experience is strongly influenced by cognitive activities
which modulate attention and alertness (avoidance and escape), vigi-
lance processes (information search and problem solving), person–
situation interactions (denial, distancing, cognitive restructuring,
positive reappraisal, etc.), and actions, which change the person–envi-
ronment relationship [49]. Coping research has identified typical cog-
nitive strategies to regulate arousal during an emotional episode such
as rejection (venting, disengagement) and accommodation strategies
(relaxation, cognitive work) [50]. Cognitive activities subsume
engagement (reconceptualization, reevaluation strategies such as
rationalization or reappraisal) and distraction techniques.
These behavioral and cognitive regulation processes have been
studied for many decades [51]. This research has shown that the
outcome of coping processes crucially depends upon the valence,
ambiguity, controllability, and changeability of a stressor. Input-­
related regulation (denial, distraction, defense, or cognitive restruc-
turing; [52]) or antecedent-focused regulation (selection,
modification, or cognitive restructuring of situational antecedents;
[53]) have been differentiated from response-focused processes
(suppression of expressive behavior and physiological arousal; [53]).
While the behavioral procedures mentioned above are mostly
unstandardized, a vast number of standardized psychometric instru-
ments are available to assess the higher-order emotional processes (for
a review see Ref. [31]). Questionnaires are the most frequently used
method, being followed by behavior ratings by experts or significant
others. However, these data assess subjective representations, that is,
personal constructs and may be obscured by biased responding

2.2 Research Design The requirements for experimental research [32] are not always
and Validity fulfilled by many early research designs of emotional neuroimaging
work. This is typical for the pilot stage of scientific progress. In
2.2.1 Research Design
many cases, only preliminary or correlational interpretations are
possible due to incomplete or missing control conditions (e.g.,
with respect to the “awareness” of emotional stimuli; [54]). In
 fMRI of Emotion 457

contrast, more recent work increasingly makes use of full factorial

designs or applies parametric variations of the independent variable
[55]. Moreover, new techniques of covariance analysis are available
to explore the causal predictive value of structural data on emo-
tional brain activation. The relationship of structural and func-
tional connectivity data has been explored by means of Structural
Equation Modeling [56–58] and Dynamic Causal Modeling [59].
Moreover, functional brain imaging has been successfully com-
bined with the lesion approach to elucidate the modulating influ-
ences of interconnected brain regions [60]. Thus, by means of
appropriate research plans and advanced techniques of analysis, an
“effective connectivity” can be identified that elucidates the causal
relations of one neural system to another [61]. For example, the
functional connectivity of the prefrontal cortex (PFC) that is
­supposed to modulate amygdala activity [62] might thus be better
evaluated in terms of causality.
To avoid operationalization errors, the quality of the emotion
induction procedure needs to be scrutinized, that is, it must be
evaluated whether the intended emotion has actually been elicited.
For example, since a variety of emotional and nonemotional stimu-
lus situations may trigger amygdala activations [5], it is necessary
to evaluate whether the intended emotion (such as fear) has actu-
ally been elicited. Since subjective report is not always an appropri-
ate manipulation check, additional psychophysiological criteria are
needed to validate the intended emotion. Sympathetic activity as
indexed by electrodermal activity (EDA) has been assessed during
imaging procedures for this purpose. Nevertheless, this does not
validate fear since skin conductance responses represent the end-
point of many different processes [63].

2.2.2 Construct Brain imaging work implements specific neuropsychological con-

Validation struct validation strategies by associating behavioral measurement
of emotion with functional brain activation data for different
localizations [31]. Here, functional (physiological) data are
related to but still remain categorically distinct from the psycho-
logical data that emerge from a particular behavioral paradigm.
During the process of construct validation, indicators of connec-
tional or neurophysiological constructs are related to the indica-
tors of psychological constructs. Thus, different operationalizations
of a certain psychological construct (procedures or task) are
expected to be correlated with activations of a certain area or
cluster of areas. A different construct is expected to correlate with
another but not the previous area and vice versa. This corre-
sponds to the double dissociation approach, which inspects task
by localization interactions. This process of neuropsychological
concept formation typically starts at a relatively broad level and
proceeds downward in the above hierarchy finally specifying
within-systems localization constructs [64].
458 Simon Robinson et al.

However, depending on limitations of the measurement device

described below, the reliability of psychological or activation data
declines at lower levels of structural constructs complicating this
validation process. The diverse validation attempts typically draw
upon convergent or divergent associations of constructs that are
located at quite different levels of generality. However, successful
construct validation very much depends upon whether brain activa-
tion and psychological measures are analyzed on the same level of
generality. In cases of asymmetry, low relationships may result that
provoke misinterpretations and confuse the validation process. Thus,
successful construct validation in the affective neurosciences requires
emotional constructs and brain activation data to be ­measured on
the same (symmetrical) level of generality or aggregation [31].
Emotional neuroimaging is typically guided by neuropsychologi-
cal construct validation strategies. Here, the constructs are opera-
tionally defined by the complementary methods of emotion
psychology and of neurophysiology. Both construct types are embed-
ded in hierarchically organized networks with lower- and higher-
order levels of generality. Both types of data are associated with each
other during validation. However, it is necessary to define neural and
emotional constructs on the same level of generality. For example,
when a relatively broad behavioral category or set of functions (“emo-
tion regulation”) is being associated with isolated cerebral substruc-
tures, the relationship is likely to be asymmetrical and disappointing
low correlations might result confusing the validation process.

2.2.3 Internal Validity FMRI is known to be a highly reactive measure because the scanner
setting (gradient noise and the supine position) causes the subject to
respond to the experimental situation as a stressor. Unless habitua-
tion sessions are included in the procedure, tonic stress and arousal
effects may be induced that modulate responding as discussed above.
For example, a decreasing rate of response of the amygdala to a con-
ditioned stimulus during the late phase of acquisition [10, 24, 26,
65] may also be attributable to testing effects (sensitization to the
setting, acquaintance with the procedure, and type of unconditioned
stimulation) rather than fast amygdala habituation per se (other fac-
tors might also explain reduced amygdala perfusion measures such
as potential ceiling effects, baseline dependencies, and regression to
the mean). In general, familiarity with emotionally activating proce-
dures in the scanner induces states of expectation, sensitizing or
desensitizing effects that may confound follow-up measurement. In
addition to these testing effects, history, that is, occurrences other
than the treatment and individual experiences between a first and a
second measurement are likely to endanger the assessment of emo-
tion (e.g., when assessing psychotherapy effects).
Changes in the observational technique, the measurement
device or sequence and other instrumentation effects may also
obscure emotion-related treatment variance during an fMRI
 fMRI of Emotion 459

session or across sessions. From the discussion of MR methods it is

clear that longitudinal changes of measurement precision are also
to be expected from inconsistent acquisition geometry and shim,
as well as system instabilities and hardware changes.
It is well known from psychophysiological research that the
interpretation of repeated measurement factors is complicated by
initial value dependencies [66]. When the hemodynamic response is
fitted relative to the prestimulus baseline, a physiological or ­statistical
dependency of tonic perfusion levels and the phasic reaction may
prevail [67]. While the first experimental blocks may show extreme
effects, subsequent measurements are likely to be closer to the
mean. Moreover, it has been pointed out above that the reliability
of blood oxygen level-dependent (BOLD) measurements may be
compromised by distortions or signal loss. When emotional para-
digms with inconsistent effects are used or when subjects with an
extreme variability of emotional responsivity are investigated,
experimental effects are likely to show “regression to the mean.”
Subjects change as a function of time and these maturation
effects may occur during the time range of the experiment (psy-
chophysiological changes of organismic state or psychological
stance, in particular during aversive paradigms). State-dependent
influences or maturation effects may hamper within-subject repli-
cation or evaluations of long-term psychotherapy effects.
Subject groups with an elevated emotionality are more likely to
show greater dropout rates in stressful experiments, that is, subjects
of one group drop out as a consequence of their specific reactivity
to the emotionally strain of the challenge paradigm. If exit from an
emotionally activating study is not random, this effect of “experi-
mental mortality” may confound comparison between groups.
Selection effects, that is, group differences from the outset of
the study, are likely in functional imaging studies with very small
numbers of participants. Selective recruitment of volunteers or drop
out of participants may lead to decreased reactivity and lower emo-
tionality in the remaining study group. Poor recruitment techniques
(e.g., drafting subjects from the social circle of the lab partially
acquainted with the procedures) or lack of random assignment to
groups may further limit the validity of emotional fMRI studies.
Interactions of selection with maturation may occur when
groups that differ with respect to maturation processes are com-
pared (e.g., administering a social stress test for cortisol stimula-
tion at different times of the day). Gender, personality traits, or
psychopathology are all associated with specific individual differ-
ences of emotional regulation behavior. When these behaviors
change over time as a function of personal development, follow-up
measurements may be confounded by this type of effect. Thus,
poor randomization or lack of control of personality-specific vari-
ance may jeopardize brain activation studies of emotional behavior.
Finally, an interaction of selection with instrumentation occurs
460 Simon Robinson et al.

when experimental subjects and controls show pre-experimental

differences with respect to the shape of their responses such as
floor or ceiling effects.
In general, emotional responses show an intraindividual insta-
bility due to measurement artifacts (see Sect. 3), state-dependent
influences, or characteristics of the subject (age, gender, experience,
temperament) all impose additional effects on functional neuroim-
aging results [5]. A considerable degree of within- and between-
subject variation in the time course of emotional responding
depends on habitual, subject-specific mechanisms. First, the phasic
activation pattern reflects the short-term modulation in response to
the emotional stimulus. Due to the temporal within-­trial variability
of BOLD responses in different brain regions, averaging across sub-
jects may obscure the detection of activation in a specific region and
reduce effect sizes specifically for higher-level reactions. Second,
activation also varies across the time course of the experiment. Most
subjects show a constant increase in autonomic arousal depending
on the degree of emotional stimulation. This is not only accompa-
nied by a systemic response (tonic increase of sympathetic activation
including blood pressure, cardiac contractility, and variability), but
also by variations of tonic perfusion. These changes may show
divergent trends for cortical and limbic regions imposing an
unknown error on the measurement of the phasic BOLD reaction.
These tonic and phasic variations appear to reflect the subject-spe-
cific mechanisms of emotional regulation behavior.

2.2.4 External Validity The majority of current paradigms have focused on lower-level
and Generalizability perceptual or learning processes pertaining to basic or secondary
emotional categories. Since the results depend on the selected task
Generalization to Other
parameters (degree of induced arousal, hedonic strength, and
Procedures and Paradigms
motivational value; degree of involvement of memory processes;
reinforcement schedule; conditioning to cues or contexts; etc.), a
comparison with and generalization to other operationalizations
remains difficult. Systematic neuroimaging approaches to higher-­
level appraisal processes are still sparse. These involve evaluations
of the motivational conditions and coping potential, that is, the
ability to overcome obstructions or to adapt to unavoidable con-
sequences [29]. An expanded range of constructs would involve
an assessment of social communication processes, beliefs, prefer-
ences, predispositions, high-level evaluation checks, as well as
modulating sociocultural influences. Higher-order appraisal pro-
cesses involve the evaluation of whether stimulus events are com-
patible with social standards and values or with the self-concept.
Another function to be explored concerns the degree to which a
stimulus event may increase, decrease, or even block goal attain-
ment or need satisfaction, and activate a reorientation of the indi-
vidual’s goal/need hierarchy and behavioral planning (goal/need
priority setting) [29].
 fMRI of Emotion 461

Whereas frontostriatal mechanisms of motor control have been

increasingly investigated, recent work has made efforts toward devel-
oping an understanding of how emotion and motivation are linked
to the frontal mechanisms controlling the preparation and execution
of behavior [68, 69]. Behavior preparation and execution represent
closely integrated components within an emotional episode.
Mobilization of energy is required to prepare for a certain class of
behavior. Action planning and motor preparation requires sequenc-
ing of actions and generation of movements. However, an emotion
preceding behavior is only one of a number of factors, including
situational pressures, strategic concerns, or instrumentality, involved
in eliciting the concrete action. Additional research is needed to
trace the information flow from motivational to motor systems.
Another component is the verbal or nonverbal communication of
emotions such as facial expression or vocal prosody [70]. The ability
to verbally conceptualize emotions and to communicate emotional
experiences plays an important role in the regulation of an ongoing
emotional episode. For example, explicit emotion-­labeling tasks have
been shown to decrease the activation level of the amygdala [71, 72].
Finally, sociocultural factors may shape attitudes (relatively
enduring, affectively colored beliefs, preferences, and predisposi-
tions toward objects or persons) as well as interpersonal stances
(affective stance taken toward another person in a specific interac-
tion). The ability of the individual to form representations of
beliefs, intentions, and affective states of others has a considerable
importance for affective and interpersonal interaction. However,
the effects of beliefs, preferences, and predispositions on lower lev-
els of emotional responding have attracted little attention. Top-­
down processes may induce considerable variations of task and
stimulus parameters by modulating lower-level automatic processes
and by controlling the late behavior preparation stages during the
emotional process. Thus, generalization to other paradigms and
constructs has limitations because higher-level behavioral and cog-
nitive strategies that are part of the individual emotion regulation
system ([50]; see later) modulate the emotion process.

Generalization to Other The study groups of many fMRI studies have been relatively small
Subjects and Populations and poorly described with respect to personality dimensions. Since
several studies provide evidence for trait-dependent differences in
responding [73–76], it remains unclear to what extent the results
may have been influenced by interindividual differences of the par-
ticipating subjects. The representativeness of results is particularly
poor if members of the social circle of the lab serve as participants
instead of independently recruited participants. Thus, when the
effects of an emotional paradigm interact with characteristics of the
study groups (such as a low level of emotionality in subjects willing to
participate in an activating scanning condition), this selection × treat-
ment effect may endanger generalizations to other populations.
462 Simon Robinson et al.

Generalization to Other The prediction of future emotional or psychopathological disor-

Times and Settings ders on the basis of emotional behavior assessed in the scanner
remains difficult [77]. Eliciting emotions in the imaging scanner is
a highly artificial situation. It remains unclear to what extent these
results can be generalized to other settings and, in particular, to
real life settings. Small and Nusbaum [78] have criticized the
unnatural MRI scanner setting and suggested an “ecological func-
tional brain imaging approach” that includes monitoring of natural
behaviors using a multimodal assessment and environmental con-
text of presentation or behavior. Nevertheless, in contrast to the
scanner, emotion in real settings is not restricted to simple reac-
tions but includes the full range of regulatory actions. By correlat-
ing fMRI and field data, such as, for example, generated by emotion
monitoring during everyday life [79], the “ecological validity,”
that is, the predictive value of cerebral perfusion patterns for real-­
life emotions could be better evaluated.

3 fMRI Methods

3.1 Methodological A host of fMRI studies have identified the amygdalae as central
Challenges structures in emotion processing (see Sect. 1 and Zald et al. [5], for
example, for a review). The amygdalae lie in the anterior medial
3.1.1 Introduction
temporal lobe (MTL), bounded ventrolaterally by the lateral ven-
tricles and medially by the sphenoid sinuses (Fig. 1). The differing
magnetic susceptibilities of these tissues cause large deviations in
the static magnetic field, B0. There is also a strong gradient in B0 in
the MTL, and differing precession frequencies lead to dephasing of

Fig. 1 The amygdalae, central brain structures in emotion processing, lie in a region of moderate deviation
from the static magnetic field (left) and very high static magnetic field gradients (right). The planes intersect in
the amygdala at MNI coordinate (18, −2, −18), marked by arrows. Single subject measurement at 4.0 T
 fMRI of Emotion 463

the bulk magnetization and loss of signal in images. This problem

is not restricted to the amygdala, however. Inferior frontal and
orbitofrontal regions, likewise involved in emotion processing
[80], are also zones of high static magnetic field gradient. In addi-
tion to signal loss, static magnetic field gradients also lead to echo
times (TE) becoming shifted, so that BOLD sensitivity may be
reduced, or signal may not be acquired at all (termed “Type 2” loss
[81]). These problems are examined in Sect. 3.1.2.
Local variations in the static magnetic field strength confound
spatial encoding of the MR signal, leading to image distortion.
Particular considerations for the MTL in this regard are discussed
in Sect. 3.1.3. Even at high field, deviations from B0 immediately
in the amygdala are relatively moderate (Fig. 1 left; 10 Hz mea-
sured at the arrow position, for data acquired at 4.0 T) but the
field gradient is high (2 Hz/mm at the same position), leading to
very large distortions in neighboring structures, which can cause
signal to encroach into the amygdalae.
The ventral brain is also prone to physiological artifacts of car-
diac and respiratory origin, as described in Sect. 3.1.4, which may
be mitigated to some extent by simultaneous measurement of car-
diac and respiratory processes and the application of postprocess-
ing corrections. In addition to the measurement challenges of
ventral brain imaging, the presence of large magnetic field gradi-
ents makes the ventral brain susceptible to stimulus-correlated
motion (SCM) artifacts, as discussed in Sect. 3.1.5. These can lead
to the appearance of neuronal activation (Fig. 2) arising from sub-
tle head movements which are time locked to stimuli.

10

8
Patient 13

6 Patient 8
Displacement (mm)

2
Patient 1

0 Control 3

Emo Age Emo

Instruction

0 20 40 60 80 100 120 140 160 180 200 220 240 260

Scan number

Fig. 2 Large static magnetic field gradients make the amygdala region prone to the artifactual appearance of
neuronal activation when stimulus-correlated motion (SCM) is present. Left : Observed patterns of SCM of
schizophrenic patients and controls in a 3.0-T experiment with three stimulus blocks (facial emotion and age
discrimination “EMO” and “AGE”). Right : a baseline (no stimulus) study in which a subject executed submil-
limeter SCM similar to that of Patient 1. The contrast corresponds to the “EMO” periods (uncorrected p < 0.0001;
t threshold = 5, Montreal Neurological Institute coordinates 22, −6, −16)
464 Simon Robinson et al.

Fig. 3 Signal changes in the amygdala in emotion experiments have to be measured against a background of
resting state fluctuations. A resting state network recently been reported, covering the amygdala and basal
ganglia (3.0 T, group independent component analysis of 26 young healthy adults). Adapted from [106] with
permission from the ISMRM

A further potential confound is the presence of RSNs which

colocalize with regions under study. These show slow fluctuations
in the absence of stimuli and constitute sources of unmodeled
noise and intertrial variation. The existence of a RSN in the amyg-
dalae (Fig. 3) offers a possible explanation of why small signal
changes are generally recorded in these structures, despite the high
neurovascular reactivity of deep gray matter nuclei. This and other
RSNs which may involve the amygdala are described in Sect. 3.1.6.
In Sect. 3.1 we expand on the problems outlined here, and go
on in Sect. 3.2 to detail approaches to optimizing conventional
single-shot 2D gradient-recalled echo-planar imaging (EPI) to
mitigate their effects, alternative sequences which are less sensitive
to static magnetic field gradients and, in Sect. 3.3, methods to cor-
rect for image distortion, physiological noise, and SCM artifacts.

3.1.2 Signal Loss It is worthwhile to briefly review the problem of signal loss from an
and BOLD Sensitivity Loss empirical perspective. A temporal resolution of 1–3 s is usually
desirable in fMRI. The whole brain may be covered in this time by
acquiring images with voxels of typically 3-mm size (or 27 μl).
 fMRI of Emotion 465

Relatively long TEs are employed, partly also as a technical neces-

sity—to allow time for gradient switching and echo sampling—but
also to confer T2* weighting. As well as providing sensitivity to
BOLD effects, however, this allows time for dephasing from mac-
roscopic inhomogeneities to develop. The severe signal loss seen in
EPI in the anterior MTL with typical parameters is illustrated
Fig. 4 in the lower left two images.
In gradient-echo imaging, the MR signal decays with a time
constant T2*, comprising the transverse relaxation time, T2 (reflect-
ing irreversible decay arising from time-varying microscopic spin-­
spin processes), and T2′, the reversible contribution to the transverse
decay rate and the major source of BOLD contrast. T2,′ itself can be
separated into “mesoscopic” contributions (which operate on a
scale smaller than the voxel, e.g., dephasing in the capillary bed),
and “macroscopic” contributions (meaning larger than the voxel)
which stem from bulk field inhomogeneities and which are depen-
dent on the tissues present, on the quality of shim, and on the
scanning parameters such as voxel size and slice orientation.
Separating these effects, the MR signal S in a gradient-echo experi-
ment decays such that at the TE it can be expressed [82] as:

Fig. 4 Effects of voxel size and acceleration factor on T2* and echo-planar imag-
ing (EPI) image quality at high field (4.0 T). Top: T2* in coronal and axial slices
through the amygdala at two voxel sizes. Bottom: corresponding EPI in slices
through the amygdala with acquisition voxel sizes of 4 × 4 × 4 mm, 3 × 3 × 3 mm,
2 × 2 × 2 mm, and 2 × 2 × 2 with GRAPPA acceleration of factor 2, all with echo
time (TE) = 32 ms
466 Simon Robinson et al.

æ -TE ö
S ( TE ) = S ( 0 ) ´ exp ç ÷ ´ F ( TE ) , (1)
è T2 ø
where an approximation to F(TE) for linear field variations over
voxels, ΔBi, in the x, y, and z directions is

æ ‡ Bx TE ö æ ‡ By TE ö æ ‡ Bz TE ö
F ( TE ) = sinc ç ÷ ´ sinc ç ÷ ´ sinc ç ÷ . (2)
è 2 ø è 2 ø è 2 ø

This illustrates that the signal decay rate may be reduced by decreas-
ing the voxel size—to reduce the gradients across voxels, ΔBi—or
by reducing the TE.
The aim of any attempt to optimize an EPI sequence is not just
to maximize signal, described above, but also BOLD sensitivity (BS),
which is equal to the product of image intensity and TE; for magneti-
cally homogeneous regions is a maximum when the EPI effective TE
is equal to the T2* of the target region [83]. In homogeneous regions,
however, the presence of field gradients shifts the location of signal in
k-space, mainly in the phase-encode direction (because of the low
bandwidth), changing the local TE [81]. Through-plane field gradi-
ents lead to signal loss and reduce BS. If the component of the in-
plane susceptibility gradient in the phase-­ encode direction is
antiparallel to the phase-encode gradient “blip” direction, then the
TE is also reduced, reducing BS further. Conversely, if it is parallel to
the phase-encode “blips” then TE increases. While this increases BS,
to some extent compensating for signal loss, if the shift of TE is too
large the echo will fall outside the acquisition window, leading to
complete signal dropout. This is commonly observed in the anterior
MTL for a negative-going phase-encode scheme.
This description motivates the optimization approaches to EPI
in susceptibility-affected regions which will be outlined later in this
section; compensating through-plane gradients, selecting image
orientation and gradient direction to minimize echo shifts, and
reducing voxel sizes to reduce field gradients. These techniques
will be shown to increase both signal and BS.

3.1.3 Image Distortion Accurate spatial encoding in MRI is founded upon a homogeneous
static magnetic field in the object. The location of signal is deduced
from the local field strength under the application of small orthog-
onal, linear magnetic fields in directions usually referred to as slice
select, readout, and phase-encode. The method is confounded if
there are regional variations in the static magnetic field, which lead
to signal mislocalization (distortion). Typical field offsets are illus-
trated in Fig. 1 (left) and lead to EPI distortions of the image
shown in Fig. 4.
The extent of distortion, expressed as the number of pixels by
which signal is mislocalized, is equal to the local magnetic field
 fMRI of Emotion 467

deviation divided by the bandwidth per pixel (the reciprocal of the

time between measuring adjacent points in k-space), expressed in the
same units. The bandwidth per pixel in the readout direction
(rBWread/pix) is equal to the total imaging bandwidth (the signal
sampling rate) divided by the image matrix size in the readout direc-
tion. In EPI, the pixel bandwidth in the phase-encode direction is
smaller than this again by a factor of the image matrix size in the
phase-encode direction. The fact that total bandwidth is often
increased in proportion with the readout matrix dimension in order
to keep rBWread/pix constant means that distortion (in distance
rather than number of pixels) is approximately constant as a function
of matrix size (and thereby resolution, at constant matrix size). To
illustrate the size of expected distortions, in a 64 × 64 matrix acquisi-
tion, a typical rBWread might be 1500 Hz/pixel, giving (as 1500/64)
a rBWphase of 23 Hz/pixel. A value of ΔB0 of 50 Hz (common at
high fields, see Fig. 1) would lead to a shift of 0.03 voxels in the read-
out direction, but 2 voxels in the phase-­encode direction, or 7 mm
for a typical field of view for brain imaging. In a higher resolution
acquisition with a 128 × 128 matrix and the same rBWread, rBW-
phase would be 12 Hz/pixels and the distortion 4 voxels, but also
7 mm because of the proportionately smaller voxel size.
The relationship between EPI distortion and field strength is
not simple, depending both on hardware and usage. Susceptibility-­
induced field changes increase linearly with static magnetic field
strength while gradient amplitude (the factor which limits sam-
pling rate) is approximately constant in the standard to high field
regimes. While theoretically this leads to an approximate propor-
tionality between distortion and field strength, in practice higher
acquisition bandwidths are often used at high field to the achieve
shorter effective TEs, to match reduced T2* times.
Image distortion frustrates attempts to coregister data from
many subjects to a common probabilistic atlas [84], which can
reduce significance in fMRI even in relatively homogeneous areas
[85]. Established methods for correcting image distortion are
compared for their performance in the amygdala in Sect. 3.3.1.

3.1.4 Physiological A number of physiological processes give rise to fluctuations in the

Artifacts MR signal which are unrelated to neuronal activation, and should
therefore be corrected for or modeled in a statistical analysis. The
amygdala area is particularly prone to cardiac artifacts due to the
proximity of the arteries in the Circle of Willis, and to respiratory
artifacts because of the susceptibility gradients.
Respiration leads to head motion, changes in the magnetic field
distribution in the head due to changes of gas volume or oxygen
concentration in the chest [86], and variation in the local oxyhemo-
globin concentration, probably due to flow changes in draining
veins [87]. Subtle changes in respiration rate and depth are thought
to be the origin of spontaneous changes in arterial carbon dioxide
468 Simon Robinson et al.

level at about 0.03 Hz which have been shown to lead to significant

low-frequency variations in BOLD signal [88]. The lag of 6 s in this
process corresponds to the time taken for blood to transit from the
lungs to the brain, and for cerebral blood flow volume to respond
to CO2, a cerebral vasodilator. Magnetic field changes in the head
particularly affect ventral brain imaging due high field gradients.
Respiration-related artifacts typically affect the image periphery,
making them problematic for the amygdala, which is usually at the
anterior boundary of the signal-­providing region.
Cardiac pulsatility causes expansion of the arteries, bulk motion
of the brain, and cerebrospinal fluid flow and leads to the influx of
fully relaxed spins into an imaging slice. As a consequence, the
signal may increase in many of the arteries that lie close to the
amygdala, such as the middle cerebral artery and other elements of
the Circle of Willis [89]. Cardiac artifacts are particularly complex
with regard to emotion studies as the amygdala innervates the
autonomic nervous system via the hypothalamus and brainstem,
increasing heart rate, as has been shown in fMRI [90], and human
depth electrode studies [91]. Recently, fluctuations in cardiac rate
have been shown to explain almost as much variation in the BOLD
signal as the oscillations related to each cardiac cycle, as revealed by
shifted cardiac rate regressors [92].
Cardiac and respiratory cycles are connected by a number of
processes [93], leading to many regions showing BOLD fluctua-
tions of cardiac origin [92] being also observed in studies of respi-
ratory effects [94].
Cardiac and respiratory artifacts may be corrected for by a
number of approaches, some of which require additional measure-
ments at the time of imaging. The effectiveness of these techniques
in the ventral brain is outlined in Sect. 3.3.2.

3.1.5 Motion Artifacts Motion artifacts affect all regions of the brain, but are particularly
problematic in emotion studies because the nature of the task
material is prone to induce SCM as a startle, attention, or repulse
response. Patients with disorders with emotional components
(such as schizophrenia and posttraumatic stress disorder) are less
likely to remain still throughout the experiment and the interac-
tion between motion and distortion in regions of high susceptibil-
ity gradient produces nonlinear pixel shifts that are not well
corrected with rigid-body methods. Partial brain coverage proto-
cols, such as those that may be used to allow z-shimming or high
spatial and temporal resolution fMRI in the amygdala, are also
more prone to partial voluming in the outermost slices and spin
history effects, in which motion between the acquisition of adja-
cent slices leads to some spins being excited twice within one rep-
etition time (TR) while others are not excited at all.
Head motion can be minimized using bite bars, vacuum cush-
ions, thermoplastic masks, or plaster head casts. As well as effective
 fMRI of Emotion 469

immobilization, casts allow for repositioning in longitudinal studies

[95]. Such devices are not appropriate for emotion studies, however,
due to the added degree of discomfort and distraction they provide.
SCM was originally investigated by Hajnal et al. [96] in hybrid
simulations with quite large (3 mm) introduced pixel shifts, which
led to peripheral correlations. A study by Field et al. [97] found
that small-amplitude motion can lead to false positive results, par-
ticularly in regions of high field gradient. Likewise, larger motions
can reduce significance and lead to false negative results. Two dis-
tinct patterns of SCM are often observed in fMRI experiments. As
in the example of identified motion with sample schizophrenic
patients and controls (Fig. 2, left), patients may execute large
motions at the first presentation of a stimulus, and many patients
and controls show very small displacements which endure for
entire blocks. Reproducing the submillimeter head motions
observed in that experiment in a separate session (without stimuli),
these have been shown to lead to highly significant correlations in
the amygdala which are difficult to distinguish from genuine acti-
vation (Fig. 2, right), a problem not mitigated by standard motion
correction methods [98].

3.1.6 Colocalized Resting An additional methodological confound comes in the form of

State Networks RSNs, which constitute additional sources of signal fluctuations
unrelated to experimental task. In the absence of tasks or stimuli,
the brain undergoes slow (0.01–0.1 Hz) fluctuations in function-
ally related networks of brain regions [99, 100]. These endure dur-
ing task execution, and have been shown to account not only for
much of the intertrial variation in the BOLD response in evoked
brain response [101], but also to the intertrial variability in behav-
ior [102]. Approximately ten such RSNs have been discovered
over the past decade [99, 100, 103–105] in networks relating to
sensory or cognitive function. A network with similar low-­
frequency characteristics has recently been identified in the amyg-
dala and basal ganglia [106].
The network illustrated in Fig. 3 shows the results from a group
of independent component analysis (ICA), performed with
MELODIC [107], of resting state data acquired from 26 subjects. It
is continuous, fully incorporating symmetrically the striate nuclei
(pallidum, puitamen, and caudate nuclei), extending inferiorly to the
amygdaloid complexes. The network is weaker than those previously
reported (measured by the amount of variance it explains in the data),
but is reproducible across subgroups of subjects, runs, and resting
state conditions (fixation and eyes closed) and offers a tantalizing
explanation as to why, despite the fact that neurovascular reactivity is
high in deep gray nuclei, BOLD signal changes are weaker and less
consistent in the amygdalae and basal ganglia than in the cortex.
This may not be the only RSN in which the amygdala is
involved. Correlations were observed between the amygdalae, and
470 Simon Robinson et al.

between the amygdalae and hippocampi and anterior temporal

lobes in one of the earliest resting state analyses, using functional
connectivity [100]. The amygdala was also listed as an element in
the “default mode” network [108], when originally reported as
regions showing deactivations across a number of tasks in PET
[109]. The fact that the amygdala has not been observed as part of
this network in this context may relate to the technical challenges
of measurement discussed in this chapter.

3.2 MR Methods, While the signal to noise ratio (SNR), the magnitude of BOLD
Sequences, signal changes, and the specificity of the BOLD response to micro-
and Protocols vascular contributions all increase with field strength, so do physi-
ological noise, field inhomogeneities, and physiological artifacts
3.2.1 Field Strength
which specifically affect the anterior MTL. The advantages of high
field for emotion studies are therefore restricted to particular
regimes and methods in which these problems are minimized.
Human emotion fMRI studies have been carried out at field
strengths from 1.0 to 7.0 T. In line with the development of
sequences and approaches to EPI in susceptibility-affected area
which are discussed in Sect. 3.2.2–3.2.8 (high-resolution single
and multishot EPI, multiecho and spiral acquisitions, gradient
compensation, and parallel imaging), emotion fMRI in the high
field regime (3.0–4.0 T) has become commonplace, although
applied studies have generally used standard sequences and param-
eters despite the problems which have received attention in the
MR literature [110] and a number of promising remedies (see the
following sections). Ultra-high field strength studies of emotion
are still sparse, however, and it is likely that they will be restricted
to highly specific questions during the next 5–10 years of hardware
and sequence development.
Theoretical gains in SNR at high field are limited by physio-
logical noise, which increases both with field strength and voxel
size, and causes time-series SNR (tSNR) to reach as asymptotic
limit with voxel volume [111]. This limit was found to increase
only modestly with field strength, being 65 at 1.5 T, 75 at 3 T,
and 90 at 7 T, so that for large (5 × 5 × 3 mm) voxels, tSNR was
only 11 % higher at 3 T than at 1.5 T, and only 25 % higher at 7 T
than 1.5 T. The tendency toward asymptotic behavior began at
relatively small volume volumes, with 80 % of the asymptotic
maximum being reached at 28.6, 15.0, and 11.7 mm3 at 1.5, 3,
and 7 T, respectively. For small voxels, however, where thermal
noise dominates, tSNR gains were almost linear with field
strength. In the same study, the authors found that with
1.5 × 1.5 × 3 mm3 voxels, tSNR increased by 110 % at 3 T com-
pared to 1.5 T, and by 245 % at 7 T compared to 1.5 T [111].
This study clearly shows that tSNR gains are to be made at high
field in the small voxel volume regime.
 fMRI of Emotion 471

These tSNR results also explain the often modest gains achieved
in fMRI studies at higher field, particularly in regions affected by
signal dropout. Krasnow et al. [112] compared activation in
response to perceptual, cognitive, and affective tasks at 1.5 and 3 T
with a relatively large voxel protocol (3 × 3 × 4 mm) and observed
only moderate increases in activated volume at 3 T for the percep-
tual and cognitive tasks (23 and 36 %, respectively), but no signifi-
cant improvement in the activated amygdala volume due to
increased susceptibility-related signal loss. A high-resolution, high-­
field approach has been exemplified in the only human study of
amygdala function at 7 T to date of which we are aware, which was
carried out at submillimeter resolution [113].
These studies define the regime in which field strength gains
are to be made, but it is fair to ask why one should move to high-­
resolution measurements if the neuroscience question does not
require, for instance, subnuclei of the amygdala to be resolved,
but—as is more commonly the case—the study of interactions
between the amygdalae and the cortex, for which whole brain cov-
erage is essential. The use of high resolution here is not principally
to distinguish activation in small structures, but to reduce both
physiological noise and susceptibility artifacts. A number of works
have shown the value of averaging thin slices, downsampling, and
smoothing data acquired at high resolution [114–116] and using
multichannel coils [115] to regain losses in SNR inherent to small
voxels generally and yielding net gains in susceptibility affected
areas [115, 117].

3.2.2 z-Shimming, The effect of signal dephasing arising from through-plane gradi-
Gradient Compensation, ents may be reduced by creating a composite image from a number
Tailored RF Pulses of acquisitions in which different slice-select gradients are applied
[118], a process known as z-shimming. In each image the applied
gradient pulse is appropriate to counteract susceptibility gradients
in particular regions. The method is effective in regaining signal in
the anterior MTL, but clearly reduces temporal resolution by a fac-
tor equal to the number of images acquired, usually a minimum of
3. Alternatively, a single, moderate preparation pulse may be used.
This reduces through-plane dephasing in affected areas at limited
cost to BS and signal in homogeneous areas, and allows slices to be
orientated so that TE shifts are small, reducing signal loss due to
in-plane gradients [119]. z-Shimming and other compensation
schemes have been applied in a number of other sequences
described in this section.
Spins may also be refocused using tailored radio frequency
pulses which create uniform in-plane phase but quadratic phase
variation through the slice, allowing dephasing to be “precompen-
sated” [120]. Analogous to z-shimming, in the original implemen-
tation a number of acquisitions with different precompensations
were required, suited to different regions. More recently 3D
472 Simon Robinson et al.

versions have been developed, and while these are promising the
pulse lengths are long, and the distribution of susceptibilities must
be known [121], or calculated iteratively online [122]. These are,
however, important steps toward single-shot compensation of sus-
ceptibility dropout.

3.2.3 Slice Orientation Divergent findings and recommendations for the optimum slice
and Gradient Directions orientation for amygdala fMRI are due to the absence, until rela-
tively recently, of an adequate description of signal loss and BS in
the presence of field gradients [81, 119, 123].
In many early studies, quite nonisotropic voxels were used to
achieve short TR while minimizing demands on scanner hardware,
with slice thickness being substantially larger than the in-plane
voxel size. Gradients across voxels were highest then, and signal
loss most severe, if the direction of strongest field gradient was
along the slice (through-plane) direction [124]. With many studies
finding that the direction of the field vector across the amygdala
was principally superior-inferior [125], this prescription precluded
an axial orientation. As bilateral structures, the amygdala could be
imaged in the same slice in the coronal but not the sagittal planes,
leading to the coronal orientation being preferred by many [110].
The optimum imaging plane is also dependent on whether
gradient compensation is used [81]. If so, through-plane gradients
may be compensated for with a moderate gradient in the slice
direction, although this will lead to a small decrease in BS in unaf-
fected areas. The slice can then be orientated so that in-plane
­gradients are below the critical threshold for Type 2 signal loss.
The value of this has been demonstrated in the orbitofrontal cortex
[119] but the approach yields lower rewards in the amygdala
region [126] as gradients are higher (making it more difficult to
find a suitable value for compensation), and are more variable
between subjects.
The simulations of Chen et al. [125] for the amygdala sug-
gested that the maximum BS was to be achieved by orienting the
slice direction perpendicular to the maximum gradient vector and
the readout direction parallel to it, indicating an (oblique) coronal
orientation with superior–inferior readout. The angle between the
gradient vector and the superior–inferior direction was shown to
vary widely between subjects (from −7° to +26° at 1.5 T, from −5°
to +34° at 3 T), meaning that field gradients need to be mapped
for each subject before measurement. This scheme also invokes
distortions which are asymmetric about the midline (left–right). If
erroneous conclusions about lateralization are to be avoided, resid-
ual distortions in the amygdala should be symmetric, requiring the
phase–encode direction to be superior–inferior for coronal slices or
anterior–posterior for axial slices.
As well as the direction of imaging gradients, the sign of phase-­
encode blips is important for signal loss and BS [123]. Encoding in
 fMRI of Emotion 473

EPI can be either with a large positive phase-encode “prewinder”

followed by a succession of small negative “blips,” or a negative
prewinder followed by positive blips. In homogeneous fields these
schemes are equivalent, but we have seen that in the presence of
susceptibility gradients echo positions are shifted away from the cen-
ter of k-space, along the phase-encode axis. Positive and negative
blip schemes have quite different properties, therefore, depending
on whether the component of susceptibility gradient in the phase-
encode direction is itself positive or negative [123]. The phase-
encode direction (PE), slice angle, and z-shimming prepulse gradient
moments (PP) that lead to maximum BS for EPI with otherwise
standard EPI parameters (TE = 50/30 ms at 1.5 T/3 T,
3 × 3 × 2 mm3 voxels) have been measured throughout the brain by
Weiskopf et al. at 1.5 and 3 T [126]. They define positive slice angles
as being those in which, beginning from the axial plane, the anterior
edge is tilted toward the feet, and a positive PE as being that in
which the prewinder gradient points from the posterior to the ante-
rior of the brain. In the amygdala they find that the highest BS is
achieved with positive PE, a −45° slice tilt and a PP = +0.6 mT/m ms
at 3 T, and positive PE, −45° slice tilt and PP = 0.0 mT/m ms at
1.5 T. These values led to a 14 % increase in BS at 3 T over a standard
acquisition (with positive PE, a −0° slice tilt and a PP = −0.4 mT/m ms)
but only 5 % at 1.5 T. This indicates that BS can be increased by
selecting optimum geometry parameters and compensations gradi-
ents, although improvement is more modest than that which has
been demonstrated with the more technically challenging or time-
consuming strategies described in this chapter. The gradient and
geometry values suggested in Weiskopf et al. [126] should be
adopted for EPI with standard parameters at these field strengths. At
other field strengths their analysis could be followed, or interpolated
values adopted from the trends evident in that study.

3.2.4 Voxel Size Among many solutions to the problem of signal loss in the anterior
MTL, reduced voxel size was established very early as an effective
means of mitigating susceptibility-related signal loss [127, 128].
Equation (2) describes how the rate of signal decay is reduced with
voxel size by lowering field gradients across voxels. The effective-
ness of this can be seen in the 4-T images of Fig. 4 over a range of
resolutions, with T2* in the amygdala (measured with a multiple
gradient-echo sequence with the same geometry as the EPI)
increasing from 22 to 38 ms when the voxel size is reduced from
64 to 8 mm3, with corresponding EPI signal increase apparent in
the anterior MTL.
Reducing voxel size comes at the expense of temporal resolu-
tion (or brain coverage) and SNR. The relationship between image
SNR and voxel volume, ΔV, is
474 Simon Robinson et al.

Nx N y Nz
SNR = V , (3)
rBW

where Ni is the number of samples in direction i and rBW the

receiver bandwidth [129]. The commonly held view that voxel
volume is simply proportional to SNR is premised on changing
the volume via the field of view [130], or that, in addition to
increasing Nx and Ny by a factor f, (considering only in-plane
resolution) receiver bandwidth is also increased by the same fac-
tor. If receiver bandwidth and field of view are held constant,
however, then we see from Eq. (3) (because N x N y N z = k / V |FOV ,
where k is the total imaged volume) that SNR is proportional to
the square root of the voxel volume, and SNR may be restored by
downsampling high-resolution images. In this time-consuming
scheme, partial k-space acquisition may be used to achieve the
desired TE, SNR can be increased with multichannel coils, as has
been validated for the MTL [115] and parallel imaging used to
reduce an otherwise long TR.
While this analysis provides the basis for the dependence of
image signal on imaging parameters, it neglects the effects of phys-
iological noise. The most important measure of signal in this con-
text is tSNR, which translates into the feasibility of detecting a
specified signal change in fMRI [131] and has been shown to be
useful in assessing the viability of amygdala fMRI in individual
­subjects [132]. In a study of optimum parameters for GE-EPI for
3-T amygdala EPI with a volume coil, a protocol with approxi-
mately 2-mm isotropic voxels was found to yield 60 % higher tSNR
than a protocol with standard parameters (with approximately
4-mm isotropic voxels) [117], despite having been measured at
twice the receiver bandwidth. Additional gains with smaller voxels
(thinner slices) were not large, because T2* had already increased to
a value close to that in homogeneous regions. This is in concor-
dance with models calculations which suggest that 2 mm repre-
sents the smallest voxel size that should be used for amygdala
imaging providing the activated size is itself at least 2 mm [125].
There are many differences between the conditions and met-
rics of the methodological work cited and typical fMRI studies. It
is encouraging, therefore, that these findings have been confirmed
in the significance and extent of amygdala activation in fMRI
experiments [133, 134].
In summary, small voxels should be used in high field strength
studies in order to operate in a regime dominated by technical,
rather than physiological noise. In inhomogeneous regions this
results in reduced field gradients, reducing signal loss and echo
shifts, making BS more uniform in the volume. Time-series SNR
may be increased by using multichannel coils and downsampling
small voxels.
 fMRI of Emotion 475

3.2.5 Echo Time Taking the simplest approach of matching effective echo time
(TEeff) to the T2* of the structures of interest in GE-EPI might be
seen as being problematic in large voxel size acquisitions, with T2*
s varying quite widely (e.g., between the amygdala and the fusi-
form face area). One solution is to use a multiecho sequence, in
which the each time of each image is appropriate for regions with
particular field gradients, as will be described in more detail in
Sect. 3.2.8. A novel solution to matching TEeff to T2* in the amyg-
dala without sacrificing BS in more dorsal slices is to use an axial
acquisition with slice-specific TE, demonstrated at 1.5 T with
TEeff = 60 ms in dorsal slices, TEeff = 40 ms in ventral slices, and a
transition zone with intermediate effective TE [135].
It should be remembered, though, that the maximum of BS is
quite flat as a function of TE, and TE is itself not well defined in
EPI. In the previous sections, we also saw that in-plane susceptibil-
ity gradients change local TE [81]. This exposes the limitation of
the approach of simply reducing the TEeff of the sequence. In the
common, negative blip scheme, signal in the anterior MTL will in
fact be shifted to a longer TE. Using a short TEeff makes the
sequence more prone to complete (type 2) signal loss.
This explains the experimental findings of Gorno-Tempini
et al. [136] and Morawetz et al. [134]. In 2-T dual-echo EPI with
large voxels, Gorno-Tempini et al. found that although signal loss
was reduced at the short TE (26 ms) BOLD activation was signifi-
cantly greater in the hippocampus at the longer TE (40 ms).
Morawetz et al. [134] studied four EPI protocols in their efficacy
at mapping amygdala activation, using variants with two different
TE (27 and 36 ms) and slices thicknesses (2 and 4 mm), all with
high in-plane resolution (2 mm). Activation results were poor in
the 4-mm protocols, even at the shorter TE.
A more effective approach than reducing TEeff is to reduce
susceptibility gradients, and thereby signal dephasing and echo
shifts, using the techniques described earlier; gradient compensa-
tion, selection of appropriate gradient direction and slice orienta-
tion, and the use of smaller voxels. This increases T2* in
susceptibility-affected regions and, by reducing echo shifts, makes
BS more homogeneous throughout the imaging volume.
Conditions then approach those with a homogeneous static field,
where BS is maximized by using TE eff = T2* .
The increase in T2* in the amygdala with reduced voxel size is
illustrated at 4 T in Fig. 4; from 22 ms in a 4 × 4 × 4-mm acquisition
to 38 ms in 2 × 2 × 2-mm data, consistent with previous results at
3 T [117]. Likewise, increase in BS was illustrated in the Morawetz
et al. study [134], in which robust amygdala activation was only
detectable in the high-resolution acquisition.

3.2.6 Parallel Imaging The previous sections have shown that many of the techniques
which mitigate susceptibility-related signal loss in the amygdala,
476 Simon Robinson et al.

hypothalamus, and MTL are also time consuming, limiting either

temporal resolution or brain coverage. This is undesirable where
brain coverage cannot be reduced to the amygdala. Parallel imaging
allows acceleration by undersampling k-space and using the sensi-
tivity profiles of a number of receiver channel to reconstruct data
without image fold-over [137, 138]. By this means it is possible to
reduce TEeff, which reduces susceptibility loss, and to reduce TR by
the acceleration factor. Image distortions and echo shifts are like-
wise reduced by the acceleration factor so that even at the same
effective TE as in a conventional acquisition, signal loss in the
amygdala region is lower (Fig. 4, bottom right). The noise proper-
ties of images reconstructed from parallel acquisition lead to BS
reductions of the order of 15–20 % in other regions, however [139].
The effectiveness of parallel imaging and suitable acceleration fac-
tors for the MTL have been studied by Schmidt et al. [140]. Statistical
power in the study of MTL activation was higher in the parallel-acqui-
sition data with an acceleration factor of 2 than in the acquisition with-
out acceleration, but neither image quality nor statistical power
improved with higher acceleration factors, as noise and reconstruction
artifacts reduced tSNR prohibitively. Particular gains in BS can be
made in the MTL using parallel imaging with a modest acceleration
factor combined with high-resolution imaging [115]. Combining
parallel imaging, high-resolution and high field has even allowed dif-
ferential response of the hypothalamus to be recorded in response to
funny as opposed to neutral stimuli at 3 T [141, 142], which could
potentially be used to diagnose narcolepsy and cataplexy.

3.2.7 Flip Angle The following is a consideration which is common to fMRI studies in
all brain regions. The flip angle that should be used in a sequence is
that which maximizes the signal with a particular experimental TR. In
a spoiled gradient-echo sequence this is the Ernst angle, θE, given by
æ TR
ö
q E = arccos ç e T1
÷.
ç ÷
è ø

T1 values can be taken from the literature, if available, or mapped in a

single study of a representative group of subjects, mostly simply using
an inversion recovery sequence and a range of inversion times. At
high (3.0–4.0 T) and very high field (7.0 T or higher), dielectric
effects lead to B1 inhomogeneity, and flip angles achieved deviate
from nominal values. Particularly at 7.0 T it is worthwhile to map the
RF field [e.g., using the 180° signal null point using a simple spoiled
gradient-echo sequence [143] to calibrate nominal flip angles].

3.2.8 Alternatives to 2D, If multiple echo images are acquired following a single excitation, the
Single-Shot, Gradient-Echo range of TEeff in these provides near-optimum BS for a number of
EPI regions [144, 145]. Images acquired at different TEs may be analyzed
separately, or combined to maximize BOLD contrast-to-­noise ratio
[145]. Acquiring multiple images in a single shot also allows
 fMRI of Emotion 477

additional features to be built into the sequence, such as 3D gradient

compensation, in which different combinations of compensation gra-
dients are applied to each echo [146], leading to excellent signal
recovery in the amygdala in the combined image [147]. Alternatively,
the phase-encoding gradient polarity may be reversed to yield images
with distortions in opposite directions, allowing for their correction
[148].
Similar multiecho and compensation techniques have been
applied to spiral acquisitions. A spiral-in trajectory has been shown
to reduce signal loss compared to a conventional spiral–out scheme
with the same TE, and SNR and BS could be increased with a spiral
in–out scheme by combining images optimally from the two acqui-
sitions [149]. A number of variants of this have been developed to
further reduce susceptibility artifacts, including applying a z-shim
gradient to the second echo [150] or subject-dependent slice-­
specific z-shims to both echoes [151].
A number of segmented methods are being developed to over-
come the temporal constraints of multiecho and high-resolution
acquisitions. In conventional segmented EPI, subsets of inter-
leaved k-space lines are acquired after successive excitations. The
higher phase-encode bandwidth leads to reduced distortions and
smaller echo shifts, but the method is inherently slow and prone to
motion and physiological fluctuations, as each image is built up
over a number of TRs. In the MESBAC sequence, navigator echoes
are acquired in both the readout and phase-encode directions
between each segment. Multiple echoes are acquired with different
amounts of compensation for each echo [152], and combined to
give impressive signal in inferior frontal areas.

3.2.9 Summary In the subsections of Sect. 3.2 we have looked at the influence of
field strength, gradient compensation, slice orientation, voxel size,
TE, and acquisition acceleration factor on susceptibility-related
signal and BS reduction in the anterior MTL, as well as discussing
some variants of multiecho and spiral schemes which have been
tailored for this region. While the interdependent nature of EPI
parameters and changing considerations at different field strength
necessarily make some considerations complex, we would like to
pick out two lines of approach presented here as being particularly
effective, and clarify recommendations.
The first approach is high-field, high-resolution single-shot
EPI with gradient compensation and acceleration. BOLD signal
changes are greater at high field (3.0–4.0 T), and the tSNR advan-
tages of high field strength are capitalized upon by measuring with
small (circa 8-μl voxels), where thermal noise rather than physio-
logical noise dominates. Measuring with small voxels reduces sig-
nal dephasing, making T2* more homogeneous. Shifts in local TE
are also less, reducing Type 2 signal loss and increasing BOLD
sensitivity. Moderate slice select gradient compensation and an
oblique axial acquisition with a tilt between 20 and 45° (anterior
478 Simon Robinson et al.

slice edge toward the head) reduces in-plane gradients and echo
shifts further. With susceptibility gradients reduced—evidenced by
T2* values close to those in magnetically homogeneous regions—
BS can be maximized by setting the TE eff = T2* . The TEeff can be
reached using parallel imaging acceleration (e.g., factor 2), which
further reduces both TE shifts and image distortion. Images
acquired with these parameters have high signal in the anterior
MTL, low distortion, and quite homogenous BS. Time-series SNR
can be increased before statistical analysis by downsampling or
smoothing images. This approach is attractive in that it may be
achieved on most modern high field systems.
Not only the value of gradient compensation was discussed in
Sect. 3.2.2, but also the high cost in temporal resolution, if images
with a number of compensation gradients are acquired. The s­ econd
approach we wish to highlight involves the application of a range
of compensation gradients to each of a number of echoes acquired
after a single excitation, so reducing the time penalty. Both the
multiecho echo-planar [146] and multiecho spiral acquisitions
[151] described in Sect. 3.2.8 have been shown to be effective in
reducing susceptibility-related signal loss in the anterior MTL.

3.3 Correction The field map (FM) method was first described by Weisskoff and
Methods Davis [153] and developed by Jezzard and Balaban [154]. In
Sect. 3.1.2 we saw that distortion in EPI is only significant in the
3.3.1 Distortion
phase-encode direction and that the number of pixels by which
Correction with the Field
signal is mislocated is equal to the local field offset divided by the
Map and Point-Spread
bandwidth per pixel in the phase-encode direction. In the fieldmap
Function Methods
method, static magnetic field deviations, ΔB, are calculated from
the phase difference, Δϕ, between two scans with TE separated by
ΔTE (or a dual-echo scan), using the relation DB = 2pg TEDj .
This map is distorted (forward-warped) to provide a map of the
voxel shifts required to reverse the distortion at each EPI location.
Gaps in the corrected image are filled by interpolation.
While undemanding from the sequence perspective, considerable
postprocessing is required to produce FMs that do not contain errors.
Phase imaging is only capable of encoding phase values in a 2π range,
with values outside this range being aliased, causing “wraps” in the
image. These can be removed in the spatial domain using a number of
freely available algorithms (e.g., PRELUDE [155] or ΠUN [156]),
or by examining voxel-wise phase evolution in time if three or more
echoes are acquired [157]. If imaging is being carried out with a mul-
tichannel radiofrequency receive coil, phase images created via the
sum-of-squares reconstruction [158] will show nonphysical disconti-
nuities from arbitrary phase offsets between the coil channels (incon-
gruent wraps) unless these offsets are removed [159, 160].
Alternatively, images from channels may be processed separately and
individual FMs, weighted by coil sensitivities, combined. In 2D spatial
unwrapping, additional global, erroneous 2π phase changes are occa-
sionally inferred between TE when the algorithm begins to unwrap
 fMRI of Emotion 479

from different sides of a phase wrap at the two TE. In multichannel

imaging, these slice phase shifts may be identified by examining the
consistency between coil channels [161], as may unreliable voxels at
the image edge and in regions of high-field gradient. The FM may
finally need to be smoothed to remove high frequency features and
dilated to ensure that it extends to the periphery of the brain.
In the point spread function (PSF) approach [162] applied to
distortion correction [163], the imaging sequence is similar to
EPI, but with the initial phase prewinder gradient replaced by a
phase gradient table, the values are applied in a loop. The PSF of
each voxel is the Fourier transform of the acquired data, and the
displacement of the voxel is the shift of the center of the PSF (e.g.,
if the center of this is at zero additional phase, this corresponds to
no local field offset). For one major scanner manufacturer, this
method has been robustly implemented with the flexibility to be
used for parallel imaging with high acceleration factors [164].
The FM and PSF methods have been compared at 1.5 T [163].
The PSF was found to be generally superior, although some con-
clusions were based on deficiencies in FMs in regions of high field
gradient which may be improved upon.
The effectiveness of the two methods in correcting larger
distortions at 4.0 T is shown in Fig. 5, focusing on a section

Fig. 5 Distortion correction of echo-planar imaging (EPI) at high field (4.0 T). A comparison of field-map (col-
umn 3) and point-spread function (column 4) correction of distortion in EPI (column 2) at the level of the
amygdala (top row) compared to a more dorsal section (bottom row). Salient features have been copied from
a gradient-echo geometric reference scan (column 1)
480 Simon Robinson et al.

through the amygdala (top row), and comparing this with the
situation in a more dorsal slice (bottom row). Raw and corrected
EPIs are compared to a gradient-echo reference which has the
same (subvoxel) distortion in the readout direction, but no dis-
tortion in the phase-­ encode direction. The distortion at the
anterior boundary of the amygdala (A) is circa 3 mm—moderate
compared to the displacement of the ventricles (9 mm at B) and
the frontal gray-white matter border indicated at C (12 mm). If
the multiplicity of phase information available from multichan-
nel coils is used in the FM method [161], both FM and PSF
methods perform very well in all areas, with only minor errors at
the periphery of the FM-corrected images due to residual field
map inaccuracies at those locations (at D, not present in the
PSF-corrected images).
The choice of correction method is often a pragmatic one
based on which is more robustly and conveniently implemented.

3.3.2 Correction Physiological fluctuation in a sequence of gradient-echo images can

of Physiological Artifacts be corrected using a navigator echo technique [165]. A single echo
is acquired before the encoding scheme is begun and used to amend
the phase changes in the image data which arise from susceptibility
effects. This “global” correction approach, using the central k-space
point only, can be extended to 1D [166] and 2D [167]. These
methods are effective, but have as drawbacks an increase in TR.
To avoid them being aliased in EPI time series, respiratory
fluctuations (circa 0.2–0.3 Hz) and cardiac fluctuations (circa
1 Hz) would need to be sampled at least at 2 Hz. That is, the TR
of the sequence would need to be 500 ms or less. Typical TRs in
whole-brain fMRI are 1–4 s, and the previous sections have indi-
cated that many of the strategies that should be implemented to
improve data quality in fMRI for emotion studies lead to longer
repetition times. Respiratory and cardiac fluctuations will nor-
mally be aliased, then, and not generally into a particular fre-
quency band [168]. Simple band-pass filtering is therefore not
generally ­ possible; although a range of alternative correction
methods have been developed.
A class of correction methods requires additional physiological
measurement to be made concurrent with the fMRI time-series,
using a respiration belt to monitor breathing and an electrocardio-
gram or pulse oximeter to monitor heart rate. Applied in image
space, the RETROICOR correction method involves plotting pix-
els according to their acquisition time within the respiratory cycle
(classified also by respiration depth) and subtracting a fit to fluctua-
tions over the cycle [169]. Despite the many reasons why physio-
logical artifacts are expected to particularly affect amygdala fMRI,
their correction with RETROICOR was found to bring only mod-
est improvements in group fMRI results in an emotion processing
task; up to 13 % in t statistic values depending on the degree of
 fMRI of Emotion 481

smoothing [170]. Those improvements were mostly due to cor-

rection of cardiac effects. Recent findings that cardiac rate changes
lead to signal changes of similar size to the effect due to cardiac
action itself [88], which are not modeled in the RETROICOR
approach, suggest that further gains are possible.
Modeling physiological fluctuations [171] by including mea-
sured signal as “Nuisance Variable Regressors (NVRs)” is a conve-
nient alternative to fitting and removing them. A detailed
examination of these and other sources of noise showed respiratory-­
induced noise particularly at the edge of the brain, larger veins and
ventricles, and cardiac-induced noise focused on the middle cere-
bral artery and Circle of Willis, close to the amygdala [168], which
could be well modeled.
A number of image-based methods for physiological artifact
correction have been developed, which do not require physio-
logical monitoring data. Physiological fluctuations can be mod-
eled with NVRs based on ventricular and white matter ROI
values [172]. Alternatively, the data can be decomposed using
ICA (e.g., MELODIC [173] or GIFT [174]) and components
relating to physiological processes identified with automated or
semiautomated methods. These can be based on experimental
thresholds [175], statistical testing [176], automatic threshold-
ing [177], or supervised classifiers [178]. Once identified, these
components can be removed from the data. While in their
infancy, these methods are very promising, particularly for the
ventral brain. Tohka et al., for instance, demonstrated marked
Z-score increases in frontal ventral regions and other areas close
to susceptibility artifacts.

3.3.3 Correction In patient group studies, Bullmore et al. [179] have shown the
of Stimulus-­Correlated need to compare the extent to which SCM explains variance
Motion Artifacts between the groups, and suggest that this be identified using an
analysis of covariance (ANCOVA). Without this approach, differ-
ences between the groups arising from higher SCM in the
­schizophrenic group in their study would have been attributed to
differential activation in response to the task.
In the example of Fig. 2 (left), realignment of the time series
in the motion-only replication did not substantially reduce the
amygdala SCM artifact (right), but including identified motion
parameters in the model as NVRs was effective [168, 180].
Alternatively, a boxcar NVR corresponding to presentation and
response periods can be included in the model [181]. This and a
number of other studies [182] have shown that the temporal
shift in response introduced by the hemodynamic response func-
tion (HRF) makes it possible to separate motion from activation
for short presentation periods, making event-related designs less
sensitive to motion than block designs.
482 Simon Robinson et al.

4 Summary and Discussion

Emotional neuroimaging is a rapidly expanding area that provides

an interface between neurobiological work and psychophysiologi-
cal emotion research. One important view that has emerged from
the area of behavioral neuroscience is that emotional processes play
a central role in the adaptive modulation of perceptual encoding,
learning and memory, attention, decision-making, and control of
action [9]. Many of neuroimaging studies have demonstrated that
amygdala activation, for example, modulates attention and mem-
ory storage in other brain regions such as the hippocampus, stria-
tum, and neocortex. Such interactions may occur as facilitations or
modulations of neurocognitive function at several levels of process-
ing. Conversely, recent work has shown that the organism is pre-
vented from excessive emotional activation not only by low-level
habituation or negative feedback mechanisms but also as a result of
protective inhibition processes. Diverse behavioral and cognitive
strategies have been identified that modulate and downregulate
the ongoing emotion process [6]. The modulating effects on emo-
tional arousal during an emotional episode such as rejection (vent-
ing and disengagement) or accommodation (relaxation, distraction,
reconceptualization, rationalization, or reappraisal) deserve further
inspection with respect to the involved neural mechanisms.
Although important advances have been made in the area of
human emotion perception, learning, and autonomic conditioning,
research has typically been limited to a small number of primary and
mostly negative emotions such as fear, anger, or disgust. Limiting
the range of investigated categories (neglecting shame, guilt, inter-
est, etc.), dimensions (neglecting positive emotions such as care,
support, etc.) and behavioral procedures does not do justice to the
complexity of the multistage emotional appraisal process described
above [29]. It is equally important but more d ­ ifficult to identify the
correlates of complex emotions such as those resulting from beliefs,
preferences, predispositions, or interpersonal exchange. Not only
the social dimensions such as untrustworthiness or dishonesty [183,
184], but also positive aspects such as social fairness [185], trust,
and supportiveness play a role. Moreover, an understanding of
modulating sociocultural influences is essential for a comprehensive
conceptualization of human emotion [29].
Current neuroimaging research on emotion can be described as
an ongoing construct validation process [186], which draws upon
convergent and divergent associations of local activation variables and
psychological constructs. The experimental measures (operational-
izations of psychological constructs) are expected to be correlated
with regional brain activations. It is evaluated whether topographi-
cally distinct patterns of activation in a certain region consistently
predict engagement of different processes (for an example in the area
 fMRI of Emotion 483

of cognitive processing, see Ref. [187]). Indicators of a different con-

struct are expected to correlate with activations of different areas.
This corresponds to the well-known double dissociation strategy that
inspects task by localization interactions in neuropsychology [64].
This validation process typically starts at a relatively broad con-
struct level and proceeds downward in the hierarchy of constructs
to finally specify within-systems constructs. Previous studies have
demonstrated a relatively high cross-laboratory repeatability of
emotional brain activation patterns at a higher systems level. At
lower levels, however, the reliability of psychological or activation
data may decline depending on limitations of the instruments.
Nonetheless, high-field fMRI scanners permit an improved dis-
crimination of activations, for example, within the different subnuclei
of the amygdala [5]. It is evident that increased discrimination on the
neural side must be accompanied by a refined technology to assess
more fine-grained emotional constructs on the behavioral side.
Neuropsychological construct validation requires additional
physiological data to obtain some kind of convergent information
about the indicator variable. At the neurophysiological level, the
perfusion mechanisms has been elucidated by combining the greater
spatial resolution of fMRI with the real-time resolution of intracorti-
cal local field ERP (LFP) recordings. The neurophysiological cou-
pling mechanisms of neural activity and the BOLD response can
thus be assessed [188]. An application of both fMRI methods and
electrophysiological approaches (e.g., surface and deep electrode
recordings from limbic brain structures) is useful [189]. The combi-
nation of brain perfusion changes and electrophysiological correlates
of oscillatory coupling will foster the understanding of the neural
interaction processes within frontal and temporal networks [190].
On the level of the autonomic nervous system, multivariate
coregistrations of psychophysiological response patterns including
emotion modulated startle, heart rate variability, or cortisol secre-
tion alleviate the validation of experimentally induced emotions or
presence of specific emotional disorders.
Emotional neuroimaging has continuously profited from
improvement in scanning techniques and the adaptation and stan-
dardization of signal processing strategies. However, this area has
not only benefited from the diverse contributions of its subdisci-
plines but also inherited their methodological problems. An inspec-
tion of brain imaging studies of emotion showed that measurement
quality may be influenced by many factors: by a rapid and differen-
tial habituation of responses to emotional stimuli in some regions;
by artifacts of certain signal scaling techniques that are applied by
default; by situational or state-dependent influences; and by insuf-
ficient validation of the emotion to be elicited (manipulation
check). Interindividual differences of emotional regulation behav-
ior appear to modulate event-related reactions during the time
course of the experiment.
484 Simon Robinson et al.

Some of the many approaches to reducing signal loss in EPI in

the anterior MTL have been outlined here, as well as some of the
methods for identifying and correcting artifacts arising from SCM,
distortion, and physiological artifacts. Despite the gravity of the
problem and the effectiveness of some of these strategies, the over-
whelming majority of fMRI studies of the emotions use the same
measurement protocols and analysis methods as have been applied
to study cognitive function over the last decade.
Combining many of the simpler strategies described here—high
field strength, small voxel volumes, partial k-space acquisition with the
correction of physiological and SCM artifacts—allows reliable results
to be achieved in the anterior MTL [191]. Figure 6 demonstrates
such an example; the detection of subtle differences in amygdala acti-
vation between explicit and implicit emotion processing [192].
Moreover, new research designs and analysis methods such as
Structural Equation Modeling or Dynamic Causal Modeling are
now available to inspect the effective or causal connectivity that,
for example, permits the PFC to modulate amygdala activity [62].
The influences of individual brain regions on each another can also
been studied by combining functional brain imaging with the
lesion approach or transcranial magnetic stimulation [193].
We have raised a number of caveats that highlight some of the
limitations of emotion assessment in a scanner environment. As has
been argued above, a lack of representativeness must be noted, that
is, emotion includes a much broader conceptual network than

Fig. 6 High-resolution imaging detailed in this chapter allows the acquisition of low-artifact echo-planar imag-
ing (EPI) and allows subtle processing effects to be distinguished. Group results from 29 subjects for the condi-
tions (a) emotion recognition (b) implicit emotion processing (age discrimination) and (c) the difference
between the two conditions (3.0 T). Results, showing activation in the amygdala and fusiform gyrus (as well as
cerebellum and brainstem) are overlaid on mean EPI and thresholded at p = 0.05, family-wise error corrected.
Reprinted from [192], with permission
 fMRI of Emotion 485

c­ urrently covered by neuroimaging research. Thus, generalizations

to other areas of functioning remain difficult. Representative
designs are needed that pay greater attention to high-level strate-
gies that depend on sociocultural factors and initiate, modulate, or
regulate emotions. Moreover, the representativeness of results is
limited due to small, selected, and poorly described study groups.
Finally, since emotion elicitation in the scanner has been highly
artificial, the power to predict emotions outside the neuroimaging
context remains questionable. An ecological functional brain imag-
ing approach that includes natural behaviors and environmental
contexts of presentation may help to obtain a more representative
view of real-life emotions.
Subject-specific mechanisms regulating the strength and tem-
poral pattern of response to emotional stimuli and the balance of
excitatory and inhibitory processes are of particular interest. The
variability of the BOLD response between trials and across the
time course of the experiment needs to be explained. Future
research may therefore examine individual and group differences
with a view to resolving inconsistencies in the literature [5, 12,
77]. Investigations into personality disorders or psychiatric diseases
will provide further insight into the dispositional factors modifying
the response to situational stressors. Paradigms specifically adapted
to the investigated disorder may help to identify prefrontal dys-
function and associated failure to tonically inhibit amygdala output
or to recognize safety signals eventually inducing sympathetic
overactivity [194]. It may be that—as is the case in motor tasks—a
large proportion of the intertrial variation not only in the behav-
ioral response [102], but also in the BOLD signal [101] is explained
by fluctuations in underlying RSNs.
Eliciting emotions in the environment of an imaging scanner
remains a highly artificial process. This raises the question as to the
predictive value of current neuroimaging data for explaining the emo-
tional modulations in real-life contexts. This is particularly important
for applied areas such as psychotherapy and coping research. Thus, in
addition to identifying the neurobiological basis of emotional regula-
tion behavior, the generalizability or predictive validity of imaging
data for real-life emotions should be systematically evaluated.

5 Conclusions

Neuroimaging has replicated and extended earlier findings of neu-

ropsychological studies in brain damaged subjects. It has signifi-
cantly contributed to unraveling the organization of neural systems
subserving the different components of emotional stimulus-­
response mediation along the neuraxis in healthy human subjects.
Improved operational definitions and paradigms have contributed
to differentiating subcomponents of emotional functions such as,
486 Simon Robinson et al.

for example, perceptual decoding, anticipation, associative learn-

ing, awareness, and response mediation. However, despite obvious
advances, a comprehensive model integrating the diverse emo-
tional behaviors on the basis of involved cerebral mechanisms is
still unavailable. Moreover, the interpretation of findings is compli-
cated by technical and methodological difficulties.
Research advances not only depend upon the technical
refinement of imaging methodology but also on the improve-
ment of behavioral procedures and measurement models.
Neuropsychological construct validation procedures imply that
an increase of localization precision of the imaging technology
would also require an enhanced precision on the side of behav-
ioral operationalizations. However, this seems not to be case as
many studies still use unsophisticated stimulus materials or
global instructions involving multiple or undefined subfunc-
tions. As much as relatively global operationalizations are applied,
however, the obtained neuropsychological correlations (for
example, regarding activations of the PFC) will remain
incomprehensible.
We have suggested here the framework of a lense-type assess-
ment model, wherein activations in well-characterized neural
structures may be used as predictors of particular emotional pro-
cesses. According to this, a hierarchy of latent constructs consti-
tutes the behavioral level, an idea, which is largely accepted in
psychology. On the level of brain activity, patterns or families of
topographically distinct activity can be identified in a similar way
and used as a predictor of behavioral function. Following the
assumptions of a methodological parallelism, neuropsychologi-
cal construct validation procedures make uses of this framework
of activity–behavior associations on different levels of the hierar-
chy. It can be extrapolated from multivariate personality theory,
that the prediction of behavior will only be successful if activa-
tion measures and psychological data are analyzed on a similar
level of generality or aggregation.
In view of the complexities of emotional regulation behavior in
human subjects, it is equally important to advance assessment the-
ory, psychological conceptualization, and behavioral methodology
[29]. Future work should therefore more closely inspect issues
related to model construction, symmetry of neural and behavioral
variables, and their aggregation levels. Multidisciplinary approaches
that combine improvement in brain activation measurement with
enhanced psychological data theory may thus foster construct valid-
ity, reliability, and predictive power of emotional neuroimaging.
Knowledge pertaining to the localization of brain activations
and its functional connectivity is also an important input to inform
and constrain cognitive theories of emotion psychology. Thus,
insights from the brain will thus help to explain the incoherences
of psychophysiological, behavioral, and subjective indicators of
 fMRI of Emotion 487

emotion that are so frequently observed in psychophysiological

studies. Activation data may also help to establish models that pos-
sess a better “breakdown compatibility,” that is, power to predict
behavioral change as a consequence of brain damage.
The introduction of structural/connectional and functional
data has considerably bolstered scientific construct validation
processes in the affective neurosciences and emotion psychol-
ogy. Topographically distinct activity patterns are increasingly
identified that possess a certain incremental validity, that is, an
increasing power to predict the individual dynamics of emo-
tional regulation behavior. Establishing a representative and
valid model of emotional functioning is a necessary precondi-
tion for many areas of application such as the categorization of
patients with emotional disorders and the assessment of
psychotherapy.
Greater attention to methodological issues may help to
bring more rigors to experimentation in the field of emotional
neuroimaging, promote interdisciplinary research, and alleviate
cross-­laboratory replication. A wealth of approaches have been
presented to countering BS loss in the amygdala, many of which
are available as standard on commercial scanners or simply
require the adoption of suitable imaging parameters [117, 125,
134]. Also, in the absence of a measurement theory that
describes validated procedures or instruments for assessing
emotional constructs, single findings cannot be trusted.
Although absence of validation is acceptable for early stages of
the research cycle, current emotional neuroimaging work has
only just begun to approach the confirmatory stage. To estab-
lish confidence in the suggested models, additional efforts are
required to empirically validate assessment strategies and
instrumentation.

Acknowledgments

The author’s own work reported in this chapter was supported by

the Austria FWF grant P16669-B02, grant 11437 from the Austrian
National Bank, the government of the Provincia Autonoma di
Trento, Italy, the private foundation Fondazione Cassa di Risparmio
di Trento e Rovereto, the University of Trento, Italy, and by grant
Pe 499/3–2 from the Deutsche Forschungsgemeinschaft to M.P. J.
Jovicich is thanked for helpful comments.
488 Simon Robinson et al.

References
1. Büchel C, Dolan RJ (2000) Classical fear 17. Adolphs R, Tranel D, Damasio H, Damasio
conditioning in functional neuroimaging. A (1994) Impaired recognition of emotion in
Curr Opin Neurobiol 10:219–223 facial expressions following bilateral damage
2. Phan KL, Wager T, Taylor SF, Liberzon I to the human amygdala. Nature 372:669–672
(2002) Functional neuroanatomy of emotion: 18. Broks P, Young AW, Maratos EJ et al (1998)
a meta-analysis of emotion activation studies Face processing impairments after encephali-
in PET and fMRI. NeuroImage 16:331–348 tis: amygdala damage and recognition of fear.
3. Dolan RJ, Vuilleumier P (2003) Amygdala Neuropsychologia 36:59–70
automaticity in emotional processing. Ann N 19. Sprengelmeyer R, Young AW, Schröder U
Y Acad Sci 985:348–355 et al (1999) Knowing no fear. Proc R Soc
4. Wager TD, Phan KL, Liberzon I, Taylor SF Lond B Biol Sci 266:2451–2456
(2003) Valence, gender, and lateralization 20. Adolphs R, Gosselin F, Buchanan T, Tranel
of functional brain anatomy in emotion: a D, Schyns P, Damasio A (2005) A mechanism
meta-­analysis of findings from neuroimaging. for impaired fear recognition after amygdala
NeuroImage 19:513–531 damage. Nature 433:68–72
5. Zald DH (2003) The human amygdala and 21. Adolphs R, Tranel D, Hamann S et al (1999)
the emotional evaluation of sensory stimuli. Recognition of facial emotion in nine indi-
Brain Res Brain Res Rev 41:88–123 viduals with bilateral amygdala damage.
6. Ochsner KN, Gross JJ (2005) The cognitive Neuropsychologia 37:1111–1117
control of emotion. Trends Cogn Sci 9:242–249 22. Schmolck H, Squire LR (2001) Impaired
7. Panksepp J (1998) Affective neuroscience: perception of facial emotions following bilat-
the foundations of human and animal emo- eral damage to the anterior temporal lobe.
tions. Oxford University Press, Oxford Neuropsychology 15:30–38
8. Davidson RJ, Jackson DC, Kalin NH (2000) 23. LeDoux JE (1995) Emotion: clues from the
Emotion, plasticity, context, and regulation: brain. Annu Rev Psychol 46:209–235
perspectives from affective neuroscience. 24. LaBar KS, Gatenby JC, Gore JC, LeDoux JE,
Psychol Bull 126:890–909 Phelps EA (1998) Human amygdala activa-
9. Dolan RJ (2002) Emotion, cognition, and tion during conditioned fear acquisition and
behavior. Science 298:1191–1194 extinction: a mixed-trial fMRI study. Neuron
10. Breiter HC, Rauch SL (1996) Functional 20:937–945
MRI and the study of OCD: from symptom 25. Morris JS, Büchel C, Dolan RJ (2002) Parallel
provocation to cognitive-behavioral probes neural responses in amygdala subregions and
of cortico-striatal systems and the amygdala. sensory cortex during implicit fear. Neurobiol
NeuroImage 4:127–138 12:169–177
11. Johnson PA, Hurley RA, Benkelfat C, Herpertz 26. Büchel C, Morris J, Dolan RJ, Friston KJ
SC, Taber KH (2003) Understanding emo- (1998) Brain systems mediating aversive
tion regulation in borderline personality conditioning: an event-related fMRI study.
disorder: contributions of neuroimaging. Neuron 20:947–957
J Neuropsychiatry Clin Neurosci 15:397–402 27. Krech D (1950) Dynamic systems as open neu-
12. Hamann S, Canli T (2004) Individual dif- rological systems. Psychol Rev 57:345–361
ferences in emotion processing. Curr Opin 28. Cacioppo JT, Tassinary LG, Berntson GG
Neurobiol 14:233–238 (eds) (2000) Handbook of psychophysiology.
13. Paquette V, Levesque J, Mensour B et al Cambridge University Press, Cambridge
(2003) Change the mind and you change the 29. Scherer KR, Peper M (2001) Psychological
brain: effects of cognitive-behavioral therapy theories of emotion and neuropsychological
on the neural correlates of spider phobia. research. In: Gainotti G (ed) Handbook of
Neuroimage 18:401–409 neuropsychology. Vol. 5: Emotional behav-
14. Popma A, Raine A (2006) Will future forensic ior and its disorders, 2nd edn. Elsevier,
assessment be neurobiologic? Child Adolesc Amsterdam, pp 17–48
Psychiatr Clin N Am 15:429–444 30. Coan JA, Allen JJB (eds) (2007) Handbook
15. Bräutigam S (2005) Neuroeconomics – from of emotion elicitation and assessment. Oxford
neural systems to economic behaviour. Brain University Press, New York, NY
Res Bull 67:355–360 31. Peper M, Vauth R (2008) Socio-emotional pro-
16. Walter H, Abler B, Ciaramidaro A, Erk S cessing competences: assessment and clinical
(2005) Motivating forces of human actions: application. In: Vandekerckhove M, von Scheve
neuroimaging reward and social interaction. C, Ismer S, Jung S, Kronast S (eds) Regulating
Brain Res Bull 67:368–381 emotions (ch. 9). Wiley, Hoboken, NJ
 fMRI of Emotion 489

32. Kerlinger FN, Lee HB (2000) Foundations 47. Lang P, Rice DG, Sternbach RA (1972) The
of behavioral research (4th edition). Harcourt psychophysiology of emotion. In: Greenfield
College Publishers, Fort Worth, TX NS, Sternbach RA (eds) Handbook of
33. Scherer KR (1993) Neuroscience projections psychophysiology. Holt, New York, NY,
to current debates in emotion psychology. pp 623–643
Cogn Emot 7:1–41 48. Fahrenberg J (1983) Psychophysiologische
34. Scherer KR (2000) Psychological models of Methodik. In: Groffman K-J, Michel
emotion. In: Borod J (ed) The neuropsychol- L (eds) Enzyklopädie der Psychologie:
ogy of emotion. Oxford University Press, Themenbereich B Methodologie und
Oxford, pp 137–162 Methoden, Serie II Psychologische
35. Scherer KR (1999) Appraisal theories. In: Diagnostik, Band 4 Verhaltensdiagnostik.
Dalgleish T, Power M (eds) Handbook of cogni- Hogrefe, Göttingen, pp 1–192
tion and emotion. Wiley, Chichester, pp 637–663 49. Lazarus RS (1991) Emotion and adaptation.
36. Peper M, Fahrenberg J (2008) Oxford University Press, New York, NY
Psychophysiologie. In: Sturm W, Herrmann 50. Parkinson B, Totterdell P (1999) Classifying
M, Münte TF (eds) Lehrbuch der Klinischen affect-regulation strategies. Cogn Emot
Neuropsychologie. Spektrum Akademischer 13:277–303
Verlag, Heidelberg 51. Gross JJ, Levenson RW (1991) Emotional
37. Stemmler G, Fahrenberg J (1989) suppression: physiology, self-report, and
Psychophysiological assessment: conceptual, expressive behavior. J Pers Soc Psychol
psychometric, and statistical issues. In: Turpin 64:970–986
G (ed) Handbook of clinical psychophysiol- 52. Lazarus RS, Folkman S (1984) Stress,
ogy. Wiley, Chichester, pp 71–104 appraisal, and coping. Springer, New York, NY
38. Stemmler G (1992) Differential psycho- 53. Gross JJ (1998) The emerging field of emo-
physiology: persons in situations. Springer, tion regulation: an integrative review. Rev
Heidelberg Gen Psychol 2:271–299
39. Johnsen BH, Thayer JF, Hugdahl K (1995) 54. Whalen PJ, Rauch SL, Etcoff NL, McInerney
Affective judgment of the Ekman faces: SC, Lee MB, Jenike MA (1998) Masked
a dimensional approach. J Psychophysiol presentations of emotional facial expressions
9:193–202 modulate amygdala activity without explicit
40. Peper M, Irle E (1997) The decoding of emo- knowledge. J Neurosci 18:411–418
tional concepts in patients with focal cerebral 55. Pessoa L, Japee S, Sturman D, Ungerleider
lesions. Brain Cogn 34:360–387 LG (2006) Target visibility and visual aware-
41. Ekman P (1994) Strong evidence for uni- ness modulate amygdala responses to fearful
versals in facial expressions. Psychol Bull faces. Cereb Cortex 16:366–375
115:268–287 56. McIntosh AR, Gonzalez-Lima F (1994)
42. Russell JA (1994) Is there universal recog- Network interactions among limbic cor-
nition of emotion from facial expressions? tices, basal forebrain, and cerebellum dif-
Psychol Bull 115:102–141 ferentiate a tone conditioned as a Pavlovian
43. Stemmler G (1998) Emotionen. In: F excitor or inhibitor: fluorodeoxyglucose map-
Rösler (Ed) Enzyklopädie der Psychologie: ping and covariance structural modeling.
Themenbereich C Theorie und Forschung, Serie J Neurophysiol 72:1717–1733
1 Biologische Psychologie, Band 5 Ergebnisse 57. Büchel C, Friston KJ (1997) Modulation of
und Anwendungen der Psychophysiologie. connectivity in visual pathways by attention:
Göttingen: Hogrefe, pp 95–163 cortical interactions evaluated with structural
44. Scherer KR (1984) On the nature and equation modelling. Magn Reson Imaging
function of emotion: a component process 15:763–770
approach. In: Scherer KR, Ekman P (eds) 58. Büchel C, Friston KJ (2000) Assessing interac-
Approaches to emotion. Erlbaum, Hillsdale, tions among neuronal systems using functional
NJ, pp 203–317 neuroimaging. Neural Netw 13:871–882
45. Lacey JI (1967) Somatic response patterning 59. Friston KJ, Harrison L, Penny W (2003)
and stress: some revisions of activation theory. Dynamic causal modelling. NeuroImage
In: Appley MH, Trumbull R (eds) Psychological 19:1273–1302
stress: issues in research. Appleton-Century 60. Vuilleumier P, Richardson MP, Armony JL,
Crofts, New York, NY, pp 14–42 Driver J, Dolan RJ (2004) Distant influences
46. Peper M (2000) Awareness of emotions: of amygdala lesion on visual cortical activa-
a neuropsychological perspective. Adv tion during emotional face processing. Nat
Consciousness Stud 16:245–270 Neurosci 7:1271–1278
490 Simon Robinson et al.

61. Friston KJ (1994) Functional and effective 75. Canli T, Sivers H, Whitfield SL, Gotlib IH,
connectivity in neuroimaging: a synthesis. Gabrieli JDE (2002) Amygdala response to
Hum Brain Mapp 2:56–78 happy faces as a function of extraversion.
62. Ochsner KN, Ray RD, Cooper JC et al (2004) Science 296:2191
For better or for worse: neural systems support- 76. Schienle A, Schafer A, Stark R, Walter B, Vaitl
ing the cognitive down- and up-­regulation of D (2005) Relationship between disgust sensi-
negative emotion. Neuroimage 23:483–499 tivity, trait anxiety and brain activity during
63. Fahrenberg J, Peper M (2000) disgust induction. Neuropsychobiology
Psychophysiologie. In: Sturm W, Herrmann 51:86–92
M, Wallesch CW (eds) Lehrbuch der neuro- 77. Canli T, Amin Z (2002) Neuroimaging of
psychologie, chapter 1. 10. Swets and emotion and personality: scientific evidence
Zeitlinger, Amsterdam, pp 154–68 and ethical considerations. Brain Cogn
64. Passingham RE, Stephan KE, Kötter R 50:414–431
(2002) The anatomical basis of functional 78. Small SL, Nusbaum HC (2004) On the neu-
localization in the cortex. Nat Rev Neurosci robiological investigation of language under-
3:606–616 standing in context. Brain Lang 89:300–311
65. Büchel C, Dolan RJ, Armony JL, Friston KJ 79. Fahrenberg J, Myrtek M (eds) (2001)
(1999) Amygdala-hippocampal involvement in Progress in ambulatory assessment. Hogrefe
human aversive trace conditioning revealed and Huber, Seattle, WA
through event-related functional magnetic res- 80. Adolphs R (2002) Neural systems for recog-
onance imaging. J Neurosci 19:10869–10876 nizing emotion. Curr Opin Neurobiol
66. Foerster F (1995) On the problems of initial-­ 12:169–177
value-­dependencies and measurement of 81. Deichmann R, Josephs O, Hutton C, Corfield
change. J Psychophysiol 9:324–341 DR, Turner R (2002) Compensation of
67. Peper M, Herpers M, Spreer J, Hennig J, susceptibility-­induced BOLD sensitivity losses
Zentner J (2006) Functional neuroimaging in echo-planar fMRI imaging. NeuroImage
studies of emotional learning and autonomic 15:120–135
reactions. J Physiol Paris 99:342–354 82. Yablonskiy DA (1998) Quantitation of intrin-
68. Cardinal RN, Parkinson JA, Hall J, Everitt BJ sic magnetic susceptibility-related effects in a
(2002) Emotion and motivation: the role of tissue matrix. Phantom study. Magn Reson
the amygdala, ventral striatum, and prefrontal Med 39:417–428
cortex. Neurosci Biobehav Rev 26:321–352 83. Lipschutz B, Friston KJ, Ashburner J, Turner
69. O’Doherty J, Dayan P, Schultz J, Deichmann R, Price CJ (2001) Assessing study-specific
R, Friston K, Dolan RJ (2004) Dissociable regional variations in fMRI signal.
roles of ventral and dorsal striatum in instru- Neuroimage 13:392–398
mental conditioning. Science 304:452–454 84. Toga AW, Thompson PM (2001) Maps of the
70. Grandjean D, Sander D, Pourtois G et al brain. Anat Rec 265:37–53
(2005) The voices of wrath: brain responses 85. Hutton C, Bork A, Josephs O, Deichmann R,
to angry prosody in meaningless speech. Nat Ashburner J, Turner R (2002) Image distor-
Neurosci 8:145–146 tion correction in fMRI: a quantitative evalu-
71. Critchley H, Daly E, Phillips M et al (2000) ation. NeuroImage 16:217–240
Explicit and implicit neural mechanisms for 86. Raj D, Paley DP, Anderson AW, Kennan RP,
processing of social information from facial Gore JC (2000) A model for susceptibility
expressions: a functional Magnetic Resonance artefacts from respiration in functional echo-­
Imaging study. Hum Brain Mapp 9:93–105 planar magnetic resonance imaging. Phys
72. Hariri S, Bookheiner SY, Mazziotta JC Med Biol 45:3809–3820
(2000) Modulating emotional responses: 87. Windischberger C, Langenberger H, Sycha T
effects of a neocortical network on the limbic et al (2002) On the origin of respiratory arti-
system. Neuroreport 11:43–48 facts in BOLD-EPI of the human brain.
73. Furmark T, Fischer H, Wik G, Larsson M, Magn Reson Imaging 20:575–582
Fredrikson M (1997) The amygdala and indi- 88. Wise RG, Ide K, Poulin MJ, Tracey I (2004)
vidual differences in human fear conditioning. Resting fluctuations in arterial carbon dioxide
Neuroreport 8:3957–3960 induce significant low frequency variations in
74. Canli T, Zhao Z, Desmond JE, Kang E, Gross BOLD signal. Neuroimage 21:1652–1664
J, Gabrieli JDE (2001) An fMRI study of per- 89. Dagli MS, Ingeholm JE, Haxby JV (1999)
sonality influences on brain reactivity to emo- Localization of cardiac-induced signal change
tional stimuli. Behav Neurosci 115:33–42 in fMRI. Neuroimage 9:407–415
 fMRI of Emotion 491

90. Critchley HD, Rotshtein P, Nagai Y, 103. Beckmann CF, De Luca M, Devlin JT, Smith
O’Doherty J, Mathias CJ, Dolan RJ (2005) SM (2005) Investigations into resting-state
Activity in the human brain predicting differ- connectivity using independent component
ential heart rate responses to emotional facial analysis. Philos Trans R Soc Lond B Biol Sci
expressions. Neuroimage 24:751–762 360:1001–1013
91. Frysinger RC, Harper RM (1989) Cardiac 104. Damoiseaux JS, Rombouts SA, Barkhof F
and respiratory correlations with unit dis- et al (2006) Consistent resting-state networks
charge in human amygdala and hippocampus. across healthy subjects. Proc Natl Acad Sci U
Electroencephalogr Clin Neurophysiol S A 103:13848–13853
72:463–470 105. De Luca M, Beckmann CF, De Stefano N,
92. Shmueli K, van Gelderen P, de Zwart JA et al Matthews PM, Smith SM (2006) fMRI rest-
(2007) Low-frequency fluctuations in the ing state networks define distinct modes of
cardiac rate as a source of variance in the long-distance interactions in the human brain.
resting-­state fMRI BOLD signal. Neuroimage Neuroimage 29:1359–1367
38:306–320 106. Robinson S, Soldati N, Basso G et al (2008)
93. Cohen MA, Taylor JA (2002) Short-term car- A resting state network in the basal ganglia.
diovascular oscillations in man: measuring Proc Intl Soc Magn Res Med 16:746
and modelling the physiologies. J Physiol 107. Beckmann CF, Smith SM (2005) Tensorial
542:669–683 extensions of independent component analy-
94. Birn RM, Diamond JB, Smith MA, Bandettini sis for multisubject FMRI analysis.
PA (2006) Separating respiratory-variation-­ Neuroimage 25:294–311
related fluctuations from neuronal-activity-­ 108. Raichle M, MacLeod A, Snyder A, Powers W,
related fluctuations in fMRI. Neuroimage Gusnard D, Shulman G (2001) A default
31:1536–1548 mode of brain function. Proc Natl Acad Sci U
95. Edward V, Windischberger C, Cunnington R S A 98:676–682
et al (2000) Quantification of fMRI artifact 109. Shulman G, Fiez J, Corbetta M et al (1997)
reduction by a novel plaster cast head holder. Common blood flow changes across visual
Hum Brain Mapp 11:207–213 tasks: II. Decreases in cerebral cortex. J Cogn
96. Hajnal J, Myers R, Oatridge A, Schwieso J, Neurosci 9:648–663
Young I, Bydder G (1994) Artifacts due to 110. Merboldt KD, Fransson P, Bruhn H, Frahm
stimulus correlated motion in functional J (2001) Functional MRI of the human
imaging of the brain. Magn Reson Med amygdala? Neuroimage 14:253–257
31:283–291 111. Triantafyllou C, Hoge RD, Krueger G et al
97. Field A, Yen Y, Burdette J, Elster A (2000) (2005) Comparison of physiological noise at
False cerebral activation on BOLD functional 1.5 T, 3 T and 7 T and optimization of fMRI
MR images: study of low-amplitude motion acquisition parameters. Neuroimage 26:
weakly correlated to stimulus. AJNR Am 243–250
J Neuroradiol 21:1388–1396 112. Krasnow B, Tamm L, Greicius MD et al (2003)
98. Robinson S, Moser E (2004) Positive results Comparison of fMRI activation at 3 and 1.5 T
in amygdala fMRI: Emotion or head motion? during perceptual, cognitive, and affective pro-
NeuroImage 22:S47, WE 294 cessing. Neuroimage 18:813–826
99. Biswal B, Yetkin FZ, Haughton VM, Hyde JS 113. Dickerson BC, Wright CI, Miller S et al
(1995) Functional connectivity in the motor (2006) Ultrahigh-field differentiation of
cortex of resting human brain using echo-­ medial temporal lobe function: sub-­millimeter
planar MRI. Magn Reson Med 34:537–541 fMRI of amygdala and hippocampal activa-
100. Lowe MJ, Mock BJ, Sorenson JA (1998) tion at 7 Tesla. NeuroImage 31:S154
Functional connectivity in single and mul- 114. Merboldt KD, Finsterbusch J, Frahm J (2000)
tislice echoplanar imaging using resting-state Reducing inhomogeneity artifacts in func-
fluctuations. Neuroimage 7:119–132 tional MRI of human brain activation-thin
101. Fox MD, Snyder AZ, Zacks JM, Raichle ME sections vs gradient compensation. J Magn
(2006) Coherent spontaneous activity accounts Reson 145:184–191
for trial-to-trial variability in human evoked 115. Bellgowan PS, Bandettini PA, van Gelderen
brain responses. Nat Neurosci 9:23–25 P, Martin A, Bodurka J (2006) Improved
102. Fox MD, Snyder AZ, Vincent JL, Raichle ME BOLD detection in the medial temporal
(2007) Intrinsic fluctuations within cortical region using parallel imaging and voxel
systems account for intertrial variability in volume reduction. Neuroimage 29:
human behavior. Neuron 56:171–184 1244–1251
492 Simon Robinson et al.

116. Triantafyllou C, Hoge RD, Wald LL (2006) 129. Haacke E, Brown R, Thompson M,
Effect of spatial smoothing on physiological Venkatesan R (1999) Magnetic resonance
noise in high-resolution fMRI. Neuroimage imaging: physical principles and sequence
32:551–557 design. Wiley-Liss, New York, NY
117. Robinson S, Windischberger C, Rauscher A, 130. Scouten A, Papademetris X, Constable RT
Moser E (2004) Optimized 3 T EPI of the (2006) Spatial resolution, signal-to-noise
amygdalae. NeuroImage 22:203–210 ratio, and smoothing in multi-subject func-
118. Frahm J, Merboldt KD, Hänicke W (1988) tional MRI studies. Neuroimage
Direct FLASH MR imaging of magnetic field 30:787–793
inhomogeneities by gradient compensation. 131. Parrish T, Gitelman D, LaBar K, Mesulam M
Magn Reson Med 6:474–480 (2000) Impact of signal-to-noise on func-
119. Deichmann R, Gottfried JA, Hutton C, tional MRI. Magn Reson Med 44:925–932
Turner R (2003) Optimized EPI for fMRI 132. LaBar K, Gitelman D, Mesulam M, Parrish T
studies of the orbitofrontal cortex. (2001) Impact of signal-to-noise on func-
Neuroimage 19:430–441 tional MRI of the human amygdala.
120. Cho Z, Ro Y (1992) Reduction of suscepti- Neuroreport 12:3461–3464
bility artifact in gradient-echo imaging. Magn 133. Robinson S, Hoheisel B, Windischberger C,
Reson Med 23:193–200 Habel U, Lanzenberger R, Moser E (2005)
121. Stenger VA, Boada FE, Noll DC (2000) FMRI of the emotions, towards an improved
Three-dimensional tailored RF pulses for the understanding of amygdala function. Curr
reduction of susceptibility artifacts in T(*) Med Imaging Rev 1:115–129
(2)-weighted functional MRI. Magn Reson 134. Morawetz C, Holz P, Lange C et al (2008)
Med 44:525–531 Improved functional mapping of the human
122. Yip CY, Fessler JA, Noll DC (2006) Advanced amygdala using a standard functional magnetic
three-dimensional tailored RF pulse for signal resonance imaging sequence with simple modi-
recovery in T2*-weighted functional mag- fications. Magn Reson Imaging 26:45–53
netic resonance imaging. Magn Reson Med 135. Stocker T, Kellermann T, Schneider F et al
56:1050–1059 (2006) Dependence of amygdala activation
123. De Panfilis C, Schwarzbauer C (2005) Positive on echo time: results from olfactory fMRI
or negative blips? The effect of phase encod- experiments. Neuroimage 30:151–159
ing scheme on susceptibility-induced signal 136. Gorno-Tempini M, Hutton C, Josephs O,
losses in EPI. Neuroimage 25:112–121 Deichmann R, Price C, Turner R (2002)
124. Ojemann JG, Akbudak E, Snyder AZ, Echo time dependence of BOLD contrast and
McKinstry RC, Raichle ME, Conturo TE susceptibility artifacts. NeuroImage
(1997) Anatomic localization and quantitative 15:136–142
analysis of gradient refocused echo-planar fMRI 137. Sodickson DK, Manning WJ (1997)
susceptibility artifacts. Neuroimage 6:156–167 Simultaneous acquisition of spatial harmonics
125. Chen N, Dickey CC, Guttman CRG, Panych (SMASH): fast imaging with radiofrequency
LP (2003) Selection of voxel size and slice ori- coil arrays. Magn Reson Med 38:591–603
entation for fMRI in the presence of suscepti- 138. Pruessmann K, Weiger M, Scheidegger M,
bility field gradients: application to imaging of Boesiger P (1999) SENSE: sensitivity encod-
the amygdala. NeuroImage 19:817–825 ing for fast MRI. Magn Reson Med
126. Weiskopf N, Hutton C, Josephs O, Deichmann 42:952–962
R (2006) Optimal EPI parameters for reduc- 139. Lutcke H, Merboldt KD, Frahm J (2006)
tion of susceptibility-induced BOLD sensitiv- The cost of parallel imaging in functional
ity losses: a whole-brain analysis at 3 T and 1.5 MRI of the human brain. Magn Reson
T. Neuroimage 33:493–504 Imaging 24:1–5
127. Young IR, Cox IJ, Bryant DJ, Bydder GM 140. Schmidt CF, Degonda N, Luechinger R,
(1988) The benefits of increasing spatial reso- Henke K, Boesiger P (2005) Sensitivity-­
lution as a means of reducing artifacts due to encoded (SENSE) echo planar fMRI at 3 T in
field inhomogeneities. Magn Reson Imaging the medial temporal lobe. NeuroImage
6:585–590 25:625–641
128. Hyde S, Biswal B, Jesmanowicz A (2001) 141. Fürsatz M, Windischberger C, Karlsson KÆ,
High-resolution fMRI using multislice partial Moser E (2008) Successful fMRI of the
k-space GR-EPI with cubic voxels. Magn hypothalamus at 3T. Proc Intl Soc Magn
­
Reson Med 46:114–125 Reson Med 16:2501
 fMRI of Emotion 493

142. Fürsatz M, Windischberger C, Karlsson KÆ, 156. Witoszynskyj S, Rauscher A, Reichenbach JR,
Mayr W, Moser E. Valence-dependent modu- Barth M (2007) ΠUN (Πhase UNwrapping)
lation of hypothalamic activity. Neuroimage validation of a 2D region-growing phase
(in press) unwrapping program. Proc Intl Soc Magn
143. Dowell NG, Tofts PS (2007) Fast, accurate, Reson Med 15:3436
and precise mapping of the RF field in vivo 157. Windischberger C, Robinson S, Rauscher A,
using the 180 degrees signal null. Magn Barth M, Moser E (2004) Robust field map
Reson Med 58:622–630 generation using a triple-echo acquisition.
144. Speck O, Hennig J (1998) Functional imag- J Magn Reson Imaging 20:730
ing by I0- and T2*-parameter mapping using 158. Roemer PB, Edelstein WA, Hayes CE, Souza
multi-image EPI. Magn Reson Med SP, Mueller OM (1990) The NMR phased
40:243–248 array. Magn Reson Med 16:192–225
145. Posse S, Wiese S, Gembris D et al (1999) 159. Bernstein MA, Grgic M, Brosnan TJ, Pelc NJ
Enhancement of BOLD-contrast sensitivity (1994) Reconstructions of phase contrast,
by single-shot multi-echo functional MR phased array multicoil data. Magn Reson Med
imaging. Magn Reson Med 42:87–97 32:330–334
146. Posse S, Shen Z, Kiselev V, Kemna LJ (2003) 160. Hammond KE, Lupo JM, Xu D et al (2008)
Single-shot T(2)* mapping with 3D compen- Development of a robust method for generat-
sation of local susceptibility gradients in mul- ing 7.0 T multichannel phase images of the
tiple regions. Neuroimage 18:390–400 brain with application to normal volunteers
147. Posse S, Holten D, Gao K, Rick J, Speck O and patients with neurological diseases.
(2006) Evaluation of interleaved XYZ-­ Neuroimage 39:1682–1692
shimming with multi-echo EPI in prefrontal 161. Robinson S, Jovicich J (2008) EPI distortion
cortex and amygdala at 4 Tesla. NeuroImage corrections at 4 T: Multi-channel field map-
31:S154 ping and a comparison with the point-spread
148. Weiskopf N, Klose U, Birbaumer N, Mathiak function method. Proc Intl Soc Magn Reson
K (2005) Single-shot compensation of image Med 16:3031
distortions and BOLD contrast optimization 162. Robson MD, Gore JC, Constable RT (1997)
using multi-echo EPI for real-time Measurement of the point spread function in
fMRI. Neuroimage 24:1068–1079 MRI using constant time imaging. Magn
149. Glover GH, Law CS (2001) Spiral-in/out Reson Med 38:733–740
BOLD fMRI for increased SNR and reduced 163. Zeng H, Constable RT (2002) Image distor-
susceptibility artifacts. Magn Reson Med tion correction in EPI: Comparison of field
46:515–522 mapping with point spread function mapping.
150. Guo H, Song AW (2003) Single-shot spiral Magn Reson Med 48:137–146
image acquisition with embedded z-­shimming 164. Zaitsev M, Hennig J, Speck O (2004) Point
for susceptibility signal recovery. J Magn spread function mapping with parallel imag-
Reson Imaging 18:389–395 ing techniques and high acceleration factors:
151. Truong TK, Song AW (2008) Single-shot fast, robust, and flexible method for echo-­
dual-z-shimmed sensitivity-encoded spiral- planar imaging distortion correction. Magn
in/out imaging for functional MRI with
­ Reson Med 52:1156–1166
reduced susceptibility artifacts. Magn Reson 165. Hu X, Kim SG (1994) Reduction of signal
Med 59:221–227 fluctuation in functional MRI using navigator
152. Li Z, Wu G, Zhao X, Luo F, Li SJ (2002) echoes. Magn Reson Med 31:495–503
Multiecho segmented EPI with z-shimmed 166. Bruder H, Fischer H, Reinfelder HE,
background gradient compensation Schmitt F (1992) Image reconstruction for
(MESBAC) pulse sequence for fMRI. Magn echo planar imaging with nonequidistant
Reson Med 48:312–321 k-space sampling. Magn Reson Med
153. Weisskoff RM, Davis TL (1992) Correcting 23:311–323
gross distortion on echo planar images. Paper 167. Barry RL, Klassen LM, Williams JM, Menon RS
presented at the SMRM, Berlin (2008) Hybrid two-dimensional navigator cor-
154. Jezzard P, Balaban RS (1995) Correction for rection: a new technique to suppress respiratory-­
geometric distortion in echo planar images from induced physiological noise in multi-shot
B0 field variations. Magn Reson Med 34:65–73 echo-planar functional MRI. Neuroimage
155. Jenkinson M (2003) Fast, automated, 39:1142–1150
N-dimensional phase-unwrapping algorithm. 168. Lund TE, Madsen KH, Sidaros K, Luo WL,
Magn Reson Med 49:193–197 Nichols TE (2006) Non-white noise in fMRI:
494 Simon Robinson et al.

does modelling have an impact? Neuroimage 181. Preibisch C, Raab P, Neumann K et al (2003)
29:54–66 Event-related fMRI for the suppression of
169. Glover GH, Li TQ, Ress D (2000) Image-­based speech-associated artifacts in stuttering.
method for retrospective correction of physio- Neuroimage 19:1076–1084
logical motion effects in fMRI: RETROICOR. 182. Birn RM, Bandettini PA, Cox RW, Shaker R
Magn Reson Med 44:162–167 (1999) Event-related fMRI of tasks involving
170. Windischberger C, Friedreich S, Hoheisel B, brief motion. Hum Brain Mapp 7:106–114
Moser E (2004) The importance of correct- 183. Phelps EA, O’Connor KJ, Cunningham WA
ing for physiological artifacts for functional et al (2000) Performance on indirect mea-
MRI in deep brain structures. NeuroImage sures of race evaluation predicts amygdala
22:S28 activation. J Cogn Neurosci 12:729–738
171. Josephs O, Howseman A, Friston K, Turner 184. Winston JS, Strange BA, O’Doherty J, Dolan
R (1997) Physiological noise modelling for RJ (2002) Automatic and intentional brain
multi-slice EPI fMRI using SPM. Proc Intl responses during evaluation of trustworthi-
Soc Magn Reson Med 5:1682 ness of faces. Nat Neurosci 5:277–283
172. Weissenbacher A, Windischberger C, 185. Singer T, Kiebel SJ, Winston JS, Dolan RJ,
Lanzenberger R, Moser E (2008) Efficient Frith CD (2004) Brain responses to the acquired
correction for artificial signal fluctuations in moral status of faces. Neuron 41:653–662
resting-state fMRI-data. Proc Intl Soc Magn 186. Campbell DT, Stanley JC (1966)
Reson Med 16:2467 Experimental and quasi-experimental designs
173. Beckmann CF, Smith SM (2004) Probabilistic for research. Houghton Mifflin, Boston
independent component analysis for func- 187. Poldrack RA, Wagner AD (2004) What can
tional magnetic resonance imaging. IEEE neuroimaging tell us about the mind? Insights
Trans Med Imaging 23:137–152 from prefrontal cortex. Curr Dir Psychol Sci
174. Calhoun V, Adali T, Stevens M, Kiehl K, 13:177–181
Pekar J (2005) Semi-blind ICA of fMRI: a 188. Logothetis NK, Pauls J, Augath M, Trinath
method for utilizing hypothesis-derived time T, Oeltermann A (2001) Neurophysiological
courses in a spatial ICA analysis. Neuroimage investigation of the basis of the fMRI signal.
25:527–538 Nature 412:150–157
175. Thomas CG, Harshman RA, Menon RS 189. Janz C, Heinrich SP, Kornmayer J, Bach M,
(2002) Noise reduction in BOLD-based Hennig J (2001) Coupling of neural activity
fMRI using component analysis. Neuroimage and BOLD fMRI response: new insights by
17:1521–1537 combination of fMRI and VEP experiments
176. Kochiyama T, Morita T, Okada T, Yonekura Y, in transition from single events to continu-
Matsumura M, Sadato N (2005) Removing the ous stimulation. Magn Reson Med 46:
effects of task-related motion using independent-­ 482–486
component analysis. Neuroimage 25:802–814 190. Baas D, Aleman A, Kahn RS (2004)
177. Perlbarg V, Bellec P, Anton JL, Pelegrini-Issac Lateralization of amygdala activation: a sys-
M, Doyon J, Benali H (2007) CORSICA: tematic review of functional neuroimaging
correction of structured noise in fMRI by studies. Brain Res Brain Res Rev 4:96–103
automatic identification of ICA components. 191. Robinson S, Pripfl J, Bauer H, Moser M
Magn Reson Imaging 25:35–46 (2005) Empirical evidence for the minimum
178. Tohka J, Foerde K, Aron AR, Tom SM, Toga voxel size required for reliable 3 T fMRI of
AW, Poldrack RA (2008) Automatic indepen- the amygdala. NeuroImage 26:S795
dent component labeling for artifact removal 192. Habel U, Windischberger C, Derntl B et al
in fMRI. Neuroimage 39:1227–1245 (2007) Amygdala activation and facial expres-
179. Bullmore ET, Brammer MJ, Rabe-Hesketh S sions: explicit emotion discrimination versus
et al (1999) Methods for diagnosis and treat- implicit emotion processing.
ment of stimulus-correlated motion in generic Neuropsychologia 45:2369–2377
brain activation studies using fMRI. Hum 193. Sack AT, Linden DE (2003) Combining transcra-
Brain Mapp 7:38–48 nial magnetic stimulation and functional imaging
180. Morgan VL, Dawant BM, Li Y, Pickens DR in cognitive brain research: possibilities and limita-
(2007) Comparison of fMRI statistical soft- tions. Brain Res Brain Res Rev 43:41–56
ware packages and strategies for analysis of 194. Thayer JF, Brosschot JF (2005)
images containing random and stimulus-­ Psychosomatics and psychopathology: look-
correlated motion. Comput Med Imaging ing up and down from the brain.
Graph 31:436–446 Psychoneuroendocrinology 30:1050–1058
 Chapter 16

fMRI of Pain
Emma G. Duerden, Roberta Messina, Maria A. Rocca,
Massimo Filippi, and Gary H. Duncan

Abstract
Pain was first considered to be a hard-wired system in which noxious input was passively transmitted along
sensory channels to the brain. However, today it is generally accepted that the experience of pain is not
simply driven by noxious stimulus characteristics, but that the brain is the structure where the subjective
perception of pain emerges and is critically linked with other cognitive processes.
The field of pain research has progressed immensely due to the advancement of brain imaging tech-
niques. The initial goal of this research was to expand our understanding of the cerebral mechanisms
underlying the perception of pain; more recently the research objectives have shifted toward chronic
pain—understanding its origins, developing methods for its diagnosis, and exploring potential avenues for
its treatment. While several different neuroimaging approaches have certain advantages for the study of
pain, fMRI has ultimately become the most widely utilized imaging technique over the past decade because
of its noninvasive nature, high-temporal and spatial resolution, and general availability; thus, the following
chapter will focus on fMRI and the special aspects of this technique that are particular to pain research.

Key words Pain, Functional neuroimaging, Brain, Perception

1 Introduction

The history of pain imaging is relatively short, although it has

advanced immensely within the last decade due to improvements in
imaging techniques, statistical analysis, and specialized equipment
for the delivery of painful stimuli. Initially, brain-imaging studies
sought simply to examine the brain areas that are involved in pain
processing, to make comparisons with the long established neuro-
physiological studies reported in this field. Many of these initial
imaging studies were prompted by electrophysiological data from
patients undergoing brain surgery in the early part of the twentieth
century [1], which had questioned the role of the cortex in nocicep-
tive processing. It was initially believed that the thalamus was pri-
marily responsible for nociceptive processing as suggested by deficits
in pain perception observed in patients with thalamic lesions [2].

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_16, © Springer Science+Business Media New York 2016

495
496 Emma G. Duerden et al.

In the early 1990s, activation in the human brain evoked by

experimental pain stimuli was studied using positron emission
tomography (PET) [3, 4] and single photon emission tomography
(SPECT) [5]. Then in 1995 the first fMRI studies examining the
cortical representation of pain [6] were conducted largely to con-
firm the findings of previous PET studies and to examine whether
the cortical nociceptive signal could be detected using fMRI. In
more recent years, the field of pain imaging has expanded
immensely, allowing researchers to answer complex questions con-
cerning pain processing, such as how cortical regions are connected
and modified during the perception of pain and, most importantly,
how the cortex responds during the modulation of pain. These
experimental studies were conducted in healthy humans in order
to answer broad questions regarding pain processing, with the
eventual goal of applying this knowledge to a better understanding
and alleviation of pain and suffering associated with chronic pain
syndromes. The use of fMRI and other imaging techniques have
revealed a number of cortical and subcortical changes that may
occur as a result of prolonged exposure to pain—or possibly as
causal factors in chronic pain conditions [7–9]. Indeed, with the
advent of high-speed image acquisition and computational pro-
cessing, not only has the technology of fMRI revealed areas of
cortical plasticity associated with chronic pain, but it is also now
possible to use fMRI in real-time to furnish feedback to subjects
(and patients) to teach them how to modulate their cortical activa-
tion in response to chronic pain [10, 11].
This chapter reviews and discusses the various advances in our
knowledge of cerebral pain processing that have been achieved
using fMRI, the response properties of cortical nociceptive neu-
rons in relation to both imaging techniques and stimuli used to
evoke pain, the applications of this research to treat clinical pain in
patients, and the future of pain research using fMRI.

2 Use of fMRI to Study Nociceptive Processing

Compared to other brain mapping techniques currently used to

study pain experimentally in humans—such as PET, electroencepha-
lography (EEG), magnetoencephalography (MEG), or optical imag-
ing—fMRI is the tool of choice, given its high spatial resolution,
noninvasiveness, and reasonable temporal resolution, which allow
the study of rapid dynamic processes involved in pain processing.
However, a number of methodological issues concerning the use of
the BOLD signal in research involving cortical, and more recently,
spinal mechanisms of pain perception has to be considered.

2.1 Nociceptive For cortical nociceptive processing related to cutaneous heat stimuli,
BOLD Signal the hemodynamic response function (HRF) peaks slightly later and
 fMRI of Pain 497

lasts longer in comparison to innocuous stimuli. Chen et al. [12]

performed a direct comparison of the temporal properties of the
HRF in response to noxious thermal heat pain and innocuous brush-
ing stimuli in SI and SII. While both stimuli were of the same dura-
tion, the time course for innocuous stimuli peaked ~10 s after the
onset of the stimulus and dissipated quickly after its removal.
However, noxious thermal heat stimuli produced a time course
peaking at ~15 s after the onset of the stimulus and the response was
sustained for several seconds. Similar results have been reported in
response to painful electrical stimuli [13]; identical trains of noxious
and innocuous stimuli produced differential time courses, with the
HRF for painful stimulation lasting twice as long as that produced
by nonpainful stimuli.
Time course information on the BOLD response to noxious
stimuli is crucial for interpreting data analyzed using the standard
canonical HRFs available in the majority of fMRI analysis software,
which approximate this time period at ~6 s. Ideally to establish a
more representative model of painful stimuli, a canonical HRF
should be created based on data from independent studies employ-
ing similar noxious stimulation. The BOLD signal can then be
regressed against this canonical HRF to reveal activation more spe-
cific to the nociceptive signal.
A related issue in analyzing data recorded during experimental
pain studies is the critical importance of considering the rise time
of thermal stimuli when establishing time periods in the event
design matrix. As the temperature of the thermode gradually
increases, warm and pain fibers will become increasingly activated.
In order to maximize sensitivity for detection of the pain-related
BOLD signal, it is important to enter into the design matrix solely
the period of time during which the thermode has exceeded the
subjects’ pain threshold—not the initial rise-time of the stimulus
period, which would be associated with the innocuous warm sensa-
tions perceived before the actual onset of pain.

2.2 BOLD fMRI A newly developing field in pain fMRI is spinal cord imaging, which
of Spinal Nociceptive is crucial for a better understanding of central nervous system (CNS)
Signals pain processing. The spinal cord and brainstem receive input from
the periphery before relaying this information on to the cortex. These
subcortical regions are involved in the modulation of nociceptive
input and the potentially abnormal processing of that input that may
lead to chronic pain syndromes. Therefore, knowledge concerning
the peripheral mechanisms of nociceptive processing is crucial to
understanding a number of pathological pain conditions resulting
from nerve injury or inflammation. These factors contribute to the
generation and maintenance of two key components of chronic pain,
namely hyperalgesia and allodynia. Hyperalgesia is the phenomenon
where an exaggerated response occurs after exposure to a noxious
stimulus. Allodynia is an exaggerated response toward nonpainful
498 Emma G. Duerden et al.

mechanical stimuli. Both occur when nociceptive fibers become sen-

sitized, after exposure to a noxious stimulus, causing the release of
“painful” substances in the periphery. Peripheral sensitization can
occur due to inflammation of peripheral tissues as a result of a burn
or cut. Because of this barrage of input, peripheral nociceptors can
become hyperexcitable. This peripheral sensitization can also occur
due to ectopic firing of peripheral nerves resulting from an amputa-
tion or injury. Central sensitization can occur in the dorsal horns of
the spinal cord, when peripheral nerves that were once insensitive to
nociceptive input switch their firing patterns and begin to transmit
nociceptive information, causing the area of affected skin to become
painful to the slightest touch. Much research in this area is focused at
the periphery, although these processes have been shown to have
supra-spinal effects resulting in aberrant cortical activity and the reor-
ganization of body maps in somatosensory cortices.
To fulfill this need to study spinal mechanisms of nociception,
experimental models directed toward spinal fMRI have begun in
humans [14, 15].
Applications of spinal fMRI to the study of chronic pain could
have vast clinical applications. Use of a noninvasive functional imag-
ing modality could shed light on the spinal mechanisms involved in
the generation of neuropathic pain, such as dysesthetic pain in
patients with spinal cord injury or syringomyelia. In addition to
understanding the effects of chronic pain on neuroplasticity of the
spinal cord, spinal fMRI could provide insight into the potential
mechanisms of medications and their efficacy at treating chronic pain.

3 Methods for fMRI Pain Experiments

3.1 Pain Assessment A key issue in functional imaging of the cortical nociceptive signal
is to ensure that the stimuli delivered to the subjects are perceived
as noxious. Pain thresholds are commonly determined during a
separate session prior to the scan. This procedure also serves to
familiarize participants with the stimuli and reduce anxiety, thereby
minimizing anxiety-related fluctuations in cardiovascular activity
[16]. Stimuli utilized for the scanning session are frequently tai-
lored to each individual’s pain threshold; conversely, all subjects
can be administered the same level of noxious stimulation, which
has been determined to evoke the perception of pain in all subjects.
A corollary to the appropriate choice of noxious stimuli is the con-
firmation that predetermined levels of stimulation are actually per-
ceived as painful, within the scanning environment. A number of
contextual factors can alter the perception of stimuli that were
originally considered painful during a pre-scanning test, including
the temperature of the scanning suite, the position of the body in
the scanner, and distractions of noise, possible feelings of claustro-
phobia, and other conditions specific to the scanning paradigm.
 fMRI of Pain 499

It is also important to note that the perception of pain can

change during the course of a scanning session, due either to habit-
uation, sensitization, or the potential changes in attention during a
long scanning experiment [17, 18].
To address this issue, pain assessment ratings can be obtained dur-
ing the fMRI scanning session through subjective reports from par-
ticipants using a variety of methods. Subjects can rate their perception
after each stimulus, continuously during the stimuli, or at the end of
the scanning run by giving an average rating of all the stimuli. Subjects’
scores are recorded typically using numerical or visual analog scales
(VAS) [19]. In fMRI experiments, ratings can be obtained during or
immediately after the presentation of each stimulus. Conversely, due
to methodological issues, pain ratings within the context of PET stud-
ies can be taken only at the end of a scanning session several minutes
after stimulus presentation. Increased time between stimulus presen-
tation and assessment can cause inaccuracies in subject responses [20].
This is a special consideration in studies examining mechanisms of
analgesic relief since retrospective ratings can be inflated with increased
time after stimulus presentation [21, 22].
In addition to ensuring that the noxious stimuli are actually
painful, pain assessment ratings (and other behavioral measures) can
be used as regressors in the fMRI design matrix to aid in identifying
cortical regions involved in various aspects of pain processing.
Behavioral data can be incorporated into the fMRI design matrix as
a weighting factor applied to the canonical HRF. Alternatively, con-
tinuous pain ratings (recorded during the stimulus presentations)
can be modeled in the design matrix (e.g., see ref. [23]). The result-
ing contrasts produce activation sites that are more closely based on
the degree to which a region’s activity correlates with the perceived
intensity of the stimuli rather than with the physical intensity of the
stimulus—in other words a “percept-related” activation as opposed
to a “stimulus-related” activation [24].
This experimental approach may have important implications for
studying the dissociation that sometimes occurs between the intensity
of peripheral stimulation and the perception of pain. For example, pre-
sentation of noxious mechanical stimuli over longer durations (~2 min)
has been shown to disrupt the relationship between the firing fre-
quency of nociceptive afferents and the perceived intensity of pain
evoked by the stimuli [25, 26]. This paradoxical relationship may be
explained by the process of temporal summation—a disproportionate
increase in the firing rate of dorsal horn neurons over time, whereby
their response threshold to sensory input is substantially lowered.
Additionally, repeated exposure to short-duration heat pain stimuli can
cause habituation to both the perceived intensity and unpleasantness of
the stimuli [27]. Therefore, subjective pain ratings can play a key role
in the interpretation of nociceptive processing in the cortex, as opposed
to utilizing simply the duration or intensity of the noxious stimuli that
may not aptly reflect the resulting activations.
500 Emma G. Duerden et al.

Several studies have explored the possible cerebral

mechanisms underlying habituation or sensitization to painful
stimuli, showing activation of cerebral regions involved in anti-
nociception. However, these gave conflicting results concerning
any specific association between cerebral activity and ratings of
pain intensity [17, 28–30].
On the whole, however, these ambiguities in the correspondence
between stimulus delivery, evoked nociceptive signal, and subjective
reports of pain intensity, underscore the importance of accessing the
level of perceived pain during scanning sessions, rather than assuming
a fixed relationship between stimulation and percept.
A number of advantages and potential disadvantages are asso-
ciated with obtaining continuous pain ratings of stimuli during
fMRI experiments. Clearly, participants’ perceptual evaluations
will rely less on memory and will tend to be more accurate, com-
pared with evaluations made after the scanning run. In turn, the
resulting brain activation will be less reflective of mnemonic or
error detection processes. Additionally, continuous ratings can be
used to deduce the time lag between the application of the stimu-
lus and the onset of pain perceived by the subject, and to provide
further details about the time course of pain perception and the
underlying neural activity.
While continuous ratings provide real-time information about
a subject’s perception of the stimuli, a clear disadvantage to their
use is that the motor activity and motor-related activation can pro-
duce a confound that complicates interpretation of sensory-related
activity. However, this can be accounted for by including the
movements as covariates in the fMRI design matrix.

4 Neuroanatomy of Pain Processing

Before describing how fMRI measures the cortical and spinal noci-
ceptive signal, it is important to understand how this signal is
transferred to the cortex. In the periphery, a painful stimulus
applied to the body is transmitted to the CNS through nociceptors
[31]. Myelinated A-delta fibers transmit sharp pricking pain [32],
while unmyelinated C-fibers transmit slow burning pain, often
referred to as second pain [33]. The cell bodies of A-delta and
C-fibers are located in the dorsal root ganglia, receiving afferent
input from the periphery and then sending the information into
the spinal cord to terminate in the dorsal horn [34, 35]. Axons
from the second-order dorsal horn neurons rise through several
ascending pathways that transmit nociceptive information to the
thalamus, reticular formation, and cortex [8, 36]. Pain and tem-
perature information applied to the face is relayed through cranial
nerves to the spinal nucleus V terminating in the thalamus via the
trigeminothalamic tract, which is then relayed to the cortex.
 fMRI of Pain 501

A number of spinal and cortical neurons respond to noxious

stimuli including nociceptive-specific (NS) and wide dynamic
range (WDR) projection neurons, the latter of which respond to
both noxious and innocuous stimuli. Additionally, the dorsal horns
and cortical somatosensory regions contain neurons responsive
solely to innocuous stimuli called low threshold mechanical (LTM)
neurons and thermoreceptive neurons responsive to temperatures
in the warm and cold range. However, recent studies demonstrated
that nociceptive, tactile, auditory and visual stimuli can elicit spa-
tially indistinguishable cortical responses, thus indicating that the
bulk of the brain responses to nociceptive stimuli reflects multi-
modal neural activity (i.e., activity that can be triggered by any
kind of stimulus independently of sensory modality) [37]. This
range of responses is an important consideration when interpreting
results from fMRI studies of pain in terms of exactly what the acti-
vation pattern is reflecting.
Typically, pain-evoked brain activation is achieved by applying
contact thermodes to the skin. This technique involves an increase
in temperature at the rate of 1–10 °C s−1. Depending on the base-
line temperature it can take several seconds to reach perceived pain
threshold. In addition to activating NS neurons with noxious heat,
contact thermodes may activate both LTM and WDR neurons
through innocuous mechanical and thermal stimulation of the skin
as the stimulation temperature rises towards pain threshold.
Therefore, to examine pain-specific cortical activations, it is neces-
sary to compare pain-related activations to those associated with
the presentation of innocuous warm stimuli.
In addition to conductive heating of the skin using contact
thermodes, nociceptive afferents can be activated using thermal
radiation administered through infrared laser stimulators [38, 39].
Lasers can deliver heat stimuli without the need for a contact probe,
thus selectively stimulating C-fibers and A-delta fibers without con-
taminant activation of A-beta fibers that transmit touch informa-
tion. Additionally, laser stimuli can activate nociceptive nerve
endings at rapid rates for short durations (1 ms) [40, 41] and are
therefore well suited for rapid event-related fMRI studies. However,
an important consideration associated with the use of laser stimuli
is the difficulty of measuring and controlling skin temperature,
which is the primary factor triggering the cascade of neural responses
that culminate in the processing of heat-related nociceptive infor-
mation in the brain and likewise the assessment of pain by the sub-
jects [42]. Laser and contact heat stimuli have been shown to
produce similar patterns of BOLD activation in anterior cingulate
cortex (ACC), insula, primary motor cortex, prefrontal cortex
(PFC), parahippocampal gyrus, thalamus, basal ganglia, periaqua-
ductal gray (PAG), and cerebellum. However, stronger activation in
response to contact heat stimuli was noted in secondary somatosen-
sory cortex (SII), posterior insula, posterior ACC, and regions in
502 Emma G. Duerden et al.

parietal and frontal cortices [43]. Thus, these two modes of

delivering noxious heat stimulation cannot be considered identical
in terms of the evoked pain-related BOLD activations, and the
advantages and disadvantages of each should be weighed in relation
to the research questions and appropriate stimulation paradigms.

4.1 Supraspinal During the past 3–5 years, neuroimaging studies have extensively
Processing investigated the neural basis of pain perception, thus showing that
of Nociceptive Stimuli nociceptive stimuli commonly elicit activity within a very wide array
of subcortical and cortical brain structures [44, 45]. Regions most
frequently activated by painful stimuli include primary somatosen-
sory cortex (SI), SII, ACC, the insula, the PFC, and the thalamus.
Regions responsible for pain processing are categorized along
two functional lines—the first being the sensory-discriminative (lat-
eral pain system) component involved in the perception of temporal,
intensity, and localization aspects of pain processing, and the second,
the affective-motivational (medial) component associated with the
emotional aspects of pain [46, 47]. Dissociations between the two
systems are made through subjective reports on pain scales. After
exposure to noxious stimuli, subjects are asked to quantify separately
how intense and how unpleasant is the perceived pain. Regions
implicated in the lateral pain system include SI, SII, posterior insula,
and lateral thalamus, while the medial pain system consists of the
medial thalamic nuclei, the ACC, and the PFC. Much of what is
known regarding the two components in pain processing was ini-
tially explored through single-unit recordings in nonhuman pri-
mates and lesion studies in humans. However, the more recent
ability to study these functional components noninvasively in
humans using fMRI and other brain mapping techniques has allowed
pain researchers to advance rapidly in their understanding of the role
of these cortical regions in pain processing and how they interact.

4.1.1 Primary SI is located in the postcentral gyrus, is composed of four areas (areas
Somatosensory Cortex 3a, 3b, 1, and 2) [48], and is involved in the processing of both
tactile and noxious stimuli [49]. It was long debated whether SI was
necessary to perceive pain. Early studies of patients with brain lesions
suggested that deficits in nociceptive processing were rather com-
mon following lesions to the thalamus, but were very rare when
damage was restricted to the area believed to incorporate SI [2].
Likewise, later studies, using electrical stimulation of the human cor-
tex during awake brain surgery, reported that direct stimulation of
SI rarely evoked any perception of pain in patients [1].
The advent of imaging technology allowed a more global
exploration of the role of SI and other cortical regions involved in
pain processing, and these studies could be conducted in healthy
volunteers, rather than in patients with brain injuries that might
alter normal function. The first of these studies involved PET and
demonstrated that noxious stimuli applied to the hands were
 fMRI of Pain 503

associated with robust activation in SI [3]. Several other early

studies failed to detect SI activation [4, 5], and subsequent reports,
using either PET or fMRI, have resulted in contradictory findings
(for SI activation, see for example: [50–53]; for absence of SI acti-
vation, see refs. [54, 55]).
In particular, several studies showed that the anterior portion
of SI (the area 3a) receives input originating predominantly from
unmyelinated nociceptors, distinguishing it from posterior SI
(areas 3b and 1), long recognized as receiving input predominantly
from myelinated afferents, including nociceptors [56].
The inconsistency of SI activation reported across imaging
studies could be due to several factors. Wide variations in the loca-
tion of the central sulcus across subjects may lead to a wash out in
signal across averaged group data. In addition, a reduction in SI
activity below statistically significant levels could be caused in some
paradigms by inhibitory effects induced by noxious stimuli on tac-
tile inputs [4, 57]. In a review discussing the issue of pain-related
activation of SI, Bushnell et al. concluded that the BOLD signal in
SI largely depends on task design that is likely to influence the
attentional state of the subject [58]. Results from subsequent
studies have likewise indicated that pain-related BOLD activation
of SI is increased when subjects attend to pain and decreased when
they are distracted [59].
On the contrary, attention may also show a deleterious effect
on SI activation as noted by Oshiro et al. [60] in their fMRI study
examining the neural correlates involved in processing spatial local-
ization of pain. The authors failed to find activation in SI in
response to painful stimulation of the calf. However, the authors
noted that this lack of activation may have been a result of the
response properties of the cortical nociceptive neurons.
Nociceptive input to SI is somatotopically organized [61–63],
and the small receptive fields of SI [64] suggest that this region is
well suited to make fine spatial discriminations of noxious stimuli
applied to the body. Oshiro et al. [60] required subjects to focus
on stimulation applied to their calves, and this increased attention
on the leg area may have caused a reduction in the receptive field
sizes of nociceptive neurons, which would enhance spatial acuity
needed to perform the task—but cause deterioration in resulting
brain activation. In another study using a discrimination task,
Albanese et al. [65] explored short-term memory for the spatial
location and intensity of painful thermal stimuli applied to the
palms. In contrast to the study by Oshiro et al. [60], Albanese
et al. [65] reported robust pain-related activation in SI/posterior
parietal cortex, which was sustained during the memory period of
the trial, suggesting that this region has a role in the encoding and
retention of noxious stimuli. Differences between the two studies
may be due to the larger somatotopic organization of the hand
representation of SI. Additionally, subjects in the Albanese study
504 Emma G. Duerden et al.

were required to detect the end of each stimulus, a strategy that

may have heightened attention toward the stimuli and contributed
to a temporal summation of the BOLD signal in SI.
Somatosensory brain regions have also been found to play a
role during pain anticipation. A recent study [66] revealed greater
changes in activation of SI in nocebo responders compared to con-
trols, supporting the notion that anticipatory activation of a
prefrontal-limbic network is involved in nocebo hyperalgesia.

4.1.2 Secondary SII is also considered to be an important region for processing the
Somatosensory Cortex (SII) sensory-discriminative component of pain. SII is located in the
parietal operculum in the dorsal bank of the lateral sulcus. Like SI,
this region receives projections from the ventroposterior lateral
nucleus (VPL) of the thalamus, but its major nociceptive input
comes directly from the ventroposterior inferior (VPI) nucleus
[52]. Studies of patients with lesions that include SII have demon-
strated deficits in the perception of pain intensity [67, 68]; how-
ever, lesions comprised additional cortical regions that may work in
concert with SII to process this piece of information. In addition
to these clinical findings, converging evidence from a number of
studies supports the notion that SII possesses a functional capacity
to discriminate between different intensities of noxious stimuli pre-
sented to the contralateral side of the body. Evidence from PET
provides a role for this region in intensity processing in that sub-
jects’ ratings of pain intensity in response to thermal heat pain have
been shown to be highly correlated with activation of SII [69].
Additionally, an fMRI study by Maihofner et al. [70] found
increased activation in SII in response to painful mechanical stimuli
compared to thermal heat pain. In turn, ratings of subjective inten-
sity were correlated with the intensity of mechanical pain. However,
dissociative processing was noted in this region as ratings of
unpleasantness were not found to correlate with SII activation.
Contrary to these findings, evidence from fMRI suggests this
region may be involved in some emotional aspects of pain process-
ing. For example, Gracely et al. [71] found that fibromyalgia
patients who scored higher on a pain catastrophizing questionnaire
showed increased activation in both the ACC and SII in response
to noxious stimuli. Catastrophizing (and in turn anxiety about
painful stimuli) is inherently linked with pain perception, where
the individual’s emotional state augments neural processing of
these stimuli. In line with these findings are data that show
increased activity in SII during the anticipation of painful stimuli,
indicative of an enhanced emotional response [66, 72].

4.1.3 Insular Cortex The insula is extensively connected to other brain regions such as
the prefrontal cortex, cingulum, amygdala, SI, SII, and also tha-
lamic nuclei (VPI, the centromedian-parafasicular, the medial dor-
sal [MD], and the ventral medial posterior [Vmpo] nuclei). It may
 fMRI of Pain 505

therefore act as a relay integrating afferent nociceptive information

with working memory, affect, and attention, and may selectively
gate nociceptive information at the cortical level to modulate vary-
ing levels of appreciation of the nociceptive stimulus [73].
Functional neuroimaging studies suggested that the anterior,
mid, and posterior division of the insula subserve different func-
tions in the perception of pain [74].
Early clinical reports and quantitative studies [75–77], have indi-
cated that patients with lesions encompassing the insula do not
exhibit normal withdrawal or emotional responses to noxious stimuli,
indicating an altered or deficient perception of pain affect. Accordingly,
fMRI activity in the anterior insula in response to noxious stimuli is
correlated with subjective ratings of pain unpleasantness [78, 79].
The insula, in particular its posterior region, has also been found
to process sensory-discriminative features of nociceptive informa-
tion, making it a likely area of convergence of the two pain systems.
Evidence for the role of the insula in sensory-discriminative process-
ing comes from direct electrical stimulation to the region during
awake brain surgery, demonstrating evoked painful sensations in the
body [80], and also from fMRI studies that showed a significant cor-
relation between posterior insula activity and painful stimulus inten-
sity [79, 81]. Furthermore, several other lines of evidence indicate
that this region may be involved in the localization of painful stimuli,
as it contains a somatotopic map of the body. The dorsal posterior
insula receives pain and temperature information from a somato-
topically organized region of the thalamus—the VMpo [82], which
in turn receives projections from thermoreceptive and nociceptive
neurons residing in lamina I of the spinal cord [79, 83] (Fig. 1).
Neuroimaging studies of pain perception frequently report
insular activation, making it difficult to dissociate it from activation
seen in adjacent regions of SII [84]. Resolving the precise somato-
topic organization of the insula using fMRI has become feasible
with the availability of high-field strength magnets. Several fMRI
studies at 3 T have revealed a nociceptive somatotopic organiza-
tion in the dorsal posterior insula in response to both cutaneous
and muscle pain [85, 86].
It is further notable that the posterior insula has been suggested
to be involved also in directing pain-related motor responses [73].

4.1.4 Anterior Cingulate The ACC plays a prominent role in pain processing. This region
Cortex (ACC) receives thalamo-cortical input from nociceptive neurons in the
thalamus and contains nociceptive-specific neurons responsive to
noxious stimuli [87]. Additionally, the ACC is implicated in medi-
ating antinociceptive responses as it contains high numbers of opi-
ate receptors [88, 89].
Historically, the ACC was considered key to affective process-
ing, as it was classified along with the retrosplenial cortex, hippo-
campus, amygdala, and several basal forebrain structures as part of
506 Emma G. Duerden et al.

Fig. 1 Pain (a) and temperature (d) processing brain regions: brain areas with significantly increased activation
during noxious than innocuous stimulation and during warm stimulation are coded red, while regions coded
blue show significantly increased activation during innocuous than noxious stimulation and during cold stimu-
lation. Insular clusters (seed clusters) with pain- (b) and temperature- (e) specific activity divided in aINS
(green and yellow) and pINS (red and blue), that were used for the insular functional connectivity analysis: both
aINS and pINS were functionally connected to a large brain network, which predominantly includes areas
involved in nociception and thermoception (SI, SII, cingulate gyrus, PFC, and parietal association cortices).
Comparison of pain- (c) and temperature- (f) specific functional connectivity of the two insular areas (areas
with significantly stronger functional connectivity to aINS than to pINS are coded red–yellow, while areas with
significantly stronger functional connectivity to pINS than to aINS are coded blue–green): the aINS was more
strongly connected to PFC and to ACC than was pINS; pINS meanwhile was more strongly connected to SI and
to the primary motor cortex. From [79] with permission

the limbic lobe, which was considered central in mediating emotion

[90]. Likewise, the ACC was targeted for surgical lesions to allevi-
ate the suffering of chronic pain [91]; patients reported that they
still experienced the pain they felt prior to surgery, but its emo-
tional unpleasantness was dampened [92].
The ACC is subdivided cytoarchitectonically into several
Brodmann areas (BA), namely 24 and 32 [93], with two further
 fMRI of Pain 507

subdivisions: BA33 located in the perigenual region, and BA25

located in the subcallosal region. The ACC is functionally divided,
rather independent of the cytoarchitectonic borders, into a caudal
cognitive division involved in attention (BA24 and BA32) and a ros-
tral affective division, which is more involved in emotional processes
(BA24, 25, 33) [94, 95]. Dissociation between the cognitive divi-
sion and pain-related processing region was elegantly demonstrated
using fMRI by Davis et al. [96], who compared BOLD activation
evoked by noxious stimuli to that seen during a demanding cogni-
tive task. Activation associated with the noxious stimuli was found to
be inferior and caudal to that produced by the cognitive task.
The first direct evidence for the role of ACC in processing
affective components of pain came from a PET study, in which
subjects under hypnosis were instructed to modulate the perceived
unpleasantness of a painful stimulus while maintaining perceived
pain intensity [97]. Results showed that activation of the ACC was
highly correlated with the subjects’ ratings of pain unpleasantness,
while activation of the SI was unaltered by emotional processes.
More recently, a resting state (RS) functional connectivity (FC)
study [98] demonstrated that the dorsal ACC is connected with
regions which comprise the affective/motivational network (ante-
rior insula and medial thalamus), the cognitive/evaluative network
(dorsolateral prefrontal cortex, inferior parietal lobule), and the
motor network (pre-supplementary motor area (SMA), and SMA).
Nevertheless, these imaging and lesion data should not be
interpreted too rigidly, since the ACC has been shown to have
some sensory-discriminative characteristics, such as a crude noci-
ceptive somatotopic organization [99]. Furthermore, reductions
in both pain intensity and unpleasantness have been described fol-
lowing a neurosurgical capsulotomy—interruption of fiber tracts
to the ACC [3]. The rostral ACC has also been supposed to play a
relevant role in coding the variability of pain perception [100].

4.1.5 Prefrontal Cortex Regions of the PFC have been implicated in both pain processing
(PFC) and pain modulation. PFC activation seen in brain imaging studies
of pain is believed to reflect attention toward the stimuli [69, 101],
but it has also been shown to be directly involved in modulating
responses to painful stimuli. Recently, Lobanov et al. [102] dem-
onstrated that attention to both spatial and intensity feature of the
noxious stimulus was associated with activation of fronto-parietal
areas, including the PFC.
Functional imaging studies have shown that activity in the
PFC reduces the pain magnitude or hyperalgesia, suggesting that
the PFC can regulate the amount of pain an individual perceives.
Activity in the PFC is also associated with episodes of emotional
detachment, when the “suffering” element of pain is absent.
Negative emotional responses can heighten the experience of pain,
and the ventrolateral PFC seems to regulate these responses via
interactions with the nucleus accumbens and the amygdala [103].
508 Emma G. Duerden et al.

Using fMRI, Wager et al. have demonstrated increased PFC

activity during the anticipation of pain, which was interpreted as a
preemptive anticipatory response triggering a descending modula-
tion of the pending nociceptive signals via activation of midbrain
structures [104].
PFC activity is consistently seen in studies employing experi-
mental models of chronic pain. Most commonly, sensitization was
associated with a signal increase in the DLPFC. The functional
significance of this activation is, however, still under debate: a posi-
tive correlation with the unpleasantness of pain indicates that
DLPFC activation reflects altered cognitive-affective processing in
the pathological pain state. Increased DLPFC activations might
also reflect the recruitment of endogenous mechanisms of pain
control [77].

4.1.6 Amygdala The amygdala, buried beneath the uncus and located at the tail of
the caudate nucleus, is a key limbic structure involved in the pro-
cessing of emotional stimuli. The amygdala is suited for such pro-
cessing as it is the sole subcortical structure to receive projections
from every sensory area.
Functional neuroimaging studies utilizing various types of
aversive stimuli including pain, habitually report amygdala activa-
tion [105]. Studies using fMRI have demonstrated that amygdala
activation is associated with extremely unpleasant noxious stimuli,
suggesting an involvement of this region in processing the affective
component of pain [106, 107]. Other evidence from fMRI has
implicated the amygdala in processing uncertainty associated with
painful stimuli [108].

4.1.7 Brainstem In addition to cortical regions, a host of midbrain structures are

also involved in processing pain affect including the PAG, superior
colliculus, red nucleus, nucleus cuneiformis, Edinger-Westphal
nucleus, nucleus of Darkschewitsch, pretectal nuclei, interstitial
nucleus, and intercolliculus nucleus [109].
Several of these structures are involved in pain modulation—
the best characterized being the PAG. The PAG surrounds the
cerebral aqueduct in the midbrain. Inhibitory encephalin-
containing neurons in the PAG disinhibit local interneurons and in
turn excite neurons in the rostral ventral medulla (RVM) and/or
the locus coeruleous (LC). The aminergic efferents from the RVM
and LC then project to the spinal cord and dampen pain transmis-
sion in dorsal horn neurons through several different mechanisms
[103, 110–112]. Activity within the RVM can provide important
antinociceptive effects, which can be beneficial during stressful cir-
cumstances. However, the RVM can also enhance nociception fol-
lowing inflammation and injury. Clearly, this pronociceptive effect
is protective during recovery from injury and promotes tissue
repair, but the failure of such an effect to resolve after the tissue has
healed may result in chronic pain [103].
 fMRI of Pain 509

Fig. 2 (a) Correlation analysis between the BOLD response in the PAG and pain threshold, recorded in seconds,
during the cold pressor test: the pain threshold directly correlated with BOLD activation in the PAG (cluster level
corrected threshold p < 0.05, Pearson’s r = 0.63). (b) Correlation analysis between the BOLD response in the
PAG and pain intensity ratings, as assessed by the numerical rating scale, during the cold pressor test: the pain
rating inversely correlated with BOLD activation in the PAG (cluster level corrected threshold p < 0.05, Pearson’s
r = 0.45). From [113] with permission

Activity within the PAG and the mesencephalic pontine reticular

formation (MPRF) has been strongly associated with the develop-
ment of hyperalgesia, and evidence suggests that the MPRF is spe-
cifically involved in the maintenance of central sensitization [103].
However, recently, La Cesa et al. showed that the greater the
PAG activation the higher the pain threshold and the weaker the
pain intensity perceived, thus highlighting the key role of the PAG
in inhibiting the pain afferent pathway function [113] (Fig. 2).
The sensitivity and in-plane resolution of 1.5 and 3.0 T MRI
scanners are limited in their ability to resolve fine spatial localiza-
tion of many brainstem structures. In addition, brainstem func-
tional imaging is also limited by image distortion and is susceptible
510 Emma G. Duerden et al.

to local magnetic field inhomogeneities and pulsation artifacts.

Therefore, optimized approaches to study brainstem fMRI are
needed [114–116].

4.1.8 Motor Cortices A number of other cortical and subcortical regions are commonly
activated during fMRI studies of pain including many regions
involved in motor processing. Motor regions include the primary
motor cortex, premotor cortex, supplementary motor area,
cerebellum, and basal ganglia. Frequently, these regions are con-
comitantly activated along with those involved with affective and
sensory aspects of pain processing [117].
The perception of a painful stimulus involves an orienting
response and subsequent retraction of the body part being tar-
geted. Activation of motor areas during functional neuroimaging
studies is believed to reflect motor preparatory responses. However,
several of these areas, such as the nuclei associated with the basal
ganglia, are directly responsive to noxious stimuli [118, 119].
Using fMRI, a reliable somatotopic organization has been shown
in the putamen [120, 121] in response to noxious stimuli, which
indicates that this region may be involved in sensory-discriminative
processing of pain.

4.1.9 Thalamus Higher-resolution imaging studies coupled to surgical investiga-

tions have confirmed the relevance of thalamic nuclei in nocicep-
tive processing. As a critical relay site, it is not surprising that the
thalamus is implicated in chronic pain [8]. In particular, recent
evidence indicates that neuroinflammation in the thalamus might
contribute to chronic pain states [122].

4.2 Spinal Cord To date, only a few reports have assessed the feasibility of studying
Processing nociception using fMRI of the spinal cord [123]. One study by
of Nociceptive Stimuli Brooks et al. [14] examined the spinal nociceptive signal at 1.5 T
in response to noxious heat pain stimuli. Using a tailored, high-
resolution scanning protocol and postprocessing techniques for
controlling physiological noise, they demonstrated reliable pain-
related activation in the ipsilateral dorsal horn.
A recent connectivity analysis revealed functional coupling
between the spinal cord dorsal horn and typical ascending thalamo-
cortical pain pathways. More importantly, the spinal cord was also
functionally connected with brain regions involved in descending
pain modulation, such as the PAG. A positive correlation between
the individual strength of connectivity within this descending pain
modulatory pathway and the behavioral pain ratings was found,
thus pointing to the functional relevance of this system during the
processing of physiological nociceptor pain [124]. In a similar
study, BOLD fMRI response in the spinal cord was correlated with
individual pain ratings, further supporting the contribution of
spinal cord activity to the perception of pain [125].
 fMRI of Pain 511

5 fMRI and the Study of Higher Cognitive Pain Processing

5.1 Pain Modulation What is clear from several studies is that nociceptive information
processing, and consequent pain perception, is subject to signifi-
cant pro- and anti-nociceptive modulations that can be influenced
dramatically by cognitive, emotional, and contextual factors [126].
Pain modulation can occur through both endogenous mecha-
nisms and as a result of exogenously administered agents. One final
common pathway for analgesic mechanisms is believed to be through
the release of endogeneous opioids [127] acting on sites in the
brainstem and midbrain that block the nociceptive signal through
their descending pathways; the final effects of this descending mod-
ulation are exerted either on the spinal cord and/or at the site of
peripheral nerves that transmit the nociceptive stimuli. Additionally,
recent research has implicated endocannibinoids in pain modula-
tion, which may act on similar descending pathways [128].
fMRI is a useful tool for examining cerebral mechanisms of pain
modulation, whereby subjects experience either analgesia or hyperal-
gesia—a decrease or increase in perceived pain, respectively. First, the
anatomical resolution of fMRI is sufficient to localize some of the
small brain regions involved in pain modulation, such as the RVM or
PAG [112, 129], and the temporal resolution allows an assessment of
the time course of activations within those regions. fMRI is also well
suited to study procedures that evoke changes in pain perception
since it accommodates the use of parametric data, whereby experi-
mental parameters such as pain ratings (intensity, expectation,
unpleasantness) can be correlated with brain activations and thus
used to characterize cortical structures according to their response
profile to various experimental parameters. As a corollary of increased
temporal resolution, a major advantage of using fMRI to study pain
modulation is the possibility of utilizing event-related designs
whereby the time course of brain activations over different phases of
the modulation period can be studied—the anticipation of the nox-
ious stimulus, the onset of pain perception, changes in pain percep-
tion over time, and post-stimulus ratings. Anticipation of the painful
stimulus is a crucial phase of the pain modulation process, since at
this time point neural mechanisms act on descending modulatory
systems to diminish or enhance the response to the stimulus [130].
fMRI has been widely applied to study modulatory processes
triggered either through endogenous mechanisms utilizing cogni-
tive strategies, such as attention [77, 131], hypnosis [132–134],
placebo and nocebo effects [104, 135], or through exogenous
agents, such as pharmacological [126, 136–138] and non pharma-
cological interventions [139].
Several lines of evidence strengthen the notion that pain mod-
ulation occurs via an integrated “frontal to brainstem to spinal
cord” system. Disruption of the descending pain modulatory
system may represent a point of vulnerability for the development
and maintenance of chronic pain [126].
512 Emma G. Duerden et al.

5.2 Pain Empathy Inherent to processing the emotional component of pain is the
ability to understand the emotional reactions of other people who
are experiencing pain—i.e., pain empathy [140]. This rapidly
growing field of empathy research is directed toward studying the
mental representation of pain—both that which is perceived to be
experienced by others, as well as that which is perceived as one’s
own. Several different types of experimental stimuli implicating
other people in pain have been used in these fMRI paradigms,
including photographic images [141–145], or short animations
[146] of body parts in potentially tissue-damaging situations, view-
ing the faces of actors evoking facial expressions of pain [147], or
subjects actually receiving painful stimuli [148], or those of chronic
pain patients [149], or being cued that a loved one in the room
was receiving painful stimuli [150].
A common finding from these studies is that the processing of
pain in others recruits brain regions involved in affective processing—
namely the ACC and insula. In a recent meta-analysis, Lamm et al.
[151] compiled brain activation coordinates from 32 studies that
had investigated empathy for pain using fMRI. Authors identified a
core network, consisting of bilateral anterior insular cortex and
medial/anterior cingulate cortex, that was associated with empathy
for pain. Activation in these areas overlapped with activation during
directly experienced pain, thus linking their involvement to repre-
senting global feeling states and the guidance of adaptive behavior
for both self- and other-related experiences. Moreover, the analysis
demonstrates that—depending on the type of experimental para-
digm—this core network was coactivated with distinct brain regions:
viewing pictures of body parts in painful situations recruited areas
underpinning action understanding (inferior parietal/ventral pre-
motor cortices) to a stronger extent; eliciting empathy by means of
abstract visual information about the other’s affective state more
strongly engaged areas associated with inferring and representing
mental states of self and others (precuneus, ventral medial prefrontal
cortex, superior temporal cortex, and temporo-parietal junction).
Several transcranial magnetic stimulation studies reported modu-
lation of sensory-discriminative regions associated with pain empathy,
thus suggesting a somatotopic specificity in the perceived pain of oth-
ers. Although early functional imaging studies suggested that somato-
sensory areas contribute very little to the neural response when seeing
others’ pain or empathizing with it, Morrison et al. have recently dem-
onstrated that just viewing others’ painful actions biases participants
to report tactile stimulation even when none occurred [145]. Such
discrepancies might have resulted from differences in experimental
paradigms, since it was shown that only the picture-based paradigms
activated somatosensory areas during empathy for pain [151].
fMRI has provided considerable insight into the neural mecha-
nisms of processing pain in others, and suggests a number of inter-
esting clinical implications. Since pain is a sensory and emotional
 fMRI of Pain 513

phenomenon that is primarily experienced by the patient—as

opposed to an easily measured sign of illness, such as fever or
weight loss, for example—health-care professionals who are con-
fronted with patients in pain must be able to infer their discomfort
accurately and treat them accordingly. Further understanding of
the neural mechanisms underlying how we interpret pain in others
is an initial step toward how these neural circuits can change—
depending on the clinical context or after years of repeated expo-
sure to those in pain.

6 Combining fMRI with Morphometry

For the study of nociceptive processing and pain perception, MRI-

based morphometric analyses can be used to examine neuroana-
tomical changes that are correlated with particular chronic pain
states or to examine differences in the anatomy of specific brain
regions that may underlie the variability in pain perception that is
observed within a population in healthy volunteers. A number of
recent studies have reported changes in cortical and subcortical
brain regions in individuals with chronic pain [152, 153].
To date, a few studies have combined morphometric and func-
tional neuroimaging analysis. The combination of functional and
structural brain measures has revealed that patients with fibromyal-
gia had overlapping decreases of cortical thickness, brain volumes,
and regional functional activity in the rACC [154]. These findings
provide a neuroanatomical basis for reduced cortical activity,
strengthening the importance of relating anatomical structure to
physiological function.
However the future of pain fMRI lies in the development of
complimentary brain imaging analysis techniques to improve our
understanding of pain processing.

7 fMRI as a Therapy for Chronic Pain

Recent improvements in the speed of analysis of fMRI data have

led to the possibility that “real-time” fMRI (rt-fMRI) can be devel-
oped as a potential “therapy” for chronic pain patients. In princi-
ple, if patients can be given feedback regarding the level of activity
in specific areas of the brain that are associated with the perception
of pain or its unpleasantness, then learning to (self)-regulate this
activity can allow them to control their own chronic pain—in much
the same way as neurosurgeons attempt to control a patient’s pain
by stimulating a specific area of the brain or by placing a lesion in a
targeted area. Self-regulation training with EEG has provided the
basis for much of the neurofeedback research; however, due to
several methodological limitations, EEG offers relatively poor
514 Emma G. Duerden et al.

spatial specificity within the brain [155, 156]. By contrast, fMRI

offers superior spatial resolution, especially for deeper brain
regions, and is more suitable for targeting activity in a small, local-
ized brain region [157, 158].
Neurofeedback, using real-time analysis of fMRI data, was ini-
tially developed by Cox et al. [159], and several groups have used this
technology to study learned control over brain activity during a num-
ber of tasks [160–163]. Recently, the use of rt-fMRI has been applied
to several clinical conditions whose etiology or symptoms might be
linked to abnormal activity in known areas of the brain, including pain
syndromes [11, 164]. In one study testing the feasibility of rt-fMRI as
a neuroimaging therapy for chronic pain patients [10], normal sub-
jects receiving experimental noxious stimuli were trained to control
activity in a targeted region within the ACC. Results demonstrated
that these subjects were able to use the feedback provided by rt-fMRI
to either increase or decrease, on command, ACC activity, and that
the level of this activity correlated with their estimates of pain evoked
by the experimental stimuli. Likewise, a small cohort of chronic pain
patients, following a similar rt-fMRI training paradigm, reported a
significant reduction in their level of chronic pain in comparison to
that of a control patient group, which received feedback training
based only on autonomic measures. Furthermore, the patients in the
rt-fMRI group demonstrated a direct correlation between their ability
to control ACC activation and their degree of pain reduction.
In the future, rt-fMRI could also be applied to modify corti-
cal hyperactivity that has been described for a number of other
pain syndromes.

8 Future of Pain Imaging

The last decade has seen a considerable improvement in the sensi-

tivity of fMRI in both the spatial and temporal localization of
regions of activation. Moreover, the shift to higher field strengths
of 4.0 and 7.0 T scanners has been shown to significantly enhance
the SNR, compared to that observed with the 1.5–3.0 T scanners,
which have been used in most pain studies. Pain imaging is poised
to benefit from these advances more than other disciplines,
because—unlike visual or motor tasks, for example, which produce
changes in CBF on the order of ~40 % [165], BOLD response to
nociceptive stimuli produce signal changes only in the range of
~5 % [54]. Improved spatial localization of fMRI pain protocols
would provide better information regarding the specificity of
somatosensory regions involved in noxious processing and their
somatotopic organization; likewise, improved spatial localization
and SNR will aid greatly to investigations of small brainstem struc-
tures that have been implicated in modulating pain processing at
both spinal and supra-spinal levels.
 fMRI of Pain 515

Another burgeoning field in pain imaging is that of arterial

spin labeling (ASL) perfusion MRI, which was first described
more than a decade ago [166]. ASL directly measures CBF by
magnetically labeling water molecules in inflowing arteries. Recent
application of ASL to study experimental pain in healthy subjects
[167, 168] has shown that this technique offers several advan-
tages. ASL gives a precise localization of neuronal structures and
has demonstrated great inter-individual reliability of activation.
Additionally, compared with BOLD fMRI, ASL is well suited for
pain imaging studies, since it is less susceptible to signal loss and
image distortions [169] due to magnetic field inhomogeneities at
the air–tissue interface around frontal, medial, and inferior tempo-
ral lobes [170, 171]. Although several methods are available to
reduce these susceptibility artifacts in the BOLD signal, ASL is
nevertheless an attractive alternative for pain studies that target
the limbic system where signal loss from susceptibility artifacts is
troublesome for such regions as the orbitofrontal cortex and
amygdala. ASL also has the additional advantage of permitting
longer acquisition times and is thus well suited for studying neu-
ronal processing that may take longer to develop, such as pain
modulation through hypnotic induction; fMRI, on the contrary,
is limited in terms of its length of acquisition due to drifts in the
baseline. However, ASL is limited in that it cannot detect changes
occurring faster than 30 s and is therefore not suited for event-
related designs. Additionally, the technique is limited by its tem-
poral resolution and slice coverage in which whole brain imaging
is not possible using current methods. These issues should be
resolved with advances made in fast echo planar imaging sequences.

9 Conclusions

The experience of pain is complex: both sensory and cognitive

components depend on a network of neural processing spread
throughout many cortical and subcortical regions of the CNS. The
advent of noninvasive imaging techniques has allowed us to gain a
deep understanding of this multifaceted phenomenon in humans—
the experimental preparation that is most relevant to our ultimate
goal of understanding, managing, and alleviating pain in patients.
Pain is a characteristic common to many diseases and injuries, a
consequence of many medical and dental procedures, and chronic
pain is essentially a syndrome in its own right—an insufferable sen-
sation that many times has no obvious stimulus. fMRI in human
subjects is helping us to understand the cerebral mechanisms of
pain processing and the modulation of pain by both endogenous
and exogenous factors. The results of these studies are making sub-
stantial contributions to the development of efficacious interven-
tions for treating and alleviating pain.
516 Emma G. Duerden et al.

References
1. Penfield W, Boldrey E (1937) Somatic motor to painful stimuli during sphygmomanom-
and sensory representation in the cerebral etry. Int J Psychophysiol 33(3):253–257
cortex of man as studied by electrical stimula- 17. Mobascher A et al (2010) Brain activation
tion. Brain 60(4):389–443 patterns underlying fast habituation to painful
2. Head H, Holmes G (1911) Sensory dis- laser stimuli. Int J Psychophysiol 75(1):16–24
turbances from cerebral lesions. Brain 18. Becerra LR et al (1999) Human brain acti-
34(2–3):102–254 vation under controlled thermal stimulation
3. Talbot J et al (1991) Multiple representations and habituation to noxious heat: an fMRI
of pain in human cerebral cortex. Science study. Magn Reson Med 41(5):1044–1057
251(4999):1355–1358 19. Price DD et al (1994) A comparison of pain
4. Jones AKP et al (1991) Cortical and subcor- measurement characteristics of mechanical
tical localization of response to pain in man visual analogue and simple numerical rating
using positron emission tomography. Proc R scales. Pain 56(2):217–226
Soc B Biol Sci 244(1309):39–44 20. Rainville P et al (2004) Rapid deterioration
5. Apkarian AV et al (1992) Persistent pain of pain sensory-discriminative information in
inhibits contralateral somatosensory cor- short-term memory. Pain 110(3):605–615
tical activity in humans. Neurosci Lett 21. Charron J, Rainville P, Marchand S (2006)
140(2):141–147 Direct comparison of placebo effects on
6. Davis KD et al (1995) fMRI of human clinical and experimental pain. Clin J Pain
somatosensory and cingulate cortex dur- 22(2):204–211
ing painful electrical nerve stimulation. 22. Price DD et al (1999) An analysis of factors
Neuroreport 7(1):321–325 that contribute to the magnitude of placebo
7. Flor H (2000) The functional organization analgesia in an experimental paradigm. Pain
of the brain in chronic pain. Prog Brain Res 83(2):147–156
129:313–322 23. Apkarian AV et al (1999) Differentiating corti-
8. Tracey I, Mantyh PW (2007) The cerebral cal areas related to pain perception from stim-
signature for pain perception and its modula- ulus identification: temporal analysis of fMRI
tion. Neuron 55(3):377–391 activity. J Neurophysiol 81(6):2956–2963
9. Wager TD et al (2013) An fMRI-based neu- 24. Porro CA et al (2004) Percept-related activ-
rologic signature of physical pain. N Engl ity in the human somatosensory system: func-
J Med 368(15):1388–1397 tional magnetic resonance imaging studies.
10. deCharms RC et al (2005) Control over Magn Reson Imaging 22(10):1539–1548
brain activation and pain learned by using 25. Andrew D, Greenspan JD (1999) Peripheral
real-time functional MRI. Proc Natl Acad Sci coding of tonic mechanical cutaneous pain:
102(51):18626–18631 comparison of nociceptor activity in rat
11. Rance M et al (2014) Real time fMRI feed- and human psychophysics. J Neurophysiol
back of the anterior cingulate and posterior 82(5):2641–2648
insular cortex in the processing of pain. Hum 26. Adriaensen H et al (1984) Nociceptor dis-
Brain Mapp 35(12):5784–5798 charges and sensations due to prolonged nox-
12. Chen JI et al (2002) Differentiating noxious- ious mechanical stimulation--a paradox. Hum
and innocuous-related activation of human Neurobiol 3(1):53–58
somatosensory cortices using temporal analy- 27. Gallez A et al (2005) Attenuation of sensory
sis of fMRI. J Neurophysiol 88(1):464–474 and affective responses to heat pain: evidence
13. Iramina K et al (1999) Effects of stimulus for contralateral mechanisms. J Neurophysiol
intensity on fMRI and MEG in somatosen- 94(5):3509–3515
sory cortex using electrical stimulation. IEEE 28. Bingel U et al (2007) Habituation to painful
Trans Magn 35(5):4106–4108 stimulation involves the antinociceptive sys-
14. Brooks J, Tracey I (2005) From nociception tem. Pain 131(1):21–30
to pain perception: imaging the spinal and 29. Valeriani M et al (2003) Reduced habituation
supraspinal pathways. J Anat 207(1):19–33 to experimental pain in migraine patients:
15. Mackey S et al (2006) FMRI evidence of nox- a CO2 laser evoked potential study. Pain
ious thermal stimuli encoding in the human 105(1):57–64
spinal cord. J Pain 7(4):S25 30. Nickel FT et al (2013) Brain correlates of
16. Rollnik JD, Schmitz N, Kugler J (1999) short-term habituation to repetitive electrical
Anxiety moderates cardiovascular responses noxious stimulation. Eur J Pain 18(1):56–66
 fMRI of Pain 517

31. Willis WD Jr (1985) The pain system. The DR (ed) The skin senses. Charles C. Thomas
neural basis of nociceptive transmission in the Publishers, Springfield, IL, pp 423–443
mammalian nervous system. Pain Headache 47. Tracey I (2008) Imaging pain. Br J Anaesth
8:1–346 101(1):32–39
32. Adriaensen H et al (1983) Response properties 48. Kaas J et al (1979) Multiple represen-
of thin myelinated (A-delta) fibers in human tations of the body within the primary
skin nerves. J Neurophysiol 49(1):111–122 somatosensory cortex of primates. Science
33. Ochoa J, Torebjörk E (1989) Sensations 204(4392):521–523
evoked by intraneural microstimulation of 49. Kenshalo DR Jr, Isensee O (1983) Responses
C nociceptor fibres in human skin nerves. of primate SI cortical neurons to noxious
J Physiol 415(1):583–599 stimuli. J Neurophysiol 50(6):1479–1496
34. Cervero F, Iggo A (1980) The substantia 50. Casey KL et al (1996) Comparison of human
gelatinosa of the spinal cord: a critical review. cerebral activation pattern during cutane-
Brain 103(4):717–772 ous warmth, heat pain, and deep cold pain.
35. Wilson P, Kitchener PD (1996) Plasticity J Neurophysiol 76(1):571–581
of cutaneous primary afferent projections 51. Gelnar PA et al (1998) Fingertip representa-
to the spinal dorsal horn. Prog Neurobiol tion in the human somatosensory cortex: an
48(2):105–129 fMRI study. Neuroimage 7(4):261–283
36. Craig AD et al (1994) A thalamic nucleus 52. Liang M, Mouraux A, Iannetti GD (2011)
specific for pain and temperature sensation. Parallel processing of nociceptive and non-
Nature 372(6508):770–773 nociceptive somatosensory information in the
37. Legrain V et al (2011) The pain matrix human primary and secondary somatosensory
reloaded: a salience detection system for the cortices: evidence from dynamic causal mod-
body. Prog Neurobiol 93(1):111–124 eling of functional magnetic resonance imag-
38. Bromm B, Treede RD (1984) Nerve fibre ing data. J Neurosci 31(24):8976–8985
discharges, cerebral potentials and sensa- 53. Cheng JC et al (2015) Individual dif-
tions induced by CO2 laser stimulation. Hum ferences in temporal summation of pain
Neurobiol 3(1):33–40 reflect pronociceptive and antinociceptive
39. Carmon A, Dotan Y, Sarne Y (1978) brain structure and function. J Neurosci
Correlation of subjective pain experience with 35(26):9689–9700
cerebral evoked responses to noxious thermal 54. Derbyshire GSW, Jones PAK (1998) Cerebral
stimulations. Exp Brain Res 33(3–4):445–453 responses to a continual tonic pain stimulus
40. Iannetti GD et al (2004) Aδ nociceptor measured using positron emission tomogra-
response to laser stimuli: selective effect phy. Pain 76(1):127–135
of stimulus duration on skin temperature, 55. Disbrow E et al (1998) Somatosensory cor-
brain potentials and pain perception. Clin tex: a comparison of the response to noxious
Neurophysiol 115(11):2629–2637 thermal, mechanical, and electrical stimuli
41. Spiegel J, Hansen C, Treede RD (2000) using functional magnetic resonance imaging.
Clinical evaluation criteria for the assess- Hum Brain Mapp 6(3):150–159
ment of impaired pain sensitivity by thulium- 56. Vierck CJ et al (2013) Role of primary
laser evoked potentials. Clin Neurophysiol somatosensory cortex in the coding of pain.
111(4):725–735 Pain 154(3):334–344
42. Leandri M et al (2006) Measurement of skin 57. Tommerdahl M et al (1996) Anterior pari-
temperature after infrared laser stimulation. etal cortical response to tactile and skin-
Neurophysiol Clin 36(4):207–218 heating stimuli applied to the same skin site.
43. Helmchen C et al (2008) Common neural J Neurophysiol 75(6):2662–2670
systems for contact heat and laser pain stimu- 58. Bushnell MC et al (1999) Pain perception: is
lation reveal higher-level pain processing. there a role for primary somatosensory cor-
Hum Brain Mapp 29(9):1080–1091 tex? Proc Natl Acad Sci 96(14):7705–7709
44. Iannetti GD, Mouraux A (2010) From the 59. Seminowicz DA, Mikulis DJ, Davis KD
neuromatrix to the pain matrix (and back). (2004) Cognitive modulation of pain-related
Exp Brain Res 205(1):1–12 brain responses depends on behavioral strat-
45. Apkarian AV et al (2005) Human brain mech- egy. Pain 112(1):48–58
anisms of pain perception and regulation in 60. Oshiro Y et al (2007) Brain mechanisms
health and disease. Eur J Pain 9(4):463–484 supporting spatial discrimination of pain.
46. Melzack R, Casey KL (1968) Sensory, moti- J Neurosci 27(13):3388–3394
vational and central control determinants of 61. Andersson JLR et al (1997) Somatotopic
pain: a new conceptual model. In: Kenshalo organization along the central sulcus, for
518 Emma G. Duerden et al.

pain localization in humans, as revealed by 77. Wiech K, Ploner M, Tracey I (2008)

positron emission tomography. Exp Brain Res Neurocognitive aspects of pain perception.
117(2):192–199 Trends Cogn Sci 12(8):306–313
62. DaSilva AF et al (2002) Somatotopic activa- 78. Maihöfner C, Handwerker HO (2005)
tion in the human trigeminal pain pathway. Differential coding of hyperalgesia in the
J Neurosci 22(18):8183–8192 human brain: a functional MRI study.
63. Ogino Y, Nemoto H, Goto F (2005) Neuroimage 28(4):996–1006
Somatotopy in human primary somatosen- 79. Peltz E et al (2011) Functional connectivity of
sory cortex in pain system. Anesthesiology the human insular cortex during noxious and
103(4):821–827 innocuous thermal stimulation. Neuroimage
64. Kaas JH (1983) What, if anything, is SI? 54(2):1324–1335
Organization of first somatosensory area of 80. Mazzola L, Isnard J, Mauguiere F (2006)
cortex. Physiol Rev 63(1):206–231 Somatosensory and pain responses to stimu-
65. Albanese MC et al (2007) Memory traces lation of the second somatosensory area (SII)
of pain in human cortex. J Neurosci in humans. A comparison with SI and insular
27(17):4612–4620 responses. Cereb Cortex 16(7):960–968
66. Schmid J et al (2015) Neural underpinnings of 81. Segerdahl AR et al (2015) The dorsal pos-
nocebo hyperalgesia in visceral pain: a fMRI study terior insula subserves a fundamental role in
in healthy volunteers. Neuroimage 120:114–122 human pain. Nat Neurosci 18(4):499–500
67. Greenspan JD, Lee RR, Lenz FA (1999) 82. Craig AD (2002) New and old thoughts on
Pain sensitivity alterations as a function of the mechanisms of spinal cord injury pain. In:
lesion location in the parasylvian cortex. Pain Yezierski RP, Burchiel KJ (eds) Spinal cord
81(3):273–282 injury pain: assessment, mechanisms, man-
68. Ploner M, Freund HJ, Schnitzler A (1999) agement. IASP Press, Seattle 237–264
Pain affect without pain sensation in a patient 83. Blomqvist A (2000) Cytoarchitectonic and
with a postcentral lesion. Pain 81(1):211–214 immunohistochemical characterization of a
69. Coghill RC et al (1999) Pain intensity pro- specific pain and temperature relay, the poste-
cessing within the human brain: a bilateral, rior portion of the ventral medial nucleus, in
distributed mechanism. J Neurophysiol the human thalamus. Brain 123(3):601–619
82(4):1934–1943 84. Peyron R et al (2002) Role of operculoinsular
70. Maihöfner C, Herzner B, Otto Handwerker cortices in human pain processing: converg-
H (2006) Secondary somatosensory cortex ing evidence from PET, fMRI, dipole mod-
is important for the sensory-discriminative eling, and intracerebral recordings of evoked
dimension of pain: a functional MRI study. potentials. Neuroimage 17(3):1336–1346
Eur J Neurosci 23(5):1377–1383 85. Brooks JCW et al (2005) Somatotopic organ-
71. Gracely RH et al. (2004) Pain catastrophizing isation of the human insula to painful heat
and neural responses to pain among persons studied with high resolution functional imag-
with fibromyalgia. Brain 127(4):835–843 ing. Neuroimage 27(1):201–209
72. Sawamoto N et al (2000) Expectation of 86. Henderson LA, Rubin TK, Macefield VG
pain enhances responses to nonpainful (2011) Within-limb somatotopic representa-
somatosensory stimulation in the anterior tion of acute muscle pain in the human con-
cingulate cortex and parietal operculum/pos- tralateral dorsal posterior insula. Hum Brain
terior insula: an event-related functional mag- Mapp 32(10):1592–1601
netic resonance imaging study. J Neurosci 87. Hutchison WD et al (1999) Pain-related
20(19):7438–7445 neurons in the human cingulate cortex. Nat
73. Oertel BG et al (2012) Separating brain pro- Neurosci 2(5):403–405
cessing of pain from that of stimulus intensity. 88. Jones AKP et al (1991) In vivo distribution
Hum Brain Mapp 33(4):883–894 of opioid receptors in man in relation to the
74. Wiech K et al (2014) Differential structural cortical projections of the medial and lateral
and resting state connectivity between insu- pain systems measured with positron emission
lar subdivisions and other pain-related brain tomography. Neurosci Lett 126(1):25–28
regions. Pain 155(10):2047–2055 89. Baumgärtner U et al (2007) High opiate
75. Schilder P, Stengel E (1932) Asymbolia for receptor binding potential in the human lat-
pain. Arch Neurol Psychiatry 25(3):598–600 eral pain system: a (FEDPN)PET study. Clin
Neurophysiol 118(4):e12
76. Berthier M, Starkstein S, Leiguarda R (1988)
Asymbolia for pain: a sensory-limbic discon- 90. Pessoa L (2008) On the relationship between
nection syndrome. Ann Neurol 24(1):41–49 emotion and cognition. Nat Rev Neurosci
9(2):148–158
 fMRI of Pain 519

91. Pillay PK, Hassenbusch SJ (1992) Bilateral flow: an fMRI study. Neuropsychobiology
MRI-guided stereotactic cingulotomy for 43(3):175–185
intractable pain. Stereotact Funct Neurosurg 107. Berna C et al (2010) Induction of depressed
59(1–4):33–38 mood disrupts emotion regulation neurocir-
92. Gybels JM, Sweet WH (1989) Neurosurgical cuitry and enhances pain unpleasantness. Biol
treatment of persistent pain. Physiological Psychiatry 67(11):1083–1090
and pathological mechanisms of human pain. 108. Bornhovd K et al (2002) Painful stimuli evoke
Pain Headache 11:1–402 different stimulus–response functions in the
93. Vogt BA et al (1995) Human cingulate cor- amygdala, prefrontal, insula and somatosen-
tex: surface features, flat maps, and cytoarchi- sory cortex: a single-trial fMRI study. Brain
tecture. J Comp Neurol 359(3):490–506 125(6):1326–1336
94. Devinsky O, Morrell MJ, Vogt BA (1995) 109. Schulte LH, Sprenger C, May A (2015)
Contributions of anterior cingulate cortex to Physiological brainstem mechanisms of tri-
behaviour. Brain 118(1):279–306 geminal nociception: an fMRI study at
95. Vogt BA (2005) Pain and emotion interac- 3T. Neuroimage 124(Pt A):518–525
tions in subregions of the cingulate gyrus. 110. Mason P (2005) Deconstructing endog-
Nat Rev Neurosci 6(7):533–544 enous pain modulations. J Neurophysiol
96. Davis KD et al (1997) Functional MRI of 94(3):1659–1663
pain- and attention-related activations in 111. Fields HL, Heinricher MM (1985) Anatomy
the human cingulate cortex. J Neurophysiol and physiology of a nociceptive modulatory
77(6):3370–3380 system. Philos Trans R Soc Lond B Biol Sci
97. Rainville P et al (1997) Pain affect encoded in 308(1136):361–374
human anterior cingulate but not somatosen- 112. Fields HL (2000) Pain modulation: expecta-
sory cortex. Science 277(5328):968–971 tion, opioid analgesia and virtual pain. Prog
98. Wilcox CE et al (2015) The subjective expe- Brain Res 122:245–253
rience of pain: an FMRI study of percept- 113. La Cesa S et al (2014) fMRI pain activation in
related models and functional connectivity. the periaqueductal gray in healthy volunteers
Pain Med 16(11):2121–33 during the cold pressor test. Magn Reson
99. Arienzo D et al (2006) Somatotopy of ante- Imaging 32(3):236–240
rior cingulate cortex (ACC) and supplemen- 114. Dunckley P et al (2005) A comparison
tary motor area (SMA) for electric stimulation of visceral and somatic pain processing
of the median and tibial nerves: an fMRI in the human brainstem using functional
study. Neuroimage 33(2):700–705 magnetic resonance imaging. J Neurosci
100. Kroger IL, Menz MM, May A (2015) 25(32):7333–7341
Dissociating the neural mechanisms of 115. Tracey I, Iannetti GD (2006) Brainstem
pain consistency and pain intensity in the functional imaging in humans. Suppl Clin
trigemino-nociceptive system. Cephalalgia Neurophysiol 58:52–67
[published online before print October 22, 116. Guimaraes AR et al (1998) Imaging subcorti-
2015, doi:10.1177/0333102415612765] cal auditory activity in humans. Hum Brain
101. Casey KL (1999) Forebrain mechanisms of Mapp 6(1):33–41
nociception and pain: analysis through imag- 117. Farina S et al (2003) Pain-related modula-
ing. Proc Natl Acad Sci 96(14):7668–7674 tion of the human motor cortex. Neurol Res
102. Lobanov OV et al (2013) Frontoparietal 25(2):130–142
mechanisms supporting attention to loca- 118. Chudler EH, Dong WK (1995) The role of
tion and intensity of painful stimuli. Pain the basal ganglia in nociception and pain.
154(9):1758–1768 Pain 60(1):3–38
103. Tracey I (2011) Can neuroimaging studies 119. Borsook D et al (2010) A key role of the basal
identify pain endophenotypes in humans? Nat ganglia in pain and analgesia--insights gained
Rev Neurol 7(3):173–181 through human functional imaging. Mol Pain
104. Wager TD et al (2004) Placebo-induced 6:27
changes in FMRI in the anticipation and expe- 120. Tomycz ND, Friedlander RM (2011) The
rience of pain. Science 303(5661):1162–1167 experience of pain and the putamen: a
105. Zald DH (2003) The human amygdala and new link found with functional MRI and
the emotional evaluation of sensory stimuli. diffusion tensor imaging. Neurosurgery
Brain Res Rev 41(1):88–123 69(4):N12–N13
106. Schneider F et al (2001) Subjective ratings of 121. Bingel U et al (2004) Somatotopic repre-
pain correlate with subcortical-limbic blood sentation of nociceptive information in the
520 Emma G. Duerden et al.

putamen: an event-related fMRI study. Cereb 136. Maihöfner C et al (2007) Brain imaging
Cortex 14(12):1340–1345 of analgesic and antihyperalgesic effects of
122. Loggia ML et al (2015) Evidence for brain cyclooxygenase inhibition in an experimental
glial activation in chronic pain patients. Brain human pain model: a functional MRI study.
138(Pt 3):604–615 Eur J Neurosci 26(5):1344–1356
123. Cahill CM, Stroman PW (2011) Mapping of 137. Wise RG et al (2007) The anxiolytic effects
neural activity produced by thermal pain in the of midazolam during anticipation to pain
healthy human spinal cord and brain stem: a revealed using fMRI. Magn Reson Imaging
functional magnetic resonance imaging study. 25(6):801–810
Magn Reson Imaging 29(3):342–352 138. Sanders D et al (2015) Pharmacologic modu-
124. Sprenger C, Finsterbusch J, Buchel C (2015) lation of hand pain in osteoarthritis: a double-
Spinal cord-midbrain functional connectivity blind placebo-controlled functional magnetic
is related to perceived pain intensity: a com- resonance imaging study using naproxen.
bined spino-cortical FMRI study. J Neurosci Arthritis Rheumatol 67(3):741–751
35(10):4248–4257 139. Li K et al (2015) The effects of acupuncture
125. Khan HS, Stroman PW (2015) Inter- treatment on the right frontoparietal network
individual differences in pain processing in migraine without aura patients. J Headache
investigated by functional magnetic reso- Pain 16:518
nance imaging of the brainstem and spinal 140. Thompson E (2001) Empathy and conscious-
cord. Neuroscience 307:231–241 ness. J Conscious Stud 8(5–7):1–32
126. Bingel U, Tracey I (2008) Imaging CNS 141. Jackson PL et al (2006) Empathy examined
modulation of pain in humans. Physiology through the neural mechanisms involved
(Bethesda) 23:371–380 in imagining how I feel versus how you feel
127. Levine JD et al (1978) The narcotic antago- pain. Neuropsychologia 44(5):752–761
nist naloxone enhances clinical pain. Nature 142. Jackson PL, Meltzoff AN, Decety J (2005)
272(5656):826–827 How do we perceive the pain of others? A
128. Hohmann AG, Suplita RL (2006) window into the neural processes involved in
Endocannabinoid mechanisms of pain modu- empathy. Neuroimage 24(3):771–779
lation. AAPS J 8(4):E693–E708 143. Lamm C et al (2007) What are you feeling?
129. Tracey I et al (2002) Imaging attentional Using functional magnetic resonance imaging
modulation of pain in the periaqueductal gray to assess the modulation of sensory and affec-
in humans. J Neurosci 22(7):2748–2752 tive responses during empathy for pain. PLoS
130. Porro CA (2003) Functional imaging and One 2(12):e1292
pain: behavior, perception, and modulation. 144. Moriguchi Y et al (2006) Empathy and judg-
Neuroscientist 9(5):354–369 ing other’s pain: an fMRI study of alexi-
131. Bantick SJ et al (2002) Imaging how atten- thymia. Cereb Cortex 17(9):2223–2234
tion modulates pain in humans using func- 145. Morrison I et al (2013) “Feeling” others’
tional MRI. Brain 125(2):310–319 painful actions: the sensorimotor integration
132. Roder CH et al (2007) Pain response in of pain and action information. Hum Brain
depersonalization: a functional imaging study Mapp 34(8):1982–1998
using hypnosis in healthy subjects. Psychother 146. Morrison I, Peelen MV, Downing PE (2007)
Psychosom 76(2):115–121 The sight of others’ pain modulates motor
133. Schulz-Stübner S et al (2004) Clinical hypno- processing in human cingulate cortex. Cereb
sis modulates functional magnetic resonance Cortex 17(9):2214–2222
imaging signal intensities and pain percep- 147. Simon D et al (2006) Brain responses to
tion in a thermal stimulation paradigm. Reg dynamic facial expressions of pain. Pain
Anesth Pain Med 29(6):549–556 126(1):309–318
134. Nakata H, Sakamoto K, Kakigi R (2014) 148. Botvinick M et al (2005) Viewing facial expressions
Meditation reduces pain-related neural activ- of pain engages cortical areas involved in the direct
ity in the anterior cingulate cortex, insula, sec- experience of pain. Neuroimage 25(1):312–319
ondary somatosensory cortex, and thalamus. 149. Saarela MV et al (2007) The compassionate
Front Psychol 5:1489 brain: humans detect intensity of pain from
135. Tracey I (2010) Getting the pain you another’s face. Cereb Cortex 17(1):230–237
expect: mechanisms of placebo, nocebo 150. Singer T et al (2004) Empathy for pain
and reappraisal effects in humans. Nat Med involves the affective but not sensory compo-
16(11):1277–1283 nents of pain. Science 303(5661):1157–1162
 fMRI of Pain 521

151. Lamm C, Decety J, Singer T (2011) Meta- time functional magnetic resonance imaging
analytic evidence for common and distinct (fMRI): methodology and exemplary data.
neural networks associated with directly Neuroimage 19(3):577–586
experienced pain and empathy for pain. 162. Posse S et al (2003) Real-time fMRI of tem-
Neuroimage 54(3):2492–2502 porolimbic regions detects amygdala activa-
152. Apkarian AV et al (2004) Chronic back pain tion during single-trial self-induced sadness.
is associated with decreased prefrontal and Neuroimage 18(3):760–768
thalamic gray matter density. J Neurosci 163. Weiskopf N (2012) Real-time fMRI and its
24(46):10410–10415 application to neurofeedback. Neuroimage
153. Schmidt-Wilcke T et al (2006) Affective com- 62(2):682–692
ponents and intensity of pain correlate with 164. Guan M et al (2015) Self-regulation of brain
structural differences in gray matter in chronic activity in patients with postherpetic neural-
back pain patients. Pain 125(1):89–97 gia: a double-blind randomized study using
154. Jensen KB et al (2013) Overlapping struc- real-time FMRI neurofeedback. PLoS One
tural and functional brain changes in patients 10(4):e0123675
with long-term exposure to fibromyalgia 165. Ramsey NF et al (1996) Functional map-
pain. Arthritis Rheum 65(12):3293–3303 ping of human sensorimotor cortex with 3D
155. Vernon DJ (2005) Can neurofeedback training BOLD fMRI correlates highly with H215O
enhance performance? An evaluation of the evi- PET rCBF. J Cereb Blood Flow Metab
dence with implications for future research. Appl 16(5):755–759
Psychophysiol Biofeedback 30(4):347–364 166. Detre JA et al (1992) Perfusion imaging.
156. Tao JX et al (2005) Intracranial EEG sub- Magn Reson Med 23(1):37–45
strates of scalp EEG interictal spikes. Epilepsia 167. Owen DG et al (2008) Quantification of
46(5):669–676 pain-induced changes in cerebral blood flow
157. Lantz G et al (2001) Localization of distrib- by perfusion MRI. Pain 136(1):85–96
uted sources and comparison with functional 168. Maleki N et al (2013) Pain response measured
MRI. Epileptic Disord, Special Issue:45–58. with arterial spin labeling. NMR Biomed
158. Stern JM (2006) Simultaneous electroen- 26(6):664–673
cephalography and functional magnetic reso- 169. Wang J et al (2004) Reduced susceptibility
nance imaging applied to epilepsy. Epilepsy effects in perfusion fMRI with single-shot
Behav 8(4):683–692 spin-echo EPI acquisitions at 1.5 tesla. Magn
159. Cox RW, Jesmanowicz A, Hyde JS (1995) Reson Imaging 22(1):1–7
Real-time functional magnetic resonance 170. Devlin JT et al (2000) Susceptibility-induced
imaging. Magn Reson Med 33(2):230–236 loss of signal: comparing PET and fMRI on a
160. Yoo S-S, Jolesz FA (2002) Functional MRI semantic task. Neuroimage 11(6):589–600
for neurofeedback: feasibility study on a hand 171. Ojemann JG et al (1997) Anatomic local-
motor task. Neuroreport 13(11):1377–1381 ization and quantitative analysis of gradient
161. Weiskopf N et al (2003) Physiological self- refocused echo-planar fMRI susceptibility
regulation of regional brain activity using real- artifacts. Neuroimage 6(3):156–167
 Chapter 17

fMRI of the Sensorimotor System

Massimo Filippi, Roberta Messina, and Maria A. Rocca

Abstract
The extensive application of fMRI to the assessment of the human sensorimotor system has disclosed a
complexity that is largely beyond our original understanding. From the available data, it is accepted
that this system consists of a large, and somewhat yet unknown, number of cortical and subcortical
areas, with a precise location and a specialized function. In particular, a large number of regions in the
frontal and parietal lobes contribute to different aspects of motor act performance. It is also evident
that the properties and potentialities of this network still need to be fully elucidated by further research.
Defining how the human sensorimotor system works is of outmost importance for understanding its
dysfunction in case of diseases and also to develop potential therapeutic strategies capable to enhance
its functional plasticity and reserve.

Key words Sensorimotor system, Mirror-neuron system, fMRI, Motor training

1 Introduction

During the past 15 years, fMRI has become a valuable tool to

study normal brain function, due to the development of revolu-
tionary methods for data acquisition and postprocessing, as well as
for paradigm design. Due to its noninvasiveness and relatively high
spatial and temporal resolution, fMRI has rapidly substituted other
techniques, such as positron emission tomography (PET), in the
assessment of brain function. In addition, the combination of
fMRI with neurophysiological techniques, such as transcranial
magnetic stimulation (TMS), is providing important pieces of
information for the understanding of brain function in healthy
individuals, which, on turn, is critical for the interpretation of func-
tional changes in diseased people.
This chapter summarizes the major contributions of fMRI for
the in vivo assessment of the sensorimotor network in healthy
human subjects, with a specific focus on studies of performance of
a simple motor task with the dominant upper limb.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_17, © Springer Science+Business Media New York 2016

523
524 Massimo Filippi et al.

2 Sensorimotor Paradigms

Activity of the sensorimotor system has been investigated by using

several experimental paradigms. The majority of the studies ana-
lyzed the performance of active tasks consisting of movement of
the hand, using tasks that require flexion-extension of the hand
and/or fingers, tapping the hand or fingers, closing–opening the
hand, hand manipulation, and squeezing. A few studies investi-
gated the movement of the foot, leg, arm, shoulder, and tongue,
with the main goal of defining the somatotopic location and hemi-
spheric lateralization of these body parts and assessing the complex
interplay between multiple sensorimotor areas [1–3]. Other stud-
ies analyzed the fMRI correlates of interlimb coordination [4, 5].
The brain activations associated to the performance of passive
tasks have also been evaluated [2, 6–9]. This strategy has mainly
been prompted by the need of obtaining meaningful comparisons
between controls and patients with neurological diseases that might
impair the “ability” to perform active tasks correctly [6–8]. The use
of passive tasks is also supported by the fact that there are reciprocal
projections between the motor and the related sensory cortices;
hence, patterns of brain activations that reflect local field potentials
from presynaptic activity primarily [10], even with entirely passive
movements, may identify those brain regions involved in active vol-
untary movements. This hypothesis has indeed been confirmed by
fMRI studies of healthy controls which have demonstrated that
activations associated to active and passive limb movements are sim-
ilar in localization and size [6, 7, 11]. However, recent studies
revealed a greater activation of brain areas responsible for motor
planning and visuomotor coordination during active-movements
execution and a selective recruitment of regions involved in motor
response inhibition during passive movements [9, 12]. Finally, it is
now established that motor network recruitment can be elicited
also by the imagination of movements [13–15].
One of the major caveats in the set up of fMRI experiments of
the sensorimotor system is an adequate monitoring of subjects’ per-
formance during task execution, which might require to be corrected
during the statistical analysis. Several variables have been shown to
influence the observed patterns of movement-associated cortical acti-
vations in healthy subjects during motor task execution, including:
1. Movement rate, which has been positively correlated with the
recruitment of the contralateral primary sensorimotor cortex
(SMC) [16], supplementary motor area (SMA) [17] and ipsi-
lateral cerebellar cortex [11, 18].
2. Force, as suggested by the load-dependent effect observed in
the primary SMC [16, 19] and cerebellum [20], and the dif-
ferent pattern of brain activity associated with the production
of static or dynamic force pulses [21].
 fMRI of the Sensorimotor System 525

3. Movement complexity, which has been shown to modulate

activity of the primary SMC [22, 23], SMA, and premotor cor-
tex [11, 24], as well as several regions of the parietal lobes [16].
Several strategies can be adopted to minimize these possible
confounding factors, including an accurate monitoring of task per-
formance during fMRI acquisition either visually or using more
sophisticated techniques, such as position and force sensors.
Other variables that need to be considered when dealing with
motor task investigations include:
1. Hemispheric dominance. Approximately 90 % of the popula-
tion has a left-hemispheric dominance for processing motor
acts [25]. In line with this, fMRI studies have demonstrated
that motor-related activations are usually lateralized to the left
hemisphere in right-handers and bilateralized or lateralized to
the right hemisphere in left-handers [26–30].
2. Gender. Women have larger activations of cortical motor areas
during motor tasks, while men exhibited significantly stronger
activation in the striatal regions [31].
3. Age. There appears to be greater motor task-related brain activ-
ity in a wider network of brain regions in older compared to
younger subjects [32, 33]. A study of healthy individuals has
demonstrated an age-related increased functional connectivity
of motor cortices between the two hemispheres [34]. These
results are consistent with a more general reduction of func-
tional lateralization of the motor cortex recruitment with aging,
which has been interpreted as a compensatory response to
increased functional demands (Fig. 1) [35, 36]. However, other
studies suggested that the increased cerebral recruitment might
reflect an inefficient response to an age-related higher difficulty
of task [33]. An age-specific pattern of activations of cerebral
areas during motor imagery has also been demonstrated [37].

3 Components of the Human Sensorimotor Network

The control of motor acts is a complex process that involves several

motor, sensory, and association areas, including the primary SMC,
secondary sensorimotor cortex (SII), SMA, cingulum, basal gan-
glia, cerebellum, and several regions located in the frontal and pari-
etal lobes. Thus, the sensorimotor network is a relatively complex
system, with a hierarchic organization. Anatomically, this view is
supported by the presence of large, somatotopically organized, pri-
mary cortices with converging projections to smaller association
areas. The extensive application of functional techniques to the
assessment of this system’s function in healthy subjects is contrib-
uting to increase our knowledge of its behavior and connections.
526 Massimo Filippi et al.

Fig. 1 Comparison of mean activation in old vs. young healthy subjects during the
performance of wrist extension/flexion and index finger abduction/adduction
with the left and right upper limb, respectively. Areas more significantly activated
in old subjects are coded in red spectrum, while areas more significantly acti-
vated in young subjects are coded in blue spectrum. Activations have been over-
laid on a standard T1 brain image in neurological view. For each motor task, the
contralateral primary sensorimotor cortex and the premotor cortex had signifi-
cantly greater activation in the young group and caudal supplementary motor
area had significantly greater activation in the old group. Ipsilateral sensorimotor
cortex was more significantly activated in the old group for index finger motor
tasks of both hands (From ref. [35], with permission)

In addition, this has allowed to define the role that the different
components of the network have during the performance of a
motor act.

3.1 The Primary Anatomically, the primary SMC is the cortex lying within the ante-
Sensorimotor Cortex rior and posterior banks of the central sulcus [1]. In line with clini-
cal and electrical stimulation studies, fMRI studies confirmed the
somatotopic organization of the primary SMC of the left
 fMRI of the Sensorimotor System 527

hemisphere, with distinct subregions controlling movements of

the foot, arm, and face [38, 39]. As already mentioned, there is a
large body of evidence supporting the prominent role of the pri-
mary SMC of the dominant hemisphere in the performance of
simple motor acts. In healthy subjects, the role of the ipsilateral
primary SMC in the control of movements is still controversial,
since several studies have reported conflicting results with respect
to the occurrence of ipsilateral primary SMC activation [16, 26,
40, 41]. In particular, while there is a general agreement on con-
sistent activation of the primary SMC of the ipsilateral hemisphere
with increasing motor task complexity [16, 24], only a few studies
reported its activation during simple task performance [26, 38,
40]. A mechanism that has been advocated to shed lights on pri-
mary SMC behavior is transcallosal inhibition. In healthy individu-
als, a transcallosal inhibitory pathway between the primary motor
cortices of the two hemispheres has been demonstrated by neuro-
physiological studies [42, 43] and has been postulated to be
responsible for the control of homologous hand muscles during
unilateral movements [42, 43]. These data are supported by fMRI
studies that have shown a decreased activation of the ipsilateral
primary SMC during sequential finger movements [41, 44–46].
The reason for the ipsilateral inhibition during unilateral hand
movements remains speculative. However, this decreased excitabil-
ity could improve the capacity to perform fine movements of the
fingers, for which a high level of dexterity is needed. Usually, such
movements are carried out unilaterally. A suppression of excitabil-
ity of the ipsilateral SMC would then minimize the risk of contra-
lateral interference, and improve the cortical focus on unilateral
activation [47]. The inhibitory interhemispheric interactions
decrease with age [48] and this loss of inhibition appears to be
ameliorated by physical activity [49].

3.2 The Another important component of the motor network is the SMA,
Supplementary Motor which is the cortex lying above the cingulate sulcus and anteriorly
Area within the paracentral lobule [1]. The SMA contributes to the
preparation, coordination, temporal course, and execution of
movements [50–52]. Studies of healthy individuals suggest that
the movement-related activity of the primary SMC might be
mediated by the extensive input it receives from the SMA [53],
which might act as facilitator or suppressor according to the task
conditions [54], and that the SMA recruitment might increase by
increasing task difficulty and complexity [16, 21, 24]. The extent
of SMA activation has been inversely related to the amount of
training an individual has gained with that specific task [50, 52,
55]. Inter- and intrahemispheric connections between the primary
SMC, the premotor cortex, and the SMA are likely to be mediated
primarily by the SMA [56]. In addition, strong bilateral connec-
tions exist between bilateral SMA and the basal ganglia, thus
528 Massimo Filippi et al.

playing an important role in motor planning and behavior [57]. In

agreement with this notion, lesions of the SMA typically result in
alterations of bimanually coordinated movements [58–60].
Efferents from the SMA project directly to the brainstem and the
cervical cord; as a consequence, an increased SMA activation might
represent recruitment of motor pathways that can function in par-
allel with the contralateral corticospinal tract [61]. A recent study
showed that spinal cord injury (SCI) can lead to network func-
tional changes, including increased neural activity within the SMA,
that might reflect a compensatory mechanism [62].
Functionally, the SMA can be divided into the pre-SMA
(located more rostrally) and the SMA-proper (located more cau-
dally), and event-related fMRI studies have shown that the pre-
SMA is activated preferentially during movement preparation [51,
63] and contributes to motor response inhibition [64]. In addi-
tion, it has been shown that pre-SMA recruitment precedes pri-
mary SMC activation by several seconds [65].

3.3 The Frontal The frontal cortex contains many areas contributing to the motor
Cortex network [66, 67]. In addition to the primary SMC, these areas
include the ventral premotor areas (including the inferior frontal
gyrus [IFG]), the dorsal premotor cortex (sometimes divided into
a caudal and a rostral part) (PMd), and a set of motor areas on the
medial wall of the hemispheres, such as the SMA and the cingulate
motor area (CMA). The premotor areas in the frontal lobe influ-
ence motor output through connections with the primary SMC
and direct projections to the spinal cord [68]. All previous premo-
tor areas contain corticospinal neurons that give a substantial con-
tribution to corticospinal projections, which have a high degree of
topographic organization [39, 69].
The role of the left inferior frontal lobe (ventral premotor cor-
tex/Broca’s area) in motor sequence control is well documented
by several studies [70–74]. Activation of Broca’s area has been
reported in various functional imaging studies based on finger
movements [71, 72], movement imagination, and motor learning
[70]. This area is supposed to receive rich sensory information
originating from the parietal lobe (including the SII) and to use it
for action [75]. In addition, modulation of this area’s activity by
task complexity has been clearly documented [73]. Studies in
humans have shown that this region is important for encoding
hand/object interactions [3, 75] and mediating motor response
inhibition via connectivity with the preSMA [74].
The PMd has an important role in motor preparation, selec-
tion, and initiation of voluntary actions [76–78]. Imaging and
TMS experiments suggest that the PMd cortex of the left hemi-
sphere is dominant in right-handed people [79]. This area is recip-
rocally connected with the ipsilateral and contralateral primary
SMC, as well as with the parietal cortex and the contralateral PMd
 fMRI of the Sensorimotor System 529

[79–81]. Using a labeling retrograde strategy, Marconi et al. [82]

showed transcallosal homotopic and heterotopic connections
between different portions of the PMd of the two hemispheres and
between the two PMd and the SMA. A correlation between pres-
ervation of motor performance after disruption of the left PMd
activity by means of TMS and increased activation of the right
PMd cortex, the SMA, and the cingulate cortex has been demon-
strated in healthy individuals. This pattern was not seen after TMS
inhibition of the left SMC, while TMS of the reorganized right
PMd disrupted motor performance. These findings suggest that
adaptive changes of PMd function might contribute to maintain
motor behavior despite the presence of structural damage [83]. A
recent experiment confirmed the same results, revealing a stronger
TMS-related increase in activity in medial and premotor areas in
association with external cues [81].
The anatomical variability of the cingulate sulcus in humans
hampers functional analysis of this region. The caudal CMA is con-
sidered to be primarily involved in movement execution [1, 66,
84], while the rostral portion of the CMA has been shown to have
a role in action selection [85], initiation, motivation, and goal-
directed behaviors [86]. A recent study demonstrated that the
CMA has a key role in realizing intentional motor control (Fig. 2)
[87]. Activation of this region has also been found to be related to
the presentation of new motor tasks and perhaps its recruitment
reflects relative task difficulty [24, 88]. This cortical area is involved
in attentional tasks and subserves several executive functions [89].
In addition, the CMA has been suggested to play an important
role in conflict monitoring [90, 91]. The role of the CMA in the
execution of spatially complex coordination tasks has been under-
lined by a study of Wenderoth et al. [92], where an increased CMA
activation was detected during the performance of a bimanual task.

3.4 The Parietal The parietal cortex is formed by a multiplicity of independent

Cortex areas, each of which deals with specific aspects of sensory
information [93]. Physiological and imaging techniques have been
extensively applied to define the location and functional specializa-
tion of parietal cortex regions in humans. Although this effort
resulted in the identification of several areas related to the
sensorimotor network, understanding their precise function and
relationship still require further experiments.
Among the regions of the parietal cortex, the SII is considered
to function as a high-order processing area for somatosensory per-
ception, and its activation seems also to be related to attention,
manual dexterity, and coordination [94, 95]. SII activity has been
associated with processing of the temporal features of somatic sen-
sations, sensorimotor integration [96, 97], tactile recognition, and
tactile learning and memory [98]. In addition, neurons from SII
project directly to the spinal cord [99], indicating that this region
530 Massimo Filippi et al.

Fig. 2 Comparison of the anterior mid-cingulate cortex (aMCC) overall functional

connectivity (FC) with neural correlates of intentional movement generation.
Intentional movement initiation yielded increased neural activity in virtually the
same brain regions involved in the FC network of the aMCC. This observation
was quantitatively verified by the conjunction of the FC analysis of the aMCC (a)
and the neural network of intentional movement initiation (b), revealing a “core
network” that comprised the aMCC, supplementary motor area (SMA), pre-SMA,
dorsolateral prefrontal cortex, dorsolateral premotor cortex, area 44, anterior
insula, inferior parietal lobule, intraparietal sulcus, cerebellum, anterior putamen,
and the right caudate nucleus (c) (From ref. [87], with permission)

might provide alternative pathways for motor control in case of

primary SMC injury. SII is known to have extensive connections
with the prefrontal cortex, the parietal lobe, and the insula. Similarly
to the primary SMC, SII has a somatotopic representation of differ-
ent body parts, with the upper limb areas located more anteriorly
and more inferiorly than the lower limb areas [39, 100].
 fMRI of the Sensorimotor System 531

Numerous areas along the intraparietal sulcus (IPS) have also

been associated with processing of sensorimotor tasks. The ante-
rior part of the IPS contains neurons that discharge in response to
3D object presentation and during grasping movements [101,
102], and it is connected to the IFG for control of action in object
manipulation [3, 71]. This area has a central role for visuomotor
integration (crossmodal information process) [93]. In addition,
increased activity of the IPS has also been described in normal sub-
jects during complex finger movement sequences [22]. Activity of
the IPS has been associated to object matching and grasping, with
a selective involvement in processing intrinsic object attributes,
such as size and shape, for the execution of an efficient grasp [93,
102]. The precuneus has been related to the execution of spatially
complex coordination tasks [92], which require shifting attention
between different locations in space. Finally, the superior parietal
gyrus (SPG), that has a well-demonstrated hand/finger represen-
tation [71], is thought to be involved in the elaboration of somato-
sensory modalities and computations underlying a transformation
from spatial target to movement vector [103].

3.5 The Basal The basal ganglia have extensive connections to the motor and
Ganglia, Insula, somatosensory cortices and are involved in motor programming,
and Thalamus execution, and control [104, 105]. In particular, basal ganglia activ-
ity has been associated with motor program selection and suppres-
sion at early stages of motor planning, as well as with control of
movement simulation [57, 106]. In addition, they are implicated in
the formation of motor skills and are part of subsystems whose
activity has been associated with timing of motor acts [107–110].
The thalamus [104, 111] and the insula [112] have extensive
connections with the motor and somatosensory cortices and are
involved in motor execution [104, 112]. Interestingly, the thala-
mus is an important relay station of the complex re-entrant cir-
cuitry that links the motor and the prefrontal cortices to the basal
ganglia, which is part of the feedback loops of the limbic system
able to modulate the cortical motor output [57, 113].
The insular cortex has been shown to play a role in crossmodal
transfer of information [114]. In addition, the insular cortex,
which has connections with numerous cortical and subcortical
motor regions, is involved in the synchronization of movement
kinematic [115] and skeletomotor body orientation [116].

3.6 The Cerebellum The cerebellum integrates sensory information and motor pro-
grams to coordinate fine movements. Functionally, the cerebellum
is organized in modules arranged in the medio-lateral direction,
being the medial part responsible for control of posture and the
lateral regions for coordination and movements. Anatomically, the
cerebellum is divided along the rostro-caudal axis in the anterior
lobe, which contains a somatotopic representation of movement of
532 Massimo Filippi et al.

the ipsilateral side and contributes to motor control [117], and the
posterior lobe, which is thought to be related to motor imagery
[75] and motor learning [118, 119]. The posterior lobe of the
cerebellum has projections from and to regions of the parietal cor-
tex, involved in the processing of sensory information [120–122],
which is then used to correct movements. However, recent studies
revealed functionally distinct areas within and across cerebellar lob-
ules, thus demonstrating a functional parcellation that is indepen-
dent of anatomical lobular divisions [122].
Several imaging studies have reported a cerebellar recruitment
associated to timing of rhythmic movements [123]. Some studies
also described increased cerebellar activation corresponding to
increase in movement frequency and velocity [50, 110, 124, 125].
The cerebellum has also been involved in the “automatization”
(improvement of motor performance) of learned skills, establish-
ment of movement strategies, and consolidation of such a motor
knowledge [110, 126, 127]. Further evidence supporting the role
of the cerebellum in motor learning is based on data from patients
with focal cerebellar lesions, who have shown impairment in learn-
ing new motor skills [126, 128, 129], imaging studies that high-
lighted its contribution to motor recovery after SCI [62] and other
studies of motor learning in healthy individuals, who showed
prominent cerebellar recruitment [130, 131].

4 Cortical Reorganization During Motor Training and Motor Skill Learning

Psychophysiological studies have demonstrated that the acquisi-

tion of motor skills follows two distinct stages. The first is a fast
learning stage during which considerable improvement in perfor-
mance can be observed within a single training session; the second
is a later, slow learning stage, during which further gains can be
observed across several sessions of practice [23]. Karni et al. [23]
used a simple finger-opposition task, during which healthy indi-
viduals were trained over the course of several weeks and were
scanned at weekly interval using fMRI. Repetition of the task after
3 weeks of practice showed that there was a significant larger acti-
vation of the contralateral primary SMC as compared with the acti-
vation obtained with a control, untrained finger-opposition
sequence. These results support the notion that motor practice
induces recruitment of additional M1 units into a local network
specifically representing the motor trained sequence. These find-
ings are in agreement with the demonstration that, in healthy indi-
viduals, the recruitment of the primary SMC can be modified by
previous activities, such as playing musical instruments [124, 132]
or sports [133–135]. In the previous experiment, changes in pri-
mary SMC recruitment were also observed in the early scan ses-
sion, reflecting a sort of initial habituation-like effect, in which the
second sequence performed in a set evoked a smaller response than
 fMRI of the Sensorimotor System 533

the first sequence [23]. Different type of motor training might

induce different cortical reorganization. For instance, music-
related motor training usually requires auditory feedback. The
coupling between auditory input and motor output increases dur-
ing motor training, thus leading to a cortical reorganization not
only in motor areas but also in auditory areas [55]. On the other
hand, sport-related motor expertise might induce cortical reorga-
nization in motor, visual, and sensory-motor areas.
Although several studies demonstrated that learning of motor
tasks is associated with an increased activation of the contralateral
primary SMC, the pattern of functional reorganization is still
unclear. The evaluation of activation patterns associated with repeti-
tion of simple movements gave conflicting results, since some stud-
ies reported reductions, and others increases of task-related
activations [23, 136–138]. These discrepancies among studies
might be related to variability in number and length of sessions,
length of training, as well as monitoring of motor performance.
Recent evidence suggests that the repetition of a simple sequence
within a brief time window typically results in a reduced recruit-
ment of the primary SMC, due to habituation [23, 136, 139].
Activity decreases may also indicate a more efficient organization of
task-related brain networks through intensive training [55, 140].
In addition, a change in the degree of activation of the pari-
etal lobe from healthy volunteers has also been described after
motor training [139].
Dynamic activation changes during acquisition of motor skills
have also been seen in different subcortical areas [135, 141]. In
details, the dorsal parts of the putamen and the more rostral striatal
areas have been shown to be active only during the early learning
stage. On the contrary, activations of the posteroventral regions of
the putamen and globus pallidus increase with practice (Fig. 3) [141].
Recent research has indicated that motor expertise influences
not only brain activity during motor execution, but also during
motor observation [142], motor imagery and planning [143] and
motor prediction [144].

5 The Mirror-Neuron System

The mirror-neuron system (MNS) is an observation-execution match-

ing system. Several neurophysiological [145, 146] and neuroimaging
[75, 147–149] studies have demonstrated that MNS neurons dis-
charge not only when an individual performs a specific goal-directed
action, but also when an individual observes actions made by other
individuals, implying an involvement of this system in imitation and
motor learning [55, 140, 150]. The main role of the MNS is postu-
lated to be the understanding of actions [151, 152]. This system is
also thought to be involved in motor imagery [153] and empathy
[154, 155]. In humans, neurons of this system have been described in
534 Massimo Filippi et al.

Fig. 3 Activation patterns in the basal ganglia and cerebellum during acquisition of motor skills. (a Upper)
Activation maps obtained in the putamen superimposed on a coronal T1-weighted image. There was a pro-
gressive activation decrease in the dorsal part of the putamen (arrows) and an increase in a more ventrolateral
area (arrowheads) bilaterally, which persisted after 4 weeks of training. (a Lower) Percentage signal
increase ± SEM averaged across all subjects for each run of the trained sequence confirmed the activation
decrease in the dorsal putamen and increase in the ventral putamen. (b Top) Activation maps obtained in the
substantia nigra (SN) and subthalamic nucleus (STN) superimposed on EPI images. During session 1, STN
activation was observed during the first run of T-sequence (T1). After 4 weeks of training, these areas were no
more activated during the T-sequence. There was no significant signal change in the SN across runs. (b
Bottom) Signal-to-time curves ± SEM in the STN averaged across all subjects and epochs confirm the activa-
tion decrease. (c Left) Activation maps obtained in the cerebellum during the T-sequence (T1 on day 1 and T5
on day 28). Activation in the lateral cerebellar hemispheres, the left dentate nucleus (DN), and the pons
decreased with training. (c Right) Percentage signal increase ± SEM averaged across all subjects for each run
of the trained sequence in the left and right DN. In the right DN, activation increased transiently during T2
(10 min of practice) and returned to pretraining values (From ref. [141], with permission)

the IFG, the adjacent premotor cortex [156], and the rostral part of
the inferior parietal lobule [157]. The MNS is connected with the
superior temporal sulcus (STS) that provides a higher-order visual
description of the observed action [152, 158]. Mirror neurons are
likely to be multimodal, as they respond to both the visual observation
of an action as well as the sound associated with specific actions [159].
A recent fMRI study showed that regions supporting visuomotor
integration and MNS abilities are multimodal convergent zones of the
visual and motor streams (Fig. 4). In addition, the MNS seems to be
a privilege position for incorporating and integrating basic sensory-
motor information into higher-order cognitive centers [160].
In humans, mirror neurons are part of a system serving the
imitation of actions and speech generation. Therefore, the MNS
 fMRI of the Sensorimotor System 535

Stepwise Convergence of
Visual and Motor Cortices

Visual
MNS: Mirror Neuron System
Motor
vPM/I: ventral Premotor / Insula
Major Visuo-Motor Integration aOP4: anterior Operculum Parietale-4
Other Visuo-Motor Integration OPI: Operculum Parietale-I
Functional Connections
SPL: Superior Parietal Lobe
(no white matter tracts)

Fig. 4 Diagram showing the convergence of the visual (green nodes) and motor
(red nodes) systems into the multimodal integration network (blue nodes) or mir-
ror neuron system. Visual cortex streams converge into a common destiny in the
brain network. The main motor functional stream connects motor areas with the
same network as the visual system does. Therefore, motor and visual functional-
related streams meet in a multimodal integration network. Other regions such as
premotor, dorsolateral prefrontal, anterior lateral occipital, or even in situ primary
motor areas may be relevant for visuomotor integration as well, although they
engage fewer convergent functional pathways than the multimodal integration
core (light blue nodes) (From ref. [160] with permission)

might constitute a bridge between action and language processing

and might represent the neuronal substrate from which human lan-
guage evolved [161, 162]. Activity of this system is elicited by both
the execution and observation of object-related transitive and
intransitive actions [148, 158, 163]. These observations suggest
that the MNS is a network that has been preserved during evolution
and has developed “new” functions. Therefore, the MNS seems to
be an extremely plastic system, with the capacity to adapt to new
cognitive, social, or behavioral requirements to which an individual
is exposed [162, 164]. This hypothesis assumes that these “evolu-
tionary” changes have occurred over an extremely long time win-
dow (phylogenetic plasticity). It is plausible that, as shown for other
536 Massimo Filippi et al.

brain networks, including the motor one, disease-related changes of

such a plastic system might occur in case of CNS injury (adaptive
plasticity). This hypothesis has indeed been supported by the results
of a recent study in patients with multiple sclerosis, which demon-
strated that these patients tend to activate regions that are part of
the MNS during the performance of a simple motor task [165].
Defining the role of the MNS after brain injury may be central to a
better understanding of the clinical manifestations of various neuro-
logical conditions and, as a consequence, to develop new rehabilita-
tive strategies. Indeed, mirror therapy has been administered to
treat various neurological conditions leading to improvement in
motor function [166]. A recent study suggested that mirror therapy
represents an appropriate method to recruit the contralesional
motor areas to promote functional recovery through interhemi-
spheric transfer of information (Fig. 5) [167].
The majority of the MNS studies has been focused on the
attempt to better define the exact role of this system and its precise
location in healthy individuals. In this perspective, it has been dem-
onstrated that: (1) the MNS has a bilateral representation [168];
(2) mirror neurons in the premotor cortex have a somatotopic
organization, as shown for the classical motor cortex homunculus
[163]; and, finally, (3) there are gender differences in MNS func-
tion [169]. Recently, it has been suggested that there might be a
relation between activity of the MNS and handedness [165].
Functional studies have suggested a role of MNS dysfunction,
in combination with limbic system impairment, in patients with
autism [170, 171], suggesting that this neuronal system may play
a role in autistic social impairment. These studies described a
reduced activity in the IFG and premotor cortex during action/
face imitation and observation in adults [170] and children [171]
with autism spectrum disorders. However, recently, some studies
showed that the capacity of children with autism to understand the
goal of observed motor acts is preserved, thus bringing the mirror
hypothesis of autism into question [152].

6 Conclusions

Functional neuroimaging has dramatically changed our under-

standing of the human sensorimotor system by showing that it is
constituted by a large number of cortical and subcortical areas,
with a precise location and a specialized function. It is also evident
that this system functions in cooperation with other brain networks
in order to integrate all the information coming from the environ-
ment and to finalize the performance of motor acts. Defining this
system’s behavior is of the outmost importance for the understand-
ing of its dysfunction in case of disease and to develop potentially
successful therapeutic strategies capable to enhance its plasticity.
 fMRI of the Sensorimotor System 537

Fig. 5 Brain activity patterns between healthy participants who were trained on simple motor tasks with their
right hand with (mirror training group—MG) and without a mirror (control training group—CG). The fMRI analy-
sis revealed activation changes within the right dorsal premotor cortex (dPMC), left inferior parietal lobule
(vPMC), left ventral premotor cortex (IPL) and left primary sensorimotor cortex (SMC) in the MG in comparison
with the CG (schema 1). The functional connectivity (FC) analysis revealed an increased FC between previous
regions and the left supplementary motor area (SMA) in the MG over the CG (schema 2). The dynamic causal
modeling, which estimates and makes inferences about the coupling among brain areas, revealed that the
right dPMC and left vPMC interacted with the left SMA, which in turn had access to the left SMC (and the left
IPL interacted with the left vPMC) (schema 3) (From ref. [167], with permission)
538 Massimo Filippi et al.

References

1. Fink GR et al (1997) Multiple nonpri- 15. van der Meulen M et al (2014) The influence
mary motor areas in the human cortex. of individual motor imagery ability on cere-
J Neurophysiol 77(4):2164–2174 bral recruitment during gait imagery. Hum
2. Ciccarelli O et al (2005) Identifying brain Brain Mapp 35(2):455–470
regions for integrative sensorimotor process- 16. Wexler BE et al (1997) An fMRI study of
ing with ankle movements. Exp Brain Res the human cortical motor system response to
166(1):31–42 increasing functional demands. Magn Reson
3. Nowak DA, Glasauer S, Hermsdorfer Imaging 15(4):385–396
J (2013) Force control in object manipula- 17. Deiber MP et al (1999) Mesial motor areas
tion—a model for the study of sensorimotor in self-initiated versus externally triggered
control strategies. Neurosci Biobehav Rev movements examined with fMRI: effect of
37(8):1578–1586 movement type and rate. J Neurophysiol
4. Debaere F et al (2001) Brain areas involved 81(6):3065–3077
in interlimb coordination: a distributed net- 18. VanMeter JW et al (1995) Parametric analy-
work. Neuroimage 14(5):947–958 sis of functional neuroimages: application
5. Rocca MA et al (2007) Influence of body to a variable-rate motor task. Neuroimage
segment position during in-phase and anti- 2(4):273–283
phase hand and foot movements: a kinematic 19. Dettmers C et al (1995) Relation between
and functional MRI study. Hum Brain Mapp cerebral activity and force in the motor areas of
28(3):218–227 the human brain. J Neurophysiol 74:802–815
6. Reddy H et al (2001) Altered cortical acti- 20. Keisker B et al (2009) Differential force scal-
vation with finger movement after peripheral ing of fine-graded power grip force in the
denervation: comparison of active and passive sensorimotor network. Hum Brain Mapp
tasks. Exp Brain Res 138(4):484–491 30(8):2453–2465
7. Reddy H et al (2002) Functional brain reor- 21. Neely KA et al (2013) Segregated and over-
ganization for hand movement in patients lapping neural circuits exist for the produc-
with multiple sclerosis: defining distinct tion of static and dynamic precision grip
effects of injury and disability. Brain 125(Pt force. Hum Brain Mapp 34(3):698–712
12):2646–2657 22. Schlaug G, Knorr U, Seitz R (1994) Inter-
8. Petsas N et al (2013) Evidence of impaired subject variability of cerebral activations in
brain activity balance after passive sensorimo- acquiring a motor skill: a study with posi-
tor stimulation in multiple sclerosis. PLoS tron emission tomography. Exp Brain Res
One 8(6):e65315 98(3):523–534
9. Jaeger L et al (2014) Brain activation associ- 23. Karni A et al (1995) Functional MRI evidence
ated with active and passive lower limb step- for adult motor cortex plasticity during motor
ping. Front Hum Neurosci 8:828 skill learning. Nature 377(6545):155–158
10. Logothetis NK et al (2001) Neurophysiological 24. Rao SM et al (1993) Functional magnetic
investigation of the basis of the fMRI signal. resonance imaging of complex human move-
Nature 412(6843):150–157 ments. Neurology 43(11):2311–2318
11. Mehta JP et al (2012) The effect of move- 25. Annett M (1973) Handedness in families.
ment rate and complexity on functional Ann Hum Genet 37(1):93–105
magnetic resonance signal change during 26. Kim SG et al (1993) Functional magnetic
pedaling. Motor Control 16(2):158–175 resonance imaging of motor cortex: hemi-
12. Francis S et al (2009) fMRI analysis of active, spheric asymmetry and handedness. Science
passive and electrically stimulated ankle dorsi- 261(5121):615–617
flexion. Neuroimage 44(2):469–479 27. Singh LN et al (1998) Comparison of ipsilat-
13. Decety J et al (1994) Mapping motor repre- eral activation between right and left handers:
sentations with positron emission tomogra- a functional MR imaging study. Neuroreport
phy. Nature 371(6498):600–602 9(8):1861–1866
14. Porro CA et al (1996) Primary motor and 28. Solodkin A et al (2001) Lateralization of
sensory cortex activation during motor per- motor circuits and handedness during finger
formance and motor imagery: a functional movements. Eur J Neurol 8(5):425–434
magnetic resonance imaging study. J Neurosci 29. Verstynen T et al (2005) Ipsilateral motor
16(23):7688–7698 cortex activity during unimanual hand
 fMRI of the Sensorimotor System 539

movements relates to task complexity. 45. Nirkko AC et al (2001) Different ipsilateral

J Neurophysiol 93(3):1209–1222 representations for distal and proximal move-
30. Pool EM et al (2015) Functional resting-state ments in the sensorimotor cortex: activa-
connectivity of the human motor network: tion and deactivation patterns. Neuroimage
differences between right- and left-handers. 13(5):825–835
Neuroimage 109:298–306 46. Stefanovic B, Warnking JM, Pike GB (2004)
31. Lissek S et al (2007) Sex differences in corti- Hemodynamic and metabolic responses
cal and subcortical recruitment during simple to neuronal inhibition. Neuroimage
and complex motor control: an fMRI study. 22(2):771–778
Neuroimage 37(3):912–926 47. Geffen GM, Jones DL, Geffen LB (1994)
32. Ward NS (2006) Compensatory mechanisms Interhemispheric control of manual motor
in the aging motor system. Ageing Res Rev activity. Behav Brain Res 64(1–2):131–140
5(3):239–254 48. Davidson T, Tremblay F (2013) Age and
33. Loibl M et al (2011) Non-effective increase hemispheric differences in transcallosal inhi-
of fMRI-activation for motor perfor- bition between motor cortices: an ispsilateral
mance in elder individuals. Behav Brain Res silent period study. BMC Neurosci 14:62
223(2):280–286 49. McGregor KM et al (2011) Physical activ-
34. Taniwaki T et al (2007) Age-related altera- ity and neural correlates of aging: a com-
tions of the functional interactions within bined TMS/fMRI study. Behav Brain Res
the basal ganglia and cerebellar motor loops 222(1):158–168
in vivo. Neuroimage 36(4):1263–1276 50. Sadato N et al (1997) Role of the supplemen-
35. Hutchinson S et al (2002) Age-related tary motor area and the right premotor cortex
differences in movement representation. in the coordination of bimanual finger move-
Neuroimage 17(4):1720–1728 ments. J Neurosci 17(24):9667–9674
36. Mattay VS et al (2002) Neurophysiological 51. Lee KM, Chang KH, Roh JK (1999) Subregions
correlates of age-related changes in human within the supplementary motor area activated
motor function. Neurology 58(4):630–635 at different stages of movement preparation and
37. Wang L et al (2014) Age-specific activation execution. Neuroimage 9(1):117–123
of cerebral areas in motor imagery—a fMRI 52. Ohara S et al (2000) Movement-related
study. Neuroradiology 56(4):339–348 change of electrocorticographic activity in
38. Alkadhi H et al (2002) Reproducibility of human supplementary motor area proper.
primary motor cortex somatotopy under con- Brain 123(Pt 6):1203–1215
trolled conditions. AJNR Am J Neuroradiol 53. Arai N et al (2012) Effective connectivity
23(9):1524–1532 between human supplementary motor area
39. Cunningham DA et al (2013) Functional and primary motor cortex: a paired-coil TMS
somatotopy revealed across multiple cortical study. Exp Brain Res 220(1):79–87
regions using a model of complex motor task. 54. Gao Q et al (2014) Differential contribution
Brain Res 1531:25–36 of bilateral supplementary motor area to the
40. Chiou SY et al (2013) Co-activation of pri- effective connectivity networks induced by
mary motor cortex ipsilateral to muscles task conditions using dynamic causal model-
contracting in a unilateral motor task. Clin ing. Brain Connect 4(4):256–264
Neurophysiol 124(7):1353–1363 55. Yang J (2015) The influence of motor exper-
41. McGregor KM et al (2015) Reliability of neg- tise on the brain activity of motor task perfor-
ative BOLD in ipsilateral sensorimotor areas mance: a meta-analysis of functional magnetic
during unimanual task activity. Brain Imaging resonance imaging studies. Cogn Affect
Behav 9(2):245–254 Behav Neurosci 15(2):381–394
42. Netz J, Ziemann U, Homberg V (1995) 56. Rouiller EM et al (1994) Transcallosal con-
Hemispheric asymmetry of transcallosal inhi- nections of the distal forelimb representations
bition in man. Exp Brain Res 104(3):527–533 of the primary and supplementary motor cor-
tical areas in macaque monkeys. Exp Brain
43. Liepert J et al (2001) Inhibition of ipsilateral Res 102(2):227–243
motor cortex during phasic generation of low
force. Clin Neurophysiol 112(1):114–121 57. Marchand WR et al (2013) Functional
architecture of the cortico-basal ganglia cir-
44. Allison JD et al (2000) Functional MRI cere- cuitry during motor task execution: correla-
bral activation and deactivation during finger tions of strength of functional connectivity
movement. Neurology 54(1):135–142 with neuropsychological task performance
540 Massimo Filippi et al.

among female subjects. Hum Brain Mapp of sequential movements. J Cogn Neurosci
34(5):1194–1207 12:56–77
58. Brinkman C (1981) Lesions in supplemen- 73. Haslinger B et al (2002) The role of lateral
tary motor area interfere with a monkey’s premotor-cerebellar-parietal circuits in motor
performance of a bimanual coordination task. sequence control: a parametric fMRI study.
Neurosci Lett 27(3):267–270 Brain Res Cogn Brain Res 13(2):159–168
59. Brinkman C (1984) Supplementary motor 74. Duann JR et al (2009) Functional connec-
area of the monkey’s cerebral cortex: short- tivity delineates distinct roles of the inferior
and long-term deficits after unilateral ablation frontal cortex and presupplementary motor
and the effects of subsequent callosal section. area in stop signal inhibition. J Neurosci
J Neurosci 4(4):918–929 29(32):10171–10179
60. Akkal D, Dum RP, Strick PL (2007) 75. Grafton ST et al (1996) Localization of grasp
Supplementary motor area and presupple- representations in humans by positron emission
mentary motor area: targets of basal gan- tomography. 2. Observation compared with
glia and cerebellar output. J Neurosci imagination. Exp Brain Res 112(1):103–111
27(40):10659–10673 76. Scott SH, Sergio LE, Kalaska JF (1997)
61. Martino AM, Strick PL (1987) Corticospinal Reaching movements with similar hand paths
projections originate from the arcuate premo- but different arm orientations. II. Activity
tor area. Brain Res 404(1–2):307–312 of individual cells in dorsal premotor cor-
62. Hou JM et al (2014) Alterations of resting- tex and parietal area 5. J Neurophysiol
state regional and network-level neu- 78(5):2413–2426
ral function after acute spinal cord injury. 77. Grafton ST, Fagg AH, Arbib MA (1998)
Neuroscience 277:446–454 Dorsal premotor cortex and conditional
63. Humberstone M et al (1997) Functional movement selection: a PET functional map-
magnetic resonance imaging of single motor ping study. J Neurophysiol 79(2):1092–1097
events reveals human presupplementary 78. Bestmann S et al (2008) Dorsal premotor
motor area. Ann Neurol 42(4):632–637 cortex exerts state-dependent causal influ-
64. Zhang S, Ide JS, Li CS (2012) Resting-state ences on activity in contralateral primary
functional connectivity of the medial superior motor and dorsal premotor cortex. Cereb
frontal cortex. Cereb Cortex 22(1):99–111 Cortex 18(6):1281–1291
65. Weilke F et al (2001) Time-resolved fMRI 79. Schluter ND et al (2001) Cerebral dominance
of activation patterns in M1 and SMA for action in the human brain: the selection of
during complex voluntary movement. actions. Neuropsychologia 39(2):105–113
J Neurophysiol 85(5):1858–1863 80. Schluter ND et al (1998) Temporary inter-
66. Picard N, Strick PL (1996) Motor areas of ference in human lateral premotor cortex
the medial wall: a review of their location suggests dominance for the selection of
and functional activation. Cereb Cortex movements. A study using transcranial mag-
6(3):342–353 netic stimulation. Brain 121(Pt 5):785–799
67. Rizzolatti G, Luppino G (2001) The cortical 81. Moisa M et al (2012) Uncovering a
motor system. Neuron 31(6):889–901 context-specific connectional fingerprint of
68. Dum RP, Strick PL (1991) The origin of cor- human dorsal premotor cortex. J Neurosci
ticospinal projections from the premotor areas 32(21):7244–7252
in the frontal lobe. J Neurosci 11(3):667–689 82. Marconi B et al (2003) Callosal connec-
69. Dum RP, Strick PL (2002) Motor areas in tions of dorso-lateral premotor cortex. Eur
the frontal lobe of the primate. Physiol Behav J Neurosci 18(4):775–788
77(4–5):677–682 83. O’Shea J et al (2007) Functionally specific
70. Stephan KM et al (1995) Functional anat- reorganization in human premotor cortex.
omy of the mental representation of upper Neuron 54(3):479–490
extremity movements in healthy subjects. 84. Cunnington R et al (2006) The selection
J Neurophysiol 73:373–386 of intended actions and the observation of
71. Binkofski F et al (1999) A fronto-parietal others’ actions: a time-resolved fMRI study.
circuit for object manipulation in man: evi- Neuroimage 29(4):1294–1302
dence from an fMRI-study. Eur J Neurosci 85. Deiber MP et al (1991) Cortical areas and
11(9):3276–3286 the selection of movement: a study with posi-
72. Harrington DL et al (2000) Specialized tron emission tomography. Exp Brain Res
neural systems underlying representations 84(2):393–402
 fMRI of the Sensorimotor System 541

86. Devinsky O, Morrell MJ, Vogt BA (1995) 101. Culham JC, Kanwisher NG (2001)
Contributions of anterior cingulate cortex to Neuroimaging of cognitive functions in
behaviour. Brain 118:279–306 human parietal cortex. Curr Opin Neurobiol
87. Hoffstaedter F et al (2014) The role of ante- 11(2):157–163
rior midcingulate cortex in cognitive motor 102. Monaco S et al (2015) Neural correlates of
control: evidence from functional connectivity object size and object location during grasp-
analyses. Hum Brain Mapp 35(6):2741–2753 ing actions. Eur J Neurosci 41(4):454–465
88. Paus T et al (1993) Role of the human anterior 103. Barany DA et al (2014) Feature interactions
cingulate cortex in the control of oculomo- enable decoding of sensorimotor transforma-
tor, manual, and speech responses: a positron tions for goal-directed movement. J Neurosci
emission tomography study. J Neurophysiol 34(20):6860–6873
70(2):453–469 104. Parent A, Hazrati LN (1995) Functional
89. Vogt BA, Finch DM, Olson CR (1992) anatomy of the basal ganglia. I. The cortico-
Functional heterogeneity in cingulate cortex: basal ganglia-thalamo-cortical loop. Brain Res
the anterior executive and posterior evaluative Brain Res Rev 20(1):91–9127
regions. Cereb Cortex 2(6):435–443 105. Choi EY, Yeo BT, Buckner RL (2012) The
90. Botvinick M et al (1999) Conflict monitoring organization of the human striatum esti-
versus selection-for-action in anterior cingu- mated by intrinsic functional connectivity.
late cortex. Nature 402(6758):179–181 J Neurophysiol 108(8):2242–2263
91. Carter CS et al (1998) Anterior cingulate cor- 106. Kessler K et al (2006) Investigating the human
tex, error detection, and the online monitoring mirror neuron system by means of cortical syn-
of performance. Science 280(5364):747–749 chronization during the imitation of biological
92. Wenderoth N et al (2005) The role of anterior movements. Neuroimage 33(1):227–238
cingulate cortex and precuneus in the coor- 107. Harrington DL, Haaland KY, Knight
dination of motor behaviour. Eur J Neurosci RT (1998) Cortical networks underlying
22(1):235–246 mechanisms of time perception. J Neurosci
93. Rizzolatti G, Fogassi L, Gallese V (1997) 18(3):1085–1095
Parietal cortex: from sight to action. Curr 108. Ivry RB, Keele SW, Diener HC (1988)
Opin Neurobiol 7(4):562–567 Dissociation of the lateral and medial cer-
94. Karhu J, Tesche CD (1999) Simultaneous ebellum in movement timing and movement
early processing of sensory input in human pri- execution. Exp Brain Res 73(1):167–180
mary (SI) and secondary (SII) somatosensory 109. Jantzen KJ, Steinberg FL, Kelso JAS (2004)
cortices. J Neurophysiol 81(5):2017–2025 Brain networks underlying human timing
95. Hamalainen H, Hiltunen J, Titievskaja I behavior are influenced by prior context. Proc
(2000) fMRI activations of SI and SII cortices Natl Acad Sci U S A 101(17):6815–6820
during tactile stimulation depend on atten- 110. Walz AD et al (2014) Changes in corti-
tion. Neuroreport 11(8):1673–1676 cal, cerebellar and basal ganglia repre-
96. Huttunen J et al (1996) Significance of sentation after comprehensive long term
the second somatosensory cortex in senso- unilateral hand motor training. Behav Brain
rimotor integration: enhancement of sen- Res 278C:393–403
sory responses during finger movements. 111. Brooks DJ (1995) The role of the basal gan-
Neuroreport 7(5):1009–1012 glia in motor control: contributions from
97. Shergill SS et al (2013) Modulation of somato- PET. J Neurol Sci 128(1):1–13
sensory processing by action. Neuroimage 112. Mesulam MM (1998) From sensation to cog-
70:356–362 nition. Brain 121(Pt 6):1013–1052
98. Mima T et al (1998) Attention modulates 113. Chaudhuri A, Behan PO (2000) Fatigue and
both primary and second somatosensory corti- basal ganglia. J Neurol Sci 179(S 1–2):34–42
cal activities in humans: a magnetoencephalo- 114. Hadjikhani N, Roland PE (1998) Cross-
graphic study. J Neurophysiol 80(4):2215–2221 modal transfer of information between the
99. Dobkin BH (2003) Functional MRI: a poten- tactile and the visual representations in the
tial physiologic indicator for stroke rehabilita- human brain: a positron emission tomo-
tion interventions. Stroke 34(5):e23–e28 graphic study. J Neurosci 18(3):1072–1084
100. Del Gratta C et al (2002) Topographic orga- 115. Mosier K, Bereznaya I (2001) Parallel cortical
nization of the human primary and second- networks for volitional control of swallowing
ary somatosensory cortices: comparison in humans. Exp Brain Res 140(3):280–289
of fMRI and MEG findings. Neuroimage 116. Taylor KS, Seminowicz DA, Davis KD (2009)
17(3):1373–1383 Two systems of resting state connectiv-
542 Massimo Filippi et al.

ity between the insula and cingulate cortex. tron emission tomography. Rev Neurol
Hum Brain Mapp 30(9):2731–2745 (Paris) 149:647–653
117. Nitschke MF et al (1996) Somatotopic motor 131. Jenkins IH et al (1994) Motor sequence
representation in the human anterior cerebel- learning: a study with positron emission
lum. A high-resolution functional MRI study. tomography. J Neurosci 14(6):3775–3790
Brain 119(Pt 3):1023–1029 132. Krings T et al (2000) Cortical activation pat-
118. Sakai K et al (1998) Separate cerebel- terns during complex motor tasks in piano
lar areas for motor control. Neuroreport players and control subjects. A functional
9(10):2359–2363 magnetic resonance imaging study. Neurosci
119. Kim JJ, Thompson RF (1997) Cerebellar Lett 278(3):189–193
circuits and synaptic mechanisms involved 133. Pearce AJ et al (2000) Functional reorgan-
in classical eyeblink conditioning. Trends isation of the corticomotor projection to the
Neurosci 20(4):177–181 hand in skilled racquet players. Exp Brain Res
120. Ehrsson HH, Kuhtz-Buschbeck JP, Forssberg 130(2):238–243
H (2002) Brain regions controlling nonsyn- 134. Milton J et al (2007) The mind of expert
ergistic versus synergistic movement of the motor performance is cool and focused.
digits: a functional magnetic resonance imag- Neuroimage 35(2):804–813
ing study. J Neurosci 22(12):5074–5080 135. Bishop DT et al (2013) Neural bases for
121. Allen GI, Tsukahara N (1974) anticipation skill in soccer: an FMRI study.
Cerebrocerebellar communication systems. J Sport Exerc Psychol 35(1):98–109
Physiol Rev 54(4):957–951006 136. Dirnberger G et al (2004) Habituation in
122. Kipping JA et al (2013) Overlapping and a simple repetitive motor task: a study with
parallel cerebello-cerebral networks contrib- movement-related cortical potentials. Clin
uting to sensorimotor control: an intrinsic Neurophysiol 115(2):378–384
functional connectivity study. Neuroimage 137. Loubinoux I et al (2001) Within-session and
83:837–848 between-session reproducibility of cerebral
123. Ramnani N, Passingham RE (2001) Changes sensorimotor activation: a test–retest effect
in the human brain during rhythm learning. evidenced with functional magnetic reso-
J Cogn Neurosci 13(7):952–966 nance imaging. J Cereb Blood Flow Metab
124. Jancke L, Shah NJ, Peters M (2000) Cortical 21(5):592–607
activations in primary and secondary motor 138. Tracy JI et al (2001) A comparison of ‘Early’
areas for complex bimanual movements in and ‘Late’ stage brain activation during brief
professional pianists. Brain Res Cogn Brain practice of a simple motor task. Brain Res
Res 10(1–2):177–183 Cogn Brain Res 10(3):303–316
125. Wenzel U et al (2014) Functional and struc- 139. Morgen K et al (2004) Kinematic specificity of
tural correlates of motor speed in the cerebel- cortical reorganization associated with motor
lar anterior lobe. PLoS One 9(5):e96871 training. Neuroimage 21(3):1182–1187
126. Doyon J et al (1998) Role of the stria- 140. Kruger B et al (2014) Parietal and premo-
tum, cerebellum and frontal lobes in the tor cortices: activation reflects imitation
automatization of a repeated visuomotor accuracy during observation, delayed imita-
sequence of movements. Neuropsychologia tion and concurrent imitation. Neuroimage
36(7):625–641 100:39–50
127. Jueptner M, Weiller C (1998) A review of 141. Lehericy S et al (2005) Distinct basal ganglia
differences between basal ganglia and cer- territories are engaged in early and advanced
ebellar control of movements as revealed motor sequence learning. Proc Natl Acad Sci
by functional imaging studies. Brain 121(Pt U S A 102(35):12566–12571
8):1437–1449 142. Kim YT et al (2011) Neural correlates related
128. Sanes JN, Dimitrov B, Hallett M (1990) to action observation in expert archers. Behav
Motor learning in patients with cerebellar Brain Res 223(2):342–347
dysfunction. Brain 113(Pt 1):103–120 143. Baeck JS et al (2012) Brain activation patterns
129. Bracha V et al (2000) The human cerebel- of motor imagery reflect plastic changes asso-
lum and associative learning: dissociation ciated with intensive shooting training. Behav
between the acquisition, retention and extinc- Brain Res 234(1):26–32
tion of conditioned eyeblinks. Brain Res 144. Balser N et al (2014) Prediction of human
860(1–2):87–94 actions: expertise and task-related effects on
130. Jenkins IH, Frackowiak RS (1993) Functional neural activation of the action observation
studies of the human cerebellum with posi- network. Hum Brain Mapp 35(8):4016–4034
 fMRI of the Sensorimotor System 543

145. Fadiga L et al (1995) Motor facilitation dur- 158. Iacoboni M (2005) Neural mechanisms of imi-
ing action observation: a magnetic stimula- tation. Curr Opin Neurobiol 15(6):632–637
tion study. J Neurophysiol 73(6):2608–2611 159. Kohler E et al (2002) Hearing sounds, under-
146. Hari R et al (1998) Activation of human pri- standing actions: action representation in
mary motor cortex during action observation: mirror neurons. Science 297:846–848
a neuromagnetic study. Proc Natl Acad Sci U 160. Sepulcre J (2014) Integration of visual and
S A 95(25):15061–15065 motor functional streams in the human brain.
147. Grezes J et al (2003) Activations related to Neurosci Lett 567:68–73
“mirror” and “canonical” neurones in the 161. Rizzolatti G, Arbib MA (1998) Language
human brain: an fMRI study. Neuroimage within our grasp. Trends Neurosci
18(4):928–937 21(5):188–194
148. Rizzolatti G et al (1996) Localization of 162. Oztop E, Kawato M, Arbib MA (2013)
grasp representations in humans by PET: 1. Mirror neurons: functions, mechanisms and
Observation versus execution. Exp Brain Res models. Neurosci Lett 540:43–55
111(2):246–252 163. Buccino G et al (2001) Action observa-
149. Mengotti P, Corradi-Dell’acqua C, Rumiati tion activates premotor and parietal areas in
RI (2012) Imitation components in the a somatotopic manner: an fMRI study. Eur
human brain: an fMRI study. Neuroimage J Neurosci 13(2):400–404
59(2):1622–1630 164. Filippi M et al (2013) The “vegetarian brain”:
150. Buccino G et al (2004) Neural circuits under- chatting with monkeys and pigs? Brain Struct
lying imitation learning of hand actions: Funct 218(5):1211–1227
an event-related fMRI study. Neuron 165. Rocca MA et al (2008) The mirror-neuron
42(2):323–334 system and handedness: a “right” world?
151. Rizzolatti G, Craighero L (2004) The Hum Brain Mapp 29(11):1243–1254
mirror-neuron system. Annu Rev Neurosci 166. Buccino G (2014) Action observation
27:169–192 treatment: a novel tool in neurorehabilita-
152. Rizzolatti G, Sinigaglia C (2010) The func- tion. Philos Trans R Soc Lond B Biol Sci
tional role of the parieto-frontal mirror cir- 369(1644):20130185
cuit: interpretations and misinterpretations. 167. Hamzei F et al (2012) Functional plastic-
Nat Rev Neurosci 11(4):264–274 ity induced by mirror training: the mirror as
153. Johnson SH et al (2002) Selective activation of a the element connecting both hands to one
parietofrontal circuit during implicitly imagined hemisphere. Neurorehabil Neural Repair
prehension. Neuroimage 17(4):1693–1704 26(5):484–496
154. Leslie KR, Johnson-Frey SH, Grafton ST 168. Aziz-Zadeh L et al (2006) Lateralization of
(2004) Functional imaging of face and hand the human mirror neuron system. J Neurosci
imitation: towards a motor theory of empa- 26(11):2964–2970
thy. Neuroimage 21(2):601–607 169. Cheng Y-W et al (2006) Gender differ-
155. Filippi M et al (2010) The brain functional ences in the human mirror system: a mag-
networks associated to human and animal netoencephalography study. Neuroreport
suffering differ among omnivores, vegetarians 17(11):1115–1119
and vegans. PLoS One 5(5):e10847 170. Theoret H et al (2005) Impaired motor facili-
156. Cerri G et al (2015) The mirror neuron tation during action observation in individu-
system and the strange case of Broca’s area. als with autism spectrum disorder. Curr Biol
Hum Brain Mapp 36(3):1010–1027 15(3):R84–R85
157. Rizzolatti G, Fogassi L, Gallese V (2001) 171. Dapretto M et al (2006) Understanding emo-
Neurophysiological mechanisms underlying tions in others: mirror neuron dysfunction in
the understanding and imitation of action. children with autism spectrum disorders. Nat
Nat Rev Neurosci 2(9):661–670 Neurosci 9(1):28–30
 Chapter 18

Functional Imaging of the Human Visual System

Guy A. Orban and Stefania Ferri

Abstract
The human visual system consists of a large, yet unknown number of cortical areas. We summarize the efforts
which have led to the identification of 19 retinotopic areas in human occipital cortex, using the macaque
visual cortex as a guide. In this process retinotopic mapping has proven far superior to the study of functional
properties. Macaques and humans share early areas (V1, V2, and V3), a motion-sensitive middle temporal
(MT/V5) cluster as well as six other areas. The remaining human occipital areas either result from reorgani-
zation of a group of monkey areas or seem to be specifically human. Several regions sensitive to motion and
even higher-order motion have been described in parietal cortex, the retinotopic organization of which is still
under debate. On the other hand, both dorsal and ventral regions are sensitive to shape, which is most pro-
nounced in the lateral occipital complex (LOC) extending into the fusiform gyrus. The anterior part of this
complex is flanked by specialized regions devoted to processing faces and bodies and represents “visual
objects” rather than image properties. Its exact organization requires further investigation.

Key words Vision, Retinotopy, Cortical area, Visual field, Motion, 2D and 3D Shape, Actions

1 Introduction

The human visual system is located in the occipital lobe and extends
rostrally into the parietal and temporal lobes. It is estimated to
encompass 30 % of human cortex [1]. Functional imaging gives us
direct access to the function of this important part of human cortex.
One way to study this system is to consider a number of perceptual
or visual cognitive functions and to localize their neural correlates.
An alternative is to consider the visual system as an anatomically orga-
nized collection of cortical areas and subcortical centers that process
retinal information and transform it into messages appropriate for
processing in the nonvisual cerebral regions to which the visual sys-
tem projects. A critical aim in visual neuroscience is to define the
different cortical areas that make up the human visual system. In
other species, such as the nonhuman primates, cortical areas are
defined by the combination of four criteria: (1) cyto- and myelo-
architectonics, (2) anatomical connections with other (known) areas,
(3) topographic organization, i.e., retinotopic organization, and

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_18, © Springer Science+Business Media New York 2016

545
546 Guy A. Orban and Stefania Ferri

(4) functional properties. It is important to note that while not all

criteria may apply to each area, it is essential to obtain as much con-
verging information as possible. In the nonhuman primate, 30 or
more visual cortical areas have been identified using these criteria,
although it is fair to state that even in these species there is discussion
about the exact definition of areas, especially those at the higher levels
in the system [1]. The definition of the visual cortical areas is only a
first step in understanding the visual system; next is the investigation
of the nature of the processing performed by these areas and the flow
of information through the areas as a function of the behavioral con-
text and task demands.
Recent advances in brain imaging have provided powerful
tools for the definition and mapping of cortical areas. Functional
magnetic resonance imaging (fMRI) provides insights into the
functional characteristics of cortical areas by means of specific con-
trasts of brain activity that isolate a functional property. For exam-
ple, in the monkey in which a number of visual areas have been
identified using anatomical and neurophysiological measurements,
fMRI has shown that a small number of functional characteristics,
defined by a few subtractions, allow the definition of six motion-
sensitive regions in the monkey superior temporal sulcus (STS)
[2]. Functional MRI can also provide evidence for retinotopic
organization. It actually is more powerful than single cell studies in
this respect, as it is less biased in its sampling and the measure
required is simply responsiveness. It has been suggested that the
topology of an area, that is, its localization with respect to neigh-
boring areas, might be a valuable addition for the identification of
areas [3]. Imaging has not yet provided clear means to obtain his-
tological structure, although at high field (7.0 T) the stria of
Gennari becomes visible, and myelin density can be measured indi-
rectly at 3 T [4]. The situation is slightly better for anatomical
connections, as diffusion tensor imaging (DTI) [5, 6] is increas-
ingly seen as a potential measure of connectivity between areas,
although the methodological issues remain formidable [7]. In the
present chapter we will provide an overview of how these two
fMRI strategies, functional specialization and retinotopic organi-
zation, have been used for defining cortical areas.
Despite all its strengths functional imaging has severe limita-
tions due to its limited temporal and spatial resolution. With the
present 3 T systems a few millimeters can be resolved. While this is
ample to define cortical regions it is a long way from the resolution
of the single neuron. In fact, fMRI signals are only indirect, hemo-
dynamic reflections of average activity of thousands of neurons.
Hence, fMRI is very sensitive at detecting average activity levels,
but it has great difficulty in measuring neuronal selectivity. It has
been proposed that repetition suppression can be used to measure
neuronal tuning, but the case for it might be overstated as in single
neurons the tuning of the adaptation is narrower than the response
 Functional Imaging of the Human Visual System 547

tuning [8–10]. Recent developments using multivoxel pattern

analysis (MVPA) [11] provide sensitive tools for studying neural
representations beyond the resolution of conventional fMRI
approaches. Yet the estimation provided by this analysis depends
heavily on the clustering of neurons with similar properties, like
those in cortical columns, and the discrimination provided falls
quite short of what single neurons can achieve. For example, single
V1 neurons can signal orientation differences of 5°–10° with an
84 % chance of success [12]. MVPA of human V1 has so far yielded
values of 35° [13]. Therefore, much can be gained by combining
functional imaging in humans with knowledge derived from inva-
sive studies, such as single cell recordings in nonhuman primates.
The combination has become possible with the advent of fMRI in
the awake monkey [14]. Indeed this allows parallel imaging experi-
ments leading to the definition of cortical regions and their charac-
teristics in the two species, paving the way for establishing
homologies. Once a homology is established, one can test whether
the neuronal properties in that area apply to the human homolog.
Indeed, comparison of the single cell recordings and fMRI in the
monkey using similar stimuli allows one to derive an fMRI signa-
ture of a neuronal property. One can then verify that the human
homolog also exhibits this fMRI signature [15, 16]. Hence, the
definition of cortical areas in both species is a critical step for knowl-
edge transfer from animal models to the human visual system.

2 Methodological Issues

2.1 Stimulus Definition of the visual stimulus is important as it determines to a

Definition large degree the brain activation pattern and thus the experimental
findings reported. It is important to note that a precise stimulus
description is crucial for repeating an experiment and replicating
the results. For example, very different stimuli are used for defin-
ing motion-responsive areas. A motion localizer used to localize
human middle temporal (hMT/V5) region often consists of ran-
dom dots, but may also consist of gratings, either rectangular or
circular. Random dots may have different densities, sizes, lumi-
nance, etc., or the whole pattern may be of a different size. Random
dots may translate in one or several directions, but may also rotate
or move radially. All these paradigms, using very different stimuli,
are referred to as motion localizers, but because of their differences
they may result in activation of different cortical regions, reducing
the value of the localization.

2.2 Tasks One of the main challenges in brain imaging is investigating the
link between neural activity and human behavior. Recent studies
using parametric stimulus manipulation employ detection or dis-
crimination tasks [17–19] rather than passive viewing of the
548 Guy A. Orban and Stefania Ferri

stimuli. These paradigms allow correlation between behavioral

data (psychometric functions) and fMRI activations. This approach
is important for discerning the functional role of different cortical
areas and evaluating their contribution to behavior. Further, atten-
tionally demanding tasks (e.g., detection of changes in the fixation
target, 1-back matching task) are used during scanning to ensure
that observers pay attention across all stimulus conditions and that
activation differences across conditions are not simply due to dif-
ferences in the general arousal of the participants or the task diffi-
culty across conditions. For example, when mapping the lateral
occipital complex (LOC), participants view intact and scrambled
images of objects. It is possible that higher activations for intact
images of objects are due to the fact that these images attract the
participants’ attention more than scrambled images. To control for
this potential confound observers are instructed to perform a task
on different properties of the fixation target or the stimulus (e.g.,
dimming of the fixation point or part of the shape) [20] that entail
similar attention across all stimulus conditions. Another task that
has been adopted for controlling attentional confounds is the
1-back matching task (detect a repeat of an intact or scrambled
image) [21]. This task is more demanding for scrambled than
intact images, thus excluding the possibility that higher activations
for intact images of objects are due to attentional differences.

2.3 Control of Eye Control of fixation is mandatory in motion response studies, reti-
Movements notopic mapping experiments, and in spatial attention studies.
Although in the past it was acceptable to show that the subjects
fixated well based on off-line measurements, standards have
evolved. In addition, precise eye movement records, provided by
infrared corneal reflection methods, allow one to remove the effect
of residual eye movements that occur despite fixation. In general,
in all visual experiments, control of fixation will ensure that the
part of visual field stimulated is known and will remove eye move-
ments as a source of unwanted and uncontrolled activations.

2.4 fMRI Designs The conventional fMRI approach for identifying cortical areas
and Paradigms involved in different processes and cognitive tasks entails a
subtraction of activations between different stimulus types that are
presented in blocked or event-related designs.
One of the limitations of these fMRI paradigms is that they
average across neural populations that may respond homogeneously
across stimulus properties or may be differentially tuned to different
stimulus attributes. Thus, in most cases, it is impossible to infer the
properties of the underlying imaged neural populations. fMRI
adaptation (or repetition suppression) paradigms [22–27] have
recently been employed to study the properties of neuronal popula-
tions beyond the limited spatial resolution of fMRI. These para-
digms capitalize on the reduction of neural responses for stimuli
 Functional Imaging of the Human Visual System 549

that have been presented for prolonged time or repeatedly [28, 29].
A change in a specific stimulus dimension that elicits increased
responses (i.e., rebound of activity) identifies neural populations
that are tuned to the modified stimulus attributes. fMRI adaptation
paradigms have been used in both monkey and human fMRI stud-
ies as a sensitive tool that allows us to investigate: (a) the sensitivity
of the neural populations to stimulus properties, and (b) the invari-
ance of their responses within the imaged voxels. Adaptation across
a change between two stimuli suggests a common neural represen-
tation invariant to that change, while recovery from adaptation sug-
gests neural representations sensitive to specific stimulus properties.
For example, recent imaging studies tested whether fMRI measure-
ments can reveal neural populations in early visual areas sensitive to
elementary visual features, e.g., orientation, color, and direction of
motion [30–34]. Consider the case of motion direction: after pro-
longed exposure to the adapting motion direction, observers were
tested with the same stimulus in the same or in an orthogonal
motion direction. Decreased fMRI responses were observed in MT
when the test stimuli were at the same motion direction as the
adapting stimulus. However, recovery from this adaptation effect
was observed for stimuli presented at an orthogonal direction.
These studies suggest that the neural populations in human MT are
sensitive to direction of motion [31, 34]. Using the same procedure
in the monkey, Nelissen et al. [2] indeed observed adaptation in
MT/V5 but also in other motion-sensitive regions, such as the
medial superior temporal (MST) region. Similarly, recent studies
have shown stronger adaptation in hMT/V5+ for coherently than
transparently moving plaid stimuli. These findings provide evidence
that fMRI adaptation responses are linked to the activity of pattern-
motion rather than component-motion cells in MT/MST [32].
Thus, these studies suggest that the fMRI signal can reveal neural
selectivity consistent with the selectivity established by neurophysi-
ological methods. However, recent studies comparing fMRI adap-
tation and neurophysiology in monkeys call for cautious
interpretation of the relationship between fMRI adaptation effects
and neural selectivity or invariance at higher levels in the system
[10]. In particular, fMRI adaptation in a given cortical area may be
the result of adaptation at earlier or later stages of processing that is
propagated along the visual areas. Hence in higher-order areas
receiving from multiple inputs fMRI adaptation might reflect adap-
tation of one of the inputs, while recordings show that local neuro-
nal responses driven by the other inputs are not adapted.
Interestingly, novel MVPA methods [11, 35, 36] provide an alter-
native approach for investigating neural selectivity based on fMRI sig-
nals. Unlike conventional univariate analysis, MVPA takes advantage of
the information across multiple voxels in a cortical area and allows us
to characterize neural representations of features that are encoded at a
higher spatial resolution in the brain than the typical resolution of
550 Guy A. Orban and Stefania Ferri

fMRI. These classification analyses have been used successfully for the
decoding of elementary visual features (e.g., orientation [13, 37],
motion direction [38], and object categories [39–42]). The weakness
of the MVPA approach is its dependence on the clustering of neurons
with similar properties. This is also the case for a third technique which
is has been proposed to infer neuronal selectivity from fMRI measure-
ments: measuring the tuning of individual voxels [43]. Just as MVPA,
tuning of voxels is prone to false negative results, as the grouping of
neurons for higher-order selectivity is frequently unknown. In contrast
adaptation fMRI is prone to false positives as inputs may adapt and not
the local neuronal activity. For all these methods greater caution is
required at higher level in the cortex.

2.5 Whole Brain The statistical evaluation of activation differences between stimulus
Versus Region and tasks is typically conducted by comparing responses for each voxel
of Interest Analyses using the general linear model. Analysis of activation patterns across
the whole brain (whole brain analysis) reveals clusters of activations in
different anatomical regions that show significant differences in their
functional processing. This approach has allowed researchers to iden-
tify and localize cortical regions with different functions and evaluate
their involvement in various cognitive tasks. In contrast, region of
interest (ROI) analysis focuses on specific cortical areas identified ana-
tomically or functionally following standard mapping procedures
(e.g., retinotopic mapping). The advantage of this approach is that it
allows us to zoom in on specific cortical regions and investigate their
neural computations using parametric stimulus manipulations. Such
manipulations result in fine stimulus variations and differences in
behavioral performance. Identifying fMRI activations that reflect
these fine differences in neural processing may require the high signal-
to-noise ratio that is possible when scanning and analyzing smaller
regions of cortex. However, ROI analyses are limited in two respects:
(a) the ROI may be outside the volume scanned or analyzed, (b) the
voxels of interest (i.e., voxels that show differential activations across
conditions) may cover a smaller cortical volume than the ROI; as a
result, the differential activations may be averaged out within the
ROI. Taken together, whole brain and ROI analyses can be used as
complementary tools for studying the functional roles of cortical
regions. Whole brain analyses search the entire brain for regions
involved in the analysis of a given stimulus or a cognitive task, while
ROI methods are more appropriate for finer investigation of the neu-
ral processing in these cortical regions [44, 45].

3 Retinotopic Organization

3.1 Early Visual Initially, positron emission tomography (PET) studies have concen-
Areas (V1, V2, V3) trated on the retinotopy of V1 [46], which is a large area of known
localization in the calcarine sulcus. With the advent of fMRI,
 Functional Imaging of the Human Visual System 551

mapping was extended to areas neighboring V1 [47] (but see also

[48]). An additional step was the introduction of angular and
eccentricity periodic sweeping stimuli that generate eccentricity and
polar angle maps based on phase encoding of stimulus position
[49]. This allowed the mapping of all three early areas (V1, V2, V3,
Fig. 1) [51–53], in which polar angle and eccentricity vary along
orthogonal directions on the cortical surface. The eccentricity var-
ies from the central representation at the posterior tip of the calca-
rine fissure to that of large eccentricities rostrally along the calcarine.
Polar angle varies in dorsoventral direction with the lower field
being represented above the calcarine and the upper field below
(Fig. 1). The three early visual areas are also shown on the flatmaps
of Fig. 2 which cover a smaller eccentricity range (0.25°–7.75°)
compared to that in Fig. 1 (0°–12°). Figures 1 and 2 show retino-
topic maps of individual subjects and these maps exhibit quite some
variability. To derive a more general representation one generates
maximum probability maps (MPM) which plot in each voxel the
area with the highest probability for a given set of subjects. These
maps depend heavily on the quality of the inter-subject alignment
[20, 56, 57] and this is dramatically improved by using the novel
multimodal surface matching technique [57]. The resulting MPM
of the left hemisphere is shown on the inflated brain in Fig. 3a, b,
and on the flatmap in Fig. 3c. These maps are freely available in
Caret [56], and can be used to identify activations without the need
to spend valuable time mapping the retinotopic areas in the subjects

Fig. 1 The sweeping stimulus retinotopy paradigm. Two stimuli are used to measure the retinotopic maps in
the cortex. Expanding ring stimuli map eccentricity, and rotating wedge stimuli map polar angle. The phase of
the best-fitting sinusoid for each voxel indicates the position in the visual field that produces maximal activa-
tion for that voxel. Thus, these pseudocolor phase maps are used to visualize the retinotopic maps. Data area
is shown for the left hemisphere (medial view) of one subject. Because of the heavy folding of human cortex,
these retinotopic maps are best seen on flattened hemispheres (from Dougherty et al. [50])
552 Guy A. Orban and Stefania Ferri

Fig. 2 The 18 retinotopic areas defined in the polar angle (a) and eccentricity (b) maps by Georgieva et al. [54], and
Kolster et al. [55]; right hemisphere of subject 1. Stars: central visual field, purple: eccentricity ridge, white dotted
lines: horizontal meridian, black full and dashed lines: lower and upper vertical meridian (from Abdollahi et al. [56])

under study. Of course, when investigating the actual properties of

retinotopic areas, the direct mapping remains a superior strategy.
The three early visual cortical areas all have a large, complete rep-
resentation of the contralateral hemifield, with the upper quadrant
projecting ventrally and lower quadrant dorsally. The representation
of the vertical meridian (VM) constitutes the boundary between V1
and V2 as well as the anterior boundary of V3. The representations of
the horizontal meridian (HM) split the V1 representation and consti-
tute the boundary between V2 and V3. The central representations of
the three areas are fused in the central confluence (Figs. 1, 2, and 3).
This retinotopic organization is very similar in humans and macaques
(Fig. 4). This is not surprising as the presence of three early visual areas
is a feature of primates [59, 60]. In all three areas the central represen-
tation is magnified compared to that of the periphery [51]. Duncan
and Boynton [61] observed a correlation between magnification fac-
tor in V1 of human subjects and Vernier acuity but not grating acuity.
The surface of V1 has been estimated from histological specimens to
range between 2000 and 4500 mm2, while the central 12° occupy
2200 mm2 according to one imaging study [50]. Comparison
between histological and fMRI estimates is difficult because of the dif-
ficulty of estimating the shrinkage in the histological specimens and
the portion of V1 occupied by the central representation [62]. In
Fig. 3 (a and b) MPMs of the retinotopic areas (MSM-retino registration) in the left hemisphere on the inflated
cortical surface, lateral (a) and medial (b) views; (c) schematic representation of retinotopic organization of the
18 areas shown on the retinotopic MPM: upper (+) and lower (−) fields and central vision (stars); same color
code as in (a). The location of V6 and V6A in the parieto-occipital sulcus is indicated in (b); Note that the MPM
of areas V2 and V3 does not include the large eccentricities, otherwise V3 would abut V6

Fig. 4 Comparison of retinotopic layout of monkey (a) and human (b) visual cortex. Adapted with permission
from Vanduffel et al. [58]. Stars indicate central representations
554 Guy A. Orban and Stefania Ferri

both types of studies large variation between individuals (a factor of 2)

were observed. A similar range of variation has been observed in the
macaque, in which the average surface of V1 is roughly half the size of
its human counterpart [63]. The surface of human V2 is estimated to
be 80 % of that of V1—that of V3 60 %. Hence, cortical magnification
is somewhat lower in V2 and V3 than in V1 [50, 51], but magnifica-
tion factors decrease with eccentricity at similar rates in V1, V2, and
V3 [50]. In fact the relative size of V1, V2, and V3 depend on the
eccentricity range explored, e.g. in the Abdollahi et al. study [56] V2
is actually slightly larger than V1.
The retinotopic maximum probability maps allow also com-
parison with other parcellations of the same region, in particular
those based on morphological features. Figure 5a comparers the
retinotopic regions with the average myelin density maps based on
the T1/T2 ratio [4]. It shows that the three early visual cortical
areas are heavily myelinated. The three early areas correspond rela-
tively closely to the cytoarchitectonic areas hOc1, hOc2, and the
combination of hOc3d and hOc3v, respectively (Fig. 5b). On the
other hand the retinotopic parcellation has little in common with
earlier attempts to parcel occipital cortex using DTI [65]. The
comparisons in Fig. 5 also indicate that the central 7.75° of the
visual field are represented in roughly half of the V1–V3 surface.

Fig. 5 (a) Outlines (black lines) of retinotopic MPMs superimposed on myelin density(blue to red color) maps
of left hemisphere (L) of 196 subjects. Color code: myelin content in percentiles of the normalized T1w/T2w
distribution. (b) outlines (white) of retinotopic MPM superimposed on the cytoarchitectonic MPM of left (L)
hemisphere. Black lines: 50 % contours of PAs of hOc1, hOc2, hOc5 from Fischl et al. [64]. Inset color code of
cytoarchitectonic areas (from Abdollahi et al. [56])
 Functional Imaging of the Human Visual System 555

3.2 Two Middle- V5 or the Middle Temporal (MT) area in humans was initially
Level Areas: Human localized in the ascending branch of the inferior temporal sulcus
MT/V5 and V3A (ITS) [66, 67]. This identification was supported by the fMRI
study of Tootell et al. [68], showing that this region of human cor-
tex has properties, such as luminance and color contrast sensitivity,
similar to those of macaque MT/V5. Subsequently this region has
been referred to as human MT/V5+ [52] to indicate that probably
it corresponds not just to MT/V5 of the macaque but also to sev-
eral of its satellites. It has proven difficult to demonstrate a retino-
topic organization in this region. Huk et al. [69] have suggested
that the MT/V5 complex in humans contains a posterior retino-
topic part, considered the homolog of MT/V5, and an anterior
part driven by ipsilateral stimuli [70], considered the homolog of
MST. One of the drawbacks of this parcellation was the absence of
an homolog of the fundus of the superior temporal (FST) area. Also
the retinotopic organization of what was believed to be MT in
humans [69, 71] seems opposite to that of macaque MT in which
the lower visual field projects in the dorsal part of MT [72, 73]. The
breakthrough occurred when refining the sweeping technique
proved that MT and its satellites could be mapped in the macaque
(Fig. 4) [74]. Applying the same strategy to humans yielded a MT
cluster organized exactly as in the monkey and including four reti-
notopic areas, considered homologues of MT, MSTv, FST, and V4t
(Figs. 2 and 3). The critical point was to identify the central visual
field representation in the eccentricity maps, as it corresponds to the
center of the cluster from which the four areas radiate. This center
is distinct from the central confluence (Fig. 2) and separated from
it by a representation of the periphery, the so-called peripheral edge
(purple in Fig. 2), which was initially noted by Tootell and cowork-
ers [75]. It is noteworthy that in both species the cluster does not
included MSTd involved in optic flow processing [76]. There is at
present little consensus on the criteria to define the human counter-
part of this MST component [69, 77].
In humans, V3A has a similar retinotopic organization as in
macaque: it is defined by a hemifield representation in which the
representations of the two quadrants, separated by the HM, are
neighbors and occupies the banks of the transverse sulcus [78]. The
posterior quadrant is the lower quadrant, separated from that of
V3d by a lower VM. In contrast to macaque V3A, hV3A is motion
sensitive [14, 78, 79]. In the initial mapping study [68] the central
representation of V3A was considered to be fused with that of V1–
V2–V3. Subsequent studies [80–82] have shown that the central
representation is separated from and located more dorsal than that
of the V1–3 confluence, as it generally is in monkeys (5/8 hemi-
spheres in [73]). It has also been noted in humans that this foveal
projection, which V3A shares with V3B (see below), can vary con-
siderably in clarity, being well defined in about half (13/30) hemi-
spheres [83]. In fact the retinotopic organization of dorsal occipital
556 Guy A. Orban and Stefania Ferri

cortex is more complex than initially assumed based on the monkey

model, and this part of cortex includes one or more retinotopic
areas in addition to V3A (see below). It is noteworthy that in all
primates the visual cortex includes an MT area, but that the pres-
ence of an area V3A in new world monkeys in unclear [60, 84].

3.3 The Fate of V4 In their 1995 study, Sereno et al. [51] reported an upper quadrant
in Human representation anterior to V3v, that they labeled V4v as it occupied
Visual Cortex the same position as ventral V4 in macaque. Many studies have
replicated that finding of a lower quadrant in front of V3v, but it
has proven difficult to identify a corresponding dorsal V4 quadrant
in front of dorsal V3 [75]. One possible explanation was that stan-
dard mapping technique locating meridians did not apply. Indeed,
in the macaque the horizontal meridian, which represents the ante-
rior border of ventral V4, forms the boundary of dorsal V4 only
over a short distance, as it curves to join the HM splitting MT/V5
into two halves [73, 85]. Hence, we [3] and others [75] have sug-
gested that the region between V3/V3A and hMT/V5+, which we
refer to as LOS [20], is the homolog of macaque dorsal V4. Indeed
it is located in a position similar to that of dorsal V4 and has func-
tional properties relatively similar to those of macaque dorsal V4,
for example, is sensitive to 3D shape from motion (Fig. 7), to 2D
shape [20], and kinetic boundaries [87, 88].
Yet, subsequent mapping studies concentrating on the central
6° of the visual field have suggested that the two halves of macaque
V4 have become separated in humans and are each integrated into
a separate representation of the contralateral hemifield. Brewer
et al. [89] have shown that a lower quadrant was located in front
of the upper quadrant initially labeled V4v, with the eccentricity
running at right angle to the polar variations. They proposed that
this hemifield, located in front of V3v (Figs. 2 and 3) should be
considered human V4. They went on to describe two additional
maps located in front of hV4: ventral occipital (VO)1 and VO2,
each supposedly containing a hemifield representation.
Interestingly, the two face areas, the fusiform and occipital face
areas are located just lateral to hV4 and VO2, respectively. In the
same vein, Larsson and Heeger [83] have described a complete
hemifield representation in front of V3d, which they refer to as
lateral occipital (LO)1. The posterior half of this region is a lower
quadrant that was initially described by Smith et al. [90] as
V3B. Thus, the posterior parts of hV4 and LO1 apparently seem
more responsive, explaining why they were discovered first. Just as
is the case ventrally, a second hemifield representation has been
described in front of LO1: LO2, of which the anterior border is
close to hMT/V5+. The LO1–2/hV4 scheme led to the sugges-
tion that beyond V1–3 the monkey occipital cortex was not an
adequate model for human cortex [81], prompting some [91] to
attempt to rescue the monkey model by suggesting that human V4
was similar to that of the monkey.
 Functional Imaging of the Human Visual System 557

Our mapping results also favor the LO1–2/hV4 scheme [54,

92]. The final resolution of this problem came with the recogni-
tion that the retinotopic organization of occipito-temporal cortex
in the monkey is more complex than initially appreciated. A recent
study showed that cytoarchitectonic area TEO, located just in
front of V4, and which initially was thought to contain a single
retinotopic map [93] in fact corresponds to four retinotopic areas:
V4A, OTd and PITv and PITd (Fig. 4 [94, 95]). This resolved the
problem in the sense that human cortex also includes a cluster sim-
ilar to the PIT cluster [55] and considered homologous to the
monkey PITs, and that the region between V3 and the PIT cluster
contains six quadrant representations in both species. In the mon-
key four of these quadrants are part of a split organization and only
two combine into a hemifield, while in humans all quadrants form
hemifield representations.

3.4 Dorsal Occipital Human V3A has been suggested to share its central representation
and Intraparietal Areas with an area referred to as V3B, located in front of V3A and dor-
sally from the LO1/LO2 pair [83, 96]. V3B occupied in this
scheme a position initially referred to as V7 [97]. V7 is now instead
described as an area rostro-dorsal to a complex of dorsal occipital
areas, the V3A complex, which includes four hemifields organized
pairwise (Figs. 3 and 4). The lower pair, V3A/V3B shares its
peripheral representation (P-cluster) like hV4 and VO1, while the
upper pair V3C/V3D shares a central representation (C-cluster).
Area V7 instead is a parietal area corresponding to IPS0 of Swisher
et al. [98] and seems to correspond to the ventral intraparietal sul-
cus (VIPS) motion-sensitive region [79, 82, 99], located in the
most ventral part of the occipital part of human intraparietal sulcus
(IPS) [100]. In fact V7 is part of another C-cluster sharing its cen-
ter with V7A [101], corresponding to IPS1 and likely the homo-
logue of the pair CIP1–2 described in the monkey [102, 103].
In the human parieto-occipital sulcus (POS) Pitzalis et al. [104]
have described human V6, which borders the dorsal parts of V2 and
V3, representing large eccentricities in the lower visual field (Fig. 3b),
and seems to be homologous in both species. It represents the con-
tralateral hemifield, but with an emphasis on the periphery of the
visual field rather than the center. Pitzalis et al. [105] described
lower-field only representation in the opposite bank of the POS
(Fig. 3b), which they labeled human V6A. This area shows strong
pointing responses, unlike V6, and likely belongs to parietal cortex.
Finally, several attempts have been made to parcel visual regions
in human IPS. Using standard retinotopic mapping, Swisher et al.
[98] described four retinotopic maps, labeled IPS1–4, separated by
VM representations. Konen and Kastner [106] added IPS5 and
SPL1, relying again only on polar angle maps. Responses to stan-
dard retinotopic stimuli are weak in this region, and within ante-
rior parts of IPS moving stimuli are more appropriate to map
558 Guy A. Orban and Stefania Ferri

retinotopic organization than are black and white flickering

checkerboards (Fig. 1). Others have used attentional stimuli [107],
delayed saccade stimuli [108–110] or stereoscopic stimuli [101] to
map retinotopic organization. Progress will come not just from
using more appropriate stimuli but also recognizing that eccentric-
ity has to be mapped in addition to polar angle in order to identify
correctly retinotopic clusters, which seem to be the dominant
organization. Our preliminary results suggest that human IPS
includes two additional C-clusters including together four to eight
areas. Further work is need to understand the retinotopic organi-
zation of this part of human parietal cortex and its relationship to
the monkey organization in four areas (LIPv, LIPd, VIP, and AIP)

3.5 Conclusions Human occipital cortex is now almost completely mapped and
includes 19 areas: early areas V1–V3, middle areas LO1–2, hV4,
ventral areas VO1–2, dorsal areas V3A–D and hV6, plus the
occipito-temporal MT and PIT clusters. The competing scheme
using only polar angle maps to define areas [111] only lists 12
occipital retinotopic areas.
Most (13/19) areas are similar to those in the monkey (Fig. 4),
if we admit the proposal of Orban et al. [112] that TFO1–2 located
ventrally to V4/PITv in the monkey are the homologues of VO1–
2. The main inter-species differences are the reorganization of V4/
V4A/OTd into LO1–2/hV4, perhaps related to the separation of
the PITs from the central confluence [55], and the emergence of
areas V3B–D. These latter areas seem to have no counterpart in
the monkey in which V3A neighbors CIP1–2, and may relate to
the expansion of IPL in humans giving rise to the occipital part of
IPS. It is noteworthy that clear homologies are present both at
early and high-order level in the occipital cortex, refuting the idea
that the human visual system divergence more and more from its
monkey counterpart as one ascends into the hierarchy. Also homol-
ogous areas may differ in functional properties, e.g. V3A is motion
sensitive in humans and not in monkeys.
In human occipital cortex all areas beyond V1–3 have a hemi-
field organization, while in macaque hemifield representations
seemed for a long time the exception and split representations, with
separate dorsal and ventral quadrants, the rule. Indeed most initially
known areas (V1–4) had split organization with MT/V5 and V3A
being the exceptions. With most areas mapped, only 5/16 areas
have a split representation in the monkey (Fig. 4), still a larger pro-
portion than in humans (3/19). What is the benefit of the hemi-
field arrangement? As noticed earlier the dorsal region between
V3/V3A and hMT/V5+, in macaque as well as in human, has some
particular functional characteristics, such as 3D shape from motion
sensitivity. The advantage of the human arrangement is that this
sensitivity applies to the whole visual field, while in macaque it
applies only to the lower field. This might be an evolutionary
 Functional Imaging of the Human Visual System 559

advantage explaining the changes in this region, which has expanded

considerably in humans. More generally the hemifield organization
shortens the distance between neurons with RFs in upper and lower
field allowing a better integration across the visual field. This appar-
ently outweighs the need for shorter distances across neighboring
areas which favors split representations.
Finally it is worth mentioning that most if not all of occipital
cortex is retinotopically organized in both species, and that this
organization, again in both species, is maintained in the visual parts
of parietal cortex but not temporal cortex [103], with the excep-
tion of parahippocampal cortex [113].

4 Motion-Sensitive Regions

4.1 Low-Level The two most prominent motion-sensitive regions in human visual
Motion Regions cortex are human MT/V5+ and V3A (see earlier). They display the
highest z scores in a contrast between moving and static random
dots. Their activation remains significant at low stimulus contrasts
typical of the magnocellular stream [78]. In the occipital cortex
motion responses have also been noted in lingual gyrus, probably
corresponding to ventral V2, V3, and in parts of LOS [20, 79,
114, 115]. This activation pattern depends heavily on the size of
the stimuli. With large stimuli, lower-order motion additionally
recruits hV6 [116].
In the early studies it was noted that some parietal regions
were also responsive to motion in a contrast between moving and
static random dots. Sunaert et al. [79] described four motion-
sensitive regions in the IPS. The ventral IPS (VIPS) region is
located at the bottom of the IPS near hV3A. This region, we
believe corresponds to V7 (see above). The parieto-occipital IPS
(POIPS) region is located dorsally with respect to VIPS, at the
junction of the parieto-occipital sulcus and IPS, in the vicinity of
hV6. Not surprisingly, it represents mainly the peripheral visual
field [82] (Fig. 6). The dorsal IPS medial and anterior (DIPSM
and DIPSA) regions are located in the horizontal part of IPS, and
both represent mainly the central visual field [82] (Fig. 6). They
are considered the homolog of anterior part of lateral intraparietal
(LIP) region (DIPSM) and posterior part of anterior intraparietal
(AIP) region (DIPSA), and indeed DIPSA is located just behind
the region referred to as human homolog of AIP based on activa-
tion by grasping actions [117]. All these regions are also activated
by 3D shape from motion [100], which just as motion itself has a
much more extensive representation in human IPS than in macaque
IPS (Fig. 7) [86, 118]. We have speculated that this might in part
be due to the more extensive tool use in humans than in monkeys,
and using a tool indeed activates DIPSM and DIPSA [119]. These
different parietal regions may be engaged in different visuomotor
Fig. 6 Human motion-sensitive regions: distinction between central and peripheral visual field. (a) Stimulus
configuration in experiment 1: the randomly textured pattern (RTP) was positioned either centrally or 5° into
left and right visual field (red dot indicates fixation point). (b and c) Statistical parametric maps (SPMs) showing
voxels significant (yellow: p < 0.0001 uncorrected for multiple comparisons, corresponding to a false discovery
rate of less than 5 % false positives; red: p < 0.001 uncorrected) in the group random-effects analysis (experi-
ment 1, n = 16) for the subtraction moving minus stationary conditions for the centrally (b) and peripherally
(right visual field) (c) positioned stimulus, rendered on the posterior and superior views of the standard human
brain. Further statistical testing revealed that the interaction between type of stimulus (motion, stationary) and
location (center, periphery) was significant (random effects analysis) in DIPSA (Z = 3.12, p < 0.001 uncorrected
and Z = 3.58, p < 0.001 uncorrected for right and left, respectively), DIPSM (Z = 3.58, p < 0.001 uncorrected
and Z = 4.35, p < 0.0001 uncorrected for right and left, respectively) and weakly in POIPS (Z = 2.69, p < 0.01
uncorrected and Z = 2.24, p < 0.01 uncorrected for right and left, respectively). (d) Overlap of voxels (p < 0.001
uncorrected; yellow) in the group random-effects analysis for the subtraction moving minus stationary condi-
tions for the centrally (red) and peripherally (right and left visual field; green) positioned stimulus (experiment
1), rendered on the posterior and superior views of the standard human brain. (e) Stimulus configuration in
experiment 2: RTP was positioned centrally or at 5° eccentricity on upper or lower vertical or horizontal merid-
ian. (f–i) SPMs showing voxels significant (p < 0.05 corrected) in experiment 2 (n = 3) for the subtraction mov-
ing minus stationary conditions for the stimuli positioned in the central visual field (f, red), peripherally left and
right on the horizontal meridian (g, green), and on the lower (h, blue) and upper vertical meridian (i, white),
rendered on the superior view of the standard human brain (posterior part). (j) SPM showing voxels that are
active only in the central condition (obtained by exclusive masking of the subtraction in (f) with those in (g–i)).
 Functional Imaging of the Human Visual System 561

Fig. 7 Visual cortical regions sensitive to 3D shape from motion in human and macaque. Statistical parametric
maps (SPMs) for the subtraction viewing of 3D rotating lines minus viewing of 2D translating lines (p < 0.05,
corrected) of a single human (a) and monkey (M4) (b) subject projected on the posterior part of the flattened
right hemisphere. White stippled and solid lines: vertical and horizontal meridian projections (from separate
retinotopic mapping experiments); black stippled lines: motion-responsive regions from separate motion local-
izing tests; purple stippled lines: region of interspecies difference encompassing V3 and intraparietal sulcus.
PCS post-central sulcus, IPS intraparietal sulcus, LaS lateral sulcus, POS parieto-occipital sulcus, CAS calca-
rine sulcus, STS superior temporal sulcus, ITS inferior temporal sulcus, CoS collateral sulcus, IOS inferior
occipital sulcus, OTS occipito-temporal sulcus, PMTS posterior middle temporal sulcus, AMTS anterior middle
temporal sulcus (modified from Vanduffel et al. [86])

control circuits, for example, in the control of heading [120], or

tracking [121]. Furthermore, it has been shown that flicker is
rejected gradually from hMT/V5+ to the more anterior IPS regions
[72, 99, 122].
Further regions sensitive to motion, but not to 3D shape
from motion, are V6 and premotor regions corresponding to the
frontal eye field (FEF) [79, 100, 116], as well as a region in the
posterior insula, caudal to the somatosensory opercular complex

Fig. 6 (continued) The opposite procedure, subtractions (g–i) masked by that in (f), yielded no active voxels.
R right, L left, VF visual field. White and yellow numbers in (a) and (e) indicate eccentricity and diameter (diam-
eter), respectively. Numbers in (b–d) correspond to the activation sites listed: 1 and 8: hV3A; 2 and 9: lingual
gyrus; 3, 10, and 11: hMT/V5+; 4: LOS; 5 and 12: VIPS; 13: POIPS; 6: DIPSM; and 7: DIPSA [82]
562 Guy A. Orban and Stefania Ferri

[123], which we refer to as posterior insular cortex (PIC) region

[79, 118] and which might be the homolog of a visual region
located next to the posterior insular vestibular cortex (PIVC) in
macaques [14, 124, 125].

4.2 The Kinetic Using kinetic gratings, that is, stimuli in which random dots
Occipital (KO) Region move in opposite directions in alternate stripes, and comparing
them to luminance gratings or uniform motion, our group [87,
126, 127] discovered a region located between V3/V3A and
hMT/V5+ that appeared selective for kinetic boundaries and
that we referred to as the kinetic occipital (KO) region. Recent
work by Zeki et al. [128] has proposed that KO responds to
boundaries defined by other cues (e.g., colors). These findings
do not dispute the responsiveness of KO to kinetic gratings as
several groups have observed these responses [83, 129].
Although they have been presented differently, these findings are
in fact consistent with our PET [127] and fMRI studies [87]
showing responses in KO for both kinetic and luminance grat-
ings, suggesting that KO responds to contours of different
nature, not just kinetic contours. However, it is important to
emphasize that in contrast with responses in hMT/V5+ and
other motion-sensitive regions, KO is selective for kinetic con-
tours as opposed to uniform motion. Thus, we meant selectivity
in the motion domain, not in the domain of cues defining con-
tours, when we stated [87] that KO is selective for kinetic
boundaries. In the Van Oostende et al. study [87] we observed
overlap of the KO region with response to the LO localizer.
Indeed, Larsson and Heeger [83] in their study identifying
LO1/2 showed that the maximal response to kinetic gratings
compared to transparent motion, the contrast most sharply
defining KO [87], was strongest in LO1 and V3A/B. The coor-
dinates of LO1 [83] are very similar to those of KO (±31, −91,
0, and −32, −92, 0 [87]), supporting the identifying LO1 as the
core region of KO. Thus KO is another functionally defined
region that is incorporated into retinotopic regions, as those
become known, the human motion area [66], or hMT/V5+,
being the primary example, and EBA [130] another one [92].

4.3 High-Level All these motion-sensitive regions are low-level motion regions in
Motion Area the sense that they are driven by motion of light over the retina.
Claeys et al. [99] provided evidence for an attention-based motion-
sensitive region in the inferior parietal lobule (IPL). This region
has activated equiluminant color gratings in which one of the col-
ors is more salient than the other, a paradigm tapping third-order
motion [131, 132]. In addition this region has a bilateral represen-
tation of the visual field, while all other motion-sensitive areas have
mainly a contralateral representation.
 Functional Imaging of the Human Visual System 563

5 Shape-Sensitive Regions

There is accumulating evidence that neuronal processes supporting

object recognition are coarsely localized in the ventral visual stream
[133] that contains a hierarchy of cortical processing stages
(V1 → V2 → V4 → IT). The highest stages of this stream (i.e., ante-
rior inferior temporal cortex, AIT or anterior TE in the monkey,
and the rostral part of LOC in the human [20, 21, 134, 135]) are
thought to be involved in shape processing and support object
recognition (Fig. 8). But how are these neuronal representations

Fig. 8 (a) Shape-sensitive regions in human occipital, temporal, and parietal cortex adapted from Sawamura et al.
[136]: yellow: voxels significant (p < 0.05; corrected) in the subtraction 32-objects minus identical condition; black
and blue lines: borders of shape-sensitive regions [i.e., voxels significant in the subtractions intact vs. scrambled
images] obtained by Sawamura et al. [136] and Denys et al. [20] respectively. Numbers: local maxima listed in
black. (b) Face, place, and body patches in human occipital and occipito-temporal cortex: activations are projected
onto flattened right hemisphere of fsaverage atlas. Faces (red)- and body (dark blue) selective regions: approximate
probabilistic data of Engell and McCarthy [137]; Dynamic facial expressions (ocre): real data from Zhu et al. [138];
ATFP: approximate data from (Rajimehr et al. [139]; aSTS: approximate data from (Pitcher et al. [140], showing
Talairach coordinates of individual activations (local maxima). Scene patches (light blue) from Nasr et al. [141].
Biological motion sensitivity (green shape main effect, white kinematics main effect, white interaction): approxi-
mate data from Jastorff and Orban [142]. Regions sensitive for primate vocalizations (black outlines) and intelligible
speech (dashed yellow outlines): real data from Joly et al. [143]. Retinotopy: approximate probabilistic data from
PALS-B12 atlas; VIPS, POIPS, DIPSM, DIPSA, and phAIP (green dashed outlines): are approximate locations from
Jastorff et al. [144],. White star is central representation in V6
564 Guy A. Orban and Stefania Ferri

that support object recognition constructed in the brain? In the

monkey, the visual system has been suggested to recruit a hierar-
chical network of areas across the ventral visual pathway [133,
145] with selectivity for features of increasing complexity from
early to later stages of processing [146]. Recent neuroimaging
studies suggest a similar organization in the human brain. That is,
local image features (e.g., position, orientation) are shown to be
processed at the first stage of cortical processing (V1) [11, 37]
while complex shapes and even abstract object categories (faces,
bodies, places) are represented toward the end of the pathway in
the LOC [147–150]. Combined monkey and human fMRI studies
showed that the perception of global shapes involves both early
(retinotopic) and higher (occipito-temporal) visual areas that may
integrate local elements to global shapes at different spatial scales
[151, 152]. However, unlike neurons in early visual areas that inte-
grate local information about global shapes within the neighbor-
hood of their receptive fields, neural populations in the LOC
represent the perceived global form of objects. In particular, recent
imaging studies [153] have shown fMRI adaptation in LOC when
the perceived shape of visual stimuli was identical but the image
contours differed (because occluding bars occurred in front of the
shape in one stimulus and behind the shape in the other). In con-
trast, recovery from adaptation was observed when the contours
were identical but the perceived shapes were different (because of
a figure-ground reversal).
The idea of a single, general ventral stream processing objects,
has been contradicted by the recent findings of multiple specialized
regions processing faces, bodies, and scenes (Fig. 8b [58])This has
led to the view that in addition to a general purpose object process-
ing system housed in LOC, the human ventral pathway includes also
category specific processing regions [154]. This compromise is not
very satisfactory as it implied a dissociation between semantic and
visual definition of categories and the fact that general purpose
mechanisms for categorization have been located in prefrontal and
parietal cortex [155] and not in inferotemporal cortex of the mon-
key [156]. Therefore, we have recently proposed that the ventral
visual pathway is organized in three stages [103]: first a retinotopic
stage which included the phPIT cluster, processing visual features of
the image; second the anterior part of LOC, corresponding to mon-
key TE, processing real world entities (RWE), a general term cover-
ing objects, faces, and bodies, and third the temporal pole, processing
known, complete RWEs. Furthermore the second stage operates in
parallel with more dorsal regions processing actions and more ven-
tral regions processing scenes. This middle stage is subdivided into a
more dorsal substream processing shape and a more ventral sub-
stream processing material properties (color, texture etc.).
The parallel streams and substreams for general shape, faces,
bodies, and material properties start in the rostral part of the
 Functional Imaging of the Human Visual System 565

retinotopic cortex, as shown by the overlap between the caudal

face, body and color patches, and retinotopic cortex. For example
OFA overlaps with retinotopic cortex but also the posterior two
thirds of EBA [92]. At more anterior levels, i.e. the second and
third stage, retinotopy is absent, as stated above. Central-periphery
organization has been reported at this level [157, 158] but this is
the simple consequence of the fact that faces require detail avail-
able in central vision while scenes require at least moderately large
eccentricities. Finally along these streams and substreams the visual
information is gradually abstracted away from the image proper-
ties. This is best documented for the LOC, and its monkey coun-
terpart TE [23, 136, 146, 148, 159], i.e. the general shape
substream, but likely applies to all (sub)streams [160]. In particu-
lar, representations in the anterior subregion of the LOC in the
fusiform gyrus (pFs) were shown to be largely invariant to size and
position, but not invariant to the direction of illumination and
rotation around the vertical axis. In contrast, representations in the
posterior subregion of the LOC in the lateral occipital (LO) cortex
did not show size or position invariance [23, 24].

6 Depth Processing and 3D Shape Perception

Neurophysiological studies have revealed selectivity for binocular

disparity at multiple levels of the visual hierarchy in the monkey
brain from early visual areas, to object- and motion-selective areas
and the parietal cortex (for reviews: [161–164]). Imaging studies
have identified multiple human brain areas in the visual, temporal,
and parietal cortex that show stronger activations for stimuli
defined by binocular or monocular depth cues than for 2D versions
of these stimuli. In particular, areas V3A [165–168] and V3B/
LO1/KO [87, 128, 129, 169] have been implicated in the analysis
of disparity-defined surfaces and boundaries. Furthermore, studies
have employed parametric manipulations to investigate the neural
correlates of surface depth (i.e., near vs. far) judgments [167, 170]
and 3D shape perception [19]. Finally, several recent studies sug-
gest that areas involved in disparity processing, primarily in the
temporal and parietal cortex, are also engaged in the processing of
monocular cues to depth (e.g., texture, motion, shading) [20, 86,
100, 171–178] and the combination of binocular and monocular
cues for depth perception [179].
Depth relates to the distance from the fixation point and needs
to be combined with eye position information to yield distance
from the observer. The derivatives of depth provide information
about surface orientation and object shape. Gradient selective neu-
rons extracting these derivatives from monocular or binocular
image(s) shave been amply documented in parietal and temporal
cortex of the monkey [15]. A systematic set of fMRI studies [15]
566 Guy A. Orban and Stefania Ferri

have documented the parietal and temporal regions involved in

this extraction in humans. While 3D shape from texture, motion,
and disparity is extracted both in dorsal and ventral pathways, 3D
shape from shading is predominantly processed in ventral regions
close to the phPIT cluster. Systematic combination of single cell
recordings, monkey and human fMRI with identical stimuli have
allowed to infer the presence of gradient selective neurons in some
of the human regions such as pFST or DIPSA [15, 58].

7 Processing of Observed Actions

The visual processing of actions performed by others has been

largely neglected in studies of the visual system [180]. Recent
studies [144, 181] have shown that this information is processed in
regions homologous to the upper and lower bank of middle and
rostral STS of the monkey [112, 182]: posterior MTG/pSTG and
posterior OTS/posterior fusiform cortex respectively. These areas
are also involved in processing biological motion [112, 142, 183].

8 Conclusions

The human visual system likely includes about 40–45 cortical

areas. About two/thirds of these have been identified so far, using
retinotopic mapping, which proved more efficient than functional
properties or morphological features. Further progress can be
expected from mapping retinotopic organization with functionally
more specific stimuli than black and white checkerboards and from
mapping higher-order visual attributes, such as 3D shape or actions,
combined with detection of gradients in maps relying on morpho-
logical features and/or connections [184].

References

1. Van Essen DC (2004) Organization of visual content as revealed by t1- and t2-weighted
areas in macaque and human cerebral cortex. MRI. J Neurosci 31:11597–11616
In: Chalupa LM, Werner JS (eds) The visual 5. Dougherty RF et al (2005) Occipital-callosal
neurosciences, vol 1. MIT Press, Cambridge, pathways in children validation and atlas devel-
MA, pp 507–521 opment. Ann N Y Acad Sci 1064:98–112
2. Nelissen K, Vanduffel W, Orban GA (2006) 6. Schmahmann JD et al (2007) Association
Charting the lower superior temporal region, a fibre pathways of the brain: parallel observa-
new motion-sensitive region in monkey supe- tions from diffusion spectrum imaging and
rior temporal sulcus. J Neurosci 26:5929–5947 autoradiography. Brain 130:630–653
3. Orban GA, Van Essen D, Vanduffel W (2004) 7. Van Essen DC, Jbabdi S, Sotiropoulos SN,
Comparative mapping of higher visual areas Chen C, Dikranian K, Coalson T, Harwell
in monkeys and humans. Trends Cogn Sci J, Behrens TE, Glasser MT (2013) Mapping
8:315–324 connections in humans and non-human pri-
4. Glasser MF, Van Essen DC (2011) Mapping mates: aspirations and challenges for diffu-
human cortical areas in vivo based on myelin sion imaging. In: Johansen-Berg H, Behrens
 Functional Imaging of the Human Visual System 567

TE (eds) Diffusion MRI from quantitative 22. Buckner RL et al (1998) Functional-anatomic

measurement to in vivo neuroanatomy. correlates of object priming in humans
Academic, Amsterdam revealed by rapid presentation event-related
8. Krekelberg B, Boynton GM, van Wezel fMRI. Neuron 20:285–295
RJA (2006) Adaptation: from single cells to 23. Grill-Spector K et al (1999) Differential pro-
BOLD signals. Trends Neurosci 29:250–256 cessing of objects under various viewing con-
9. Kovács G, Kaiser D, Kaliukhovich DA, ditions in the human lateral occipital complex.
Vidnyánszky Z, Vogels R (2013) Repetition Neuron 24:187–203
probability does not affect fMRI rep- 24. Grill-Spector K, Malach R (2001) fMR-
etition suppression for objects. J Neurosci adaptation: a tool for studying the functional
33:9805–9812 properties of human cortical neurons. Acta
10. Sawamura H, Orban GA, Vogels R (2006) Psychol (Amst) 107:293–321
Selectivity of neuronal adaptation does not 25. Grill-Spector K, Henson R, Martin A (2006)
match response selectivity: a single-cell study Repetition and the brain: neural models of
of the fMRI adaptation paradigm. Neuron stimulus-specific effects. Trends Cogn Sci
49:307–318 10:14–23
11. Haynes J-D, Rees G (2006) Decoding mental 26. Koutstaal W et al (2001) Perceptual specificity
states from brain activity in humans. Nat Rev in visual object priming: functional magnetic
Neurosci 7:523–534 resonance imaging evidence for a laterality dif-
12. Vogels R, Orban GA (1990) How well do ference in fusiform cortex. Neuropsychologia
response changes of striate neurons signal dif- 39:184–199
ferences in orientation: a study in the discrim- 27. Vuilleumier P, Henson RN, Driver J, Dolan
inating monkey. J Neurosci 10:3543–3558 RJ (2002) Multiple levels of visual object
13. Haynes J-D, Rees G (2005) Predicting the constancy revealed by event-related fMRI of
orientation of invisible stimuli from activity repetition priming. Nat Neurosci 5:491–499
in human primary visual cortex. Nat Neurosci 28. Lisberger SG, Movshon JA (1999) Visual
8:686–691 motion analysis for pursuit eye movements
14. Vanduffel W et al (2001) Visual motion pro- in area MT of macaque monkeys. J Neurosci
cessing investigated using contrast agent- 19:2224–2246
enhanced fMRI in awake behaving monkeys. 29. Müller JR, Metha AB, Krauskopf J, Lennie P
Neuron 32:565–577 (1999) Rapid adaptation in visual cortex to the
15. Orban GA (2011) The extraction of 3D shape structure of images. Science 285:1405–1408
in the visual system of human and nonhuman 30. Tootell RB et al (1995) Visual motion after-
primates. Annu Rev Neurosci 34:361–388 effect in human cortical area MT revealed
16. Orban GA (2002) Functional MRI in the by functional magnetic resonance imaging.
awake monkey: the missing link. J Cogn Nature 375:139–141
Neurosci 14:965–969 31. Huk AC, Ress D, Heeger DJ (2001) Neuronal
17. Boynton GM, Demb JB, Glover GH, Heeger basis of the motion aftereffect reconsidered.
DJ (1999) Neuronal basis of contrast discrim- Neuron 32:161–172
ination. Vision Res 39:257–269 32. Huk AC, Heeger DJ (2002) Pattern-motion
18. Zenger-Landolt B, Heeger DJ (2003) responses in human visual cortex. Nat
Response suppression in v1 agrees with psy- Neurosci 5:72–75
chophysics of surround masking. J Neurosci 33. Engel SA, Furmanski CS (2001) Selective
23:6884–6893 adaptation to color contrast in human pri-
19. Chandrasekaran C, Canon V, Dahmen JC, mary visual cortex. J Neurosci 21:3949–3954
Kourtzi Z, Welchman AE (2007) Neural 34. Tolias AS, Smirnakis SM, Augath MA, Trinath
correlates of disparity-defined shape discrimi- T, Logothetis NK (2001) Motion process-
nation in the human brain. J Neurophysiol ing in the macaque: revisited with functional
97:1553–1565 magnetic resonance imaging. J Neurosci
20. Denys K et al (2004) The processing of 21:8594–8601
visual shape in the cerebral cortex of human 35. Norman KA, Polyn SM, Detre GJ, Haxby JV
and nonhuman primates: a functional mag- (2006) Beyond mind-reading: multi-voxel
netic resonance imaging study. J Neurosci pattern analysis of fMRI data. Trends Cogn
24:2551–2565 Sci 10:424–430
21. Kourtzi Z, Kanwisher N (2000) Cortical 36. Cox DD, Savoy RL (2003) Functional mag-
regions involved in perceiving object shape. netic resonance imaging (fMRI) “brain read-
J Neurosci 20:3310–3318 ing”: detecting and classifying distributed
568 Guy A. Orban and Stefania Ferri

patterns of fMRI activity in human visual cor- 52. DeYoe EA et al (1996) Mapping striate and
tex. Neuroimage 19:261–270 extrastriate visual areas in human cerebral cor-
37. Kamitani Y, Tong F (2005) Decoding the tex. Proc Natl Acad Sci U S A 93:2382–2386
visual and subjective contents of the human 53. Engel SA, Glover GH, Wandell BA (1997)
brain. Nat Neurosci 8:679–685 Retinotopic organization in human visual
38. Kamitani Y, Tong F (2006) Decoding seen cortex and the spatial precision of functional
and attended motion directions from activ- MRI. Cereb Cortex 7:181–192
ity in the human visual cortex. Curr Biol 54. Georgieva S, Peeters R, Kolster H, Todd JT,
16:1096–1102 Orban GA (2009) The processing of three-
39. Williams MA, Dang S, Kanwisher NG (2007) dimensional shape from disparity in the
Only some spatial patterns of fMRI response human brain. J Neurosci 29:727–742
are read out in task performance. Nat 55. Kolster H, Peeters R, Orban GA (2010) The
Neurosci 10:685–686 retinotopic organization of the human middle
40. O’Toole AJ, Jiang F, Abdi H, Haxby JV temporal area MT/V5 and its cortical neigh-
(2005) Partially distributed representations of bors. J Neurosci 30:9801–9820
objects and faces in ventral temporal cortex. 56. Abdollahi RO et al (2014) Correspondences
J Cogn Neurosci 17:580–590 between retinotopic areas and myelin maps in
41. Hanson SJ, Matsuka T, Haxby JV (2004) human visual cortex. Neuroimage 99:509–524
Combinatorial codes in ventral temporal lobe 57. Robinson EC et al (2013) Multimodal surface
for object recognition: Haxby (2001) revisited: matching: fast and generalisable cortical reg-
is there a “face” area? Neuroimage 23:156–166 istration using discrete optimisation. In: Lect
42. Haxby JV et al (2001) Distributed and Notes Comput Sci (including Subser. Lect
overlapping representations of faces and Notes Artif Intell Lect Notes Bioinformatics)
objects in ventral temporal cortex. Science 7917 LNCS. pp 475–486
293:2425–2430 58. Vanduffel W, Zhu Q, Orban GA (2014)
43. Serences JT, Saproo S, Scolari M, Ho T, Monkey cortex through fMRI glasses.
Muftuler LT (2009) Estimating the influence Neuron 83:533–550
of attention on population codes in human 59. Lyon DC, Kaas JH (2002) Evidence for a
visual cortex using voxel-based tuning func- modified V3 with dorsal and ventral halves in
tions. Neuroimage 44:223–231 macaque monkeys. Neuron 33:453–461
44. Friston KJ, Rotshtein P, Geng JJ, Sterzer P, 60. Rosa MGP, Tweedale R (2005) Brain maps,
Henson RN (2006) A critique of functional great and small: lessons from comparative
localisers. Neuroimage 30:1077–1087 studies of primate visual cortical organiza-
45. Saxe R, Brett M, Kanwisher N (2006) Divide tion. Philos Trans R Soc Lond B Biol Sci
and conquer: a defense of functional localiz- 360:665–691
ers. Neuroimage 30:1088–1096 61. Duncan RO, Boynton GM (2003) Cortical
46. Fox PT et al (1986) Mapping human visual magnification within human primary visual
cortex with positron emission tomography. cortex correlates with acuity thresholds.
Nature 323:806–809 Neuron 38:659–671
47. Schneider W, Noll DC, Cohen JD (1993) 62. Adams DL, Sincich LC, Horton JC (2007)
Functional topographic mapping of the corti- Complete pattern of ocular dominance
cal ribbon in human vision with conventional columns in human primary visual cortex.
MRI scanners. Nature 365:150–153 J Neurosci 27:10391–10403
48. Shipp S, Watson JD, Frackowiak RS, Zeki S 63. Van Essen DC, Newsome WT, Maunsell JHR
(1995) Retinotopic maps in human prestriate (1984) The visual field representation in stri-
visual cortex: the demarcation of areas V2 and ate cortex of the macaque monkey: asymme-
V3. Neuroimage 2:125–132 tries, anisotropies, and individual variability.
49. Engel SA et al (1994) fMRI of human visual Vision Res 24:429–448
cortex. Nature 369:525 64. Fischl B et al (2008) Cortical folding pat-
50. Dougherty RF et al (2003) Visual field represen- terns and predicting cytoarchitecture. Cereb
tations and locations of visual areas V1/2/3 in Cortex 18:1973–1980
human visual cortex. J Vis 3:586–598 65. Hagmann P et al (2008) Mapping the struc-
51. Sereno MI et al (1995) Borders of multiple tural core of human cerebral cortex. PLoS
visual areas in humans revealed by func- Biol 6:1479–1493
tional magnetic resonance imaging. Science 66. Zeki S et al (1991) A direct demonstration of
268:889–893 functional specialization in human visual cor-
tex. J Neurosci 11:641–649
 Functional Imaging of the Human Visual System 569

67. Watson JD et al (1993) Area V5 of the human 82. Orban GA et al (2006) Mapping the parietal
brain: evidence from a combined study using cortex of human and non-human primates.
positron emission tomography and magnetic Neuropsychologia 44:2647–2667
resonance imaging. Cereb Cortex 3:79–94 83. Larsson J, Heeger DJ (2006) Two retinotopic
68. Tootell RB et al (1995) Functional analy- visual areas in human lateral occipital cortex.
sis of human MT and related visual cortical J Neurosci 26:13128–13142
areas using magnetic resonance imaging. 84. Lyon DC, Kaas JH (2002) Evidence from V1
J Neurosci 15:3215–3230 connections for both dorsal and ventral sub-
69. Huk AC, Dougherty RF, Heeger DJ (2002) divisions of V3 in three species of new world
Retinotopy and functional subdivision of monkeys. J Comp Neurol 449:281–297
human areas MT and MST. J Neurosci 85. Gattass R, Sousa AP, Gross CG (1988)
22:7195–7205 Visuotopic organization and extent of V3 and
70. Dukelow SP et al (2001) Distinguishing sub- V4 of the macaque. J Neurosci 8:1831–1845
regions of the human MT+ complex using 86. Vanduffel W et al (2002) Extracting 3D from
visual fields and pursuit eye movements. motion: differences in human and monkey
J Neurophysiol 86:1991–2000 intraparietal cortex. Science 298:413–415
71. Smith AT, Wall MB, Williams AL, Singh KD 87. Van Oostende S, Sunaert S, Van Hecke P,
(2006) Sensitivity to optic flow in human Marchal G, Orban GA (1997) The kinetic
cortical areas MT and MST. Eur J Neurosci occipital (KO) region in man: an fMRI study.
23:561–569 Cereb Cortex 7:690–701
72. Van Essen DC, Maunsell JH, Bixby JL 88. Nelissen K, Vanduffel W, Sunaert S, Janssen
(1981) The middle temporal visual area in P, Tootell RB, Orban GA (2000) Processing
the macaque: myeloarchitecture, connec- of kinetic boundaries investigated using fMRI
tions, functional properties and topographic and double-label deoxyglucose technique in
organization. J Comp Neurol 199:293–326 awake monkeys. Soc Neurosci Abstr 26:1584
73. Fize D et al (2003) The retinotopic organi- 89. Brewer AA, Liu J, Wade AR, Wandell BA
zation of primate dorsal V4 and surround- (2005) Visual field maps and stimulus selec-
ing areas: a functional magnetic resonance tivity in human ventral occipital cortex. Nat
imaging study in awake monkeys. J Neurosci Neurosci 8:1102–1109
23:7395–7406 90. Smith AT, Greenlee MW, Singh KD, Kraemer
74. Kolster H et al (2009) Visual field map clus- FM, Hennig J (1998) The processing of first-
ters in macaque extrastriate visual cortex. and second-order motion in human visual cor-
J Neurosci 29:7031–7039 tex assessed by functional magnetic resonance
75. Tootell RB, Hadjikhani N (2001) Where imaging (fMRI). J Neurosci 18:3816–3830
is “dorsal V4” in human visual cortex? 91. Hansen KA, Kay KN, Gallant JL (2007)
Retinotopic, topographic and functional evi- Topographic organization in and near human
dence. Cereb Cortex 11:298–311 visual area V4. J Neurosci 27:11896–11911
76. Tanaka K et al (1986) Analysis of local and 92. Ferri S, Kolster H, Jastorff J, Orban GA
wide-field movements in the superior tem- (2013) The overlap of the EBA and the MT/
poral visual areas of the macaque monkey. V5 cluster. Neuroimage 66:412–425
J Neurosci 6:134–144 93. Boussaoud D, Desimone R, Ungerleider LG
77. Morrone MC et al (2000) A cortical area that (1991) Visual topography of area TEO in the
responds specifically to optic flow, revealed by macaque. J Comp Neurol 306:554–575
fMRI. Nat Neurosci 3:1322–1328 94. Janssens T, Zhu Q, Popivanov ID, Vanduffel
78. Tootell R et al (1997) Functional analysis of W (2014) Probabilistic and single-subject ret-
V3A and related areas in human visual cortex. inotopic maps reveal the topographic organi-
J Neurosci 17:7060–7078 zation of face patches in the macaque cortex.
79. Sunaert S, Van Hecke P, Marchal G, Orban J Neurosci 34:10156–10167
GA (1999) Motion-responsive regions of the 95. Kolster H, Janssens T, Orban GA, Vanduffel
human brain. Exp Brain Res 127:355–370 W (2014) The retinotopic organization of
80. Press WA, Brewer AA, Dougherty RF, Wade macaque occipitotemporal cortex anterior to
AR, Wandell BA (2001) Visual areas and spa- V4 and caudoventral to the middle temporal
tial summation in human visual cortex. Vision (MT) cluster. J Neurosci 34:10168–10191
Res 41:1321–1332 96. Wandell BA, Brewer AA, Dougherty RF
81. Wandell BA, Dumoulin SO, Brewer AA (2005) Visual field map clusters in human
(2007) Visual field maps in human cortex. cortex. Philos Trans R Soc Lond B Biol Sci
Neuron 56:366–383 360:693–707
570 Guy A. Orban and Stefania Ferri

97. Tootell RBH, Tsao D, Vanduffel W (2003) raphy in human cortex. Cereb Cortex
Neuroimaging weighs in: humans meet 25(10):3911–3931
macaques in “primate” visual cortex. 112. Orban GA, Jastorff J (2014) Functional map-
J Neurosci 23:3981–3989 ping of motion regions in human and non-
98. Swisher JD, Halko MA, Merabet LB, human primates. In: Chalupa LM, Werner
McMains SA, Somers DC (2007) Visual JS (eds) The new visual neuroscience, vol 1.
topography of human intraparietal sulcus. MIT Press, Cambridge, MA, pp 777–791
J Neurosci 27:5326–5337 113. Arcaro MJ, McMains SA, Singer BD,
99. Claeys KG, Lindsey DT, De Schutter E, Kastner S (2009) Retinotopic organization
Orban GA (2003) A higher order motion of human ventral visual cortex. J Neurosci
region in human inferior parietal lobule: evi- 29:10638–10652
dence from fMRI. Neuron 40:631–642 114. Sunaert S, Van Hecke P, Marchal G, Orban
100. Orban GA, Sunaert S, Todd JT, Van Hecke GA (2000) Attention to speed of motion,
P, Marchal G (1999) Human cortical regions speed discrimination, and task difficulty: an
involved in extracting depth from motion. fMRI study. Neuroimage 11:612–623
Neuron 24:929–940 115. Rees G, Friston K, Koch C (2000) A direct
101. Kolster H, Peeters R, Orban GA (2011) Ten quantitative relationship between the func-
retinotopically organized areas in human tional properties of human and macaque V5.
paritetal cortex. Soc Neurosci Abstr 851.10 Nat Neurosci 3:716–723
102. Arcaro MJ, Pinsk MA, Li X, Kastner S 116. Pitzalis S et al (2010) Human V6: the medial
(2011) Visuotopic organization of macaque motion area. Cereb Cortex 20:411–424
posterior parietal cortex: a functional mag- 117. Binkofski F et al (1998) Human anterior
netic resonance imaging study. J Neurosci intraparietal area subserves prehension: a
31:2064–2078 combined lesion and functional MRI activa-
103. Orban GA, Zhu Q, Vanduffel W (2014) The tion study. Neurology 50:1253–1259
transition in the ventral stream from feature 118. Orban GA et al (2003) Similarities and dif-
to real-world entity representations. Front ferences in motion processing between the
Psychol 5:695 human and macaque brain: evidence from
104. Pitzalis S et al (2006) Wide-field retino- fMRI. Neuropsychologia 41:1757–1768
topy defines human cortical visual area v6. 119. Stout D, Chaminade T (2007) The evo-
J Neurosci 26:7962–7973 lutionary neuroscience of tool making.
105. Pitzalis S et al (2013) The human homo- Neuropsychologia 45:1091–1100
logue of macaque area V6A. Neuroimage 120. Peuskens H et al (2001) Human brain regions
82:517–530 involved in heading estimation. J Neurosci
106. Konen CS, Kastner S (2008) Two hierar- 21:2451–2461
chically organized neural systems for object 121. Gori M et al (2012) Long integration time
information in human visual cortex. Nat for accelerating and decelerating visual, tac-
Neurosci 11:224–231 tile and visuo-tactile stimuli. Multisens Res
107. Silver MA, Ress D, Heeger DJ (2005) 26:53–68
Topographic maps of visual spatial attention 122. Braddick OJ, O’Brien JMD, Wattam-Bell
in human parietal cortex. J Neurophysiol J, Atkinson J, Turner R (2000) Form and
94:1358–1371 motion coherence activate independent, but
108. Sereno MI, Pitzalis S, Martinez A (2001) not dorsal/ventral segregated, networks in
Mapping of contralateral space in retino- the human brain. Curr Biol 10:731–734
topic coordinates by a parietal cortical area in 123. Eickhoff SB, Grefkes C, Zilles K, Fink GR
humans. Science 294:1350–1354 (2007) The somatotopic organization of
109. Schluppeck D, Curtis CE, Glimcher PW, cytoarchitectonic areas on the human parietal
Heeger DJ (2006) Sustained activity in operculum. Cereb Cortex 17:1800–1811
topographic areas of human posterior pari- 124. Grüsser OJ, Pause M, Schreiter U (1990)
etal cortex during memory-guided saccades. Vestibular neurones in the parieto-insular
J Neurosci 26:5098–5108 cortex of monkeys (Macaca fascicularis):
110. Schluppeck D, Glimcher P, Heeger DJ visual and neck receptor responses. J Physiol
(2005) Topographic organization for delayed 430:559–583
saccades in human posterior parietal cortex. 125. Grüsser O-J, Guldin WO, Mirring S, Salah-
J Neurophysiol 94:1372–1384 Eldin A (1994) Comparative physiological
111. Wang L, Mruczek REB, Arcaro MJ, Kastner and anatomical studies of the primate vestibu-
S (2015) Probabilistic maps of visual topog- lar cortex. In: Albowitz B, Albus K, Kuhnt
 Functional Imaging of the Human Visual System 571

U, Nothdurft H-C, Wahle P (eds) Structural ity for dynamic versus static information in
and functional organization of the neocortex. face-selective cortical regions. Neuroimage
Proceedings of a Symposium in the Memory 56:2356–2363
of Otto D. Creutzfeldt, May 1993, Exp Brain 141. Nasr S et al (2011) Scene-selective cortical
Res Series 24. pp 358–371 regions in human and nonhuman primates.
126. Orban GA et al (1995) A motion area in J Neurosci 31:13771–13785
human visual cortex. Proc Natl Acad Sci U S 142. Jastorff J, Orban GA (2009) Human func-
A 92:993–997 tional magnetic resonance imaging reveals
127. Dupont P et al (1997) The kinetic occipital region separation and integration of shape and
in human visual cortex. Cereb Cortex 7:283–292 motion cues in biological motion processing.
128. Zeki S, Perry RJ, Bartels A (2003) The pro- J Neurosci 29:7315–7329
cessing of kinetic contours in the brain. Cereb 143. Joly O et al (2012) Processing of vocaliza-
Cortex 13:189–202 tions in humans and monkeys: a comparative
129. Tyler CW, Likova LT, Kontsevich LL, Wade AR fMRI study. Neuroimage 62:1376–1389
(2006) The specificity of cortical region KO to 144. Jastorff J, Begliomini C, Fabbri-Destro M,
depth structure. Neuroimage 30:228–238 Rizzolatti G, Orban GA (2010) Coding
130. Downing PE, Jiang Y, Shuman M, Kanwisher observed motor acts: different organizational
N (2001) A cortical area selective for visual principles in the parietal and premotor cortex
processing of the human body. Science of humans. J Neurophysiol 104:128–140
293:2470–2473 145. Felleman DJ, Van Essen DC (1991)
131. Lu ZL, Sperling G (1995) Attention-generated Distributed hierarchical processing in the pri-
apparent motion. Nature 377:237–239 mate cerebral cortex. Cereb Cortex 1:1–47
132. Lu ZL, Lesmes LA, Sperling G (1999) 146. Tanaka K, Saito H, Fukada Y, Moriya M
The mechanism of isoluminant chromatic (1991) Coding visual images of objects in the
motion perception. Proc Natl Acad Sci U S A inferotemporal cortex of the macaque mon-
96:8289–8294 key. J Neurophysiol 66:170–189
133. Ungerleider LG, Mishkin M (1982) Two 147. Grill-Spector K, Malach R (2004) The human
cortical visual systems. Anal Vis Behav visual cortex. Annu Rev Neurosci 27:649–677
549:549–586 148. Quiroga RQ, Reddy L, Kreiman G, Koch C,
134. Malach R et al (1995) Object-related activ- Fried I (2005) Invariant visual representation
ity revealed by functional magnetic resonance by single neurons in the human brain. Nature
imaging in human occipital cortex. Proc Natl 435:1102–1107
Acad Sci U S A 92:8135–8139 149. Reddy L, Kanwisher N (2006) Coding of
135. Kanwisher N, Chun MM, McDermott J, visual objects in the ventral stream. Curr Opin
Ledden PJ (1996) Functional imaging of human Neurobiol 16:408–414
visual recognition. Cogn Brain Res 5:55–67 150. Privman E et al (2007) Enhanced category
136. Sawamura H, Georgieva S, Vogels R, tuning revealed by intracranial electroenceph-
Vanduffel W, Orban GA (2005) Using func- alograms in high-order human visual areas.
tional magnetic resonance imaging to assess J Neurosci 27:6234–6242
adaptation and size invariance of shape pro- 151. Altmann CF, Bülthoff HH, Kourtzi Z (2003)
cessing by humans and monkeys. J Neurosci Perceptual organization of local elements into
25:4294–4306 global shapes in the human visual cortex.
137. Engell AD, McCarthy G (2013) Probabilistic Curr Biol 13:342–349
atlases for face and biological motion percep- 152. Kourtzi Z, Tolias AS, Altmann CF, Augath
tion: an analysis of their reliability and over- M, Logothetis NK (2003) Integration of
lap. Neuroimage 74:140–151 local features into global shapes: monkey and
138. Zhu Q et al (2012) Dissimilar processing human fMRI studies. Neuron 37:333–346
of emotional facial expressions in human 153. Kourtzi Z, Kanwisher N (2001) Human
and monkey temporal cortex. Neuroimage lateral occipital complex representation of
66C:402–411 perceived object shape by the human lateral
139. Rajimehr R, Young JC, Tootell RBH (2009) occipital complex. Science 293:1506–1509
An anterior temporal face patch in human 154. Downing PE, Chan AW-Y, Peelen MV, Dodds
cortex, predicted by macaque maps. Proc CM, Kanwisher N (2006) Domain specificity
Natl Acad Sci U S A 106:1995–2000 in visual cortex. Cereb Cortex 16:1453–1461
140. Pitcher D, Dilks DD, Saxe RR, Triantafyllou 155. Freedman DJ, Riesenhuber M, Poggio T,
C, Kanwisher N (2011) Differential selectiv- Miller EK (2003) A comparison of primate
572 Guy A. Orban and Stefania Ferri

prefrontal and inferior temporal cortices human object-related visual areas. Hum Brain
during visual categorization. J Neurosci Mapp 15:67–79
23:5235–5246 171. Orban GA (2007) Three-dimensional shape:
156. Vogels R (1999) Categorization of com- cortical mechanisms of shape extraction. In:
plex visual images by rhesus monkeys. Masland RH, Albright T (eds) Handbook of
Part 2: Single-cell study. Eur J Neurosci the senses, vol 5, Vision. Elsevier, Amsterdam
11:1239–1255 172. Durand JB et al (2007) Anterior regions of
157. Levy I, Hasson U, Avidan G, Hendler T, Malach monkey parietal cortex process visual 3D
R (2001) Center-periphery organization of shape. Neuron 55:493–505
human object areas. Nat Neurosci 4:533–539 173. James TW et al (2002) Haptic study of three-
158. Hasson U, Levy I, Behrmann M, Hendler T, dimensional objects activates extrastriate visual
Malach R (2002) Eccentricity bias as an orga- areas. Neuropsychologia 40:1706–1714
nizing principle for human high-order object 174. Kourtzi Z, Erb M, Grodd W, Bülthoff HH
areas. Neuron 34:479–490 (2003) Representation of the perceived 3-D
159. Rolls ET (2000) Functions of the primate object shape in the human lateral occipital
temporal lobe cortical visual areas in invariant complex. Cereb Cortex 13:911–920
visual object and face recognition. Neuron 175. Murray SO, Olshausen BA, Woods DL
27:205–218 (2003) Processing shape, motion and three-
160. Freiwald WA, Tsao DY (2010) Functional dimensional shape-from-motion in the
compartmentalization and viewpoint gener- human cortex. Cereb Cortex 13:508–516
alization within the macaque face-processing 176. Sereno ME, Trinath T, Augath M, Logothetis
system. Science 330:845–851 NK (2002) Three-dimensional shape represen-
161. Cumming BG, DeAngelis GC (2001) The tation in monkey cortex. Neuron 33:635–652
physiology of stereopsis. Annu Rev Neurosci 177. Shikata E et al (2001) Surface orienta-
24:203–238 tion discrimination activates caudal and
162. Parker AJ (2007) Binocular depth perception and anterior intraparietal sulcus in humans: an
the cerebral cortex. Nat Rev Neurosci 8:379–391 event-related fMRI study. J Neurophysiol
163. Neri P, Bridge H, Heeger DJ (2004) 85:1309–1314
Stereoscopic processing of absolute and 178. Taira M, Nose I, Inoue K, Tsutsui K (2001)
relative disparity in human visual cortex. Cortical areas related to attention to 3D sur-
J Neurophysiol 92:1880–1891 face structures based on shading: an fMRI
164. Orban GA, Janssen P, Vogels R (2006) study. Neuroimage 14:959–966
Extracting 3D structure from disparity. 179. Welchman AE, Deubelius A, Conrad V,
Trends Neurosci 29:466–473 Bülthoff HH, Kourtzi Z (2005) 3D shape per-
165. Gulyas B, Roland PE (1994) Processing and ception from combined depth cues in human
analysis of form, colour and binocular dis- visual cortex. Nat Neurosci 8:820–827
parity in the human brain: functional anat- 180. Perrett DI et al (1985) Visual analysis of body
omy by positron emission tomography. Eur movements by neurones in the temporal cor-
J Neurosci 6:1811–1828 tex of the macaque monkey: a preliminary
166. Mendola JD, Dale AM, Fischl B, Liu AK, report. Behav Brain Res 16:153–170
Tootell RB (1999) The representation of illu- 181. Abdollahi RO, Jastorff J, Orban GA (2013)
sory and real contours in human cortical visual Common and segregated processing of
areas revealed by functional magnetic reso- observed actions in human SPL. Cereb
nance imaging. J Neurosci 19:8560–8572 Cortex 23:2734–2753
167. Backus BT, Fleet DJ, Parker AJ, Heeger 182. Jastorff J, Popivanov ID, Vogels R, Vanduffel
DJ (2001) Human cortical activity corre- W, Orban GA (2012) Integration of shape and
lates with stereoscopic depth perception. motion cues in biological motion processing
J Neurophysiol 86:2054–2068 in the monkey STS. Neuroimage 60:911–921
168. Tsao DY et al (2003) Stereopsis activates V3A 183. Grossman E et al (2000) Brain areas involved
and caudal intraparietal areas in macaques and in perception of biological motion. J Cogn
humans. Neuron 39:555–568 Neurosci 12:711–720
169. Brouwer GJ, van Ee R, Schwarzbach J (2005) 184. Glasser MF, Robinson EC, Coalson TS, Smith
Activation in visual cortex correlates with the SM, Jenkinson M, Hacker CS, Laumann TO,
awareness of stereoscopic depth. J Neurosci Van Essen DC (2014) Partial correlation
25:10403–10413 functional connectivity gradients for cortical
170. Gilaie-Dotan S, Ullman S, Kushnir T, Malach parcellation: methods and multi-modal com-
R (2002) Shape-selective stereo processing in parisons SFN abstract WCC 147B
 Chapter 19

fMRI of the Central Auditory System

Deborah Ann Hall and Aspasia Eleni Paltoglou

Abstract
Over the years, blood oxygen level-dependent (BOLD) fMRI has made important contributions to the under-
standing of central auditory processing in humans. Although there are significant technical challenges to over-
come in the case of auditory fMRI, the unique methodological advantage of fMRI as an indicator of population
neural activity lies in its spatial precision. It can be used to examine the neural basis of auditory representation
at a number of spatial scales, from the micro-anatomical scale of population assemblies to the macro-anatomical
scale of cortico-cortical circuits. The spatial resolution of fMRI is maximized in the case of mapping individual
brain activity, and here it has been possible to demonstrate known organizational features of the auditory system
that have hitherto been possible only using invasive electrophysiological recording methods. Frequency coding
in the primary auditory cortex is one such example that we shall discuss in this chapter. Of course, noninvasive
procedures for neuroscience are the ultimate aim and as the field moves towards this goal by recording in awake,
behaving animals so human neuroimaging techniques will be increasingly relied upon to provide an interpretive
link between animal neurophysiology at the multi-unit level and the operation of larger neuronal assemblies, as
well as the mechanisms of auditory perception itself. For example, the neural effects of intentional behavior on
stimulus-driven coding have been explored both in animals, using electrophysiological techniques, and in
humans, using fMRI. While the feature-specific effects of selective attention are well established in the visual
cortex, the effect of auditory attention in the auditory cortex has generally been examined at a very coarse spatial
scale. Ongoing research in our laboratory has started to address this question and here we present preliminary
evidence for frequency-specific effects of attentional enhancement in the human auditory cortex. We end with
a brief discussion of several future directions for auditory fMRI research.

Key words Technical challenges, Frequency coding, Selective attention, Perceptual representation,
Task specificity

1 Challenges of Auditory fMRI

The construction of a brain image using MR imaging depends

upon the magnetic properties of hydrogen ions that, when placed
in a static magnetic field, can absorb pulses of radiowave energy of
a specific frequency. The time taken for the ion alignments to
return to equilibrium after the radiofrequency (RF) pulse differs
according to the surrounding tissue, thus providing the image
contrast, for example between gray matter, white matter, cerebro-
spinal fluid, and bone. The use of MR techniques for detecting

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_19, © Springer Science+Business Media New York 2016

573
574 Deborah Ann Hall and Aspasia Eleni Paltoglou

functional brain activation relies on two factors: first that local neu-
ral activity is a metabolically demanding process that is closely asso-
ciated with a local increase in the supply of oxygenated blood to
those active parts of the brain, and second that the different para-
magnetic properties of oxygenated and deoxygenated blood pro-
duce measurable effects on the MR signal. The functional signal
detected during fMRI is known as the blood oxygen level-
dependent (BOLD) response. Essentially, the functional image
represents the spatial distribution of blood oxygenation levels in
the brain, and the small fluctuations in these levels over time are
correlated with the stimulus input or cognitive task.
MR scanners operate using three different types of electromag-
netic fields: a very high static field generated by a superconducting
magnet, time-varying gradient magnetic fields, and pulsed RF fields.
The latter two fields are much weaker than the first, but all pose a
number of unique and considerable technical challenges for conduct-
ing auditory fMRI research within this hostile environment. In the
first place, the static and time-varying magnetic fields preclude the use
of many types of electronic sound presentation equipment, as well as
preventing the safe scanning of patients who are wearing listening
devices such as hearing aids or implants. Additionally, the high levels
of scanner noise generated by the flexing of the gradient coils in the
static magnetic field can potentially cause hearing difficulties. The
scanner noise masks the perception of the acoustic stimuli presented
to the subject in the scanner making it difficult to calibrate audible
hearing levels and adding to the difficulty of the listening task. And
finally, the scanner noise not only activates parts of the auditory brain,
but also interacts with the patterns of activity evoked by experimental
stimuli. Auditory fMRI poses a number of other challenges, not
related to the hostile environment of the MR scanner, but related
instead to the nature of the neural coding in the auditory cortex. The
response of auditory cortical neurons to a particular class of sound is
determined not only by the acoustic features of that sound, but also
by its presentation context. For example, neurons respond strongly to
the onset of sound events and thereafter tend to show rapid adapta-
tion to that sound in terms of a reduction in their firing rate. Thus, the
result of any particular auditory fMRI experiment will depend not
only on the physical attributes of a stimulus, but also on the way in
which the stimuli are presented. In this first section, we shall take each
one of these issues in turn, introducing the problems in more detail as
well as proposing some solutions.

1.1 Use of Electronic The ideal requirement is a sound presentation system that produces
Equipment for Sound a range of sound levels [up to 100-dB sound pressure level (SPL)],
Presentation in the MR with low distortion, a flat frequency response, and a smooth and
Scanner predictable phase response. The first commercially available solu-
tion utilized loudspeakers, placed away from the high static mag-
1.1.1 Problems
netic field, from which the sound was delivered through plastic
tubes inserted into the ear canal (Fig. 1a) through a protective ear
 fMRI of the Central Auditory System 575

Fig. 1 MR compatible headsets for sound delivery and noise reduction: (a) tube phones system with foam ear
inserts, (b) circum-aural ear defenders, plus foam ear plugs for passive noise reduction, (c) MRC IHR sound
presentation headset combining commercially available electrostatic transducers in an industry standard ear
defender, and (d) modified MRC IHR headset for sound presentation and for active noise cancellation (ANC),
including an optical error microphone positioned underneath the ear defender

defender (Fig. 1b). One general disadvantage of the tube phone

system is that the tubing distorts both the phase and amplitude of
the acoustic signal, for example, by imposing a severe ripple on the
spectra and reducing sound level, especially at higher frequencies.
Another limitation is the leak of the scanner noise through the pipe
walls to the pipe inner and hence the ear. Despite alternative sys-
tems now being readily available, tube phone systems are still com-
mercially manufactured (e.g. Avotec Inc. Stuart, Florida, USA,
www.avotec.org/). The Avotec system has been specifically designed
for fMRI use and boasts an equalizer to provide a reasonably flat
audio output (±5 dB) across its nominal bandwidth
(150 Hz–4.5 kHz) and a procedure for acoustic calibration that
feeds a known electrical input signal to the audio system input and
makes a direct acoustic output measurement at the headset.
Alternative electronic systems often used for psychoacoustical
research deliver high-quality signals, but these systems are gener-
ally unsuitable for use in the MR environment because most head-
phones use an electromagnet to push and pull on a diaphragm to
vibrate the air and generate sound. Of course, this electromagnet
576 Deborah Ann Hall and Aspasia Eleni Paltoglou

is rendered inoperable by the magnetic fields in the MR scanner.

Headphone components constructed from ferromagnetic material
also disrupt the magnetic fields locally and induce signal loss or
spatial distortion in areas close to the ears. In addition, the elec-
tronic components can be damaged by the static magnetic field,
while electromagnetic interference generated by the equipment is
detected by the MR receiver head coil. Electronic sound delivery
systems for use in auditory fMRI research have been designed spe-
cifically to overcome these difficulties.

1.1.2 Solutions Despite the restriction on the materials that can be used in a
scanner, a number of different MR-compatible active head-
phone driving units have been produced. An ingenious system
has been developed and marketed by one auditory neuroimag-
ing research group (MR confon GmbH, Magdeburg, Germany,
www.mr-confon.de). This system incorporates a unique, elec-
trodynamic driver that uses the scanner’s static magnetic field in
place of the permanent magnets that are found in conventional
headphones and loudspeakers. It produces a wide frequency
range (less than 200 Hz–35 kHz) with a flat frequency response
(±6 dB). Another company manufactures and supplies high-
quality products for MRI, with a special focus on the fast-grow-
ing field of functional imaging (NordicNeuroLab AS, Bergen,
Norway, www.nordicneurolab.com/). Their audio system uses
electrostatic transducers to ensure high performance.
Electrostatic headphones generate sound using a conductive
diaphragm placed next to a fixed conducting panel. A high volt-
age polarizes the fixed panel and the audio signal passing
through the diaphragm rapidly switches between a positive and
a negative signal, attracting or repelling it to the fixed panel and
thus vibrating the air. Their technical specification claims a flat
frequency response from 8 Hz to 35 kHz. The signal is trans-
ferred from the audio source to the headphones in the RF
screened scanner room using either filters through a filter panel
or fiber-optic cable through the waveguide.
Here at the MRC Institute of Hearing Research, we became
engaged in auditory fMRI research well before such commercial sys-
tems were widely available and so, for our own purposes, we developed
an MR-compatible headset (Fig. 1c) based on commercially available
electrostatic headphones, modified to remove or replace their ferro-
magnetic components, and combined with standard industrial ear
defenders to provide good acoustic isolation [1]. Our custom-built
system delivers a flat frequency response (±10 dB) across the frequency
range 50 Hz–10 kHz and has an output level capability up to 120-dB
SPL. Again, the digital audio source, electronics, and power supply
that drive the system are housed outside the RF screened scanner room
to avoid electromagnetic interference with MR scanning, and all elec-
trical signals passing into the screened scanner room are RF filtered.
 fMRI of the Central Auditory System 577

1.2 Risk to Patients No ferromagnetic components can be placed in the scanner bore as
Who Are Wearing they would experience a strong attraction by the static magnetic
Listening Devices field and potentially cause damage not only to the scanner and the
in the MR Scanner listening device, but also to the patient. Induced currents in the
electronics, caused directly by the time-varying gradient magnetic
1.2.1 Problems fields or the RF pulses, are an additional hazard to the electronic
devices themselves, while some materials can also absorb the RF
energy causing local tissue heating and even burns if in contact with
soft tissue. For these reasons, there are restrictions on scanning peo-
ple who have electronic listening devices. These include hearing
aids, cochlear implants, and brainstem implants. Hearing aids
amplify sound for people who have moderate to profound hearing
loss. The aid is battery-operated and worn in or around the ear.
Hearing aids are available in different shapes, sizes, and types, but
they all work in a similar way. They all have a built-in microphone
that picks up sound from the environment. These sounds are pro-
cessed electronically and made louder, either by analogue circuits or
digitally, and the resulting signals are passed to a receiver in the
hearing aid where they are converted back into audible sounds. In
contrast, cochlear and brainstem implants are both small, complex
electronic devices that can help to provide a sense of sound to peo-
ple who are profoundly deaf or severely hard-of-hearing. Cochlear
implants bypass damaged portions of the inner ear (the cochlea)
and directly stimulate the auditory nerve, while auditory brainstem
implants bypass the vestibulocochlear nerve in cases when it is dam-
aged by tumors or surgery and directly stimulate the lower part of
the auditory brain (the cochlear nucleus). In general, both types of
implant consist of an external portion that sits behind the ear and a
second portion that is surgically placed under the skin. They con-
tain a microphone, a sound processor (which converts sounds
picked up by the microphone into an electrical code), a transmitter
and receiver/stimulator (which receive signals from the processor
and convert them into electric impulses), and finally an electrode
array (which is a set of electrodes that collect the impulses from the
stimulator and stimulate groups of auditory neurons). Coded infor-
mation from the sound processor is delivered across the skin via
electromagnetic induction to the implanted receiver/stimulator,
which is surgically placed on a bone behind the ear.

1.2.2 Solutions Official approval for the manufacture of implant devices requires
rigorous testing for susceptibility to electromagnetic fields, radi-
ated electromagnetic fields, and electrical safety testing (including
susceptibility to electrical discharge). However, such tests are con-
ducted under normal conditions, not in the magnetic fields of an
MR scanner. Some implant designs have been proven to be MR
compatible [2–5], but they are not routinely supplied in clinical
practice. Standard listening devices do not meet MR compatibility
criteria and, for the patient, risks include movement of the device
578 Deborah Ann Hall and Aspasia Eleni Paltoglou

and localized heating of brain tissue, whereas, for the device, the
electronic components may be damaged. Magnetic Resonance
Safety Testing Services (MRSTS) is a highly experienced testing
company that conducts comprehensive evaluations of implants,
devices, objects, and materials in the MR environment (MRSTS,
Los Angeles, CA, www.magneticresonancesafetytesting.com/).
Testing includes approved assessment of magnetic field interac-
tions, heating, induced electrical currents, and artifacts. A database
of the devices and results of implant testing is accessible to the
interested reader (www.mrisafety.com/). However, auditory
devices have generally been tested only at low magnetic fields (up
to 1.5 T) because most clinical MR systems operate at this field
strength. Since research systems typically operate at 3.0 T (for
improved BOLD signal-to-noise ratio, BOLD SNR) it may be nec-
essary for individual research teams to ensure the safety of their
patients. For example, here at the MRC Institute of Hearing
Research, we have recently assessed the risks of movement and
localized tissue heating for two middle ear piston devices [6]. For
the safety reasons discussed in this subsection, listeners who nor-
mally wear hearing aids could be scanned without their aid but, to
compensate, have been presented with sounds amplified to an
audible level. Given that implanted devices cannot be removed
without surgical intervention, clinical imaging research of implan-
tees has generally used other brain imaging methods, namely posi-
tron emission tomography [7].

1.3 Intense MR The scanning sequence used to measure the BOLD fMRI signal
Scanner Noise and Its requires rapid on and off switching of electrical currents through
Effects on Hearing the three gradient coils of wire in order to create time-varying mag-
netic fields that are required for selecting and encoding the three-
1.3.1 Problems
dimensional image volume (in the x, y, and z planes). This rapid
switching in the static magnetic field induces bending and buckling
of the gradient coils during MRI. As a result, the gradient coils act
like a moving coil loudspeaker to produce a compression wave in
the air, which is heard as acoustic noise during the image acquisi-
tion. Scanner noise increases nonlinearly with static magnetic field
strength, such that ramping from 0.5 to 2 T could account for a rise
in sound level of as much as 11-dB SPL [8]. A brain scan is com-
posed of a set of two-dimensional “slices” through the brain.
Gradient switching is required for each slice acquisition and so an
intense scanner “ping” occurs each time a brain slice is collected.
Each ping lasts about 50 ms and so during fMRI, each scan is audi-
ble as a rapid sequence of such “pings” (see inset in Fig. 2 for an
example of the amplitude envelope of the scanner noise).
The dominant components of the noise spectrum are com-
posed of a peak of sound energy at the gradient switching frequency
plus its higher harmonics. Most of the energy lies below 3 kHz.
Secondary acoustic noise can be produced if the vibration of the
 fMRI of the Central Auditory System 579

−18

−30

Relative sound level (dB)

−42

−54

−66

−78
uncancelled
−90

uncancelled

55 110 220 440 880 1760 3520

Frequency (Hz)

Fig. 2 Typical frequency spectrum of the scanner noise generated during blood
oxygen level-dependent (BOLD) fMRI. This example was measured in the bore of
a Philips Intera 3.0 Tesla scanner. The black line (uncancelled) indicates the
acoustic energy of the noise recorded under normal scanning conditions. The
gray line (canceled) indicates the residual acoustic energy at the ear when the
active noise cancellation (ANC) system is operative. The inset (upper right) shows
an example of the amplitude envelope of the scanner noise for a brain scan
consisting of 16 slices corresponding to a sequence of 16 intense “pings”

coils and the core on which they are wound conducts through the
core supports to the rest of the scanner structure. These secondary
noise characteristics depend more on the mechanical resonances of
the coil assemblies than on the type of imaging sequence and they
tend to be the dominant contributor to the bandwidth and the
spectral envelope of the noise. In this example of the frequency
spectrum captured from a BOLD fMRI scanning sequence that was
run on a Philips 3 Tesla Intera (Fig. 2), the spectrum has a peak
component at 600 Hz with several other prominent pseudo-har-
monics at 300, 1080, and 1720 Hz. The sound level measured in
the bore of the scanner is typically 99-dB SPL [98 dB(A) using an
A-weighting], measured using the maximum “fast” root-mean-
square (RMS) time constant (125 ms). Clearly, exposure to such an
intense sound levels without protection is likely to cause a tempo-
rary threshold shift in hearing and tinnitus, and it could be perma-
nently damaging over a prolonged dosage [9].

1.3.2 Solutions The simplest way to treat the intense noise is to use ear protection in
the form of ear defenders and/or ear plugs (shown in Fig. 1b). Foam
ear plugs can compromise the acoustic quality of the experimental
sounds delivered to the subject and so ear defenders are preferable.
580 Deborah Ann Hall and Aspasia Eleni Paltoglou

Fig. 3 Acoustic waveforms of the scanner noise measured with and without a lining of acoustic damping foam
in the bore of the scanner. Our data demonstrate that the foam reduces the sound pressure level (SPL) at the
position of the subject’s head and in scanner room by a significant margin (about 8 dB). The segment of scan-
ner noise that is illustrated here has a duration of approximately 1 s

Typically, transducers are fitted into sound attenuating earmuffs to

reduce the ambient noise level at the subject’s ears. Attenuation of
the external sound by up to 40 dB can be achieved in this manner,
although the level of reduction drops off at the high-frequency end
of the spectrum. Commercial sound delivery systems all incorporate
passive noise attenuation of this sort. An additional method of noise
reduction is to line the bore of the scanner with a sound-energy
absorbing material ([10]; see also www.ihr.mrc.ac.uk/research/
technical/soundsystem/). The results of a set of measurements
directly comparing the sound intensity of the scanner noise with and
without the foam lining are shown in Fig. 3. However, this strategy
does not provide a feasible solution because neither the design of the
scanner bore nor the automated patient table are suited to the per-
manent installation of a foam lining and some types of acoustic foam
can present risks of noxious fumes if they catch fire.
Some manufacturers have attempted to minimize scanner
sound levels by modifying the design of the scanner hardware. For
example, MR scanners manufactured by Toshiba (Toshiba America
Medical systems, Inc., www.medical.toshiba.com/) incorporate
Pianissimo technology—employing a solid foundation for gradient
support, integrating sound dampening material in the gradient
coils and enclosing them in a vacuum to reduce acoustic noise,
even at full gradient power. This technology claims to reduce scan-
ner noise by up to 90 % [11]. Subjects are reported to hear sounds
at the volume of gentle drumming instead of the jackhammer noise
level of other MR systems.
 fMRI of the Central Auditory System 581

Another solution is to run modified pulse sequences that

reduce acoustic noise by slowing down the gradient switching.
This approach is based on the premise that the spectrum of the
acoustic noise is determined by the product of the frequency spec-
trum of the gradient waveforms and the frequency response func-
tion of the gradient system [12]. The frequency response function
is generally substantially reduced at low frequencies (i.e. below
200 Hz) and so the sound level can be reduced by using gradient
pulse sequences whose spectra are band limited to this low-
frequency range using pulse shapes with smooth onset and offset
ramps [13]. A low-noise fast low-angle shot (FLASH) sequence
can be modified to have a long gradient ramp time (6000 μs) and
it generates a peak sound level of 48-dB SPL measured at the posi-
tion of the ear. This type of sequence has been used for mapping
central auditory function [14]. However, the low noise is achieved
at the expense of slower gradient switching, extending the acquisi-
tion time. Low-noise sequences are not suitable for rapid BOLD
imaging in which the fundamental frequency of the gradient wave-
form is greater than 200 Hz.

1.4 The Effect Not only is the intense scanner noise a risk for hearing, but it also
of Scanner Noise masks the perception of the acoustic stimuli presented to the sub-
on Stimulus Audibility ject. The exact specification of the acoustic signal-to-scanner-noise
ratio (acoustic SNR) in fMRI studies using auditory stimuli is a
1.4.1 Problems
potentially complicated matter. Nevertheless, we have sought to
establish the relative difference between the stimulus level and the
scanner noise level at the ear, by measuring these signals using a
reference microphone placed inside the cup of the ear defender
while participants perform a signal detection in noise task.
Detection thresholds for a narrow band noise centered at the peak
frequency of the scanner noise (600 Hz) are elevated when the
target coincides with the scanner noise. We have demonstrated an
average 11-dB shift in the 71 % detection threshold for the 600-Hz
target when we modulate the perceived level of the scanner noise
using active noise cancelation (ANC) methods (see later).
This evidence suggests that even with hearing protection,
whenever the scanner noise coincides with the presented sound
stimulus it produces changes in task performance and probably
also increases the attentional demands of the listening task. The
frequency range of the scanner acoustic noise is crucial for speech
intelligibility, and speech experiments can be particularly compro-
mised by a noisy environment ([15]; for review, see [16]). A recent
study has quantified the effect of acoustic SNR using four listening
tasks: pitch discrimination of complex tones, same/different judg-
ments of minimal-pair nonsense syllables, lexical decision, and
judgement of sentence plausibility [17]. Across these tasks, perfor-
mance was assessed in silence (acoustic SNR = infinity) and in a
background of MR scanner noise at the three acoustic SNR levels
582 Deborah Ann Hall and Aspasia Eleni Paltoglou

Fig. 4 Mean performance in a simulated scanning environment across four acoustic signal-to-noise ratios
[17]. The top panel plots the proportion of correct responses on the individual tasks, while the bottom panel
shows the overall mean performance (SNR signal-to-noise ratio, dB decibels)

(−6, −12, and −18 dB). Performance of normally hearing listeners

significantly decreased as a function of the noise (Fig. 4). Even at
−6 dB acoustic SNR, participants made many more errors than in
quiet listening conditions (p < 0.01). Thus, across a range of audi-
tory tasks that vary in linguistic complexity, listeners are highly
susceptible to the disruptive impact of the intense noise associated
with fMRI scanning.
 fMRI of the Central Auditory System 583

1.4.2 Solutions The aggregate noise dosage can be reduced by acquiring either a
single or at least very few brain slices, but at the expense of only a
partial view of brain activity [18]. For whole brain fMRI, other
strategies are required.
One novel method that has been developed and evaluated at
our Institute combines optical microphone technology with an
active noise controller for significant attenuation of ambient noise
received at the ears [19]. The canceller is based upon a variation of
the single channel feed-forward filtered-x adaptive controller and
uses a digital signal processor to achieve the noise reduction in real
time. The canceler minimizes the noise pressure level at a specific
control point in space that is defined by the position of the error
microphone, positioned underneath the circum-aural ear defender
of the headset (see Fig. 1d). In 2001, we published a psychophysical
assessment of the system using a prototype system built in the labo-
ratory that utilized a loudspeaker as the noise generator [19]. This
system produced 10–20 dB of subjective noise reduction between
250 Hz and 1 kHz and smaller amounts at higher frequencies.
More recently, we have obtained psychophysical threshold data in a
Philips 3 Tesla scanner confirming that the same level of cancella-
tion is achieved in the real scanner environment (Fig. 5; [20]).
Again, the subjective impression of the scanner noise is the volume
of gentle drumming when the sound system is operating in its can-
celed mode. Thus, it is possible to achieve a high level of noise
attenuation by combining both passive and active methods.
A much more common strategy for reducing the masking influ-
ence of the concomitant scanner noise combines a passive method of
ear protection with an experimental protocol that carefully controls

Detection thresholds for the 600 Hz signal

71% detection threshold (dB)

35

30

25
uncancelled
20
cancelled
15

10

5
1 2 3 4 5
Participant

Fig. 5 Performance on a signal detection in noise task measured in a real scan-

ning environment [Philips 3.0 Tesla MR scanner during blood oxygen level-
dependent (BOLD) fMRI]. The data show that when the noise canceller was
operative, the sound level of the signal could be 8–16 dB softer (depending upon
the listener) in order to achieve the same detection performance
584 Deborah Ann Hall and Aspasia Eleni Paltoglou

the timing between stimulus presentation and image acquisition so

that sound stimuli can be delivered during brief periods of quiet in
between successive brain scans [21]. Specific details of several pulse
sequence protocols that reduce the masking effects of scanner noise
are discussed in more detail in the next subsection.

1.5 The Effect of To increase the BOLD SNR, it is necessary to acquire a large num-
Scanner Noise on ber of scans in each condition in an fMRI experiment. Typically, an
Sound-Related experimenter would collect many hundreds of brain scans in a sin-
Activation in the Brain gle session, with the time in between each scan chosen to be as
short as the scanner hardware and software will permit. Remember
1.5.1 Problems
that, for fMRI, an intense “ping” is generated for each slice of the
scan and so of course this means that the participant can easily be
subjected to several thousand repeated “pings” of noise during the
experiment. Not only does this scanner noise acoustically mask the
presented sound stimuli, but the elevated baseline of sound-evoked
activation due to the ambient scanner noise also makes the experi-
mentally induced auditory activation more difficult to detect statis-
tically. Much of the work examining the influence of acoustic
scanner noise has been directed toward its capacity to interfere
with the study of audition or speech perception by producing acti-
vation of various brain regions, especially the auditory cortex [22–
25]. Several studies highlight the reduced activation signal (i.e. the
difference between stimulation and baseline conditions) in the
auditory cortex when the amount of prior scanner noise is increased,
demonstrating that the scanner noise effectively masks the detec-
tion of auditory activation [22, 26, 27]. In another example, taken
from one of the early fMRI experiments conducted at the MRC
Institute of Hearing Research, we used a specially tailored scanning
protocol to measure the amplitude and the time course of the
BOLD response to a high-quality recording of a single burst of
scanner noise presented to participants over headphones [24]. Our
results revealed a reliable transient increase in the BOLD signal
across a large part of the auditory cortex. As in many other brain
regions, the evoked response to this single brief stimulus event was
smoothed and delayed in time. It rose to a peak by 4–5 s after
stimulus onset and decayed by 5–8 s after stimulus offset [24]. Its
amplitude reached about 1.5 % of the overall signal change, which
is considerable considering that stimulus-related activation usually
accounts for a BOLD signal change of approximately 2–5 %.
Figure 6 illustrates the canonical BOLD response to a noise onset.
In many fMRI experimental paradigms, regions of stimulus-
evoked activation are detected by comparing the BOLD scans
acquired during one sound condition with the BOLD scans acquired
during another condition, which could be either a condition in
which a different type of sound was presented or no sound (known
as a baseline “silent” condition) was presented. Activation is defined
as those parts of the brain that demonstrate a statistically significant
 fMRI of the Central Auditory System 585

3.5

BOLD response in right auditory cortex

3
L R
2.5

1.5

0.5

–0.5

–1

–1.5
0 5 10 15 20 25 30
Peristimulus time (seconds)

Fig. 6 Transient blood oxygen level-dependent (BOLD) response to a noise onset.

The graph shows the fitted response and the 90 % confidence interval. This
example illustrates all the characteristic features of the transient BOLD
response—a peak at 4-s post-stimulus onset followed by an undershoot and
then return to baseline at 16 s

difference between the two conditions. For example, let us consider

the simplest case in which one condition contains a sound and the
other does not. Since the scanner noise is present throughout, the
sound condition effectively contains both stimulus and scanner
noise, while the baseline condition also contains the scanner noise
(i.e. it is not silent). Given the spectrotemporal characteristics of the
scanner noise, it generates widespread sound-related activity across
the auditory cortex. Thus, the subtraction analysis for detecting acti-
vation is sensitive only to whatever is the small additional contribu-
tion of the sound stimulus to auditory neural activity.

1.5.2 Solutions A number of different scanning protocols have been used to mini-
mize the effect of the scanner acoustic noise on the measured pat-
terns of auditory cortical activation. In this section, we will describe
two of these, but before we do, we need to consider some impor-
tant details about the time course of the BOLD response to the
scanner noise and introduce some new terms.
During an fMRI experiment, the BOLD response to the scan-
ner noise spans two different temporal scales. First, the “ping”
generated by the acquisition of one slice early in the scan may
induce a BOLD response in a slice, which is acquired later in the
same scan if that later scan is positioned over the auditory cortex.
We shall call this inter-slice interference. Inter-slice interference is
maximally reduced when all slices in the scan are acquired in rapid
586 Deborah Ann Hall and Aspasia Eleni Paltoglou

a Sparse sampling
true scans
EPI readout

inter-scan interval = 10 s

b Interleaved silent steady state sampling

dummy scans true scans

silent slice-selective RF excitation EPI readout

inter-scan interval = 2.5 s

Fig. 7 Two scanning protocols that have been used to minimize the effect of the
scanner acoustic noise on the measured patterns of auditory cortical activation.
See text for further explanation (s seconds, EPI echo-planar imaging, RF
radiofrequency)

succession and the total duration of the scan is not more than 2 s
[26]. A common term for the scanning protocol that uses a mini-
mum inter-slice interval is a clustered-acquisition sequence.
Edmister et al. [28] found that the clustered-acquisition sequence
provides an advantageous auditory BOLD SNR compared with a
conventional scanning protocol. The second form of interference
is called inter-scan interference. This occurs when the scanner noise
evokes an auditory BOLD response that extends across time to
subsequent scans, predominantly when the interval between scans
is as short as the MR system will permit. Reducing the inter-scan
interference can easily be achieved by extending the period between
scans (the inter-scan interval). By separately manipulating the tim-
ing between slices and between scans, we can reduce the inter-slice
and inter-scan interference independently of one another. When
the clustered-acquisition sequence is combined with a long (e.g.
10 s) inter-scan interval, the activation associated with the experi-
mental sound can be separated from the activation associated with
the scanner sound (Fig. 7a). Furthermore, because the scanner
sound is temporally offset, it does not produce acoustical masking
and does not distract the listener. This scanning protocol is com-
monly known as sparse sampling [21]. Sparse sampling is often the
scanning protocol of choice for identifying auditory cortical evoked
responses in the absence of scanner noise (see e.g. [29–33]).
However, it requires a scanning session that is longer than that of
conventional “continuous” protocols in order to acquire the same
amount of imaging data, and participants can be intolerant of long
 fMRI of the Central Auditory System 587

sessions. It also relies upon certain assumptions about the time to

peak of the BOLD response after stimulus onset and a sustained
plateau of evoked activity for the duration of the stimulus.
A second type of scanning protocol acquires a rapid set of scans
following each silent period in order to avoid some of the afore-
mentioned difficulties—“interleaved silent steady state” sampling
[34]. The increased number of scans permits a greater proportion
of scanning time to be used for data acquisition and at least partial
mapping of the time course of the BOLD response (Fig. 7b).
However, some pulse programming is required to avoid T1-related
signal decay during the data acquisition, hence ensuring that signal
contrast is constant across successive scans. The software modifica-
tion maintains the longitudinal magnetization in a steady state
throughout the scanning session by applying a train of slice-selective
excitation pulses (quiet dummy scans) during each silent period.

1.6 The Effect of The acoustic environment is typically composed of one or more
Stimulus Context: sound sources that change over time. Over the years, both psycho-
Neural Adaptation to physical and electrophysiological studies have amply demonstrated
Sounds that stimulus context strongly influences the perception and neural
coding of individual sounds, especially in the context of stream seg-
1.6.1 Problems regation and grouping [35–37]. A simple example of the influence
of stimulus context is forward masking, which occurs when the
presence of one sound increases the detection threshold for the
subsequent sound. The perceptual effects of forward masking are
strongest when the spectral content of the first sound is similar to
the second sound, when there is no delay between the two sounds,
and when the masker duration is long [38]. Forward inhibition
typically lasts from 70 to 200 ms. This type of suppression has not
only been demonstrated in anesthetized preparations, but also in
awake primates. In the latter case, suppression was seen to extend
up to 1 s in time [39]. As well as tone–tone interactions, neural fir-
ing rate is sensitive to stimulus duration. Neurons respond strongly
to the onset of a sound and their response decays thereafter. Many
illustrative examples can be found in the literature, especially in
cases where longer duration sounds are presented (e.g. 750–
1500 ms in the case of Bartlett and Wang [38], see their Fig. 4).
By transporting these well-established paradigms into a neuro-
imaging experiment, researchers are beginning to address the con-
text dependency of neural coding in humans. One way in which
the effect of sound context on the auditory BOLD fMRI signal has
been examined is in terms of different repetition rates [19, 40].
This is conceptually analogous to the presentation rate manipula-
tions of the forward masking studies described earlier, but goes
beyond the simple case of two-tone interactions. In the fMRI stud-
ies, stimuli were long trains of noise bursts presented at different
rates. The slowest rate was 2 Hz and the fastest rate was 35 Hz,
with intermediate rates being 10 and 20 Hz. Noise bursts at each
588 Deborah Ann Hall and Aspasia Eleni Paltoglou

repetition rate were presented in prolonged blocks of 30 s, each

followed by a 30-s “silent” period. During sound presentation,
scans were acquired at a short inter-scan interval (approximately
2 s) so that the experimenters could reconstruct the 30-s time
course of the BOLD response to each of the different repetition
rates, hence determining the multi-second time pattern of neural
activity. The scans were positioned so that a number of different
auditory sites in the ascending auditory system could be measured:
(1) the inferior colliculus in the midbrain, (2) the medial genicu-
late nucleus in the thalamus, and (3) Heschl’s gyrus and the supe-
rior temporal gyrus in the cortex. The plots of the BOLD time
course demonstrated a systematic change in its shape from mid-
brain up to cortex. In the inferior colliculus, the amplitude of the
BOLD response increased as a function of repetition rate while its
shape was sustained throughout the 30-s stimulus period. In the
medial geniculate body, increasing rate also produced an increase
in BOLD amplitude with a moderate peak in the BOLD shape just
after stimulus onset. Repetition rate exerted its largest effect in the
auditory cortex. The most striking change was in the shape of the
BOLD response. The low repetition rate (2 Hz) elicited a sus-
tained response, whereas the high rate (35 Hz) elicited a phasic
response with prominent peaks just after stimulus onset and offset.
The follow-up study [40] confirmed that it was the temporal enve-
lope characteristics of the acoustic stimulus, not its sound level or
bandwidth, that strongly influenced the shape of the BOLD
response. The authors offer a perceptual interpretation of the neu-
ral response to different repetition rates. The shift in the shape of
the cortical BOLD response from sustained to phasic corresponds
to a shift from a stimulus in which component noise bursts are
perceptually distinct to one in which successive noise bursts fuse to
become individually indistinguishable. The onset and offset
responses of the phasic response coincide with the onset and offset
of a distinct, meaningful event. The logical conclusion to this argu-
ment is that the succession of individual perceptual events in the
low repetition rate conditions defines the sustained BOLD response
observed at the 2-Hz rate. It is clear from these results that while
the amplitude of the BOLD response to sound can inform us about
the tuning properties of the underlying neural population (e.g.
sensitivity to repetition rate), other properties of the BOLD
response, such as its shape, provide different information about
neural coding (e.g. segmentation of the auditory environment into
perceptual events).
It is crucial that these contextual influences on the BOLD sig-
nal are accounted for in the design and/or interpretation of audi-
tory fMRI experiments. To illustrate this case in point, I use a set
of our own experimental data [41]. In this experiment, one of the
sound conditions was a diotic noise (identical signal at the two
ears) presented continuously for 32 s at a constant sound level
 fMRI of the Central Auditory System 589

(∼86-dB SPL) and at a fixed location in the azimuthal plane. Scans

were acquired every 4 s throughout the stimulus period. When the
scans acquired during this sound condition were combined
together and contrasted against the scans acquired during the
“silent” baseline condition, no overall significant activation was
obtained (p > 0.001). We interpret this lack of activation as evi-
dence that the auditory response had rapidly habituated to a static
signal. This conclusion is confirmed by plotting out the time course
of the response at one location within the auditory cortex. The
initial transient rise in the BOLD response at the onset of the
sound begins to decay at about 4 s and this reduction continues
across the stimulus epoch. The end of the epoch is characterized by
a further rise in the BOLD response, elicited by the other types of
sound stimuli that were presented in the experiment (Fig. 8a).

1.6.2 Solutions It is common for auditory fMRI experiments to use a blocked

design in which a sound condition is presented over a pro-
longed time period that extends over many seconds, even tens
of seconds. Indeed as we described in Sect. 1.5, the blocked
design is at the core of the sparse sampling protocol, and so the
risk of neural adaptation is a legitimate one. The BOLD signal
detection problem caused by neural adaptation is often circum-
vented by presenting the stimulus of interest as a train of stimu-
lus bursts at a repetition rate that elicits the sustained cortical
response (e.g. 2 Hz). Many of the auditory fMRI experiments
that have been conducted over the years in our research group
have taken this form [30, 31, 42–44]. Alternatively, if the stim-
ulus contains dynamic spectrotemporal changes, then it is not
always necessary to pulse the stimulus on and off. To illustrate
this case in point, I return to a set of our own experimental data
[41]. In this experiment, one of the sound conditions was a
broadband noise convolved with a generic head-related transfer
function to give the perceptual impression of a sound source
that was continuously rotating around the azimuthal plane of
the listener. Although the sound was presented continuously
for 32 s, the filter functions of the pinnae imposed a changing
frequency spectrum and the head shadow effect imposed low-
rate amplitude modulations in the sound envelope presented to
each ear. When the scans acquired during this sound condition
were combined together and contrasted against the scans
acquired during the “fixed sound source” condition, wide-
spread activation was obtained (p < 0.001) across the posterior
auditory cortex (planum temporale): an area traditionally
linked with spatial acoustic analysis. The time course of activa-
tion demonstrates a sustained BOLD response across the entire
duration of the epoch (Fig. 8b). The sustained response con-
trasts with the transient response observed for the fixed sound
source condition (Fig. 8a).
590 Deborah Ann Hall and Aspasia Eleni Paltoglou

Sound from a fixed source

0.4

0.3

0.2
BOLD response in right posterior auditory cortex
0.1

0.0

−0.1

−0.2

−0.3

−0.4

Sound from a rotating

0.4 source
0.3

0.2

0.1

0.0

−0.1

−0.2

−0.3

−0.4

0 8 16 24 32 40 48
Peristimulus time (secs)
Fig. 8 Adjusted blood oxygen level-dependent (BOLD) response (measured in arbitrary units) across the 32-s
stimulus epoch shaded in gray (a) for a sound from a fixed source and (b) for a sound from a rotating source.
Adjusted values are combined for all six participants and the trend line is indicated using a polynomial sixth
order function. The response for both stimulus types is plotted using the same voxel location in the planum
temporale region of the right auditory cortex (coordinates x 63, y −30, z 15 mm). The position of this voxel is
shown in the inserted panel. The activation illustrated in this insert represents the subtraction of the fixed sound
location from the rotating sound conditions (p < 0.001)

2 Examples of Auditory Feature Processing

2.1 The Within the inner ear, an incoming sound is separated into its indi-
Representation of vidual frequency components by the way in which the energy at
Frequency in the different frequencies travels along the cochlear partition [45].
Auditory Cortex High-frequency tones maximally stimulate those nerve fibers near
the base of the cochlea while low-frequency tones are best coded
 fMRI of the Central Auditory System 591

towards the apex. This cochleotopic representation persists

throughout the auditory pathway where it is referred to as a tono-
topic map. Within the mammalian auditory cortex, electrophysio-
logical recordings have revealed many tonotopic maps [46, 47].
Within each map, neurons tuned to the same sound frequency are
colocalized in a strip across the cortical surface, with an orderly
progression of frequency tuning across adjacent strips. Frequency
tuning is sharper in the primary auditory fields than it is in the sur-
rounding nonprimary fields, and so the most complete representa-
tions of the audible frequency range are found in the primary fields.
Primates have at least two tonotopic maps in primary auditory cor-
tex, adjacent to one another and with mirror-reversed frequency
axes. It is possible to demonstrate tonotopy by fMRI as well as by
electrophysiology, even though frequency selectivity deteriorates at
the moderate to high sound intensities required for fMRI sound
presentation. As a recent example, mirror-symmetric frequency
gradients have been confirmed across primary auditory fields using
high-resolution fMRI at 4.7 T in anesthetized macaques and at
7.0 T in awake behaving macaques [48]. This section describes
results from several fMRI experiments that have sought to demon-
strate tonotopy in the human auditory cortex.
fMRI is an ideal tool for exploring the spatial distribution of
the frequency-dependent responses across the human auditory
cortex because it provides good spatial resolution and the analysis
requires few a priori modeling assumptions (see [49] for a review).
In addition, it is possible to detect statistically significant activation
using individual fMRI analysis. This is important when determin-
ing fine-grained spatial organization because averaging data across
different listeners would inevitably blur the subtle distinctions. A
number of recent studies have sought to determine the organiza-
tion of human tonotopy [29, 33, 50–52]. To avoid the problem of
neural adaptation discussed in Sect. 1.6, experimenters chose stim-
uli that would elicit robust auditory cortical activation. For exam-
ple, Talavage et al. [51, 52] presented amplitude-modulated
signals, while Schönwiesner et al. [50] presented sine tones that
were frequency modulated across a narrow bandwidth. Langers
et al. [33] used a signal detection task in which the tone targets at
each frequency were briefly presented (0.5 s). In agreement with
the primate literature, evidence for the presence of tonotopic orga-
nization is at its most apparent within the primary auditory cortex
while frequency preferences in the surrounding nonprimary areas
are more erratic [33]. Thus, we shall consider in more detail the
precise arrangement of tonotopy in the primary region.
In their first study, Talavage et al. [51] contrasted pairs of low
(<66 Hz) and high (>2490 Hz) frequency stimuli of moderate
intensity and sufficient spectral separation to produce spatially
resolvable differences in activation (low > high and high > low)
across the auditory cortical surface. These activation foci were con-
sidered to define the endpoints of a frequency gradient. In total,
592 Deborah Ann Hall and Aspasia Eleni Paltoglou

a b
A

6 L M

1a
FTTS
P

1b HG
2

3
8 4
HS
High-Frequency sensitive area

Low-Frequency sensitive area

c d

Fig. 9 (a) Sagittal view of the brain with the oblique white line denoting the approximate location and orienta-
tion of the schematic view shown in panel (b) along the supratemporal plane. (b) Schematic representation of
the most consistently found high (red) and low (blue) frequency-sensitive areas across the human auditory
cortex reported by Talavage et al. [50, 51]. The primary area is shown in white and the nonprimary areas are
shown by dotted shading. Panels (c) and (d) illustrate the high- (red) and low- (blue) frequency sensitive areas
across the left auditory cortex of one participant (unpublished data). Two planes in the superior-inferior dimen-
sion are shown (z = 5 mm and z = 0 mm above the CA-CP line). A anterior, P posterior, M medial, L lateral, HG
Heschl’s gyrus, HS Heschl’s sulcus, FTTS first transverse temporal sulcus, STP supratemporal plane

Talavage et al. identified eight frequency-sensitive sites across

Heschl’s gyrus (HG, the primary auditory cortex) and the
surrounding superior temporal plane (STP, the nonprimary audi-
tory cortex). Each site was reliably identified across listeners and
the sites were defined by a numerical label [1–8].
Foci 1–4 occurred around the medial two-thirds of HG and
are good candidates for representing frequency coding within
the primary auditory cortex (Fig. 9). Finding several endpoints
does not provide direct confirmation of tonotopy because tono-
topy necessitates a linear gradient of frequency sensitivity.
Nevertheless, Talavage et al. argued that the foci 1–3 were at
least consistent with predictions from primate electrophysiology.
 fMRI of the Central Auditory System 593

The arrangement of the three foci encompassed the primary

auditory cortex, suggested a common low-frequency border,
and had a mirror-image reversed pattern. This interpretation was
criticized by Schönwiesner et al. [50] who stated that it was
wrong to associate these foci with specific tonotopic fields
because pairs of low- and high-frequency foci could not clearly
be attributed to specific frequency axes nor to anatomically
defined fields. Indeed, in their own study, Schönwiesner et al.
[50] did not observe the predicted gradual decrease in fre-
quency-response amplitude at locations away from the best-fre-
quency focus, but instead found a rather complex distribution of
response profiles. Their explanation for this finding was that the
regions of frequency sensitivity reflected not tonotopy, but dis-
tinct cortical areas that each preferred different acoustic features
associated with a limited bandwidth signal.
Increasing the BOLD SNR might be necessary for charac-
terizing some of the more subtle changes in the response away
from best frequency and more recent evidence using more
sophisticated scanning techniques does support the tonotopy
viewpoint. Frequency sensitivity in the primary auditory cortex
was studied using a 7-T ultra-high field MR scanner to improve
the BOLD SNR and to provide reasonably fine-grained (1 mm3)
spatial resolution [29]. Formisano et al. [29] sought to map the
progression of activation as a smooth function of tone frequency
across HG. Frequency sensitivity was mapped by computing the
locations of the best response to single frequency tones pre-
sented at a range of frequencies (0.3, 0.5, 0.8, 1, 2, and 3 kHz).
Flattened cortical maps of best frequency revealed two mirror-
symmetric gradients (high-to-low and low-to-high) traveling
along HG from an anterolateral point to the posteromedial
extremity. In general, the amplitude of the BOLD response
decreased as the stimulating tone frequency moved away from
the best frequency tuning characteristics of the voxel. A receiver
coil placed close to the scalp over the position of the auditory
cortex is another way to achieve a good BOLD SNR and this
was the method used by Talavage et al. [52]. Talavage et al.
measured best-frequency responses to an acoustic signal that
was slowly modulated in frequency across the range 0.1–8 kHz.
Again, the results confirmed the presence of two mirror-
symmetric maps that crossed HG (extending from the anterior
first transverse temporal sulcus to the posterior Heschl’s sulcus)
and shared a low-frequency border.
Although more evidence will be required before a clear con-
sensus is established, the studies presented in this section have
made influential contributions to the understanding of frequency
representation in the human auditory cortex and its correspon-
dence to primate models of auditory coding.
594 Deborah Ann Hall and Aspasia Eleni Paltoglou

2.2 The Influence of We live in a complex sound environment in which many different
Selective Attention on overlapping auditory sources contribute to the incoming acoustical
Frequency signal. Our brains have a limited processing capacity and so one of
Representations in the most important functions of neural coding is to separate out
Human Auditory these competing sources of information. One way to achieve this is
Cortex by filtering out the uninformative signals (the “ground”) and
attending to the signal of interest (the “figure”). Competition
between incoming signals can be resolved by a bottom-up, stimulus-
driven process (such as a highly salient stimulus that evokes an
involuntary orienting response), or it can be resolved by a top-
down, goal-directed process (such as selective attention). Selective
attention provides a modulatory influence that enables a listener to
focus on the figure and to filter out or attenuate the ground [53].
Visual scientists have shown that attention can be directed to
the features of the figure (feature-based attention, for a review see
[54]) or to the entire figure (object-based attention, for a review see
[55]). Given that so little is known about the mechanisms by which
auditory objects are coded [56], we shall focus on those studies of
auditory feature-based attention. A sound can be defined according
to many different feature dimensions including frequency spec-
trum, temporal envelope, periodicity, spatial location, sound level,
and duration. The experimenter can instruct listeners to attend to
any feature dimension in order to investigate the effect of selective
attention on the neural coding of that feature. Different listening
conditions have been used for comparison with the “attend” condi-
tion. The least controlled of these is a passive listening condition in
which participants are not given any explicit task instructions [30,
57, 58]. Even if there are cases where a task is required, but the
cognitive demand of that task is low, participants are able to divide
their attention across both relevant and irrelevant stimulus dimen-
sions (see [59] for a review on attentional load). Again, this leads to
an uncontrolled experimental situation. For greater control, some
studies have employed a visual distractor task to compete for atten-
tional resources and pull selective attention away from the auditory
modality [60, 61]. However, there is some evidence that the mere
presence of a visual stimulus exerts a significant influence on audi-
tory cortical responses [62, 63] and hence modulation related to
selective attention might interact with that related to the presence
of visual stimuli in a rather complex manner. This can make com-
parison between the results from bimodal studies [60, 61] and uni-
modal auditory studies [32, 64] somewhat problematic.
One paradigm that has been commonly used to examine
feature-based attention manipulates two different feature dimen-
sions independently within the same experimental session and lis-
teners are required to make a discrimination judgement to one
feature or the other. Studies have compared attention to spatial
features such as location, motion, and ear of presentation with
attention to nonspatial features such as pitch and phonemes [60,
 fMRI of the Central Auditory System 595

64]. Results typically demonstrate a response enhancement in

nonprimary auditory regions. For example, Degerman et al. [60]
found auditory enhancement in left posterior nonprimary regions,
but only for attending to location relative to pitch and not the
other way round. Ahveninen et al. [64] used a novel paradigm in
which they measured the effect of attention on neural adaptation.
Their fMRI results showed smaller adaptation effects in the right
posterior nonprimary auditory cortex when attending to location
(relative to phonemes), but again not the other way round. Both
studies reported enhancement for attending to location in addi-
tional nonauditory regions, notably the prefrontal and right pari-
etal areas. This asymmetry in the effects observed across spatial and
nonspatial attended domains is worthy of further exploration since
spatial analysis is well known to engage the right posterior auditory
and right parietal cortex [65].
Another experimental design that has been used to examine
feature-based attention presents concurrent visual and auditory
stimuli and participants are required to make a discrimination judge-
ment to stimuli in one modality or the other. One example of this
design used novel melodies and geometric shapes, and participants
were required to respond to either long note targets or vertical line
targets [57, 58]. When “attending to the shapes” was subtracted
from “attending to the melodies” the results revealed relative
enhancement bilaterally in the lower boundary of the superior tem-
poral gyrus. This finding supports the view that there is sensory
enhancement when attending to the auditory modality. In addition,
it was shown that when “attending to the shapes,” the auditory
response was suppressed relative to a bimodal passive condition.
This is tentative evidence for neural suppression when ignoring the
auditory modality. A novel feature of the experiment by Degerman
et al. [66] was that in one selective attention condition, participants
had to respond to a target defined by a particular combination of
cross-modal features (e.g. high pitch and red circle). The conven-
tional general linear analysis did not show any significant difference
in the magnitude of the auditory response in the cross-modal condi-
tion compared with a condition in which participants simply attended
to the high- and low-pitch targets in the audiovisual stimulus.
However, a region of interest analysis (defining a region in the pos-
terolateral superior temporal gyrus) did suggest some enhancement
for the audiovisual attention condition compared with the auditory
attention condition. Thus, it is possible that nonprimary auditory
regions are involved in attention-dependent binding of synchronous
auditory and visual events into coherent audio–visual objects.
In audition, it has long been established behaviourally that
when participants expect a tone at a specific frequency, their ability
to detect a tone in a noise masker is significantly better when the
tone is at the expected frequency than when it is at an unexpected
frequency (the probe-signal paradigm [67]). The benefit of
596 Deborah Ann Hall and Aspasia Eleni Paltoglou

selective attention for signal detection thresholds can be plotted as

a function of frequency. The ability to detect tones at frequencies
close to the expected frequency is also enhanced, and this benefit
drops off smoothly with the distance away from the expected fre-
quency [67, 68]. The width of this attention-based listening band
is comparable to the width of the critical band related to the
frequency-tuning curve, which can be measured psychophysically
using notched noise maskers [68]. This equivalence suggests that
selective attention might be operating at the level of the sensory
representation of tone frequency.
Evidence from electrophysiological recordings demonstrates
frequency-specific attentional modulation at the level of the primary
auditory cortex, consistent with a neural correlate of the psycho-
physical phenomena found in the probe-signal paradigm. In a series
of experiments, awake behaving ferrets were trained to perform a
number of spectral tasks including tone detection and frequency dis-
crimination [69]. In the tone detection task, ferrets were trained to
identify the presence of a tone against a background of broadband
rippled noise. The spectro-temporal receptive fields measured dur-
ing the noise for frequency-tuned neurons showed strong facilita-
tion around the target frequency that persisted for 30–40 ms. In the
two-tone discrimination task, ferrets performed an oddball task in
which they responded to an infrequent target frequency. Again, the
spectro-temporal receptive fields showed an enhanced and persistent
response for the target frequency, plus a decreased response for the
reference frequency. These opposite effects serve to magnify the
contrast between the two center frequencies, and thus facilitate the
selection of the target. The results of these two tasks confirm that
the acoustic filter properties of auditory cortical neurons can dynam-
ically adapt to the attentional focus of the task.
Recently, we have addressed the question of attentional enhance-
ment for selective attention to frequency using a high-resolution scan-
ning protocol (1.5 mm2 × 2.5 mm) (unpublished data). To control for
the demands on selective attention, we presented two concurrent
streams (low- and high-frequency tones). Participants were requested
to attend to one frequency stream or the other and these attend con-
ditions were presented in an interleaved manner throughout the
experiment. Behavioral testing confirmed that performance signifi-
cantly deteriorated when these sounds were presented in a divided
attention task. To be able to identify high- and low-frequency sensi-
tive areas around the primary auditory cortex we designed two types
of stimuli using different rhythms for each of the two streams. For
example, one stimulus contained a “fast” high-frequency rhythm and
a “slow” low-frequency rhythm so that the stimulus contained a
majority of high-frequency tones. The other stimulus was the con-
verse. Areas of high-frequency sensitivity were identified by subtract-
ing the low-frequency majority stimulus from the high-frequency
majority stimulus, and vice-versa (Fig. 9c, d). For each of the three
 fMRI of the Central Auditory System 597

2.295

(95% confidence intervals) 2.290

Log MR signal

2.285

2.280

2.275
Just listen Attend Attend
BF off BF

Fig. 10 Response to the three listening conditions: just listen, attend to the best frequency (BF) tones, and
attend OFF BF. The data shown are for those stimuli in which BF tones formed the majority (80 %) of the tones
in the sound sequence, combining responses across areas 1–3. The error bars denote the 95 % confidence
intervals

participants, we selected those frequency-specific areas that best

corresponded to areas 1–4 (defined by Talavage et al. [51, 52]; see
Sect. 2.1). Within these areas, we extracted the BOLD signal time
course for every voxel and performed a log transform to standardize
the data. We collapsed the data across low- and high-frequency sensi-
tive areas [1–4] according to their “best frequency” (BF). The best
frequency of an area corresponds to the frequency that evokes the
largest BOLD response. A univariate ANOVA showed response
enhancement when participants were attending to the BF of that area,
compared with attending to the other frequency (p < 0.01). In addi-
tion, response enhancement was also found when attending to the BF
of that area, compared with passive listening (p < 0.05) (Fig. 10). Note
that for these results area 4 was excluded from the analysis, because it
showed different pattern of attentional modulation. The response
profile of area 4 might differ from that of areas 1–3 in other ways
because it is not consistently present in all listeners [51]. Our finding
of frequency-specific attentional enhancement in primary auditory
regions contrasts with that of Petkov et al. [61], who reported atten-
tion-related modulation to be independent of stimulus frequency and
to engage mainly the nonprimary auditory areas. However, our result
is more in keeping with the predictions made by the neurophysiologi-
cal data reported by Fritz et al. [69].
598 Deborah Ann Hall and Aspasia Eleni Paltoglou

3 Future Directions

It is increasingly likely that auditory cortical regions compute aspects

of the sound signal that are more complicated in their nature than
the simple physical acoustic attributes of the sound. Thus, the
encoded features of the sound reflect an increasingly abstract repre-
sentation of the sound stimulus. We have already presented some
evidence for this in terms of the way in which the auditory cortical
response is exquisitely sensitive to the temporal context of the sound,
particularly the way in which the time course of the BOLD response
represents the temporal envelope characteristics of the sound,
including sound onsets and offsets [18, 40], (see Sect. 1.4). However,
there are many other ways in which neural coding reflects higher
level processing. In this final section, we shall introduce two impor-
tant aspects of the listening context that determine the auditory
BOLD response: the perceptual experience of the listener and the
operational aspects of the task. A number of fMRI studies have dem-
onstrated ways in which activity within the human auditory cortex is
modulated by auditory sensations, including loudness, pitch, and
spatial width. Other studies have revealed that task relatedness is also
a significant determining factor for the pattern of activation. These
findings highlight how future auditory fMRI studies could usefully
investigate these contributory factors in order to provide a more
complete picture of the neural basis of the listening process.

3.1 Cortical One approach used in auditory fMRI to investigate perceptually

Activation Reflects relevant coding imposes systematic changes to the listener’s per-
Perceptually Relevant ception of a sound signal by parametrically manipulating certain
Coding acoustic parameters and subsequently correlating the perceptual
change with the variation in the pattern of activation. For example,
by increasing sound intensity (measured in SPL), one also increases
its perceived loudness (measured in phons). Loudness is a percep-
tual phenomenon that is a function of the auditory excitation pat-
tern induced by the sound, integrated across frequency. Sound
intensity and loudness are measures of different phenomena. For
example, if the bandwidth of a broadband signal is increased while
its intensity is held constant, then loudness nevertheless increases
because the signal spans a greater number of frequency channels.
In an early fMRI study, Hall et al. [31] presented single-frequency
tones and harmonic-complex tones that were matched either in
intensity or in loudness. The results showed that the complex tones
produced greater activation than did the single-frequency tones,
irrespective of the matching scheme. This result indicates that
bandwidth had a greater effect on the pattern of auditory activa-
tion than sound level. Nevertheless, when the data were collapsed
across stimulus class, the amount of activation was significantly
correlated with the loudness scale, not with the intensity scale.
 fMRI of the Central Auditory System 599

In people with elevated hearing thresholds, the perception of

sound level is distorted. They typically experience the same dynamic
range of loudness as normally hearing listeners despite having a com-
pressed range of sensitivity to sound level. The BOLD response to
sound level is reflected in a disproportionate increase in loudness
with intensity. A recent study has characterized the BOLD response
to frequency-modulated tones presented at a broad range of intensi-
ties (0–70 dB above the normal hearing threshold) [33]. Both nor-
mally hearing and hearing impaired groups showed the same
steepness in the linear increase in auditory activation as a function of
loudness, but not of intensity (Fig. 11). The results from this study
clearly demonstrate that the BOLD response can be interpreted as a
correlate of the subjective strength of the stimulus percept.
Pitch can be defined as the sensation whose variation is associ-
ated with musical melodies. Together with loudness, timbre, and
spatial location, pitch is one of the primary auditory sensations.
The salience of a pitch is determined by several physical properties
of the pitch signal, one being the numbered harmonic components
comprising a harmonic-complex tone. The cochlea separates out
the frequency components of sounds to a limited extent, so that
the first eight harmonics of a harmonic-complex tone excite dis-
tinct places in the cochlea and are said to be “resolved,” whereas
the higher harmonics are not separated and are said to be “unre-
solved.” Pitch discrimination thresholds for unresolved harmonics
are substantially higher than those for resolved harmonics, consis-
tent with the former type of stimulus evoking a less salient pitch

3 3
Intensity above normal Equivalent loudness
hearing threshold percept
(% signal change)

2 2
BOLD activation

1 1

0 0
Normal hearing
Hearing impaired
−1 −1
0 20 40 60 80 0 20 40 60 80
intensity (dB) loudness (dB)

Fig. 11 Growth in the blood oxygen level-dependent (BOLD) response (measured

in % signal change) for high (4–8 kHz) frequency-modulated tone presented
across a range of sound levels for ten normally hearing participants (black lines)
and ten participants with a high-frequency hearing loss (gray lines). The left hand
panel shows the growth as a function of sound intensity, while the left hand panel
plots the same data as a function of the equivalent loudness percept. The upper
and lower lines denote the 95 % confidence interval of the quadratic polynomial
fit to the data. This graph summarizes data presented by Langers et al [33]
600 Deborah Ann Hall and Aspasia Eleni Paltoglou

[70]. A pairwise comparison between the activation patterns for

resolved (strong pitch) and unresolved (weak pitch) harmonic-
complex tones has identified differential activation in a small, spa-
tially localized region of nonprimary auditory cortex, overlapping
the anterolateral end of Heschl’s gyrus [71]. The authors claim
that this finding reflects the cortical representation for pitch
salience. Another way to determine the salience of a pitch is by the
degree of fine temporal regularity in the stimulus (i.e. the monau-
ral repeating pattern within frequency channels). This is true even
for signals in which there are no distinct frequency peaks in the
cochlear excitation pattern from which to calculate the pitch. A
range of pitch saliencies can be created by parametrically varying
the degree of temporal regularity in an iterated-ripple noise stimu-
lus (using 0, 1, 2, 4, 8, and 16 add-and-delay iterations during
stimulus generation) [72]. Again the anterolateral end of Heschl’s
gyrus appeared highly responsive to the change in pitch salience, in
a linear manner.
Spatial location is another important auditory sensation that is
determined by the fine temporal structure in the signal, this time it
being the binaural temporal characteristics across the two ears. The
interaural correlation (IAC) of a sound represents the similarity
between the signals at the left and right ears. Changes in the IAC
of a wideband signal result in changes in sound’s perceived “width”
when presented through headphones. A noise with an IAC of 1.0
is typically perceived as sound with a compact source located at the
center of the head. As the IAC is reduced the source broadens. For
an IAC of 0.0, it eventually splits into two separate sources, one at
each ear [73]. Again the parametric approach has been employed
to measure activation across a range of IAC values (1.00, 0.93,
0.80, 0.60, 0.33, and 0.00) [74]. The authors found a significant
positive relationship between BOLD activity and IAC, which was
confined to the anterolateral end of Heschl’s gyrus, the region that
is also responsive to pitch salience. The slope of the function was
not precisely linear but the BOLD response was more sensitive to
changes in IAC at values near to unity than at values near zero.
This response pattern is qualitatively compatible with previous
behavioral measures of sensitivity to IAC [75].
There is some evidence to support the claim that the neural
representations of auditory sensations (including loudness, pitch,
and spatial width) evolve as one ascends the auditory pathway.
Budd et al. [74] examined sensitivity to values of IAC associated
with spatial width within the inferior colliculus, the medial genicu-
late nucleus, as well as across different auditory cortical regions,
but the effects were significant only within the nonprimary audi-
tory cortex. Griffiths et al. [72, 76] also examined sensitivity to the
increases in temporal regularity associated with pitch salience
within the cochlear nucleus, inferior colliculus, medial geniculate
nucleus, as well as across different auditory cortical regions. Some
 fMRI of the Central Auditory System 601

degree of sensitivity to pitch salience was found at all sites, but the
preference appeared greater in the higher centers than in the
cochlear nucleus [76]. Thus, the evidence supports the notion of
an increasing responsiveness to percept attributes of sound
throughout the ascending auditory system, culminating in the
nonprimary auditory cortex. These findings are consistent with the
hierarchical processing of sound attributes.
Encoding the perceptual properties of a sound is integral to
identifying the object properties of that sound source. The nonpri-
mary auditory cortex probably plays a key role in this process
because it has widespread cortical projections to frontal and parietal
brain regions and is therefore ideally suited to access distinct higher
level cortical mechanisms for sound identification and localization.
Recent trends in auditory neuroscience are increasingly concerned
with auditory coding beyond the conventional limits of the audi-
tory cortex (the superior temporal gyrus in humans), particularly
with respect to the hierarchical organization of sensory coding via
dorsal and ventral auditory processing routes. At the top of this
hierarchy stands the brain’s representation of an auditory “object.”
The concept of an auditory object still remains controversial [56].
Although it is clear that the brain needs to code information about
the invariant properties of a sound source, research in this field is
considerably underdeveloped. Future directions are likely to begin
to address critical issues such as the definition of an auditory object,
whether the concept is informative for auditory perception, and
optimal paradigms for studying object coding.

3.2 Cortical Listeners interact with complex auditory environments that, at any
Activation Also one time point, contain multiple auditory objects located at
Reflects Behaviourally dynamically varying spatial locations. One of the primary chal-
Relevant Coding lenges for the auditory system is to analyze this external environ-
ment in order to inform goal-directed behavior. In Sect. 2.2 we
introduced some of the neurophysiological evidence for the impor-
tance of the attentional focus of the task in determining the pattern
of auditory cortical activity [69]. Here, we consider the contribu-
tion of human auditory fMRI research to this question. In
particular, we present the interesting findings of one group who
have started to address how the auditory cortex responds to the
context and the procedural and cognitive demands of the listening
task (see [77] for a review).
In that review, Scheich and colleagues report a series of research
studies in which they suggest that the function of different audi-
tory cortical areas is not determined so much by stimulus features
(such as timbre, pitch, motion, etc.), but rather by the task that is
performed. For example, one study reported the results of two
fMRI experiments in which the same frequency-modulated stimuli
were presented under different task conditions [78]. Top-down
influences strongly affected the strength of the auditory response.
602 Deborah Ann Hall and Aspasia Eleni Paltoglou

When a pitch-direction categorization task was compared with pas-

sive listening, a greater response was found in right posterior
nonprimary auditory areas (planum temporale). Moreover, hemi-
spheric differences were also found when comparing the response
to two different tasks. The right nonprimary areas responded more
strongly when the task required a judgement about pitch direction
(rising or falling), whereas the left nonprimary areas responded
more strongly when the task required a judgement about the
sound duration. It is not the case that the right posterior nonpri-
mary areas were only engaged by sound categorization because this
region was more responsive to the critical sound feature (frequency
modulation) during passive listening than were surrounding audi-
tory areas (see also ref. [32]). These results broadly indicate an
interaction between the stimulus and the task, which influences the
pattern of auditory cortical activity. The precise characteristics of
this interaction are worthy of future studies.

References

1. Palmer AR, Bullock DC, Chambers JD (1998) 10. Ravicz ME, Melcher JR (2001) Isolating the
A high-output, high-quality sound system for auditory system from acoustic noise during func-
use in auditory fMRI. Neuroimage 7:S357 tional magnetic resonance imaging: examination
2. Chou CK, McDougall JA, Chan KW (1995) of noise conduction through the ear canal, head,
Absence of radiofrequency heating from audi- and body. J Acoust Soc Am 109(1):216–231
tory implants during magnetic-resonance 11. Price DL, De Wilde JP, Papadaki AM, Curran
imaging. Bioelectromagnetics 16(5):307–316 JS, Kitney RI (2001) Investigation of acoustic
3. Heller JW, Brackmann DE, Tucci DL, noise on 15 MRI scanners from 0.2 T to 3 T. J
Nyenhuis JA, Chou CK (1996) Evaluation Magn Reson Imaging 13(2):288–293
of MRI compatibility of the modified nucleus 12. Hedeen RA, Edelstein WA (1997) Characterization
multichannel auditory brainstem and cochlear and prediction of gradient acoustic noise in MR
implants. Am J Otol 17(5):724–729 imagers. Magn Reson Med 37(1):7–10
4. Shellock FG, Morisoli S, Kanal E (1993) MR 13. Hennel F, Girard F, Loenneker T (1999)
procedures and biomedical implants, mate- “Silent” MRI with soft gradient pulses. Magn
rials, and devices—1993 update. Radiology Reson Med 42:6–10
189(2):587–599 14. Brechmann A, Baumgart F, Scheich H (2002)
5. Weber BP, Neuburger J, Battmer RD, Lenarz Sound-level-dependent representation of fre-
T (1997) Magnetless cochlear implant: rel- quency modulations in human auditory cortex: a
evance of adult experience for children. Am low-noise fMRI study. J Neurophysiol 87:423–433
J Otol 18(6):S50–S51 15. Sumby WH, Pollack I (1954) Visual contribu-
6. Wild DC, Head K, Hall DA (2006) Safe tion to speech intelligibility in noise. J Acoust
magnetic resonance scanning of patients with Soc Am 26(2):212–215
metallic middle ear implants. Clin Otolaryngol 16. Assmann P, Summerfield Q (2004) Perception
31(6):508–510 of speech under adverse conditions. In:
7. Giraud AL, Truy E, Frackowiak R (2001) Greenberg S, Ainsworth WA, Popper AN, Fay
Imaging plasticity in cochlear implant patients. RR (eds) Speech processing in the auditory sys-
Audiol Neurootol 6(6):381–393 tem. Springer, New York, pp 231–308
8. Moelker A, Wielopolski PA, Pattynama PM 17. Healy EW, Moser DC, Morrow-Odom KL,
(2003) Relationship between magnetic field Hall DA, Fridriksson J (2007) Speech percep-
strength and magnetic-resonance-related tion in MRI scanner noise by persons with
acoustic noise levels. MAGMA 16:52–55 aphasia. J Speech Lang Hear Res 50:323–334
9. Foster JR, Hall DA, Summerfield AQ, Palmer 18. Harms MP, Melcher JR (2002) Sound repetition
AR, Bowtell RW (2000) Sound-level measure- rate in the human auditory pathway: representa-
ments and calculations of safe noise dosage during tions in the waveshape and amplitude of fMRI
fMRI at 3T. J Magn Reson Imaging 12:157–163 activation. J Neurophysiol 88:1433–1450
 fMRI of the Central Auditory System 603

19. Chambers JD, Akeroyd MA, Summerfield absence of background scanner noise. J Acoust
AQ, Palmer AR (2001) Active control of the Soc Am 109(4):1559–1570
volume acquisition noise in functional mag- 32. Hart HC, Palmer AR, Hall DA (2004) Different
netic resonance imaging: method and psy- areas of human non-primary auditory cortex are
choacoustical evaluation. J Acoust Soc Am activated by sounds with spatial and nonspatial
110(6):3041–3054 properties. Hum Brain Mapp 21:178–190
20. Hall DA, Chambers J, Foster J, Akeroyd MA, 33. Langers DRM, Backes WH, Van Dijk P (2007)
Coxon R, Palmer AR (2009) Acoustic, psycho- Representation of lateralization and tonotopy
physical, and neuroimaging measurements of in primary versus secondary human auditory
the effectiveness of active cancellation during cortex. Neuroimage 34:264–273
auditory functional magnetic resonance imag- 34. Schwarzbauer C, Davis MH, Rodd JM,
ing. J Acoust Soc Am 125(1):347–359 Johnsrude I (2006) Interleaved silent steady
21. Hall DA, Haggard MP, Akeroyd MA, Palmer state (ISSS) imaging: a new sparse imaging
AR, Summerfield AQ, Elliott MR, Gurney EM, method applied to auditory fMRI. Neuroimage
Bowtell RW (1999) ‘Sparse’ temporal sampling 29(3):774–782
in auditory fMRI. Hum Brain Mapp 7:213–223 35. Bregman AS (1990) Auditory scene analysis:
22. Bandettini PA, Jesmanowicz A, Van Kylen J, the perceptual organisation of sound. MIT,
Birn RM, Hyde JS (1998) Functional MRI of Cambridge, MA
brain activation induced by scanner acoustic 36. Fishman YI, Arezzo JC, Steinschneider M
noise. Magn Reson Med 39:410–416 (2004) Auditory stream segregation in mon-
23. Bilecen D, Scheffler K, Schmid N, Tschopp K, key auditory cortex: effects of frequency sepa-
Seelig J (1998) Tonotopic organization of the ration, presentation rate, and tone duration.
human auditory cortex as detected by BOLD- J Acoust Soc Am 116(3):1656–1670
FMRI. Hear Res 126:19–27 37. Fishman YI, Reser DH, Arezzo JC, Steinschneider
24. Hall DA, Summerfield AQ, Gonçalves MS, M (2001) Neural correlates of auditory stream
Foster JR, Palmer AR, Bowtell RW (2000) segregation in primary auditory cortex of the
Time-course of the auditory BOLD response to awake monkey. Hear Res 151:167–187
scanner noise. Magn Reson Med 43:601–606 38. Brosch M, Schreiner CE (1997) Time course of
25. Shah NJ, Jäncke L, Grosse-Ruyken M-L, forward masking tuning curves in cat primary
Müller-Gärtner HW (1999) Influence of auditory cortex. J Neurophysiol 77:923–943
acoustic masking noise in fMRI of the auditory 39. Bartlett EL, Wang X (2005) Long-lasting
cortex during phonetic discrimination. J Magn modulation by stimulus context in primate
Reson Imaging 9(1):19–25 auditory cortex. J Neurophysiol 94:83–104
26. Talavage TM, Edmister WB, Ledden PJ, 40. Harms MP, Guinan JJ, Sigalovsky IS, Melcher
Weisskoff RM (1999) Quantitative assessment JR (2005) Short-term sound temporal enve-
of auditory cortex responses induced by imager lope characteristics determine multisecond time
acoustic noise. Hum Brain Mapp 7(2):79–88 patterns of activity in human auditory cortex as
27. Elliott MR, Bowtell RW, Morris PG (1999) shown by fMRI. J Neurophysiol 93:210–222
The effect of scanner sound in visual, motor, 41. Palmer AR, Hall DA, Sumner C, Barrett DJK,
and auditory functional MRI. Magn Reson Jones S, Nakamoto K, Moore DR (2007)
Med 41(6):1230–1235 Some investigations into non-passive listening.
28. Edmister WB, Talavage TM, Ledden PJ, Hear Res 229:148–157
Weisskoff RM (1999) Improved auditory cor- 42. Hall DA, Edmondson-Jones M, Fridriksson
tex imaging using clustered volume acquisi- J (2006) Periodicity and frequency coding
tions. Hum Brain Mapp 7:89–97 in human auditory cortex. Eur J Neurosci
29. Formisano E, Kim DS, Di Salle F, van de 24:3601–3610
Moortele PF, Ugurbil K, Goebel R (2003) 43. Hall DA, Johnsrude IS, Haggard MP, Palmer
Mirror-symmetric tonotopic maps in human pri- AR, Akeroyd MA, Summerfield AQ (2002)
mary auditory cortex. Neuron 40(4):859–869 Spectral and temporal processing in human
30. Hall DA, Haggard MP, Akeroyd MA, auditory cortex. Cereb Cortex 12:140–149
Summerfield AQ, Palmer AR, Elliott MR, 44. Hart HC, Hall DA, Palmer AR (2003) The
Bowtell RW (2000) Modulation and task sound-level-dependent growth in the extent of
effects in auditory processing measured using fMRI activation in Heschl’s gyrus is different
fMRI. Hum Brain Mapp 10(3):107–119 for low- and high-frequency tones. Hear Res
31. Hall DA, Haggard MP, Summerfield AQ, 179(1–2):104–112
Akeroyd MA, Palmer AR, Bowtell RW (2001) 45. Von Békésy G (1947) The variations of phase
Functional magnetic resonance imaging mea- along the basilar membrane with sinusoidal
surements of sound-level encoding in the vibrations. J Acoust Soc Am 19:452–460
604 Deborah Ann Hall and Aspasia Eleni Paltoglou

46. Kosaki H, Hashikawa T, He J, Jones EG (1997) 62. Kayser C, Petkov CI, Augath M, Logothetis
Tonotopic organization of auditory cortical fields NK (2007) Functional imaging reveals visual
delineated by parvalbumin immunoreactivity in modulation of specific fields in auditory cortex.
macaque monkeys. J Comp Neurol 386:304–316 J Neurosci 27(8):1824–1835
47. Merzenich MM, Brugge JF (1973) 63. Lehmann C, Herdener M, Esposito F, Hubl
Representation of the cochlear partition on the D, di Salle F, Scheffler K, Bach DR, Federspiel
superior temporal plane of the macaque mon- A, Kretz R, Dierks T, Seifritz E (2006)
key. Brain Res 50:275–296 Differential patterns of multisensory interac-
48. Petkov CL, Kayser C, Augath M, Logothetis tions in core and belt areas of human auditory
NK (2006) Functional imaging reveals numer- cortex. Neuroimage 31(1):294–300
ous fields in the monkey auditory cortex. PLoS 64. Ahveninen J, Jaaskelainen IP, Raij T, Bonmassar
Biol 4(7):213–226 G, Devore S, Hamalainen M, Levanen S, Lin
49. Hall DA, Hart HC, Johnsrude IS (2003) F-H, Sams M, Shinn-Cunningham BG, Witzel
Relationships between human auditory corti- T, Belliveau JW (2006) Task-modulated “what”
cal structure and function. Audiol Neurootol and “where” pathways in human auditory cortex.
8(1):1–18 Proc Natl Acad Sci U S A 103(39):14608–14613
50. Schönwiesner M, Von Cramon DY, Rubsamen 65. Lewald J, Meister IG, Weidemann J, Topper R
R (2002) Is it tonotopy after all? Neuroimage (2004) Involvement of the superior temporal
17:1144–1161 cortex and the occipital cortex in spatial hearing:
51. Talavage TM, Ledden PJ, Benson RR, Rosen evidence from repetitive transcranial magnetic
BR, Melcher JR (2000) Frequency-dependent stimulation. J Cogn Neurosci 16(5):828–838
responses exhibited by multiple regions in 66. Degerman A, Rinne T, Pekkola J, Autti T,
human auditory cortex. Hear Res 150:225–244 Jaaskelainen IP, Sams M, Alho K (2007)
52. Talavage TM, Sereno MI, Melcher JR, Ledden Human brain activity associated with audio-
PJ, Rosen BR, Dale AM (2004) Tonotopic visual perception and attention. Neuroimage
organization in human auditory cortex 34(4):1683–1691
revealed by progressions of frequency sensitiv- 67. Greenberg GZ, Larkin WD (1968) Frequency-
ity. J Neurophysiol 91:1282–1296 response characteristic of auditory observers
53. Kastner S, Ungerleider LG (2000) Mechanisms detecting signals of a single frequency in noise:
of visual attention in the human cortex. Annu the probe-signal method. J Acoust Soc Am
Rev Neurosci 23(1):315–341 44(6):1513–1523
54. Maunsell JHR, Treue S (2006) Feature-based 68. Schlauch RS, Hafter ER (1991) Listening
attention in visual cortex. Trends Neurosci bandwidths and frequency uncertainty in
29(6):317–322 pure-tone signal detection. J Acoust Soc Am
90(3):1332–1339
55. Scholl BJ (2001) Objects and attention: the
state of the art. Cognition 80(1–2):1–46 69. Fritz JB, Elhilali M, David SV, Shamma SA
(2007) Does attention play a role in dynamic
56. Griffiths TD, Warren JD (2004) What receptive field adaptation to changing acoustic
is an auditory object? Nat Rev Neurosci salience in A1? Hear Res 229:186–203
5(11):887–892
70. Shackleton TM, Carlyon RP (1994) The role
57. Johnson JA, Zatorre RJ (2005) Attention to of resolved and unresolved harmonics in pitch
simultaneous unrelated auditory and visual perception and frequency modulation discrimi-
events: behavioral and neural correlates. Cereb nation. J Acoust Soc Am 95:3529–3540
Cortex 15(10):1609–1620
71. Penagos H, Melcher JR, Oxenham AJ (2004)
58. Johnson JA, Zatorre RJ (2006) Neural sub- A neural representation of pitch salience in
strates for dividing and focusing attention nonprimary human auditory cortex revealed
between simultaneous auditory and visual with functional magnetic resonance imaging.
events. Neuroimage 31(4):1673–1681 J Neurosci 24(30):6810–6815
59. Lavie N (2005) Distracted and confused? 72. Griffiths TD, Büchel C, Frackowiak RSJ,
Selective attention under load. Trends Cogn Patterson RD (1998) Analysis of temporal
Sci 9(2):75–82 structure in sound by the human brain. Nat
60. Degerman A, Rinne T, Salmi J, Salonen O, Neurosci 1:422–427
Alho K (2006) Selective attention to sound 73. Blauert J, Lindemann W (1986) Spatial map-
location or pitch studied with fMRI. Brain Res ping of intracranial auditory events for various
1077(1):123–134 degrees of interaural coherence. J Acoust Soc
61. Petkov CI, Kang X, Alho K, Bertrand O, Yund Am 79(3):806–813
EW, Woods DL (2004) Attentional modula- 74. Budd TW, Hall DA, Goncalves MS,
tion of human auditory cortex. Nat Neurosci Akeroyd MA, Foster JR, Palmer AR, Head
7(6):658–663 K, Summerfield AQ (2003) Binaural
 fMRI of the Central Auditory System 605

specialisation in human auditory cortex: an human brainstem. Nat Neurosci

fMRI investigation of interaural correla- 4:633–637
tion sensitivity. Neuroimage 77. Scheich H, Brechmann A, Brosch M, Budinger
20(3):1783–1794 E, Ohl FW (2007) The cognitive auditory cor-
75. Culling JF, Colburn HS, Spurchise M (2001) tex: task-specificity of stimulus representations.
Interaural correlation sensitivity. J Acoust Soc Hear Res 229:213–224
Am 110(2):1020–1029 78. Brechmann A, Scheich H (2005) Hemispheric
76. Griffiths TD, Uppenkamp S, Johnsrude I, shifts of sound representation in auditory cor-
Josephs O, Patterson RD (2001) Encoding tex with conceptual listening. Cereb Cortex
of the temporal regularity of sound in the 15(5):578–587
 Part III

fMRI Clinical Application

 Chapter 20

Application of fMRI to Multiple Sclerosis and Other

White Matter Disorders
Massimo Filippi and Maria A. Rocca

Abstract
The variable effectiveness of reparative and recovery mechanisms following tissue damage is among the
factors that might contribute to explain, at least partially, the paucity of the correlation between clinical
and magnetic resonance imaging (MRI) findings in patients with white matter disorders. Among the
mechanisms of recovery, brain plasticity is likely to be one of the most important with several possible dif-
ferent substrates (including increased axonal expression of sodium channels, synaptic changes, increased
recruitment of parallel existing pathways or “latent” connections, and reorganization of distant sites). The
application of fMRI has shown that plastic cortical changes do occur after white matter injury of different
etiology, that such changes are related to the extent of white matter damage, and that they can contribute
in limiting the clinical consequences of brain damage. Conversely, the failure or exhaustion of the adaptive
properties of the cerebral cortex might be among the factors responsible for the accumulation of “fixed”
neurological deficits in patients with white matter disorders.

Key words Multiple sclerosis, Functional magnetic resonance imaging, White matter, Adaptation,
Maladaptation, Myelitis, Vasculitides

1 Introduction

Over the past decade, modern structural magnetic resonance imag-

ing (MRI) techniques have been extensively used to study patients
with white matter disorders with the ultimate goal of increasing
the understanding of the mechanisms responsible for the accumu-
lation of irreversible disability [1–3]. Although the application of
these techniques has provided important insight into the pathobi-
ology of many of these disorders, the magnitude of the correlation
between MRI and clinical findings remains suboptimal [1–3]. This
might be explained, at least partially, by the variable effectiveness of
reparative and recovery mechanisms following tissue damage.
Cortical reorganization has been suggested as a potential contribu-
tor to the recovery or to the maintenance of function in the pres-
ence of irreversible white matter damage [4, 5]. Brain plasticity is a

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_20, © Springer Science+Business Media New York 2016

609
610 Massimo Filippi and Maria A. Rocca

well-known feature of the human brain, which is likely to have

several different substrates (including increased axonal expression
of sodium channels, synaptic changes, increased recruitment of
parallel existing pathways or “latent” connections, and reorganiza-
tion of distant sites) [6]. The application of functional MRI (fMRI)
has shown that plastic cortical changes do occur after central ner-
vous system (CNS) white matter injury of different etiology, that
such changes are related to the extent of WM damage, and that
they can contribute in limiting the clinical consequences of wide-
spread disease-related tissue damage [4, 5]. Conversely, the failure
or the exhaustion of the adaptive properties of the cerebral cortex
might be among the factors responsible for the accumulation of
“fixed” neurological deficits in patients affected by white matter
disorders (WMD) [4, 5].
This chapter summarizes the major contributions of fMRI for
the in vivo monitoring of several white matter diseases. Since fMRI
has been mostly applied to improve our understanding of the
pathophysiology of multiple sclerosis (MS), a special focus is
devoted to this condition and allied WMD.

2 fMRI in MS

2.1 General The main problem in the interpretation of fMRI studies in dis-
Considerations eased people is that the observed changes might be biased by
differences in task performance between patients and controls.
Clearly, this is a major issue in MS, which typically causes impair-
ment of various functional systems. Therefore, despite providing
several important pieces of information, the value of the earliest
fMRI studies of patients with MS [7–13] has to be weight against
this background. For this reason, more recent fMRI studies in
MS have been based on larger and more selected patients’ groups
than the seminal studies. These studies have investigated the
brain patterns of cortical activations during the performance of a
number of motor, visual, and cognitive tasks in patients with all
the major clinical phenotypes of the disease. Another appealing
strategy which has been introduced for the study of functional
network rewiring in clinically impaired patients is based on the
assessment of functional abnormalities at rest in the main brain
functional networks (resting state networks). One of the most
solid conclusions that can be drawn from fMRI studies of MS is
that cortical reorganization does occur in patients affected by
this condition. The correlation between various measures of
structural MS damage and the extent of cortical activations also
suggests an adaptive role of such cortical changes in contributing
to clinical recovery and maintaining a normal level of function-
ing in patients with MS, despite the presence of irreversible
axonal/neuronal loss.
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 611

2.2 Visual System The method usually applied to investigate the visual system con-
sists of the application of a 8 Hz photic stimulation to one or both
eyes [8, 12–18]. A study of the visual system [12] in patients who
had recovered from a single episode of acute unilateral optic neu-
ritis demonstrated that these patients, relative to healthy volun-
teers, had an extensive activation of the visual network, including
the claustrum, lateral temporal and posterior parietal cortices, and
thalamus, in addition to the primary visual cortex, when the clini-
cally affected eye was studied. When the unaffected eye was stimu-
lated, only activations of the visual cortex and the right insula/
claustrum were observed. A strong correlation was found in these
patients between the volume of the extra-occipital activation and
the latency of the visual evoked potential (VEP) P100, suggesting
that the functional reorganization of the cortex might represent an
adaptive response to a persistently abnormal visual input. The
results of this preliminary study have been confirmed and extended
by subsequent studies [14, 15, 17]. Using fMRI and VEP to mon-
itor the functional recovery after an acute unilateral optic neuritis,
Russ et al. [15] found a strong relationship between fMRI and
VEP latencies, suggesting that fMRI might contribute to the
assessment of the temporal evolution of the visual deficits during
recovery. Levin et al. [17] showed reduced activation of the pri-
mary visual cortex and increased activation of the lateral occipital
complex (LOC) in eight subjects who recovered clinically from an
episode of optic neuritis, but who still had prolonged VEP laten-
cies in comparison with healthy controls.
Structural MRI, electrophysiology, and fMRI have been com-
bined in another study to investigate why ON patients exhibit a
wide variation in severity of acute visual loss. Optic nerve lesion
length and VEP amplitude were associated with visual loss. Bilateral
activation in the extra-striate occipital cortex correlated directly
with vision, after adjusting for optic nerve lesion length, VEP
amplitude, and demographic characteristics [19] (Fig. 1).
These data suggest that acute visual loss is associated with the
extent of inflammation and conduction block in the optic nerve,
but not with pathology in the optic radiations or occipital cortex.
The association of better vision with greater fMRI responses, after
accounting for factors which reduce afferent input, suggested a
role for adaptive neuroplasticity within the association cortex of
the dorsal stream of higher visual processing [19]. In a 1-year
follow-up study, Toosy et al. [14], using a novel technique that
modeled the fMRI response and optic nerve structure together
with clinical function, demonstrated a potential adaptive role of
cortical reorganization within the extra-striate visual areas. An
increased optic nerve gadolinium-enhanced lesion length at base-
line was associated with a reduced functional activation within the
visual cortex and poorer vision. At 3 months, more severe optic
nerve damage was associated with an increased fMRI response in
612 Massimo Filippi and Maria A. Rocca

2
2 1

1
BOLD_response
0

0
-1 -2

0 .5 1 1.5 2
logmar_aff1

Fig. 1 Statistical parametric maps showing group correlations between functional magnetic resonance imag-
ing (fMRI) response and visual acuity, after correcting for age, gender, side affected, gadolinium-enhanced
lesion length, fast spin-echo lesion length, and visual evoked potential (VEP) amplitude. A correlation is seen
in the region of the cuneus bilaterally, where better visual acuity is associated with a greater fMRI response.
The graph plots logMAR visual acuity against the mean corrected fMRI response (approximate percentage
blood oxygenated level dependent signal change), at the peak voxel. The statistical parametric maps are thres-
holded at cluster level p < 0.05 (corrected), and the scale bar indicates the voxel level t-scores (from ref. [19])

the bilateral temporal cortices, whereas at 1 year, the right tempo-

ral cortex correlation reversed. These results illustrate how the
same regions may play different roles at different times during
recovery, reflecting the complexity of brain plasticity and the MS
process. This notion has been supported by a region-of-interest
longitudinal study [18] that demonstrated dynamic changes in
the fMRI response following visual stimulation not only in V1,
V2, and the LOC, but also in the lateral geniculate nucleus (LGN)
in patients with isolated acute optic neuritis. In this study, abnor-
mal LGN response was found not only following stimulation of
the affected eye, but also after stimulation of the unaffected one,
indicating that the visual pathways undergo early functional
changes following tissue injury.
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 613

Another 1-year longitudinal study demonstrated that early func-

tional abnormalities in the LOC contribute to predict visual out-
come after a clinically isolated ON [20]. Specifically, greater baseline
fMRI responses in the LOCs were associated with better visual out-
come at 12 months. This was evident on stimulation of either eye
and was independent of measures of demyelination and neuroaxonal
loss. A negative fMRI response in the LOCs at baseline was associ-
ated with a relatively worse visual outcome. These findings suggest
that early neuroplasticity in higher visual areas is likely to be an
important determinant of recovery from ON, independent of tissue
damage in the anterior or posterior visual pathways.
In patients with established MS and a relapsing-remitting (RR)
course with a unilateral optic neuritis, a reduced recruitment of the
visual cortex after stimulation of the affected and the unaffected
eyes was found when compared with healthy subjects. On average,
patients with optimal clinical recovery showed increased visual cor-
tex activation than those with poor or no recovery, although the
extent of the activation remained reduced compared with controls
[8]. Another study [13] of nine patients with previous optic neuri-
tis confirmed the previous results [8] and showed that these
patients not only have a reduced activation of the primary visual
cortex, but also a reduced fMRI percentage signal change in this
region, again suggesting an abnormality of the synaptic input.

2.3 Motor System The investigation of the motor system in patients with MS has
mainly focused on the analysis of the performance of simple motor
tasks with the dominant right upper limbs [9–11, 21–40]. Such
tasks were either self-paced or paced by a metronome. A few stud-
ies assessed the performance of simple motor tasks with the domi-
nant right lower limbs [23, 27, 33], while even fewer studies have
investigated the performance of more complex tasks, including
phasic movements of dominant hand and foot [27, 33], object
manipulation [41], and visuomotor integration tasks [42].
An altered brain pattern of movement-associated cortical acti-
vations, characterized by an increased recruitment of the contralat-
eral primary sensorimotor cortex (SMC) during the performance of
simple tasks [23, 27] and by the recruitment of additional “classi-
cal” and “higher-order” sensorimotor areas during the performance
of more complex tasks [27], has been demonstrated in patients with
clinically isolated syndrome (CIS) suggestive of MS. The clinical
and conventional MRI follow-up of these patients has shown that,
at disease onset, CIS patients with a subsequent evolution to clini-
cally definite MS tend to recruit a more widespread sensorimotor
network than those without short-term disease evolution [36].
These findings suggest that in CIS patients the extent of early corti-
cal reorganization might be a factor associated with a different clini-
cal evolution. This would support the notion that, whereas increased
recruitment of a widespread sensorimotor network contributes to
614 Massimo Filippi and Maria A. Rocca

limiting the impact of structural damage during the course of MS,

its early activation might be counterproductive, as it might result in
an early exhaustion of the adaptive properties of the brain. This
notion is also supported by studies of stroke patients, where a per-
sistent overrecruitment of a widespread cortical network has been
related to an unfavorable clinical outcome [43].
An increased recruitment of several sensorimotor areas, mainly
located in the cerebral hemisphere ipsilateral to the limb that per-
formed the task, has also been demonstrated in patients with early
RRMS and a previous episode of hemiparesis [29]. In patients with
similar characteristics, but who presented with an episode of optic
neuritis, this increased recruitment involved sensorimotor areas that
were mainly located in the contralateral cerebral hemisphere [30].
In patients with established MS and a RR course, functional
cortical changes, mainly characterized by an increased recruitment
of “classical” motor areas, including the primary SMC, the supple-
mentary motor area (SMA), and the secondary sensorimotor cor-
tex (SII), have been shown during the performance of simple
motor [9–11, 22] and visuomotor integration tasks [42].
Movement-associated cortical changes, characterized by the activa-
tion of highly specialized cortical areas, have also been described in
patients with secondary progressive (SP) MS [24] during the per-
formance of a simple motor task and in patients with primary pro-
gressive (PP) MS during the performance of active [21, 33] and
passive [44] motor experiments. Interestingly enough, contrary to
what happens in SPMS, the movement-associated pattern of acti-
vations seen in benign (B) MS resembles that of healthy people,
and its abnormalities are restricted to the sensorimotor network
[45]. These results suggest that the long-term preservation of
brain functional adaptive mechanisms might be among the factors
contributing to the favorable clinical course of BMS.
The concept that movement-associated cortical reorganiza-
tion varies across patients at different stages of the disease has
been shown by a fMRI study of patients with different disease
phenotypes [28] (including 16 patients with a CIS suggestive of
MS, 14 with RRMS and no disability, 15 with RRMS and mild
clinical disability, and 12 with SPMS) acquired during the perfor-
mance of a simple motor task with their unimpaired dominant
hand. CIS patients had an increased activation of the contralateral
primary SMC when compared with those with RRMS and no dis-
ability, whereas patients with RRMS and no disability had an
increased activation of the SMA when compared with those with
CIS (Fig. 2). Patients with RRMS and no disability had an
increased activation of the primary SMC, bilaterally, and ipsilateral
SMA when compared with patients with RRMS and mild clinical
disability. Conversely, patients with RRMS and mild clinical dis-
ability had an increased activation of the contralateral SII, inferior
frontal gyrus (IFG), and ipsilateral precuneus. Patients with
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 615

Fig. 2 Comparisons of patients at presentation with clinically isolated syndrome (CIS) suggestive of MS and
patients with relapsing-remitting (RR) MS and no disability during a simple, right-hand, motor task. Patients
with CIS showed an increased activation of the contralateral primary sensorimotor cortex when compared with
patients with RRMS and no disability (top row). Patients with RRMS and no disability had a more significant
activation of the supplementary motor area, bilaterally, when compared with patients with a CIS (bottom row).
Images are color-coded for activation and arrows show t cut-off values. Activations were superimposed on a
high-resolution T1-weighted scan obtained from one healthy individual and normalized into a standard statisti-
cal parametric mapping space (neurological convention) (from ref. [28])

RRMS and mild clinical disability had an increased activation of

the contralateral thalamus and ipsilateral SII when compared with
those with SPMS. The opposite contrast showed that patients
with SPMS had an increased activation of the IFG, bilaterally,
middle frontal gyrus (MFG), bilaterally, contralateral precuneus,
and ipsilateral cingulate motor area (CMA) and inferior parietal
lobule. This study suggests that early in the disease course more
areas typically devoted to motor tasks are recruited, then a bilat-
eral activation of these regions is seen, and late in the disease
course, areas that healthy people recruit to perform novel or com-
plex tasks are activated [28], perhaps in an attempt to limit the
functional consequences of accumulating tissue damage.
616 Massimo Filippi and Maria A. Rocca

To provide a more comprehensive view of abnormalities of

cerebral response to task stimulation in MS patients at different
stages of the disease, activation and deactivation patterns during
passive hand movements have also been investigated [46]. In line
with previous findings, analysis of activations showed a progressive
extension to the ipsilateral brain hemisphere according to the
group and the clinical form (HC < RRMS < SPMS). Compare to
controls, MS patients had reduced deactivation of the ipsilateral
cortical sensorimotor areas. Deactivation of posterior cortical areas
belonging to the default mode network (DMN) was increased in
RRMS, but not in SPMS, with respect to HC.
As described in another chapter of this book, fMRI has been
also applied for the investigation of cervical cord neuronal activity
during a proprioceptive and a tactile stimulation of the right upper
limb from patients with the main MS clinical phenotypes [47–51].
During the application of both stimulations, healthy controls and
MS patients had significant activations of the cervical cord between
C5 and C8. On average, MS patients had 20 % higher cord fMRI
signal changes during either stimulations, suggesting an abnormal
cord function in these patients. Compared to controls, MS patients
tend to show a more distributed pattern of cord recruitment with
additional activations of regions located in the anterior and contra-
lateral portions of the cord. Such a behavior was more pronounced
in patients with SPMS vs. those with RRMS and PPMS [49, 50].

2.4 Cognition Several fMRI studies have suggested that functional cortical
changes might have an adaptive role also in limiting MS-related
cognitive impairment [52–68]. Therefore, brain plasticity might,
in part, explain the weak relationship found in MS between neuro-
psychological deficits and conventional MRI measures of disease
burden [69].
Several cognitive domains have been investigated in MS
patients with fMRI. Working memory has been the most exten-
sively studied by means of the Paced Auditory Serial Addition Test
(PASAT) or the Paced Visual Serial Addition Task (PVSAT) [52–
55, 60, 64, 70] (which also involve sustained attention, information
processing speed, and simple calculation), the n-back task [59,
61–63, 65, 71], or a task adapted from the Sternberg paradigm
[57]. Additional cognitive domains including attention [58], epi-
sodic memory [72], planning [68], and emotional processing [73]
have also been interrogated.
In patients at presentation with CIS suggestive of MS, an
altered pattern of cortical activations has been described during the
performance of the PASAT [55, 56, 70], confirming the presence
of cortical reorganization at the earliest clinical stage of the disease.
Staffen et al. [52] found that, during the performance of the
PVSAT, MS patients with intact task performance had an increased
activation of several regions located in the frontal and parietal
lobes, bilaterally, compared with healthy volunteers, suggesting the
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 617

Fig. 3 Brain patterns of cortical activations on a rendered brain during the execution of the Paced Auditory
Serial Addition Task (PASAT) in (a) 22 healthy controls and in (b) 22 patients with MS. (b1, b2) Rendered images
for patients with MS subgrouped according to their performance at the PASAT during fMRI showing significant
activated foci in (b1) for 12 patients whose performance was similar to that of healthy controls and in (b2) the
activations found in the 10 patients who exhibited lower scores (from ref. [60])

presence of functional compensatory mechanisms. An increased

recruitment of several cortical areas during the performance of a
simple cognitive task has also been shown in patients with RRMS
and mild clinical disability (Fig. 3) [60].
An increased activation of regions exclusively located in the right
cerebral hemisphere (in particular in the frontal and temporal lobes)
has also been found in MS patients when testing rehearsal within
working memory [57]. The degree of right hemisphere recruitment
was strongly related to patient neuropsychological performance
[57]. In patients with RRMS and no cognitive deficits, using fMRI
during an n-back test, a reduced activation of the “core” areas of the
working memory circuitry (including prefrontal and parietal regions)
and an increased activation of other regions within and beyond the
typical working memory circuitry (including areas in the frontal,
parietal, temporal, and occipital lobes) have been found [63]. This
shift of activation was most prominent with increased working mem-
ory demands. These findings suggest that, as shown for motor and
visual tasks, dynamic changes of brain activation patterns can occur
in RRMS patients during cognitive tasks. Other studies [61, 62, 66],
which also investigated working memory performance in MS
618 Massimo Filippi and Maria A. Rocca

Fig. 4 Analysis of functional connectivity during the performance of the n-back task with different levels of
difficulty in healthy individuals and patients with MS. The most significant correlations between activation in
regions involved in processing increasing task demand are indicated in (a). In (b) are those connections more
significant for controls (p < 0.05). The image in (c) shows connections that were more significant in patients
than controls (p < 0.05). C cingulate, SF superior medial frontal, RF right dorsolateral prefrontal, LF left dorso-
lateral prefrontal, RP right parietal, LP left parietal (from ref. [65])

patients, demonstrated: (1) an increased recruitment of regions

related to sensorimotor functions and anterior attentional/executive
components of the working memory system in patients compared
with healthy controls, and (2) a reduced recruitment of several
regions in the right cerebellar hemisphere in patients compared with
healthy individuals [66], thus suggesting that the cerebellum might
play a role in the working memory impairment of MS.
In a fMRI study [65], working memory was investigated with an
n-back task and functional connectivity analysis in a group of 21
RRMS patients and 16 age- and sex-matched healthy controls. With
similar task performances, activations were found in similar regions for
both groups. However, patients had relatively reduced activations of
the superior frontal and anterior cingulate gyri. Patients also showed a
variable, but generally substantially smaller increase of activation than
healthy controls with greater task complexity, depending on the spe-
cific brain regions assessed. These findings suggest that, despite similar
brain regions were recruited in both groups, patients have a reduced
functional reserve for cognition relevant to memory. The functional
connectivity analysis revealed increased correlations between right
dorsolateral prefrontal and superior frontal/anterior cingulate activa-
tions in controls, and increased correlations between activations in the
right and left prefrontal cortices in patients (Fig. 4), suggesting that
altered interhemispheric interactions between dorsal and lateral pre-
frontal regions may yet be an additional adaptive mechanism distinct
from recruitment of novel processing regions [65].
Using a 3 T scanner, more significant activations of several areas
of the cognitive network involved in the performance of the Stroop
test have also been demonstrated in a group of 15 cognitively pre-
served patients with benign MS (BMS) when compared with 19
healthy controls (Fig. 5) [67]. BMS patients also showed an
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 619

Fig. 5 Areas showing increased activations in patients with BMS in comparison with healthy controls during
the analysis of the Stroop facilitation condition (random effect interaction analysis, ANOVA, p < 0.05 corrected
for multiple comparisons). BMS patients had increased activations of several areas located in the frontal and
parietal lobes, bilaterally, including the anterior cingulate cortex, the superior frontal sulcus, the inferior frontal
gyrus, the precuneus, the secondary sensorimotor cortex, the bilateral visual cortex, and the cerebellum, bilat-
erally. Note that the color-encoded activations have been superimposed on a rendered brain and normalized
into standard SPM space (neurological convention) (see ref. [67])

increased connectivity of several cortical areas of the sensorimotor

network, including the left IFG, the anterior cingulated cortex and
the left SII, with the right IFG and the right cerebellum, as well as
a decreased connectivity between some areas (including the left SII,
the prefrontal cortex, and the right cerebellum), and the anterior
cingulate cortex. These results suggest an altered interhemispheric
balance in favor of the right hemisphere in BMS patients in com-
parison with healthy controls, when performing cognitive tasks.
fMRI during the performance of the Stroop task has also been
used to determine whether modification of the connections between
cerebellar and prefrontal areas might vary among MS phenotypes and
might be associated with cognitive failure [74]. Activation and effec-
tive connectivity analyses showed that, compared with the other
groups, RRMS patients had abnormal recruitment of regions of the
left frontoparietal lobes, whereas compared with RRMS, SPMS
patients had abnormal recruitment of the posterior cingulum/precu-
neus. BMS patients had increased activation of the right prefrontal
cortex, and increased interaction between these regions and the right
cerebellum. In healthy controls, reaction times (RTs) inversely corre-
lated with activity of right cerebellum and several frontoparietal
regions. In MS, RTs inversely correlated with bilateral cerebellar activ-
ity and directly correlated with right precuneus activity. In MS, disease
duration inversely correlated with right cerebellar activity and directly
correlated with left inferior frontal gyrus and right precuneus activity.
Higher T2 lesion volume and lower brain volumes were related to
activity in these areas. These results suggest that MS patients who have
various clinical phenotypes experience different abnormalities in acti-
vation and effective connectivity between the right cerebellum and
frontoparietal areas, which contribute to inefficient cortical reorgani-
zation, with increasing cognitive load.
620 Massimo Filippi and Maria A. Rocca

2.5 Correlations In addition to an abnormal pattern of functional activations, the

Between the Extent majority of the previous studies described a variable relationship
of Functional Cortical between the extent of fMRI activation and several measures of tis-
Reorganization sue damage [7, 8, 10, 11, 21–25, 30, 33, 42, 53, 56, 58, 60].
and the Extent of Brain An increased recruitment of several brain areas with increasing
Damage in MS T2 lesion load has been shown in patients with RR [7, 8, 54, 60,
72] and PPMS. The severity of intrinsic T2-visible lesion damage,
measured using T1-weighted images [30], magnetization transfer
(MT), and diffusion tensor (DT) MRI [22], has been found to
modulate the activity of some cortical areas in these patients. The
severity of normal appearing brain tissue (NABT) injury, measured
using proton MR spectroscopy [10, 11, 25], MT MRI [21, 22, 53,
56], and DT MRI [21, 23, 25], is another important factor associ-
ated to an increased recruitment of motor- and cognitive-related
brain regions, as shown by studies of patients at presentation with
CIS suggestive of MS [23, 53, 56], patients with RRMS and vari-
able degrees of clinical disability [10, 11, 22, 25], patients with
PPMS [21], and with SPMS [10, 11]. Finally, subtle GM damage,
which goes undetected when using conventional MRI, may also
influence functional cortical recruitment, as demonstrated, for the
motor system, in patients with RRMS [42], SPMS [24], and
patients with clinically definite MS and nonspecific (less than three
focal white matter lesions) conventional MRI findings [25]. In
cognitively intact MS patients, the increased activation of a left
prefrontal region during the counting Stroop task has been corre-
lated with the normalized brain parenchymal volume [58].

2.6 Functional Structural damage of white matter pathways that connect functional
Cortical relevant areas for a given task has been shown to modify the observed
Reorganization brain patterns of cortical activations in patients with MS. Damage to
and Regional Damage the corticospinal tract [30, 31] (Fig. 6) as well as damage to the
in MS corpus callosum (CC) [75, 76] has been related to a more bilateral
movement-associated brain pattern of cortical activations.
The role of the CC in interhemispheric connectivity and in elic-
iting functional cortical changes has been underpinned by a study by
Lowe et al. [32], who showed, by measuring low-frequency BOLD
fluctuations, a reduced functional connectivity between the right
and the left hemisphere primary motor cortices in MS patients.
The recent development of diffusion-based tractography
methods that allow to define with precision the pathways connect-
ing different CNS structures and their application to patients with
MS resulted in an improvement of the correlation between struc-
tural and functional abnormalities. Several studies combined
measures of abnormal functional connectivity with DT MR mea-
sures of damage within selected white matter fiber bundles in
patients with RRMS [40], BMS [67] and PPMS [77]. In patients
with RRMS and no clinical disability [40], measures of abnormal
connectivity inside the motor network were correlated with
Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 621

Fig. 6 Brain patterns of cortical activations on a rendered brain from MS patients

without (a and c) and with (b and d) lesions in the left corticospinal tracts, during
the performance of a simple motor task with their clinically unimpaired and fully
normal functioning, dominant right hands. In patients with corticospinal tract
lesions, a more bilateral pattern of activations is visible. Note that the activations
are color-coded according to their t values. Images are in neurological conven-
tion (see ref. [31])

structural MRI metrics of tissue damage of the corticospinal and

the dentatorubrothalamic tracts, while no correlation was found
with measures of damage within “not-motor” white matter fiber
bundles. These findings suggest an adaptive role of functional con-
nectivity changes in limiting the clinical consequences of structural
damage to selected white matter pathways in RRMS patients [40].
In patients with BMS [67], measures of abnormal connectivities
inside the cognitive network were moderately correlated with
structural MRI metrics of tissue damage within intra- and inter-
hemispheric cognitive-related white matter fiber bundles, while no
correlations were found with the remaining fiber bundles studied,
suggesting that functional cortical changes in patients with BMS
might represent an adaptive response driven by damage to specific
white matter structures [67].
In patients with PPMS [21], a relationship has been demon-
strated between the severity of spinal cord pathology, measured
using MT MRI, and the extent of movement-associated cortical
activations.
622 Massimo Filippi and Maria A. Rocca

2.7 Adaptive Role Although the actual role of cortical reorganization on the clinical
of Functional Cortical manifestations of MS remains to be established, there are several
Reorganization in MS pieces of evidence which suggest that cortical adaptive changes are
likely to contribute in limiting the clinical consequences of MS-related
structural damage. In nondisabled patients with RRMS [22], an
increased activation of several motor regions, mainly located in the
contralateral cerebral hemisphere, has been seen during the perfor-
mance of a simple motor task. The correlations found in this study
[22] between the extent of fMRI activations and several MT and DT
MRI metrics of structural brain damage suggested that an increased
recruitment of movement-associated cortical network contributes to
limiting the functional impact of MS-related damage.
The notion that an increased recruitment of areas that are usu-
ally activated by healthy individuals when performing different/
more complex motor tasks might be one of the mechanisms play-
ing a role in MS recovery/maintenance of function has been high-
lighted by the results of two experiments [39, 41]. The first showed
that MS patients, during the performance of a simple motor task,
activate some regions that are part of a fronto-parietal circuit,
whose recruitment occurs typically in healthy subjects during
object manipulation (Fig. 7) [41]. The second, which assessed the
fMRI patterns of activation during the performance of a simple
motor task and of a task aimed at investigating the mirror-neuron
system, demonstrated activations of regions that are part of the
mirror-neuron system in patients with MS during the performance
of the simple motor task [39].
The compensatory role of cortical reorganization has also been
demonstrated by studies investigating the cognitive domains,
which showed increased recruitment of several cortico-subcortical
areas in cognitively preserved MS patients [52–58]. In patients
complaining of fatigue, when compared with matched nonfatigued
MS patients [78], a reduced activation of a complex movement-
associated cortical/subcortical network, including the cerebellum,
the thalamus, and regions in the frontal lobes, has been shown.
The correlation found in these patients between the reduction of
thalamic activity and the clinical severity of fatigue indicates that a
“pseudoreduction” of brain functional recruitment might be asso-
ciated with the appearance of MS symptomatology. Additional
work has shown that the pattern of movement-associated cortical
activations in MS is determined by both the extent of brain injury
and disability and that these changes are distinct [26, 33].

2.8 Maladaptive Role The results of several studies suggest that an increased cortical
of Functional Cortical recruitment might not always be beneficial for patients with MS. As
Reorganization already mentioned, disease progression and accrual of disability has
been observed in patients with SPMS, despite the widespread acti-
vations of regions in the frontal and parietal lobes during the per-
formance of simple motor tasks [24, 28]. fMRI studies of the
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 623

Fig. 7 Comparison of simple vs. complex task with the dominant right hands in healthy subjects (top row; a–c)
and patients with MS (bottom row; d–g) (paired t test for each group, corrected p value <0.05). The ipsilateral
anterior lobe of the cerebellum (a and d), bilateral insula/basal-ganglia (b, e, f) and contralateral primary sen-
sorimotor cortex and supplementary motor area (c and g) were identified in both groups. Compared with
healthy subjects, MS patients also had a significant activation of the contralateral inferior frontal gyrus and
bilateral secondary sensorimotor cortex (e). Note that the activations are color-coded according to their t val-
ues. Images are in neurological convention (see ref. [41])

motor system [21, 33, 44] of patients with PPMS suggested a lack
of “classical” adaptive mechanisms as a potential additional factor
contributing to the accumulation of disability. In these patients
during the performance of different motor tasks with the nonim-
paired dominant limbs, a recruitment of a widespread movement-
associated cortical network usually considered to function in
motor, sensory, and multimodal integration processing (i.e., the
frontal and temporal lobes, and the insula) was detected [21, 33,
44]. The absence of a concomitant recruitment of the “classical”
motor areas, including the primary SMC, the SMA, the infrapari-
etal sulcus, and the SII, was interpreted as a failure of part of the
adaptive capacity of the cerebral cortex in this severely disabling
phenotype of the disease [21, 33]. The notion that multimodal
integration areas might have a critical role in PPMS patients has
been strengthened by another study which showed increased
624 Massimo Filippi and Maria A. Rocca

activations of these regions in PPMS patients, in comparison with

healthy controls, during passive movements [44]. Similar findings
have led to similar conclusions in patients with cognitive decline
[59, 64], in whom a “poor” pattern of cortical activations in the
expected areas [59] and the activation of regions that are not nor-
mally devoted to the performance of the investigated task [64]
were detected during the performance of the administered tasks.
The comparison of the movement-associated brain patterns of
cortical activations between RRMS patients complaining of revers-
ible fatigue after weekly interferon (IFN) beta-1a administration
and those without fatigue suggested an association between the
presence of fatigue and an increased activation of several areas of
the motor network, including the thalamus, the cingulum, and
several regions located in the frontal lobes, including the SMA and
the primary SMC bilaterally [37] (Fig. 8). These results suggest
that the overrecruitment of brain networks in MS might, at least to
some degree, have a detrimental effect.
The assessment and interpretation of fMRI results (cerebral
activity) during cognitive tasks can be difficult when task
performance differs across patients (e.g., poorer performance in
cognitively impaired patients). As such, the analysis of resting state
(RS) brain functional connectivity has been proposed as a valid
alternative to task-active fMRI investigations, particularly in clini-
cally impaired populations [79].
The analysis of brain activity at rest has shown an increased
synchronization of the majority of the resting-state networks in
CIS patients [80]. In patients with cognitive impairment a reduced
functional connectivity of anterior regions of the brain, mostly
located in the frontal lobes [81, 82] is related not only to the sever-
ity of cognitive impairment, but also to structural disruption of
connecting WM tracts [81]. In another study [83] better cognitive
performance in MS patients was associated with an increased func-
tional connectivity among several regions of the attention network,
thus supporting the adaptive role of RS functional connectivity
modifications. However, findings from other studies [84–86] seem
to contradict the adaptive/compensation hypothesis since a cor-
relation between increased functional connectivity and worse cog-
nitive performance was found.
A recent study assessing intra- and inter-network functional
connectivity at rest in the brain from RRMS patients has demon-
strated a distributed pattern of abnormal functional connectivity,
which was correlated to the extent of T2 lesions and the severity of
disability [87]. If such functional abnormalities confer a systematic
vulnerability to disease progression or, conversely, protect against
the onset of clinical deficits needs to be investigated. The “cogni-
tive reserve hypothesis” has also been considered to explain the
incomplete relationship between brain disease and cognitive status
in MS; it has been demonstrated that MS patients with a greater
Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 625

Fig. 8 Relative cortical activations of MS patients with reversible fatigue after

interferon (IFN) beta-1a injection during the performance of a simple motor task
with their clinically unimpaired and fully normal functioning, dominant right hands.
At entry (a and b) (when they did not complain of fatigue), compared with MS
patients without reversible fatigue, these patients showed an increased recruit-
ment of the contralateral primary SMC (a), the thalamus (b), the superior frontal
sulcus (a and b), and the cingulate motor area (b). At day 1 (c and d) (after IFN
beta-1a administration, when fatigue was present), compared with MS patients
without fatigue, these patients showed increased recruitment of the ipsilateral
thalamus (d), and contralateral middle frontal gyrus (c). Note that the color-coded
activations have been superimposed on a high-resolution T1-weighted scan
obtained from a single, healthy subject and normalized into standard statistical
parametric mapping space (neurological convention) (from ref. [37])

intellectual enrichment require less deactivation of the brain at rest

and less recruitment of prefrontal cortices to perform tasks in a way
equal to that of patients with lesser enrichment [88].
Despite the clear advantages of RS fMRI, it is important to note
that there might be some mechanisms related to cognitive network
function which are unlikely to be captured properly by the use of a
non task-active/resting paradigm. In particular, it has been sug-
gested that, compared to healthy controls, MS patients might have
626 Massimo Filippi and Maria A. Rocca

a limited ability to increase the activation/deactivation within

cognitive-related networks with increasing task difficulty. Such an
inability to optimize cognitive network recruitment, which reflects
an impaired cognitive functional reserve (that is the ability to match
brain activity to increasing cognitive demand) [65, 89–91], is likely
to be a maladaptive mechanism contributing to the clinical manifes-
tations of the disease, since it is more pronounced in patients with
SPMS [89] as well as in those with cognitive impairment [92].

2.9 Use of fMRI Dynamic functional changes have been described in an MS patient
to Assess Longitudinal following an acute relapse [10]. These results have been confirmed
Changes of Cortical and extended by another study which assessed the early cortical
Reorganization changes following acute motor relapses secondary to pseudotu-
moral lesions in 12 MS patients and the evolution over time of
cortical reorganization in a subgroup of these patients [38]. Short-
term cortical changes were mainly characterized by the recruitment
of pathways in the unaffected hemisphere. A recovery of function
of the primary SMC of the affected hemisphere was found in
patients with clinical improvement, while in patients without clini-
cal recovery, there was a persistent recruitment of the primary
SMC of the unaffected hemisphere, suggesting that the restoration
of function of motor areas of the affected hemisphere might be a
critical factor for a favorable recovery (Fig. 9).
A longitudinal (time interval of 15–26 months) fMRI study of
the motor system has been conducted in a group of patients with
early RRMS [93]. Patients exhibited greater bilateral activations
than controls in both fMRI studies. Although no significant
differences between the two fMRI scans were observed in controls,
a reduction of the functional activity of the ipsilateral SMC and the
contralateral cerebellum was seen in patients at follow-up.
Moreover, activation changes in ipsilateral motor areas correlated
inversely with age, extent, and progression of T1 lesion load, and
occurrence of a new relapse, suggesting that younger patients with
less structural brain damage and a favorable clinical course demon-
strate brain plasticity that follows a more lateralized pattern of
brain activations [93].
Longitudinal modifications of cognitive networks’ recruitment
and their impact on patients’ cognitive status have been marginally
explored. A 1-year longitudinal study in patients with early MS
found an association between increased levels of activation in the
right dorsolateral prefrontal cortex during a cognitive task and
improved working memory and processing speed performance
[94]. A 20 month longitudinal study [95] showed that worsening
of SDMT performance in RRMS patients is correlated with
increased activity of the left inferior parietal lobule over time, prob-
ably reflecting a maladaptive mechanism.
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 627

Fig. 9 Longitudinal evolution of cortical activations in the primary sensorimotor cortex (SMC), bilaterally, during
task performance with impaired hand compared with unimpaired hand in one patient with good clinical recov-
ery during follow-up (a and b) and in one patient with poor/absent clinical recovery (c and d). Scans obtained
during left hand motor task have been flipped to keep the left hemisphere contralateral to movement. At
baseline, both patients showed an increased activation of the primary SMC of the unaffected (ipsilateral)
hemisphere (a and c). During follow-up, the patient with good clinical recovery showed an increased recruit-
ment of the primary SMC of the affected hemisphere (b), while the patient with poor/absent clinical recovery
continued to show an increased recruitment of the primary SMC of the unaffected hemisphere (d). Note that
the activations are color-coded according to their t values (see ref. [38])

2.10 fMRI to Monitor The potential of fMRI in a multicentre setting has been explored by
Treatment a few studies of the motor [96, 97] and cognitive [92] networks.
Only a few fMRI studies have been performed to monitor the
effect of treatments in MS [58, 98, 99]. Different patterns of
brain response to lower dose acute and higher dose chronic
administration of rivastigmine have been demonstrated in MS
patients using cognitive tasks [58, 99]. Rivastigmine was shown to
enhance the prefrontal function and alter the functional connec-
tivity associated with cognition. In MS patients, increased activa-
tion in the ipsilateral primary SMC and SMA has been observed
after a single dose of 3,4-diaminopyridine (a potassium channel
628 Massimo Filippi and Maria A. Rocca

blocker), suggesting that this treatment may modulate brain

motor activity in patients with MS, probably by enhancing excit-
atory synaptic transmission [98].
Active and RS fMRI have also been applied to assess modifica-
tions of the patterns of activations and functional connectivity fol-
lowing motor [100] and cognitive rehabilitation in a few
single-center studies [101–103]. A recent study demonstrated that
changes in RS functional connectivity of cognitive-related net-
works contributes to the persistence of the effects of cognitive
rehabilitation after 6 months in RRMS patients [104].

3 fMRI in MS-Allied Conditions

Among the known MS-allied conditions, only patients with neuro-

myelitis optica (NMO) have been assessed using fMRI. Rocca et al.
[35] investigated the performance of a simple motor task with the
dominant and nondominant upper limbs in ten patients with NMO
and found that, compared with matched controls, NMO patients
had an increased recruitment of several regions of the sensorimotor
network (primary SMC, postcentral gyrus, MFG, rolandic opercu-
lum, SII, precuneus, and cerebellum) and of several other regions
mainly in the temporal and occipital lobes, such as MT/V5, the
fusiform gyrus, the cuneus, and the parahippocampal gyrus during
the performance of both tasks. For both tasks, strong correlations
were found between relative activations of cortical sensorimotor
areas and the severity of cervical cord damage, suggesting that the
observed functional cortical changes might have an adaptive role in
limiting the clinical outcome of NMO structural pathology.
Using RS fMRI, abnormal functional connectivity in the
majority of RS networks has also been detected in NMO patients
and has been correlated to the severity of clinical disability [105].

4 fMRI in Other WMD and Conditions Associated with “Significant”

White Matter Damage

4.1 Isolated Spinal Studies of patients with spinal cord injury of different etiology (i.e.,
Cord Injury traumatic and/or demyelinating) with no or only partial clinical
recovery have shown movement-associated cortical changes, consist-
ing of an abnormal location of the activated areas and in a more wide-
spread recruitment of motor areas, mainly located in the hemisphere
contralateral to the limb used to perform the task [106–108].
In patients with isolated myelitis of probable demyelinating
origin and normal function in the investigated limbs, an abnormal
pattern of movement-associated cortical activation has been
described [34, 109, 110] and has been related to the degree of
daily hand use [109], the severity of cervical cord damage [34,
110], and the level of spinal cord involvement [110].
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 629

4.2 CNS Vasculitides In patients with neuropsychiatric systemic lupus erythematosus

(NPSLE) without overt motor impairment, movement-associated
functional cortical changes, characterized by more significant acti-
vations of the contralateral primary SMC, putamen, dentate
nucleus, several regions located in the frontal and parietal lobes,
MT/V5, and the middle occipital gyrus, bilaterally, have been
observed when compared with matched healthy controls [111]
(Fig. 10). The correlations found in these patients between relative
activations of sensorimotor areas and the extent and severity of
brain damage suggest that also in these patients functional cortical
changes might contribute to the maintenance of their normal func-
tional capacities [111] (Fig. 10).

4.3 Migraine Several fMRI studies have contributed to the classification of

migraine as a neurovascular or even a brain disorder. The pioneer-
ing study by Hadjikhani and coworkers [112] has shown that dur-
ing induced and spontaneous visual aura a focal increase of BOLD
signal (possibly reflecting vasodilation) developed within occipital
extrastriate cortex (area V3A) (Fig. 11).
This BOLD change progressed contiguously and slowly over
the cortex, congruent with the retinotopy of the visual percept and,
then, following the same retinotopic progression, it diminished
(possibly reflecting vasoconstriction). This spreading signal

a b e 1
5

3 0.5
LSMC response

1
0
c d
0

-0.5
0 5 10 15
Dual-echo lesion load (ml)

Fig. 10 Relative cortical activations in patients with neuropsychiatric systemic lupus erythematosus during a
simple motor task with the right hand in comparison with healthy volunteers (color-coded t values). (a)
Contralateral primary sensorimotor cortex. (b) Contralateral putamen, contralateral middle frontal gyrus (MFG),
bilateral MT/V5 complex, contralateral middle occipital gyrus (MOG). (c) Contralateral putamen, ipsilateral
inferior frontal gyrus, bilateral MT/V5 complex, ipsilateral MOG. (d) Contralateral dentate nucleus. The relative
activation of the contralateral primary sensorimotor cortex was significantly correlated with brain dual-echo
lesion load (e) (p < 0.001, r = 0.79) (from ref. [111])
630 Massimo Filippi and Maria A. Rocca

Fig. 11 Source localization and time of onset of the blood oxygenation level-dependent (BOLD) signal changes
during induced and spontaneous visual aura attack in patients with migraine: (a) data are represented on
inflated cortical surface shown from a posterior-medial view; (b) a fully flattened view of the cortical surface
of the involved region. Cortical regions showing the first BOLD perturbations are coded in red (according to the
color-coded scale representing variation in time) and locations showing the BOLD perturbations at progres-
sively later times are coded by green and blue (according to the color-coded scale representing variation in
time). The aura-related changes appeared first in extrastriate cortex (area V3A) and then progressed contigu-
ously and slowly over the cortex following the same retinotopic progression of visual disturbance. From [112]
with permission

disturbance had striking similarity with cortical spreading depression

(CSD) phenomenon, thus supporting the theory that CSD was the
electrophysiological correlate of visual aura. Notably, a spreading
neural activity suppression has also been described in migraine with-
out aura (MWoA) patients during triggered migraine attacks [113].
FMRI studies have also confirmed dysfunctional activity of
brainstem nuclei involved in pain modulation during both the ictal
and interictal phases [114]. Stankewitz et al. [115] confirmed an
increased activation of the dorsal pons in migraine patients during
induced migraine attacks. This study also revealed a selective
gradient-like activity in the spinal trigeminal nucleus: after trigemi-
nal nociceptive stimulation, interictal migraine patients exhibited
lower activations of this nucleus; however, shortly before the
migraine attack, patients had an increased activation at this level.
Of interest, the time interval to the next headache attack could be
predicted by the amplitude of signal intensities in the spinal tri-
geminal nuclei, suggesting that this oscillating behavior may repre-
sent a key phenomenon in migraine pathogenesis.
Studies which have explored cerebral activity with the pain net-
work in migraine using experimental pain stimulation have shown
abnormalities of a widespread subcortical and cortical brain net-
work involved in pain processing in these subjects. However, one
Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 631

of the main challenges in the interpretation of these results is to

differentiate findings consistent with a general pain response from
those that might be specific to migraine [116]. Using a contact
thermode as a noxious stimulation paradigm, migraine patients
were found to exhibit a greater activation in the anterior cingulate
cortex at 51 °C and less activation in the bilateral somatosensory
cortex at 53 °C [117], thus supporting the presence of an increased
antinociceptive activity in these patients, which could represent a
compensatory functional reorganization aimed at modulating pain
perception to the intensity of healthy controls. Other fMRI studies
have demonstrated alterations in pain modulatory/inhibitory cir-
cuits, which may be related to the lack of habituation after repeti-
tive painful stimulation and increased cortical excitability to painful
stimuli, that may lead to the development of allodynia [118]. The
thalamus is now considered to play a pivolat role in the manifesta-
tion of allodynia. Burstein et al. [119] showed that brush and heat
stimulation at the skin of the dorsum of the hand produced larger
BOLD responses in the posterior thalamus of patients undergoing
a migraine attack with extracephalic allodynia than the correspond-
ing responses registered when the same patients were free of
migraine and allodynia.
An increased activation of cortical regions mediating the affec-
tive dimension of pain has also been demonstrated in migraineurs.
During spontaneous and induced migraine, these patients had
increased BOLD signal intensities in limbic structures (e.g., the
amygdala and insula) and exhibited a stronger recruitment of affec-
tive cortical areas when exposed to emotional inputs [118]. Based
on these data, a model of migraine as a dysfunction of a “neurolim-
bic” pain network has been proposed [120].
Abnormalities of function of pain processing regions have also
been investigated in patients with chronic migraine, particularly
those with medication-overuse headache (MOH). Before the
withdrawal of the offending medications, these patients had
reduced pain related activity in areas of the lateral pain pathway,
including the primary and secondary sensorimotor area. Such
abnormalities regressed after treatment withdrawal [121]. In addi-
tion, patients with MOH presented dysfunctional activity of the
meso-cortico-limbic dopamine circuit, including the ventro-medial
prefrontal cortex and the substantia nigra/ventral tegmental area
complex, during the execution of a decision-making under risk
paradigm. The ventro-medial prefrontal cortex dysfunctions were
reversible and attributable to the headache, whereas the substantia
nigra/ventral tegmental area complex dysfunctions were persistent
despite treatment withdrawal, suggesting that MOH may share
some neurophysiological features with addiction [118].
It is well established that migraine patients show also hyper-
responsiveness of the primary visual cortex and a lack of habitua-
tion to visual stimuli [114]. These phenomena are more pronounced
632 Massimo Filippi and Maria A. Rocca

in patients with migraine with aura (MWA) [122].

Hyper-responsiveness of the visual cortex in migraine extends
beyond primary visual areas, even in the interictal period. Antal
et al. [123] demonstrated significantly stronger activation of the
extrastriate, motion-responsive MT area, representing the medial-
superior temporal area, in migraine patients vs. healthy controls in
response to coherent/incoherent moving dot stimuli. This cortical
hyperexcitability may represent the biological basis for the clinical
observation of heightened vulnerability to motion sickness that
migraine sufferers often report [114].
Numerous studies provided a conceptual framework for under-
standing vestibular migraine as a variant of MWA produced by the
convergence of vestibular information within migraine circuits.
Several fMRI studies showed that vestibular stimulation activate
cerebral regions that are generally involved in migraine and pain
perception, such as the posterior and anterior insula, orbitofrontal
cortex, and the posterior and anterior cingulate gyri, thus suggest-
ing that central constituents of the migraine circuit might include
components of central vestibular pathways [124]. However, so far,
fMRI has not been applied yet to investigate functional cortical
abnormalities in patients with vestibular migraine.
RS fMRI studies have shown that functional connectivity is
generally increased in pain-processing networks in migraineurs,
whereas it is decreased in pain modulatory circuits [125]. In par-
ticular, migraineurs with a history of allodynia exhibit significantly
reduced RS functional connectivity between periacqueductal gray
matter (PAG), prefrontal regions, and anterior cingulate cortex
compared with migraineurs without allodynia [126]. This RS func-
tional connectivity abnormalities have been related to the fre-
quency of migraine attacks and disease duration [125]. Significant
abnormalities of RS functional connectivity occur also in affective
networks [125], the DMN [127] and the executive network [116].

5 Conclusions

Taken all together, fMRI studies of patients with various white

matter disorders demonstrate the potential of this technique to
provide important insights into the mechanisms of cortical reorga-
nization following white matter injury. As a consequence, fMRI
holds promise to improve our understanding of the factors associ-
ated with the accumulation of irreversible disability in MS and
other white matter conditions. Although the role of cortical reor-
ganization in limiting the functional impact of white matter struc-
tural damage is still not proved definitively, the available data
support the concept that cortical adaptive responses may have an
important role in compensating for irreversible tissue damage,
such as axonal loss. Thereby, it can be concluded that the presently
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 633

available fMRI data suggest that the rate of accumulation of

disability in MS and other white matter disorders might be a func-
tion not only of tissue loss, but also of the progressive failure of the
adaptive capacity of the brain with increasing tissue damage.

References
1. Filippi M, Rocca MA (2004) Magnetization ing optic neuritis recovery. Neurosci Lett
transfer magnetic resonance imaging in 330(3):255–259
the assessment of neurological diseases. 15. Russ MO et al (2002) Functional magnetic
J Neuroimaging 14(4):303–313 resonance imaging in acute unilateral optic
2. Filippi M, Rocca MA, Comi G (2003) The neuritis. J Neuroimaging 12(4):339–350
use of quantitative magnetic-resonance-based 16. Toosy AT et al (2005) Adaptive cortical plas-
techniques to monitor the evolution of mul- ticity in higher visual areas after acute optic
tiple sclerosis. Lancet Neurol 2(6):337–346 neuritis. Ann Neurol 57(5):622–633
3. Hesselink JR (2006) Differential diagnostic 17. Levin N et al (2006) Normal and abnor-
approach to MR imaging of white matter dis- mal fMRI activation patterns in the visual
eases. Top Magn Reson Imaging 17:243–263 cortex after recovery from optic neuritis.
4. Rocca MA, Filippi M (2006) Functional MRI Neuroimage 33(4):1161–1168
to study brain plasticity in clinical neurology. 18. Korsholm K et al (2007) Recovery from optic
Neurol Sci 27(Suppl 1):S24–S26 neuritis: an ROI-based analysis of LGN and
5. Rocca MA, Filippi M (2007) Functional visual cortical areas. Brain 130(Pt 5):1244–1253
MRI in multiple sclerosis. J Neuroimaging 19. Jenkins T et al (2010) Dissecting structure-
17(Suppl 1):36S–41S function interactions in acute optic neuritis to
6. Waxman SG (1998) Demyelinating diseases— investigate neuroplasticity. Hum Brain Mapp
new pathological insights, new therapeutic 31(2):276–286
targets. N Engl J Med 338(5):323–325 20. Jenkins TM et al (2010) Neuroplasticity pre-
7. Clanet M, Berry I, Boulanouar K (1997) dicts outcome of optic neuritis independent
Functional imaging in multiple sclerosis. Int of tissue damage. Ann Neurol 67(1):99–113
MS J 4:26–32 21. Filippi M et al (2002) Correlations
8. Rombouts SA et al (1998) Visual activa- between structural CNS damage and func-
tion patterns in patients with optic neu- tional MRI changes in primary progressive
ritis: an fMRI pilot study. Neurology MS. Neuroimage 15(3):537–546
50(6):1896–1899 22. Rocca MA et al (2002) Adaptive functional
9. Lee M et al (2000) The motor cortex shows changes in the cerebral cortex of patients with
adaptive functional changes to brain injury from nondisabling multiple sclerosis correlate with
multiple sclerosis. Ann Neurol 47(5):606–613 the extent of brain structural damage. Ann
10. Reddy H et al (2000) Relating axonal injury Neurol 51(3):330–339
to functional recovery in MS. Neurology 23. Rocca MA et al (2003) Evidence for axonal
54(1):236–239 pathology and adaptive cortical reorganiza-
11. Reddy H et al (2000) Evidence for adaptive tion in patients at presentation with clinically
functional changes in the cerebral cortex with isolated syndromes suggestive of multiple
axonal injury from multiple sclerosis. Brain sclerosis. Neuroimage 18(4):847–855
123(Pt 11):2314–2320 24. Rocca MA et al (2003) A functional mag-
12. Werring DJ et al (2000) Recovery from optic netic resonance imaging study of patients
neuritis is associated with a change in the distri- with secondary progressive multiple sclerosis.
bution of cerebral response to visual stimulation: Neuroimage 19(4):1770–1777
a functional magnetic resonance imaging study. 25. Rocca MA et al (2003) Functional cortical
J Neurol Neurosurg Psychiatry 68(4):441–449 changes in patients with multiple sclerosis and
13. Langkilde AR et al (2002) Functional MRI of nonspecific findings on conventional mag-
the visual cortex and visual testing in patients netic resonance imaging scans of the brain.
with previous optic neuritis. Eur J Neurol Neuroimage 19(3):826–836
9(3):277–286 26. Reddy H et al (2002) Functional brain reorgani-
14. Toosy AT et al (2002) Functional magnetic zation for hand movement in patients with mul-
resonance imaging of the cortical response tiple sclerosis: defining distinct effects of injury
to photic stimulation in humans follow- and disability. Brain 125(Pt 12):2646–2657
634 Massimo Filippi and Maria A. Rocca

27. Filippi M et al (2004) Simple and complex 41. Filippi M et al (2004) A functional MRI
movement-associated functional MRI changes study of cortical activations associated
in patients at presentation with clinically iso- with object manipulation in patients with
lated syndromes suggestive of multiple sclero- MS. Neuroimage 21(3):1147–1154
sis. Hum Brain Mapp 21(2):108–117 42. Cerasa A et al (2006) Adaptive cortical
28. Rocca MA et al (2005) Cortical adaptation in changes and the functional correlates of visuo-
patients with MS: a cross-sectional functional motor integration in relapsing-remitting mul-
MRI study of disease phenotypes. Lancet tiple sclerosis. Brain Res Bull 69(6):597–605
Neurol 4(10):618–626 43. Calautti C, Baron J-C (2003) Functional
29. Pantano P et al (2002) Cortical motor reor- neuroimaging studies of motor recov-
ganization after a single clinical attack of mul- ery after stroke in adults: a review. Stroke
tiple sclerosis. Brain 125(Pt 7):1607–1615 34(6):1553–1566
30. Pantano P et al (2002) Contribution of 44. Ciccarelli O et al (2006) Functional response
corticospinal tract damage to cortical to active and passive ankle movements with
motor reorganization after a single clini- clinical correlations in patients with pri-
cal attack of multiple sclerosis. Neuroimage mary progressive multiple sclerosis. J Neurol
17(4):1837–1843 253(7):882–891
31. Rocca MA et al (2004) Pyramidal tract lesions 45. Rocca MA et al (2010) Preserved brain adap-
and movement-associated cortical recruitment in tive properties in patients with benign mul-
patients with MS. Neuroimage 23(1):141–147 tiple sclerosis. Neurology 74(2):142–149
32. Lowe MJ et al (2002) Multiple sclerosis: 46. Petsas N et al (2013) Evidence of impaired
low-frequency temporal blood oxygen level- brain activity balance after passive sensorimo-
dependent fluctuations indicate reduced func- tor stimulation in multiple sclerosis. PLoS
tional connectivity initial results. Radiology One 8(6):e65315
224(1):184–192 47. Agosta F et al (2008) Tactile-associated
33. Rocca MA et al (2002) Evidence for wide- recruitment of the cervical cord is altered in
spread movement-associated functional MRI patients with multiple sclerosis. Neuroimage
changes in patients with PPMS. Neurology 39(4):1542–1548
58(6):866–872 48. Agosta F et al (2008) Evidence for enhanced
34. Rocca MA et al (2003) Cord damage elic- functional activity of cervical cord in relaps-
its brain functional reorganization after ing multiple sclerosis. Magn Reson Med
a single episode of myelitis. Neurology 59(5):1035–1042
61(8):1078–1085 49. Valsasina P et al (2010) Cervical cord func-
35. Rocca MA et al (2004) A functional MRI tional MRI changes in relapse-onset MS
study of movement-associated cortical patients. J Neurol Neurosurg Psychiatry
changes in patients with Devic’s neuromyeli- 81(4):405–408
tis optica. Neuroimage 21(3):1061–1068 50. Valsasina P et al (2012) Cervical cord fMRI
36. Rocca MA et al (2005) A widespread pat- abnormalities differ between the progressive
tern of cortical activations in patients at forms of multiple sclerosis. Hum Brain Mapp
presentation with clinically isolated symp- 33(9):2072–2080
toms is associated with evolution to definite 51. Rocca MA et al (2012) Abnormal cervical
multiple sclerosis. AJNR Am J Neuroradiol cord function contributes to fatigue in mul-
26(5):1136–1139 tiple sclerosis. Mult Scler 18(11):1552–1559
37. Rocca MA et al (2007) fMRI changes in 52. Staffen W et al (2002) Cognitive function
relapsing-remitting multiple sclerosis patients and fMRI in patients with multiple sclerosis:
complaining of fatigue after IFNbeta-1a evidence for compensatory cortical activa-
injection. Hum Brain Mapp 28(5):373–382 tion during an attention task. Brain 125(Pt
38. Mezzapesa DM et al (2008) Functional corti- 6):1275–1282
cal changes of the sensorimotor network are 53. Au Duong MV et al (2005) Altered func-
associated with clinical recovery in multiple tional connectivity related to white matter
sclerosis. Hum Brain Mapp 29(5):562–573 changes inside the working memory network
39. Rocca MA et al (2008) The “mirror-neu- at the very early stage of MS. J Cereb Blood
ron system” in MS: a 3 tesla fMRI study. Flow Metab 25(10):1245–1253
Neurology 70(4):255–262 54. Au Duong MV et al (2005) Modulation of
40. Rocca MA et al (2007) Altered functional and effective connectivity inside the working mem-
structural connectivities in patients with MS: ory network in patients at the earliest stage of
a 3-T study. Neurology 69(23):2136–2145 multiple sclerosis. Neuroimage 24(2):533–538
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 635

55. Audoin B et al (2003) Compensatory cortical moderately advanced multiple sclerosis. Mult
activation observed by fMRI during a cog- Scler 10(5):549–555
nitive task at the earliest stage of MS. Hum 69. Comi G et al (2001) Clinical and MRI assess-
Brain Mapp 20(2):51–58 ment of brain damage in MS. Neurol Sci
56. Audoin B et al (2005) Magnetic resonance 22(Suppl 2):123–127
study of the influence of tissue damage and 70. Forn C et al (2012) Functional magnetic res-
cortical reorganization on PASAT perfor- onance imaging correlates of cognitive per-
mance at the earliest stage of multiple sclero- formance in patients with a clinically isolated
sis. Hum Brain Mapp 24(3):216–228 syndrome suggestive of multiple sclerosis at
57. Hillary FG et al (2003) An investigation of presentation: an activation and connectivity
working memory rehearsal in multiple scle- study. Mult Scler 18(2):153–163
rosis using fMRI. J Clin Exp Neuropsychol 71. Cerasa A et al (2010) The effects of BDNF
25(7):965–978 Val66Met polymorphism on brain function in
58. Parry AM et al (2003) Potentially adaptive controls and patients with multiple sclerosis:
functional changes in cognitive processing for an imaging genetic study. Behav Brain Res
patients with multiple sclerosis and their acute 207(2):377–386
modulation by rivastigmine. Brain 126(Pt 72. Bobholz JA et al (2006) fMRI study of epi-
12):2750–2760 sodic memory in relapsing-remitting MS: cor-
59. Penner IK et al (2003) Analysis of impairment relation with T2 lesion volume. Neurology
related functional architecture in MS patients 67(9):1640–1645
during performance of different attention 73. Jehna M et al (2011) Cognitively preserved
tasks. J Neurol 250(4):461–472 MS patients demonstrate functional differ-
60. Mainero C et al (2004) fMRI evidence ences in processing neutral and emotional
of brain reorganization during attention faces. Brain Imaging Behav 5(4):241–251
and memory tasks in multiple sclerosis. 74. Rocca MA et al (2012) Differential cerebel-
Neuroimage 21(3):858–867 lar functional interactions during an interfer-
61. Sweet LH et al (2004) Functional magnetic ence task across multiple sclerosis phenotypes.
resonance imaging of working memory among Radiology 265(3):864–873
multiple sclerosis patients. J Neuroimaging 75. Lenzi D et al (2007) Effect of corpus callosum
14(2):150–157 damage on ipsilateral motor activation in patients
62. Sweet LH et al (2006) Functional magnetic with multiple sclerosis: a functional and anatom-
resonance imaging response to increased ical study. Hum Brain Mapp 28(7):636–644
verbal working memory demands among 76. Manson SC et al (2006) Loss of interhemi-
patients with multiple sclerosis. Hum Brain spheric inhibition in patients with multiple
Mapp 27(1):28–36 sclerosis is related to corpus callosum atrophy.
63. Wishart HA et al (2004) Brain activation Exp Brain Res 174(4):728–733
patterns associated with working mem- 77. Ceccarelli A et al (2010) Structural and func-
ory in relapsing-remitting MS. Neurology tional magnetic resonance imaging correlates
62(2):234–238 of motor network dysfunction in primary
64. Chiaravalloti N et al (2005) Cerebral activa- progressive multiple sclerosis. Eur J Neurosci
tion patterns during working memory per- 31(7):1273–1280
formance in multiple sclerosis using FMRI. J 78. Filippi M et al (2002) Functional magnetic
Clin Exp Neuropsychol 27(1):33–54 resonance imaging correlates of fatigue in mul-
65. Cader S et al (2006) Reduced brain functional tiple sclerosis. Neuroimage 15(3):559–567
reserve and altered functional connectivity in 79. Raichle ME, Snyder AZ (2007) A default
patients with multiple sclerosis. Brain 129(Pt mode of brain function: a brief history of
2):527–537 an evolving idea. Neuroimage 37(4):1083–
66. Li Y et al (2004) Differential cerebellar activa- 1090, discussion 1097–1099
tion on functional magnetic resonance imag- 80. Roosendaal SD et al (2010) Resting state net-
ing during working memory performance in works change in clinically isolated syndrome.
persons with multiple sclerosis. Arch Phys Brain 133(Pt 6):1612–1621
Med Rehabil 85(4):635–639 81. Rocca MA et al (2010) Default-mode network
67. Rocca MA et al (2009) Structural and func- dysfunction and cognitive impairment in pro-
tional MRI correlates of Stroop control in gressive MS. Neurology 74(16):1252–1259
benign MS. Hum Brain Mapp 30(1):276–290 82. Bonavita S et al (2011) Distributed changes
68. Lazeron RHC et al (2004) An fMRI study of in default-mode resting-state connectivity in
planning-related brain activity in patients with multiple sclerosis. Mult Scler 17(4):411–422
636 Massimo Filippi and Maria A. Rocca

83. Loitfelder M et al (2012) Abnormalities in a multi-centre fMRI study. Eur J Neurol

of resting state functional connectivity are 15(2):113–122
related to sustained attention deficits in 97. Rocca MA et al (2009) Abnormal connectiv-
MS. PLoS One 7(8):e42862 ity of the sensorimotor network in patients
84. Hawellek DJ et al (2011) Increased functional with MS: a multicenter fMRI study. Hum
connectivity indicates the severity of cognitive Brain Mapp 30(8):2412–2425
impairment in multiple sclerosis. Proc Natl 98. Mainero C et al (2004) Enhanced brain
Acad Sci U S A 108(47):19066–19071 motor activity in patients with MS after a sin-
85. Faivre A et al (2012) Assessing brain connec- gle dose of 3,4-diaminopyridine. Neurology
tivity at rest is clinically relevant in early mul- 62(11):2044–2050
tiple sclerosis. Mult Scler 18(9):1251–1258 99. Cader S, Palace J, Matthews PM (2009)
86. Schoonheim MM et al (2015) Thalamus Cholinergic agonism alters cognitive pro-
structure and function determine severity of cessing and enhances brain functional con-
cognitive impairment in multiple sclerosis. nectivity in patients with multiple sclerosis.
Neurology 84(8):776–783 J Psychopharmacol 23(6):686–696
87. Rocca M et al (2012) Large-scale neu- 100. Tomassini V et al (2012) Relating brain dam-
ronal network dysfunction in relapsing- age to brain plasticity in patients with mul-
remitting multiple sclerosis. Neurology tiple sclerosis. Neurorehabil Neural Repair
79(14):1449–1457 26(6):581–593
88. Sumowski JF et al (2010) Intellectual enrich- 101. Filippi M et al (2012) Effects of cognitive
ment is linked to cerebral efficiency in mul- rehabilitation on structural and functional
tiple sclerosis: functional magnetic resonance MRI measures in multiple sclerosis: an explor-
imaging evidence for cognitive reserve. Brain ative study. Radiology 262(3):932–940
133(Pt 2):362–374 102. Sastre-Garriga J et al (2011) A functional
89. Loitfelder M et al (2011) Reorganization in magnetic resonance proof of concept pilot
cognitive networks with progression of mul- trial of cognitive rehabilitation in multiple
tiple sclerosis: insights from fMRI. Neurology sclerosis. Mult Scler 17(4):457–467
76(6):526–533 103. Cerasa A et al (2013) Computer-assisted cog-
90. Tortorella C et al (2013) Load-dependent nitive rehabilitation of attention deficits for
dysfunction of the putamen during atten- multiple sclerosis: a randomized trial with
tional processing in patients with clinically fMRI correlates. Neurorehabil Neural Repair
isolated syndrome suggestive of multiple scle- 27(4):284–295
rosis. Mult Scler 19(9):1153–1160 104. Parisi L et al (2014) Changes of brain resting
91. Amann M et al (2011) Altered functional state functional connectivity predict the per-
adaptation to attention and working memory sistence of cognitive rehabilitation effects in
tasks with increasing complexity in relapsing- patients with multiple sclerosis. Mult Scler
remitting multiple sclerosis patients. Hum 20(6):686–694
Brain Mapp 32(10):1704–1719 105. Liu Y et al (2011) Abnormal baseline brain
92. Rocca MA et al (2014) Functional correlates activity in patients with neuromyelitis optica:
of cognitive dysfunction in multiple sclerosis: a resting-state fMRI study. Eur J Radiol
a multicenter fMRI Study. Hum Brain Mapp 80(2):407–411
35(12):5799–5814 106. Mikulis DJ et al (2002) Adaptation in the
93. Pantano P et al (2005) A longitudinal fMRI motor cortex following cervical spinal cord
study on motor activity in patients with mul- injury. Neurology 58(5):794–801
tiple sclerosis. Brain 128(Pt 9):2146–2153 107. Curt A et al (2002) Changes of non-affected
94. Audoin B et al (2008) Efficiency of cognitive upper limb cortical representation in paraple-
control recruitment in the very early stage of gic patients as assessed by fMRI. Brain 125(Pt
multiple sclerosis: a one-year fMRI follow-up 11):2567–2578
study. Mult Scler 14(6):786–792 108. Sabbah P et al (2002) Sensorimotor cortical
95. Loitfelder M et al (2014) Brain activity activity in patients with complete spinal cord
changes in cognitive networks in relapsing- injury: a functional magnetic resonance imag-
remitting multiple sclerosis—insights from ing study. J Neurotrauma 19(1):53–60
a longitudinal FMRI study. PLoS One 109. Cramer SC et al (2001) Changes in motor
9(4):e93715 cortex activation after recovery from spinal
96. Wegner C et al (2008) Relating functional cord inflammation. Mult Scler 7(6):364–370
changes during hand movement to clinical 110. Rocca MA et al (2006) The level of spinal
parameters in patients with multiple sclerosis cord involvement influences the pattern of
 Application of fMRI to Multiple Sclerosis and Other White Matter Disorders 637

movement-associated cortical recruitment in 119. Burstein R et al (2010) Thalamic sensitiza-

patients with isolated myelitis. Neuroimage tion transforms localized pain into widespread
30(3):879–884 allodynia. Ann Neurol 68(1):81–91
111. Rocca MA et al (2006) An fMRI study of 120. Maizels M, Aurora S, Heinricher M (2012)
the motor system in patients with neuro- Beyond neurovascular: migraine as a dysfunc-
psychiatric systemic lupus erythematosus. tional neurolimbic pain network. Headache
Neuroimage 30(2):478–484 52(10):1553–1565
112. Hadjikhani N et al (2001) Mechanisms of 121. Chiapparini L et al (2010) Neuroimaging in
migraine aura revealed by functional MRI in chronic migraine. Neurol Sci 31(Suppl 1):S19–S22
human visual cortex. Proc Natl Acad Sci U S 122. Bhaskar S et al (2013) Recent progress in
A 98(8):4687–4692 migraine pathophysiology: role of cortical
113. Cao Y et al (1999) Functional MRI-BOLD spreading depression and magnetic resonance
of visually triggered headache in patients with imaging. Eur J Neurosci 38(11):3540–3551
migraine. Arch Neurol 56(5):548–554 123. Antal A et al (2011) Differential activation
114. Lakhan SE, Avramut M, Tepper SJ (2013) of the middle-temporal complex to visual
Structural and functional neuroimaging in stimulation in migraineurs. Cephalalgia
migraine: insights from 3 decades of research. 31(3):338–345
Headache 53(1):46–66 124. Furman JM, Marcus DA, Balaban CD (2013)
115. Stankewitz A et al (2011) Trigeminal noci- Vestibular migraine: clinical aspects and patho-
ceptive transmission in migraineurs predicts physiology. Lancet Neurol 12(7):706–715
migraine attacks. J Neurosci 31(6):1937–1943 125. Sprenger T, Magon S (2013) Can functional
116. Tedeschi G et al (2013) The role of BOLD- magnetic resonance imaging at rest shed
fMRI in elucidating migraine pathophysiol- light on the pathophysiology of migraine?
ogy. Neurol Sci 34(Suppl 1):S47–S50 Headache 53(5):723–725
117. Russo A et al (2012) Pain processing in 126. Mainero C, Boshyan J, Hadjikhani N (2011)
patients with migraine: an event-related fMRI Altered functional magnetic resonance imaging
study during trigeminal nociceptive stimula- resting-state connectivity in periaqueductal gray
tion. J Neurol 259(9):1903–1912 networks in migraine. Ann Neurol 70(5):838–845
118. Sprenger T, Borsook D (2012) Migraine 127. Tessitore A et al (2013) Disrupted default
changes the brain: neuroimaging makes its mode network connectivity in migraine with-
mark. Curr Opin Neurol 25(3):252–262 out aura. J Headache Pain 14(1):89
 Chapter 21

fMRI in Cerebrovascular Disorders

Nick S. Ward

Abstract
Stroke is a major cause of long-term disability worldwide. One of the key factors underpinning recovery of
function is reorganization of surviving neural networks. Noninvasive techniques such as fMRI allow this
reorganization to be studied in humans. However, the design of experiments involving patients with impair-
ment requires careful consideration and is often constrained. Difficulty with some tasks can lead to a number
of performance confounds, and so tasks and task parameters that avoid or minimize this should be selected.
Furthermore, when studying patients with cerebrovascular disease, it is important to consider the possibility
that the blood oxygen level-dependent signal may be altered and affect interpretation of results. Despite these
potential problems, careful experimental design can provide real insights into system-level reorganization
after stroke and how it is related to functional recovery. Currently, results suggest that functionally relevant
reorganization does occur in cerebral networks in human stroke patients. For example, it is apparent that
initial attempts to move a paretic limb following stroke are associated with widespread activity within the
distributed motor system in both cerebral hemispheres. This reliance on nonprimary motor output pathways
is unlikely to support full recovery, but improved efficiency of the surviving networks is associated with
behavioral gains. This reorganization can only occur in structurally and functionally intact brain regions.
Understanding the dynamic process of system-level reorganization will allow greater understanding of the
mechanisms of recovery and potentially improve our ability to deliver effective restorative therapy.

Key words fMRI, Stroke, Blood oxygen level-dependent, Motor cortex, Premotor cortex, Plasticity,
Rehabilitation

1 Introduction

Studying patients who have suffered from stroke with functional brain
imaging is difficult for a variety of reasons. The motivation behind
such studies is a desire to understand and subsequently improve the
process of functional recovery. Stroke and other forms of neurological
damage account for nearly half of all severely disabled adults [1–3].
Longitudinal studies of recovery suggest that only 50 % of stroke sur-
vivors with significant initial upper limb paresis recover useful function
of the limb [4]. Furthermore, those with poor recovery of arm func-
tion have dramatically impaired quality of life and sense of well-being
[5, 6]. It is clear that effective treatment of motor impairment after
stroke is critically important to many people.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_21, © Springer Science+Business Media New York 2016

639
640 Nick S. Ward

The mainstay of treatment is neurorehabilitation. The overall

approach is effective and the benefit of strategies aimed at helping
patients adapt to impairment well proven [7]. Treatments aimed at
reducing impairment, remain poorly developed. There is an over-
riding assumption that one way to tackle impairment in those
patients with focal brain damage is to attempt to promote func-
tionally relevant reorganization within surviving neural networks
[8]. Over the last decade, advances in our understanding of how
the normal brain is organized at the molecular, cellular, and sys-
tems level have improved enormously. Advances in our under-
standing of the mechanisms of impairment after brain injury,
including stroke, are way behind. Translating findings from proof-
of-principle studies into real treatments remains problematic and
requires urgent attention [9]. Thus, the clinical neurosciences have
the potential to make a unique contribution toward developing
rehabilitation strategies designed to reduce impairment.
The tools available for studying the working human brain are
different to those used in animal models. In human subjects,
experiments are performed at the level of neural systems rather
than single cells or molecules. This chapter will concentrate on the
way that fMRI can be used to contribute. The first half will con-
sider the specific difficulties involved in performing fMRI experi-
ments in stroke patients, in particular how fMRI signals might be
affected in cerebrovascular disease and how studying patients with
impairment should influence experimental design. In the second
half, examples of studies using fMRI in stroke patients will be dis-
cussed to illustrate advances in our understanding of post-stroke
functional brain reorganization. Many studies have been per-
formed in the somatosensory [10–12] and language systems [13–
15], but studies of the motor system are particularly numerous and
will be used to illustrate how fMRI may be used.

2 Blood Oxygen Level-Dependent Signal in Cerebrovascular Disease

fMRI relies on the blood oxygen level-dependent (BOLD) signal.

In brief, the BOLD signal relies on the close coupling between
blood flow and metabolism. During an increase in neuronal activa-
tion there is an increase in local cerebral blood flow, but only a
small proportion of the greater amount of oxygen delivered locally
to the tissue is used. This results in a net increase in the tissue con-
centration of oxyhemoglobin and a net reduction in paramagnetic
deoxyhemoglobin in the local capillary bed. The magnetic proper-
ties of hemoglobin depend on its level of oxygenation so that this
change results in an increase in local tissue-derived signal intensity
on T2*-weighted MR images [16].
The mechanism of neurovascular signaling to the blood vessels
controlling cerebral blood flow is still unclear, although it may
 fMRI in Cerebrovascular Disorders 641

involve metabolic [17, 18] or neurochemical [19, 20] mechanisms.

In addition, the generation of BOLD signal is still reliant on venous
blood volume, blood flow, blood oxygenation, and oxygen con-
sumption. Thus, it is possible that any disease state that changes
these parameters will potentially modify the BOLD signal. It is
therefore legitimate to be concerned whether the BOLD signal is
reliable in patients who have suffered stroke and in subjects with
evidence of both large and small vessel atherosclerosis. The poten-
tial problem arises because in general, the BOLD signal is assumed
to have the same shape in all subjects and in all brain regions. In one
case, the canonical hemodynamic response function (HRF) has
been derived by principle component analysis of empirical data with
a peak magnitude occurring 6 s after the neuronal activity [21].
There is evidence to suggest that the shape of the hemody-
namic response might be altered after stroke. Newton et al. [22]
demonstrated a greater time to peak BOLD response in ipsilateral
compared with contralateral primary motor cortex (M1) in con-
trols. In three chronic stroke patients, the time to peak BOLD
response was increased in ipsilesional (contralateral) M1 compared
with controls. Interestingly, in these patients the time to peak
BOLD response in contralesional M1 was equivalent or less than
that for ipsilesional M1, representing a finding opposite to that
seen in healthy controls. Pineiro et al. [23] have also described a
slower time to peak BOLD response in sensorimotor cortex bilat-
erally in 12 chronic stroke patients with lacunar infarcts. Thus,
modeling the BOLD response with a canonical HRF might be less
efficient in stroke patients. It is worth considering what the effect
of this would be in the context of a standard functional imaging
analysis using the general linear model approach. If the canonical
HRF was a poor fit for the actual response, then the residual error
of the analytical model would be greater (than if the fit was good),
thus lowering t- and Z-scores and depressing sensitivity to detec-
tion of differences. In fact, in general, most studies of stroke
patients have found increased activation in a number of brain
regions over and above healthy controls, so it might be the case
that these overactivations have been underestimated. However,
modeling differences in measured HRF from the canonical is likely
to be beneficial. The use of temporal and dispersion derivatives of
the canonical HRF to specifically capture differences in the timing
or duration of the peak response, for example, is likely to increase
sensitivity.
In addition to changes in the shape of the HRF, there is evi-
dence that in patients with impaired cerebrovascular reserve or
advanced narrowing of the cerebral arteries, the BOLD fMRI sig-
nal may be reduced, or even become negative [24–26]. Röther
et al. [27] describe a single patient, which illustrates the point. The
patient was found to have bilateral occluded internal carotid arter-
ies and an occluded vertebral artery. The cerebrovascular reactivity,
as determined by reduced change in T2* signal during
642 Nick S. Ward

hypercapnia, was severely impaired in the left hemisphere. The

finding of importance was that during a motor task with the right
hand, the BOLD response in the left motor cortex was negative for
the duration of the task. This suggests that the initial dip in BOLD
signal due to a relative decrease in oxyhemoglobin was not fol-
lowed by the normal vascular response (which would have resulted
in an increase in oxygenated and decrease in deoxygenated hemo-
globin). This subject had previously suffered from a transient isch-
emic attack involving the right arm. It is likely that these symptoms
were related to hemodynamic insufficiency, and it is interesting to
speculate that the presence of a prolonged negative dip in BOLD
signal represents a marker for those at risk from such symptoms.
Others have made the point that this impaired cerebrovascular
reactivity might be due to either large or small vessel disease [26].
Further investigation will reveal whether this idea has genuine
potential as a clinical tool.
Several studies have now suggested that impairment of normal
vasodilatation in response to hypercapnia is associated with dimin-
ished magnitude of BOLD signal [25, 26, 28, 29]. Thus, in patients
with severely impaired cerebrovascular reactivity neuronal activation
may not translate into a BOLD response in the conventional sense,
and standard models using the canonical HRF may not be sufficient.
However, the scale of the problem is not yet clear. For exam-
ple, the cerebrovascular reactivity in the right hemisphere of the
patient studied by Röther et al. [27] was moderately impaired and
the BOLD response during a motor task with the left hand was
entirely normal. Patients with hemodynamic symptoms are rare,
and are likely to be excluded from fMRI studies. In addition,
patients with severe stenosis of ipsilesional internal carotid arteries
are usually also excluded, although it is not clear that this is neces-
sary. It may also be the case that small vessel disease may also make
a significant contribution to impaired cerebrovascular reactivity.
Thus, although there is evidence that impaired cerebrovascular
reactivity can diminish the BOLD response, there is no evidence
that the BOLD signal is erroneously detected in these patients, i.e.,
this is largely a problem of false negative results. In general, the lit-
erature concerning differences between stroke patients and healthy
controls is dominated by the finding of overactivity in patients com-
pared with controls, and once again, it is possible that the issue of
cerebrovascular disease has led to an underestimation of changes in
cortical organization after stroke. It is clear that when examining for
differences between a group of patients and a group of healthy con-
trols, the nonneural factors that can influence the BOLD response
will contribute significantly to whether a difference is found or not.
However, several studies have begun to use a correlation approach.
That is to say, to attempt to explain variability in the task-related
BOLD signal with some other parameters, such as a measure of
recovery [30] or corticospinal tract integrity [31, 32]. It is unlikely
 fMRI in Cerebrovascular Disorders 643

that changes in cerebrovascular responsiveness will correlate with

recovery or an anatomical measure of corticospinal tract integrity,
and so it is unlikely to account for any significant (and biologically
plausible) results. As we have already discussed, however, the ability
to detect a real finding is likely to be diminished by alterations in
nonneural contributors to the BOLD signal. A multimodal approach
using different imaging techniques (BOLD, perfusion, hypercapnic
challenge) and concurrent neurophysiological methods (electroen-
cephalography [EEG], magnetoencephalography [MEG], transcra-
nial magnetic stimulation [TMS]) may be useful when addressing
the influence of multiple physiological variables. These issues will
require further empirical study.
The discussion regarding differences in hemodynamic coupling
is also of relevance when considering the effects of age, given that
stroke is commoner with advancing age, and that often age-matched
controls are used in studies of stroke patients. D’Esposito et al. [33]
examined the effects of age on the BOLD signal generated during a
button press task in response to a visual cue, using a sparse event-
related design. Using a standard fixed effects analysis, task-related
activation was detected in M1 above the chosen threshold in only
75 % of the older subjects but 100 % of the younger subjects.
Furthermore, for those in whom activation was detected, four times
the number of suprathreshold voxels was present in the younger
compared with the older subjects. Thus, on the face of it, M1 appears
to be less active during a button press task in older subjects. However,
the most important finding was in relation to the signal-to-noise
ratio (SNR), which was reduced in elderly subjects. Results from
single subject or group fixed effects analyses of functional imaging
data are generally presented as t statistics for each voxel (volume ele-
ment) of the brain. The result is therefore dependent on both the
magnitude of the signal change and the residual variance after this
has been accounted for. Thus, an increased SNR will lead to a lower
t statistic, and therefore fewer suprathreshold voxels. In fact,
D’Esposito et al. [33] found no difference in the magnitude of task-
related signal change in M1, supporting the notion that the dimin-
ished number of suprathreshold voxels was largely attributable to
the decreased SNR in the older subjects.
The problem of reduced SNR can be effectively dealt with by
employing the statistical technique of random effects analysis as
opposed to fixed effects analysis. Random effects analysis of func-
tional imaging data treats each subject as a random variable. The
experimental variance is dominated by between subjects variability
(as opposed to within subject variability in the case of fixed effects
models). The data for each subject comprise the voxel-wise param-
eter estimate for the task under consideration, which reflects the
magnitude of the signal change in each voxel. Appropriate statistics
can be performed on these data, which are less likely to be influ-
enced by differences in SNR [33]. Using a random effects analysis,
644 Nick S. Ward

and employing both temporal and dispersion derivatives of the

HRF, Ward et al. [34] demonstrated no change in the shape of the
hemodynamic response during a hand grip task with advancing
age, in keeping with the findings described earlier.

3 Issues in Experimental Design

The results from any functional imaging study are only as reliable
as the care with which the experiment is constructed and executed,
but studies involving patients who have had a stroke raise some
specific issues. The selection of patients, choice of experimental
paradigm, within scanner monitoring of performance, and the
approaches to data analysis all require careful consideration.

3.1 Subject Selection In general, stroke patients are a heterogeneous group differing in
several important ways, not least the site and size of infarct, patency
of the vascular system, age, comorbidities, and concurrent medica-
tion. The criteria for patient selection will to an extent depend on
the experimental question. It is unlikely that averaging the results
from a wide variety of patient types will prove useful because of this
variability, but there are two other ways of approaching the experi-
mental design. First, it may be desirable to use a group of patients
highly selected on the basis of lesion location, for example. Results
from this type of controlled study are powerful, but do not gener-
alize outside the subgroup selected. Alternatively, it might be more
useful to study a group of patients who vary in a specific factor of
interest (e.g., outcome), to explore the relationship between this
factor and task-related brain activity. Results from studies using
this approach can be generalized more easily.

3.2 Performance The choice of experimental task is critical and is dependent on the
Confounds experimental question. For example, a study of the relationship
between brain activation and outcome after stroke will by necessity
involve patients with different performance abilities. Similarly, a lon-
gitudinal study will require that patients are studied at different stages
of recovery. For an active motor or language task this can result in the
problem of performance confounds, because the ability to perform
the task is not the same across patients or sessions. A change in exper-
imental task performance can have significant effects upon the pat-
tern of brain activation. In other words, comparison across patients
or time points is made difficult if the patients are performing the task
differently. Thus, each patient must perform the same task during the
fMRI experiment, so that a meaningful comparison can be made
across subjects or scanning sessions. Maintaining a consistent task is
therefore of great importance, but in stroke recovery studies equality
of task may be interpreted in a number of ways. In particular, a task
may be consistent across patients with different abilities in terms of
absolute or relative parameters.
 fMRI in Cerebrovascular Disorders 645

This can be illustrated by considering a simple motor experiment.

The absolute task parameters can be fixed (by setting the same tar-
get force and rate for each subject or session), but performing a task
may be experienced as more or less effortful depending on the level
of recovery. Consequently, any differences in results between sub-
jects or sessions could be attributed to differences in “effort”
exerted. Alternatively, the relative task parameters (i.e., the level of
task difficulty) can be fixed across subjects/sessions. In this sce-
nario, patients will perform the task at different absolute forces and
rates, and so differences in results across subjects/sessions could be
attributed to differences in the absolute task parameters. When
using an “active task,” these factors must always be considered and
results interpreted with these confounds in mind. Increased effort is
a potentially useful strategy for overcoming motor, language, or
cognitive impairment in a real world setting. As described earlier,
some patients may use less effortful as their performance improves.
Is the focus of interest the reorganization that might be the sub-
strate for recovery, or is it the strategy that each patient uses to
perform a task to a certain level, given the constraints of their
impairment? Both may be of interest, but the choice of experimen-
tal design has an impact on which process is being studied. The
problem of performance confounds is avoided with passive tasks
(e.g., passive limb movements and passive listening), but these are
complementary approaches to active tasks, not substitutes for them.

3.3 Task Frequency The rate of task performance is also something that has implications
for both data analysis and interpretation of results. Consider once
again a simple motor task such as finger tapping or hand squeezing.
The rate of performance of a repetitive task will influence how
effortful the task is, in the same way as the target force. Most experi-
ments are conducted in a “block design”; that is to say a period of
activity (usually for 16–30 s) followed by a period of rest. If subjects
are asked to perform at the same rate (even if the target force is
scaled according to each subject’s own performance abilities), for
example, finger tapping at 1 Hz for 20 s, then differences across
subjects/sessions could be due to differences in perceived effort,
just as with equal absolute target forces. Some investigators have
varied the rate at which subjects are asked to perform a task to try
to control for effort exerted. However, comparing blocks with dif-
ferent numbers of “events” within them is problematic because the
BOLD signal summates depending on how many events there are.
The BOLD response needs approximately 10 s and longer to return
to baseline, but “events” are usually more frequent than this (e.g.,
1 Hz finger tapping). It is usually assumed that there is a summa-
tion of the overlapping BOLD responses, which is largely (but not
entirely) linear [35, 36]. The basis function (boxcar design) in the
general linear model will have the same “height,” and so more fre-
quent events will result in a larger parameter estimate, for the same
amount of event-related activity. In fact, it is the quantity of events,
646 Nick S. Ward

not their frequency, which will modulate BOLD signal in motor

cortices [37]. Thus, if a subject performs the task at 1 Hz and then
at 0.5 Hz in both cases for 20 s, and each period is modeled with
the same boxcar basis function, then the parameter estimate (or
magnitude of activity) will appear to be roughly twice as much dur-
ing the more frequent task. This reflects the quantity of “events,”
but not a change in the way the brain is organized. Thus, if patients
perform a task more slowly than healthy controls to control for
effort, then the brain activity associated with that movement will be
underestimated in comparison to those subjects who perform the
task at a faster rate. One way around this problem is to use an event-
related design in which the intertrial interval is long enough for the
task to be performed repetitively without increasing the sense of
effort. This design may be less efficient in terms of fMRI design,
but avoids the confounds described earlier [31, 32].

3.4 Task Complexity Investigators are often tempted to use more complex tasks when
studying patients with impairment in the hope that this will maxi-
mize differences between patients and control subjects. This is
sometimes done in the hope of exploring a more ecologically valid
task, i.e., one which is relevant to function in the real world.
However, it is never possible to study the neural correlates of a task
that a subject cannot themselves perform. By introducing more
complexity into the task, patients with significant impairment are
more likely to adopt new operational strategies toward these
experimental tasks in an attempt to adapt to their impairment.
These differences in strategy could therefore account for differ-
ences between subject groups. Although of clinical interest, differ-
ences in strategy across a group represent a potential experimental
confound if they are unexpected and not measured. One approach
is therefore to use a simple task that minimizes difference in strate-
gic approach to the task so that valid comparisons can then be
made across subjects/sessions.

3.5 Task Monitoring Once a paradigm has been selected it is important that task perfor-
mance is monitored during the experiment. Intersubject variability
may be greater after stroke and new sources of variability can arise,
such as mirror or associated movements. To take account of this,
some investigators record behavior during a prescan rehearsal,
whilst others incorporate the increasingly available instrumentation
that is compatible with the MRI setting. Prescan rehearsal provides
some idea of whether a task can be performed correctly, or whether
mirror movements are present, for example. However, in-scanner
recordings allow this information to be incorporated into image
analysis as a covariate, and thus improving statistical power by
accounting for correlated variance in the measured scan signal.
The experimental approach is therefore dictated by the experi-
mental question. Not all investigators will have the same question,
 fMRI in Cerebrovascular Disorders 647

but the issues discussed earlier need to be considered in all cases. For
most questions, this approach is entirely appropriate and standard-
ization of experimental paradigms, patient selection, and method of
analysis across experiments is not required. In the case of experimen-
tal questions that require a multicenter approach that is technically
feasible, standardization of such factors would be required.

4 Reorganization in the Motor System After Stroke

4.1 Residual Early studies of motor system organization after stroke compared
Functional brain activation during movement in well-recovered patients and
Architecture After normal controls. Early group studies of stroke patients with sub-
Stroke: The Story cortical lesions described greater activation within a number of
So Far? motor-related cortical regions compared with controls during a
finger tapping task [38–42]. It was suggested that nonprimary (or
secondary) cortical motor regions were thus responsible for recov-
ery of motor function in these patients. Strick [43] had proposed
this as a potential mechanism of restoration of function, some years
before based on an understanding of the organization of the corti-
cal motor system in primates. Normal distal motor function is facil-
itated largely through the corticospinal pathway, from the cortical
motor system to the spinal cord motor neurons. The majority of
corticospinal fibers originate in the M1, but there are contribu-
tions from other cortical regions [44]. In primates, the M1, arcu-
ate (or lateral) premotor cortex (PM), and supplementary motor
area (SMA) are each part of parallel, independent motor networks
with (1) separate projections to spinal cord motoneuron and (2)
interactions at the level of the cortex [43]. There is some similarity
between the corticospinal projections from the hand regions of
M1, PM, and SMA. Thus, it seemed feasible that a number of
motor networks acting in parallel could generate an output to the
spinal cord necessary for movement, and that damage in one of
these networks could be at least partially compensated for by activ-
ity in another [45, 46]. Subsequently, many studies have demon-
strated that the performance of a simple motor task with the
affected limb is associated with greater bilateral brain activation in
a number of cortical motor-related areas compared with healthy
volunteers, including dorsal PM (PMd) and ventral PM (PMv),
SMA, and cingulate motor areas (CMA) [23, 38–42, 47–52].
A critical question is whether these differences are related to
recovery. As discussed previously in this chapter, this question
requires that the group of patients examined have a wide variety of
outcomes, or else longitudinal studies should be performed. In the
first such cross-sectional study, a group of chronic stroke patients
with infarcts sparing M1 were scanned during a hand grip with
visual feedback task using fMRI [30]. The target forces used were
always a proportion of each subject’s own maximum grip force, so
648 Nick S. Ward

Fig. 1 Brain regions in which there is a negative correlation between corticospinal system integrity (as assessed
with transcranial magnetic stimulation) and task-related signal change during hand grip with the affected
hand. Increasing task-related activity is seen in a number of secondary motor areas including premotor regions
and supplementary motor area as damage to the corticospinal system increases. The affected hand was on
the left side. Results are displayed on a “glass brain” shown from the right side (top left image), from behind
(top right image), and from above (bottom left image). Voxels are significant at P < 0.001 (uncorrected), and
clusters are significant at P < 0.05 (corrected) (Reproduced from ref. [31], Oxford University Press.)

that any differences were unlikely to be due to differences in effort.

The more affected patients had greater task-related activity in sec-
ondary motor regions in both affected and unaffected hemispheres,
whereas patients with the best motor scores had activation patterns
that were indistinguishable from healthy age-matched volunteers.
A similar result was observed in a group of patients studied at
approximately 10 days post-stroke [53]. It was hypothesized that,
secondary motor areas are recruited in response to damage to cor-
ticospinal output. A subsequent study demonstrated a strong posi-
tive correlation between secondary motor area recruitment in both
hemispheres and corticospinal system damage (Fig. 1) [31]. A
more “normal” corticospinal system was associated with greater
task-related activity in contralesional M1 (hand area), suggesting a
progressive shift away from primary to secondary motor areas with
increasing disruption to corticospinal system. This has also been
 fMRI in Cerebrovascular Disorders 649

described as an increase in either bilateral or contralesional activity,

but the exact pattern is likely to depend on the anatomy of the
damage. Furthermore, several secondary motor areas have bilateral
projections to motor output systems [54, 55]. Thus, after stroke,
the brain will use what is available (i.e., what is intact and con-
nected so that motor output can be influenced) in an attempt to
generate motor output to spinal cord motoneurons.
These results do not immediately support the idea that second-
ary motor areas are the substrate for motor recovery. Labeling corti-
cospinal neurons with retrograde tracers has revealed multiple
nonprimary corticospinal output zones in both the lateral and the
medial areas of the frontal lobe (SMA, CMA, PMd, and PMv) [56–
58]. These output zones contain large numbers of corticospinal
neurons that project to the intermediate zone and ventral horn of
the spinal cord suggesting their potential for direct control of spinal
motoneurons in a way paralleling corticospinal output from M1
[45]. In primates, however, projections from secondary motor areas
to spinal cord motor neurons are less numerous and less efficient at
exciting spinal cord motoneurons than those from M1 [59, 60].
Moreover, unlike M1, facilitation of distal muscles from SMA, PMd,
and PMv is not significantly stronger than facilitation of proximal
muscles. These brain regions probably exert their influence via indi-
rect descending motor pathways such as the reticulospinal or rubro-
spinal pathways [61]. These pathways often have bihemispheric
origins, hence bihemispheric task-related activity is more common
after stroke. Furthermore, they are also more likely to supply the
upper limb flexors, hence flexor synergistic patterns of movement in
patients who are reliant on these indirect rather than direct path-
ways. Alternatively, or possibly in addition, cortico-cortical interac-
tions, presumably with surviving M1 output, may also play an
important role in supporting recovered motor function.
What is the evidence for the idea that the secondary motor
areas of both hemispheres are contributing to recovered function?
There are two ways to investigate the functional relevance of sec-
ondary motor region recruitment. One is to measure how task-
related activity co-varies with modulation of task parameters. In
healthy humans, for example, increasing force production is associ-
ated with linear increases in BOLD signal in contralateral M1 and
medial motor regions, implying that they have a functional role in
force production [62–64]. A recent study examined specifically for
regional changes in the control of force modulation after stroke
[32]. In patients with greater corticospinal system damage, force-
related signal changes were seen mainly in contralesional dorsolat-
eral PM, bilateral ventrolateral premotor cortices, and contralesional
cerebellum, but not ipsilesional M1 (Fig. 2).
Thus, not only do premotor cortices become increasingly active
as corticospinal system integrity diminishes [31], but they can take
on a new “M1-like” role during modulation of force output, which
implies a new and functionally relevant role in motor control.
650 Nick S. Ward

Force modulation with Force modulation with increasing

intact CST CST damage

contralesional PMd contralesional cerebellum (Cr I)

ipsilesional M1

contralesional PMv ipsilesional PMv

Fig. 2 Brain regions in which the blood oxygen level-dependent (BOLD) signal varies linearly with force exerted
during hand grip change as a function of corticospinal tract (CST) integrity (as assessed with transcranial
magnetic stimulation). The affected hand was on the left side. In the left panel, increasing force leads to
greater modulation of BOLD signal in ipsilesional M1 in patients with less damage to CST. In the right panel,
increasing force leads to greater modulation of BOLD signal in contralesional dorsal premotor cortex, contral-
esional cerebellum, contralesional, and ipsilesional ventral premotor cortex. This demonstrates that brain
regions involved in force modulation shift away from primary motor cortex to premotor regions with increasing
CST damage. Results are overlaid onto the average T1-weighted structural scan obtained from all stroke
patients in the study (Adapted from ref. [32], Blackwell Publishing.)

Second, experiments in which premotor activity is transiently

disrupted with TMS can lead to worsening of recovered motor
behaviors in patients with no effect on the performance of control
subjects [51, 65, 66], again implying new and functionally relevant
roles. Furthermore, TMS to contralesional PMd is more disruptive
in patients with greater impairment [51], whereas TMS to ipsile-
sional PMd is more disruptive in less impaired patients [65] in
keeping with a general shift toward functionally relevant activity in
the contralesional hemisphere of patients with greater damage to
motor output pathways.
These results are important because they tell us that the
response to focal injury does not involve simple substitution of one
cortical region for another. It is clear that nodes within remaining
motor networks can take on new functional roles. The contribution
of contralesional M1 to recovery is surprisingly unresolved. An
early view was that the contralesional M1 might be viewed much
like an extra secondary motor area, contributing what descending
signals it could. An alternative view suggested that contralesional
M1 impairs motor recovery through excessive inhibitory drive to
 fMRI in Cerebrovascular Disorders 651

ipsilesional M1 [8]. The more mundane truth is likely to be that

the role played by contralesional M1, secondary motor areas and
indeed ipsilesional M1 is likely to depend on a number of factors,
most importantly the profile of anatomical damage and the actual
task being performed.

4.2 Connectivity Subsequent studies have turned to examining how connectivity

Based Analysis between nodes in the motor network is affected by stroke [67].
of Post-stroke Motor Techniques such as Dynamic Causal Modelling have examined
Networks task-related networks, whereas other approaches have looked at
“large-scale” whole brain networks (using for example, graph-
theory). In general, both approaches have followed the same theme
in looking for differences between patients and healthy controls as
well as testing for correlations between brain imaging measures,
particularly interhemispheric connectivity, and clinical scores. Most
findings at rest point to lower connectivity between motor cortices
in patients with more impairment [68] and greater corticospinal
tract damage [69]. During movement of the affected hand the
influence of contralesional to ipsilesional M1 is more inhibitory,
but once again, only in more impaired patients [70]. Even the
interhemispheric influence of ipsilesional dorsolateral premotor
cortex is different depending on the level of motor impairment
(itself dependent on residual structural anatomy) [71].
In summary, in the chronic stroke brain, there is a reconfigura-
tion of the cerebral motor system. Task-related brain activation
varies across chronic stroke patients in a way that appears to be
predictable. It is important to stress that this reorganization is
often not successful in returning motor function to normal. It is
less effective than that in the intact brain but will nevertheless
attempt to generate some form of motor signal to spinal cord
motoneurons in the most efficient way. The exact configuration of
this new motor system will be determined most obviously by the
extent of the anatomic damage. This includes the extent to which
the damage affects cortical motor regions, white matter pathways,
and even which hemisphere is affected [72]. The more of the nor-
mal functional architecture that survives, the greater will be the
potential for full recovery. In patients with damage to primary sen-
sorimotor cortex, for example, tests of fractionated finger move-
ment correlated more strongly with the proportion of surviving
“normal” sensorimotor cortex (as defined by functional activation
maps in normal controls) than with total infarct volume [73].
This anatomic explanation accounts for why some patients do
better than others, but it does not account for the recovery of
function that occurs over weeks and months in individual patients.
How does the reorganized state evolve? Longitudinal fMRI studies
have shed further light on the process [47, 50, 74–76], although
only a handful have studied patients on more than two occasions.
One study scanned subcortical stroke patients on average eight
times over 6 months after stroke [76] and demonstrated an early
652 Nick S. Ward

overactivation in primary and many nonprimary motor regions.

Thereafter, functional recovery was associated with a focusing of
task-related brain activation patterns toward a “normal” lateralized
pattern. In general, longitudinal studies have demonstrated a
focusing of activity toward the lesioned hemisphere motor regions
that is associated with improvement in motor function [47, 74],
with some patients showing persistent recruitment [50].

5 Conclusions

In summary, the brain activation pattern of an individual patient

represents the state of reorganization within that system at the
time of study. This pattern is highly influenced by a number of
methodological factors as previously discussed. However, in appro-
priately controlled experiments, these activation patterns tell us
something about how that brain is functionally organized.
Functional improvement with treatment is likely to be associated
with changes within this network. The potential for functionally
relevant change to occur will depend on a number of other factors,
not least the biologic age of the subject and the premorbid state of
their brain, but also current drug treatments. Furthermore, levels
of neurotransmitters and growth factors that are able to influence
the ability of the brain to respond to afferent input (i.e., how plas-
tic it is) might be determined by their genetic status [77]. All of
these factors will influence the potential for activity driven change
within the intact motor networks, the putative mechanism of ther-
apy driven improvements in motor performance.

References
1. Hoffman C, Rice D, Sung HY (1996) Persons being one year after stroke. Clin Rehabil
with chronic conditions. Their prevalence and 11(2):139–145
costs. JAMA 276(18):1473–1479 7. Stroke Unit Trialists’ Collaboration (2000)
2. Office of Population Censuses and Surveys Organised inpatient (stroke unit) care for
(1988) OPCS surveys of disability in Great stroke (Cochrane Review). The Cochrane
Britain. I. The prevalence of disability among Library, Issue 2. Oxford: Update Software
adults. HMSO, London 8. Ward NS, Cohen LG (2004) Mechanisms
3. Wade DT, Hewer RL (1987) Epidemiology of underlying recovery of motor function after
some neurological diseases with special refer- stroke. Arch Neurol 61(12):1844–1848
ence to work load on the NHS. Int Rehabil 9. The Academy of Medical Sciences (2004)
Med 8(3):129–137 Restoring neurological function: putting the
4. Wade DT (1989) Measuring arm impairment neurosciences to work in neurorehabilitation.
and disability after stroke. Int Disabil Stud Academy of Medical Sciences, London
11(2):89–92 10. Loubinoux I, Carel C, Pariente J, Dechaumont
5. Nichols-Larsen DS, Clark PC, Zeringue A, S, Albucher JF, Marque P et al (2003)
Greenspan A, Blanton S (2005) Factors influ- Correlation between cerebral reorganization
encing stroke survivors’ quality of life during and motor recovery after subcortical infarcts.
subacute recovery. Stroke 36(7):1480–1484 Neuroimage 20(4):2166–2180
6. Wyller TB, Sveen U, Sodring KM, Pettersen 11. Tombari D, Loubinoux I, Pariente J, Gerdelat A,
AM, Bautz-Holter E (1997) Subjective well- Albucher JF, Tardy J et al (2004) A longitudinal
 fMRI in Cerebrovascular Disorders 653

fMRI study: in recovering and then in clinically disease on the BOLD signal in
stable sub-cortical stroke patients. Neuroimage fMRI. Neuroimage 20(2):1393–1399
23(3):827–839 26. Rossini PM, Altamura C, Ferretti A, Vernieri F,
12. Ward NS, Brown MM, Thompson AJ, Zappasodi F, Caulo M et al (2004) Does cere-
Frackowiak RS (2006) Longitudinal changes in brovascular disease affect the coupling between
cerebral response to proprioceptive input in neuronal activity and local haemodynamics?
individual patients after stroke: an FMRI study. Brain 127(Pt 1):99–110
Neurorehabil Neural Repair 20(3):398–405 27. Rother J, Knab R, Hamzei F, Fiehler J,
13. Lee A, Kannan V, Hillis AE (2006) The contri- Reichenbach JR, Buchel C et al (2002)
bution of neuroimaging to the study of language Negative dip in BOLD fMRI is caused by
and aphasia. Neuropsychol Rev 16(4):171–183 blood flow-oxygen consumption uncoupling
14. Price CJ, Crinion J (2005) The latest on func- in humans. Neuroimage 15(1):98–102
tional imaging studies of aphasic stroke. Curr 28. Krainik A, Hund-Georgiadis M, Zysset S, von
Opin Neurol 18(4):429–434 Cramon DY (2005) Regional impairment of
15. Wise RJ (2003) Language systems in normal cerebrovascular reactivity and BOLD signal in
and aphasic human subjects: functional imag- adults after stroke. Stroke 36(6):1146–1152
ing studies and inferences from animal studies. 29. Murata Y, Sakatani K, Hoshino T, Fujiwara N,
Br Med Bull 65:95–119 Kano T, Nakamura S et al (2006) Effects of
16. Buxton RB (2002) An introduction to func- cerebral ischemia on evoked cerebral blood
tional magnetic resonance imaging: principles oxygenation responses and BOLD contrast
and techniques. Cambridge University Press, functional MRI in stroke patients. Stroke
Cambridge 37(10):2514–2520
17. Magistretti PJ, Pellerin L (1999) Cellular 30. Ward NS, Brown MM, Thompson AJ,
mechanisms of brain energy metabolism and Frackowiak RS (2003) Neural correlates of
their relevance to functional brain imaging. outcome after stroke: a cross-sectional fMRI
Philos Trans R Soc Lond B Biol Sci study. Brain 126(Pt 6):1430–1448
354(1387):1155–1163 31. Ward NS, Newton JM, Swayne OB, Lee L,
18. Magistretti PJ, Pellerin L, Rothman DL, Thompson AJ, Greenwood RJ et al (2006)
Shulman RG (1999) Energy on demand. Motor system activation after subcortical stroke
Science 283(5401):496–497 depends on corticospinal system integrity.
19. Iadecola C (2004) Neurovascular regulation in Brain 129(Pt 3):809–819
the normal brain and in Alzheimer’s disease. 32. Ward NS, Newton JM, Swayne OB, Lee L,
Nat Rev Neurosci 5(5):347–360 Frackowiak RS, Thompson AJ et al (2007) The
20. Attwell D, Iadecola C (2002) The neural basis relationship between brain activity and peak
of functional brain imaging signals. Trends grip force is modulated by corticospinal system
Neurosci 25(12):621–625 integrity after subcortical stroke. Eur J Neurosci
25(6):1865–1873
21. Friston KJ, Josephs O, Rees G, Turner R
(1998) Nonlinear event-related responses in 33. D’Esposito M, Zarahn E, Aguirre GK, Rypma
fMRI. Magn Reson Med 39(1):41–52 B (1999) The effect of normal aging on the
coupling of neural activity to the bold hemody-
22. Newton J, Sunderland A, Butterworth SE, namic response. Neuroimage 10(1):6–14
Peters AM, Peck KK, Gowland PA (2002) A
pilot study of event-related functional mag- 34. Ward NS, Swayne OB, Newton JM (2008)
netic resonance imaging of monitored wrist Age-dependent changes in the neural corre-
movements in patients with partial recovery. lates of force modulation: an fMRI study.
Stroke 33(12):2881–2887 Neurobiol Aging 29(9):1434–1446
23. Pineiro R, Pendlebury S, Johansen-Berg H, 35. Pollmann S, Dove A, Yves von Cramon D,
Matthews PM (2001) Functional MRI detects Wiggins CJ (2000) Event-related fMRI: com-
posterior shifts in primary sensorimotor cortex parison of conditions with varying BOLD
activation after stroke: evidence of local adap- overlap. Hum Brain Mapp 9(1):26–37
tive reorganization? Stroke 32(5):1134–1139 36. Wager TD, Vazquez A, Hernandez L, Noll DC
24. Carusone LM, Srinivasan J, Gitelman DR, (2005) Accounting for nonlinear BOLD
Mesulam MM, Parrish TB (2002) Hemodynamic effects in fMRI: parameter estimates and a
response changes in cerebrovascular disease: model for prediction in rapid event-related
implications for functional MR imaging. AJNR studies. Neuroimage 25(1):206–218
Am J Neuroradiol 23(7):1222–1228 37. Kim JA, Eliassen JC, Sanes JN (2005) Movement
25. Hamzei F, Knab R, Weiller C, Rother J (2003) quantity and frequency coding in human motor
The influence of extra- and intracranial artery areas. J Neurophysiol 94(4):2504–2511
654 Nick S. Ward

38. Cao Y, D’Olhaberriague L, Vikingstad EM, ment and focusing of brain activation. Stroke
Levine SR, Welch KM (1998) Pilot study of 33(6):1610–1617
functional MRI to assess cerebral activation of 51. Johansen-Berg H, Rushworth MF, Bogdanovic
motor function after poststroke hemiparesis. MD, Kischka U, Wimalaratna S, Matthews PM
Stroke 29(1):112–122 (2002) The role of ipsilateral premotor cortex
39. Chollet F, DiPiero V, Wise RJ, Brooks DJ, in hand movement after stroke. Proc Natl Acad
Dolan RJ, Frackowiak RS (1991) The func- Sci U S A 99(22):14518–14523
tional anatomy of motor recovery after stroke 52. Seitz RJ, Hoflich P, Binkofski F, Tellmann
in humans: a study with positron emission L, Herzog H, Freund HJ (1998) Role of
tomography. Ann Neurol 29(1):63–71 the premotor cortex in recovery from mid-
40. Cramer SC, Nelles G, Benson RR, Kaplan JD, dle cerebral artery infarction. Arch Neurol
Parker RA, Kwong KK et al (1997) A func- 55(8):1081–1088
tional MRI study of subjects recovered from 53. Ward NS, Brown MM, Thompson AJ,
hemiparetic stroke. Stroke 28(12):2518–2527 Frackowiak RS (2004) The influence of time
41. Weiller C, Chollet F, Friston KJ, Wise RJ, after stroke on brain activations during a motor
Frackowiak RS (1992) Functional reorga- task. Ann Neurol 55(6):829–834
nization of the brain in recovery from stria- 54. Dancause N, Barbay S, Frost SB, Plautz EJ,
tocapsular infarction in man. Ann Neurol Stowe AM, Friel KM et al (2006) Ipsilateral
31(5):463–472 connections of the ventral premotor cor-
42. Weiller C, Ramsay SC, Wise RJ, Friston KJ, tex in a new world primate. J Comp Neurol
Frackowiak RS (1993) Individual patterns 495(4):374–390
of functional reorganization in the human 55. Dancause N, Barbay S, Frost SB, Mahnken JD,
cerebral cortex after capsular infarction. Ann Nudo RJ (2007) Interhemispheric connections
Neurol 33(2):181–189 of the ventral premotor cortex in a new world
43. Strick PL (1988) Anatomical organization of primate. J Comp Neurol 505(6):701–715
multiple motor areas in the frontal lobe: impli- 56. Dum RP, Strick PL (1991) The origin of cor-
cations for recovery of function. Adv Neurol ticospinal projections from the premotor areas
47:293–312 in the frontal lobe. J Neurosci 11(3):667–689
44. Porter R, Lemon RN (1993) Corticospinal 57. He SQ, Dum RP, Strick PL (1993) Topographic
function and voluntary movement. Oxford organization of corticospinal projections
University Press, Oxford, UK from the frontal lobe: motor areas on the
45. Dum RP, Strick PL (1996) Spinal cord termina- lateral surface of the hemisphere. J Neurosci
tions of the medial wall motor areas in macaque 13(3):952–980
monkeys. J Neurosci 16(20):6513–6525 58. He SQ, Dum RP, Strick PL (1995) Topographic
46. Rouiller EM, Moret V, Tanne J, Boussaoud organization of corticospinal projections from
D (1996) Evidence for direct connections the frontal lobe: motor areas on the medial
between the hand region of the supplemen- surface of the hemisphere. J Neurosci 15(5 Pt
tary motor area and cervical motoneurons 1):3284–3306
in the macaque monkey. Eur J Neurosci 59. Boudrias MH, Belhaj-Saif A, Park MC, Cheney
8(5):1055–1059 PD (2006) Contrasting properties of motor
47. Calautti C, Leroy F, Guincestre JY, Baron JC output from the supplementary motor area
(2001) Dynamics of motor network overac- and primary motor cortex in rhesus macaques.
tivation after striatocapsular stroke: a longi- Cereb Cortex 16(5):632–638
tudinal PET study using a fixed-performance 60. Maier MA, Armand J, Kirkwood PA, Yang
paradigm. Stroke 32(11):2534–2542 HW, Davis JN, Lemon RN (2002) Differences
48. Calautti C, Leroy F, Guincestre JY, Baron JC in the corticospinal projection from primary
(2003) Displacement of primary sensorimotor motor cortex and supplementary motor area
cortex activation after subcortical stroke: a lon- to macaque upper limb motoneurons: an ana-
gitudinal PET study with clinical correlation. tomical and electrophysiological study. Cereb
Neuroimage 19(4):1650–1654 Cortex 12(3):281–296
49. Cramer SC, Shah R, Juranek J, Crafton KR, 61. Baker SN, Zaaimi B, Fisher KM, Edgley SA,
Le V (2006) Activity in the peri-infarct rim Soteropoulos DS (2015) Pathways medi-
in relation to recovery from stroke. Stroke ating functional recovery. Prog Brain Res
37(1):111–115 218:389–412
50. Feydy A, Carlier R, Roby-Brami A, Bussel B, 62. Dettmers C, Fink GR, Lemon RN, Stephan
Cazalis F, Pierot L et al (2002) Longitudinal KM, Passingham RE, Silbersweig D et al
study of motor recovery after stroke: recruit- (1995) Relation between cerebral activity and
 fMRI in Cerebrovascular Disorders 655

force in the motor areas of the human brain. nectivity after subcortical stroke assessed with
J Neurophysiol 74(2):802–815 functional magnetic resonance imaging. Ann
63. Thickbroom GW, Phillips BA, Morris I, Byrnes Neurol 63(2):236–246
ML, Sacco P, Mastaglia FL (1999) Differences 71. Bestmann S, Swayne O, Blankenburg F, Ruff
in functional magnetic resonance imaging of CC, Teo J, Weiskopf N et al (2010) The role of
sensorimotor cortex during static and dynamic contralesional dorsal premotor cortex after
finger flexion. Exp Brain Res 126(3):431–438 stroke as studied with concurrent TMS-fMRI. J
64. Ward NS, Frackowiak RS (2003) Age-related Neurosci 30(36):11926–11937
changes in the neural correlates of motor per- 72. Zemke AC, Heagerty PJ, Lee C, Cramer SC
formance. Brain 126(Pt 4):873–888 (2003) Motor cortex organization after stroke
65. Fridman EA, Hanakawa T, Chung M, is related to side of stroke and level of recovery.
Hummel F, Leiguarda RC, Cohen LG (2004) Stroke 34(5):e23–e28
Reorganization of the human ipsilesional 73. Crafton KR, Mark AN, Cramer SC (2003)
premotor cortex after stroke. Brain 127(Pt Improved understanding of cortical injury by
4):747–758 incorporating measures of functional anatomy.
66. Lotze M, Markert J, Sauseng P, Hoppe J, Brain 126(Pt 7):1650–1659
Plewnia C, Gerloff C (2006) The role of mul- 74. Marshall RS, Perera GM, Lazar RM, Krakauer
tiple contralesional motor areas for complex JW, Constantine RC, DeLaPaz RL (2000)
hand movements after internal capsular lesion. Evolution of cortical activation during recov-
J Neurosci 26(22):6096–6102 ery from corticospinal tract infarction. Stroke
67. Grefkes C, Fink GR (2014) Connectivity-based 31(3):656–661
approaches in stroke and recovery of function. 75. Small SL, Hlustik P, Noll DC, Genovese C,
Lancet Neurol 13(2):206–216 Solodkin A (2002) Cerebellar hemispheric
68. Carter AR, Patel KR, Astafiev SV, Snyder AZ, activation ipsilateral to the paretic hand corre-
Rengachary J, Strube MJ et al (2012) Upstream lates with functional recovery after stroke.
dysfunction of somatomotor functional con- Brain 125(Pt 7):1544–1557
nectivity after corticospinal damage in stroke. 76. Ward NS, Brown MM, Thompson AJ,
Neurorehabil Neural Repair 26(1):7–19 Frackowiak RS (2003) Neural correlates of
69. Carter AR, Astafiev SV, Lang CE, Connor LT, motor recovery after stroke: a longitudinal
Rengachary J, Strube MJ et al (2010) Resting fMRI study. Brain 126(Pt 11):2476–2496
interhemispheric functional magnetic reso- 77. Kleim JA, Chan S, Pringle E, Schallert K,
nance imaging connectivity predicts perfor- Procaccio V, Jimenez R et al (2006) BDNF val-
mance after stroke. Ann Neurol 67(3):365–375 66met polymorphism is associated with modi-
70. Grefkes C, Nowak DA, Eickhoff SB, Dafotakis fied experience-dependent plasticity in human
M, Küst J, Karbe H et al (2008) Cortical con- motor cortex. Nat Neurosci 9(6):735–737
 Chapter 22

fMRI in Psychiatric Disorders

Erin L. Habecker, Melissa A. Daniels, Elisa Canu, Maria A. Rocca,
Massimo Filippi, and Perry F. Renshaw

Abstract
Functional neuroimaging has become an important tool for clinical research, with the potentiality to
provide information on psychiatric disease pathology and treatment response. We review functional mag-
netic resonance imaging (fMRI) research findings for five psychiatric disorders: schizophrenia, major
depressive disorder, bipolar disorder, obsessive-compulsive disorder, and posttraumatic stress disorder.
Brain functional abnormalities and possible underlying mechanisms for disease symptoms are discussed,
with a focus on future clinical implications for fMRI in psychiatric disease.

Key words fMRI, Blood oxygen level dependent, Psychiatric disorders, Schizophrenia, Major depres-
sive disorder, Bipolar disorder, Obsessive-compulsive disorder, Posttraumatic stress disorder

1 Introduction

1.1 Overview of fMRI Functional magnetic resonance imaging (fMRI) is a unique, noninva-
sive method of measuring neural activation through changes in oxida-
tion and regional blood flow. An important clinical research tool that
has been used more and more frequently in recent years, fMRI is able
to indirectly detect brain activity in the working brain, allowing for the
assessment of psychiatric disease physiology and treatment effects.
fMRI does not involve exposure to radioactive tracers, thus allowing
patients and subjects to undergo multiple scans over a short period of
time, if necessary. Most fMRI studies involve the measurement of sig-
nal arising from hydrogen nuclei [1, 2]. Common types of fMRI used
in psychiatric neuroimaging include blood oxygen level dependent
(BOLD) and arterial spin labeling (ASL).
Instead of incorporating a radioactive tracer as in positron
emission tomography (PET) or single photon emission computed
tomography (SPECT), fMRI makes use of the unique properties
of hemoglobin (BOLD and BOLD contrast methods) or the water
molecules of flowing blood (ASL) to produce images of neural
activation. Most fMRI studies today are BOLD studies that make

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_22, © Springer Science+Business Media New York 2016

657
658 Erin L. Habecker et al.

use of T2* mechanisms. ASL fMRI depends on T1 effects, which

will be further described below. Hemoglobin is present in the body
in two forms: the oxygenated form, oxyhemoglobin; and the deox-
ygenated form, deoxyhemoglobin. T2* weighed images of each
form of hemoglobin are distinctive because the two have different
magnetic properties. Neuronal activity results in greater cerebral
blood flow (CBF) to the specific brain areas involved in the pro-
cessing of a particular task, leading to an increase in T2* signal and
a more intense MR signal on the images created. Tasks and stimuli
are used during fMRI to elicit a predicted brain response—they are
intended to alter neural activity in brain regions thought to be
impacted by the disorder in question.

1.2 BOLD The advantages of BOLD fMRI—the most common psychiatric

fMRI modality—as compared with other functional imaging tech-
niques, such as PET, include the greater sensitivity of the fMRI
signal to event-related changes in neuronal blood flow and the
increased spatial resolution of fMRI images [3]. Temporal resolu-
tion, which was historically poor in previous imaging methods, has
been improved drastically through the use of high-speed MR scan-
ners with the ability to perform echo planar imaging, acquiring
single image planes in 50–100 ms [4]. However, one distinct dis-
advantage of this mode of functional imaging that must be taken
into consideration during experimental construction is the inability
of BOLD signal to differentiate between changes in CBF that are
correlated with neuronal activity and changes that are independent
of it. Such changes include activity-related signal changes in drain-
ing veins away from the brain activity [5], incidental neural activa-
tions that are unrelated to the task at hand [6], and changes in
CBF caused by changes in respiration. Even small respiration
changes can alter blood arterial carbon dioxide tension (PCO2),
which has a large effect on CBF [7, 8]. Subjects with an anxiety
disorder, or state anxiety induced by the MRI environment, are
particularly susceptible to variations in respiration, and this must
be taken into account during experiment planning using
BOLD. The effects of respiration changes on PCO2 may be man-
aged by acquiring continuous measurements of PCO2, or end-tidal
CO2, during the experimental protocol [3]. When this function is
available, investigators have the option of either acquiring data
only during steady-state CO2 levels [9], or attempting to adjust for
the effect of PCO2 on global CBF [10, 11] and integrate this mod-
ification into fMRI indices.

1.3 ASL ASL differs from BOLD in that it depends on T1 mechanisms and
the magnetic labeling of water molecules to generate images. Water
molecules in flowing blood are tagged through the saturation or
inversion of the longitudinal component of the MR signal [12];
these molecules then diffuse from capillaries into brain tissue where
 fMRI in Psychiatric Disorders 659

they alter the magnetization of the local tissue [1]. As blood flow
into the imaging slice increases, there is a more significant differ-
ence between the magnetized condition and the control condition,
during which the magnetization of arterial blood is fully relaxed
[1]. Control and tagged images are then taken, and the difference
between them is proportional to the CBF. ASL can be used to
measure global CBF changes dynamically; its use of water mole-
cules as an endogenous blood flow tracer means that the images
generated by ASL are not susceptible to neurovascular changes
that are not related to neuronal activation. ASL has another advan-
tage over BOLD in that effects of frequency drifts tend to be mini-
mized in ASL, making this method more suitable for longer
duration scans [12]. However, BOLD acquisitions tend to have
greater temporal resolution, greater maximum number of slices,
and appear to be more sensitive to parametric manipulations of
task demands [1, 13, 14]. BOLD maps also usually have larger
activation areas than ASL maps [15, 16], which could either be
due to the decreased sensitivity or improved signal localization
inherent in ASL [13]. There are three classes of ASL methods:
pulsed ASL, continuous ASL, and velocity selective ASL [1]. A
discussion of the relative methodologies and merits of the three
techniques is beyond the scope of this chapter.

2 fMRI in Psychiatry

2.1 Clinical A wide range of neuropsychiatric disorders have now been investi-
Disorders gated using fMRI techniques and protocols. This review will
explore the paradigms employed, imaging results, and future
research opportunities in five mental disorders: schizophrenia,
major depression, bipolar disorder (BD), obsessive-compulsive dis-
order (OCD), and posttraumatic stress disorder (PTSD). These
particular disorders were selected due to the fact that they repre-
sent a subset of psychotic, mood, and anxiety disorders; have a
significant prevalence in the general population (0.4–14 % depend-
ing on age and gender of the sample); are popular candidates for
fMRI research; and have each been the subject of research on diag-
nosis, disease progression, and treatment using imaging. Table 1
summarizes the nature, range, and prevalence of the selected disor-
ders in the general population.
Modern imaging techniques have been crucial to the delinea-
tion of the brain structures and functions that are negatively
impacted in psychiatric disorders such as the ones reviewed here.
Traditionally, such disorders have been characterized primarily via
clinical psychiatric evaluation of abnormal symptoms, and treat-
ments consist of a trial-and-error strategy combined with patient
self-selection of treatment options or option combinations [17].
The use of fMRI to evaluate the underlying cognitive disturbances
Table 1
660

Summary of psychiatric disorders discussed in this chapter

Disorder Type Characteristic symptoms Subtypes Prevalence Prognosis Typical treatment

Schizophrenia Psychotic Delusions, hallucinations (often Subtypes are defined by the 0.5–1.5 % among Complete remission Antipsychotic
disorder auditory), disorganized speech, predominant symptom at adults uncommon. Some medications,
grossly disorganized or catatonic the time of evaluation individuals display including
behavior, negative symptoms (paranoid, disorganized, exacerbations and clozapine,
(affective flattening, alogia, catatonic, undifferentiated, remissions of risperidone,
aviolotion) residual). However, symptoms while olanzapine,
Erin L. Habecker et al.

symptoms are not stable others remain quetiapine,

during the course of the chronically ill, either ziprasidone, and
disorder and often change on a stable course or a aripiprazole
from one subtype to progressively
another and/or present worsening one
overlapping within
subtypes
Major depressive Mood One or more major depressive Single episode, recurrent 5–9 % among Major depressive SSRIs, tricyclics,
disorder disorder episodes accompanied by adult women, episodes may end and MAOIs
changes in appetite, weight, or 2–3 % among completely (2/3 of
sleep; decreased energy; feelings adult men cases) or partially or
of worthlessness or guilt; not at all (1/3 of
difficulty thinking, cases). Up to 15 % of
concentrating, or making affected individuals
decisions; or recurrent thoughts die by suicide
of death or suicide
Bipolar disorder Mood One or more manic or mixed Bipolar I, bipolar II, 0.4–5 % among Recurrent disorder: Mood stabilizers,
disorder episodes (Bipolar I) or one or cyclothymia, bipolar adults interval between including
more hypomanic episodes disorder not otherwise episodes tends to lithium,
(Bipolar II) accompanied by one specified decrease with age. valproate,
or more major depressive 20–30 % of affected carbamazepine,
episodes. Cyclothymia individuals do not lamotrigine,
characterized by periods of return to full gabapentin, and
hypomanic and depressive functionality between topiramate
symptoms episodes. Up to 15 %
of affected individuals
die by suicide
 Obsessive- Anxiety Recurrent obsessions (persistent Checkers, washers, touchers, 1.5–2.5 % among Majority of individuals SSRIs, tricyclics
compulsive disorder ideas, thoughts, impulses, or counters, and arrangers; adults have chronic disease (clomipramine
disorder images) or compulsions Insight needs to be course with waxing only), MAOIs,
(repetitive behaviors) that are verified. and waning and
time-consuming or cause symptoms. 15 % show benzodiazepines
marked distress or significant progressive
impairment. Affected adults deterioration of
realize that obsessions or occupational and
compulsions are excessive or social functioning. 5 %
unreasonable have episodic course
with minimal
symptoms between
episodes
Posttraumatic Anxiety Development of characteristic Acute, chronic, delayed 1–14 % among Symptomatic onset SSRIs, including
stress disorder disorder symptoms following an extreme onset adults; up to usually occurs within sertraline and
traumatic stressor. Symptoms 58 % for at risk 3 months after paroxetine
include persistent reexperiencing populations trauma. Complete
of the trauma, avoidance of (combat recovery occurs
stimuli associated with the victims, victims within 3 months in
trauma, numbing of general of criminal 50 % of cases;
responsiveness, and increased violence, etc.) symptoms persist
arousal longer than 12
months in many cases
MAOIs monoamine oxidase inhibitors, NMDA N-methyl d-aspartate, SSRIs selective serotonin reuptake inhibitors
fMRI in Psychiatric Disorders
661
662 Erin L. Habecker et al.

present across a heterogeneous psychiatric disorder, or even a

range of psychiatric disorders, is important in that it allows for the
investigation of core dysfunctions that might highlight more effec-
tive treatment options. Studying differences in neural response
between psychiatric patients and normal subjects with respect to
affected brain regions, and incorporating a variety of emotional
and cognitive challenges to examine localized activation, investiga-
tors have the opportunity to evaluate subtle differences in the ways
that the brains of patients process different types of information
and perform tasks. In addition, functional imaging can be used to
evaluate the efficacy of a psychiatric medication, especially in con-
junction with the usual clinical symptom assessments. New findings
are also highlighting the ways in which fMRI could be utilized to
aid in the diagnosis of psychiatric disease [17–19].

3 Psychotic Disorders

3.1 Schizophrenia Schizophrenia is a lifelong illness associated with a high rate of

morbidity and disability due to the severity and neurologically dis-
ruptive nature of its symptoms. It is often thought of as the most
serious psychiatric disorder, and the afflicted population (about 1 %
of the general population) is impaired in one or more major areas
of functioning: interpersonal relations, work or education, or self-
care (American Psychiatric Association, Diagnostic and Statistical
Manual of Mental Disorders, 5th edition, 2014—DSMV 2014).
The disorder, which typically manifests sometime in an individual’s
mid-20s, is diagnosed through a number of characteristic symp-
toms falling into positive (an excess or distortion of normal func-
tion) or negative (attenuation or loss of normal function) categories.
Positive symptoms include delusions, hallucinations, disorganized
speech, and disorganized or catatonic behavior, while the negative
symptoms encompass affective flattening and abulia (DSM-V,
2014). Because schizophrenia is a heterogeneous disorder with a
wide range of associated impairments, the range of tasks employed
during fMRI studies has been similarly broad. Table 2 summarizes
the most common paradigms, which include verbal fluency [31],
affective pictures [22, 27, 28], working memory (WM) [20, 23,
26], and inhibitory control [21, 33]. These studies have reported
attenuation and deactivation of fMRI signal, as compared to
healthy control groups, in prefrontal and temporal lobe structures
including the amygdala, hippocampus, and parahippocampal
gyrus. In addition, increased activation of the basal ganglia and
striatum has been observed during WM and inhibitory control
tasks [20, 21, 26, 33].
fMRI assessment of cognitive verbal and memory tasks in
schizophrenic patients allows for an analysis of cognitive deficits
and enables investigation of the abnormal language functionality
Table 2
Summary of fMRI research in schizophrenia

Authors Subjects fMRI paradigm Results

Schizophrenia
Manoach 9 schizophrenic subjects (8 under stable dose of Working memory: Sternberg Schizophrenic patients exhibited deficiencies in working
et al. [20] antipsychotic medication, 1 unmedicated) and 9 Item Recognition memory, despite similar activity in the DLPFC compared to
healthy controls Paradigm adapted to healthy controls. Compared with controls, during the task,
include monetary reward schizophrenic patients showed activation in the basal ganglia
for correct responses and thalamus
Rubia et al. 6 male patients with schizophrenia (all under Inhibitory control: “stop” Schizophrenic patients exhibited reduced left prefrontal
[21] atypical antipsychotic medication) and 7 and “go/no-go” tasks activation compared to healthy controls
matched healthy controls
Hempel 10 partially remitted schizophrenic patients (all Facial affect discrimination Compared with controls, schizophrenic patients showed a
et al. [22] under atypical antipsychotic medication) and 10 and labeling significantly decreased activation in the ACC during the
healthy controls discrimination task, and decreased activity in the amygdala-
hippocampal complex bilaterally during labeling
Hofer et al. 10 male outpatients with schizophrenia (all under Episodic encoding/ Compared with controls, patients with schizophrenia
[23] atypical antipsychotic medication) and 10 male recognition of words demonstrated similar cognitive performance in word
healthy controls recognition, but decreased activation in the bilateral DLPFC
and lateral temporal cortices during both tasks
Kubicki 9 male chronic schizophrenic patients (unspecified Semantic encoding Schizophrenic patients had decreased activation of the left
et al. [24] medication) and 9 control subjects inferior prefrontal cortex and increased activation of the left
superior temporal gyrus compared to healthy controls
Habel et al. 13 male patients with schizophrenia (8 under Positive and negative mood Schizophrenic patients and brothers of schizophrenic patients
[25] typical and 4 under atypical antipsychotic evocation showed reduced amygdala activity compared to healthy
medication, and 1 unmedicated), 13 of their non controls
affected brothers (asymptomatic, unmedicated),
and 26 unrelated matched healthy controls
fMRI in Psychiatric Disorders

Ragland 14 patients with schizophrenia (2 under typical and 9 Word encoding/recognition Compared with controls, schizophrenic patients demonstrated
et al. [26] under atypical antipsychotic medication, and 3 reduced activation of the prefrontal cortex and increased
under both typical/atypical medication) and 15 activation of the parahippocampal gyri during encoding
healthy controls
663

(continued)
Table 2
664

(continued)

Authors Subjects fMRI paradigm Results

Takahashi 15 schizophrenic patients (11 under atypical Affective pictures Compared with controls, patients with schizophrenia had
et al. [27] antipsychotic medication; 4 unmedicated) and decreased activity in the right amygdala and medial
15 healthy volunteers prefrontal cortex
Williams 27 schizophrenic patients and 22 healthy controls Facial expressions of fear Compared with controls, schizophrenic patients had enhanced
et al. [28] arousal responses coupled with reduction in amygdala and
medial prefrontal activity
Erin L. Habecker et al.

Honey et al. 12 healthy volunteers were administered with Episodic memory task Subjects on ketamine showed left frontal activation during
[29] 100 ng/ml plasma ketamine or placebo semantic processing at encoding and in these subjects
successful encoding was supplemented by additional
nonverbal processing
Morey et al. 52 subjects: 10 ultra-high risk for schizophrenia, Visual oddball task Compared with controls, early and chronic schizophrenic
[30] 15 with early schizophrenia, 11 with chronic patients showed lower activation associated with the target
schizophrenia and 16 healthy controls stimuli in the ACC, inferior frontal gyrus, and the medial
frontal gyrus. Although the ultra-high-risk group did not
reach the significance when compared with controls, they
showed a trend toward the early group
Yurgelun- 12 schizophrenic patients under 8 weeks of Word fluency task Compared with patients receiving placebo, patients receiving
Todd d-cycloserine treatment or placebo d-cycloserine showed a significant increased activation,
et al. [31] associated with a reduction in negative symptoms, in the
temporal lobe
Juckel et al. 20 schizophrenic patients (10 under typical and 10 Incentive monetary delay task Schizophrenic patients showed reduced ventral striatal
[32] under atypical antipsychotic medication) and 10 activation during the presentation of reward-indicating cues
age-matched male healthy controls as compared to healthy controls. Decreased activation of the
left ventral striatum was inversely correlated with the severity
of symptoms
Vink et al. 21 schizophrenic patients (all under stable atypical Inhibitory control: stop cues Compared with controls, schizophrenic patients and
[33] antipsychotic medication), 15 unaffected unaffected siblings did not activate the striatum when
siblings, and 36 matched healthy controls responding to motor cues
ACC anterior cingulate cortex, DLPFC dorsolateral prefrontal cortex
 fMRI in Psychiatric Disorders 665

seen in some individuals with the disorder. WM performance, as

commonly measured by encoding and recognition tasks, continu-
ous performance paradigms, and the Sternberg test, to name a few,
appears to be impacted by schizophrenia. Difficulties in encoding
and free recall are common, indicating the possible involvement of
the prefrontal cortex and hippocampus, both of which have been
shown to participate in the neural mechanisms of working, epi-
sodic, and semantic memory [34–36]. Ragland et al. [26], using a
word encoding and recognition task, observed dorsolateral
prefrontal cortex (DLPFC) dysfunction manifested by bilateral
defects during encoding, left hemisphere hypoactivity during rec-
ognition, and right side signal attenuation during successful
retrieval, as compared to a healthy control group. This finding has
been replicated using a visual oddball continuous task [30]. In
addition, marked increase in parahippocampal activation in schizo-
phrenic patients as compared to healthy controls has been observed
during task performance, suggesting a core deficit in the reciprocal
connections between the hippocampus and the neocortex [26].
Other WM fMRI studies of schizophrenia have reported attenu-
ated activations in the anterior cingulate cortex (ACC) and cere-
bellum [29], and significantly increased activity in the basal ganglia
and thalamus [20]. It has been hypothesized that frontostriatal
circuitry defects could account for these deviations, and the ana-
tomical as well as functional normality of this circuitry in schizo-
phrenic patients has been targeted for further study [20].
Investigations of emotional processing, long a part of psychiatric
research into mood disorders, have begun in recent years to be used
in the study of schizophrenia. fMRI protocols have used primarily
affective facial expressions, as well as emotional pictures and words,
to evoke cortical responses in the temporal and frontal lobes.
Attenuation of amygdala response, as well as that of the amygdala-
hippocampal complex, has been noted by several investigators in
response to emotional faces, discrimination of facial affect, and the
evocation of negative mood [22, 25, 27, 28]. Some of these studies
have also shown attenuation in the ACC and medial prefrontal cor-
tex (mPFC) as compared to healthy controls [22, 27]. This reduced
activation is often accompanied by increased activation in another
area, such as the middle frontal gyrus (MFG) in one study [22]. This
suggests that the observed increased activations are secondary mech-
anisms evoked to compensate for the dysfunction in related areas.
Other fMRI studies of schizophrenia have focused on the neu-
ral underpinnings of the episodic memory, language, and learning
deficits common in the disorder. Hofer et al. [23] found decreased
activations in the bilateral DLPFC and lateral temporal cortices in
schizophrenic patients, despite the fact that recognition perfor-
mance in the schizophrenic patients was intact. A language pro-
cessing task highlighted underactivity in the temporal lobe which
improved after treatment with d-cycloserine, in conjunction with
666 Erin L. Habecker et al.

negative symptom improvement [31]. Reduced activation of the

left inferior prefrontal cortex was observed, accompanied by exag-
gerated activation in the left superior temporal gyrus, during a
semantic encoding learning task [24]. These findings are particu-
larly relevant in a clinical sense, as encoding, learning, and lan-
guage skills are important for normal social interaction and
functioning. These studies increase our level of understanding of
the neuropsychological dysfunctions that underlie some of the
most disruptive symptoms of schizophrenia, and point to a need
for further study to determine optimal treatment options.

3.1.1 fMRI Features In the last decades, several studies have attempted to define the
of Subjects at Risk prodromal traits predictive of future conversion to schizophrenia in
to Develop Schizophrenia at-risk subjects, especially after demonstration that early interven-
tion improves disease prognosis [37]. The eventual “risk” to
develop schizophrenia is defined in terms of genetic aspects, such as
a positive family history for schizophrenia (e.g., individuals who
have an affected first-degree family member-FHR), or in terms of
clinical (prodromal) features. Among individuals with a clinical
high risk (CHR) to develop schizophrenia, subjects ‘at risk mental
state’ (ARMS) [38] are those having clinically defined sub-psychotic
symptoms and subjects at ultra-high risk (UHR) are defined by the
presence of prevalent positive clinical symptoms [38, 39].
Although altered activation of frontal and prefrontal cortices
has been amply reported in people with increased risk of psychosis,
at present it is still not clear if this neurofunctional alteration
increases in line with the level of psychosis risk. On this purpose, a
study observed a relationship between the level of working mem-
ory task-related deactivation in the mPFC and precuneus and the
level of psychosis risk, with deactivation weakest in the UHR
group, at an intermediate level in the FHR group, and greatest in
healthy controls [40]. In another study, while controls showed a
negative association between age and frontal functional activation
during verbal working memory, clinical high risk youth who con-
verted to psychosis showed the opposite [41], likely reflecting an
emerging hyperactivity in frontal regions for compensative pur-
poses [41]. During an executive task, a significant reduction in the
topological centrality of the ACC in ARMS subjects which later
converted into a psychotic disorder suggested this as a potential
biomarker for the transition to psychosis [42].
In subjects at risk, functional alterations at the frontal regions
are likely to subtend not only their cognitive but also their emo-
tional features. In FHR subjects, it has been reported a reduced
coupling between amygdala and prefrontal cortex during facial
expression processing [43]. Adolescent FHR subjects have shown
reduced ACC activation during emotional processing [44] and
specific hyperactivation in the right superior frontal gyrus and right
precentral gyrus during fearful face presentations [45]. Patients
 fMRI in Psychiatric Disorders 667

with schizophrenia and their nonpsychotic relatives showed limbic

system hypoactivation similarities during facial affect attribution
tasks [46, 47] and similar hypoactivation in the amygdala while a
sad mood was elicited [25].
In terms of metacognitive abilities, while processing a decision,
subjects at risk did not demonstrate “jumping to conclusion” biases,
typical of patients with schizophrenia. However, ARMS subjects
showed a significant hypoactivation in the right ventral striatum, simi-
lar to that of schizophrenic patients, during the decision making [48].
Finally, since poor social functioning is a hallmark of schizophre-
nia and may precede the onset of illness, a number of studies focused
on the investigation of the theory of mind (ToM) in subjects at high
risk of psychosis [49]. While performing a ToM task, UHR subjects
experiencing psychotic symptoms in the past had lower activation of
the right inferior parietal lobule and parts of the prefrontal cortex
[50]. Moreover, those subjects who at the day of scanning had psy-
chotic symptoms displayed activations more similar to patients with
manifest schizophrenia [50]. In contrast, subjects at high risk who
had never experienced psychotic symptoms showed significantly
greater activation in the MFG compared to high risk subjects who
did experience psychotic symptoms in the past.

4 Mood Disorders

4.1 Major Major depression is diagnosed in patients who experience one or

Depression more major depressive episodes without any associated mania. A
major depressive episode occurs when a patient presents with per-
sistent feelings of deep despair and loss of pleasure or interest in
nearly all activities for at least 2 weeks, accompanied by at least five
of the following symptoms: two are core features, i.e., depressed
mood and markedly diminished or loss of interests and pleasure;
the other three could be among sleep disturbances, disruption of
appetite, feelings of hopelessness or worthlessness, difficulty con-
centrating, or suicidal thoughts (DSM V, 2014). Individuals with
major depressive disorder may present a range of heterogeneous
symptoms within this framework, implying that the disease impacts
more than one brain region or neurotransmitter system. Studying
mood and cognition-induced brain activations in affected individ-
uals represents a powerful way to unlock the functional discrepan-
cies between the depressed and normal nervous system. fMRI
studies have revealed a wide-range network of limbic and paralim-
bic neural regions and circuitry whose interactions appear to be
disrupted in major depressive disorder [51–53].
The limbic-cortical model of depression advanced by Mayberg
et al. hypothesizes major depression as a dysfunction among dis-
crete, but functionally integrated pathways in the dorsal, ventral, and
rostral compartments of the brain [17]. Respective dysfunctions in
668 Erin L. Habecker et al.

components of each of these compartments, which include the dor-

solateral and dorsomedial prefrontal cortex, dorsal ACC, and poste-
rior cingulate (dorsal compartment); subgenual anterior cingulate,
ventral prefrontal cortex, insula, hippocampus, and amygdala (ven-
tral compartment); and rostral ACC (rostral compartment), can all
be associated with the collection symptoms seen in major depression
(Fig. 1) [53]. In addition, it has been theorized that the failure of
the healthy elements in the system to maintain homeostasis of emo-
tionality during times of stress when a part of the system is compro-
mised is a contributor to major depressive episodes [17]. Depressed
subjects demonstrate abnormalities in regional cerebral blood flow
(rCBF) and regional cerebral glucose metabolism (regional cerebral
metabolic rate for glucose, rCMRglc) in the dorsal and ventral com-
partments. Decreases rCBF and rCMRglc that have been observed
in the DLPFC [52, 54], dorsomedial and dorsal anterolateral pre-
frontal cortex, as well as the dorsal ACC [54, 55] in depressed sub-
jects during PET and SPECT studies have highlighted these areas
for exploration with fMRI BOLD paradigms—as do the observed
increases in rCBF and rCMRglc that have been found in

Fig. 1 A limbic-cortical model of depression adapted and modified from Mayberg

et al. [53]. It involves three compartments: a dorsal, a ventral, and a rostral com-
partment. BG basal ganglia, TH thalamus, dACC dorsal anterior cingulate cortex,
rACC rostral anterior cingulate cortex
 fMRI in Psychiatric Disorders 669

components of the ventral compartment, including the ventrolat-

eral, ventromedial, orbitofrontal cortex (OFC), the subgenal pre-
frontal cortex, the amygdala, and the insular cortex [56, 57].
The amygdala has been extensively investigated for its involve-
ment in the processing of emotional stimuli and abnormal response in
individuals with major depressive disorder. Emotional paradigms used
in these fMRI studies include the exhibition of emotional film clips,
facial photographs, or the presentation of audio cues during the scan
to induce feelings of sadness, happiness, or fear [4]. During negative
emotional tasks (negative words and sad faces), it has been shown that
the amygdala in the depressed brain displays abnormally sustained
activations as compared to the amygdala in the normal brain [19, 58,
59]. Other studies utilizing emotional facial expression stimuli with
depressed subjects have also shown increased activation in the hippo-
campus, left parahippocampal gyrus, and other regions of the left
brain (Table 3) [19, 60], as well as increased dynamic range bilaterally
in the cerebellum and ACC extending to the rostral prefrontal cortex
in response to sad faces, as compared to healthy controls [19].
Cognitive disturbances associated with depression, such as con-
centration difficulties, explicit memory impairment, and impairment
in executive functioning, have been examined with fMRI paradigms
that incorporate various WM and executive control tasks. In com-
parison to healthy control subjects, depressed individuals demon-
strate slower reaction times and decreased accuracy with regards to
executive control challenges in conjunction with decreased DLPFC
activity [59]. Interestingly, depressed subjects were also shown, in
another study, to have increased activation in the DLPFC in response
to cognitive load in a WM task [64].
Ideally, identifying brain regions and circuitry impacted by
depressive disorder will lead to a greater understanding of the
effects of drug treatment and yield important information regard-
ing the feasibility and success of various treatment options. In
order to delineate the neurocircuitry involved in processing emo-
tional cues and gather information about the pharmacodynamics
of various antidepressants, studies have been undertaken that
examine the effects of antidepressant treatment in a variety of sce-
narios. PharmacoMRI (pMRI) studies of a single dose of an anti-
depressant with healthy control subjects allow any focal changes in
brain activity induced by the drug to be observed during BOLD
scanning. Pre- and posttreatment studies of antidepressants incor-
porate structural MRI and fMRI to combine activation paradigms
with antidepressant treatment and identify brain functional corre-
lates of antidepressant treatment and symptomatic response [19,
78]. Two studies investigating, respectively, the effects of citalo-
pram and mirtazapine on the healthy nervous system in a pMRI
format found that each antidepressant enhanced activations right
OFC during a Go/No-go task [61, 63]. In addition, one study
incorporating the emotional faces paradigm found that the
Table 3
670

Summary of fMRI research in mood disorders

Authors Subjects fMRI paradigm Results

Major depressive disorder
Siegle et al. 7 depressed and 10 never-depressed Alternating 15 s emotional and Depressed subjects displayed sustained amygdala responses to negative
[58] subjects nonemotional processing trials words that lasted throughout the subsequent nonemotional
processing trial (25 s later). Never-depressed subjects displayed
amygdala responses to all emotional stimuli, decaying within 10 s
Erin L. Habecker et al.

Fu et al. [19] 19 medication free, acutely depressed Facial expressions of sadness: Compared with controls, depressed subjects had decreased activation in
subjects and 19 matched healthy low, medium, and high regions of the left brain: hippocampus and parahippocampal gyrus,
volunteers intensity. Subjects asked to amygdala, insula, caudate nucleus, thalamus, dorsal cingulate gyrus,
identify sex of face inferior parietal cortex. They also showed negative relationship
between increased differential response to variable affective intensity
and activation in the rostral prefrontal cortex
Surguladze 16 individuals with major depressive Facial expressions: happy and sad Healthy individuals displayed increased activation in bilateral fusiform
et al. [60] disorder and 14 healthy controls gyri and right putamen in response to increasing happiness. Depressed
subjects demonstrated activation in left putamen, left
parahippocampal gyrus/amygdala, and right fusiform gyrus in
response to increasing sadness
Del-Ben 12 healthy male subjects, under 7.5 mg IV Go/No-go, Loss/No-loss, Citalopram enhanced activations in the right BA47 during Go/No-go
et al. [61] citalopram or placebo, single blind covert (averse) face emotion task but attenuated BA47 response to aversive faces. Citalopram
crossover design recognition attenuated the right amygdala response to aversive faces and the
BA11 response during Loss/No-loss task
Wagner et al. 16 patients with unipolar depression (free Adapted version of Stroop task No differences in reaction time and accuracy between groups.
[62] of psychotropic medication for a week at Compared with controls, patients showed increased activation in the
the time of the fMRI); 16 matched rostral ACC and left DLPFC
healthy controls
Siegle et al. 27 unmedicated, unipolar depressive Executive control task: digit Compared with controls, depressed subjects displayed sustained
[59] subjects (free of antidepressant sorting; emotional information amygdala reactivity on emotional tasks and decreased DLPFC activity
medication for 2 weeks before testing) processing: personal relevance on the digit-sorting task
and 25 never-depressed healthy controls rating of words
Vollm et al. 45 healthy male subjects under Go/No-go, Reward/No-reward, The Mirtazapine treatment was associated with enhanced activation of the
[63] Mirtazapine or placebo, double blind, Loss/No-loss right orbitofrontal cortex during Go/No-go and Reward/No-reward
placebo controlled and of the bilateral parietal cortex during Reward/No-reward
 Walter et al. 12 partially remitted, medicated inpatients Working memory: delayed match Compared with controls, depressed patients were slower and less
[64] with major depressive disorder; 17 to sample accurate in task. They showed increased activation in the left DLPFC
healthy control during highest cognitive load and in the VMPFC during control
condition
Bipolar disorder
Yurgelun- 14 BD subjects (12 under mood stabilizers; Faces: happy and fearful affect Compared with controls, BD subjects had reduced DLPFC activation
Todd et al. 13 under atypical and 1 under typical recognition paradigm and increased amygdala response to fearful facial affect
[65] antipsychotic medication) and 10 healthy
controls
Blumberg 36 BD subjects: 11 elevated mood state, Color-word Stroop task Compared with controls, elevated mood group showed small signal
et al. [66] 10 depressed mood state, 15 euthymic increase in right frontal cortex; and depressed mood group showed
mood state (13 unmedicated; 11 under large signal increase in the left frontal cortex. Regardless of the mood
lithium; 11 under anticonvulsants; 13 state, BD subjects showed increased activation in rostral region of left
under antidepressants; 3 under atypical VPFC
and 2 typical antipsychotic medication);
20 matched healthy controls
Adler et al. 12 euthymic BD patients (8 medicated Working Memory: Two-back BD patients performed more poorly than healthy controls. Compared
[67] with mood stabilizers and/or task, zero-back control/ with controls, BD patients had increased activation in fronto-polar
antipsychotic) and 10 healthy controls attention task prefrontal cortex, temporal cortex, basal ganglia, thalamus, and
posterior parietal cortex
Chang et al. 12 young (12–18 years old) male BD Working Memory: two-back task Compared with controls, BD subjects showed increased activations in
[68] subjects off medication for 24 h; 10 bilateral ACC, left putamen, left thalamus, left DLPFC, and right
age-matched healthy controls inferior frontal gyrus
Gruber et al. 14 BD patients on stable pharmacotherapy Stroop test Compared with controls, BD patients had reduced activations in the
[69] regimen (11 under mood stabilizers; 7 right subdivision of the ACC and increased activation in DLPFC
under antipsychotic medication; 4 under
antidepressants; 3 unmedicated) 10
healthy controls
Lawrence 12 euthymic BD patients (5 under SSRIs Faces: fear, happiness, and Compared with controls, BD patients had increased activations in the
et al. [70] medication; 5 under atypical sadness ventral striatal, thalamic, hippocampal, and ventral prefrontal cortical
antipsychotics; 9 under mood stabilizers) areas in response to intense fear, mild happiness, and mild sadness
fMRI in Psychiatric Disorders

9 major depressive disorder patients (all

under antidepressants), and 11 healthy
controls
(continued)
671
Table 3
(continued)
672

Authors Subjects fMRI paradigm Results

Malhi et al. 10 hypomanic-state female subjects with Negative, positive, or neutral Compared with controls, BD patients had increased activations in the
[71] BD (on mood-stabilizing psychotropic captioned pictures caudate and thalamus in response to negative-captioned pictures
medications only, with dosages unaltered
over the preceding 2 weeks), 10
matched healthy controls
Malhi et al. 10 depressed-state female subjects with BD Negative, positive or neutral Compared with controls, BD patients had increased activation in the
[72] (all under mood stabilizers with dosage captioned pictures amygdala, thalamus, hypothalamus, and medial globus pallidus in
Erin L. Habecker et al.

unaltered during the preceding 2 response to positive-captioned pictures

weeks), 10 matched healthy controls
Monks et al. 12 male BD subjects on lithium carbonate Working Memory: Two-back During the Two-back task, compared with controls, BD group had
[73] monotherapy and 12 healthy controls task and Sternberg test attenuated activity in bilateral frontal, temporal, and parietal regions;
increased activity in left precentral, right medial frontal and left
supramarginal gyri
Blumberg 17 BD patients (5 unmedicated; 8 under Faces: fear, happiness, sadness, Compared with controls, patients showed increased amygdala activation,
et al. [66] anticonvulsants; 3 under antidepressant; and neutral with greatest activations in unmedicated patients. Rostral ACC
4 under mood stabilizers; 1 under activations were attenuated in unmedicated patients as compared to
atypical antipsychotic) and 17 healthy healthy controls and medicated patients
controls
Frangou 7 euthymic BD subjects on monotherapy Working Memory: N-back task Compared with controls, BD patients exhibited decreased activation in
et al. [74] and seven healthy controls and Iowa Gambling task VPFC bilaterally and in the left DLPFC
Strakowski 16 euthymic BD patients and 16 healthy Stroop test Compared with controls, BD patients showed lower performance at the
et al. [75] controls task; greater activation in medial occipital cortex and reduced
activation in temporal cortical regions, middle frontal gyrus, putamen,
and midline cerebellum
Pavuluri 10 euthymic, unmedicated BD subjects Faces: angry, happy, and neutral Compared with controls, BD patients showed reduced activation in the
et al. [76] and 10 matched healthy controls right rostral VLPFC and increased activation in right pregenual ACC,
amygdala, and paralimbic cortex in response to angry and happy faces
Lagopoulos 10 euthymic BD patients (7 under mood Working memory Compared with controls, BD patients had slower reaction times and
et al. [77] stabilizers) and 10 healthy controls they exhibited attenuated or absent activation of frontal brain regions,
including the DLPFC, superior frontal gyri, ACC, and intraparietal
sulcus and increased activation in the inferior frontal gyrus during task
conditions

ACC anterior cingulate cortex, BD bipolar disorder, DLPFC dorsolateral prefrontal cortex, VPFC ventral prefrontal cortex
 fMRI in Psychiatric Disorders 673

administration of a serotonergic drug attenuated right amygdala

response to aversive faces [61].
Pre- and posttreatment studies of depressed subjects have posed
the question of whether clinical response to antidepressant drugs
can be predicted by indicators present at the baseline scan. One
study found that more positive activation in the ACC at the baseline
scan (in response to facial affect processing) was associated with
faster rates of symptom improvement as measured by Hamilton
Depression Rating Scale [79]. This finding needs to be replicated,
but clear differences between pre- and posttreatment have been
shown in the fMRI results: significantly attenuated activations after
treatment were seen in limbic-subcortical systems, including the
amygdala, that had shown enhanced activations in depressed sub-
jects at baseline in response to an emotional face processing task
[19]. In addition, as the treatment decreased this capacity for over-
activation in the limbic-subcortical region, a corresponding increase
was seen in prefrontal cortex activation in response to the highest
levels of affective load [19]. This could be explained by the fact that
treatment exposure induced changes in mood state have a proposed
association with reciprocal changes in limbic-subcortical systems and
frontoparietal circuitry: as limbic-subcortical regions activations to
sadness are selectively lowered by drug treatment, greater dynamic
range is available for high levels of affective load and increased acti-
vation in the prefrontal cortex is seen [17]. These results have great
implications for the future of fMRI studies in major depressive dis-
order: in theory, it should be possible in the years to come to use
quantitative measurements of brain function to determine optimal
treatment and predict treatment response patterns for a person pre-
senting with a major depressive episode, thereby increasing positive
outcomes and chances of eventual recovery [17].

4.1.1 fMRI Features As in schizophrenia, subjects at risk to develop major depression

of Subjects at Risk have been the focus of a number of fMRI studies in the field with
to Develop Major the main aim to emphasize the adverse impact of a positive family
Depression history (FH+) and/or early psychosocial stressors on cerebral vul-
nerability and risk for depression. In this perspective, in a sample of
120 adolescent girls at the age of 11 and 12 years, a study observed
that low parental warmth was associated with increased response to
potential rewards in the mPFC, striatum, and amygdala and with
increased depressive symptoms at the age of 16 years [80]. While
performing a similar task, compared with healthy controls, right-
sided ventral striatum activation was reduced in both currently
depressed and high-risk girls who were daughters of mothers
affected by major depression [81]. This ventral striatal activity cor-
related significantly with maternal depression scores suggesting
this as a vulnerable factor for major depression [81]. Another study
demonstrated an overactivity of the bilateral insula (also associated
with subject personality) in response to increasing executive and
674 Erin L. Habecker et al.

language task difficulty in FH+ subjects who developed major

depression 2 years later [82]. This pattern differentiated them from
healthy controls and from other individuals at high risk who did
not become unwell [82]. During an encoding task, FH+ subjects
showed an overrecruitment (likely reflecting a compensatory
mechanism for the task performance maintaining) of the dorsal
ACC, insula, and putamen, which are all regions involved in
processing the salience of stimuli [83]. While performing an emo-
tional face processing, never-depressed monozygotic twins with a
co-twin history of depression showed increased neural response in
dorsal ACC, dorsomedial PFC, and occipito-parietal regions; a
stronger negative coupling between the hyperactive regions and
amygdala; increased attention vigilance for fearful faces and slow-
ness at recognizing facial expressions compared to low-risk twins
(monozygotic twins without a co-twin history of depression) [84].

4.2 Bipolar Disorder BD, a prevalent neuropsychiatric illness manifesting as depressed

and manic episodes in affected individuals, is among the leading
worldwide causes of disability (DSM V, 2014). Bipolar depressed
patients exhibit symptomatology that overlaps with that of unipo-
lar depressed patients: feelings of despair, lack of motivation and
goal-setting behavior, social isolation, lethargy, and sleep distur-
bances. Bipolar patients in the manic state, meanwhile, experience
elevated mood, heightened energy levels, altered thought pro-
cesses, and sometimes irritability, while bipolar euthymic individu-
als show neither depressive or manic symptoms (DSM V, 2014).
FMRI studies coupled with analysis of bipolar clinical manifes-
tations have led to the hypothesis that, much like depression, the
mechanism of the disorder involves abnormalities in the limbic,
frontal, and subcortical cortical areas, perhaps caused by a dysfunc-
tion in the neural networks present in these regions [75, 85].
Anterior limbic networks, incorporating the amygdala, ACC,
DLPFC, and midline cerebellum, have been implicated in this dis-
order, as these areas contribute to behavioral functions seen to be
abnormal in individuals with BD [75]. Functional imaging studies
have been performed on bipolar patients in the euthymic state, the
manic state, and the depressed state to compare brain activations
across groups, and found comparatively increased activations in
prefrontal and dorsal ACC in all states that were not consistent
bilaterally [86]. There was evidence of a small increase in signal on
the right side of the ventral prefrontal cortex for patients in the
manic state, while patients in the depressed state showed much
greater signal increases as compared to healthy controls and manic
patients in the left ventral PFC [86]. This indicates that differential
signal changes across brain hemispheres may be connected with
the type of mood episode experienced by the bipolar individual.
Traditionally, two types of tasks have been used to generate neural
activations thought to be altered in BD: fronto-executive function
 fMRI in Psychiatric Disorders 675

cognitive studies and emotional processing studies [75, 85, 86], as

bipolar patients are known to have emotional regulatory impair-
ments and impaired cognitive control.
Cognitive tasks employed in fMRI study of BD include tests of
WM, interference tasks, encoding tasks, and other performance
related paradigms [85]. Lagopoulos et al. [77], noting that deficits
in WM seem to be of particular significance in bipolar individuals,
examined cortical activations in euthymic bipolar patients and
healthy controls during a parametric WM task with three load con-
ditions. As compared to the healthy subjects, bipolar patients
exhibited attenuation of activation across several frontal brain
regions. The DLPFC, which was activated across all WM condi-
tions in healthy controls, failed to activate under the same condi-
tions in bipolar patients, although these did not have significantly
poorer task performance. As the group noted that bipolar subjects
recruited the inferior frontal gyrus during all WM components,
while healthy subjects did not, it was theorized that this could rep-
resent a compensatory mechanism for normal DLPFC perfor-
mance [77]. This group also noted a failure of BD patients to
activate the parahippocampal gyrus during the delay condition. It
should be noted that these cognitive defects were observed in
euthymic bipolar patients, suggesting that BD-associated cognitive
defects are not restricted to state conditions of mania or depres-
sion. Similarly, Monks et al. [73] found that euthymic bipolar
patients on lithium therapy showed reduced activations bilaterally
in frontal, parietal, and temporal regions during two WM tasks,
coupled with increased activations as compared to the control
group in the left precentral, right medial frontal, and left supramar-
ginal gyri. This and similar data [74] suggest that fronto-executive
region function is compromised during WM tasks in bipolar
patients, which could lead to the recruitment of other neural areas
in task performance. Two other studies, also using WM, found
exaggerated task-induced activations in the left DLPFC, ACC, and
thalamus of bipolar patients [67, 68]. Unification of these data will
require further fMRI study with increased, heterogeneous sample
sizes, but all the studies consistently point to an underlying dys-
function in the region of the prefrontal-subcortical circuitry [68].
Stroop tasks have been utilized in several fMRI studies of BD
to examine variations in local activations induced by cognitive
interference. As mentioned above, Blumberg et al. [86] found
bilateral differences in the cortical activations of bipolar patients
that were state-dependent and significantly different from those of
healthy controls. In addition, Gruber et al. showed that stable
bipolar patients had reduced signal intensity in the right anterior
amygdaloid area (AAA) subdivision of the ACC, accompanied by
an increase in DLPFC activation during the interference condition
in what was hypothesized to be a compensatory manner [69]. In a
study of medicated bipolar subjects, unmedicated bipolar subjects,
676 Erin L. Habecker et al.

and healthy controls, both groups of patients were seen to exhibit

relatively increased activations as compared to the control group in
the medial occipital cortex, as well as reduced activations in the
temporal cortical regions, MFG, putamen, and midline cerebellum
[87]. These studies illustrate the various differences in neural struc-
tures involved in interference processing in the bipolar vs. healthy
brain, as well as differences in magnitude of MR signal intensity in
identical areas.
Emotional processing studies of BD involve the evocation of
transient mood reactions by presenting subjects in the scanner with
cues, such as charged facial expressions or auditory stimuli. Four
fMRI paradigms involving the presentation of some combination
of happy, fearful, sad, and neutral faces yielded varied results.
Lawrence et al. [70] noted increased subcortical and ventrolateral
PFC activation to all categories of emotional expression, as com-
pared to the healthy group and a group with major depressive dis-
order. Yurgelun-Todd et al. [65] found increased amygdala
activation in BD patients in response to fearful facial affect, accom-
panied by a reduction in DLPFC signal. Individuals with BD also
demonstrated an impaired ability to identify fearful faces as com-
pared to their ability to identify faces carrying positive emotion. An
increased amygdala response was also found by Blumberg et al.
[66], this time in response to happy faces, as well as decreased ros-
tral ACC activation in unmedicated BD patients–medicated BD
patients, meanwhile, exhibited an attenuation of emotional
response differences across the two groups, demonstrating that
mood-stabilizing medications have the ability to ameliorate
BD-induced functional abnormalities. Increased activity in the
right amygdala, right pregenual ACC, and paralimbic cortex was
noted by Pavuluri et al. [76] in pediatric BD in response to faces
displaying both a positive and a negative emotional state. Face
stimuli presentation to bipolar patients, then, appears to elicit a
range of abnormal frontotemporal responses, with consistent over-
activation of the amygdala seen across several studies.
State-dependent differences in brain activation in response to
emotional cues have been investigated using charged pictures as
well as facial affect paradigms. Malhi et al. [71] showed positive,
negative, and reference captioned pictures to hypomanic and
depressed female patients, finding that the hypomanic patients
restricted response to the negative-captioned pictures to subcorti-
cal regions while healthy controls displayed a more widespread pat-
tern of cortical activation. In depressed-state patients,
positive-captioned pictures significantly increased similar subcorti-
cal region reactions, including activations in the thalamus and
amygdala [72]. The depressed patients also showed relatively
increased right-side brain activity as compared to the healthy con-
trol group [72]. These results suggest that subcortical limbic sys-
tems are involved to a much greater extent in emotional processing
 fMRI in Psychiatric Disorders 677

in the bipolar hypomanic and the bipolar depressed individual, as

well as further illustrating the need to study euthymic, manic, and
depressed bipolar subjects as separate groups.
BD involves dysfunction in several key limbic and cortical net-
works, evidence for which is summarized above. One other addi-
tional feature reflected by BD fMRI research is the consistent
findings of abnormal PFC activations across state and trait boundar-
ies. Similar findings in major depressive disorder and schizophrenia
could indicate that the dysfunction caused by BD may share certain
underlying characteristics with other psychiatric disorders [85].

4.2.1 fMRI Features Given that the strongest risk factor for developing mania is a posi-
of Subjects at Risk tive family history (FH+) for BD [88], several studies have
to Develop Bipolar Disorder attempted to characterize the fMRI features in non symptomatic
subjects at risk using cognitive and emotional processing tasks. In
a motor inhibition task, compared to healthy controls and BD
patients, asymptomatic youths with a first-degree BD relative
exhibited increased activation of the putamen during unsuccessful
inhibition [89]. Compared with their low-risk peers, children
without disorders born from parents with BD showed aberrant
prefrontal neural responses to reward, aberrant connectivities
among reward-related regions, and neural correlates in mesolimbic
regions to novelty seeking and impulsive traits [90]. During a WM
task, compared to controls, both BD patients and non symptom-
atic FH+ subjects exhibited failure to suppress emotional arousal
and functional activity of the anterior insular and frontopolar cor-
tices [91]. While processing facial expressions, relative to HC, both
BD patients and FH+ subjects rated anger faces as less hostile and
they showed decreased modulation in the amygdala and inferior
frontal gyrus during anger face presentation [92]. Youth at
increased genetic risk for BD demonstrated reduced brain signal of
the left inferior frontal gyrus when inhibiting responses to fearful
face stimuli, compared with subjects from control families [93].

5 Anxiety Disorders

5.1 Obsessive- OCD is a complex and clinically heterogeneous disorder character-

Compulsive Disorder ized by obsessions (intrusive, unwanted, and repetitive thoughts),
compulsions (repetitive behaviors), or both. These dysfunctional
“solutions” represent the patient intention to diminish the levels of
anxiety in specific daily situations, however they cyclically lead to
always higher levels of distress (DSM V, 2014). The intensity of
symptoms is generally varied throughout a patient’s lifetime, but
complete and spontaneous remission is rare [94]. Some studies
have pointed to an association between different dimensions of the
disorder and different treatment responses: in particular, the
hoarding impulse has been associated with poorer behavioral and
678 Erin L. Habecker et al.

pharmacologic treatment response [95]. These findings illustrate

the heterogeneity of the disease; however, fMRI and other neuro-
imaging studies have tended to group together patients with a
range of symptoms, allowing for the delineation of underlying
neural mechanisms present across all categories.
Resting-state studies of functional neuroanatomy using PET
and SPECT have uncovered elements of the neurobiology of OCD
that should be taken into consideration when undertaking and
analyzing results of fMRI research. Specifically, examinations of
CBF in OCD patients during a resting state have revealed that
these patients exhibit increased metabolism in the OFC and head
of the caudate nucleus compared to healthy controls [94]. In addi-
tion, one study involving OCD patients with comorbid major
depression showed reduced CBF in the hippocampus, caudate, and
thalamus as compared to both the group of “pure” OCD patients
and the control group [96]. These resting state variations should
be taken into account when conducting BOLD research on this
disorder, as the sensitivity of fMRI to changes in brain metabolism
that are independent of task-evoked activation can be a significant
confounding factor [3]. Accordingly, fMRI techniques have been
used to investigate a number of states in patients with OCD
(Table 4): studies comparing local activations in the brains of OCD
patients and healthy controls during cognitive tasks; pre- and post-
treatment studies; and symptom-provocation studies during which
transient OCD-related anxiety symptoms are incited through
pictures or contact with “contaminated” objects [108, 109]. This
research has generally supported the involvement of frontal-
striatal-thalamic-cortical circuitry in OCD symptomatology, with
OCD patients demonstrating functional deviations from healthy
controls in the affected brain regions [98, 99, 108].
Cognitive challenge studies examine abnormal activations in
the brains of OCD patients as compared to healthy controls during
a variety of learning and inhibition control tasks. The proposal of
the frontal cortex and striatum as possible sites of dysfunction in the
disease suggests the use of tasks that have previously been found to
require processing by the frontal and subcortical systems during
performance by healthy subjects [110]. Using a Tower of London
task, van den Heuvel et al. [97] found decreased frontal-striatal
responsiveness in OCD patients as compared to the control group,
noting that this was accompanied by increased involvement of the
ACC, the ventrolateral PFC, and the parahippocampal cortex in a
possibly “compensatory” mechanism. Roth et al., using a response
inhibition “Go/no-go” task, demonstrated that the OCD group
had reduced activations in the right thalamus during response inhi-
bition [99], a finding consistent with a significant body of literature
reporting structural [111, 112] and functional [96, 113] thalamus
abnormalities in OCD patients. This study also reported a reduced
activation of the right OFC and dorsal cingulate gyrus in patients
Table 4
Summary of fMRI research in anxiety disorders, including posttraumatic stress disorder

Authors Subjects fMRI paradigm Results

Obsessive-compulsive disorder
van den Heuvel 22 unmedicated OCD patients and Tower of London Compared with healthy controls, OCD patients showed decreased frontal-
et al. [97] 22 healthy controls striatal responsiveness, mainly in dorsolateral prefrontal cortex and caudate
nucleus; and increased involvement of anterior cingulate, ventrolateral
prefrontal, and parahippocampal cortices
Remijnse et al. 20 unmedicated OCD patients and Reversal learning task Compared with healthy controls, patients with OCD showed reduced number of
[98] 27 healthy controls correct responses but similar responses to receipt of punishment and
demonstrated normal affective switching. During the task patients showed
reduced activations in the right medial and lateral OFC and in right caudate
nucleus. Patients recruited the left posterior OFC, bilateral insular cortex,
bilateral dorsolateral, and bilateral anterior prefrontal cortex to a lesser extent
than control subjects
Roth et al. [99] 12 adults with OCD (6 under SSRIs, Response inhibition: During response inhibition, healthy controls demonstrated right-hemisphere
and 6 had not taken any Go/No-go activation while the patient group showed a more diffuse and bilateral
psychotropic medication for at least pattern of activation. The OCD group had less activation than control group
6 weeks before scanning) and 14 during response inhibition in several right-hemisphere regions. Severe OCD
healthy control subjects symptoms were positively correlated with thalamic and posterior cortical
activations and inversely correlated with right OFC and anterior cingulate
gyri activations
Posttraumatic stress disorder
Rauch et al. 8 Vietnam combat veterans with Masked-fearful vs. Subjects with PTSD had an increased amygdala response to the masked-fearful
[100] PTSD and 8 Vietnam combat masked-happy faces faces as compared to control group and to the PTSD group’s responses to
veterans free from PTSD the masked-happy faces
Lanius et al. 9 traumatized subjects with PTSD, 9 Script-driven symptom Compared with controls, PTSD subjects showed reduced activation of the
[101] traumatized subjects without PTSD provocation thalamus, anterior cingulate gyrus, and medial frontal gyrus
Shin et al. [102] 8 Vietnam veterans with PTSD, 8 Emotional counting Compared with controls, PTSD group exhibited diminished response in rostral
fMRI in Psychiatric Disorders

Vietnam veterans without PTSD stroop task anterior cingulate cortex

(continued)
679
Table 4
(continued)

Authors Subjects fMRI paradigm Results

Hendler et al. 21 male veterans, 10 with PTSD (1 Parametric factorial Compared with veterans without PTSD, PTSD group showed increased
[103] unmedicated; 7 under design with combat activation in amygdala in response to all images and increased activation in
antidepressants; 1 under slides and visual cortex of PTSD group in response to combat content
antipsychotics; 1 under mood noncombat slides
stabilizers; 6 under
benzodiazepams,, and 11 (all
unmedicated) without
Lanius et al. 10 traumatized subjects with PTSD, Script-driven symptom Compared with controls, PTSD subjects had less activation of the thalamus and
[104] 10 traumatized subjects without provocation the anterior cingulate gyrus
PTSD
Driessen et al. 12 traumatized female patients with Autobiographical Subgroup without PTSD: predominant bilateral activation of OFC and Broca’s
[105] BPD, 6 with PTSD and 6 without recall of traumatic area. Subjects with PTSD: predominant activation of right anterior temporal
PTSD vs. negative but lobes, mesiotemporal areas, amygdala, posterior cingulate gyrus, occipital
nontraumatic events areas, and cerebellum
Protopopescu 11 patients with assault-related PTSD, Trauma and Compared with controls, PTSD patients had increased initial amygdala
et al. [106] 21 healthy controls nontrauma-related response to trauma-related emotional words and did not become habituated
emotional words to negative stimuli
Shin et al. [107] 13 traumatized subjects with PTSD, Emotional faces Compared with traumatized subjects without PTSD, the PTSD group
13 traumatized subjects without exhibited increased amygdala response and diminished medial prefrontal
PTSD cortex response to fearful facial expressions and decreased ability to habituate
right amygdala response to fearful faces
BPD borderline personality disorder, OCD obsessive-compulsive disorder, OFC orbitofrontal cortex, PTSD posttraumatic stress disorder
 fMRI in Psychiatric Disorders 681

with the most severe symptoms during response inhibition, while

another group also observed reduced activations in the right OFC
and in the right caudate nucleus during a reversal learning task
[98]. These findings support the involvement of the OFC, thala-
mus, and cortical circuitry in the abnormal patterns of response
inhibition that characterize OCD, as well as indicating that some of
the behavioral impairments associated with the disorder may be
attributed to dysfunction in this region [98].
Treatment studies in OCD incorporating fMRI to track local
activation changes across a course of medication have traditionally
combined symptom-provocation tasks with longitudinal study
designs [108]. The caudate nucleus has shown decreased glucose
metabolism following treatment with serotonin reuptake inhibi-
tors (SRIs) such as clomipramine and fluoxetine, suggesting that
the right anteriolateral OFC plays a role in the mediation of OCD
symptoms and response of OCD patients to pharmacotherapy
[114]. During symptom-provocation experiments, increased acti-
vations have been observed in the OFC, cingulate cortex, striatum,
thalamus, lateral PFC, amygdala, caudate, and insula among
unmedicated OCD patients [98, 114]. These results add support
to the theory that dysfunctions of the OFC and frontal-subcortical
circuitry are responsible for a great extent of OCD symptomatol-
ogy. One theory with the potential to unify a great deal of func-
tional imaging data to date involves the orbitofrontal-subcortical
circuitry, which has classically been described as having a “direct”
and an “indirect” pathway (Fig. 2) [115]. It has been hypothesized
that OCD symptoms could be caused by a captured signal in the
direct pathway creating a positive feedback loop and increasing
activity in the OFC, ventromedial caudate, and medial dorsal thala-
mus—leading to an excessive fixation on issues of hygiene, order,
danger, violence, and sex coupled with an inability to distract one-
self from these thoughts or change behavior patterns [114].
Interventions that alter and functional imaging experiments that
study the direct–indirect pathway balance within the orbitofrontal-
subcortical circuits would be particularly beneficial to the future of
OCD research, as they could directly test these theories and help
to advance the understanding of OFC functionality in the brains of
OCD patients [114].

5.1.1 fMRI Features Genetic epidemiological studies have revealed that OCD has a sig-
of Subjects at Risk nificant familial aggregation [116]. The aggregate risk in first-
to Develop Obsessive- degree relative of probands with OCD has been estimated at
Compulsive Disorder approximately 8–23 % [116]. As relatives of patients are at a signifi-
cantly higher risk of developing OCD symptoms than the general
population, young relatives at risk represent a valuable group to
examine potential neurobiological precursors of the disorder, how-
ever up to date few fMRI studies have been carried on in this popu-
lation. One of these studies observed that a WM increased task
682 Erin L. Habecker et al.

Fig. 2 The cortico-striatal model of obsessive-compulsive disorder (adapted from

ref. [115]). The striatum projects via direct and indirect pathways through the
globus pallidus to the thalamus, which, in turn, projects to the neocortex.
Excitatory connections are labeled “+”; inhibitory connections are labeled “−”

load was associated with increased task-related brain activity in

OCD and in their unaffected siblings compared with controls,
however this increase was smaller while reaching the highest task
load [117]. These findings indicate that compensatory frontopari-
etal brain activity in OCD patients and their unaffected relatives
preserves task performance at low task loads but is insufficient to
maintain performance at high task loads [117]. Authors concluded
that the frontoparietal dysfunction may constitute a neurocogni-
tive endophenotype for OCD [117]. In another study, patients
with OCD and their siblings performed a stop-signal task during
the MRI [118]. In this task, reaction times provide a behavioral
measure of response inhibition [118]. Compared with controls,
patients with OCD and their siblings showed greater activity in the
presupplementary motor area during (successful) inhibition [118].
The presupplementary motor area hyperactivity was negatively
correlated with stop-signal reaction time, likely representing an
OCD neurocognitive endophenotype which may contribute to
their inhibition deficit [118].

5.2 Posttraumatic PTSD is an anxiety disorder caused by the onset of an extreme

Stress Disorder stressor such as combat, childhood physical/sexual abuse, motor
vehicle accidents, rape, and natural disasters. Symptoms vary across
subtype of disorder and can include one or several of the following:
sleep disturbance with dreams characterized by negative emotions,
 fMRI in Psychiatric Disorders 683

intrusive memories, flashbacks, avoidance of traumatic stimuli,

numbing of emotions, and social dysfunction (DSM V, 2014).
There are four basic neural mechanisms that appear to function
abnormally in the brains of patients with PTSD: the fear response,
fear extinction, behavioral sensitization, and memory [119]. Fear
response in PTSD patients appears greatly exaggerated: in the nor-
mal brain, the pairing of potentially dangerous stimuli with fear is
an important survival mechanism, decreasing response time and
initiating fight or flight mechanisms when such a stimuli is pre-
sented. In the brain of a PTSD patient, it seems that there is an
overgeneralization of danger cues such that nonthreatening stimuli
is seen as dangerous and can then be linked to past traumatic mem-
ories, which come hand-in-hand with intrusive memories and
flashbacks. As a result of such nonthreatening yet triggering stim-
uli, patients with PTSD often exhibit avoidance of such stimuli or
numbing of emotional reactions [120]. In functional neuroimag-
ing studies of PTSD, symptom-provocation paradigms measure
brain activity when subjects are exposed to visual or auditory stim-
ulation reminiscent of their past experienced trauma to determine
the mechanism of the abnormal response [2]. The main structure
involved in the response to fearful stimuli is the central nucleus of
the amygdala, with additional involvement by the sensory cortex,
thalamus, and the mPFC [120]. PTSD patients also often exhibit
a failure of fear extinction: normally, the brain is able to process
stimuli from a dangerous situation from which there were no
adverse outcomes such that the representation of these stimuli elic-
its a smaller fear response than the initial situation. However, in
patients with PTSD, repeated encounters with fearful stimuli can
continue to result in a consistent, heightened fear response, regard-
less of the actual danger of the situation [2]. The main neuroana-
tomical structures involved in the extinction of fear response
overlap with those involved in fear conditioning, and their func-
tionality may be studied concurrently [121].
PTSD patients often have increased sensitivity to stress, which
leads to an increase in responses such as arousal and vigilance in
response to stressful stimuli [119] known as behavioral sensitization.
The neuroanatomy of the stress response is not centralized but
involves a wide range of structures and mechanisms—many of which
overlap with those involved in fear conditioning and fear response
[122]. Many of those afflicted with PTSD also exhibit memory defi-
ciencies thought to be connected to hippocampal function and
reduction in hippocampal volume [123]. It is unclear whether a
stressful event leading to PTSD causes volume reductions in the hip-
pocampus through exposure to elevated glucocorticoids accompa-
nied by a reduction of brain-derived neurotrophic factor [124, 125],
or if persons with reduced hippocampal volume from birth are sim-
ply more prone to developing PTSD [126]. However, hippocampal
volume and functionality is an important area of PTSD study.
684 Erin L. Habecker et al.

FMRI research into PTSD has focused on the examination of

the brain structures outlined above using symptom-provocation
paradigms to examine fear response and stress sensitivity, as well as
cognitive task paradigms to investigate memory deficiencies and
other cognitive problems potentially associated with the disorder.
It has been theorized that neurocircuitry links the amygdala to the
mPFC, the hippocampus, and the thalamus through excitatory and
inhibitory connections that are dysfunctional in PTSD [115]
(Fig. 3). Consistent with this, fMRI studies have found alterations
in BOLD signal in OFC, ACC, anterior temporal cortex, and
amygdala [100, 127], as well as in the hippocampus, parahippo-
campus, and thalamus [2, 128, 129]. Symptom-provocation stud-
ies find exaggerated amygdala responses and decreased activation
within medial frontal areas [101, 130, 131]. Rauch et al. [100]
used an fMRI paradigm incorporating a happy vs. fearful faces task
in healthy combat veterans and combat veterans suffering from
PTSD. The study found increased activation of the amygdala in the
PTSD group in response to fearful faces, which could be positively
correlated with PTSD symptom severity. Similarly, Hendler et al.
[103] presented combat and noncombat related slides to PTSD
and non-PTSD Israeli soldiers, and found that activity in the amyg-
dala was significantly increased, although their results demonstrated

Fig. 3 The amygdalocentric neurocircuitry model of posttraumatic stress disor-

der (adapted from ref. [115]). Excitatory connections are labeled “+”; inhibitory
connections are labeled “−”
 fMRI in Psychiatric Disorders 685

this increased activity regardless of whether the slide presented was

traumatic (combat-related) or nontraumatic (noncombat-related).
One study presenting autobiographical cues to borderline person-
ality disorder patients with and without PTSD found increased
activation in the amygdala in the group with PTSD only [105],
and another showed increased amygdala activation in response to
traumatic stimuli correlating with PTSD symptom severity [106].
The inappropriate fear responses associated with PTSD may be
linked to dysfunction of the mPFC, as well as the amygdala. The
mPFC is associated with fear response and fear extinction [132–
134]. A number of studies have reported diminished activations in
the mPFC as compared to groups of healthy control subjects [101,
102, 104, 107, 135]. These abnormal activations have been
reported during such symptom-provocation tasks as traumatic nar-
ratives, combat-related stimuli, emotional word tasks, and emo-
tional Stroop tasks. For example, using an autobiographical script,
two studies by Lanius et al. [101, 104] found reduced activity as
compared to healthy controls in both the mPFC and thalamus of
PTSD subjects, suggesting an alteration of normal neurocircuitry in
those regions. In a fMRI paradigm incorporating a happy vs. fearful
faces task, Shin et al. [107] noted that the observed decrease in
mPFC activity in the PTSD group as compared to controls was
accompanied by increased amygdala activations. The theory has
been advanced that PTSD symptoms are caused by an overactive
amygdala in PTSD patients accompanied by a failure of the mPFC,
including the ACC, and the hippocampus to inhibit this abnormal
activation [136–139]. The fact that the functionality and structure
of the hippocampus is also often altered in this disorder contributes
evidence to this hypothesis. In one study combining fMRI and PET
with verbal declarative memory tasks, the hippocampus of PTSD
patients failed to activate entirely during the same tasks that caused
activations in two control groups [125]. Other PET studies have
noted hypoactivations of the hippocampus during memory tasks
[140, 141], but one other fMRI study found increased hippocam-
pal activation during a Stroop task [78], and another PET study
noted increased resting state blood flow in the hippocampus and
parahippocampal gyrus that was positively correlated with symptom
severity [142]. Overall, reduced volume and increased resting blood
flow in the hippocampus have been seen in many PTSD patients
during both PET and fMRI studies. More study is needed to bring
a greater degree of consistency to the imaging data with regards to
this structure, incorporating subjects that are consistent for age,
gender, and subtype of PTSD (primarily manifesting as flashback
symptoms or primarily involving dissociation) [104].
The majority of functional PTSD fMRI studies to date provide
evidence for hyperactivation of the amygdala coupled with a relative
decrease in mPFC activity. It is hypothesized that the hippocam-
pus, parahippocampus, and thalamus also represent participatory
686 Erin L. Habecker et al.

areas and that dysfunction in regional neurocircuitry leads to the

symptoms of the disorder [115]. More research with larger subject
groups and standardized study guidelines is needed to unify the
fMRI data and uncover more consistent trends in PTSD imaging
research.

5.2.1 fMRI Features Only one study so far investigated the fMRI features related to the
of Subjects at Risk risk to develop PTSD [143]. Authors assessed the fMRI differ-
to Develop Posttraumatic ences between combat-exposed veterans with PTSD and their
Stress Disorder identical combat-unexposed co-twins vs. combat-exposed veterans
without PTSD and their identical combat-unexposed co-twins.
During a Multi-Source Interference Task, combat-exposed veter-
ans with PTSD and their unexposed co-twins had significantly
greater activation in the dorsal ACC and tended to have larger
response time difference scores, as compared to combat-exposed
veterans without PTSD and their co-twins [143]. This cerebral
activation in the unexposed twins was positively correlated with
their combat exposed co-twins’ PTSD symptom severity [143]
leading authors to conclude that hyperresponsivity in the dorsal
ACC appears to be a familial risk factor for the development of
PTSD following psychological trauma [143].

6 Summary and Future Directions

Table 5 summarizes the overall neural activation differences between

affected psychiatric patients and healthy subjects during a sampling
of common fMRI paradigms designed to test the normality of the
patients’ emotional, memory, inhibitory, learning, language, and
executive functionality. As expected, decreased activity in cortical
regions is common in the diseased brain, but neural structure over-
activity is just as prevalent in certain disorders. Many researchers
have theorized about the possibility of secondary effects, whereby
the functional deficits that underlie the symptomatology of certain
disorders are compensated for by involvement of accessory struc-
tures or overactivation in another area [17, 22]. Clearly, the effects
of these disorders cannot be summarized by a single regional deficit,
and current fMRI research is moving toward investigation of neural
circuitry and abnormal systemic interactions. It is hoped that the
documentation of these disorders’ underlying dysfunctionality will
yield common threads that connect the ranges of heterogeneous
symptoms and indicate new clinical strategies. However, the use of
fMRI as a diagnostic imaging method in psychiatric practice has thus
far been limited. Although some studies are beginning to hone in on
early indicators of disease, as well as the potential treatment response
of the newly diagnosed, clinical method development is hampered
by cost, complexity of typical research paradigms and protocols, and
the known and unknown effects of drug treatments of CBF [4].
Table 5
Summary of fMRI paradigms and protocols in psychiatric disorders

Stimulation
Function paradigms Observed activation differences in affected group Disorder
Emotional Evocation of happy ↓ Anterior cingulate Schizophrenia
processing and sad mood
↓ Amygdala–hippocampal complex (Hempel et al. [22])
(autobiographical
↓ Amygdala (Habel et al. [24], Williams et al. [28])
scripts, positive,
↓ Right amygdala
and negative
↓ mPFC (Takahashi et al. [27])
words, etc.),
↑ Left amygdala Major depressive disorder
affective pictures,
↑ Ventral striatum
facial expressions
↑ Left hippocampus, insula, caudate nucleus, thalamus, dorsal cingulate gyrus, inferior
parietal cortex (Fu et al. [19])
↑ Amygdala (Siegle et al. [58, 59])
↓ DLPFC (Yurgulen-Todd et al. [65]) Bipolar disorder
↑ Amygdala (Yurgulen-Todd et al. [65], Malhi et al. [72], Blumberg et al. [66],
Pavuluri et al. [76])
↑ Thalamus (Lawrence et al. [70], Malhi et al. [71, 72])
↑ Ventral striatum, ventral PFC
↑ Hippocampus (Lawrence et al. [70])
↑ Caudate nucleus (Malhi et al. [71])
↑ Hypothalamus, medial globus pallidus (Malhi et al. [72])
↓ Rostral anterior cingulate (Blumberg et al. [66])
↓ Right rostral ventrolateral PFC (Pavuluri et al. [76])
↑ Right pregenual anterior cingulate, paralimbic cortex (Pavuluri et al. [76])
↑ Amygdala (Rauch et al. [100], Hendler et al. [103], Driessen et al. [105], Posttraumatic stress disorder
Protopopescu et al. [106], Shin et al. [107])
↓ Thalamus, anterior cingulate gyrus (Lanius et al. [104])
↑ Right anterior temporal lobes, mesiotemporal areas, posterior cingulate gyrus,
occipital areas, cerebellum (Driessen et al. [105])
fMRI in Psychiatric Disorders

↓ Medial prefrontal cortex (Shin et al. [107])

(continued)
687
Table 5
688

(continued)

Stimulation
Function paradigms Observed activation differences in affected group Disorder
Working Word encoding/ ↑ Basal ganglia Schizophrenia
memory/ recognition,
↑ Thalamus (Manoach et al. [20])
attention delayed match to
↑ Parahippocampus (Ragland et al. [26])
sample, Sternberg
↓ Prefrontal cortex (Ragland et al. [26], Morey et al. [30])
test, N-back,
Erin L. Habecker et al.

↓ Anterior cingulate cortex

continuous
↓ Cerebellum (Honey et al. [29])
performance
↑ Left DLPFC Major depressive disorder
↑ Ventromedial PFC (Walter et al. [64])
↓ Left DLPFC (Rubia et al. [21], Chang et al. [68], Frangou et al. [74]) Bipolar disorder
↑ Fronto-polar PFC, temporal cortex, temporal cortex, posterior parietal cortex
↑ Thalamus, basal ganglia (Adler et al. [67])
↑ Bilateral ACC, left thalamus, left putamen, right inferior frontal gyrus (Chang et al. [68])
↓ Frontal, temporal, parietal regions (Monks et al. [73])
↑ Left precentral, right medial frontal, left supramarginal gyri (Monks et al. [73])
↓ Bilateral VPFC (Frangou et al. [74])
↓ DLPFC (Lagopoulos et al. [77])
↑ Inferior frontal gyrus (Lagopoulos et al. [77])
Inhibitory Go/No-go, stop ↑ Rostral anterior cingulate gyrus Major depressive disorder
control cues, Stroop task
↑ Left DLPFC (Wagner et al. [62])
↑ Rostral left VPFC Bipolar disorder
↑ Left frontal cortex (depressed mood group)
↑ Right frontal cortex (elevated mood group) (Blumberg et al. [86])
↓ ACC (Gruber et al. [69])
↑ DLPFC (Gruber et al. [69])
↓ Temporal cortical regions, medial frontal gyrus, putamen, midline cerebellum
(Strakowski et al. [87])
↑ Medial occipital cortex (Strakowski et al. [87])
↓ Right inferior and medial frontal gyri Obsessive-compulsive disorder
↓ Right orbitofrontal cortex, dorsal anterior cingulate gyrus (Roth et al. [99])
↓ Rostral anterior cingulate cortex (Shin et al. [102]) Posttraumatic stress disorder
 Stimulation
Function paradigms Observed activation differences in affected group Disorder
Executive Digit sorting, ↓ DLPFC (Siegle et al. [59]) Major depressive disorder
function Tower of London ↓ DLPFC Obsessive-compulsive
↓ Caudate nucleus (van den Heuvel et al. [97]) disorder
↑ Anterior cingulate, ventrolateral PFC, parahippocampal cortex (van den Heuvel
et al. [97])
Episodic Episodic encoding/ ↓ Bilateral DLPFC Schizophrenia
memory recognition ↓ Lateral temporal cortex (Hofer et al. [23])
Language Word fluency, letter ↓ Temporal lobe (Yurgelun-Todd et al. [85]) Schizophrenia
processing fluency
Reward and Loss/No loss, ↓ Ventral striatum (Juckel et al. [32]) Schizophrenia
punishment reward/no
reward, incentive
monetary tasks
Learning Picture encoding, ↓ Left inferior PFC Schizophrenia
semantic ↑ Left superior temporal gyrus (Kubicki et al. [24])
encoding
Affective Reversal learning ↓ Right medial and right lateral OFC, right caudate nucleus Obsessive-compulsive
switching task ↓ Left posterior OFC, insular cortex, DLPFC, anterior PFC (Remijnse et al. [98]) disorder

ACC anterior cingulate cortex, DLPFC dorsolateral prefrontal cortex, VPFC ventral prefrontal cortex
fMRI in Psychiatric Disorders
689
690 Erin L. Habecker et al.

An important future direction for fMRI research into these dis-

orders is the development of simple paradigms that can be relied
upon to produce distinct and consistent patterns of activation in
healthy and affected brains. In addition, the integration of current
trends in clinical and molecular genetics, cellular biology, and social
health factors such as population vulnerability factors and life events
into the study of these disorders has the potential to delineate disease
causality, risk factors, and course to an increasingly accurate extent.

References
1. Brown GG, Perthen JE, Liu TT, Buxton RB in PaCO2 on cerebral blood volume, blood
(2007) A primer on functional magnetic reso- flow, and vascular mean transit time. Stroke
nance imaging. Neuropsychol Rev 17(2): 5(5):630–639
107–125 11. Reiman EM, Raichle ME, Robins E, Butler
2. Francati V, Vermetten E, Bremner JD (2007) FK, Herscovitch P, Fox P, Perlmutter J
Functional neuroimaging studies in posttrau- (1986) The application of positron emission
matic stress disorder: review of current methods tomography to the study of panic disorder.
and findings. Depress Anxiety 24(3):202–218 Am J Psychiatry 143(4):469–477
3. Giardino ND, Friedman SD, Dager SR 12. Aguirre GK, Detre JA, Wang J (2005)
(2007) Anxiety, respiration, and cerebral Perfusion fMRI for functional neuroimaging.
blood flow: implications for functional brain Int Rev Neurobiol 66:213–236
imaging. Compr Psychiatry 48(2):103–112 13. Liu TT, Brown GG (2007) Measurement
4. Yurgelun-Todd DA, Renshaw PF, Femia LA of cerebral perfusion with arterial spin label-
(2006) Applications of fMRI to psychiatry. ing: Part 1. Methods. J Int Neuropsychol
In: Faro SH, Mohamed FB (eds) Functional Soc 13(3):517–525. doi:10.1017/
MRI: basic principles and clinical applica- S1355617707070646
tions. Springer, New York, pp 183–220 14. Rao SM, Salmeron BJ, Durgerian S, Janowiak
5. Lai S, Hopkins AL, Haacke EM, Li D, JA, Fischer M, Risinger RC, Conant LL,
Wasserman BA, Buckley P, Friedman L, Stein EA (2000) Effects of methylpheni-
Meltzer H, Hedera P, Friedland R (1993) date on functional MRI blood-oxygen-
Identification of vascular structures as a major level-dependent contrast. Am J Psychiatry
source of signal contrast in high resolution 157(10):1697–1699
2D and 3D functional activation imaging of 15. Mildner T, Zysset S, Trampel R, Driesel W,
the motor cortex at 1.5T: preliminary results. Moller HE (2005) Towards quantifica-
Magn Reson Med 30(3):387–392 tion of blood-flow changes during cognitive
6. Saad ZS, Ropella KM, DeYoe EA, Bandettini task activation using perfusion-based fMRI.
PA (2003) The spatial extent of the BOLD Neuroimage 27(4):919–926
response. Neuroimage 19(1):132–144 16. Tjandra T, Brooks JCW, Figueiredo P, Wise R,
7. Ide K, Eliasziw M, Poulin MJ (2003) Matthews PM, Tracey I (2005) Quantitative
Relationship between middle cerebral artery assessment of the reproducibility of functional
blood velocity and end-tidal PCO2 in the activation measured with BOLD and MR
hypocapnic-hypercapnic range in humans. perfusion imaging: implications for clinical
J Appl Physiol (1985) 95(1):129–137 trial design. Neuroimage 27(2):393–401
8. Poulin MJ, Liang PJ, Robbins PA (1996) 17. Mayberg HS (2003) Modulating dysfunc-
Dynamics of the cerebral blood flow tional limbic-cortical circuits in depression:
response to step changes in end-tidal PCO2 towards development of brain-based algo-
and PO2 in humans. J Appl Physiol (1985) rithms for diagnosis and optimised treatment.
81(3):1084–1095 Br Med Bull 65:193–207
9. Rostrup E, Knudsen GM, Law I, Holm S, 18. Deckersbach T, Dougherty DD, Rauch SL
Larsson HBW, Paulson OB (2005) The rela- (2006) Functional imaging of mood and anx-
tionship between cerebral blood flow and vol- iety disorders. J Neuroimaging 16(1):1–10
ume in humans. Neuroimage 24(1):1–11 19. Fu CHY, Williams SCR, Cleare AJ, Brammer
10. Grubb RL, Raichle ME, Eichling JO, Ter- MJ, Walsh ND, Kim J, Andrew CM, Pich EM,
Pogossian MM (1974) The effects of changes Williams PM, Reed LJ, Mitterschiffthaler MT,
 fMRI in Psychiatric Disorders 691

Suckling J, Bullmore ET (2004) Attenuation 29. Honey GD, Honey RAE, O’Loughlin C,
of the neural response to sad faces in major Sharar SR, Kumaran D, Suckling J, Menon
depression by antidepressant treatment: a DK, Sleator C, Bullmore ET, Fletcher PC
prospective, event-related functional mag- (2005) Ketamine disrupts frontal and hippo-
netic resonance imaging study. Arch Gen campal contribution to encoding and retrieval
Psychiatry 61(9):877–889 of episodic memory: an fMRI study. Cereb
20. Manoach DS, Gollub RL, Benson ES, Cortex 15(6):749–759
Searl MM, Goff DC, Halpern E, Saper CB, 30. Morey RA, Inan S, Mitchell TV, Perkins DO,
Rauch SL (2000) Schizophrenic subjects Lieberman JA, Belger A (2005) Imaging fron-
show aberrant fMRI activation of dorso- tostriatal function in ultra-high-risk, early, and
lateral prefrontal cortex and basal ganglia chronic schizophrenia during executive pro-
during working memory performance. Biol cessing. Arch Gen Psychiatry 62(3):254–262
Psychiatry 48(2):99–9109 31. Yurgelun-Todd DA, Coyle JT, Gruber SA,
21. Rubia K, Russell T, Bullmore ET, Soni W, Renshaw PF, Silveri MM, Amico E, Cohen
Brammer MJ, Simmons A, Taylor E, Andrew B, Goff DC (2005) Functional magnetic
C, Giampietro V, Sharma T (2001) An fMRI resonance imaging studies of schizophrenic
study of reduced left prefrontal activation in patients during word production: effects of
schizophrenia during normal inhibitory func- D-cycloserine. Psychiatry Res 138(1):23–31
tion. Schizophr Res 52(1–2):47–55 32. Juckel G, Schlagenhauf F, Koslowski M,
22. Hempel A, Hempel E, Schonknecht P, Filonov D, Wustenberg T, Villringer A,
Stippich C, Schroder J (2003) Impairment Knutson B, Kienast T, Gallinat J, Wrase J,
in basal limbic function in schizophrenia Heinz A (2006) Dysfunction of ventral stria-
during affect recognition. Psychiatry Res tal reward prediction in schizophrenic patients
122(2):115–124 treated with typical, not atypical, neuroleptics.
23. Hofer A, Weiss EM, Golaszewski SM, Psychopharmacology (Berl) 187(2):222–228
Siedentopf CM, Brinkhoff C, Kremser C, 33. Vink M, Ramsey NF, Raemaekers M, Kahn
Felber S, Fleischhacker WW (2003) An FMRI RS (2006) Striatal dysfunction in schizophre-
study of episodic encoding and recognition of nia and unaffected relatives. Biol Psychiatry
words in patients with schizophrenia in remis- 60(1):32–39
sion. Am J Psychiatry 160(5):911–918 34. Braver TS, Barch DM, Kelley WM, Buckner
24. Kubicki M, McCarley RW, Nestor PG, RL, Cohen NJ, Miezin FM, Snyder AZ,
Huh T, Kikinis R, Shenton ME, Wible CG Ollinger JM, Akbudak E, Conturo TE,
(2003) An fMRI study of semantic process- Petersen SE (2001) Direct comparison of
ing in men with schizophrenia. Neuroimage prefrontal cortex regions engaged by working
20(4):1923–1933 and long-term memory tasks. Neuroimage
25. Habel U, Klein M, Shah NJ, Toni I, Zilles 14(1 Pt 1):48–59
K, Falkai P, Schneider F (2004) Genetic load 35. Cohen NJ, Ryan J, Hunt C, Romine L,
on amygdala hypofunction during sadness Wszalek T, Nash C (1999) Hippocampal
in nonaffected brothers of schizophrenia system and declarative (relational) memory:
patients. Am J Psychiatry 161(10):1806–1813 summarizing the data from functional neuro-
26. Ragland JD, Gur RC, Valdez J, Turetsky BI, imaging studies. Hippocampus 9(1):83–98
Elliott M, Kohler C, Siegel S, Kanes S, Gur 36. Nyberg L, Marklund P, Persson J, Cabeza
RE (2004) Event-related fMRI of fronto- R, Forkstam C, Petersson KM, Ingvar M
temporal activity during word encoding and (2003) Common prefrontal activations dur-
recognition in schizophrenia. Am J Psychiatry ing working memory, episodic memory, and
161(6):1004–1015 semantic memory. Neuropsychologia 41(3):
27. Takahashi H, Koeda M, Oda K, Matsuda T, 371–377
Matsushima E, Matsuura M, Asai K, Okubo 37. Klosterkotter J, Schultze-Lutter F, Bechdolf
Y (2004) An fMRI study of differential neural A, Ruhrmann S (2011) Prediction and pre-
response to affective pictures in schizophre- vention of schizophrenia: what has been
nia. Neuroimage 22(3):1247–1254 achieved and where to go next? World
28. Williams LM, Das P, Harris AWF, Liddell BB, Psychiatry 10(3):165–174
Brammer MJ, Olivieri G, Skerrett D, Phillips 38. Yung AR, Phillips LJ, Yuen HP, McGorry PD
ML, David AS, Peduto A, Gordon E (2004) (2004) Risk factors for psychosis in an ultra
Dysregulation of arousal and amygdala- high-risk group: psychopathology and clinical
prefrontal systems in paranoid schizophrenia. features. Schizophr Res 67(2–3):131–142.
Am J Psychiatry 161(3):480–489 doi:10.1016/S0920-9964(03)00192-0
692 Erin L. Habecker et al.

39. Yung AR, Phillips LJ, Yuen HP, Francey 48. Rausch F, Mier D, Eifler S, Esslinger C,
SM, McFarlane CA, Hallgren M, McGorry Schilling C, Schirmbeck F, Englisch S, Meyer-
PD (2003) Psychosis prediction: 12-month Lindenberg A, Kirsch P, Zink M (2014)
follow up of a high-risk (“prodromal”) Reduced activation in ventral striatum and
group. Schizophr Res 60(1):21–32, ventral tegmental area during probabilistic
S0920996402001676 [pii] decision-making in schizophrenia. Schizophr
40. Falkenberg I, Chaddock C, Murray RM, Res 156(2–3):143–149. doi:10.1016/j.
McDonald C, Modinos G, Bramon E, Walshe schres.2014.04.020
M, Broome M, McGuire P, Allen P (2015) 49. Chung YS, Kang DH, Shin NY, Yoo SY,
Failure to deactivate medial prefrontal cor- Kwon JS (2008) Deficit of theory of mind
tex in people at high risk for psychosis. Eur in individuals at ultra-high-risk for schizo-
Psychiatry 30(5):633–640. doi:10.1016/j. phrenia. Schizophr Res 99(1–3):111–118.
eurpsy.2015.03.003 doi:10.1016/j.schres.2007.11.012
41. Karlsgodt KH, van Erp TG, Bearden CE, 50. Marjoram D, Job DE, Whalley HC,
Cannon TD (2014) Altered relationships Gountouna VE, McIntosh AM, Simonotto E,
between age and functional brain activation in Cunningham-Owens D, Johnstone EC, Lawrie
adolescents at clinical high risk for psychosis. S (2006) A visual joke fMRI investigation into
Psychiatry Res 221(1):21–29. doi:10.1016/j. Theory of Mind and enhanced risk of schizo-
pscychresns.2013.08.004 phrenia. Neuroimage 31(4):1850–1858.
42. Lord LD, Allen P, Expert P, Howes O, doi:10.1016/j.neuroimage.2006.02.011
Broome M, Lambiotte R, Fusar-Poli P, 51. Drevets WC (2000) Neuroimaging studies of
Valli I, McGuire P, Turkheimer FE (2012) mood disorders. Biol Psychiatry 48(8):813–829
Functional brain networks before the onset 52. Mayberg HS (1997) Limbic-cortical dys-
of psychosis: a prospective fMRI study with regulation: a proposed model of depression.
graph theoretical analysis. NeuroImage Clin J Neuropsychiatry Clin Neurosci 9(3):471–481
1(1):91–98. doi:10.1016/j.nicl.2012.09.008 53. Mayberg HS, Liotti M, Brannan SK,
43. Pulkkinen J, Nikkinen J, Kiviniemi V, Maki McGinnis S, Mahurin RK, Jerabek PA, Silva
P, Miettunen J, Koivukangas J, Mukkala JA, Tekell JL, Martin CC, Lancaster JL, Fox
S, Nordstrom T, Barnett JH, Jones PB, PT (1999) Reciprocal limbic-cortical func-
Moilanen I, Murray GK, Veijola J (2015) tion and negative mood: converging PET
Functional mapping of dynamic happy and findings in depression and normal sadness.
fearful facial expressions in young adults with Am J Psychiatry 156(5):675–682
familial risk for psychosis—Oulu brain and 54. Baxter LR, Schwartz JM, Phelps ME,
mind study. Schizophr Res 164(1–3):242– Mazziotta JC, Guze BH, Selin CE, Gerner
249. doi:10.1016/j.schres.2015.01.039 RH, Sumida RM (1989) Reduction of pre-
44. Hart SJ, Bizzell J, McMahon MA, Gu H, frontal cortex glucose metabolism common to
Perkins DO, Belger A (2013) Altered fronto- three types of depression. Arch Gen Psychiatry
limbic activity in children and adolescents 46(3):243–250
with familial high risk for schizophrenia. 55. Bench CJ, Friston KJ, Brown RG, Scott
Psychiatry Res 212(1):19–27. doi:10.1016/j. LC, Frackowiak RS, Dolan RJ (1992) The
pscychresns.2012.12.003 anatomy of melancholia—focal abnormalities
45. Li HJ, Chan RC, Gong QY, Liu Y, Liu SM, Shum of cerebral blood flow in major depression.
D, Ma ZL (2012) Facial emotion processing in Psychol Med 22(3):607–615
patients with schizophrenia and their non-psy- 56. Drevets WC, Price JL, Simpson JR, Todd
chotic siblings: a functional magnetic resonance RD, Reich T, Vannier M, Raichle ME
imaging study. Schizophr Res 134(2–3):143– (1997) Subgenual prefrontal cortex
150. doi:10.1016/j.schres.2011.10.019 abnormalities in mood disorders. Nature
46. Barbour T, Pruitt P, Diwadkar VA (2012) 386(6627):824–827
fMRI responses to emotional faces in chil- 57. Liotti M, Mayberg HS, Brannan SK, McGinnis
dren and adolescents at genetic risk for psy- S, Jerabek P, Fox PT (2000) Differential lim-
chiatric illness share some of the features of bic–cortical correlates of sadness and anxiety
depression. J Affect Disord 136(3):276–285. in healthy subjects: implications for affective
doi:10.1016/j.jad.2011.11.036 disorders. Biol Psychiatry 48(1):30–42
47. Habel U, Chechko N, Pauly K, Koch K, 58. Siegle GJ, Steinhauer SR, Thase ME, Stenger
Backes V, Seiferth N, Shah NJ, Stocker T, VA, Carter CS (2002) Can’t shake that
Schneider F, Kellermann T (2010) Neural feeling: event-related fMRI assessment of sus-
correlates of emotion recognition in schizo- tained amygdala activity in response to emo-
phrenia. Schizophr Res 122(1–3):113–123. tional information in depressed individuals.
doi:10.1016/j.schres.2010.06.009 Biol Psychiatry 51(9):693–707
 fMRI in Psychiatric Disorders 693

59. Siegle GJ, Thompson W, Carter CS, 69. Gruber SA, Rogowska J, Yurgelun-Todd DA
Steinhauer SR, Thase ME (2007) Increased (2004) Decreased activation of the anterior
amygdala and decreased dorsolateral pre- cingulate in bipolar patients: an fMRI study.
frontal BOLD responses in unipolar depres- J Affect Disord 82(2):191–201
sion: related and independent features. Biol 70. Lawrence NS, Williams AM, Surguladze
Psychiatry 61(2):198–209 S, Giampietro V, Brammer MJ, Andrew C,
60. Surguladze SA, Young AW, Senior C, Brebion Frangou S, Ecker C, Phillips ML (2004)
G, Travis MJ, Phillips ML (2004) Recognition Subcortical and ventral prefrontal cortical
accuracy and response bias to happy and sad neural responses to facial expressions distin-
facial expressions in patients with major depres- guish patients with bipolar disorder and major
sion. Neuropsychology 18(2):212–218 depression. Biol Psychiatry 55(6):578–587
61. Del-Ben CM, Deakin JF, McKie S, Delvai NA, 71. Malhi GS, Lagopoulos J, Sachdev P, Mitchell
Williams SR, Elliott R, Dolan M, Anderson IM PB, Ivanovski B, Parker GB (2004) Cognitive
(2005) The effect of citalopram pretreatment generation of affect in hypomania: an
on neuronal responses to neuropsychological fMRI study. Bipolar Disord 6(4):271–285.
tasks in normal volunteers: an FMRI study. doi:10.1111/j.1399-5618.2004.00123.x
Neuropsychopharmacology 30(9):1724– 72. Malhi GS, Lagopoulos J, Ward PB, Kumari
1734. doi:10.1038/sj.npp.1300728 V, Mitchell PB, Parker GB, Ivanovski B,
62. Wagner G, Sinsel E, Sobanski T, Kohler S, Sachdev P (2004) Cognitive generation of
Marinou V, Mentzel H-J, Sauer H, Schlosser affect in bipolar depression: an fMRI study.
RGM (2006) Cortical inefficiency in patients Eur J Neurosci 19(3):741–754
with unipolar depression: an event-related 73. Monks PJ, Thompson JM, Bullmore ET,
FMRI study with the Stroop task. Biol Suckling J, Brammer MJ, Williams SCR,
Psychiatry 59(10):958–965 Simmons A, Giles N, Lloyd AJ, Harrison CL,
63. Vollm B, Richardson P, McKie S, Elliott Seal M, Murray RM, Ferrier IN, Young AH,
R, Deakin JFW, Anderson IM (2006) Curtis VA (2004) A functional MRI study
Serotonergic modulation of neuronal of working memory task in euthymic bipolar
responses to behavioural inhibition and rein- disorder: evidence for task-specific dysfunc-
forcing stimuli: an fMRI study in healthy vol- tion. Bipolar Disord 6(6):550–564
unteers. Eur J Neurosci 23(2):552–560 74. Frangou S, Raymont V, Bettany D (2002)
64. Walter H, Wolf RC, Spitzer M, Vasic N (2007) The Maudsley bipolar disorder project. A
Increased left prefrontal activation in patients survey of psychotropic prescribing patterns in
with unipolar depression: an event-related, bipolar I disorder. Bipolar Disord 4(6):
parametric, performance-controlled fMRI 378–385
study. J Affect Disord 101(1–3):175–185 75. Strakowski SM, Delbello MP, Adler CM
65. Yurgelun-Todd DA, Gruber SA, Kanayama (2005) The functional neuroanatomy of
G, Killgore WD, Baird AA, Young AD (2000) bipolar disorder: a review of neuroimaging
fMRI during affect discrimination in bipolar findings. Mol Psychiatry 10(1):105–116
affective disorder. Bipolar Disord 2(3 Pt 2): 76. Pavuluri MN, O’Connor MM, Harral E,
237–248 Sweeney JA (2007) Affective neural cir-
66. Blumberg HP, Donegan NH, Sanislow CA, cuitry during facial emotion processing in
Collins S, Lacadie C, Skudlarski P, Gueorguieva pediatric bipolar disorder. Biol Psychiatry
R, Fulbright RK, McGlashan TH, Gore JC, 62(2):158–167
Krystal JH (2005) Preliminary evidence for 77. Lagopoulos J, Ivanovski B, Malhi GS (2007)
medication effects on functional abnormali- An event-related functional MRI study of
ties in the amygdala and anterior cingulate in working memory in euthymic bipolar disor-
bipolar disorder. Psychopharmacology (Berl) der. J Psychiatry Neurosci 32(3):174–184
183(3):308–313 78. Sheline YI, Barch DM, Donnelly JM,
67. Adler CM, Holland SK, Schmithorst V, Ollinger JM, Snyder AZ, Mintun MA (2001)
Tuchfarber MJ, Strakowski SM (2004) Increased amygdala response to masked emo-
Changes in neuronal activation in patients with tional faces in depressed subjects resolves with
bipolar disorder during performance of a work- antidepressant treatment: an fMRI study. Biol
ing memory task. Bipolar Disord 6(6):540–549 Psychiatry 50(9):651–658
68. Chang K, Adleman NE, Dienes K, Simeonova 79. Chen C-H, Ridler K, Suckling J, Williams S,
DI, Menon V, Reiss A (2004) Anomalous Fu CHY, Merlo-Pich E, Bullmore E (2007)
prefrontal-subcortical activation in familial Brain imaging correlates of depressive symp-
pediatric bipolar disorder: a functional mag- tom severity and predictors of symptom
netic resonance imaging investigation. Arch improvement after antidepressant treatment.
Gen Psychiatry 61(8):781–792 Biol Psychiatry 62(5):407–414
694 Erin L. Habecker et al.

80. Casement MD, Guyer AE, Hipwell AE, motor inhibition in children at risk for bipolar
McAloon RL, Hoffmann AM, Keenan KE, disorder. Prog Neuropsychopharmacol Biol
Forbes EE (2014) Girls’ challenging social Psychiatry 38(2):127–133. doi:10.1016/j.
experiences in early adolescence predict pnpbp.2012.02.014
neural response to rewards and depressive 90. Singh MK, Kelley RG, Howe ME, Reiss
symptoms. Dev Cogn Neurosci 8:18–27. AL, Gotlib IH, Chang KD (2014) Reward
doi:10.1016/j.dcn.2013.12.003 processing in healthy offspring of parents
81. Sharp C, Kim S, Herman L, Pane H, Reuter with bipolar disorder. JAMA Psychiatry
T, Strathearn L (2014) Major depression in 71(10):1148–1156. doi:10.1001/
mothers predicts reduced ventral striatum jamapsychiatry.2014.1031
activation in adolescent female offspring with 91. Thermenos HW, Goldstein JM, Milanovic
and without depression. J Abnorm Psychol SM, Whitfield-Gabrieli S, Makris N,
123(2):298–309. doi:10.1037/a0036191 Laviolette P, Koch JK, Faraone SV, Tsuang
82. Whalley HC, Sussmann JE, Romaniuk L, MT, Buka SL, Seidman LJ (2010) An fMRI
Stewart T, Papmeyer M, Sprooten E, Hackett study of working memory in persons with
S, Hall J, Lawrie SM, McIntosh AM (2013) bipolar disorder or at genetic risk for bipolar
Prediction of depression in individuals at high disorder. Am J Med Genet B Neuropsychiatr
familial risk of mood disorders using func- Genet 153B(1):120–131. doi:10.1002/
tional magnetic resonance imaging. PLoS ajmg.b.30964
One 8(3):e57357. doi:10.1371/journal. 92. Brotman MA, Deveney CM, Thomas LA,
pone.0057357 Hinton KE, Yi JY, Pine DS, Leibenluft E
83. Mannie ZN, Filippini N, Williams C, Near (2014) Parametric modulation of neural
J, Mackay CE, Cowen PJ (2014) Structural activity during face emotion processing in
and functional imaging of the hippocampus unaffected youth at familial risk for bipolar
in young people at familial risk of depres- disorder. Bipolar Disord 16(7):756–763.
sion. Psychol Med 44(14):2939–2948. doi:10.1111/bdi.12193
doi:10.1017/S0033291714000580 93. Roberts G, Green MJ, Breakspear M,
84. Miskowiak KW, Glerup L, Vestbo C, Harmer McCormack C, Frankland A, Wright A, Levy
CJ, Reinecke A, Macoveanu J, Siebner HR, F, Lenroot R, Chan HN, Mitchell PB (2013)
Kessing LV, Vinberg M (2015) Different Reduced inferior frontal gyrus activation dur-
neural and cognitive response to emotional ing response inhibition to emotional stimuli
faces in healthy monozygotic twins at risk of in youth at high risk of bipolar disorder. Biol
depression. Psychol Med 45(7):1447–1458. Psychiatry 74(1):55–61. doi:10.1016/j.
doi:10.1017/S0033291714002542 biopsych.2012.11.004
85. Yurgelun-Todd DA, Ross AJ (2006) 94. Whiteside SP, Port JD, Abramowitz JS (2004)
Functional magnetic resonance imaging A meta-analysis of functional neuroimaging in
studies in bipolar disorder. CNS Spectr obsessive-compulsive disorder. Psychiatry Res
11(4):287–297 132(1):69–79
86. Blumberg HP, Leung HC, Skudlarski P, 95. Mataix-Cols D, Rosario-Campos MC,
Lacadie CM, Fredericks CA, Harris BC, Leckman JF (2005) A multidimensional
Charney DS, Gore JC, Krystal JH, Peterson model of obsessive-compulsive disorder. Am
BS (2003) A functional magnetic resonance J Psychiatry 162(2):228–238. doi:10.1176/
imaging study of bipolar disorder: state- and appi.ajp.162.2.228
trait-related dysfunction in ventral prefrontal 96. Saxena S, Brody AL, Ho ML, Alborzian S, Ho
cortices. Arch Gen Psychiatry 60(6):601– MK, Maidment KM, Huang SC, Wu HM, Au
609. doi:10.1001/archpsyc.60.6.601 SC, Baxter LR Jr (2001) Cerebral metabolism
87. Strakowski SM, Adler CM, Holland SK, in major depression and obsessive-compulsive
Mills NP, DelBello MP, Eliassen JC (2005) disorder occurring separately and concur-
Abnormal FMRI brain activation in euthy- rently. Biol Psychiatry 50(3):159–170
mic bipolar disorder patients during a count- 97. van den Heuvel OA, Veltman DJ,
ing Stroop interference task. Am J Psychiatry Groenewegen HJ, Cath DC, van Balkom
162(9):1697–1705 AJ, van Hartskamp J, Barkhof F, van Dyck
88. Goodwin FK, Jamison KR (1990) Manic- R (2005) Frontal-striatal dysfunction dur-
depressive illness. Oxford University Press, ing planning in obsessive-compulsive disor-
New York der. Arch Gen Psychiatry 62(3):301–309.
89. Deveney CM, Connolly ME, Jenkins SE, Kim P, doi:10.1001/archpsyc.62.3.301
Fromm SJ, Brotman MA, Pine DS, Leibenluft 98. Remijnse PL, Nielen MMA, van Balkom
E (2012) Striatal dysfunction during failed AJLM, Cath DC, van Oppen P, Uylings HBM,
 fMRI in Psychiatric Disorders 695

Veltman DJ (2006) Reduced orbitofrontal- of amygdala and medial prefrontal cortex

striatal activity on a reversal learning task in responses to overtly presented fearful faces
obsessive-compulsive disorder. Arch Gen in posttraumatic stress disorder. Arch Gen
Psychiatry 63(11):1225–1236 Psychiatry 62(3):273–281
99. Roth RM, Saykin AJ, Flashman LA, Pixley 108. Mitterschiffthaler MT, Ettinger U, Mehta
HS, West JD, Mamourian AC (2007) Event- MA, Mataix-Cols D, Williams SCR (2006)
related functional magnetic resonance imaging Applications of functional magnetic reso-
of response inhibition in obsessive-compulsive nance imaging in psychiatry. J Magn Reson
disorder. Biol Psychiatry 62(8):901–909 Imaging 23(6):851–861
100. Rauch SL, Whalen PJ, Shin LM, McInerney 109. Saxena S, Brody AL, Schwartz JM, Baxter LR
SC, Macklin ML, Lasko NB, Orr SP, Pitman (1998) Neuroimaging and frontal-subcortical
RK (2000) Exaggerated amygdala response circuitry in obsessive-compulsive disorder. Br
to masked facial stimuli in posttraumatic J Psychiatry Suppl 35:26–37
stress disorder: a functional MRI study. Biol 110. Purcell R, Maruff P, Kyrios M, Pantelis
Psychiatry 47(9):769–776 C (1998) Cognitive deficits in obsessive-
101. Lanius RA, Williamson PC, Densmore M, compulsive disorder on tests of frontal-striatal
Boksman K, Gupta MA, Neufeld RW, Gati function. Biol Psychiatry 43(5):348–357
JS, Menon RS (2001) Neural correlates of 111. Gilbert AR, Moore GJ, Keshavan MS,
traumatic memories in posttraumatic stress Paulson LA, Narula V, Mac Master FP,
disorder: a functional MRI investigation. Am Stewart CM, Rosenberg DR (2000)
J Psychiatry 158(11):1920–1922 Decrease in thalamic volumes of pediatric
102. Shin LM, Whalen PJ, Pitman RK, Bush G, patients with obsessive-compulsive disor-
Macklin ML, Lasko NB, Orr SP, McInerney der who are taking paroxetine. Arch Gen
SC, Rauch SL (2001) An fMRI study of ante- Psychiatry 57(5):449–456
rior cingulate function in posttraumatic stress 112. Kim JJ, Lee MC, Kim J, Kim IY, Kim SI,
disorder. Biol Psychiatry 50(12):932–942 Han MH, Chang KH, Kwon JS (2001) Grey
103. Hendler T, Rotshtein P, Yeshurun Y, matter abnormalities in obsessive-compulsive
Weizmann T, Kahn I, Ben-Bashat D, Malach disorder: statistical parametric mapping of
R, Bleich A (2003) Sensing the invisible: segmented magnetic resonance images. Br
differential sensitivity of visual cortex and J Psychiatry 179:330–334
amygdala to traumatic context. Neuroimage 113. Lacerda ALT, Dalgalarrondo P, Caetano D,
19(3):587–600 Camargo EE, Etchebehere ECSC, Soares
104. Lanius RA, Williamson PC, Hopper J, JC (2003) Elevated thalamic and prefrontal
Densmore M, Boksman K, Gupta MA, regional cerebral blood flow in obsessive-
Neufeld RWJ, Gati JS, Menon RS (2003) compulsive disorder: a SPECT study.
Recall of emotional states in posttraumatic Psychiatry Res 123(2):125–134
stress disorder: an fMRI investigation. Biol 114. Saxena S, Rauch SL (2000) Functional neuro-
Psychiatry 53(3):204–210 imaging and the neuroanatomy of obsessive-
105. Driessen M, Beblo T, Mertens M, Piefke M, compulsive disorder. Psychiatr Clin North
Rullkoetter N, Silva-Saavedra A, Reddemann Am 23(3):563–586
L, Rau H, Markowitsch HJ, Wulff H, Lange 115. Rauch SL, Savage CR, Alpert NM, Fischman
W, Woermann FG (2004) Posttraumatic AJ, Jenike MA (1997) The functional neu-
stress disorder and fMRI activation patterns roanatomy of anxiety: a study of three dis-
of traumatic memory in patients with bor- orders using positron emission tomography
derline personality disorder. Biol Psychiatry and symptom provocation. Biol Psychiatry
55(6):603–611 42(6):446–452
106. Protopopescu X, Pan H, Tuescher O, Cloitre 116. Hettema JM, Neale MC, Kendler KS (2001)
M, Goldstein M, Engelien W, Epstein J, Yang A review and meta-analysis of the genetic
Y, Gorman J, LeDoux J, Silbersweig D, Stern epidemiology of anxiety disorders. Am
E (2005) Differential time courses and speci- J Psychiatry 158(10):1568–1578
ficity of amygdala activity in posttraumatic 117. de Vries FE, de Wit SJ, Cath DC, van der Werf
stress disorder subjects and normal control YD, van der Borden V, van Rossum TB, van
subjects. Biol Psychiatry 57(5):464–473 Balkom AJ, van der Wee NJ, Veltman DJ, van
107. Shin LM, Wright CI, Cannistraro PA, Wedig den Heuvel OA (2014) Compensatory fron-
MM, McMullin K, Martis B, Macklin ML, toparietal activity during working memory:
Lasko NB, Cavanagh SR, Krangel TS, Orr SP, an endophenotype of obsessive-compulsive
Pitman RK, Whalen PJ, Rauch SL (2005) A disorder. Biol Psychiatry 76(11):878–887.
functional magnetic resonance imaging study doi:10.1016/j.biopsych.2013.11.021
696 Erin L. Habecker et al.

118. de Wit SJ, de Vries FE, van der Werf YD, 129. Vermetten E, Vythilingam M, Southwick SM,
Cath DC, Heslenfeld DJ, Veltman EM, van Charney DS, Bremner JD (2003) Long-term
Balkom AJ, Veltman DJ, van den Heuvel OA treatment with paroxetine increases verbal
(2012) Presupplementary motor area hyper- declarative memory and hippocampal vol-
activity during response inhibition: a candi- ume in posttraumatic stress disorder. Biol
date endophenotype of obsessive-compulsive Psychiatry 54(7):693–702
disorder. Am J Psychiatry 169(10):1100– 130. Pitman RK, Shin LM, Rauch SL (2001)
1108. doi:10.1176/appi.ajp.2012.12010073 Investigating the pathogenesis of posttrau-
119. Charney DS, Deutch AY, Krystal JH, Southwick matic stress disorder with neuroimaging.
SM, Davis M (1993) Psychobiologic mecha- J Clin Psychiatry 62(Suppl 17):47–54
nisms of posttraumatic stress disorder. Arch 131. Villarreal G, King CY (2001) Brain imaging
Gen Psychiatry 50(4):295–305 in posttraumatic stress disorder. Semin Clin
120. Charney DS (2004) Psychobiological mecha- Neuropsychiatry 6(2):131–145
nisms of resilience and vulnerability: impli- 132. Morgan MA, LeDoux JE (1995) Differential
cations for successful adaptation to extreme contribution of dorsal and ventral medial
stress. Am J Psychiatry 161(2):195–216 prefrontal cortex to the acquisition and
121. Quirk GJ, Gehlert DR (2003) Inhibition of extinction of conditioned fear in rats. Behav
the amygdala: key to pathological states? Ann Neurosci 109(4):681–688
N Y Acad Sci 985:263–272 133. Quirk GJ, Russo GK, Barron JL, Lebron K
122. Stein MB, Simmons AN, Feinstein JS, (2000) The role of ventromedial prefrontal
Paulus MP (2007) Increased amygdala and cortex in the recovery of extinguished fear.
insula activation during emotion processing J Neurosci 20(16):6225–6231
in anxiety-prone subjects. Am J Psychiatry 134. Santini E, Ge H, Ren K, Pena de Ortiz S, Quirk
164(2):318–327 GJ (2004) Consolidation of fear extinction
123. Geuze E, Vermetten E, Bremner JD (2005) requires protein synthesis in the medial pre-
MR-based in vivo hippocampal volumetrics: frontal cortex. J Neurosci 24(25):5704–5710
2. Findings in neuropsychiatric disorders. 135. Williams LM, Kemp AH, Felmingham K,
Mol Psychiatry 10(2):160–184 Barton M, Olivieri G, Peduto A, Gordon
124. Bremner JD (2002) Neuroimaging of child- E, Bryant RA (2006) Trauma modulates
hood trauma. Semin Clin Neuropsychiatry amygdala and medial prefrontal responses
7(2):104–112 to consciously attended fear. Neuroimage
125. Bremner JD, Vythilingam M, Vermetten E, 29(2):347–357
Southwick SM, McGlashan T, Nazeer A, 136. Golier JA, Yehuda R, Lupien SJ, Harvey PD,
Khan S, Vaccarino LV, Soufer R, Garg PK, Grossman R, Elkin A (2002) Memory per-
Ng CK, Staib LH, Duncan JS, Charney formance in Holocaust survivors with post-
DS (2003) MRI and PET study of defi- traumatic stress disorder. Am J Psychiatry
cits in hippocampal structure and function 159(10):1682–1688
in women with childhood sexual abuse and 137. Rauch SL, Shin LM, Phelps EA (2006)
posttraumatic stress disorder. Am J Psychiatry Neurocircuitry models of posttraumatic stress
160(5):924–932 disorder and extinction: human neuroimag-
126. Gilbertson MW, Shenton ME, Ciszewski ing research—past, present, and future. Biol
A, Kasai K, Lasko NB, Orr SP, Pitman RK Psychiatry 60(4):376–382
(2002) Smaller hippocampal volume pre- 138. Shin LM, Shin PS, Heckers S, Krangel TS,
dicts pathologic vulnerability to psychological Macklin ML, Orr SP, Lasko N, Segal E,
trauma. Nat Neurosci 5(11):1242–1247 Makris N, Richert K, Levering J, Schacter
127. Whalen PJ, Rauch SL, Etcoff NL, McInerney DL, Alpert NM, Fischman AJ, Pitman RK,
SC, Lee MB, Jenike MA (1998) Masked Rauch SL (2004) Hippocampal function in
presentations of emotional facial expressions posttraumatic stress disorder. Hippocampus
modulate amygdala activity without explicit 14(3):292–300
knowledge. J Neurosci 18(1):411–418 139. Vermetten E, Bremner JD (2002) Circuits
128. Lanius RA, Williamson PC, Bluhm RL, and systems in stress. II. Applications to neu-
Densmore M, Boksman K, Neufeld RWJ, Gati robiology and treatment in posttraumatic
JS, Menon RS (2005) Functional connectiv- stress disorder. Depress Anxiety 16(1):14–38
ity of dissociative responses in posttraumatic 140. Bremner JD, Narayan M, Staib LH,
stress disorder: a functional magnetic reso- Southwick SM, McGlashan T, Charney DS
nance imaging investigation. Biol Psychiatry (1999) Neural correlates of memories of
57(8):873–884 childhood sexual abuse in women with and
 fMRI in Psychiatric Disorders 697

without posttraumatic stress disorder. Am Higher brain blood flow at amygdala and lower
J Psychiatry 156(11):1787–1795 frontal cortex blood flow in PTSD patients
141. Shin LM, McNally RJ, Kosslyn SM, with comorbid cocaine and alcohol abuse com-
Thompson WL, Rauch SL, Alpert NM, pared with normals. Psychiatry 63(1):65–74
Metzger LJ, Lasko NB, Orr SP, Pitman RK 143. Shin LM, Bush G, Milad MR, Lasko NB,
(1999) Regional cerebral blood flow during Brohawn KH, Hughes KC, Macklin ML,
script-driven imagery in childhood sexual Gold AL, Karpf RD, Orr SP, Rauch SL,
abuse-related PTSD: a PET investigation. Am Pitman RK (2011) Exaggerated activation
J Psychiatry 156(4):575–584 of dorsal anterior cingulate cortex during
142. Semple WE, Goyer PF, McCormick R, cognitive interference: a monozygotic twin
Donovan B, Muzic RF, Rugle L, McCutcheon study of posttraumatic stress disorder. Am
K, Lewis C, Liebling D, Kowaliw S, Vapenik J Psychiatry 168(9):979–985. doi:10.1176/
K, Semple MA, Flener CR, Schulz SC (2000) appi.ajp.2011.09121812
 Chapter 23

fMRI in Neurodegenerative Diseases: From Scientific

Insights to Clinical Applications
Bradford C. Dickerson, Federica Agosta, and Massimo Filippi

Abstract
fMRI is a technology with great promise as a tool to probe abnormalities of brain activity in neurodegenerative
diseases. The detection of functional brain abnormalities may be useful, in the appropriate clinical context, for
early diagnosis, differential diagnosis, or prognostication. Prediction of response to treatment or therapeutic
monitoring may also be possible with fMRI. In addition, fMRI has the potential to provide a variety of scien-
tific insights that may have clinical relevance, including compensatory hyperactivation of brain circuits or
genetic modulation of functional brain activity.

Key words Alzheimer’s disease, Amyotrophic lateral sclerosis, Functional MRI, Huntington disease,
Magnetic resonance imaging, Neurodegenerative diseases, Parkinson’s disease

1 Introduction

Neurodegenerative diseases are a major medical and social burden in

many societies, particularly with the growth of older population seg-
ments. Neurodegenerative diseases include many dementias, move-
ment disorders, cerebellar, and motor neuron diseases. In many cases,
these diseases involve the pathologic accumulation of abnormal pro-
tein forms. As the biology of these diseases is elucidated, hope is begin-
ning to emerge for specific treatments targeted at modification of
fundamental pathophysiologic processes [1, 2]. For this hope to be
realized, methods for early detection of specific disease processes need
to be identified. Furthermore, reliable methods for monitoring the
progression of the diseases will likely be critical in demonstrating the
effects of putative disease-modifying therapies. Although molecular
biomarkers measured via positron emission tomography (PET) or
cerebrospinal fluid (CSF) have emerged in the past 5–10 years as a
major class of biomarker, magnetic resonance imaging (MRI) contin-
ues to offer great potential in assessment and monitoring of neurode-
generative diseases [3]. Given the growing body of evidence that
alterations in synaptic function are present very early in the course of

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_23, © Springer Science+Business Media New York 2016

699
700 Bradford C. Dickerson et al.

neurodegenerative disease processes, possibly long before the

development of clinical symptoms and even significant neuropathol-
ogy [4, 5], functional MRI (fMRI) may be particularly useful for
detecting alterations in brain function that may be present very early in
the trajectory of neurodegenerative diseases. In addition, fMRI may be
a critical biomarker for the detection of physiological alterations over a
short period of time, thus serving as a measure of target engagement in
clinical trials. The uses of fMRI in neurodegenerative dementias will be
reviewed, with a focus on Alzheimer’s disease (AD), to illustrate many
points of relevance to other neurodegenerative diseases.
Before specifically discussing fMRI, though, it is worth consid-
ering the current concepts of clinicopathologic constructs of neu-
rodegenerative diseases, since the interpretation of imaging data in
patients depends critically on a detailed understanding of the clini-
cal characteristics of the patient population(s) being studied.

2 Constructs of Neurodegenerative Disease: Clinical, Prodromal,

and Presymptomatic Phases

The field of neurodegenerative dementias has increasingly shifted as

molecular biomarkers have become available to considering neuro-
pathology separately from clinical syndrome. The major neuropa-
thologies of neurodegenerative dementias include Alzheimer’s
disease (AD), frontotemporal lobar degeneration (including FTLD-
TDP43 and FTLD-tau, which many experts consider to encompass
not only classical Pick’s disease but also progressive supranuclear
palsy [PSP] and corticobasal degeneration [CBD]), the Lewy body
diseases, and Huntington’s disease. The clinical syndromes include
AD dementia, the primary progressive aphasias (PPA), behavioral
variant frontotemporal dementia (bvFTD), the PSP syndromes, cor-
ticobasal syndrome, dementia with Lewy bodies (DLB) and
Parkinson’s disease dementia (PDD), amyotrophic lateral sclerosis
with cognitive/behavioral impairment (ALS-FTD), and
Huntington’s disease [6, 7]. Clinicopathologic relationships within
the family of neurodegenerative dementias is immensely complex,
with growing data supporting many types of overlap between condi-
tions traditionally thought of as distinct.
Many neurodegenerative diseases contributing to dementia are
thought to arise from pathophysiologic processes that take place over
a decade or more prior to the development of symptoms. For exam-
ple, the clinical diagnosis of AD has traditionally been made after a
patient has developed impairment in multiple cognitive domains that
is substantial enough to interfere with routine social and/or occupa-
tional function (dementia). Previously, it was only after this point that
FDA-approved medications were currently indicated—that is, in
patients with clinically probable AD dementia. By this time, substan-
tial neuronal loss and neuropathologic change have damaged many
brain regions. Furthermore, clinical trials of amyloid-modifying drugs
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 701

have failed in patients with AD dementia, indicating that to date it is

not possible to slow progression. Thus, it would be ideal to initiate
treatment with neuroprotective medications at a time when—or even
before—AD is mildly symptomatic [3, 8]. We are moving rapidly
toward this goal with the 2011 revision of the diagnostic criteria for
AD to include prodromal and preclinical phases [9–11]. This scenario
is true for many neurodegenerative diseases, and is even more com-
pelling in diseases in which known genetic abnormalities can be iden-
tified that predict future disease, as in FTD [12], Huntington’s disease
[13], and autosomal dominant forms of AD [14].
To approach the goal of early intervention in neurodegenerative
diseases, we must improve our capability to identify individuals in
the earliest symptomatic phases of the diseases prior to significant
functional impairment. For example, individuals are categorized as
having mild cognitive impairment (MCI) when symptoms sugges-
tive of AD are present but mild enough that traditional diagnostic
criteria (which require functional impairment consistent with
dementia) are not fulfilled. This gradual transitional state often lasts
for a number of years, offering opportunities for intervention when
the disease is in its prodromal stages. Diagnostic criteria for MCI
have been developed [15] and operationalized [16] and subse-
quently revised including biomarkers to enable the identification of
patients with MCI highly likely due to AD [10], or prodromal AD,
which has been the target population in a number of clinical trials to
date. If the pathophysiologic process of AD can be slowed at this
stage of the disease, then it may be possible to preserve cognitive
function and delay the ultimate development of dementia for a
period of time, which is clearly clinically meaningful.
Finally, the presymptomatic phase of neurodegenerative dis-
eases is the phase when pathologic alterations are developing but
symptoms are not yet apparent. In the case of AD, this phase has
been studied through the identification of cohorts with particular
risk factors, such as genetic determinants (e.g., amyloid precursor
protein (APP) or presenilin mutations, Down syndrome) or sus-
ceptibility factors (e.g., apolipoprotein E (APOE) ε4) or through
biomarkers of the underlying disease process (e.g., amyloid imag-
ing or spinal fluid). The development of research diagnostic criteria
for preclinical AD has been transformative [9], and in the last few
years therapeutic clinical trials of medications in preclinical AD
have begun using all of these strategies [17].

3 Strengths and Weaknesses of fMRI as a Tool to Probe Brain Activity

in Neurodegenerative Diseases

Since functional neuroimaging tools assess inherently dynamic

processes that may change over short time intervals in relation to a
host of factors, these measures have unique characteristics that may
offer both strengths and weaknesses as potential biomarkers of
702 Bradford C. Dickerson et al.

neurologic disease. Functional neuroimaging measures may be

affected by transient brain and body states at the time of imaging,
such as arousal, attention, sleep deprivation, sensory processing of
irrelevant stimuli, or the effects of substances with pharmacologic
central nervous system activity. Imaging measures of brain func-
tion may also be more sensitive than structural measures to consti-
tutional or chronic differences between individuals, such as
genetics, intelligence or educational level, learning, mood, or med-
ication use. While these may be effects of interest in certain experi-
mental settings, they need to be controlled when the focus is on
disease-related changes and differences between subject groups or
within individuals over time.
Among functional neuroimaging techniques, fMRI has many
potential advantages in studying patients with neurodegenerative
disorders, as it is a noninvasive imaging technique that does not
require the injection of a contrast agent. It can be repeated many
times over the course of a longitudinal study and thus lends itself
well as a measure in clinical drug trials. It has relatively high spatial
and temporal resolution, and the use of event-related designs
enables the hemodynamic correlates of specific behavioral events,
such as successful memory formation [18] or momentary ratings
of emotional intensity [19], to be measured.
A caveat essential to the interpretation of task-related func-
tional neuroimaging data is that healthy individuals of any age
demonstrate differences in brain activation depending on how well
they are able to perform the particular task. For example, when
cognitively intact individuals learn new information during fMRI
scanning, the strength of this signal is related to subsequent ability
to remember the information [20–24]. AD patients typically per-
form less well on the memory tasks, which complicates the inter-
pretation of these data [25]. Conversely, the recruitment of
additional brain regions during task performance by patients with
neurodegenerative or other neurologic disease may indicate the
presence of processes attempting to compensate for damaged net-
works [26, 27]. While the task performance factor is important to
consider when designing or interpreting functional neuroimaging
studies of neurodegenerative diseases, it also indicates that these
imaging biomarkers may be particularly sensitive to changes in
cognitive or sensorimotor function, which not only provides face
validity for these measures but also supports their potential use in
short-term, early proof-of-concept drug trials.
There are additional challenges to performing fMRI studies in
patients with neurodegenerative diseases. The technique is particu-
larly sensitive to even small amounts of head motion. Finally,
although a growing number of test-retest reliability studies of task-
related [28–39] and resting state [31, 40–44] fMRI have been
published in the last 5–10 years, most of these are in healthy young
adults. A small but growing number of longitudinal fMRI studies
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 703

have been performed in patient populations; some of these have

been focused on investigating reliability of signal [45–50] and oth-
ers have been aimed at identifying changes in brain activation or
connectivity associated with longitudinal clinical change [51, 52].

4 Clinical Applications of fMRI in Neurodegenerative Diseases

Functional MRI has been applied in a number of ways in studies of

patients with neurodegenerative diseases. fMRI has been used to
identify abnormal patterns of brain activity during the performance
of a variety of tasks in patients with neurodegenerative diseases;
these abnormal patterns may reveal new insights into the disrup-
tion of brain circuits by such diseases. They may also be useful in
differential diagnosis. Similarly, studies of resting state fMRI have
begun to demonstrate how this relatively newer method can pro-
vide insights into disrupted brain circuitry in neurodegenerative
diseases. Hyperactivation or hyperconnectivity has been identified
in many of these studies, which is a fascinating area of ongoing
research [53]. fMRI has been used to assess the modulatory effects
of genetic factors on brain activation in patients with or at risk for
neurodegenerative disorders, and has been used to monitor the
effects of therapeutic interventions and is beginning to be used to
try to help predict the course of the diseases.

4.1 Patterns fMRI has been used to investigate abnormalities in patterns of

of Abnormal Regional regional brain activation during a variety of cognitive tasks in
Brain Activation patients diagnosed with mild AD compared with control subjects.
During Task It is important to keep in mind that the particular abnormalities
Performance found in an fMRI study of an AD or other patient group are heav-
ily dependent on the type of behavioral task used in the study—if
the task does not engage a particular circuit, functional abnormali-
ties will not likely be observed. Also, the nature of functional
abnormalities may depend on whether the activated brain regions
are directly affected by the disease, are indirectly affected via con-
nectivity, or are not pathologically affected. Tools are now available
to directly investigate the overlap of disease-related alterations in
brain structure and task-related functional activity (Fig. 1). Yet it
should also be kept in mind that even brain regions not usually
thought to be affected by a particular neurodegenerative disease
(e.g., sensorimotor areas in AD) have been shown to exhibit abnor-
mal function [54, 55].
In addition to memory, which is discussed next, aspects of lan-
guage and attention have been studied. Altered patterns of frontal
and temporal activation have been observed in AD patients per-
forming language tasks [56–58]. Similarly, although temporo-
parietal activation was diminished in AD during performance of
semantic memory task, increased activation in temporal and frontal
704 Bradford C. Dickerson et al.

Fig. 1 The localization, magnitude, and extent of abnormalities observed in fMRI

studies of patients with neurologic diseases depend on both localization and
severity of pathology and on functional networks engaged by the particular fMRI
task, as well as participant performance on the task. In this illustration, regions of
cortical thinning in Alzheimer’s disease from structural MRI (bottom; brighter blue
colors indicate greater degree of thinning) are compared with cortical areas acti-
vated, as measured with fMRI, in normals during an event-related study of suc-
cessful learning of new information that was able to later be freely recalled (top,
yellow colors indicate greater blood oxygen level-dependent (BOLD) signal in the
contrast of recalled items vs. fixation) (reproduced with permission from [76])

regions was also observed, suggesting possible compensatory

processes [59]. Increased activation of the semantic memory cir-
cuits was observed in MCI patients when task performance remains
at levels comparable to healthy controls [60]. This increased acti-
vation was particularly evident in the posterior cingulate cortex,
posterolateral parietal cortex, and frontal cortex [60]. During per-
formance of a visual attention task, AD patients showed alterations
in parietal activation; increased prefrontal activation was also
observed compared with controls, again suggesting possible com-
pensatory mechanisms [61]. fMRI has been used to explore the
neural basis of interesting phenomena in AD including abnormali-
ties in discriminating foreground from background sounds (the
“cocktail party effect” in auditory scene analysis) [62].
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 705

With respect to memory, a number of pioneering fMRI studies

in patients with clinically diagnosed AD, using a variety of visually
presented stimuli, have identified decreased activation in hippo-
campal and parahippocampal regions than in control subjects dur-
ing episodic encoding tasks [63–67]. Neocortical abnormalities in
AD have also been demonstrated using fMRI. A meta-analysis of
both fMRI and fluorodeoxyglucose positron emission tomography
(FDG-PET) memory activation studies of AD identified several
cortical regions as showing greater encoding-related activation in
controls than in AD patients, including the ventrolateral prefrontal
cortex, precuneus, cingulum and lingual cortex [68]. In addition
to AD-related differences in task-related blood oxygen level-
dependent (BOLD) signal amplitude or spatial extent, the tempo-
ral dynamics of activation appear to be altered in patients with AD
[69]. Increased activation in prefrontal and other regions has also
been found in AD patients performing memory tasks [67].
While memory-task related fMRI data regarding medial tempo-
ral lobe (MTL) activation in individuals with MCI are less consistent
than data from patients diagnosed with AD, with reports of both
decreased and increased activation [26, 64, 66, 70–75], they indi-
cate that differences are present in comparison to older controls.
Some of the variability in fMRI data in MTL activation appears to
relate to degree of impairment along the spectrum of MCI, which
suggests that fMRI may be sensitive to relatively subtle clinical dif-
ferences in disease severity [76, 77]. Similarly to studies of AD
patients, investigations of posteromedial cortical deactivation during
encoding in MCI patients has demonstrated reduced deactivation
[78]. Other tasks have been used to demonstrate functional brain
activation abnormalities in patients with MCI, including abnormali-
ties of lateral temporal activation during word reading [79].
Since the advent of molecular markers of brain amyloid, includ-
ing both amyloid imaging and CSF amyloid assays, research has
intensified investigating task-related fMRI in cognitively normal
(CN) older adults with brain amyloid, who are often viewed as
having preclinical AD. One of the most widely replicated findings
in CN amyloid-positive individuals is impaired posterior cingu-
late/precuneus deactivation during memory encoding [80–83], as
well as impaired activation of this region during retrieval [81], and
also in an attentional control task [84]. The MTL hyperactivates
during encoding in association with increasing amyloid load in CN
older adults [82, 85]. Prefrontal regions may also hyperactivate
during successful encoding [82, 85], although not all studies dem-
onstrate this finding [83]; in one study, the degree of hyperactiva-
tion was associated with memory performance [85], supporting
the interpretation of possible compensatory hyperactivation. As
tau PET imaging is developed [86], it will be exciting to test
hypotheses regarding the effects of local hyperphosphorylated tau
deposition on regional activation and connectivity.
706 Bradford C. Dickerson et al.

Very little work has been done with task-related fMRI in patients
with bvFTD, probably in large part due to the difficulties these
patients often have in cooperating with task instructions and in the
setting of MRI. During the viewing of faces conveying emotional
expressions, bvFTD patients exhibit abnormally reduced activation in
ventrolateral prefrontal cortex, insula, and a variety of other brain
regions, but elevated activation in posterior parietal cortex [87].
fMRI has also been used to investigate the neural basis for changes in
understanding of emotion conveyed through music [88] and in rea-
soning for personal vs. impersonal moral dilemmas [89] in bvFTD.
In PPA, an early fMRI study using simple phonological and
semantic tasks demonstrated relatively normal activation within
canonical regions of the language network but with increased
recruitment in areas not typically recruited by controls; these
increases correlated with greater language impairment [90]. Sonty
et al. subsequently used dynamic causal modeling to show that
effective connectivity between canonical language regions was
impaired in PPA [91].
In a study of syntactic comprehension in nonfluent variant
PPA patients, the caudal inferior frontal cortex did not show rela-
tively greater activation for complex sentences than for simple sen-
tences, as it did in controls [92].
Vandenbulcke et al. [93] used an associative-semantic fMRI par-
adigm to demonstrate that patients with the semantic variant of PPA
show an abnormal rightward lateralization of anterior temporal acti-
vation, which is reminiscent of a similar language laterality shift that
has been reported in patients with stroke aphasia. This paradigm was
also used in nonfluent variant PPA patients to identify a similar effect
[94]. The phenomenon of surface dyslexia was investigated in seman-
tic variant PPA; in patients but not controls the inferior parietal cor-
tex was recruited for irregular words, but mid-fusiform and superior
temporal cortex was under-recruited in PPA patients [95].
Patients with the semantic variant of PPA have also been stud-
ied using a paradigm comparing meaningful to meaningless sounds.
Compared with controls, patients showed abnormal activation of
dorsolateral temporal cortical areas for both meaningless sounds as
well as for meaningful sounds (animal sounds versus tool sounds),
suggesting that aberrant processing of sounds in semantic PPA
extends to pre-semantic perceptual processing [96]; prior work by
the same authors demonstrated abnormal auditory processing in
patients with the nonfluent variant of PPA as well [97].
fMRI has been used to study abnormal patterns of brain activa-
tion in a variety of tasks in patients with Parkinson’s disease (PD),
most commonly in tasks engaging the motor system [98, 99]. fMRI
studies investigating brain activations in PD patients during self-initi-
ated movements demonstrated evidence of reduced neural responses
in the pre-supplementary motor areas (SMA), along with hyperactiva-
tion in both the lateral premotor cortex and parietal cortex
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 707

[100–102]. Some authors interpret these findings as a shift from

medial to lateral premotor loops, possibly indicating impairment in
generating internal cues for movement and greater dependence on
external cues [103–105]. In addition, cognitive tasks have been
employed, such as a working memory paradigm that demonstrated
hypoactivation in fronto-striatal regions in PD patients with cognitive
impairment compared with those patients who were not cognitively
impaired [106]. These findings were confirmed by a large fMRI study
showing a frontostriatal hypoactivation during a verbal two-back
working memory task in PD patients [107]. In the PD group, patients
with MCI (PD-MCI) had an additional hypoactivation of the bilateral
anterior cingulate cortex and the right caudate nucleus compared with
non-MCI PD patients [107] (Fig. 2). Interestingly, PD-MCI patients
also showed a lower single-photon emission computed tomography
dopamine presynaptic uptake in the right caudate than patients with-
out MCI, which correlated with striatal fMRI signal [107]. The rela-
tionship between cognitive impairment and frontostriatal dysfunction
was also suggested by a study showing that patients with PD-MCI
had a reduced activity of the premotor cortex and cognitive corticos-
triatal loop, which includes the caudate nucleus and prefrontal cortex,
while planning a set shift during fMRI, whereas non-MCI patients
experienced activation patterns similar to those of healthy participants
[108]. Impulsive behavior is an important problem in some patients
with PD, and fMRI paradigms have been developed to understand
the neural basis of these symptoms as well; behavioral abnormalities
and their underlying neural substrates are exacerbated with dopami-
nergic therapy [109, 110].
Very little work using fMRI has been performed in patients
with diffuse Lewy body disease (DLB), with one recent study dem-
onstrating a complex set of differences between DLB and AD
patients in visual cortical activation during face, color, and motion
perceptual tasks, many of which were explainable by differences in
task performance [111].
In Huntington’s disease, a growing body of work has employed
attentional or executive tasks to probe fronto-parietal or fronto-
striatal systems using fMRI. In an early study, reduced activation was
found in multiple cortical regions in HD patients performing a serial
reaction time task, compared to controls [112]. Both increased and
decreased recruitment has been observed on working memory tasks
in HD patients [113, 114]. On a visual attention/response inhibi-
tion task, HD patients showed reduced prefrontal-anterior cingulate
interhemispheric functional connectivity, and reduced connectivity
predicted slower reaction times and increased numbers of errors on
the task [115]. On a set-shifting task, HD patients showed greater
prefrontal activation than controls; a lesser degree of activation was
associated with more prominent performance impairment on the
task and neuropsychiatric symptoms [116]. Reduced error-related
activation was also found in an anti-saccade task [117].
708 Bradford C. Dickerson et al.

Fig. 2 Pattern of fMRI signal during a verbal two-back working memory task in PD patients and cognitive
impairment. Significant under-recruitment occurred in (a) the bilateral anterior cingulate cortex and (b) the
right caudate in PD patients with MCI (n = 30) compared with those without MCI (n = 26; red). Mean beta
values in (c) the anterior cingulate cortex cluster and (d) right caudate, contrasting two-back with baseline
conditions for PD patients with (blue) and without (red) MCI and control individuals (green). (e) Mean beta
values in the right caudate, contrasting two-back with baseline for motor-matched (Unified Parkinson’s
disease Rating Scale III) groups for PD patients with MCI and without MCI. The group sizes were held con-
stant (n = 18 in each group) and matched by scanner model. Error bars are 1 standard error. Figure reprinted
from [107] with permission
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 709

In ALS, fMRI has been used to demonstrate abnormalities in

motor cortical activation. Furthermore, it has been used to investi-
gate whether there are different patterns of cortical activation dur-
ing a simple motor task in ALS patients with a primarily upper
motor neuron (UMN) pattern of clinical deficits vs. those with a
lower motor neuron (LMN) pattern [118]. The UMN patient
subgroup showed relatively greater activation of the anterior cin-
gulate and caudate than the LMN subgroup, suggesting that fMRI
may provide insights into the differentiation of disease subtypes.
Since many patients with ALS develop cognitive and/or affective
symptoms (and in some cases full-blown frontotemporal demen-
tia), fMRI paradigms including verbal fluency [119], inhibitory
control [120], and an emotional word viewing task [121] have
been used to demonstrate functional brain activation abnormalities
in relevant circuits. Further work is necessary to better understand
whether fMRI task activation abnormalities may predict the emer-
gence of symptoms impacting function in daily life.

4.2 Abnormalities Hypoperfusion/metabolism is typically seen with nuclear medical

in Functional imaging techniques (such as FDG-PET or SPECT) in temporo-
Connectivity in parietal/posterior cingulate cortical regions in AD patients during
Patients with Neuro- the “resting” state. The medial parietal/posterior cingulate cortex,
degenerative along with medial frontal and lateral parietal regions, compose a
Diseases: Resting “default mode” network (DMN) that is generally more active
State fMRI when individuals are not engaged in specific tasks [122], and which
is thought to play a role in memory, self-appraisal, self-referential
planning, and a variety of other mental states [123]. Multiple
studies in AD patients have demonstrated alterations in the deacti-
vation and functional connectivity of these regions [124–127].
Substantial overlap is present between the DMN and the localiza-
tion of amyloid PET tracer binding [128].
These early observations have been markedly enriched in the
past decade with the explosion of research using resting state fMRI.
More than 20 years ago, Biswal originally observed [129] that low-
frequency fluctuations in BOLD signal amplitude could be mea-
sured while subjects lie “at rest” in the scanner without engaging
in a specific task, and that correlations between the oscillations in a
“seed” region of interest could be used to identify correlated signal
in other areas that are likely part of the same large-scale circuit, and
that share the same topography as regions activated during a fin-
ger-tapping task. Similar findings can be obtained using data-driven
analysis methods such as independent component analysis.
Although maps of networks obtained using resting state fMRI
methods are thought to represent connections beyond strict mono-
synaptic connections, the topography of networks identified using
these methods has been validated against traditional tract tracing
methods in nonhuman primates as reflecting the major gray matter
sites of origin and termination of large-scale networks [130]. One
710 Bradford C. Dickerson et al.

advantage of using resting state fMRI methods in patients with

cognitive impairment or dementia is that the challenges of engag-
ing patients in specific tasks can be avoided.
A growing body of work has developed confirming Biswal’s
original observation across multiple networks, showing the corre-
spondence between the topography of distributed neural networks
identified at rest and those engaged by task performance in tradi-
tional task-related fMRI [131]. Major networks have been identi-
fied using these methods including somatosensory, visual, auditory,
language, attention/executive function, affective, memory, DMN,
and other networks [131, 132].
In resting state fMRI studies of AD dementia, one of the most
reproduced findings from is the consistent reduction of connectivity
within the DMN [133–138]. As has been pointed out more than a
decade ago, this should not be surprising given that amyloid deposi-
tion, hypometabolism, and atrophy convergently affect many key
nodes of the DMN in patients with AD [128]. The magnitude of
reduced connectivity within the DMN correlates with the severity of
symptoms [133, 135, 136, 139]. The sensitivity of resting state fMRI
measures in differentiating AD patients from healthy elderly controls
ranges from 72 to 85 % and specificity from 77 to 80 % [125, 140,
141]. Longitudinal data have begun to suggest that some subnet-
works within the DMN may develop hyperconnectivity early in the
disease course while others show hypoconnectivity, but all decline
over time [142]. A few studies have begun to compare connectivity
at rest to connectivity during tasks; typically, controls show greater
connectivity within the DMN at rest than they do while performing
a task (presumably because some nodes of the DMN may contribute
to the task). One recent study showed that AD patients fail to modu-
late connectivity within the DMN during a simple visual attention
task [143]. Another study showed that AD patients had greater
memory encoding task-related prefrontal activation and greater pre-
frontal resting state connectivity than controls [144].
Since AD dementia is a syndrome of multiple domains of cog-
nitive impairment, it stands to reason that other networks would
be disrupted as well. However, evidence to date is not consistent.
There is some evidence demonstrating reduced connectivity in
multiple networks in addition to the DMN [136], but not all stud-
ies demonstrate such widespread effects. This is likely in part
related to the heterogeneity of AD itself and of the patient samples
in these studies, particularly with regard to the types and severity of
symptoms. In addition, some findings point to increased connec-
tivity in the salience network [133] and frontoparietal network
[144], although conflicting data have been reported showing
reduced rather than increased connectivity within these networks
[136, 145]. The connectivity within the executive control net-
work, in contrast, appears more consistently increased in AD [144–
146], which is surprising given the common occurrence of
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 711

executive dysfunction in these patients, although in one of these

studies the increased connectivity was correlated with neuropsy-
chological performance [146].
Although AD dementia is typically a multidomain amnesic
dementia, major subtypes are well recognized. Two of these have
received preliminary investigation with regard to the overlap of the
topography of atrophy with the topography of healthy functional
connectivity network. In posterior cortical atrophy (PCA), the atro-
phy pattern mapped most closely onto higher visual networks, while
in logopenic variant PPA (lvPPA), the atrophy pattern mapped most
closely onto the language network [147]. It is also well-recognized
that early-onset AD differs in many respects from late-onset AD;
whereas in late-onset AD the DMN is most prominently affected as
described above, in early-onset AD the salience and executive control
networks seem most robustly affected [148, 149]. This fits with the
atypical clinical phenotypes commonly seen in early-onset AD [150].
Decreased DMN connectivity was also found to be predictive
of clinical conversion to AD in patients with MCI [139, 151].
With the rise of biomarkers of amyloid, an increasing number of
studies have investigated the functional connectivity abnormalities
associated with preclinical AD (asymptomatic cerebral amyloido-
sis). There is consistent evidence for disrupted DMN connectivity
in preclinical AD [152–156]. See below for a discussion of APOE
and other genetic effects.
In patients with bvFTD, the resting state fMRI connectivity of
the salience network is consistently abnormal [133, 145]; in one
study, this was seen in both bvFTD and semantic dementia [157].
The degree of abnormality of functional connectivity in the salience
network correlates with symptom severity [133, 157]. Interestingly,
some investigations have reported increased DMN connectivity in
bvFTD [133], although other studies have reported reduced DMN
connectivity [145, 158]. In another study of semantic variant PPA,
graph theoretical analyses of resting state fMRI data demonstrate
loss of integrity in multimodal ventral temporal cortex as well as in
the modality-specific caudal ventral visual stream [159]. Surprisingly,
there have been no other studies of functional connectivity in PPA.
In patients with the typical “Richardson” PSP clinical syn-
drome, resting state fMRI has been used to identify a dorsal mid-
brain tegmentum-thalamocortical network in healthy individuals
that shows reduced connectivity in patients with PSP, with the
magnitude of reduced connectivity correlating with symptom
severity [160]. The topography of this network bears remarkable
similarity to the distributed set of regions known to accrue tau
pathology in PSP [161]. In a separate study, PSP patients showed
reduced thalamo-cortical connectivity which correlated with cog-
nitive and motor symptoms [162]. To date, there have been no
resting state fMRI studies of CBD.
712 Bradford C. Dickerson et al.

In ALS, most resting state fMRI studies have focused on the

sensorimotor network, with findings of either reduced or mixed
changes in connectivity relative to controls [163]. Very little work
in this area has included participants with cognitive or behavioral
impairment and there is not yet a study using resting state fMRI in
patients with ALS-FTD. In one study, a subset of the ALS patients
exhibited cognitive or behavioral impairment [164]. As a group,
the ALS patients showed both reductions and increases in connec-
tivity within the DMN and frontoparietal network with no differ-
ences in executive or salience networks. Within the group of ALS
patients, there was an inverse correlation between performance on
the Wisconsin Card Sort Test and connectivity within the DMN as
well as within the frontoparietal network, suggesting that at least
some of the increases in connectivity relate to impairments in cogni-
tive performance.
Early resting state fMRI studies of DLB produced substantially
differing results. One study showed that connectivity was reduced
between the precuneus seed and medial prefrontal cortex and hip-
pocampus, while it was increased between the precuneus seed and
regions of the dorsal attention network and putamen [165].
Another study found increased connectivity between the posterior
cingulate cortex seed and anterior cingulate, culmen, cerebellum,
and putamen with no areas of decreased connectivity [166]. In
another analysis focused on subcortical connectivity, seed regions in
bilateral caudate, putamen, and thalamus showed greater connec-
tivity with other structures than controls [167], with no reduced
connectivity compared to controls. Using independent component
analysis, another analysis demonstrated reduced functional connec-
tivity within the DMN, salience, and executive networks with
increased basal ganglia connectivity compared to controls [168].
Several resting state fMRI studies have been conducted of PD
with mild cognitive impairment (PD-MCI) or PD dementia
(PDD). In one study which employed seed-based analysis, the
DMN showed no differences but the caudate seed showed reduced
connectivity in PDD with dorsolateral prefrontal cortex and puta-
men [169]. Another group used the posterior cingulate cortex/
precuneus as a seed for a DMN analysis and reported decreased
connectivity in the right inferior frontal gyrus in PDD as compared
to PD and controls [170]. In another study, PDD patients demon-
strated reduced connectivity within the DMN compared to PD
patients without cognitive impairment, and widespread reductions
compared with controls [171]. In another study of PD with or
without MCI, the DMN was found using independent component
analysis to be reduced in both groups compared to controls, while
the connectivity of bilateral prefrontal cortex within the frontopa-
rietal network was reduced and the strength of connectivity cor-
related with visuospatial cognitive test performance [172]. Finally,
another study examined local resting signal fluctuations in PD-MCI
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 713

or PDD and found reduced spontaneous brain activity in regions

important for motor control (e.g., caudate, supplementary motor
area, precentral gyrus, thalamus), attention and executive func-
tions (e.g., lateral prefrontal cortex), and episodic memory (e.g.,
precuneus, angular gyrus, hippocampus) [173].
Another study employed both independent component and
seed-based analyses to study PD-MCI. Independent component
analysis results revealed reduced connectivity between the dorsal
attention network and frontoinsular regions, associated with worse
performance in attention/executive functions. The DMN dis-
played increased connectivity with medial and lateral occipito-
parietal regions, associated with worse visuospatial performance.
The seed-based analyses mainly revealed reduced within both the
DMN and dorsal attention network, along with disruptions of dor-
sal attention-frontoparietal and DMN-dorsal attention network
interactions [174].
A series of recent resting state fMRI studies have investigated
group differences in PD and DLB compared to each other or to
patients with PD but no cognitive impairment. In a study investi-
gating both PDD and DLB, PDD was associated with local con-
nectivity reductions in frontal regions while DLB was associated
with local connectivity reductions in posterior cortical regions
[175]. Another study compared PDD and DLB and found reduc-
tions in both frontoparietal networks and supplementary motor
network in both diseases [176].
Other recent studies have focused on resting state fMRI corre-
lates of specific symptoms. Fluctuations in cognition in DLB were
associated with reduced connectivity in the left frontoparietal net-
work [177]. Apathy in PD was found to be associated with reduced
connectivity in limbic striatal and frontal circuits, but surprisingly,
the severity of apathy was inversely correlated with connectivity in
these circuits [178]. Visual hallucinations in PD were associated
with increased occipito-striatal [179] and DMN connectivity [180].
In HD, several resting state fMRI studies have been conducted
with mixed results in analysis of sensorimotor and basal ganglia–
thalamocortical networks [181–184]. With regard to cognitive
networks, HD patients showed widespread reduction in synchrony
in the dorsal attention network, which was associated with poorer
cognitive performance [182]. Widespread DMN changes, not cor-
relating with the atrophy of the involved nodes, also appear to be
present in symptomatic HD patients [181], and correlate with
cognitive disturbances [183].
In part from functional connectivity MRI studies (as well as
from diffusion tensor imaging structural connectivity MRI stud-
ies), the hypothesis that neurodegeneration progresses along con-
nectional pathways has received greater support. Seeley et al. [185]
provided a compelling set of results relating the spatial topographic
patterns of atrophy in five neurodegenerative syndromes (AD,
714 Bradford C. Dickerson et al.

bvFTD, semantic variant PPA, nonfluent variant PPA, and cortico-

basal syndrome) to the topography of large-scale networks in the
healthy brain as measured by resting state fMRI and gray matter
structural covariance. This seminal observation has spurred a grow-
ing body of research and links with studies of the potential cell-to-
cell transmission of misfolded proteins [186]. Technical advances in
graph theoretical and related mathematical modeling approaches to
functional and diffusion MRI data are leading to new ideas about
how the human brain connectome may provide important context
for neurodegeneration [187].

4.3 Compensatory Aside from neocortical hyperactivation in AD, consistent data sug-
Hyperactivation: gest that there is a phase of increased MTL activation in MCI
A Universal Adaptation (Fig. 3) [188]. This increase, which also may be present in cogni-
Response to Brain tively intact carriers of the APOE-ε4 allele (for review, see [189]),
Injury? may represent, at least in part, an attempted compensatory response
to AD neuropathology [190–192], given that some MCI individu-
als with smaller hippocampal volume perform similarly on memory
tasks to MCI individuals with larger hippocampal volume but have
relatively greater MTL activation [26, 74]. Additional studies
employing event-related fMRI paradigms [18, 24, 75] will be very
helpful in determining whether increased MTL activation in MCI
patients is specifically associated with successful memory, as
opposed to a general effect that is present regardless of success
(possibly indicating increased effort). Whether or not hyperactiva-
tion is associated with better memory performance and thus could
be viewed as behaviorally compensatory, it appears to be associated
with more prominent neurodegeneration [193] and poorer prog-
nosis [194]. It is possible that MTL hyperactivation reflects cholin-
ergic or other neurotransmitter upregulation in MCI patients
[195]. Alternatively, increased regional brain activation may be a
marker of the pathophysiologic process of AD itself, such as aber-
rant sprouting of cholinergic fibers [196] or inefficiency in synaptic

Fig. 3 A phase of compensatory hyperactivation appears to occur in the medial temporal lobe (MTL) in mild
cognitive impairment (MCI), prior to the clinical onset of Alzheimer’s disease (AD) dementia. Representative
single subjects from each group, showing normal memory-related MTL activation measured with fMRI in
normal older controls, hyperactivation, and very mild atrophy in MCI, and hypoactivation and more prominent
atrophy in mild AD (reproduced with permission from [76])
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 715

transmission [197] or aberrant synchronous firing along the lines

of epileptiform discharges [198]. In fact, the treatment of MTL
hyperactivation using the anticonvulsant levitiracetam in MCI is
associated with improved memory performance [199]. It is impor-
tant, however, to acknowledge that multiple nonneural factors may
confound the interpretation of changes in the hemodynamic
response measured by BOLD fMRI, such as age- and disease-
related changes in neurovascular coupling [54, 55], AD-specific
alterations in vascular physiology [200], and resting hypoperfusion
and metabolism in MCI and AD [201], which may result in an
amplified BOLD fMRI signal during activation [202, 203]. Further
research to determine the specificity of hyperactivation with respect
to particular brain regions and behavioral conditions will be valu-
able to better characterize this phenomenon [76, 77].
“Compensation” is typically defined as greater regional brain
activity (hyperactivation) in an MCI/AD group in the setting of
task performance accuracy that is similar to that of a matched con-
trol group. Regional hyperactivation may involve greater magni-
tude of activity in brain regions typically active during performance
of the task (when performed by controls), or the recruitment of
additional brain regions not normally engaged by controls.
However, it is also clear that greater task difficulty may provoke
similar alterations in regional brain activity in healthy individuals
[204–207]. It is challenging to know to what degree MCI/AD
groups find memory tasks to be “more difficult” than they would
in the absence of disease. This has led some investigators to attempt
to match task difficulty between MCI/AD patients and controls
[192]. It is also possible that different cognitive strategies during
memory task performance (e.g., semantic elaborative encoding
strategies vs. visualization strategies) may contribute to differences
in the recruitment of particular brain regions [208] and that this
may vary between patient and control groups. Further work in this
area, including longitudinal studies in MCI/AD patients [52], ide-
ally including detailed behavioral measures of reaction time as well
as accuracy and possibly self-report of task difficulty, will be impor-
tant to better clarify the situations in which activity increases can be
reasonably interpreted as compensatory for brain disease.
In PD, a potential compensatory response was demonstrated
in an fMRI study showing that maintenance of movement is
accompanied by hyperactivation in lateral premotor areas [209]. In
the same subjects, dopaminergic therapy normalized these activa-
tion patterns in the setting of constant motor performance (see
later for additional discussion of pharmacologic fMRI). Monchi
et al. [210] showed that, during a set-shifting task, PD patients
have reduced ventrolateral prefrontal activity relative to controls,
but greater dorsolateral prefrontal activity, suggesting not only
that frontostriatal circuits are dysfunctional in PD but that there
also may be attempted compensatory activity. An elegant study of
716 Bradford C. Dickerson et al.

motor imagery in PD patients with strongly lateralized symptoms

demonstrated that when patients judged the laterality of hand
images in different orientations, occipito-parietal cortex hyperacti-
vation was most prominent for imagery employing the affected
hand compared with the unaffected hand [211].
In HD, hyperactivation was observed in multiple regions dur-
ing a visual attention/interference task [212]. Notably, greater
premotor activation correlated with a greater degree of clinical
impairment, supporting the conjecture that response that is at least
attempting to compensate for the disease. The authors suggest that
the HD patients may have required increased effort to inhibit inap-
propriate motor responses. As in most studies identifying possible
compensatory hyperactivation, there are a number of other inter-
pretations of how hyperactivation may relate to severity of illness.
In ALS, sensorimotor cortical hyperactivation is present in
comparison to both healthy normal controls and to controls with
peripheral motor weakness, indicating that the hyperactivation is
not purely a reflection of weakness [213]. Schoenfeld et al. [214]
used a button-press sequencing task to investigate whether task
difficulty level was primarily responsible for greater activation
within motor circuits. Although during the simple task ALS
patients showed motor hyperactivation compared with controls,
when the task was manipulated such that controls had to respond
more rapidly and thus made more errors (equivalent to those of
ALS patients in the simpler task), motor activation was similar
between the two groups. Functional recruitment of cerebral
regions involved in motor learning has also been noted in ALS
patients, including basal ganglia, cerebellum, and brainstem [215].
Despite the caveats mentioned earlier with regard to many of
the studies of hyperactivation in neurodegenerative diseases, accu-
mulating evidence suggests that task-related regional brain hyper-
activation may be a universal neural response to insult, as it occurs
in sleep deprivation [216], aging [217], and a variety of neuropsy-
chiatric disorders and conditions, including AD/MCI, PD, ALS,
cerebrovascular disease [218, 219], multiple sclerosis [220, 221],
traumatic brain injury [222], human immunodeficiency virus
(HIV) [223], alcoholism [224], and schizophrenia [225]. In many
of these studies, task-related regional brain hyperactivation was
associated with the relative preservation of performance on the
task, suggesting that hyperactivation may be serving, at least in
part, a compensatory role for neurologic insult. The evidence dis-
cussed earlier also indicates that increased MTL activation can be
seen in MCI in the setting of minimal MTL atrophy [70], which
provides in vivo support for laboratory data suggesting that physi-
ologic alterations may precede significant structural abnormalities
very early in the course of a neurodegenerative disease such as AD
[4, 226] and may represent inefficient neural circuit function
[197]. Thus, fMRI may provide a means to detect changes in
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 717

Regional circuit hyperactivation under task demand

Behavioral Abilities
Regional brain hypometabolism

Regional brain atrophy

Presymptomatic Prodromal Overtly symptomatic

Fig. 4 Illustration of degenerative-compensatory model, proposed as a universal

response to brain insult. In the case of neurodegenerative diseases, the model
proposes that there is a phase of task-related hyperactivation of regional brain
circuits subserving task performance, followed by the gradual development of
regional hypometabolism and atrophy as the disease progresses from presymp-
tomatic to prodromal to overtly symptomatic phases

human brain circuit function that underlie the earliest symptoms of

neurodegenerative diseases and may be useful in identifying groups
of subjects at high risk for future decline prior to a clinical diagno-
sis of these diseases (Fig. 4).
It is possible, however, that hyperactivation reflects inefficient
function of neural circuits in the face of injury, and that such a
response may be deleterious in the long run. Thus, it will be critical
to elucidate the relationships between behavioral performance,
neural circuit function, and clinical course of disease, with the ulti-
mate goal of determining how best to use these fMRI measures as
biomarkers of putative therapeutic response in clinical trials.
Numerous resting state fMRI studies have also found increased
connectivity [227], and efforts are also ongoing to better interpret
these findings in the context of clinical or longitudinal data.

4.4 The Modulatory In the last decade, there has been an explosion in literature on
Effects of Genetic Risk imaging and genetics, primarily in psychiatric disorders [228] and
Factors for Neurologic the basic science of genetic modulators of brain function [229,
Disease on Brain 230]. This is an area that is ripe for study in neurologic disease,
Activation with a number of studies having been done in populations at ele-
vated genetic risk for AD, FTD, ALS, and HD.
The APOE ε4 allele is a major genetic susceptibility factor
associated with increased risk for AD. Several fMRI studies have
718 Bradford C. Dickerson et al.

investigated regional brain activation during task performance in

cognitively intact subjects stratified by their APOE allele status.
Smith et al. reported decreased activation in inferior temporal
regions on a visual naming and a letter fluency fMRI paradigm
(there was no hippocampal or other medial temporal activation
reported with these tasks) in APOE ε4 carriers [231]. In a subse-
quent report, this group reported increased parietal activation in
women with an APOE ε4 allele [232]. Bookheimer et al. reported
increased activation in left hippocampal, parietal, and prefrontal
regions among APOE ε4 carriers, compared with noncarriers,
using a word-pair associative memory paradigm [233]. In addi-
tion, an increased number of activated regions in the left hemisphere
at baseline was associated with a decline in memory at the 2-year
follow-up among the APOE ε4 carriers. The authors hypothesized
that this increase in activation in the APOE ε4 carriers might rep-
resent the additional cognitive effort or neuronal recruitment
required to adequately perform the task. Similarly increased activa-
tion in multiple brain regions was recently reported in cognitively
intact older APOE ε4 carriers compared with ε3 carriers, although
the effect was lateralized to the right MTL region (left hippocam-
pal activation was greater in ε3 carriers) [234]. Among a group of
29 controls, MCI subjects, and AD patients, Dickerson et al.
reported that 13 APOE ε4 carriers demonstrated greater entorhi-
nal activation than noncarriers, in the absence of genotype-related
differences in the volumes of these regions [70]. Other studies
suggest that decreased medial temporal activation may also be seen
in APOE ε4 carriers [235].
With regard to functional connectivity, a number of studies
have identified reduced DMN connectivity in cognitively normal
middle-aged or older adults who carry the APOE ε4 allele [236–
239], suggesting the possibilities that either early preclinical patho-
logical changes associated with AD may be present or that this
genetic risk factor itself may alter brain network function (Fig. 5).
In these studies, the amyloid status of participants was unknown. In
another investigation, amyloid-negative APOE ε4 carriers were
compared to noncarriers and found to have reduced DMN connec-
tivity [240]. This latter finding raises the question of whether APOE
itself influences the efficiency of this neural network independently
of its promotion of amyloid pathology. In support of the hypothesis
that APOE modulates network efficiency, two studies of healthy
young adults have found increased DMN connectivity in APOE ε4
carriers [241, 242]. These findings are challenging to interpret at
present, but raise the possibility that APOE ε4 may have both devel-
opmental effects on the connectivity of the DMN as well as its well-
known effect of increasing the likelihood of amyloid pathology as a
function of age.
A number of studies of functional activation and connectivity in
autosomal dominant forms of AD have been conducted in the past
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 719

Fig. 5 APOE ε4 status reduces DMN connectivity in healthy older adults. Functional connectivity is decreased
in ε4 carriers compared to ε3 homozygotes in both the (a) anterior, and (b) posterior DMN. The bar graphs
depict the distribution of the effects across subgroups. They show the mean parameter estimates of a selected
region within the anterior DMN (the right anterior cingulate gyrus) and the left posterior cingulate cluster within
the posterior DMN, for male and female ε3 homozygotes and male and female ε4 carriers. The difference
across both genotype and gender is significant for the anterior cingulate gyrus. For the posterior cingulate
gyrus only the difference across genotype is significant. The statistical maps are overlaid on the Montreal
Neurological Institute (MNI) 152 brain; MNI coordinates (in mm) of the slices are displayed. Figure modified
from [239] with permission

decade. During memory encoding, presymptomatic carriers of pre-

senilin (PSEN) mutations show hippocampal hyperactivation [243,
244] and reduced precuneus deactivation [244, 245]. DMN func-
tional connectivity is reduced in presymptomatic carriers of autoso-
mal dominant mutations as they approach estimated age of onset in
their family [134, 246]. Yet children who carry these mutations
show increased DMN functional connectivity [245], again suggest-
ing a possible inverse U-shaped curve of hyperconnectivity prior to
loss of connectivity. These findings are supported by similar previ-
ous results in a small study of two PSEN mutation carriers com-
pared to three noncarriers [247]. In summary, these findings
suggest that the increased activation seen in the APOE ε4 allele
carriers may not be due solely to compensatory mechanisms but
may indicate, in part, an independent physiological mechanism.
With regard to FTD, a few studies of resting state fMRI have
been performed in presymptomatic carriers of autosomal domi-
nant mutations associated with these FTD and/or ALS. In eight
presymptomatic MAPT carriers, reduced DMN connectivity was
720 Bradford C. Dickerson et al.

observed with no change in salience network connectivity [158]. In

a mixed group of MAPT and GRN carriers, presymptomatic muta-
tion carriers showed an age-related reduction in connectivity
between the anterior insula and the anterior cingulate cortex [248]
which was not seen in controls, suggesting that as carriers approach
symptom onset the integrity of this network declines. In a group of
GRN carriers, presymptomatic carriers showed increased connec-
tivity in the salience network on one study [249] and in another
study showed reduced connectivity in the frontoparietal network/
dorsal attention network with increased connectivity within the
executive control network [250]. In this same study, the authors
investigated the modulatory effects of a polymorphism in
TMEM106B which has been shown to influence age of onset in
GRN carriers; they found that carriers of the increased risk allele
showed reduced connectivity in the frontoparietal network and
ventral salience network compared to noncarriers [250]. One rest-
ing state fMRI study has been conducted of symptomatic carriers of
the C9orf72 repeat expansion, a major genetic factor contributing
to FTD and ALS. FTD patients with this mutation show reductions
in connectivity in the salience and sensorimotor networks; salience
network connectivity reduction correlated with atrophy in the pul-
vinar thalamic nucleus, which is known to exhibit pathology in this
form of the disease [251].
A few task-related fMRI studies have been performed in indi-
viduals with presymptomatic HD. An attentional interference task
was used to identify reduced anterior cingulate activation in pres-
ymptomatic huntingtin mutation carriers compared to controls;
carriers also had subtle performance abnormalities on the task
[252]. Two studies have focused on mood-related abnormalities.
In the first, negative feedback on performance was given to partici-
pants, and abnormally reduced amygdala activation and aberrant
amygdala-orbitofrontal coupling was observed in presymptomatic
HD individuals [253]. In the other study, a mood induction task
was used to identify increased activation in presymptomatic HD
individuals relative to controls of pulvinar, cingulate cortex, and
somatosensory association cortex; pulvinar activation correlated
inversely with putaminal gray matter volume and directly with clin-
ical ratings of irritability [254].
Reduced dorsolateral prefrontal cortical activation was
observed in presymptomatic HD individuals on a working mem-
ory task [255]; this reduction was stable over a 2-year period
[256]. Working-memory task-related functional connectivity was
reduced in presymptomatic HD [257]. Presymptomatic HD indi-
viduals showed increased recruitment of prefrontal regions but
reduced task-related connectivity between these regions longitudi-
nally over an 18-month period [258].
In a resting state functional connectivity study, presymptom-
atic HD individuals showed decreased correlated activity in the
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 721

sensorimotor and dorsal attention networks; decreased level of

synchrony in the sensorimotor network was associated with poorer
motor performance [182]. In presymptomatic HD individuals,
DMN connectivity was reduced [259]. In another study of pres-
ymptomatic HD, the authors found no longitudinal change in
resting-state connectivity over 3-year period [260].

4.5 fMRI Very little work has been done on the use of fMRI as a predictive
as a Predictive biomarker for prognosis. Miller et al. pursued such a study of a
Biomarker group of 25 senior citizens spanning the spectrum of MCI, none
of whom were demented at the time of baseline assessment, but
who exhibited varying degrees of mild symptoms of cognitive
impairment clinically [as measured using the CDR sum-of-boxes
(CDR-SB)] [194]. At baseline, subjects performed a visual scene-
encoding task during fMRI scanning and were clinically followed
longitudinally after scanning. Over about 6 years of follow-up after
scanning, subjects demonstrated a wide range of cognitive decline,
with some showing no change and others progressing to dementia
(change in CDR-SB ranged from 0 to 6). The degree of cognitive
decline was predicted by hippocampal activation at the time of
baseline scanning, with greater hippocampal activation predicting

Fig. 6 fMRI as a predictive quantitative imaging biomarker. In a group of mild

cognitive impairment patients, hippocampal activation at baseline predicts the
degree of cognitive decline over 6 years after scanning. Scatterplot shows, on the
Y-axis, parameter estimates (representing percent BOLD signal change) of dif-
ferential hippocampal activation in novel versus repeated contrast. The X-axis
shows estimated rate of change in CDR SB score per year after fMRI scan in
participants who remained classified as having MCI and in those who were diag-
nosed with probable AD during the follow up interval. Figure reprinted from [194]
with permission
722 Bradford C. Dickerson et al.

greater decline (Fig. 6). This finding was present even after con-
trolling for baseline degree of impairment (CDR-SB), age, educa-
tion, and hippocampal volume, as well as gender and APOE status.
Similarly, a longitudinal fMRI study showed that healthy subjects
with more rapid cognitive decline over a 2-year period had both
the highest hippocampal activation at baseline and the greatest loss
of hippocampal activation over follow up, where the rate of activa-
tion loss correlated with the rate of cognitive decline [52]. These
data suggest that fMRI may provide a physiologic imaging bio-
marker useful for identifying the subgroup of MCI individuals at
highest risk of cognitive decline for potential inclusion in disease-
modifying clinical trials. Thus, hyperactivation might represent an
early response to AD pathology, which may predict forthcoming
hippocampal failure and memory decline.

4.6 Uses of fMRI FMRI may be particularly valuable in evaluating acute and subacute
in Understanding effects of therapeutic interventions—whether pharmacologic, non-
and Monitoring pharmacologic (e.g., transcranial magnetic stimulation), or behav-
Neurotherapeutics ioral/rehabilitative—on neural activity [261]. This may be useful
for showing that the intervention modulates targeted circuits and
may help elucidate mechanisms of action. Additionally, it may help
identify or even predict treatment responders or nonresponders.
Alterations in memory-related activation related to the admin-
istration of pharmacologic agents known to impair memory can be
detected with pharmacologic fMRI [262–264]. The effects of cog-
nitive enhancing drugs on brain activation during cognitive task
performance have shown that fMRI can detect changes after
administration of cholinesterase inhibitors in patients with AD and
MCI [265, 266]. In one of the earliest studies, after receiving a
single dose of galanthamine, AD patients demonstrated increased
fusiform activity during a face encoding task and increased prefron-
tal activity during a working memory task [265]. Another study
showed that acute dosing increased hippocampal activation during
a memory task while chronic dosing was associated with decreased
activation [267]. Although these pilot studies did not include pla-
cebo-control groups to reduce potential confounding factors, such
as learning effects, they indicate that fMRI is sensitive to both
acute and subacute medication effects, some of which relate to
behavioral change. Similar findings have been observed in subse-
quent placebo-controlled acute dosing studies [268]. Another
placebo-controlled acute dose physostigmine study demonstrated
normalization of both visual stimulus-specific activation in occipi-
tal and attention-dependent activation in frontoparietal regions in
patients with AD [269].
Pharmacological fMRI has also been used to evaluate the effects
of chronic treatment over 2–6 months in patients with AD demen-
tia. For the most part, these studies have demonstrated treatment-
related increased activation in brain regions engaged by the variety
of tasks used [270–272]. Whereas some studies have found
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 723

decreased activation in areas engaged by the task at baseline [267,

273, 274]. However, some studies have demonstrated a more com-
plex effect, with relatively greater prefrontal activation increases
being associated with a more advanced stage of cognitive impair-
ment [268], supporting the perspective that task-related activation
in the MCI-AD dementia spectrum whether on or off drug changes
with the course of the disease, presenting challenges for attempts at
group comparisons. Nevertheless, at least two studies have demon-
strated that increased task-related activation in association with
cholinesterase inhibitor therapy is associated with improved cogni-
tive function in patients with AD dementia [275, 276].
Pharmacological fMRI can be used to extend our understand-
ing of the effects of standard AD therapies on brain function.
During the auditory encoding of sentences, healthy individuals
show a suppression of activity in auditory cortex; AD patients do
not, but treatment with donepezil normalizes this activity and is
associated with better memory [277]. A subsequent analysis of
these data also demonstrated attenuated activity within the execu-
tive function network during verbal recall, which was also partially
normalized after donepezil treatment [278].
Fewer pharmacological fMRI investigations have been done of
MCI. In one of the first, MCI patients who received 6 weeks of
donepezil showed increased prefrontal activation after this course
of medication, which related to improvements in performance of a
working memory task [266]. Similarly, after MCI patients received
galanthamine for approximately 1 week, performance on a work-
ing memory task was improved in conjunction with increased acti-
vation in precuneus and middle frontal regions. In addition,
increases in hippocampal, prefrontal, cingulate, and occipital
regions were seen during an episodic encoding task, although per-
formance did not improve on this memory task [279]. In another
study, increased hippocampal activation in MCI patients after 7
days of galathamine treatment was associated with neuropsycho-
logical improvement [280]. In a more recent study, investigators
compared task-related activity during a memory encoding para-
digm in an MCI group before and after 3 months of treatment
with donepezil. After treatment, the medial temporal lobe hypoac-
tivation and medial parietal hypo-deactivation seen at baseline were
both normalized in association with improved cognitive perfor-
mance [281], similar to findings observed in other chronic cholin-
ergic therapy pharmacological fMRI studies [282, 283]. In the
latter study, after 3 months of donepezil treatment, functional con-
nectivity during episodic encoding increased between the fusiform
cortex and hippocampus [283].
Several recent studies have begun to employ resting-state fMRI
as a probe to assess the modulatory effects of pharmacologic thera-
pies on AD and MCI, in part because of the ease of its implementa-
tion. Functional connectivity of the hippocampus at rest with other
brain regions was shown to be modulated by 6 weeks of donepezil
724 Bradford C. Dickerson et al.

therapy in patients with AD, with connectivity changes in parahip-

pocampal and frontal regions being correlated with cognitive
improvement [284]. A follow-up analysis of the same dataset along
with arterial spin labeled perfusion MRI showed increases in
regional cerebral blood flow after therapy, with functional connec-
tivity changes in the medial prefrontal cortex being correlated with
cingulate perfusion and cognition [285]. Furthermore, increased
functional connectivity in the medial prefrontal areas demonstrated
an association with Alzheimer’s disease Assessment Scale-Cognitive
subscale (ADAS-cog) score changes (Fig. 7). In a similar study of
mild AD dementia patients before and after 8 weeks of treatment
with donepezil, the investigators observed increases in prefrontal
connectivity [286]. Lorenzi et al. [287] studied AD patients before
and after 6 months of treatment with memantine relative to pla-
cebo, showing a greater DMN connectivity in the precuneus in the
group treated with memantine than the placebo group. A recent
study incorporated both task-related visual scene encoding and
resting-state functional MRI in eight patients with AD dementia
receiving 3 months of donepezil therapy. Resting-state fMRI
showed changes mainly in the parahippocampal cortex while task-
related activation fMRI showed stable middle temporal gyrus acti-
vation in contrast with the control group who showed declining
middle temporal gyral activation [288]. Another study found that
APOE genotype modulated resting-state functional connectivity
measures in patients with AD dementia being treated with cholin-
esterase inhibitors. They found that APOE e4 carriers showed
greater responses to donepezil in functional connectivity of multi-
ple cognitive networks compared to APOE e4 noncarriers with AD
dementia [289]. Taken together, these studies suggest that func-
tional connectivity can be a feasible and valuable biomarker for
tracking treatment-related changes in AD.
In addition to studies of pharmacological interventions, rest-
ing state fMRI has been used to study the effects of acupuncture,
meditation, and cognitive rehabilitative training in the MCI/AD
spectrum. In one study of patients with AD, baseline resting state
scans showed reduced connectivity (compared to controls) between
the hippocampus and frontal and temporal cortex which was
increased following 3 min of acupuncture [290]. Ongoing work is
attempting to determine whether acupuncture stimulation at par-
ticular acupoints is associated with stronger effects than stimula-
tion at other acupoints [291].
In AD dementia patients who participated in meditation train-
ing, there was increased functional connectivity between the poste-
rior cingulate cortex and bilateral medial prefrontal cortex and left
hippocampus compared to controls [292]. An fMRI encoding and
recognition paradigm was used before and after 8 weeks of cogni-
tive rehabilitation training in a group of seven patients with AD
dementia, compared with a control group of eight AD patients
who received either relaxation therapy or no intervention [293].
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 725

Fig. 7 Behavioral significance of functional connectivity changes in the middle and posterior cingulate net-
works in AD patients after 12-week donepezil treatment. In the middle congulate connectivity network (a),
functional connectivity changes in the ventral anterior cingulate cortex and the ventral prefrontal cortex were
significantly correlated with the ADAS-cog score change. Functional connectivity change in the ventral anterior
cingulate cortex was correlated with the ADAS-cog score change also in the posterior cingulate connectivity
network (b). The blue solid circles and dotted lines represent the seed regions and the functional connections,
respectively. Figure modified from [285] with permission

On the recognition task, the group who received cognitive

rehabilitation showed increased activation in bilateral prefrontal
cortex and insula, while the control group showed decreases in
these areas. Another study used memory strategy training in
patients with MCI, and found that following training, hippocampal
activation during a memory task was increased compared to the
control group [294].
726 Bradford C. Dickerson et al.

PD is an excellent clinical scenario in which to apply pharmaco-

logical fMRI, given the typical rapid responsiveness of symptoms to
dopaminergic therapy. In drug-naïve patients with mild PD, SMA
and contralateral motor cortex hypoactivation during a simple finger
movement task normalized with l-dopa therapy [295]. Motor per-
formance was constant across conditions, suggesting that any change
could be ascribed to pharmacological modulation within basal gan-
glia–thalamocortical loops. A l-dopa-induced spatial remapping of
the cortico-striatal resting state fMRI connectivity has been detected
in chronically treated PD patients [296, 297]. Recent studies have
suggested that dopaminergic-related changes in resting state func-
tional connectivity occur also in drug-naïve PD cases [298–300].
In addition to motor behavior, the modulatory effects of dopa-
minergic therapy on cognition and emotion have also been studied
in PD. In one well-controlled study, the modulatory effects of
dopamine replacement were studied using both sensorimotor and
working memory tasks [301]. The cortical motor regions activated
during the motor task showed greater activation during the
dopamine-replete state, but the cortical regions subserving work-
ing memory displayed greater activation during the hypodopami-
nergic state. Interestingly, the greater cortical activation during the
working memory task in the hypodopaminergic state correlated
with errors in task performance, while the increased activation in
the cortical motor regions during the dopamine-replete state was
correlated with improvement in motor function. These results are
consistent with evidence that the hypodopaminergic state is associ-
ated with decreased efficiency of prefrontal cortical information
processing and that dopaminergic therapy improves the physiolog-
ical efficiency of this region, and also indicate that hyperactivation
does not necessarily reflect better behavioral performance.
In a study of emotional face processing in PD, amygdala activa-
tion was reduced compared with controls during a hypodopaminer-
gic state, and partly normalized with dopaminergic therapy [302].
Moving beyond its uses in pharmacologic studies, fMRI has
been used to study the effects of deep brain stimulation (DBS).
DBS is a well-accepted therapeutic modality for PD and is finding
a growing number of applications in other neurologic and psychi-
atric disorders. Initial studies focused on safety and the measure-
ment of BOLD signal changes during on versus off stimulator
activity [303, 304]. More recently, fMRI is elucidating mecha-
nisms through which DBS modulates neural circuits [305, 306].

5 Beyond Exclusion: The Use of Imaging Measures as Disease Biomarkers

At present, the potential efficacy of disease-modifying therapies for

AD and other neurodegenerative diseases is evaluated primarily using
clinical measures of cognition, movement, and other behaviors.
fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 727

In clinical trials, outcome measures are typically performance-based

instruments, such as the ADAS-Cog, or structured surveys of clini-
cian/caregiver impression of change. Although the efficacy of dis-
ease-modifying treatments for AD and other neurodegenerative
diseases must ultimately be demonstrated using clinically meaningful
outcome measures such as the slowing of decline in progression of
symptoms or functional impairment, such trials will likely require
hundreds of patients studied for a minimum of 1–2 years. Thus, sur-
rogate markers of efficacy with less variability than clinical assessments
are desperately needed to reduce the number of subjects. These
markers may also prove particularly valuable in the early phases of
drug development to detect a preliminary “signal of efficacy” over a
shorter time period.
Since the pathophysiologic process underlying cognitive
decline in AD and other neurodegenerative diseases involves the
progressive degeneration of particular brain regions, repeatable
in vivo neuroimaging measures of brain anatomy, chemistry, physi-
ology, and pathology hold promise as an important class of poten-
tial biomarkers [3]. A growing body of data indicates that the
natural history of gradually progressive cognitive decline in AD can
be reliably related to changes in such imaging measures.
Furthermore, regionally specific changes in brain anatomy, chem-
istry, and physiology can be detected by imaging prior to the point
at which the disease is symptomatic enough to make a typical clini-
cal diagnosis. Finally, evidence is accumulating that alterations in
synaptic function are present very early in the disease process, pos-
sibly long before the development of clinical symptoms and signifi-
cant cell loss, which may relate closely to symptomatic progression
in manifest disease [4, 5, 307, 308]. Thus, potential disease-
modifying therapies may act by impeding the accumulation of neu-
ropathology, slowing the loss of neurons, altering neurochemistry,
or preserving synaptic function; neuroimaging modalities exist to
measure each of these putative therapeutic goals, and fMRI could
potentially play a valuable role in the development of new scientific
insights into functional brain abnormalities early in the course of
these diseases and in the development of therapeutic agents.
Although measures of brain structure (MRI) and brain metab-
olism (FDG-PET) are well-accepted as imaging biomarkers of neu-
rodegeneration in AD and other neurodegenerative diseases and
are used clinically, fMRI received approval for current procedural
technology (CPT) codes by the American Medical Association,
one application of which is toward the assessment of complex cog-
nitive and sensorimotor function in patients with neuropsychiatric
disorders [309]. It will be extremely valuable for studies to con-
tinue to collect fMRI data in the context of structural MRI, FDG-
PET, and other multimodal imaging data to begin to understand
the relationships between these data types and the ultimate clinical
or research utility of fMRI in neurodegenerative diseases.
728 Bradford C. Dickerson et al.

6 Conclusions

Functional MRI is a particularly attractive method for use by clinical

investigators to study task-related brain activation or large-scale
functional network connectivity in patients with neurodegenerative
diseases. There has been a dramatic growth of promising fMRI
studies in various neurodegenerative disorders that highlight the
potential uses of fMRI in both basic and clinical spheres of investi-
gation. Functional MRI may provide novel insights into the neural
correlates of cognitive, affective, and sensorimotor abilities, and
how they are altered by neurologic disease and by medications. The
technique may help elucidate fundamental aspects of brain-behavior
relationships, such as the genetic influences on task-related brain
physiology. Functional MRI measures hold promise for multiple
clinical applications, including the early detection and differential
diagnosis, predicting future change in clinical status, and as a marker
of alterations in brain physiology related to neurotherapeutic agents.
The greatest potential of fMRI may lie in the study of very early and
preclinical stages of progressive neurologic diseases, at the point of
subtle neuronal dysfunction prior to overt anatomic pathology, or
as an early readout of target engagement in intervention studies.
There is a need for further validation and reliability studies and con-
tinued technical advances to fully realize the potential of fMRI.

References

1. Perry D et al (2015) Building a roadmap for 8. Sperling RA, Jack CR Jr, Aisen PS (2011)
developing combination therapies for Testing the right target and right drug at the
Alzheimer’s disease. Expert Rev Neurother right stage. Sci Transl Med 3(111):111cm33
15(3):327–333 9. Sperling RA et al (2011) Toward defining the
2. Selkoe DJ (2013) The therapeutics of preclinical stages of Alzheimer’s disease: rec-
Alzheimer’s disease: where we stand and ommendations from the National Institute on
where we are heading. Ann Neurol Aging and the Alzheimer’s Association work-
74(3):328–336 group. Alzheimers Dement 7(3):280–292
3. DeKosky ST, Marek K (2003) Looking back- 10. Albert MS et al (2011) The diagnosis of mild
ward to move forward: early detection of neu- cognitive impairment due to Alzheimer’s dis-
rodegenerative disorders. Science ease: recommendations from the National
302(5646):830–834 Institute on Aging-Alzheimer’s Association
4. Selkoe DJ (2002) Alzheimer’s disease is a syn- workgroups on diagnostic guidelines for
aptic failure. Science 298(5594):789–791 Alzheimer’s disease. Alzheimers Dement
5. Coleman P, Federoff H, Kurlan R (2004) A 7(3):270–279
focus on the synapse for neuroprotection in 11. McKhann GM et al (2011) The diagnosis of
Alzheimer disease and other dementias. dementia due to Alzheimer’s disease: recom-
Neurology 63(7):1155–1162 mendations from the National Institute on
6. Dickerson BC, Atri A (2014) Dementia: Aging and the Alzheimer’s Association work-
comprehensive principles and practice. group. Alzheimers Dement 7(3):263–269
Oxford University Press, New York 12. Boxer AL et al (2013) The advantages of
7. Dickerson BC (2015) Hodges’ frontotempo- frontotemporal degeneration drug develop-
ral dementia, 2nd edn. Cambridge University ment (part 2 of frontotemporal degeneration:
Press, Cambridge, UK the next therapeutic frontier). Alzheimers
Dement 9(2):189–198
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 729

13. Paulsen JS et al (2014) Clinical and biomarker 28. Bennett CM, Miller MB (2013) fMRI reli-
changes in premanifest Huntington disease show ability: influences of task and experimen-
trial feasibility: a decade of the PREDICT-HD tal design. Cogn Affect Behav Neurosci
study. Front Aging Neurosci 6:78 13(4):690–702
14. Mills SM et al (2013) Preclinical trials in 29. Brandt DJ et al (2013) Test-retest reliability
autosomal dominant AD: implementation of fMRI brain activity during memory encod-
of the DIAN-TU trial. Rev Neurol (Paris) ing. Front Psychiatry 4:163
169(10):737–743 30. Brown GG et al (2011) Multisite reliabil-
15. Petersen RC et al (1999) Mild cognitive ity of cognitive BOLD data. Neuroimage
impairment: clinical characterization and out- 54(3):2163–2175
come. Arch Neurol 56(3):303–308 31. Cao H et al (2014) Test-retest reliability of
16. Grundman M et al (2004) Mild cogni- fMRI-based graph theoretical properties dur-
tive impairment can be distinguished from ing working memory, emotion processing,
Alzheimer disease and normal aging for clini- and resting state. Neuroimage 84:888–900
cal trials. Arch Neurol 61(1):59–66 32. Chase HW et al (2015) Accounting for
17. Sperling R, Mormino E, Johnson K (2014) dynamic fluctuations across time when exam-
The evolution of preclinical Alzheimer’s ining fMRI test-retest reliability: analysis of
disease: implications for prevention trials. a reward paradigm in the EMBARC study.
Neuron 84(3):608–622 PLoS One 10(5):e0126326
18. Dickerson BC et al (2007) Prefrontal- 33. de Bertoldi F et al (2015) Improving the
hippocampal-fusiform activity during encod- reliability of single-subject fMRI by weight-
ing predicts intraindividual differences in ing intra-run variability. Neuroimage
free recall ability: an event-related functional- 114:287–293
anatomic MRI study. Hippocampus 17(11): 34. Frassle S et al (2015) Test-retest reli-
1060–1070 ability of dynamic causal modeling for
19. Touroutoglou A et al (2014) Amygdala fMRI. Neuroimage 117:56–66
task-evoked activity and task-free connectiv- 35. Gee DG et al (2015) Reliability of an fMRI
ity independently contribute to feelings of paradigm for emotional processing in a mul-
arousal. Hum Brain Mapp 35(10):5316–5327 tisite longitudinal study. Hum Brain Mapp
20. Brewer JB et al (1998) Making memories: 36(7):2558–2579
brain activity that predicts how well visual 36. Golestani AM et al (2015) Mapping the end-
experience will be remembered. Science tidal CO2 response function in the resting-
281(5380):1185–1187 state BOLD fMRI signal: spatial specificity,
21. Kirchhoff BA et al (2000) Prefrontal-temporal test-retest reliability and effect of fMRI sam-
circuitry for episodic encoding and subsequent pling rate. Neuroimage 104:266–277
memory. J Neurosci 20(16):6173–6180 37. Lukasova K et al (2014) Test-retest reliabil-
22. Wagner AD et al (1998) Building memories: ity of fMRI activation generated by differ-
remembering and forgetting of verbal expe- ent saccade tasks. J Magn Reson Imaging
riences as predicted by brain activity. Science 40(1):37–46
281(5380):1188–1191 38. McGonigle DJ (2012) Test-retest reliabil-
23. Daselaar SM et al (2003) Neuroanatomical ity in fMRI: or how I learned to stop wor-
correlates of episodic encoding and retrieval rying and love the variability. Neuroimage
in young and elderly subjects. Brain 126(Pt 62(2):1116–1120
1):43–56 39. Plichta MM et al (2012) Test-retest reliability
24. Sperling R et al (2003) Putting names to of evoked BOLD signals from a cognitive-
faces: successful encoding of associative mem- emotive fMRI test battery. Neuroimage
ories activates the anterior hippocampal for- 60(3):1746–1758
mation. Neuroimage 20(2):1400–1410 40. Aurich NK et al (2015) Evaluating the reli-
25. Price CJ, Friston KJ (1999) Scanning patients ability of different preprocessing steps to esti-
with tasks they can perform. Hum Brain mate graph theoretical measures in resting
Mapp 8(2–3):102–108 state fMRI data. Front Neurosci 9:48
26. Dickerson BC et al (2004) Medial temporal 41. Birn RM et al (2013) The effect of scan length
lobe function and structure in mild cognitive on the reliability of resting-state fMRI con-
impairment. Ann Neurol 56(1):27–35 nectivity estimates. Neuroimage 83:550–558
27. Grady CL et al (2003) Evidence from func- 42. Braun U et al (2012) Test-retest reliability of
tional neuroimaging of a compensatory resting-state connectivity network character-
prefrontal network in Alzheimer’s disease. istics using fMRI and graph theoretical mea-
J Neurosci 23(3):986–993 sures. Neuroimage 59(2):1404–1412
730 Bradford C. Dickerson et al.

43. Patriat R et al (2013) The effect of resting 57. Johnson SC et al (2000) The relationship
condition on resting-state fMRI reliabil- between fMRI activation and cerebral atrophy:
ity and consistency: a comparison between comparison of normal aging and Alzheimer
resting with eyes open, closed, and fixated. disease. Neuroimage 11(3):179–187
Neuroimage 78:463–473 58. Saykin AJ et al (1999) Neuroanatomic substrates
44. Wisner KM et al (2013) Neurometrics of intrin- of semantic memory impairment in Alzheimer’s
sic connectivity networks at rest using fMRI: disease: patterns of functional MRI activation.
retest reliability and cross-validation using a J Int Neuropsychol Soc 5(5):377–392
meta-level method. Neuroimage 76:236–251 59. Grossman M et al (2003) Neural basis for
45. Atri A et al (2011) Test-retest reliability of semantic memory difficulty in Alzheimer’s dis-
memory task functional magnetic resonance ease: an fMRI study. Brain 126(Pt 2):292–311
imaging in Alzheimer disease clinical trials. 60. Woodard JL et al (2009) Semantic memory
Arch Neurol 68(5):599–606 activation in amnestic mild cognitive impair-
46. Turner JA et al (2012) Reliability of the ment. Brain 132(Pt 8):2068–2078
amplitude of low-frequency fluctuations in 61. Thulborn KR, Martin C, Voyvodic JT
resting state fMRI in chronic schizophrenia. (2000) Functional MR imaging using a visu-
Psychiatry Res 201(3):253–255 ally guided saccade paradigm for comparing
47. Eaton KP et al (2008) Reliability of fMRI activation patterns in patients with probable
for studies of language in post-stroke aphasia Alzheimer’s disease and in cognitively able
subjects. Neuroimage 41(2):311–322 elderly volunteers. AJNR Am J Neuroradiol
48. Kurland J et al (2004) Test-retest reliability of 21(3):524–531
fMRI during nonverbal semantic decisions in 62. Golden HL et al (2015) Functional neu-
moderate-severe nonfluent aphasia patients. roanatomy of auditory scene analysis in
Behav Neurol 15(3–4):87–97 Alzheimer’s disease. Neuroimage Clin
49. Clement F, Belleville S (2009) Test-retest 7:699–708
reliability of fMRI verbal episodic memory 63. Kato T, Knopman D, Liu H (2001)
paradigms in healthy older adults and in per- Dissociation of regional activation in mild
sons with mild cognitive impairment. Hum AD during visual encoding: a functional MRI
Brain Mapp 30(12):4033–4047 study. Neurology 57(5):812–816
50. Zanto TP, Pa J, Gazzaley A (2014) Reliability 64. Machulda MM et al (2003) Comparison
measures of functional magnetic resonance of memory fMRI response among normal,
imaging in a longitudinal evaluation of mild cog- MCI, and Alzheimer’s patients. Neurology
nitive impairment. Neuroimage 84:443–452 61(4):500–506
51. Poudel GR et al (2015) Functional changes 65. Rombouts SA et al (2000) Functional MR
during working memory in Huntington’s imaging in Alzheimer’s disease during mem-
disease: 30-month longitudinal data from ory encoding. AJNR Am J Neuroradiol
the IMAGE-HD study. Brain Struct Funct 21(10):1869–1875
220(1):501–512 66. Small SA et al (1999) Differential regional dys-
52. O’Brien JL et al (2010) Longitudinal fMRI function of the hippocampal formation among
in elderly reveals loss of hippocampal acti- elderly with memory decline and Alzheimer’s
vation with clinical decline. Neurology disease. Ann Neurol 45(4):466–472
74(24):1969–1976 67. Sperling RA et al (2003) fMRI studies of asso-
53. Scheller E et al (2014) Attempted and suc- ciative encoding in young and elderly con-
cessful compensation in preclinical and early trols and mild Alzheimer’s disease. J Neurol
manifest neurodegeneration—a review of task Neurosurg Psychiatry 74(1):44–50
FMRI studies. Front Psychiatry 5:132 68. Schwindt GC, Black SE (2009) Functional
54. Buckner RL et al (2000) Functional brain imaging studies of episodic memory in
imaging of young, nondemented, and Alzheimer’s disease: a quantitative meta-
demented older adults. J Cogn Neurosci analysis. Neuroimage 45(1):181–190
12(Suppl 2):24–34 69. Rombouts SARB et al (2005) Delayed rather
55. D’Esposito M, Deouell LY, Gazzaley A than decreased BOLD response as a marker
(2003) Alterations in the BOLD fMRI signal for early Alzheimer’s disease. Neuroimage
with ageing and disease: a challenge for neu- 26(4):1078–1085
roimaging. Nat Rev Neurosci 4(11):863–872 70. Dickerson BC et al (2005) Increased hip-
56. Grossman M et al (2003) Neural basis for verb pocampal activation in mild cognitive
processing in Alzheimer’s disease: an fMRI impairment compared to normal aging and
study. Neuropsychology 17(4):658–674 AD. Neurology 65(3):404–411
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 731

71. Johnson SC et al (2006) Activation of brain to white matter pathology. Cereb Cortex
regions vulnerable to Alzheimer’s disease: 22(5):1038–1051
the effect of mild cognitive impairment. 85. Mormino EC et al (2012) Abeta Deposition
Neurobiol Aging 27(11):1604–1612 in aging is associated with increases in brain
72. Johnson SC et al (2004) Hippocampal adap- activation during successful memory encod-
tation to face repetition in healthy elderly and ing. Cereb Cortex 22(8):1813–1823
mild cognitive impairment. Neuropsychologia 86. Villemagne VL et al (2015) Tau imaging:
42(7):980–989 early progress and future directions. Lancet
73. Petrella JR et al (2006) Mild cognitive Neurol 14(1):114–124
impairment: evaluation with 4-T functional 87. Virani K et al (2013) Functional neural corre-
MR imaging. Radiology 240(1):177–186 lates of emotional expression processing deficits
74. Hamalainen A et al (2007) Increased in behavioural variant frontotemporal demen-
fMRI responses during encoding in mild tia. J Psychiatry Neurosci 38(3):174–182
cognitive impairment. Neurobiol Aging 88. Agustus JL et al (2015) Functional MRI of
28(12):1889–1903 music emotion processing in frontotemporal
75. Kircher T et al (2007) Hippocampal activa- dementia. Ann N Y Acad Sci 1337:232–240
tion in MCI patients is necessary for success- 89. Chiong W et al (2013) The salience network
ful memory encoding. J Neurol Neurosurg causally influences default mode network
Psychiatry 78(8):812–818 activity during moral reasoning. Brain 136(Pt
76. Dickerson BC, Sperling RA (2008) Functional 6):1929–1941
abnormalities of the medial temporal lobe 90. Sonty SP et al (2003) Primary progressive
memory system in mild cognitive impair- aphasia: PPA and the language network. Ann
ment and Alzheimer’s disease: insights from Neurol 53(1):35–49
functional MRI studies. Neuropsychologia 91. Sonty SP et al (2007) Altered effective con-
46(6):1624–1635 nectivity within the language network in
77. Dickerson BC, Sperling RA (2009) Large- primary progressive aphasia. J Neurosci
scale functional brain network abnormali- 27(6):1334–1345
ties in Alzheimer’s disease: insights from 92. Wilson SM et al (2010) Neural correlates of
functional neuroimaging. Behav Neurol syntactic processing in the nonfluent variant
21(1):63–75 of primary progressive aphasia. J Neurosci
78. Petrella JR et al (2007) Cortical deactivation 30(50):16845–16854
in mild cognitive impairment: high-field- 93. Vandenbulcke M et al (2005) Anterior tem-
strength functional MR imaging. Radiology poral laterality in primary progressive aphasia
245(1):224–235 shifts to the right. Ann Neurol 58(3):362–370
79. Vandenbulcke M et al (2007) Word reading 94. Nelissen N et al (2011) Right hemisphere
and posterior temporal dysfunction in amnes- recruitment during language processing in fron-
tic mild cognitive impairment. Cereb Cortex totemporal lobar degeneration and Alzheimer’s
17(3):542–551 disease. J Mol Neurosci 45(3):637–647
80. Vannini P et al (2012) Age and amyloid- 95. Wilson SM et al (2009) The neural basis of
related alterations in default network habitu- surface dyslexia in semantic dementia. Brain
ation to stimulus repetition. Neurobiol Aging 132(Pt 1):71–86
33(7):1237–1252
96. Goll JC et al (2012) Nonverbal sound pro-
81. Vannini P et al (2013) The ups and downs of cessing in semantic dementia: a functional
the posteromedial cortex: age- and amyloid- MRI study. Neuroimage 61(1):170–180
related functional alterations of the encod-
ing/retrieval flip in cognitively normal older 97. Goll JC et al (2010) Non-verbal sound pro-
adults. Cereb Cortex 23(6):1317–1328 cessing in the primary progressive aphasias.
Brain 133(Pt 1):272–285
82. Sperling RA et al (2009) Amyloid deposition
is associated with impaired default network 98. Elsinger CL et al (2003) Neural basis for
function in older persons without dementia. impaired time reproduction in Parkinson’s
Neuron 63(2):178–188 disease: an fMRI study. J Int Neuropsychol
Soc 9(7):1088–1098
83. Kennedy KM et al (2012) Effects of beta-
amyloid accumulation on neural function 99. Rowe JB, Siebner HR (2012) The motor system
during encoding across the adult lifespan. and its disorders. Neuroimage 61(2):464–477
Neuroimage 62(1):1–8 100. Sabatini U et al (2000) Cortical motor reorgani-
84. Hedden T et al (2012) Failure to modulate zation in akinetic patients with Parkinson’s dis-
attentional control in advanced aging linked ease: a functional MRI study. Brain 123(Pt 2):
394–403
732 Bradford C. Dickerson et al.

101. Wu T, Hallett M (2005) A functional MRI Huntington’s disease. J Neurol Neurosurg

study of automatic movements in patients with Psychiatry 78(2):127–133
Parkinson’s disease. Brain 128(Pt 10):2250–2259 116. Gray MA et al (2013) Prefrontal activ-
102. Wu T et al (2010) Neural correlates of biman- ity in Huntington’s disease reflects cogni-
ual anti-phase and in-phase movements in tive and neuropsychiatric disturbances: the
Parkinson’s disease. Brain 133(Pt 8):2394–2409 IMAGE-HD study. Exp Neurol 239:218–228
103. Rowe JB et al (2010) Dynamic causal 117. Rupp J et al (2011) Abnormal error-
modelling of effective connectivity from related antisaccade activation in premani-
fMRI: are results reproducible and sensi- fest and early manifest Huntington disease.
tive to Parkinson’s disease and its treatment? Neuropsychology 25(3):306–318
Neuroimage 52(3):1015–1026 118. Tessitore A et al (2006) Subcortical motor
104. Ceballos-Baumann AO (2003) Functional plasticity in patients with sporadic ALS: an
imaging in Parkinson’s disease: activation fMRI study. Brain Res Bull 69(5):489–494
studies with PET, fMRI and SPECT. J Neurol 119. Abrahams S et al (2004) Word retrieval in
250(Suppl 1):I15–I23 amyotrophic lateral sclerosis: a functional
105. Grafton ST (2004) Contributions of func- magnetic resonance imaging study. Brain
tional imaging to understanding parkin- 127(Pt 7):1507–1517
sonian symptoms. Curr Opin Neurobiol 120. Witiuk K et al (2014) Cognitive deteriora-
14(6):715–719 tion and functional compensation in ALS
106. Lewis SJ et al (2003) Cognitive impairments measured with fMRI using an inhibitory task.
in early Parkinson’s disease are accompanied J Neurosci 34(43):14260–14271
by reductions in activity in frontostriatal neu- 121. Palmieri A et al (2010) Right hemisphere dys-
ral circuitry. J Neurosci 23(15):6351–6356 function and emotional processing in ALS: an
107. Ekman U et al (2012) Functional brain fMRI study. J Neurol 257(12):1970–1978
activity and presynaptic dopamine uptake in 122. Raichle ME et al (2001) A default mode of
patients with Parkinson’s disease and mild brain function. Proc Natl Acad Sci U S A
cognitive impairment: a cross-sectional study. 98(2):676–682
Lancet Neurol 11(8):679–687 123. Buckner RL, Andrews-Hanna JR, Schacter
108. Nagano-Saito A et al (2014) Effect of mild DL (2008) The brain’s default network: anat-
cognitive impairment on the patterns of omy, function, and relevance to disease. Ann
neural activity in early Parkinson’s disease. N Y Acad Sci 1124:1–38
Neurobiol Aging 35(1):223–231 124. Lustig C et al (2003) Functional deactiva-
109. Voon V et al (2011) Dopamine agonists and tions: change with age and dementia of the
risk: impulse control disorders in Parkinson’s Alzheimer type. Proc Natl Acad Sci U S A
disease. Brain 134(Pt 5):1438–1446 100(24):14504–14509
110. Voon V et al (2011) Impulse control disor- 125. Greicius MD et al (2004) Default-mode
ders in Parkinson disease: a multicenter case– network activity distinguishes Alzheimer’s
control study. Ann Neurol 69(6):986–996 disease from healthy aging: evidence from
111. Sauer J et al (2006) Differences between functional MRI. Proc Natl Acad Sci U S A
Alzheimer’s disease and dementia with Lewy 101(13):4637–4642
bodies: an fMRI study of task-related brain 126. Rombouts SARB et al (2005) Altered resting
activity. Brain 129(Pt 7):1780–1788 state networks in mild cognitive impairment
112. Kim JS et al (2004) Functional MRI study of and mild Alzheimer’s disease: an fMRI study.
a serial reaction time task in Huntington’s dis- Hum Brain Mapp 26(4):231–239
ease. Psychiatry Res 131(1):23–30 127. Celone KA et al (2006) Alterations in
113. Wolf RC et al (2009) Cortical dysfunction memory networks in mild cognitive impair-
in patients with Huntington’s disease during ment and Alzheimer’s disease: an inde-
working memory performance. Hum Brain pendent component analysis. J Neurosci
Mapp 30(1):327–339 26(40):10222–10231
114. Georgiou-Karistianis N et al (2014) 128. Buckner RL et al (2005) Molecular, struc-
Functional magnetic resonance imaging of tural, and functional characterization of
working memory in Huntington’s disease: Alzheimer’s disease: evidence for a relation-
cross-sectional data from the IMAGE-HD ship between default activity, amyloid, and
study. Hum Brain Mapp 35(5):1847–1864 memory. J Neurosci 25(34):7709–7717
115. Thiruvady DR et al (2007) Functional 129. Biswal B et al (1995) Functional connectivity
connectivity of the prefrontal cortex in in the motor cortex of resting human brain
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 733

using echo-planar MRI. Magn Reson Med 145. Filippi M et al (2013) Functional net-
34(4):537–541 work connectivity in the behavioral vari-
130. Vincent JL et al (2007) Intrinsic functional ant of frontotemporal dementia. Cortex
architecture in the anaesthetized monkey 49(9):2389–2401
brain. Nature 447(7140):83–86 146. Agosta F et al (2012) Resting state fMRI in
131. Smith SM et al (2009) Correspondence of Alzheimer’s disease: beyond the default mode
the brain’s functional architecture during network. Neurobiol Aging 33(8):1564–1578
activation and rest. Proc Natl Acad Sci U S A 147. Lehmann M et al (2013) Intrinsic connec-
106(31):13040–13045 tivity networks in healthy subjects explain
132. Yeo BT et al (2011) The organization of the clinical variability in Alzheimer’s disease. Proc
human cerebral cortex estimated by intrin- Natl Acad Sci U S A 110(28):11606–11611
sic functional connectivity. J Neurophysiol 148. Gour N et al (2014) Functional connec-
106(3):1125–1165 tivity changes differ in early and late-onset
133. Zhou J et al (2010) Divergent network con- Alzheimer’s disease. Hum Brain Mapp
nectivity changes in behavioural variant fron- 35(7):2978–2994
totemporal dementia and Alzheimer’s disease. 149. Lehmann M et al (2015) Loss of functional
Brain 133(Pt 5):1352–1367 connectivity is greater outside the default
134. Thomas JB et al (2014) Functional connectivity mode network in nonfamilial early-onset
in autosomal dominant and late-onset Alzheimer Alzheimer’s disease variants. Neurobiol Aging
disease. JAMA Neurol 71(9):1111–1122 36(10):2678–2686
135. Binnewijzend MA et al (2012) Resting-state 150. Barnes J et al (2015) Alzheimer’s disease
fMRI changes in Alzheimer’s disease and first symptoms are age dependent: evidence
mild cognitive impairment. Neurobiol Aging from the NACC dataset. Alzheimers Dement
33(9):2018–2028 11(11):1349–1357
136. Brier MR et al (2012) Loss of intranetwork 151. Petrella JR et al (2007) Prognostic value of
and internetwork resting state functional con- posteromedial cortex deactivation in mild cog-
nections with Alzheimer’s disease progres- nitive impairment. PLoS One 2(10):e1104
sion. J Neurosci 32(26):8890–8899 152. Hedden T et al (2009) Disruption of func-
137. Zhang HY et al (2010) Resting brain con- tional connectivity in clinically normal older
nectivity: changes during the progress of adults harboring amyloid burden. J Neurosci
Alzheimer disease. Radiology 256(2):598–606 29(40):12686–12694
138. Sheline YI et al (2009) The default mode 153. Drzezga A et al (2011) Neuronal dysfunc-
network and self-referential processes in tion and disconnection of cortical hubs in
depression. Proc Natl Acad Sci U S A non-demented subjects with elevated amyloid
106(6):1942–1947 burden. Brain 134(Pt 6):1635–1646
139. Petrella JR et al (2011) Default mode net- 154. Brier MR et al (2014) Functional connectivity
work connectivity in stable vs progres- and graph theory in preclinical Alzheimer’s
sive mild cognitive impairment. Neurology disease. Neurobiol Aging 35(4):757–768
76(6):511–517 155. Wang L et al (2013) Cerebrospinal fluid
140. Li SJ et al (2002) Alzheimer disease: evalu- Abeta42, phosphorylated Tau181, and
ation of a functional MR imaging index as a resting-state functional connectivity. JAMA
marker. Radiology 225(1):253–259 Neurol 70(10):1242–1248
141. Supekar K et al (2010) Development of func- 156. Sheline YI et al (2010) Amyloid plaques dis-
tional and structural connectivity within the rupt resting state default mode network con-
default mode network in young children. nectivity in cognitively normal elderly. Biol
Neuroimage 52(1):290–301 Psychiatry 67(6):584–587
142. Damoiseaux JS et al (2012) Functional 157. Farb NA et al (2013) Abnormal network
connectivity tracks clinical deterioration connectivity in frontotemporal dementia:
in Alzheimer’s disease. Neurobiol Aging evidence for prefrontal isolation. Cortex
33(4):828.e19–30 49(7):1856–1873
143. Schwindt GC et al (2013) Modulation of 158. Whitwell JL et al (2011) Altered functional
the default-mode network between rest and connectivity in asymptomatic MAPT sub-
task in Alzheimer’s Disease. Cereb Cortex jects: a comparison to bvFTD. Neurology
23(7):1685–1694 77(9):866–874
144. Zamboni G et al (2013) Resting functional 159. Agosta F et al (2014) Disrupted brain con-
connectivity reveals residual functional activ- nectome in semantic variant of primary
ity in Alzheimer’s disease. Biol Psychiatry progressive aphasia. Neurobiol Aging
74(5):375–383 35(11):2646–2655
734 Bradford C. Dickerson et al.

160. Gardner RC et al (2013) Intrinsic connectiv- 175. Borroni B et al (2015) Structural and func-
ity network disruption in progressive supra- tional imaging study in dementia with Lewy
nuclear palsy. Ann Neurol 73(5):603–616 bodies and Parkinson’s disease dementia.
161. Williams DR et al (2007) Pathological tau Parkinsonism Relat Disord 21(9):1049–1055
burden and distribution distinguishes pro- 176. Peraza LR et al (2015) Resting state in
gressive supranuclear palsy-parkinsonism Parkinson’s disease dementia and demen-
from Richardson’s syndrome. Brain 130(Pt tia with Lewy bodies: commonalities
6):1566–1576 and differences. Int J Geriatr Psychiatry
162. Whitwell JL et al (2011) Disrupted thalamo- 30(11):1135–1146
cortical connectivity in PSP: a resting-state 177. Peraza LR et al (2014) fMRI resting state
fMRI, DTI, and VBM study. Parkinsonism networks and their association with cognitive
Relat Disord 17(8):599–605 fluctuations in dementia with Lewy bodies.
163. Filippi M et al (2015) Progress towards a neu- Neuroimage Clin 4:558–565
roimaging biomarker for amyotrophic lateral 178. Baggio HC et al (2015) Resting-state fronto-
sclerosis. Lancet Neurol 14(8):786–788 striatal functional connectivity in Parkinson’s
164. Agosta F et al (2013) Divergent brain net- disease-related apathy. Mov Disord
work connectivity in amyotrophic lateral scle- 30(5):671–679
rosis. Neurobiol Aging 34(2):419–427 179. Yao N et al (2015) Resting activity in visual
165. Galvin JE et al (2011) Resting bold fMRI and corticostriatal pathways in Parkinson’s
differentiates dementia with Lewy bod- disease with hallucinations. Parkinsonism
ies vs Alzheimer disease. Neurology Relat Disord 21(2):131–137
76(21):1797–1803 180. Yao N et al (2014) The default mode net-
166. Kenny ER et al (2012) Functional connectiv- work is disrupted in Parkinson’s disease
ity in cortical regions in dementia with Lewy with visual hallucinations. Hum Brain Mapp
bodies and Alzheimer’s disease. Brain 135(Pt 35(11):5658–5666
2):569–581 181. Dumas EM et al (2013) Reduced functional
167. Kenny ER et al (2013) Subcortical connectivity brain connectivity prior to and after disease
in dementia with Lewy bodies and Alzheimer’s onset in Huntington’s disease. Neuroimage
disease. Br J Psychiatry 203(3):209–214 Clin 2:377–384
168. Lowther ER et al (2014) Lewy body 182. Poudel GR et al (2014) Abnormal synchrony
compared with Alzheimer dementia is of resting state networks in premanifest
associated with decreased functional connec- and symptomatic Huntington disease: the
tivity in resting state networks. Psychiatry Res IMAGE-HD study. J Psychiatry Neurosci
223(3):192–201 39(2):87–96
169. Seibert TM et al (2012) Interregional cor- 183. Werner CJ et al (2014) Altered resting-state
relations in Parkinson disease and Parkinson- connectivity in Huntington’s disease. Hum
related dementia with resting functional MR Brain Mapp 35(6):2582–2593
imaging. Radiology 263(1):226–234 184. Quarantelli M et al (2013) Default-mode
170. Rektorova I et al (2012) Default mode network network changes in Huntington’s disease:
and extrastriate visual resting state network in an integrated MRI study of functional con-
patients with Parkinson’s disease dementia. nectivity and morphometry. PLoS One
Neurodegener Dis 10(1–4):232–237 8(8):e72159
171. Chen B et al (2015) Changes in anatomical 185. Seeley WW et al (2009) Neurodegenerative
and functional connectivity of Parkinson’s diseases target large-scale human brain net-
disease patients according to cognitive status. works. Neuron 62(1):42–52
Eur J Radiol 84(7):1318–1324 186. Sanders DW et al (2014) Distinct tau
172. Amboni M et al (2015) Resting-state func- prion strains propagate in cells and mice
tional connectivity associated with mild cog- and define different tauopathies. Neuron
nitive impairment in Parkinson’s disease. 82(6):1271–1288
J Neurol 262(2):425–434 187. Filippi M et al (2013) Assessment of sys-
173. Possin KL et al (2013) Rivastigmine is asso- tem dysfunction in the brain through
ciated with restoration of left frontal brain MRI-based connectomics. Lancet Neurol
activity in Parkinson’s disease. Mov Disord 12(12):1189–1199
28(10):1384–1390 188. Sperling R (2011) Potential of functional
174. Baggio HC et al (2015) Cognitive impair- MRI as a biomarker in early Alzheimer’s dis-
ment and resting-state network connectiv- ease. Neurobiol Aging 32(Suppl 1):S37–S43
ity in Parkinson’s disease. Hum Brain Mapp 189. Wierenga CE, Bondi MW (2007) Use of
36(1):199–212 functional magnetic resonance imaging in
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 735

the early identification of Alzheimer’s disease. 203. Davis TL et al (1998) Calibrated functional
Neuropsychol Rev 17(2):127–143 MRI: mapping the dynamics of oxida-
190. Backman L et al (1999) Brain regions asso- tive metabolism. Proc Natl Acad Sci U S A
ciated with episodic retrieval in normal 95(4):1834–1839
aging and Alzheimer’s disease. Neurology 204. Gur RC et al (1988) Effects of task diffi-
52(9):1861–1870 culty on regional cerebral blood flow: rela-
191. Becker JT et al (1996) Compensatory real- tionships with anxiety and performance.
location of brain resources supporting ver- Psychophysiology 25(4):392–399
bal episodic memory in Alzheimer’s disease. 205. Grasby PM et al (1994) A graded task
Neurology 46(3):692–700 approach to the functional mapping of brain
192. Stern Y et al (2000) Different brain networks areas implicated in auditory-verbal memory.
mediate task performance in normal aging Brain 117(Pt 6):1271–1282
and AD: defining compensation. Neurology 206. Grady CL (1996) Age-related changes in cor-
55(9):1291–1297 tical blood flow activation during perception
193. Putcha D et al (2011) Hippocampal hyper- and memory. Ann N Y Acad Sci 777:14–21
activation associated with cortical thinning 207. Rypma B, D’Esposito M (1999) The roles
in Alzheimer’s disease signature regions in of prefrontal brain regions in components of
non-demented elderly adults. J Neurosci working memory: effects of memory load and
31(48):17680–17688 individual differences. Proc Natl Acad Sci U S
194. Miller SL et al (2008) Hippocampal activation A 96(11):6558–6563
in adults with mild cognitive impairment pre- 208. Kirchhoff BA, Buckner RL (2006) Functional-
dicts subsequent cognitive decline. J Neurol anatomic correlates of individual differences
Neurosurg Psychiatry 79(6):630–635 in memory. Neuron 51(2):263–274
195. DeKosky ST et al (2002) Upregulation of 209. Haslinger B et al (2001) Event-related
choline acetyltransferase activity in hippo- functional magnetic resonance imaging in
campus and frontal cortex of elderly subjects Parkinson’s disease before and after levodopa.
with mild cognitive impairment. Ann Neurol Brain 124(Pt 3):558–570
51(2):145–155 210. Monchi O et al (2004) Neural bases of
196. Hashimoto M, Masliah E (2003) Cycles set-shifting deficits in Parkinson’s disease.
of aberrant synaptic sprouting and neuro- J Neurosci 24(3):702–710
degeneration in Alzheimer’s and demen- 211. Helmich RC et al (2007) Cerebral compensa-
tia with Lewy bodies. Neurochem Res tion during motor imagery in Parkinson’s dis-
28(11):1743–1756 ease. Neuropsychologia 45(10):2201–2215
197. Stern EA et al (2004) Cortical synaptic inte- 212. Georgiou-Karistianis N et al (2007) Increased
gration in vivo is disrupted by amyloid-beta cortical recruitment in Huntington’s dis-
plaques. J Neurosci 24(19):4535–4540 ease using a Simon task. Neuropsychologia
198. Palop JJ, Chin J, Mucke L (2006) A network 45(8):1791–1800
dysfunction perspective on neurodegenerative 213. Stanton BR et al (2007) Altered cortical acti-
diseases. Nature 443(7113):768–773 vation during a motor task in ALS: evidence
199. Bakker A et al (2012) Reduction of hippo- for involvement of central pathways. J Neurol
campal hyperactivity improves cognition in 254(9):1260–1267
amnestic mild cognitive impairment. Neuron 214. Schoenfeld MA et al (2005) Functional motor
74(3):467–474 compensation in amyotrophic lateral sclerosis.
200. Mueggler T et al (2002) Compromised J Neurol 252(8):944–952
hemodynamic response in amyloid precur- 215. Konrad C et al (2006) Subcortical reorgani-
sor protein transgenic mice. J Neurosci zation in amyotrophic lateral sclerosis. Exp
22(16):7218–7224 Brain Res 172(3):361–369
201. El Fakhri G et al (2003) MRI-guided SPECT 216. Drummond SP et al (2000) Altered brain
perfusion measures and volumetric MRI in response to verbal learning following sleep
prodromal Alzheimer disease. Arch Neurol deprivation. Nature 403(6770):655–657
60(8):1066–1072 217. Cabeza R et al (2002) Aging gracefully: com-
202. Cohen ER, Ugurbil K, Kim SG (2002) Effect pensatory brain activity in high-performing
of basal conditions on the magnitude and older adults. Neuroimage 17(3):1394–1402
dynamics of the blood oxygenation level- 218. Carey JR et al (2002) Analysis of fMRI and
dependent fMRI response. J Cereb Blood finger tracking training in subjects with
Flow Metab 22(9):1042–1053 chronic stroke. Brain 125(Pt 4):773–788
736 Bradford C. Dickerson et al.

219. Johansen-Berg H et al (2002) Correlation 235. Johnson SC et al (2006) The influence of

between motor improvements and altered Alzheimer disease family history and apolipo-
fMRI activity after rehabilitative therapy. protein E epsilon4 on mesial temporal lobe
Brain 125(Pt 12):2731–2742 activation. J Neurosci 26(22):6069–6076
220. Morgen K et al (2004) Training-dependent 236. Patel KT et al (2013) Default mode network
plasticity in patients with multiple sclerosis. activity and white matter integrity in healthy
Brain 127(Pt 11):2506–2517 middle-aged ApoE4 carriers. Brain Imaging
221. Reddy H et al (2000) Evidence for adaptive Behav 7(1):60–67
functional changes in the cerebral cortex with 237. Machulda MM et al (2011) Effect of APOE
axonal injury from multiple sclerosis. Brain epsilon4 status on intrinsic network connec-
123(Pt 11):2314–2320 tivity in cognitively normal elderly subjects.
222. McAllister TW et al (1999) Brain activation Arch Neurol 68(9):1131–1136
during working memory 1 month after mild 238. Heise V et al (2014) Apolipoprotein E geno-
traumatic brain injury: a functional MRI type, gender and age modulate connectiv-
study. Neurology 53(6):1300–1308 ity of the hippocampus in healthy adults.
223. Ernst T et al (2002) Abnormal brain acti- Neuroimage 98:23–30
vation on functional MRI in cognitively 239. Damoiseaux JS et al (2012) Gender modu-
asymptomatic HIV patients. Neurology lates the APOE epsilon4 effect in healthy
59(9):1343–1349 older adults: convergent evidence from func-
224. Desmond JE et al (2003) Increased fron- tional brain connectivity and spinal fluid tau
tocerebellar activation in alcoholics during levels. J Neurosci 32(24):8254–8262
verbal working memory: an fMRI study. 240. Sheline YI et al (2010) APOE4 allele dis-
Neuroimage 19(4):1510–1520 rupts resting state fMRI connectivity in the
225. Callicott JH et al (2003) Complexity of pre- absence of amyloid plaques or decreased CSF
frontal cortical dysfunction in schizophre- Abeta42. J Neurosci 30(50):17035–17040
nia: more than up or down. Am J Psychiatry 241. Filippini N et al (2009) Distinct patterns of
160(12):2209–2215 brain activity in young carriers of the APOE-
226. Walsh DM, Selkoe DJ (2004) Deciphering {varepsilon}4 allele. Proc Natl Acad Sci U S A
the molecular basis of memory failure in 106(17):7209–7214
Alzheimer’s disease. Neuron 44(1):181–193 242. Dennis NA et al (2010) Temporal lobe func-
227. Pievani M et al (2014) Brain connectivity tional activity and connectivity in young
in neurodegenerative diseases—from phe- adult APOE varepsilon4 carriers. Alzheimers
notype to proteinopathy. Nat Rev Neurol Dement 6(4):303–311
10(11):620–633 243. Quiroz YT et al (2010) Hippocampal hyperac-
228. Winterer G et al (2005) Neuroimaging and tivation in presymptomatic familial Alzheimer’s
human genetics. Int Rev Neurobiol 67:325–383 disease. Ann Neurol 68(6):865–875
229. Hariri AR, Weinberger DR (2003) Functional 244. Reiman EM et al (2012) Brain imaging and
neuroimaging of genetic variation in seroto- fluid biomarker analysis in young adults
nergic neurotransmission. Genes Brain Behav at genetic risk for autosomal dominant
2(6):341–349 Alzheimer’s disease in the presenilin 1 E280A
230. Nikolova YS, Hariri AR (2015) Can we kindred: a case-control study. Lancet Neurol
observe epigenetic effects on human brain 11(12):1048–1056
function? Trends Cogn Sci 19(7):366–373 245. Quiroz YT et al (2015) Brain imaging and
231. Smith CD et al (1999) Altered brain activa- blood biomarker abnormalities in children
tion in cognitively intact individuals at high with autosomal dominant Alzheimer dis-
risk for Alzheimer’s disease. Neurology ease: a cross-sectional study. JAMA Neurol
53(7):1391–1396 72(8):912–919
232. Smith CD et al (2002) Women at risk for AD 246. Chhatwal JP et al (2013) Impaired default
show increased parietal activation during a network functional connectivity in autoso-
fluency task. Neurology 58(8):1197–1202 mal dominant Alzheimer disease. Neurology
81(8):736–744
233. Bookheimer SY et al (2000) Patterns of brain
activation in people at risk for Alzheimer’s 247. Mondadori CR et al (2006) Enhanced
disease. N Engl J Med 343(7):450–456 brain activity may precede the diagnosis of
Alzheimer’s disease by 30 years. Brain 129(Pt
234. Bondi MW et al (2005) fMRI evidence of 11):2908–2922
compensatory mechanisms in older adults at
genetic risk for Alzheimer disease. Neurology 248. Dopper EG et al (2013) Structural and func-
64(3):501–508 tional brain connectivity in presymptomatic
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 737

familial frontotemporal dementia. Neurology 263. Thiel CM, Henson RN, Dolan RJ (2002)
80(9):814–823 Scopolamine but not lorazepam modulates
249. Borroni B et al (2012) Granulin mutation face repetition priming: a psychopharmacolog-
drives brain damage and reorganization from ical fMRI study. Neuropsychopharmacology
preclinical to symptomatic FTLD. Neurobiol 27(2):282–292
Aging 33(10):2506–2520 264. Leslie RA, James MF (2000) Pharmacological
250. Premi E et al (2014) Effect of TMEM106B magnetic resonance imaging: a new applica-
polymorphism on functional network con- tion for functional MRI. Trends Pharmacol
nectivity in asymptomatic GRN mutation car- Sci 21(8):314–318
riers. JAMA Neurol 71(2):216–221 265. Rombouts SA et al (2002) Alterations in
251. Lee SE et al (2014) Altered network con- brain activation during cholinergic enhance-
nectivity in frontotemporal dementia with ment with rivastigmine in Alzheimer’s
C9orf72 hexanucleotide repeat expansion. disease. J Neurol Neurosurg Psychiatry
Brain 137(Pt 11):3047–3060 73(6):665–671
252. Reading SA et al (2004) Functional brain 266. Saykin AJ et al (2004) Cholinergic enhance-
changes in presymptomatic Huntington’s dis- ment of frontal lobe activity in mild cognitive
ease. Ann Neurol 55(6):879–883 impairment. Brain 127(Pt 7):1574–1583
253. Kloppel S et al (2010) Irritability in pre-clinical 267. Goekoop R et al (2006) Cholinergic chal-
Huntington’s disease. Neuropsychologia lenge in Alzheimer patients and mild cognitive
48(2):549–557 impairment differentially affects hippocampal
254. Van den Stock J et al (2015) Functional brain activation—a pharmacological fMRI study.
changes underlying irritability in premani- Brain 129(Pt 1):141–157
fest Huntington’s disease. Hum Brain Mapp 268. Miettinen PS et al (2011) Effect of cho-
36(7):2681–2690 linergic stimulation in early Alzheimer’s
255. Wolf RC et al (2007) Dorsolateral prefron- disease—functional imaging during a rec-
tal cortex dysfunction in presymptomatic ognition memory task. Curr Alzheimer Res
Huntington’s disease: evidence from event- 8(7):753–764
related fMRI. Brain 130(Pt 11):2845–2857 269. Bentley P, Driver J, Dolan RJ (2008)
256. Wolf RC et al (2011) Longitudinal functional Cholinesterase inhibition modulates visual
magnetic resonance imaging of cognition in and attentional brain responses in Alzheimer’s
preclinical Huntington’s disease. Exp Neurol disease and health. Brain 131(Pt 2):409–424
231(2):214–222 270. Shanks MF et al (2007) Regional brain activ-
257. Wolf RC et al (2008) Altered frontostriatal ity after prolonged cholinergic enhancement
coupling in pre-manifest Huntington’s dis- in early Alzheimer’s disease. Magn Reson
ease: effects of increasing cognitive load. Eur Imaging 25(6):848–859
J Neurol 15(11):1180–1190 271. Kircher TT et al (2005) Cortical activation
258. Georgiou-Karistianis N et al (2013) during cholinesterase-inhibitor treatment
Functional and connectivity changes dur- in Alzheimer disease: preliminary findings
ing working memory in Huntington’s dis- from a pharmaco-fMRI study. Am J Geriatr
ease: 18 month longitudinal data from the Psychiatry 13(11):1006–1013
IMAGE-HD study. Brain Cogn 83(1):80–91 272. Thiyagesh SN et al (2010) Treatment effects
259. Wolf RC et al (2012) Default-mode network of therapeutic cholinesterase inhibitors on
changes in preclinical Huntington’s disease. visuospatial processing in Alzheimer’s disease:
Exp Neurol 237(1):191–198 a longitudinal functional MRI study. Dement
Geriatr Cogn Disord 29(2):176–188
260. Odish OF et al (2015) Longitudinal resting
state fMRI analysis in healthy controls and 273. McGeown WJ, Shanks MF, Venneri A (2008)
premanifest Huntington’s disease gene carri- Prolonged cholinergic enrichment influ-
ers: a three-year follow-up study. Hum Brain ences regional cortical activation in early
Mapp 36(1):110–119 Alzheimer’s disease. Neuropsychiatr Dis
Treat 4(2):465–476
261. Hampel H et al (2014) Perspective on
future role of biological markers in clinical 274. Bokde AL et al (2009) Decreased activa-
therapy trials of Alzheimer’s disease: a long- tion along the dorsal visual pathway after a
range point of view beyond 2020. Biochem 3-month treatment with galantamine in mild
Pharmacol 88(4):426–449 Alzheimer disease: a functional magnetic reso-
nance imaging study. J Clin Psychopharmacol
262. Sperling R et al (2002) Functional MRI 29(2):147–156
detection of pharmacologically induced
memory impairment. Proc Natl Acad Sci U S 275. McLaren DG et al (2012) Tracking cogni-
A 99(1):455–460 tive change over 24 weeks with longitudi-
738 Bradford C. Dickerson et al.

nal functional magnetic resonance imaging 288. Sole-Padulles C et al (2013) Donepezil treat-
in Alzheimer’s disease. Neurodegener Dis ment stabilizes functional connectivity during
9(4):176–186 resting state and brain activity during mem-
276. Venneri A, McGeown WJ, Shanks MF (2009) ory encoding in Alzheimer’s disease. J Clin
Responders to ChEI treatment of Alzheimer’s Psychopharmacol 33(2):199–205
disease show restitution of normal regional 289. Wang L et al (2014) The effect of APOE
cortical activation. Curr Alzheimer Res epsilon4 allele on cholinesterase inhibitors
6(2):97–111 in patients with Alzheimer disease: evalua-
277. Dhanjal NS et al (2013) Auditory corti- tion of the feasibility of resting state func-
cal function during verbal episodic memory tional connectivity magnetic resonance
encoding in Alzheimer’s disease. Ann Neurol imaging. Alzheimer Dis Assoc Disord
73(2):294–302 28(2):122–127
278. Dhanjal NS, Wise RJ (2014) Frontoparietal 290. Wang Z et al (2014) Acupuncture modulates
cognitive control of verbal memory resting state hippocampal functional con-
recall in Alzheimer’s disease. Ann Neurol nectivity in Alzheimer disease. PLoS One
76(2):241–251 9(3):e91160
279. Goekoop R et al (2004) Challenging the cho- 291. Jia B et al (2015) The effects of acupuncture
linergic system in mild cognitive impairment: at real or sham acupoints on the intrinsic
a pharmacological fMRI study. Neuroimage brain activity in mild cognitive impairment
23(4):1450–1459 patients. Evid Based Complement Alternat
280. Gron G et al (2006) Inhibition of hippo- Med 2015:529675
campal function in mild cognitive impair- 292. Wells RE et al (2013) Meditation’s impact
ment: targeting the cholinergic hypothesis. on default mode network and hippocampus
Neurobiol Aging 27(1):78–87 in mild cognitive impairment: a pilot study.
281. Risacher SL et al (2013) Cholinergic enhance- Neurosci Lett 556:15–19
ment of brain activation in mild cognitive 293. van Paasschen J et al (2013) Cognitive reha-
impairment during episodic memory encod- bilitation changes memory-related brain
ing. Front Psychiatry 4:105 activity in people with Alzheimer disease.
282. Petrella JR et al (2009) Effects of donepezil Neurorehabil Neural Repair 27(5):448–459
on cortical activation in mild cognitive impair- 294. Hampstead BM et al (2012) Mnemonic strat-
ment: a pilot double-blind placebo-controlled egy training partially restores hippocampal
trial using functional MR imaging. AJNR Am activity in patients with mild cognitive impair-
J Neuroradiol 30(2):411–416 ment. Hippocampus 22(8):1652–1658
283. Pa J et al (2013) Cholinergic enhancement 295. Buhmann C et al (2003) Pharmacologically
of functional networks in older adults with modulated fMRI—cortical responsiveness
mild cognitive impairment. Ann Neurol to levodopa in drug-naive hemiparkinsonian
73(6):762–773 patients. Brain 126(Pt 2):451–461
284. Goveas JS et al (2011) Recovery of hippo- 296. Kwak Y et al (2010) Altered resting state
campal network connectivity correlates with cortico-striatal connectivity in mild to mod-
cognitive improvement in mild Alzheimer’s erate stage Parkinson’s disease. Front Syst
disease patients treated with donepezil Neurosci 4:143
assessed by resting-state fMRI. J Magn Reson 297. Wu T et al (2009) Regional homogeneity
Imaging 34(4):764–773 changes in patients with Parkinson’s disease.
285. Li W et al (2012) Changes in regional cere- Hum Brain Mapp 30(5):1502–1510
bral blood flow and functional connectivity 298. Agosta F et al (2014) Cortico-striatal-
in the cholinergic pathway associated with thalamic network functional connectivity
cognitive performance in subjects with mild in hemiparkinsonism. Neurobiol Aging
Alzheimer’s disease after 12-week donepezil 35(11):2592–2602
treatment. Neuroimage 60(2):1083–1091 299. Choe IH et al (2013) Decreased and increased
286. Zaidel L et al (2012) Donepezil effects cerebral regional homogeneity in early
on hippocampal and prefrontal functional Parkinson’s disease. Brain Res 1527:230–237
connectivity in Alzheimer’s disease: pre- 300. Esposito F et al (2013) Rhythm-specific
liminary report. J Alzheimers Dis 31(Suppl modulation of the sensorimotor network in
3):S221–S226 drug-naive patients with Parkinson’s disease
287. Lorenzi M et al (2012) Effect of meman- by levodopa. Brain 136(Pt 3):710–725
tine on resting state default mode network 301. Mattay VS et al (2002) Dopaminergic modu-
activity in Alzheimer’s disease. Drugs Aging lation of cortical function in patients with
28(3):205–217
 fMRI in Neurodegenerative Diseases: From Scientific Insights to Clinical Applications 739

Parkinson’s disease. Ann Neurol 51(2): movements in Parkinson’s disease. PLoS One
156–164 7(12):e50270
302. Tessitore A et al (2002) Dopamine modulates 306. Kahan J et al (2014) Resting state functional
the response of the human amygdala: a study MRI in Parkinson’s disease: the impact of
in Parkinson’s disease. J Neurosci deep brain stimulation on ‘effective’ connec-
22(20):9099–9103 tivity. Brain 137(Pt 4):1130–1144
303. Arantes PR et al (2006) Performing functional 307. Scheff SW et al (2006) Hippocampal synaptic
magnetic resonance imaging in patients with loss in early Alzheimer’s disease and mild cog-
Parkinson’s disease treated with deep brain nitive impairment. Neurobiol Aging
stimulation. Mov Disord 21(8):1154–1162 27(10):1372–1384
304. Phillips MD et al (2006) Parkinson disease: 308. Jack CR Jr et al (2013) Tracking pathophysi-
pattern of functional MR imaging activation ological processes in Alzheimer’s disease: an
during deep brain stimulation of subthalamic updated hypothetical model of dynamic bio-
nucleus—initial experience. Radiology 239(1): markers. Lancet Neurol 12(2):207–216
209–216 309. Bobholz JA et al (2007) Clinical use of func-
305. Kahan J et al (2012) Therapeutic subthalamic tional magnetic resonance imaging: reflec-
nucleus deep brain stimulation reverses tions on the new CPT codes. Neuropsychol
cortico-thalamic coupling during voluntary Rev 17(2):189–191
 Chapter 24

fMRI in Epilepsy
Rachel C. Thornton, Louis André van Graan, Robert H. Powell,
and Louis Lemieux

Abstract
This chapter provides an overview of the application of functional MRI applied to the field of Epilepsy and
is divided into two sections, covering cognitive mapping and imaging of paroxysmal activity, respectively.
In addition to a review of the most scientifically and clinically relevant findings, technical and methodologi-
cal background information is provided to help the reader better understand the data acquisition process.
We show how both approaches may play a role in the presurgical evaluation of patients with drug-resistant
focal epilepsy and provide opportunities for new insights into the neuropathological processes that under-
lie both focal and generalized epilepsy.

Key words Epilepsy, Focal epilepsy, Generalized epilepsy, Interictal, Ictal, Imaging, Functional mag-
netic resonance imaging, fMRI, Electroencephalography, EEG, Multi-modal imaging, EEG-
correlated fMRI, Cognitive mapping, Functional mapping, Brain activity mapping, Language
lateralization, Memory mapping, Presurgical evaluation

1 Cognitive fMRI in Epilepsy

The commonest surgical procedure for patients with drug resistant

temporal lobe epilepsy (TLE) is anterior temporal lobe resection
(ATLR). Complications of this operation include a decline in lan-
guage and memory abilities, and an important part of the presurgi-
cal assessment lies in the careful selection of patients to minimize
these adverse cognitive sequelae. This has traditionally been the
role of neuropsychology and the intracarotid amytal test (IAT).
Since the advent of functional MRI (fMRI), however, there has
been much interest in its possible role in the presurgical assessment
of those with epilepsy, principally in the identification of eloquent
cortex to be spared during surgery.
There is a substantial body of literature reporting on cognitive
function in epilepsy—employing a spectrum of indices and param-
eters, including task performance, lesion type and location, func-
tional and effective connectivity measures and activation study

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_24, © Springer Science+Business Media New York 2016

741
742 Rachel C. Thornton et al.

outcomes that investigate and report cognitive impairment and

idiosyncrasies in epilepsy, including working memory [1–3], long-
term memory [4] and language organization [5, 6].
Neuropsychology has played a prominent role throughout the
modern era of epilepsy surgery, mainly because of the importance of
the temporal lobes in memory function. The principal role of base-
line neuropsychological assessments is in establishing a baseline
quantification of relevant cognitive function whilst providing an
indication of the potential impact of surgery. It does so in the con-
text of a conventional understanding of lateralization and localiza-
tion of cerebral disturbance, predicting the impact of surgery on
memory, providing data on lateralization and localization of cerebral
disturbance, and providing evidence for cerebral reorganization.
The IAT plays a role in the presurgical assessment of TLE in
some centers. Its uses are in assessing the capacity of the contralat-
eral temporal lobe to maintain useful memory functions thus
guarding against a severe postoperative amnesic syndrome, and
provides a means of lateralizing language function. The procedure
involves the injection of sodium amytal into one carotid artery,
inactivating the corresponding hemisphere for around 10 min, and
thus crudely mimicking the effects of surgery on the medial tem-
poral lobe (MTL) structures. During this time, the patient’s lan-
guage and memory abilities are tested. Although still commonly
used, the IAT has considerable disadvantages, not least the fact
that it is an expensive, invasive procedure with potentially serious
complications. Doubts also exist about its reliability and validity in
predicting postoperative amnesia. In contrast to the traditional
neuropsychological assessment, which relies on standardized tests
of cognitive abilities and yields results that are easily validated, IAT
procedures vary significantly between institutions with respect to
the testing protocol used, choice of behavioral stimuli, dosage, and
administration of amytal, all of which can lead to variations in the
results [7]. The IAT is also poor at predicting verbal memory
decline as deactivation of the language dominant hemisphere may
impede verbal process that are constituent to verbal memory,
thereby causing increased errors on verbal memory testing [8].
fMRI has the potential for replacing the IAT and for providing
additional information to that provided by baseline neuropsycho-
logical assessment in the lateralization and localization of language
and memory function. Practically, fMRI is cheaper than the IAT,
noninvasive, and repeatable. There are, however, important poten-
tial caveats when considering the role of fMRI. First, areas acti-
vated by a particular fMRI paradigm are not necessarily crucial for
the performance of that task. Second, it does not necessarily follow
that all areas involved in a task will be activated by a particular
fMRI paradigm. Third, the extent of activation seen in a task, in
terms of both the area activated and the magnitude of the peak,
 fMRI in Epilepsy 743

may bear no relation to the competence with which that task is

performed. The development of insight and greater appreciation
of engagement of multiple haemodynamic networks in cognitive
tasks together with technical improvement in acquisition and pro-
cessing is likely to elaborate the value of fMRI paradigms in map-
ping cognitive functions.
Whilst being neither very sensitive nor specific with regards the
neurobiological substrates of these cognitive functions—psychomet-
ric data provides some measurement and quantification. When
understood that the psychometric testing itself provides a behavioral
sample of cognitive processes rather than a true ecological measure
it compounds the interpretation of the correlation between fMRI
data and the test scores, as sensitive or specific measure of changes in
brain function. Caution will also be needed in the interpretation of
results bearing in mind that fMRI techniques, while useful for the
localization of cognitive function, may not reliably indicate the
capacity of unilateral temporal lobe structures.
In the following section, we review how fMRI is used to lateral-
ize language function. We then discuss the current state of research
efforts to localize brain regions involved in language and memory,
and study the effects of epilepsy upon these. Furthermore, the efforts
to assess the reliability of fMRI in the prediction of postoperative
language and memory deficits following ATLR are discussed.

1.1 Language fMRI The aims of preoperative language fMRI are primarily to lateralize
and localize language functions and to use this information to pre-
1.1.1 Paradigm Design
dict and avert postoperative complications. A number of task para-
and Analysis
digms to engage anterior/expressive as well as posterior/
comprehension language areas [9–14] have been used to identify
language representation. Complementary indices provided by ver-
bal fluency, verb generation and semantic decision tasks, are vari-
ously and commonly employed in the clinical context [15]. The
most widely used tasks in language fMRI experiments are verbal
fluency tasks. These are generally strongly lateralizing and reliably
identify “expressive language functions” in the dominant inferior
frontal gyrus (IFG) (Brodmann Areas [BA], 44, 45). Specifically,
verbal fluency tasks show more prominent activity in left frontal
regions, corresponding to Broca’s area, than in the medial tempo-
ral lobe in healthy controls and TLE patients [16, 17]. Although
these tasks are usually covert (i.e. performed silently without per-
formance monitoring), they have been reliably replicated in numer-
ous studies in both normal and patient populations. Their
within-subject reproducibility has been demonstrated, with frontal
activations shown to be more reliable than temporoparietal ones
[18]. In addition they can be applied to patients with a wide range
of cognitive abilities, with language lateralization results appearing
to be relatively unaffected by patients’ performance levels [19].
744 Rachel C. Thornton et al.

Although tasks of verbal fluency do show language-related activa-

tions, they are not pure language tasks, containing substantial
components of executive processing and of working and verbal
memory. These activations are typically seen in the middle frontal
gyrus (MFG) (BA 46, 49) (Fig. 1).
Paradigms that specifically recruit the areas that are removed
during ATLR should be established [17, 20]: Activation paradigms
for naming functions that could provide greater specificity in the
context of ATLR [21] include object naming paradigms involving
visual [22, 23] and auditory stimuli [24, 25]. The multidimensional
structure of language representation is illustrated by discrete contri-
bution of different task paradigms. For example, verbal fluency
identifies areas that are not activated with verb generation. In turn
verb generation causes more discrete activation than verbal fluency
[15]. Fluency tasks are also less reliable in identifying “receptive”
language areas located in the dominant temporal lobe. These pro-
cessing areas are best assessed by tasks that probe language compre-
hension such as reading sentences or stories, which tend to activate
superior temporal cortex extending to supramarginal gyrus (BA 20,
21, 39) [26, 27], but are less strongly lateralizing than verbal flu-
ency tasks (Fig. 1). Using a panel of fMRI tasks (verbal fluency,
reading comprehension, and auditory comprehension) was shown
to be helpful in reducing inter-rater variability and helped in the
evaluation of language laterality in patients with focal epilepsy [28].

Fig. 1 Verbal fluency and reading comprehension: Typical fMRI findings in a patient performing tasks of verbal
fluency (left) and reading comprehension (right) showing activation in the dominant frontal lobe and bilateral
superior temporal lobes, respectively. Areas of activation are overlaid on a distortion matched high resolution
echo-planar image
 fMRI in Epilepsy 745

These functional imaging experiments have used block design

paradigms to detect regions of the brain showing greater activation
during task blocks when compared with rest blocks. The advantage
of block designs over event-related designs is that they are efficient
in detecting differences between two conditions; however, they
offer less flexibility in the experimental design required for study-
ing complex cognitive functions.
The degree of lateralization is often quantified using a lateral-
ization (LI) or asymmetry index, (AI) = (L − R)/(L + R), where L
and R represent the strength of activation for the left (L) and right
(R) sides, respectively, based on the number of activated voxels for
the whole hemisphere or using regions of interest (ROIs) targeted
to known as language areas [29]. A positive value represents left
language lateralization and a negative, right-sided dominance,
although AI values between −0.2 and +0.2 are often classified as
bilateral. This can be determined by counting the number of voxels
exceeding a specified threshold of significance. This type of AI cal-
culation has some problems, in particular, the fact that the AI can
differ according to the significance threshold chosen for the activa-
tion map from which it is calculated [30]. One suggested solution
to this is to calculate AIs from all voxels that correlated positively
with a task, but with each weighted by their own statistical signifi-
cance [31]. Using this method, AIs were less variable than those
calculated from suprathreshold voxels only. Alternative approaches
to estimating the degree of asymmetry include measuring the mean
signal intensity change induced by the task within a brain volume
of interest [30], and performing a statistical comparison of the
magnitude of task-induced activation in homotopic regions of the
two hemispheres [32]. These methods measure the magnitude of
the mean signal change and have the advantage of not being
threshold dependent. Language networks in focal epilepsy have, in
fact, been localized using data driven approaches: These algorithms
yielded significant observations not seen with conventional fMRI,
specifically in relation to the effects of epilepsy on language repre-
sentation [33–35]. Other studies have suggested that visual rating
appears to work as well as calculating AIs [36] by using pattern
classifying algorithms that have demonstrated concordance with
existing LI and visual rating classification methods [37, 38].
The simplest forms of study designs (including those described
earlier) employ cognitive subtraction designs. These involve selecting
a task that activates the cognitive process of interest and a baseline
task that controls for all but the process of interest. One problem of
this type of design is that it depends on an assumption known as pure
insertion, which supposes that a new cognitive component can be
inserted without affecting those processes that are also engaged by
the baseline task [39]. Another problem with cognitive subtraction is
in finding baseline tasks that activate all but the process of interest.
These problems can be overcome by using more complex experi-
mental designs, such as factorial designs and cognitive conjunctions.
746 Rachel C. Thornton et al.

Factorial designs use two or more variables (e.g. sentence vs.

word presentation and auditory vs. visual presentation) and allow
the effect that one variable has on the other to be measured explic-
itly. The analysis of this type of design involves calculating the main
effects of each variable and the interaction between them [39].
Cognitive conjunctions are an extension of cognitive subtraction
paradigms. Cognitive subtraction looks for activation differences
between a single pair of tasks, while cognitive conjunction looks at
two or more task pairs, which share a common processing differ-
ence [40]. The advantages of this approach are that it allows greater
freedom in selecting the baseline task as it is not necessary to con-
trol for all but the component of interest, and that it does not
depend on the assumption of pure insertion.

1.1.2 Language Focal epilepsy may be associated with disrupted lateralization and
Lateralization in Epilepsy localization of language regions; therefore, one would expect a
higher probability of abnormal language lateralization. Nevertheless,
significant differences have been reported between centers in the
relative proportions of right and left hemisphere dominant patients
using the IAT, some of which may be due to the different criteria
used for assessing dominance. The percentage of left hemisphere
dominant right-handed patients has ranged from 63 to 96 % [41]
while for left-handers a similar variation has been reported between
38 and 70 % [42, 43]. Results of fMRI studies have also shown
greater atypical language dominance in patients. In a comparison
between 100 right-handed healthy subjects and 50 right-handed
epilepsy patients, 94 % of the normal subjects were considered as left
hemisphere dominant and 6 % had bilateral representation. The epi-
lepsy group showed greater variability of language dominance, with
78 % showing left hemisphere dominance, 16 % symmetric activa-
tion, and 6 % showing right hemisphere dominance. Atypical lan-
guage dominance was associated with an earlier age of brain injury
and with weaker right hand dominance [44].
The localization of the epileptogenic lesion and epileptic
activity [45] has also been shown to influence language organiza-
tion. In a retrospective study of patients with hippocampal sclero-
sis (HS) who had undergone presurgical evaluation, atypical
speech dominance occurred in 24 % of those with left-sided HS,
whereas all those with right-sided HS had left-sided speech domi-
nance. In addition, atypical speech representation was associated
with higher spiking frequency and in those with sensory auras
suggesting ictal involvement of the lateral temporal structures.
No association was demonstrated between either age at epilepsy
onset or age at initial precipitating injury and atypical speech rep-
resentation [46].
Comparing the degree of reorganization of frontal and tempo-
ral lobe language functions has shown a significantly more left
lateralized pattern of language activation in controls and right TLE
 fMRI in Epilepsy 747

patients than left TLE patients [47]. In patients with atypical

language representation, the degree of reorganization towards the
right hemisphere was greater in the temporal lobes than in the
frontal lobes. In a study of 50 patients with focal epilepsy, greater
atypical language dominance was seen in those with left hemi-
sphere seizure focus [48]. Left TLE patients who did not have
atypical language also had lower asymmetry indices in both frontal
and temporal ROIs, mainly because of greater activation in homol-
ogous right hemisphere regions. Atypical language representation
in Wernicke’s area is more frequently observed in TLE patients,
whereas FLE, conversely, appears to have a greater effect on the
organization of anterior language areas [49].
The degree of language lateralization has also been related to
the nature of the epileptogenic lesion with early acquired lesions,
such as HS, considered to be associated with greater incidence of
atypical language lateralization compared with developmental
lesions originating in utero, such as malformations of cortical devel-
opment (MCDs). Atypical language organization has been observed
in a number of other clinical presentations: including stroke, medial
temporal sclerosis, focal cortical lesions as well as patients with a
normal structural MRI [50]. A higher degree of atypical language
dominance, in both frontal and temporal language areas, has been
demonstrated in patients with left HS compared with patients with
left frontal and lateral temporal lesions [51], suggesting that the
hippocampus itself may play an important role in the establishment
of language dominance. Another study, however, demonstrated no
difference in the frequency of atypical language lateralization
between left TLE patients with HS and those with developmental
tumors [52]. Interestingly, HS has been associated with altered
functional organization of cortical networks involved in lexical and
semantic processing [53] in TLE patients.
It is interesting to speculate on how TLE affects language
lateralization, and it is possible that strong connectivity between
inferior frontal and temporal areas make frontal lobe functions
particularly sensitive to temporal pathology. Language lateraliza-
tion is not associated with type and location of lesion (acquired
or developmental), symptoms and gender [54] or age of seizure
onset [49]. However, an association between language lateraliza-
tion and handedness, location and nature of pathologic substrate
and duration of epilepsy has been identified [55]. Verbal memory
scores on psychometric tests have been associated with lateraliza-
tion of language implicating connectivity between inferior frontal
cortex and hippocampus [56, 57]. The increased incidence of
atypical language dominance in epilepsy illustrates the impor-
tance of establishing language dominance prior to performing
surgical resection and as a consequence much of the work on
fMRI in epilepsy has been directed towards trying to replace the
IAT as a means of doing this.
748 Rachel C. Thornton et al.

1.1.3 Comparison Studies comparing fMRI and the IAT are summarized in Table 1.
of fMRI, IAT, Just as IAT protocols differ between centers, a number of fMRI
and Electrocortical paradigms to determine language dominance have been employed
Stimulation Findings but agreement of approximately 80–90 % is seen between the two
techniques [11, 58]. The remaining cases generally exhibit partial
disparity where one method shows bilateral language representa-
tion and the other lateralized language dominance and outright
disagreement between fMRI and IAT is rare. In an interesting
study that assessed the relative accuracy of Wada and fMRI in dis-
cordant cases fMRI provided a more accurate prediction of naming
at postsurgical outcome in seven patients, Wada was more accurate
in two patients. The two methods provided comparable accuracy
in one patient [58].
One study suggested that fMRI may be less reliable in left-sided
neocortical epilepsy (25 % disparity) in comparison with left-sided
medial TLE (3 % disparity) [36]. Another showed that concordance
between fMRI-based laterality and IAT was much lower in left TLE
patients than in patients with right TLE [59]. One interesting case
of false lateralization of language function in a post-ictal patient
with left HS also illustrates the need for caution in the interpreta-
tion of results in individual patients. No activation was seen in the
left temporal lobe during multiple language tasks after a cluster of
left temporal lobe seizures but in a repeat fMRI experiment 2 weeks
later, activation was seen predominantly over the left temporal
region [60]. However, bearing in mind the previously mentioned
limitations of the IAT, it is even debatable whether fMRI and IAT
are directly comparable as they probe different aspects of language.
fMRI language localization can replace Wada test in the majority of
patients. However, the Wada test is still a valuable adjunct and can
be employed when a patient cannot undergo fMRI. It can also be
used for validation of fMRI results or for the assessment of selective
language areas near structural abnormalities [61].
Comparisons have also been performed between fMRI activa-
tion maps and regions showing disruption of function during
intraoperative electrocortical stimulation (ECS). In order for fMRI
to be used instead of ECS, it must demonstrate a high predictive
power for the presence as well as the absence of critical language
function in regions of the brain. As with IAT, these studies show
strong, but incomplete agreement with fMRI, with high sensitivity
but lower specificity [69–71]. Although false-positive activation
(fMRI activation but no ECS disruption) is relatively common,
this is not surprising given that fMRI activates whole networks of
regions, not all of which are essential for the task in question. False-
negative findings (regions showing disruption by ECS but no
fMRI activation) are more critical when planning a surgical resec-
tion, and these were identified in 2 patients out of 21 reported in
two series. Activation and disruption was typically within 5 mm in
frontal regions and 10 mm in temporal areas.
 fMRI in Epilepsy 749

Table 1
Concordance between fMRI language lateralization and the IAT

Sample
Authors sizea fMRI language lateralization tasks Concordance
Desmond et al. 7 Semantic decision task 100 %
(1995) [62]
Binder et al. 22 Semantic decision task r = 0.96
(1996) [63]
Hertz-Pannier 6b Verbal fluency paradigm 100 %
et al. (1997)
[64]
Yetkin et al. 13 Word generation task r = 0.93
(1998) [65]
Benson et al. 12 Verb generation task 100 %
(1999) [66]
Lehericy et al. 10 Semantic fluency Semantic fluency > story
(2000) [27] listening > sentence
Sentence repetition Story listening
repetition
Greater concordance between
IAT results and activation
asymmetry in frontal than
temporal lobes
Carpentier et al. 10 Identification of syntactic/semantic errors 80 %
(2001) [67] in target sentences
Gaillard et al. 21 Reading paradigm 85 %
(2002) [29]
Woermann 100 Word generation 91 %
et al. (2003)
[36]
Sabbah et al. 20c Word generation 95 %
(2003) [68]
Semantic decision
Benke et al. 68 Semantic decision 89 %—right TLE
(2006) [59] 40 Reading sentence comprehension/
72 %—left TLE
Arora et al. 229 auditory sentence comprehension and a
91.3 %
(2009) [11] verbal fluency task.
86 %
Janecek et al. Semantic decision/tone decision
(2013) [58]
IAT intracarotid amytal test, TLE temporal lobe epilepsy
a
Some of these studies report fMRI data on larger samples. However, only the patients with fMRI and IAT data are
included here
b
Age range 8–18
c
Patients with suspected atypical language lateralization were selected
750 Rachel C. Thornton et al.

A combination of four different language tasks has shown

more reliable and robust lateralization in normal subjects by tar-
geting brain regions common to different tasks, thereby focusing
on areas critical to language function. Regions of activation
detected in this way corresponded well with ECS findings in the
temporoparietal region [72]. Sensitivity was 100 % in all but one
patient. This high negative predictive value suggested that areas
where no significant fMRI activity was present could be safely
resected without using ECS. fMRI activity, however, was not
always absent at noncritical language areas limiting its positive pre-
dictive value for the presence of critical language. Although this
suggests that fMRI is not yet ready to replace ECS, it could be
used to speed up intracranial mapping procedures and to guide the
extent of the craniotomy.

1.1.4 Language Language deficits have been reported following language-dominant

Localization and Prediction ATLR, with naming the most commonly affected function [73, 74].
of Postoperative Language It has also been suggested that the risk for post-operative decline in
Deficits naming abilities increases with age of seizure onset and the extent of
lateral temporal neocortex resected [75]. Preoperative cortical stimu-
lation via subdural grid electrodes has been used to localize language
function, suggesting that early onset of dominant temporal lobe sei-
zure foci leads to a more widespread or atypical distribution of lan-
guage areas, particularly naming and reading areas [76]. A subsequent
study also reported that markers of early left hemisphere damage
(such as early seizure onset, poor verbal IQ, left handedness, and right
hemisphere memory dominance) increase the chances of essential lan-
guage areas being located in more anterior temporal regions. Again
these areas were identified using naming and reading tasks [77].
These findings suggest that naming and reading abilities are
the language skills most at risk following dominant temporal lobe
surgery. In patients with right TLE, preserved naming function is
associated with activation of the left hippocampus by the verbal
fluency task. Patients with left TLE, who show preservation of
naming, conversely appear to involve the left frontal lobe, in an
apparent compensatory response to epileptic activity in the left
hippocampus [13]. Although the IAT may provide a useful index
of language laterality, it does not provide detailed information on
the localization of these specific language skills. As these may also
vary in location between individuals, the role IAT can play in the
prediction of postoperative deficits in individual patients is there-
fore limited. Designing fMRI paradigms that specifically probe
naming and reading skills would provide a useful clinical tool for
mapping relevant language skills that could be used in the predic-
tion of postoperative deficits. Specifically, auditory and visual nam-
ing paradigms may yield greater predictive specificity with regard
to naming difficulties after ATLR [21].
 fMRI in Epilepsy 751

One study has used preoperative functional neuroimaging to

predict language deficits following left ATLR: Temporal lobe fMRI
asymmetry was found to be predictive of deficits seen on a postop-
erative naming test with a greater degree of language lateralization
toward the left hemisphere related to poorer naming outcome and
language lateralization towards the right hemisphere associated
with less or no decline. The correlation between temporal lobe
fMRI AI and naming deficits was stronger than that seen in the
frontal lobes and also stronger than that between IAT and naming
deficits [78]. In this regard, fMRI activation of the left middle
frontal gyrus (MFG) with a verbal fluency task was seen to predict
significant postsurgical naming decline in patients with left TLE,
showing good sensitivity but rather poor specificity [17].
Interestingly, many patients do not suffer any language deficits
following ATLR, suggesting that multiple sets of neural systems
may exist that are capable of performing the same cognitive func-
tion, and that some of these may be engaged following focal brain
injuries. In a study of patients who had undergone left ATLR but
did not have deficits in sentence comprehension, decreased activa-
tion was demonstrated in undamaged areas of the normal left
hemisphere system but increased activation was seen in several
right frontal and temporal regions not usually engaged by normal
subjects [79]. This suggests that there is more than one neural
system capable of sustaining sentence comprehension. This study
was, however, unable to tell whether this functional reorganization
to the right IFG occurred pre- or postoperatively. A separate study
looked at the role of the right IFG by comparing its functional
activation on a verbal fluency task in controls with left TLE patients
[80]. The patients were shown to activate a more posterior right
IFG region compared with controls, although left IFG activation
did not differ significantly between the two groups. Further, verbal
fluency-related activation in the right IFG was not anatomically
homologous to left IFG activation in either patients or controls.
This suggests that reorganization takes place preoperatively in
patients with chronic left TLE, and that the prediction of language
outcome following left ATLR may depend not only on the extent
of preoperative right hemisphere activation, but also its location.

Complex behaviors such as language and memory rely upon net-

1.1.5 Combination of works of neurons, which integrate the functions of spatially remote
MRI and MR-Tractography brain regions. The combination of fMRI to identify cortical regions
involved in language function and MR-tractography to visualize
white matter pathways connecting these regions offers an opportu-
nity to study the relationship between structure and function in the
language system (Fig. 2). Studies have revealed structural asym-
metries in controls, with greater left-sided frontotemporal connec-
tions in the dominant hemisphere [81]. Patients with left TLE had
reduced left-sided and greater right-sided connections than both
752 Rachel C. Thornton et al.

Fig. 2 Combined MR tractography and functional mapping. Frontal lobe connections overlaid on a structural
template along with group fMRI effects for word generation (solid arrows) and reading comprehension (dashed
arrows), showing how the tracts connect together the frontal and temporal lobe functionally active regions

controls and right TLE patients, reflecting the altered functional

lateralization seen in left TLE patients, and significant correlations
were demonstrated between structure and function in controls and
patients, with subjects with more highly lateralized language func-
tion having a more lateralized pattern of connections [82]. The
combination of fMRI with information on the structural connec-
tions of these normally and abnormally functioning areas offers the
opportunity to improve understanding of the relationship between
brain structure and function and may improve the planning of sur-
gical resections to maximize the chance of seizure remission and to
minimize the risks of cognitive impairment.

1.2 Memory fMRI A range of memory functions are commonly affected in epilepsy
including modality specific processes in working and long term
memory. fMRI can reliably localize and assess the impact of surgery
on memory networks [83, 84]. In addition LIs have been estab-
lished to evaluate the effects of epilepsy and surgical intervention on
memory [85]. MTL structures are associated with memory func-
tioning, and surgical resection is known to cause reduced memory
function in some cases. The study of patients following temporal
lobe surgery has provided considerable evidence supporting the
 fMRI in Epilepsy 753

critical role that the hippocampi play in memory functioning.

Bilateral injury to these areas leads to a characteristic amnesic syn-
drome [86], while unilateral lesions lead to material-specific defi-
cits, and a decline in verbal memory following surgery to the
language-dominant hemisphere has been consistently reported and
studied [87], along with deficits in topographical memory follow-
ing nondominant ATLR [88]. Although rare, some patients have a
severe anterograde amnesic syndrome following a unilateral
ATLR. Most of these, however, have subsequently been found to
have evidence of contralateral hippocampal pathology, either on
postoperative electroencephalography (EEG) [89], post-mortem
pathological findings [90], or post-operative volumetric MRI [91].
Immediate recall in the verbal and visual modalities demon-
strate significantly less activation in patients with HS as compared
to healthy subjects [92]. A recent study shows that patients employ
the contralateral hippocampus and the ipsilateral parahippocampal
gyrus, reflecting mechanisms of functional adaptation [93]. Two
different models of hippocampal function have previously been
proposed to explain memory deficits following unilateral ATLR:
hippocampal reserve and functional adequacy [94]. According to
the hippocampal reserve theory, postoperative memory decline
depends on the capacity or reserve of the contralateral hippocam-
pus to support memory following surgery, while the functional
adequacy model suggests that it is the capacity of the hippocampus
that is to be resected that determines whether changes in memory
function will be observed. Evidence from baseline neuropsychol-
ogy [95], the IAT [96], histological studies of hippocampal cell
density [97], and MRI volumetry [98] has suggested that of the
two, it is the functional adequacy of the ipsilateral MTL, rather
than the functional reserve of the contralateral MTL that is most
closely related to the typical material-specific memory deficits seen
following ATLR. However, compensatory reorganization in the
context of HS have been shown to be elaborate involving temporal
and extra temporal structures [92, 93, 99, 100].
The assessment of ability to sustain memory is critical for plan-
ning ATLR as memory decline is not an inevitable consequence of
temporal lobe surgery. Accurate prediction of likelihood and sever-
ity of postoperative memory decline is necessary to make an
informed decision regarding surgical treatment. Much work has
been focused on the identification of prognostic indicators for risk
of memory loss after ATLR. Language lateralization, verbal as well
as visual memory fMRI activation patterns, age at onset of epilepsy
and memory performance all serve as predictors of verbal memory
decline in left ATLR. However, these factors appear to be less sen-
sitive in prediction of postsurgical visual memory impairment in
right ATLR [101]. Recent results [102] confirm age at onset of
epilepsy, shorter duration of epilepsy and lower seizure frequency
as critical factors that influenced verbal memory encoding in
754 Rachel C. Thornton et al.

patients with TLE. Conversely this study showed that longer dura-
tion and higher seizure frequency were associated with greater
inefficient, extra-temporal reorganization. The severity of HS on
MRI is an important determinant, being inversely correlated with
a decline in verbal memory following left ATLR, with less severe
HS increasing the risk of memory decline [98, 103, 104].
Specifically, the extent of verbal memory decline after left ATLR is
correlated with greater BOLD activation of the diseased left hip-
pocampus and its connectivity to ipsilateral posterior cingulate
[105]. Preoperative memory performance has been related to
degree of postoperative memory impairment, with better perfor-
mance increasing the risk of memory decline [95, 106, 107]. These
risk factors reflect the functional integrity of the resected temporal
lobe and suggest that patients with residual memory function in
the pathological hippocampus are at greater risk of memory impair-
ment postoperatively. Recently, fMRI has also been shown to be a
potential predictor of postoperative material-specific memory
decline following ATLR. Comparison of pre- and postoperative
fMRI activation and correlation with better verbal memory out-
come after left ATLR indicate preoperative reorganization of ver-
bal memory function to the ipsilateral posterior medial temporal
lobe [85]. Other results indicate that visual and verbal memory
function following ATLR is correlated to activity of the contralat-
eral medial temporal lobe and its connectivity to the posterior cin-
gulate cortex ipsilateral to the damaged hippocampus [105, 108].

Impairment in memory encoding following ATLRs suggests that

1.2.1 The Difficulty anterior MTL regions are critical for successful memory encoding,
in Seeing Anterior and in the patient HM, who was rendered amnesic following bilat-
Hippocampal Activation eral temporal lobe resections, more posterior MTL structures
remained intact [109]. Intracranial electrophysiological recordings
during verbal encoding tasks have also shown greater responses in
anterior hippocampal and parahippocampal regions for words
remembered than those forgotten [110]. However, functional
imaging studies have proved contradictory, with many showing
encoding-related activations in posterior hippocampal and para-
hippocampal regions, which would be left intact following ATLRs.
One possible explanation for this apparent conflict is that ante-
rior temporal regions are subject to signal loss during fMRI
sequences. It has been demonstrated that signal loss due to suscep-
tibility artifact is most prominent in the inferior frontal and infero-
lateral temporal regions [111], and as the hippocampus rises from
anterior to posterior, one would expect greater susceptibility-
induced signal loss in the anterior (inferior) relative to posterior
(superior) hippocampus. This may have been one reason for the
relative lack of anterior hippocampal activation in early fMRI
studies of memory [112]. One study has directly examined the
effects of susceptibility artifact on hippocampal activation by
 fMRI in Epilepsy 755

demonstrating its differential effect on the anterior vs. the poste-

rior hippocampus. The averaged resting voxel intensity in an ante-
rior hippocampal ROI was significantly less than in a posterior
hippocampal ROI and intensity decreases were substantial enough
to leave many voxels below the threshold at which BOLD effects
could be detected [112]. Moreover, it has been shown that the
sensitivity to BOLD changes is proportional to signal intensity at
rest so that voxels with a lower baseline signal (such as those in
anterior hippocampal regions) would be more difficult to activate
than those with higher baseline signals [113].
An alternative explanation for the lack of anterior hippocampal
activation seen in many early memory fMRI experiments is that the
paradigms used were not optimal for detecting subsequent mem-
ory effects. The use of fMRI in studying memory function is more
challenging than for language. This is partly due to the different
components involved in memory processing, such as encoding and
retrieval, and the fact that the nature of the material being encoded
or retrieved influences which brain areas are activated. A further
difficulty is how to separate brain activity related specifically to
memory from that related to other cognitive processes. In conse-
quence, more complex paradigms are required when studying
memory than for examining language function.
Standard fMRI experiments initially used block design para-
digms looking for regions of the brain showing greater activation
during task blocks compared with rest blocks. A problem when
designing memory fMRI experiments was how to separate brain
activity specifically due to memory from that due to other cognitive
processes being used in the task. Early fMRI studies of memory
encoding employed block experimental designs to contrast tasks
promoting differing memory performance, using the “depth of
encoding” principle [114]. This states that if you manipulate mate-
rial in a “deep” way (e.g. make a semantic decision about a word),
then it is more likely to be recalled successfully than material manip-
ulated in a “shallow” way (e.g. make a decision of whether the first
letter of a word is alphabetically before the last letter). These studies
tended to show consistent activation in left prefrontal cortical
regions along with less reliable MTL activation [115–118]. Similar
assumptions underlie the use of “novelty” paradigms in probing
memory encoding. During these experiments, alternating blocks of
novel and repeated stimuli are presented, with the hypothesis being
that more memory encoding takes place while viewing a block of
novel stimuli than when viewing the same repeated stimulus [119].
The advantage of block designs is that they are generally the
most efficient in detecting differences between two conditions.
The main problem in their interpretation, however, lies in the
inference that the effects shown by these contrasts reflect differ-
ences in memory encoding, rather than any other differences
between the two conditions (e.g. response to novelty and semantic
756 Rachel C. Thornton et al.

processing) that are independent from differences in memory

encoding. Attempts were made to overcome this problem using
parametric block designs but were soon superseded by the advent
of event-related studies.
Event-related fMRI is defined as the detection of transient hemo-
dynamic responses to brief stimuli or tasks. This technique, derived
from those used by electrophysiologists to study event-related poten-
tials, enables trial-based rather than block-based experiments to be
carried out. Trial-based designs have a number of methodological
advantages, in particular that trials can be categorized post-hoc
according to a subject’s performance on a subsequent test to obtain
fMRI data at the individual item level. Therefore, when studying
memory encoding, activations for individual items presented can be
contrasted according to whether they are remembered or forgotten in
a subsequent memory test. This type of analysis allows the identifica-
tion of brain regions showing greater activation during the encoding
of items that are subsequently remembered compared with items sub-
sequently forgotten (subsequent memory effects), which are then
taken as candidate neural correlates of memory encoding [120].
Although event-related designs are less powerful than block designs at
detecting differences between two brain states and may be more vul-
nerable to alterations in the hemodynamic response function (e.g. due
to pathology), they have the advantage of permitting specifically the
detection of subsequent memory effects due to successful encoding.
One study looking at encoding of words, pictures, and faces in
healthy controls employed an experimental design, which allowed
data analysis either as a block design, or as an event-related design
of successful encoding [121]. The results demonstrated a functional
dissociation between anterior and posterior hippocampus. The
main effects of memory encoding, demonstrated specifically using
an event-related analysis, were seen in the anterior hippocampus
(Fig. 3), with the main effects of viewing stimuli, demonstrated
using a block analysis, being located in more posterior regions.

1.2.2 The Effect of TLE Deficits in verbal memory following left ATLR and topographical
on Memory Processes memory following right ATLR suggest a material-specific lateral-
ization of function in MTL structures. Functional imaging studies
have been used to look for lateralization of cerebral activation pat-
terns during episodic memory processes. Many have shown
material-specific lateralization in prefrontal regions but this has
been more difficult to demonstrate in the MTL [115, 119, 121,
122]. Working memory can be affected in TLE patients with HS,
with indication of altered connectivity between regions [123].
Specifically, a disruption of the regional balance between task-pos-
itive and task-negative functional networks is associated with work-
ing memory dysfunction in TLE [124]. Reduced right superior
parietal lobe activity is associated with suppression of activity in the
healthy hippocampus in the context of an increasing WM load with
 fMRI in Epilepsy 757

Fig. 3 Left hippocampal activation in a single subject performing a word encod-

ing task

maintained levels of performance [3]. In FLE, patients particularly

recruit the frontal lobe contralateral to the seizure focus in a com-
pensatory pattern that sees patients employ a wider range of net-
works than healthy controls [84].
A number of studies have used fMRI to look at the lateraliza-
tion of memory in patients with TLE compared with that seen in
normal subjects, and also compared the findings with the results of
the IAT. These are summarized in Table 2. These employed block
design studies, demonstrating predominantly posterior MTL acti-
vation, and therefore cannot claim that subsequent memory effects
have been specifically examined.
Studies performed in patients with TLE showing patient
groups studied, experimental design employed, and principal find-
ings HS hippocampal sclerosis, IAT intracarotid amytal test, MTL
medial temporal lobe, TLE temporal lobe epilepsy.
More recently event-related studies have demonstrated a
material-specific lateralization of memory encoding within anterior
MTL regions that would be resected during standard ATLR (Fig. 4)
[121]. In addition, a reorganization of function has been demon-
strated in patients with unilateral TLE due to HS, with reduced ipsi-
lateral activation, and increased contralateral activation in patients
compared with controls [128, 129] (Fig. 5). Comparing groups of
patients with controls demonstrated a functional reorganization away
from the pathological hemisphere; however, it is not clear whether
758 Rachel C. Thornton et al.

Table 2
fMRI memory studies in TLE

Authors Sample size fMRI tasks Findings

Detre et al. Controls n = 8 Block design
Symmetric MTL activation in normal subjects.
(1998) Lateralization of memory concordant with
Patients n = 10 Complex visual scenes
[122] IAT in 9/10 subjects
vs. abstract pictures
Bellgowan Patients n = 28, Block design Greater activation in the left MTL in right TLE
et al. 14 left TLE, Semantic decision compared to left TLE group
(1998) 14 right vs. auditory
[125] TLE perception task
Dupont Controls n = 10 Block design Left occipitotemporoparietal network activated
et al. in controls. Reduced MTL activation and
Patients n = 7, Verbal encoding and
(2000) increased activation in left dorsolateral
left HS retrieval vs. fixation
(79) frontal cortex in patients
on the letter A
Jokeit et al. Controls n = 17 Block design Roland’s No asymmetry of MTL activation in controls,
(2001) Hometown greater activation in the MTL contralateral
Patients n = 30
[126] Walking vs. baseline to seizure focus in 90 % of patients
Golby et al. Patients n = 9 Block design Group level—greater activation in the MTL
(2002) comparing novel contralateral to seizure focus for all encoding
[127] vs. repeated stimuli stimuli
Four encoding stimuli Single subjects—lateralization of memory
used—patterns, concordant with IAT in 8/9 subjects
faces, scenes and
words

0.4

0.3
fMRI activation

0.2
Left
0.1
Right
0

−0.1

−0.2
words pictures faces
Material type

Fig. 4 Material-specific lateralization of memory encoding in the anterior hippo-

campus. fMRI activation within left and right hippocampal ROIs in healthy con-
trols demonstrating left lateralized activation for word encoding, right-lateralized
activation for face encoding and bilateral activation for picture encoding
 fMRI in Epilepsy 759

this represents an effective way of maintaining memory function in

individual patients. By correlating fMRI activation and performance
on standard neuropsychological memory tests, it has been shown
that MTL activation ipsilateral to the pathology is correlated with
better performance while contralateral, compensatory activation cor-
relates with poorer performance [129]. The conclusion that memory
function in unilateral TLE is better when sustained by the activation
within the damaged hippocampus is consistent with the observation
that preoperative memory performance is a predictor of postopera-
tive memory decline, with better performance predicting worse
decline [106, 107], and adds further support to the functional ade-
quacy model of hippocampal function.

C P
0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4
0.5

0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4

−0.5

L R
Fig. 5 fMRI memory encoding experiment: Left TLE patients vs. healthy controls. Regions showing significant
differences in activation between left temporal lobe epilepsy (TLE) patients and controls are highlighted.
Contrast estimates are shown on the right of the images. Controls (C) are on the left and patients (P) on the
right. A reorganization of function is seen in the left TLE patients with reduced activation in the left hippocam-
pus, and greater activation in the right hippocampus, compared with healthy controls
760 Rachel C. Thornton et al.

1.2.3 The Prediction Prediction of postoperative memory decline is necessary to make

of Postoperative Memory an informed decision regarding surgical treatment. To date a small
Changes number of studies have used fMRI to predict the effect of left or
right ATLR on verbal and nonverbal memory. In patients with left
HS, greater verbal memory encoding activity in the left hippocam-
pus compared with the right hippocampus predicted the extent of
verbal memory decline following left ATLR [130]. In a further
analysis of the same patients, it was demonstrated that greater acti-
vation within the left hippocampus predicted a greater postopera-
tive decline in verbal memory [131]. These findings have since
been replicated and extended to patients undergoing right ATLR
[132]. Other groups have demonstrated correlations between
MTL activation asymmetry ratios and postsurgical memory
outcome in patients with both left and right TLE, with increased
activation ipsilateral to the seizure focus correlating with greater
memory decline [133, 134]. A recent study showed that bilateral
posterior hippocampal activation correlated with less verbal mem-
ory decline postoperatively whereas left frontal and anterior medial
temporal activations in left TLE patients correlated significantly
with greater verbal memory decline [102].
As discussed earlier, two different models of hippocampal
function have been proposed to explain memory deficits follow-
ing unilateral ATLR: hippocampal reserve and functional ade-
quacy [94]. Studies using asymmetry indices are unable to
address this important issue; however, the findings of some of
the above studies that greater preoperative activation within the
ipsilateral, to-be-resected hippocampus, correlated with greater
postoperative decline in memory support the functional ade-
quacy theory [131, 132].

1.3 Challenges When designing paradigms for patients with neurological deficits, it
of Clinical Cognitive is important to use tasks that they are able to perform. A differential
fMRI pattern of activation between patients and normal subjects is only
interpretable if patients are performing the task adequately [135].
In addition, one must be aware of differences in the questions
being asked by cognitive neuroscientists and clinicians, which can
lead to different approaches to data analysis. Generally, neuroscien-
tists look at groups of matched controls performing the same task
and determine which brain regions are commonly activated across
the group. The emphasis is on avoiding false-positive results (Type
I errors) and conservative statistical thresholds need to be used,
which may lead to an under representation of brain areas truly
involved. Conversely, clinicians are considering individual patients
where the priority is to identify all brain regions involved in a task,
i.e., avoiding false negatives (Type II errors). As a result, less strin-
gent statistical thresholds are required and indeed thresholds used
may need to vary on an individual basis.
 fMRI in Epilepsy 761

1.4 Cognitive fMRI fMRI is a noninvasive and widely available tool, which has had a
in Epilepsy: Summary dramatic impact on cognitive neuroscience. Much of the progress
made will benefit clinical neuroimaging, although some problems
exist in the application of fMRI to patients with neurological defi-
cits. fMRI allows the noninvasive assessment of language function
to be performed and offers a valid alternative to the IAT for estab-
lishing language dominance. By tailoring paradigms towards the
localization of the specific language skills most at risk following
temporal and frontal resections, it will be possible to map relevant
language functions in the epilepsy surgery population. This in turn
will allow better assessment of the risks posed by surgery in each
individual patient.
Considerable effort is also being made in the development of
memory paradigms that can lateralize MTL functions and provide
meaningful data at the single subject level. This information, in
combination with structural MRI to evaluate hippocampal pathol-
ogy and baseline neuropsychology, will enable preoperative predic-
tion of likely material specific memory impairments seen following
unilateral ATLR to be made with greater accuracy. In consequence,
it will be possible to modify surgical approaches in those patients
most at risk and to improve preoperative patient counseling.
Clinically it is what happens to individual patients that is
important and the next step in the validation of these techniques
will involve similar studies with larger numbers of patients. These
should include more heterogeneous samples, including both left
and right TLE undergoing ATLR. As well as showing group level
correlations either at the voxel-level or within a predefined ROI, it
will be important to establish methods for using this data to pre-
dict language and memory changes in individual cases. Investigating
how the brain sustains memory postoperatively also requires fur-
ther investigation. Longitudinal fMRI studies with pre- and post-
operative imaging, including correlations with neuropsychological
measures of language and memory, will be required to look at
functional reorganization following surgery, and it is anticipated
that these will offer valuable insights into brain plasticity.

2 fMRI of Paroxysmal Activity

Despite major developments in the field of neuroimaging over the

last two decades, the localization of the brain regions involved in
seizure onset is problematic in a significant proportion of patients
with focal epilepsy, thereby precluding surgical treatment.
Furthermore, our understanding of the neurobiological mecha-
nisms underlying epileptogenic networks in focal and generalized
epilepsies is incomplete.
Scalp EEG and magnetoencephalography (MEG) are compara-
ble in their ability to detect and measure synchronized neuronal
762 Rachel C. Thornton et al.

activity taking place mainly over relatively superficial parts of the

brain with exquisite temporal resolution. Although both remain
extremely active areas of investigation, the interpretation of EEG
and MEG data and in particular their utility in localizing brain gen-
erators is severely limited as a consequence of the principle of super-
position and its corollary, the non-unicity of the inverse solution
[136, 137]. This is in contrast to tomographic functional imaging
modalities such as positron emission tomography (PET) or fMRI,
which do not suffer from the problem of non-unicity and sampling
bias is relatively minor. Although much superior to PET, the tempo-
ral resolution of fMRI, which is essentially governed by the local
hemodynamic response, remains inferior to that of EEG by roughly
three orders of magnitude. Nonetheless, fMRI allows hemodynamic
changes linked to brief (∼ms) neuronal events to be detected and
localized with a fair degree of reliability. Although our understand-
ing of the blood oxygen level-dependent (BOLD) fMRI signal is
constantly improving, in part due to combined EEG and fMRI
experimental data, as a general rule it remains an indirect and relative
measure of neuronal activity. Although combined MEG-MRI seems
a distant prospect, combined EEG-fMRI experiments were per-
formed only a few years following the advent of fMRI [138].
Often presented as combining the advantages of its constituent
parts, a concept that motivated the technique’s pioneers, inevitably
combined EEG-fMRI also suffers from some of their individual lim-
itations. Whatever the technique’s pros and cons, it will soon become
clear to the reader that EEG-fMRI is unique in allowing the hemo-
dynamic correlates of brief, unpredictable bursts of neuronal activity
observed on scalp EEG, such as interictal spikes, to be investigated.
Prior to the possibility of EEG-fMRI experiments, studies of
paroxysmal activity using fMRI were limited to ictal events and often
relied on the correlation of the image time-series with observed clin-
ical manifestations but sometimes did not [139–142].
The first studies of paroxysmal brain activity using fMRI were
predominantly in patients with focal epilepsy, clinically motivated
by the possibility of noninvasively localizing seizure focus. This
continues to be an important source of motivation for this rapidly
moving field, but much current research focuses its attention on
the understanding of the networks underlying the generation of
seizures in both focal and generalized epilepsies.
Although an exciting development with potential clinical
value, the technique currently remains within the realm of
advanced, exploratory imaging modalities that require resources
not available in most epilepsy clinics. We therefore begin this
review by discussing some of the technique’s key technological and
methodological aspects. We will then present an overview of the
state of EEG-fMRI applied to the investigation of focal and gener-
alized epilepsies.
 fMRI in Epilepsy 763

The analysis and interpretation of fMRI data acquired from

patients lying in the resting state with simultaneous EEG recording
differs fundamentally from that of paradigm-driven fMRI in at
least two ways: a lack of a prior experimental control and uncer-
tainty about the nature of the relationship between EEG event and
putative hemodynamic effects. This important topic will be the
subject of a discussion.

2.1 EEG-Correlated The recording of EEG inside the MR scanner still presents safety,
fMRI in Epilepsy: image data quality and EEG data quality challenges. Historically, the
Technical Issues issue of EEG data quality has been the determining factor in the
technique’s evolution, from interleaved to simultaneous EEG-
fMRI. This reflects in part the fact that a gradual degradation in
EEG quality mainly linked to cardiac activity can be readily observed
in most subjects as they are moved inside the MR scanner (without
scanning), posing an immediate challenge ahead of any other con-
siderations such as safety (albeit this should also be at the forefront
of the considerations of investigators introducing any new equip-
ment in the scanner room) or the effect of scanning on EEG quality
and the possible impact of the EEG recording equipment on image
quality. In the following, we provide an overview of the state of
EEG-fMRI technology, which remains an active area of research in
particular in the area of EEG quality, although mostly for the pur-
pose of evoked response recordings. The focus will be on the impli-
cations for studies in epilepsy and in particular at field strengths
commonly used in neurological studies (≤3.0 T); the reader inter-
ested in the implementation of combined EEG and fMRI record-
ings at higher field strengths (e.g. 7 T) is directed towards two recent
specialized reports that address data quality and safety [143, 144].
The electro-magnetic processes that take place during MR image
2.1.1 Physical Principles acquisition and that are susceptible to interactions with the EEG
of EEG-MR System system are: strong static magnetic field (∼1.0–3.0 T), switching
Interactions magnetic gradient fields (∼100 T/m/s), and radio frequency
(RF) pulses (∼10 μT and 100 MHz). In addition, although MR
scanners are designed to optimize the magnetic component of the
RF pulses, an electrical component is unavoidable. This may lead
to linear antenna effects with possible safety implications [145].
EEG recording, on the other hand, requires electrodes and leads
to be placed within the imaging field of view and electronic
components, depending on the exact equipment and setup, in
proximity to the scanner coils and antenna(s).
Four main mechanisms are at the origin of EEG-MR instru-
mentation interactions:
1. Magnetic induction: any change in magnetic flux (essentially
the component of the magnetic field that is perpendicular to a
surface) over time through a conducting medium (loop, sur-
face, volume) gives rise to an electromotive force in the mate-
764 Rachel C. Thornton et al.

rial and hence an induced current. This phenomenon is

governed by Faraday’s law of induction. Changes in magnetic
flux, and the associated induced currents, can be caused by
movement (change of position, orientation, or shape) of the
conducting medium in a magnetic field or change in the mag-
netic field to which the conducting medium is exposed;
2. Magnetic susceptibility differences: static interactions due to
the magnetic properties of the components of the EEG record-
ing system;
3. RF radiation emanating from active components of the EEG
recording system;
4. Magnetic force on ferro- or paramagnetic components with
associated risks of foreign instruments or their elements
becoming projectiles in the scanner room; in the following we
will assume that all usual design and manipulation precautions
have been taken to avoid these effects.
Magnetic induction can result in EEG quality degradation in
the form of pulse-related and image acquisition (gradient-switching
and RF)-related artifacts. Magnetic susceptibility differences and RF
radiation linked to the EEG system can give rise to image artifacts.

2.1.2 Safety Health hazards not normally encountered when MR or EEG are
performed separately can arise due to induced currents flowing
through loops or the heating of EEG components in proximity or
contact with the subject. For a specific 1.5 T scanner, and based on
a worst case scenario, this study recommended that one 10 kΩ
current-limiting resistor be inserted serially at each electrode lead
and the possibility of large (EEG lead-electrode-head-electrode-
lead-amplifier circuit) loops being formed reduced to a minimum
by lead twisting. In experiments using a different custom-made
EEG system, no significant heating was observed [146]. An impor-
tant general consideration when placing wires in contact with the
body is the type of RF transmit coil used and length of wire exposed
to the electrical component of the RF field [147]. A number of
MR-compatible EEG system or electrode cap vendors have incor-
porated current-limiting resistors in their product design. To the
authors’ knowledge, no adverse incident linked specifically to
EEG-fMRI data acquisition has been formally reported to date.

2.1.3 Image Quality Image quality remains an important issue throughout the field of
MRI and the subject of investigation, particularly for echo-planar
imaging (EPI), which is particularly prone to distortion and local
signal dropout [148]. Artifacts caused by electrodes and leads were
observed in early EEG-fMRI experiments [138]. Therefore, one
must consider carefully the choice of materials and components
placed within the field-of-view [146, 149–152]. It has been shown
that the presence of high-density (256 channels) EEG caps can
significantly impair structural MR imaging [153].
 fMRI in Epilepsy 765

2.1.4 EEG Quality In the literature it is common to categorize the artifacts observed
on EEG recorded inside the MR scanner into two types: heart
beat-related (whether scanning is taking place or not) and MR
image acquisition-related. These can be considered distinct in
terms of their generating mechanism, and deserve to be addressed
separately in terms of remedies to minimize them, as reflected in
the structure of this section. However, in practice they are linked
by a third phenomenon, widely recognized as a nuisance in fMRI,
namely subject motion. This is in part because the heart beat-related
artifact is thought to mostly originate from the body motion linked
to the heart beat, but also because subject (and EEG electrode and
lead) motion can have an important impact on the ability to cor-
rect both types of artifact, depending on the approach taken.
Therefore body motion is nefarious for EEG recording quality
(and almost without saying, fMR image quality) and should be
minimized. This will be the subject of Sect. 2.1.4.3.

EEG Quality: Pulse The first attempts at recording EEG inside MR scanners revealed
Artifact, Reduction, and the presence of pulse-related artifacts delayed in relation to the
Correction Methods QRS complexes on ECG [138]. This effect has been shown to be
common across subjects and has a slight frontal emphasis [154].
The pulse artifacts can have amplitude of the order 50 μV (at 1.5 T)
and resemble epileptic spikes. Because of natural heart beat variabil-
ity, it is considered a more challenging problem than that of image
acquisition artifacts. EEG artifacts linked to subject movement are
also amplified in the scanner’s strong static magnetic field.
The precise mechanism through which the circulatory system
exposed to a strong magnetic field gives rise to these artifacts
remains uncertain, but it is thought to represent a combination of
the motion of the electrodes and leads (induction) and the Hall
effect (voltage induced by flow of conducting blood in proximity
of electrodes) [155]. Electrode motion can result from local arte-
rial pulsation, brain and head motion or whole-body motion (bal-
listocardiogram, or BCG, in the latter case) [156, 157].
Methods to reduce artifacts at the source include: careful lay-
ing out and immobilization of the leads, twisting of the leads,
bipolar electrode chain arrangement [158], head vacuum cushion
[159] and the introduction of a reference electrode layer insulated
from the EEG-measuring electrodes to capture and subtract the
artifact from the EEG prior to amplification [160]. Such measures
do not eliminate the problem completely resulting in degraded
EEG quality, impeding the identification of epileptiform dis-
charges. The first pulse artifact reduction algorithm published, and
to this day still the gold standard against which most methods are
compared, is based on subtraction of a running average estimate of
the artifact based on automatic QRS detection, and is commonly
referred to as the average artifact subtraction (AAS) method [154].
Using this method, the residual artifact is of the order of a few
microvolts. The reliance of the algorithm on ECG is a common,
though not universal, feature among subsequently developed
766 Rachel C. Thornton et al.

techniques (some of which use the signal from the standard scan-
ner pulse oxymeter). The method has been and continues to be
used successfully in our lab allowing the satisfactory identification
of ictal and interictal epileptiform discharges (IED) in real time (at
1.5 T) [161] and for the purpose of source analysis [162], and has
been implemented in widely used commercial MR-compatible
EEG recording systems (see Fig. 6).
The artifact amplitude is theoretically directly proportional to
the scanner static field strength (B0). This phenomenon, and an
increasing interest in recording evoked potentials in the MR scan-
ner, has motivated an important research effort towards improving
existing pulse-related artifact reduction methods and the develop-
ment of new ones. Variants of the AAS method have been pro-
posed, ranging from different ways of estimating the artifact
waverform, for example to account for a greater degree of inter-
beat variability [158, 166–168], more general motion effects [151,
169, 170], to improving QRS detection [171] and removing the
need for ECG recording [172].

Fig. 6 IED-related BOLD pattern in patient with drug-resistant focal epilepsy. The patient had refractory focal epilepsy,
lateralized to the right with a normal structural MRI. Frequent mid and posterior temporal sharp waves were recorded
on EEG.(a) Representative segment of 32-channel EEG showing a sharp wave, maximum at the right mid-posterior
temporal region, recorded during two 20-min fMRI sessions. Top left: EEG prior to artifact correction [154, 163].
Fig. 6 (continued) (b) Design matrix: BOLD signal changes related to 40 sharp waves were modeled by convolution
of the EEG event onsets with a canonical HRF and its time-derivative. Signal changes linked to head motion and
heartbeat were modeled as nuisance effects [164, 165]. (c) Top: SPM showing significant sharp wave-related BOLD
response in glass brain display (p < 0.05 corrected for multiple comparisons). The red arrow marks the global maxi-
mum, located in the BA 28 (superior temporal gyrus). Bottom: BOLD response overlaid onto the patient’s normalized
T1-weighted volumetric scan. Intracranial recording confirmed a right posterior temporal lobe onset. No significant
sharp wave-related deactivation was revealed. The activation clusters were labeled using the Talairach Daemon,
http://ric.uthscsa.edu/project/talairachdaemon.html
768 Rachel C. Thornton et al.

Spatial EEG filtering methods have been proposed based on

temporal principal components analysis (PCA) or independent
components analysis (ICA) [152, 159, 173–180]. It is important
to note that some PCA and ICA-based correction methods may
not be applicable in real time, making it difficult to visualize epilep-
tiform activity during the experiments with possible practical and
safety implications. In studies in which residual noise was quanti-
fied, improvements of the order of 0.1–1 μV compared with vari-
ous implementations of the AAS method have been demonstrated
[172, 179, 181]. Two important pulse artifact correction meth-
ods, AAS and optimal basis set, have been evaluated and compared
independently [182].

EEG Quality: Image In the absence of any special measures, the EEG recorded inside
Acquisition Artifact, the MR scanner becomes un-interpretable during image acquisi-
Special EEG Recording tion because of the presence of repetitive artifact waveforms caused
Equipment, and by the time-varying fields employed in the scanning process super-
Correction Methods imposed on the physiological signal [163, 183]. In addition, on
some MR instruments, the helium cooling pump can introduce
significant amount of noise in the signals measured using the EEG
equipment, and a method to remedy this problem has been pro-
posed in cases where the pump cannot be switched off for the
duration of the scan [184].
One way of circumventing this problem is to leave time gaps in
the fMRI acquisition (e.g. between EPI volumes) of sufficient
duration to capture the EEG features of interest (assuming suffi-
cient data quality, e.g., following pulse artifact removal); this is
interleaved EEG-fMRI [158, 185, 186]. This approach relies on
artifact not persisting following each acquisition (e.g. due to ampli-
fier saturation). Interleaved EEG-fMRI can be most useful to study
predictable events (evoked responses) or slowly varying phenom-
ena, such as brain rhythms. EEG-triggered fMRI and in particular
spike-triggered fMRI, which involves limiting fMRI acquisition to
single or multi-volume blocks, each triggered following the
identification of an EEG event of interest is a form of interleaved
EEG-fMRI with obvious relevance to epilepsy [183, 187–190].
Although interleaved EEG-fMRI is capable of providing useful
data in many circumstances, it imposes a limit on experimental effi-
ciency due to EEG quality degradation during scanning.
We now review the technical developments that have made it
possible to record EEG of sufficient quality throughout fMRI
acquisition (so-called continuous EEG-fMRI), by the image acquisi-
tion artifact to be corrected. For all practical purposes, and assum-
ing that the time gap between volumes is the same as between slices,
the artifact’s spectral signature ranges from 1/TR (TR: slice acqui-
sition repetition time) to around 1 kHz (corresponding to the read-
out gradient). In fact it extends into the mega-hertz (RF) range,
well beyond the recording capability of any EEG equipment. It can
appear artificially benign when captured using standard EEG
 fMRI in Epilepsy 769

equipment [138]. Only using equipment with sufficient bandwidth,

sampling rate and dynamic range, can one capture the artifact with
adequate accuracy [163, 191].
Experiments have shown that the gradient switching-related
effects generally dominate over RF in terms of amplitude and extent
in time, although the balance between the two mechanisms will vary
depending on the specific MR sequence used [163, 171, 191, 192].
For standard EPI sequences, the pattern of gradients is repeated
exactly across slices. Compared with the problem of cardiac-related
artifact reduction, this determinism greatly facilitates the task of
image acquisition artifact removal; however, the induced waveforms
will be subjected to variations in time due to changes in the elec-
trode/lead configuration caused by subject motion.
Before discussing artifact reduction postprocessing techniques,
let us review some of the hardware modifications and other mea-
sures that can facilitate the recording of good quality EEG during
fMRI. First, some of the tricks described previously to reduce the
pulse-related artifact at the source, and in particular those to limit
the area of loops formed by EEG leads and head motion, can also
help to lessen the image acquisition artifact problem. Second, low-
pass filtering at the front end of the EEG system may be used to
reduce the artifact significantly, although not sufficiently to result
in adequate EEG quality [163]. Third, a scheme has been devised
to reduce the amplitude of the artifact at the source by modifica-
tion of the MR sequence and careful synchronization with EEG
sampling [191]. It has also been noted that the subject’s head posi-
tion can have an important impact on the magnitude of the
gradient-switching artifact and therefore this can be reduced at the
source through simple manipulation [193].
In the studies by Allen et al. [163] and Anami et al. [191],
custom-built EEG recording systems with high sampling rates
(1–20 KHz) and large dynamic range (∼20 mV), based on the
notion that the artifact must be captured accurately to be under-
stood, were measured and eliminated. Specially designed
“MR-compatible” EEG recording hardware has now become the
norm in the field, with a number of commercial products now
available on the market (see also [158, 192, 194] for a description
of other modified or purpose-built apparatus).
Postprocessing methods to reduce image acquisition artifacts can
be categorized as filtering, template subtraction methods or PCA/
ICA. Filtering based on the identification and subsequent suppres-
sion of frequencies linked to the image acquisition process can lead to
an improvement in EEG quality and be used for simultaneous EEG-
fMRI [168, 194, 195]. However, it is severely limited by the spectral
overlap between the artifact and physiological signals, ringing effects
and has been shown to be inferior to template subtraction [159].
The most commonly used image acquisition artifact reduction
method is based on average template artifact subtraction (sometimes
770 Rachel C. Thornton et al.

referred to as AAS) method [163] (see Fig. 6). It relies on the lack of
correlation between physiological signals and the artifacts, enabling
the latter to be estimated by averaging the EEG over a number of
epochs, corresponding to individual scan repetitions, for example.
The success of the artifact (template) estimation and its subsequent
subtraction from the ongoing EEG depend critically on the sam-
pling rate, the number of averaging epochs, and the precision of
their timing. In Allen’s original implementation, this is addressed by
the use of the scanner’s scan trigger pulse to mark each scan acquisi-
tion and interpolation. Following subtraction, residual artifacts are
reduced using Adaptive Noise Cancellation. The method can be
used in real time, allowing continuous EEG-fMRI studies in patients
with epilepsy [196–199]. Possibly the most important practical
development has been the demonstration that synchronized MR
acquisition and EEG digitization lead to significantly improved EEG
quality, and in particular over a wider frequency range, when com-
bined with an AAS-like method [225]. The method has been found
to perform well for spiral EPI [226]. As noted previously, changes in
the artifact waveform due to subject motion will lead to suboptimal
template estimation. To address this, refinements of Allen’s method
which incorporate PCA of the residual artifact have been proposed
[171, 227]. The shape of the image acquisition artifact may be cap-
tured in a separate experiment for subsequent subtraction [192]. In
the study by Wan et al. [228], a method designed to bypass the
requirement for a slice acquisition signal from the scanner is pro-
posed. As is commonly the case for artifact reduction methods based
on ICA, identification of the components containing artifact is
mainly done visually [179]. A difficulty encountered when compar-
ing the various methods available for artifact reduction is the range
of methodologies used. This has been addressed to some degree in
a rigorous comparative study [229].

EEG Quality: Subject (head) motion, and almost inevitably motion of the EEG
The Impact of Subject recording circuit (formed by the head, electrodes and leads) which
Motion results from it, will cause fluctuations in the recorded signal addi-
tive to the effects discussed in the previous sections. In most situa-
tions, this motion is not synchronized with either the scanning
process or heartbeat; due to the associated change in geometry of
the EEG circuit in relation to the scanner, it therefore can intro-
duce random fluctuations in the magnitude of both types of arti-
fact. In terms of the AAS algorithm, this phenomenon imposes a
limit on the quality of the artifact correction: the greater and more
random the motion, the less efficient AAS will be. The previously
mentioned measures to reduce the pulse artifacts based on subject
immobilization and minimization of EEG loops are obviously
worth reiterating at this point [158, 159]. More recently, a method
to measure head motion and use the information to reduce pulse
and scanning-related artifacts has been proposed [230, 231].
 fMRI in Epilepsy 771

2.2 Application fMRI can be used to investigate the hemodynamic correlates of

of fMRI to the Study paroxysmal activity, and in particular to reveal regional changes in
of Paroxysmal Activity the BOLD signal thereby potentially providing new localizing
information. See Table 3 for a list of notable published studies.
The conventional approach to the analysis of fMRI data is
predicated on the correlation of the fMRI time series with
experimentally-determined stimuli within the framework of the
general linear model (GLM). This methodology allows voxel-by-
voxel testing of the degree of fit of predicted and observed BOLD
time courses, and subsequent inferences. The application of fMRI
for the assessment of spontaneous paroxysmal activity in epilepsy
offers a number of additional challenges, namely: the lack of exper-
imental paradigm (subjects scanned in the resting state), the iden-
tification of paroxysmal activity in relation to the fMRI time series,
and the representation and translation of this activity into a
GLM. The latter point is crucial and as we will see, has been an area
of continuing investigation in the field.
In addition to the difficulties associated with the observation of
subjects within the confined space of an MR scanner, and perhaps
more importantly the use of behaviourally derived (seizure manifes-
tations) time markers when possible, although crucial, does not
provide as complete a picture of the event as one would wish given
the importance of putative concomitant EEG abnormalities.
The advent of EEG-correlated fMRI has been a major advance
in this respect, providing an established, albeit imperfect, marker of
paroxysmal activity and more generally brain state. Importantly, it
allows the study of interictal activity, particularly IED, which only
manifest on EEG. Although this type of data allows the experimen-
talist to study hemodynamic changes linked to specific EEG events,
it presents a number of challenges linked to the subjective nature
of EEG interpretation. The investigator is also soon confronted
with a “chicken and egg” type problem of not knowing precisely
what the time course of the changes is, and in particular whether
the hemodynamic “response” function associated with paroxysmal
discharges deviates from normality, a necessary element of the
modeling, or its spatial substrate, the latter being precisely the
motivation for undertaking the experiment given its clinical
implication.

2.2.1 Ictal fMRI in Focal As mentioned previously, a number of case studies of ictal events
Epilepsy captured using fMRI alone were published prior to the advent of
EEG-fMRI [139–142]. Despite the fact that these contain inter-
esting observations, particularly with regard to the signal change
around the time of seizure onset, the availability of simultaneously
recorded EEG would have contributed important information.
The development of the ability to record good quality EEG inside
the MR scanner has offered the possibility of improved models of
ictal fMRI signal changes by the inclusion of precisely timed
Table 3 EEG-fMRI studies of paroxysmal activity: early milestones and important series

Number of subjects/no.
Study in which IED recorded Results Conclusion Comment

Focal epilepsy 1/1 frequent IED Bilateral activation where EEG suggested left “We cannot make conclusions about the source of the
Warach et al. temporal localization and anterior cingulate discharge from the present data”
1996 [183] activation in relation to generalized
epileptiform activity
Seeck et al. 1998 1/1 frequent IED Multiple areas of signal enhancement on The combination of EEG-triggered fMRI and 3D EEG
[200] fMRI. Confirmed on 3D-EEG source source analysis, represents a promising additional tool
localization with evidence of a focal onset. for presurgical epilepsy evaluation allowing precise
Focus later confirmed on subdural recordings noninvasive identification of the epileptic foci
Symms et al. 1999 1/1 frequent IED Reproducible and concordant activation across four sessions
[201]

Patel et al. 1999 20/10 frequent IED 9/10 overall reported as showing “activation corresponding to the EEG focus”
[202]
Krakow et al. 10 frequent IED Reproducible activations (same lobe and overlapping) obtained in 6/10 patients in close spatial relation
1999 [187] to EEG focus
Krakow et al. 1 frequent IED Focal activation within a large malformation of cortical development in response to focal epileptiform
1999 [203] discharges
Lazeyras et al. 11 frequent IED Activation confirmed clinical diagnosis in 7/11. In 5/6 intracranial EEG confirmed result
2000 [204]
Lazeyras et al. 1/1 frequent IED Area of signal enhancement concordant with hyperintensity seen on ictal FLAIR images in a patient with
2000 [205] nonlesional partial epilepsy
Krakow et al. 24/14 frequent IED 12/24 patients showed activations concordant with EEG focus, 7/12 of which also had concordant
2001 [189] structural lesions. 2/24 were discordant and 10/24 showed no significant activation
Lemieux et al. 1/1 frequent, In a case with stereotyped frequent IED, BOLD Localization of BOLD activation was consistent with First description of
2001 [197] stereotyped IED signal change concordant with the seizure previous findings and EEG source modeling application of
onset zone was recorded continuous
EEG-fMRI
Jager et al. 2002 10/5 frequent IED, Focal activation in 5/5 patients, concordant with EEG amplitude mapping. Mean signal increase was
[206] focal epilepsy 15 ± 9 %. Spike amplitude correlated with volume of activation
Benar et al. 2002 4/4 frequent IED The average HRF presented a wider positive There was no clear correlation between the amplitudes
[207] lobe in three patients and a longer undershoot of individual BOLD responses and EEG spikes
in two.
Al-Asmi et al. 48/31 frequent IED BOLD activation in 39 % of studies. Concordant Combining EEG and fMRI in focal epilepsy yields
2003 [190] with seizure focus in almost all. four patients regions of activation that are presumably the source
had concordant intracranial recording (by of spiking activity and these are high
lobe)
Benar et al. 2006 5/5 presurgical When an intracranial electrode is in the vicinity of an EEG or fMRI peak, it usually includes one active Largest series of
[208] candidates having contact intracranial
sEEG EEG correlated
fMRI activation
Aghakhani et al. 64/40 focal epilepsy A positive thalamic response was seen in 12.5 % The thalamus is involved in partial epilepsy during
2006 [209] with frequent of studies with unilateral and 55 % with interictal discharges. This involvement and also
unilateral or bilateral spikes. Cortical acitvation was more cortical deactivation are more commonly seen with
bilateral IED concordant with focus than deactivation bilateral spikes than focal discharges
Salek-Hadaddi 63/34 focal epilepsy Significant hemodynamic correlates were These findings provide important new information on
et al. 2006 frequent IED detectable in over 68 % of patients and were the optimal use and interpretation of EEG-fMRI in
[199] highly, but not entirely, concordant with site focal epilepsy
of presumed seizure onset
Ziljmans et al. 29/15 focal epilepsy, 8/15 subjects: IED correlated BOLD response EEG-fMRI provides additional information about the First evaluation of
2007 [210] declined for surgery at site of focus. Multifocal in 4, unifocal in 4. epileptic source in the presurgical work-up of impact on
Thornton et al. 23/12 focal epilepsy, Concordant with IC data in 2 complex cases presurgical
2011 [211] focal cortical 11/12 showed significant IED-correlated Widely distributed discordant regions of IED-related evaluation
Pittau et al. 2012 dysplasia, BOLD. BOLD matched icEEG SOZ and hemodynamic change appear to be associated with a EEG-fMRI (with
[212] intracranial EEG outcome was > 50 % seizure reduction in 5; widespread SOZ and poor postsurgical outcome video) of
Chaudhary et al. and surgery BOLD was widespread in 6 and outcome EEG-fMRI may contribute to the localization of the seizures can be
2012 [213] 43/33 focal epilepsy poor in 5/6 interictal epileptic generator in patients with focal done with
Coan et al. 2015 20/15 seizures in 21/33 IED-related BOLD contributed to the epilepsy acceptable risk,
[214] focal epilepsy delineation of the focus compared to scalp Preictal and ictal haemodynamic changes in refractory and provides
30 patients who EEG; icEEG validation was positive in 12/14 focal seizures can noninvasively localize seizure important new
underwent surgery patients onset at sublobar/gyral level when ictal scalp localizing
for TLE; in 14 Widespread preictal BOLD changes followed by electroencephalography is not helpful information
cases with no IED more focused early ictal and spread Interictal EEG-fMRI retrospectively confirmed the
during fMRI, a Good surgical outcome in 13/16 patients with epileptogenic zone of TLE patients
topographic concordant BOLD changes and in 3/14
method was used patients with discordant BOLD
[215]

(continued)
Table 3 (continued)

Number of subjects/no.
Study in which IED recorded Results Conclusion Comment

Tousseyn et al. 28/27 refractory High congruence between maps derived from Hemodynamic changes related to seizures and spikes Largest
2015 [216] focal epilepsy ictal the two techniques; some discrepancies varied spatially within a common network. Overlap comparison of
SPECT and observed nearby and distant from discharge origin interictal
interictal EEG-fMRI and
EEG-fMRI ictal SPECT
Generalized 1/1 prolonged GSW Thalamic activation and widespread cortical Supports thalamo-cortical model of GSW
epilepsy epochs (ictal) deactivation
Salek-Haddadi
et al. 2003
[217]
Baudewig et al. 1/1 frequent GSW Unilateral insular activation shown in relation to Strategy resulted in robust BOLD MRI responses to
2001 [218] generalized epileptiform discharges epileptic activity that resemble those commonly
observed for functional challenges
Hamandi et al. 46/30 interictal GSW Thalamic activation and cortical deactivation Observed cortical deactivation may represent correlate
2006 [196] in IGE and SGE observed at group level. Deactivation in the of clinical absence seizure.
default brain areas. Cortical pattern mixed at
individual level
Aghakhani et al. 15/14 interictal GSW Bilateral thalamic activation in 80 % of BOLD Cortical deactivation mediated by hyperpolarization of
2004 [219] response. Cortical deactivation in 93 % the thalamus
Hamandi et al. 4/4 interictal GSW Qualitatively reproducible BOLD and blood Consistent with preserved neurovascular coupling in
2007 perfusion patterns; Cortical deactivation GSW and decreased cortical activity
corresponds to decrease in blood flow
Yang et al. 2013 10/10 drug-naïve GSW discharge-related alterations in the default Interictal GSWDs can cause dysfunction in specific First study
[220] CAE patients; mode network (DMN), cognitive control networks important for psychosocial function specifically
interictal GSW network (CCN), and affective networks focused on
functional
connectivity
alterations
during GSWD
in drug-naive
patients
Vaudano et al., 15 with eyelid Elevated eye closure-related BOLD signal in the Supports concept of EMA as a distinct epileptic
2014 [221] myoclonus with visual cortex, posterior thalamus, and eye condition
absences motor control areas in EMA group compared
(EMA) + 14 with to IGE
IGE
Children 6/6 focal epilepsy Concordant activation with presumed focus in EEG-fMRI is a promising tool to noninvasively First series
De Tiege et al. (lesional and four cases. IC recording corroborative in 1 localize epileptogenic regions in children with specifically
2007 [222] nonlesional) pharmacoresistant focal epilepsy addressing use
of EEG-fMRI in
children
Jacobs et al. 2007 9/9, mixed focal All had BOLD activation or deactivation EEG-fMRI at 3 T could be useful to localize focus in
[223] epilepsy, sedated concordant with seizure focus. Deactivation children with focal epilepsies. Nature and origin of
appears more common in adults the negative BOLD requires investigation
Jacobs et al. 2007 13/13 symptomatic Activation corresponding with the lesion was “Good results could be obtained from the EEG-fMRI
[224] (lesional) epilepsy), seen in 20 % and deactivation in 52 % of the recordings, performed in sedated children”
sedated studies
776 Rachel C. Thornton et al.

EEG-derived information in the model, and also by providing an

indication of head motion due to artifacts on the EEG which may
assist interpretation. Clearly the problem of motion is particularly
pertinent to the acquisition of seizure data, as it can severely
degrade image data quality and consequently the ability to detect
and map regional hemodynamic changes accurately in relation to
those manifestations [232–234].
Given the above limitations and concerns regarding patient
safety, EEG-fMRI data of seizures is often acquired incidentally
[198, 205, 235–237]; however, a systematic study of patients
recruited specifically in the expectation of capturing seizures has
provided new information on the transition from the interictal
state, and ictal onset and evolution (spread) [213]. A noteworthy
observation was the common involvement of multiple regions
beyond the conventionally defined EZ, during the preictal period.
The electrophysiological substrate of ictal BOLD increases and
decreases were specifically investigated in a case who underwent
SEEG [238], showing distinct signatures.

2.2.2 Interictal fMRI: Although the study of ictal events may be useful in pursuing the
EEG-fMRI aim of noninvasively identifying the seizure onset zone and thereby
contributing most to presurgical evaluation, its acquisition remains
challenging. Attention has, therefore, focused on interictal activity
and the insight gained from it into the function of epileptic net-
works. Spike-triggered fMRI acquisition was used in the first appli-
cations of EEG-fMRI in epilepsy, which tended to focus on the
study of subjects pre-selected for the large number of IED observed
in prior routine surface EEG or as part of presurgical assessment.
The acquisition and analysis of spike-triggered fMRI data generally
rests on the assumption that interictal spikes are associated with a
BOLD signal change pattern similar to the so-called canonical
HRF. In these series, regions of positive BOLD signal change asso-
ciated with IED were observed in approximately 50 % of the cases
overall, and occasional negative changes [183, 187, 189, 190,
200–202, 204, 239, 240]. Using bursts of BOLD EPI scans,
Krakow et al. made an initial attempt at estimating the shape of the
IED-related HRF [189]. We note that in the work by Seeck and
colleagues, Clonazepam was used to suppress interictal discharges,
thereby creating a control scan state [200, 204]. Interleaved EEG-
fMRI, whereby fMRI data are acquired in blocks with inter-block
gaps of a sufficient duration to allow interpretation of part of the
EEG was employed in a patient with IGE for the mapping of
BOLD changes related to spike-wave complexes [218].
Following the implementation of image acquisition artifact
removal [163], the technique of continuous, simultaneous EEG-
fMRI acquisition was demonstrated along with estimation of the
shape of the IED-related HRF in a subject with Raussmussen’s
encephalitis and stereotyped high amplitude sharp waves on the
 fMRI in Epilepsy 777

EEG [197]. See Fig. 6 for example of EEG-fMRI in patient with

focal epilepsy. When applied to relatively large case series of subjects
with predominantly focal epilepsy selected on frequent interictal dis-
charges on routine EEG [190, 199], a yield (proportion of signifi-
cant BOLD activations or deactivations) of 60–70 % was observed
where IED were recorded. This result has been reproduced in fur-
ther studies despite variation in the analysis strategies and patient
groups. Sensitivity can be increased using spike template matching
based on EEG recorded outside the MRI scanner [241]. Positive
BOLD changes tended to correspond with the site of the presumed
seizure onset zone (based on electroclinical localization), but occa-
sionally appeared at sites distant from this region, the significance of
which remains unclear. Negative BOLD changes were more often
observed remote from the presumed focus, with a striking pattern of
retrosplenial deactivation in a significant proportion of cases [199].
It was shown that activations were more likely when there was good
electroclinical localization, frequent stereotyped spikes, less head
motion, and less background EEG abnormality. Furthermore, the
findings suggest that significant activation is more likely for runs of
spikes than for isolated discharges, when the event duration is taken
into account in the modeling [199, 242].
In TLE, IED-related activation of the MTL ipsi-lateral to the
presumed focus was found to be common, and is reminiscent of
the thalamic pattern observed in generalized spike and wave dis-
charges (GSW) [243].
The possible explanations for lack of activation include: com-
bination of insufficient number of events and limited fMRI sensi-
tivity, incorrect model due to suboptimal EEG event identification
and classification, choice of HRF and limited extent to which scalp
EEG reflects ongoing activity (throughout the brain). Automated
and semi-automated spike detection methods have been proposed
to attempt to reduce the level of subjectivity of EEG event identi-
fication [162]. Nonetheless, although the ability to record EEG
during fMRI is necessary to study the hemodynamic correlates of
EEG events observed on the scalp, total reliance on scalp EEG can
also be seen as a limitation when attempting to interpret the BOLD
signal throughout the brain and in particular the part which may
not be linked to activity reflected on the scalp. In the absence of
clear epileptiform discharges, the presence of slow activity may
provide useful localizing information [244, 245]. However, if data
from clinical video-telemetry EEG are available showing IED, the
corresponding topographic map can be correlated with that of the
intra-MRI EEG as a function of time to attempt to reveal BOLD
changes corresponding to the epileptic topographic pattern [246].
Data-driven or region-based fMRI analysis techniques may offer
another way forward in this respect, although interpretation of the
observed patterns in the former is significantly impaired by the lack
of an a priori model and numerous possible confounds [247, 248].
778 Rachel C. Thornton et al.

2.2.3 The HRF in Focal The normal hemodynamic response associated with neuronal activ-
Epilepsy ity arising from brief external stimuli in humans has a characteristic
shape with a peak at around 5–6 s following the event, the so-
called canonical HRF, with a significant degree of inter-subject
variability [249]. The shape of the HRF is a key element of fMRI
signal modeling with an important impact on sensitivity, and it is
therefore important to attempt to characterize it. In epilepsy, par-
ticularly in relation to deactivations it has been proposed that neu-
rovascular coupling is abnormal with possible consequences on the
shape of the HRF, and this was suggested to be one possible expla-
nation for the significant widespread BOLD deactivations observed
in both focal and generalized epilepsy [250, 251].
This has lead to increased interest in estimating the shape of
the HRF in epilepsy. An efficient way of estimating the shape of the
hemodynamic changes linked with epileptiform discharges is by
using a set of functions (sometimes referred to as “basis set”) that
can, by linear superposition, fit signal changes with almost any time
course. Various such schemes have been used in studies of epilepsy
including sets of gamma response functions, Fourier basis sets and
a linear combination of canonical HRF, its time derivative and a
dispersion derivative [197, 199, 207, 252, 253]. Although a
degree of variability has been observed, the HRF linked to IED
was found to be principally canonical in shape [254]. In some cases
deviation from the norm at locations distant to the IED may reflect
artifacts, while in others, deviant activation in proximity to the pre-
sumed focus, early responses may reflect brain activity that system-
atically precedes the event captured on the scalp or propagation
[252, 255]. In a recent study, the development of penicillin-
induced epileptic activity was correlated with increase in BOLD
signal prior to the onset of IED [256]. These studies coupled with
those combining EEG-fMRI with multiple source analysis [257]
suggest that in some cases the maximum BOLD response may be
detected just prior to seizure or even IED onset. It may be, how-
ever, that this reflects BOLD changes linked to interictal abnor-
malities, which are not included in the model by virtue of the fact
they are not seen on the scalp EEG.

2.2.4 Clinical Relevance Having shown that EEG-fMRI is capable of providing a unique
of fMRI of Paroxysmal form of localizing information, the issue of its clinical value arises.
Activity in Focal Epilepsy The evaluation of new noninvasive imaging modalities in the pre-
surgical assessment of patients with drug-resistant epilepsy is a
complex issue in part due to the lack of an established method-
ological consensus (baseline) across centers. Added value and clini-
cal relevance are a function of the new test’s sensitivity and
specificity. In the case of EEG-fMRI, neither has been properly
assessed to date. Thus far the field has focused on proof of principle
demonstrations, usually in patients selected based on high rates of
EEG abnormalities, with a success rate of roughly 50–60 % [183,
 fMRI in Epilepsy 779

187, 189, 199, 202, 204, 240]. Therefore, one may anticipate a
lower success rate in the most clinically challenging cases for which
the need for noninvasive assessment is most pressing.
When available, the new localizing information must be evalu-
ated relative to a surgically confirmed irritative and epileptogenic
zone [258] using the current gold standard of invasive recording
and outcome data. In practice, a gradual approach to validation is
often taken, whereby the face validity of new localizing informa-
tion is tested against other existing techniques in cases with well
characterized syndromes, such as mesial TLE.
In our laboratory, we have assessed the value of the fMRI find-
ings by comparing the localization of the BOLD cluster containing
the most significant activation to the seizure onset zone defined
electro-clinically, when possible, and found a very good degree of
concordance at the lobar level [187, 189, 199]. Additional regions
of activation were observed in roughly 50 % of the cases with
significant activation. The finding of localized BOLD activation in
cases with poor electroclinical localization suggests a means of
obtaining target areas for intracranial EEG [199]. In TLE, the
yield has been characterized as relatively high [259] and the degree
of concordance of BOLD activations with the presumed focus gen-
erally good in one study [199], but more varied in another [259].
Comparison of EEG-fMRI with source localization suggests
that IED-related BOLD activation are often in proximity to those
detected by conventional EEG source localization, but some stud-
ies have suggested a distance of up to 50 mm. The possible sources
of discrepancy between BOLD and electrical (or magnetic) source
localization include: differences in the nature of the observed phe-
nomena and neurovascular coupling, vascular architecture and
scanner field-related effects on sensitivity, instrumental and physi-
ological noise, source reconstruction limitations, fMRI sensitivity
limitations [208, 260–262].
Presurgically, intracerebral EEG data are widely recognized as
the localization gold standard. Comparison of scalp EEG-fMRI
against invasive EEG has been performed in small groups and using
widely varying fMRI analysis techniques and comparison criteria
[190, 200, 250], noting a degree of concordance between the epi-
leptogenic zone and the area of maximal BOLD activation in some,
but not all subjects [204, 208]. The potential role of EEG-fMRI in
presurgical evaluation was assessed in a series of patients with focal
epilepsy in whom surgery was not offered following conventional
electroclinical evaluation [210]. The impact of the EEG-fMRI find-
ings was assessed in eight cases with unclear foci or suspected muti-
focality (based on the center’s usual battery of tests) in whom
significant IED-related BOLD activation were observed: IED-
related changes suggested a more restricted seizure onset zone in
four, and multifocality in a further patient. The EEG-fMRI finding
was concordant with intracranial recordings in two. The authors
780 Rachel C. Thornton et al.

suggest that this supports a possible role for EEG-fMRI in presurgi-

cal evaluation where a potential, but possibly widespread focus is
identified. In a series of 23 patients with FCD who underwent inva-
sive EEG, EEG-fMRI mapping of IED was successful in 50 %; the
pattern of IED-related BOLD changes tended to reflect the inva-
sive EEG findings and in particular widespread involvement was
associated with poor surgical outcome or widespread SOZ, sug-
gesting a significant clinical role [211]. In another substantial retro-
spective case series, Pittau et al., found a high degree of concordance
of the interictal EEG-fMRI localization with the confirmed or pre-
sumed IZ, and showed the test to provide added value (“contribu-
tory” to the definition of the epileptic focus) in a large proportion
of cases [212]. A comparison of the results of IED mapping and
ictal SPECT in a series of 28 patients has revealed a common high
degree of spatial overlap often beyond the presumed EZ, possibly
reflecting common propagation pathways, with some notable mis-
matches which the authors explain by the different nature of the
two techniques and inter-session effects [216].

2.2.5 Relationship Focal epilepsy can be divided by pathological subtype. Given the
with Pathology in Focal knowledge that the irritative and epileptogenic zones may extend
Epilepsy beyond the area of pathological abnormality and that animal models
suggest abnormal subpopulations of neurons within dysplastic areas,
there have been studies of EEG-fMRI aimed at evaluating the hemo-
dynamic response in abnormal tissue revealed on MRI [203, 211,
244, 263–266]. In a series of 14 cases with heterotopia notable vari-
ability of the BOLD response across abnormal tissue was observed,
but the area of BOLD signal increase was often concordant with the
area of pathology, with a more mixed pattern of deactivations [266].
In MCDs and particularly Taylor type focal cortical dysplasia, BOLD
signal increase was observed within the lesion while peri-close and
distant from the lesional activity displayed a negative BOLD response
in four out of six cases [237]. Other studies in cases with MCD have
supported these findings. IED-related BOLD signal changes have
been observed in patients with cavernomas [267] close and distant
from the lesion. The frequently observed negative BOLD responses,
particularly in MCD have been attributed to loss of neuronal inhibi-
tion (in the presence of normal neurovascular coupling) in the
regions surrounding the abnormality or abnormalities in neurovas-
cular coupling itself. The significance of these deactivations will be
discussed in more detail later.

2.2.6 fMRI EEG-fMRI has been used to attempt to localize sources in specific
in the Investigation epilepsy syndromes, in addition to adding evidence to the under-
of Epilepsy Syndromes standing of the differences in subtypes of focal epilepsy. TLE is of
particular interest in this context as surface EEG may not detect
the deep sources involved in TLE and surgery for TLE, where it is
correctly localized, is associated with excellent outcome [268].
 fMRI in Epilepsy 781

A large series of subjects with temporal and extra-temporal lobe

epilepsy showed that temporal lobe spikes are seen at the presumed
seizure focus, but also unsurprisingly on the opposite homologous
cortex, but did not give further localizing information [259].
A further more recent study compared IED-related BOLD
responses between subjects with temporal and extra-temporal lobe
epilepsy and found that those brain areas involved in the so-called
“default mode network” [269] were commonly deactivated during
temporal lobe IED whereas other areas were involved in BOLD
activation and deactivation in extra-temporal lobe epilepsy [243].
Other subtypes of epilepsy studied include benign rolandic
epilepsy of childhood [262, 270] and other focal epilepsies in
children, which have demonstrated concordant IED-correlated
BOLD activation in approximately 60 % of cases [222, 223]. A
study of lesional epilepsy in children not only revealed positive
BOLD response concordant with the seizure onset zone in a sig-
nificant number of subjects, but also a higher number of deactiva-
tions than those observed in adult studies [223]. However, sedation
was used in this study, the effect of which has not been investigated
in detail to date. EEG-fMRI is a particularly attractive option for
the study of childhood epilepsies as it is noninvasive, but experi-
ments are long and require a high degree of subject cooperation.
In a recent study comparing the BOLD patterns (and cortical
morphology) associated with eye closure and (spontaneous and
triggered) GSW in patients with eyelid myoclonia with absences (a
form of reflex epilepsy) has provided evidence of specific alterations
in the visual (and other, related) system, thereby supporting the
notion of a distinct epileptic syndrome [221].

2.2.7 EEG-fMRI The generalized epilepsies (IGE) (that is the syndromes of

in Generalized Epilepsy Absences, Myoclonic epilepsies, and primary generalized tonic
clonic seizures coupled with generalized spike and wave discharges
on the EEG [271] are not currently amenable to surgical treat-
ment. Therefore, the primary focus of EEG-fMRI investigations of
generalized epilepsy has been the exploration of hypotheses devel-
oped in vitro and animal models [196, 217, 219, 261, 272–276].
Specific BOLD activation and deactivation of resting state net-
works are found to be correlated with interictal GSW activity. In
particular, the so-called Default Mode Network (DMN), normally
found to be particularly active at rest, is commonly deactivated and
thalamic-cortical network activated [273, 277, 278]. Moeller and
colleagues [279] found changes in functional connectivity during
GSW and GSW-free period.
It is commonly assumed that decreased DMN activity is a con-
sequence of the epileptic discharges, perhaps reflecting alterations
in awareness. This may be the case. However, a study using con-
nectivity analysis has suggested that the deactivation of the DMN
drives thalamic-cortical network changes and can lead to the pro-
duction of generalized spikes waves (GSW) suggesting a large-scale
782 Rachel C. Thornton et al.

interaction between different brain networks. In particular, it has

been proposed that the precuneus has a crucial role in facilitating
the production of GSW and in alter the state of consciousness
[280]. Other evidence based on the analysis of the temporal
sequence of BOLD response suggesting an early frontal activation
in support of the cortical focus theory [281]. Moreover, other
results have shown that the thalamus is the pacemaker of the
abnormal status in brain networks in this patient population [282].
As demonstrated, there is no clear evidence regarding the direction
of the interaction between resting state networks and GSW.
Simultaneous EEG-Arterial Spin Labelling (ASL) MRI in
patients with spike and wave discharges revealed degrees of corre-
lation between BOLD and CBF consistent with normal neurovas-
cular coupling [277]. This study also revealed a remarkable degree
of within subject, inter-session (and inter-scanner) reproducibility,
albeit in a small group.

2.2.8 The Significance BOLD signal deactivation has been observed in the monkey visual
of BOLD Deactivation system and found to have a linear relationship with CBF in the
in Epilepsy same way as positive BOLD signal change in normal physiological
conditions [283]. The pattern of cortical deactivation commonly
observed in relation to GSW using EEG-fMRI in humans has been
discussed earlier. Gotman et al. proposed that the reason for
observing the widespread cortical deactivation may be hypersyn-
chronization of the thalamus, supporting Avoli’s proposed model
of the thalamus driving the cortex during GSW [273]. The results
of Hamandi et al. using EEG-ASL are consistent with GSW-related
deactivation reflecting decreased cortical activity [277].
Focal IED-related deactivations are less common than activa-
tions and seem particularly linked to the presence of activation,
possibly reflecting a smaller hemodynamic effect of individual
events [199, 251]. In cases with MCD IED-related negative
BOLD signal change adjacent to the seizure onset zone or area of
dysplastic cortex, has led to several possible explanations including
“vascular steal” from more metabolically active regions, abnormal
neuronal coupling, or perhaps more likely, loss of inhibitory neu-
ronal activity in these regions supporting the link between BOLD
deactivation and underlying decrease in neuronal activity [251,
264].
Deactivation of the default mode areas is a typical feature of
IED-related BOLD changes in TLE in contrast to cases with extra-
TLE [243]. The observed pattern may reflect IED-related effects
in areas involved in cognition, with a possible link to transient cog-
nitive impairment [284].

2.2.9 fMRI Although limited by the sluggish BOLD response, we have seen
and the Neurobiology that EEG-fMRI is able to reveal multiple regions more or less
of Epileptic Networks simultaneously activated or deactivated in relation to EEG events
 fMRI in Epilepsy 783

[196, 199, 243]. The involvement of the thalamus in GSW has

been discussed earlier, but it has also been observed that subcorti-
cal BOLD activation can be seen in focal epilepsy [209, 259, 264].
A spike-triggered EEG-fMRI study of subjects with MCD identi-
fied BOLD activation concordant with seizure onset in all patients
studied, and subcortical activations in various structures were
observed in 50 % of subjects [264], and in Agakhani et al.’s study
of focal epilepsies, thalamic activations were observed correlated
with interictal IED, particularly where these were bilateral and syn-
chronous [209].

2.2.10 Limitations, The study of spontaneous, pathological brain activity using fMRI
Challenges, and Future presents the experimentalist numerous challenges. The advent of
Work simultaneously recorded EEG greatly facilitates this endeavor. EEG-
fMRI was originally held to be an excellent tool incorporating the
spatial coverage and resolution of imaging techniques and the tem-
poral resolution of EEG. However, a number of challenges remain.
Technically, postprocessing methods are now available to offer
EEG quality sufficient to allow a degree of abnormality identifica-
tion reliability comparable to that of routine clinical EEG. Although
EEG quality is sufficient to detect most IED with a good degree of
certainty (at 1.5 T) [161], pulse-related artifacts can sometimes
interfere with EEG interpretation particularly at 3 T. Improved
artifact correction and the use of techniques based on EEG source
reconstruction may yield more localized and reproducible results
[162, 257]. In addition, very little work has been done on this
aspect of EEG interpretation and in particular the differences
between the clinical and experimental approaches, with the former
using a summary of the abnormalities observed over the whole
EEG rather than categorization and quantification of all IED
required for fMRI. Concerning the effects of motion, which is par-
ticularly problematic in patients with epilepsy (and other neuro-
logical conditions), on MR image and EEG quality, the advent of
prospective motion correction offers a possible way forward [285].
Reliance on scalp EEG for fMRI modeling is both a strength,
as it allows to answer the question “what, if any, are the BOLD
correlates of EEG pattern X?”, and weakness because it is bur-
dened with certain limiting aspects of scalp EEG, such as sensitivity
bias subjectivity in interpretation, and the unpredictable nature of
the phenomena of interest. This presents the investigator with sig-
nificant challenges in terms of unpredictable experimental effi-
ciency (and consequently yield) and EEG interpretation for the
purpose of GLM building. The former may require a more aggres-
sive approach possibly using drug management as a tool for modu-
lating EEG activity [200]. Although EEG event classification
remains problematic, possible solutions may give the opportunity
of using fMRI to inform EEG interpretation [162].
784 Rachel C. Thornton et al.

The “yield” of EEG-fMRI studies is to some extent a function

of the previous points. A particular problem is that subjects are
scanned for a limited period of time, and interictal discharges are
not always observed during this period. A significant BOLD
response is also only present in 50 % of cases giving an overall
“yield” of the order of only 25 % in many studies.
Attempts at interpreting resting state fMRI patterns in patients
with epilepsy without reference to EEG (or clinical manifestations)
using data-driven analysis techniques have had mixed success
[248], but one can envisage the possibility of being able to reliably
identify sets of patterns that are potentially related to epileptic
activity, with 100 % yield [286].
Despite providing whole-brain coverage of putative BOLD
changes, the technique is effectively limited to the study of the events
seen on the scalp EEG. We know from intracranial EEG that a large
amount of pathological activity is not seen on the scalp, despite orig-
inating from a fixed location [287]. Therefore, the possible mis-
match between scalp-derived model and ongoing activity raises the
question of the nature of the baseline, with implications for sensitiv-
ity. This may be reflected in event onset time offset reflected in “early
responses” [252, 253]. Furthermore, this means that EEG-fMRI
cannot be used to exclude regions from epileptogenic zone.
An additional degree of modeling uncertainty arises with
regards to the IED-related hemodynamic response function linked
with the potential effects of pathology and the nature of scalp EEG
[250]. However, the latest evidence points to a preserved neuro-
vascular coupling and shape of the HRF in line with the physiolog-
ical response in healthy brains [252, 277].
The sluggishness of the BOLD response in relation to EEG
means that activation patterns effectively represent a time averaged
picture of a sequence of neuronal events, such as propagation and
loops. The causal relationship between the activity in these regions
on the one hand and the EEG on the other cannot be untangled
based on the correlation-based machinery that is the
GLM. Nonetheless, these sets of regions may be thought of as a
static or averaged picture of evolving networks, which may be sub-
jected to further investigation. This may be particularly relevant to
spontaneous brain activity compared with experimentally con-
trolled experiments because of the greater uncertainty in the origin
and sequence of neuronal events. The neurophysiology underlying
the BOLD response is not fully understood. Much remains to be
elucidated on the spatial-temporal relationship between IED and
time-locked BOLD signal changes, for example the significance of
responses distant from the presumed focus for which extensive
comparison of the fMRI with the gold standards of intracranial
EEG and postsurgical outcome will be required. The possibility of
using MRI to directly detect local changes in magnetic field is an
exciting prospect [288].
 fMRI in Epilepsy 785

The interpretation of BOLD activation patterns consisting of mul-

tiple clusters in relation to focal IED remains an area of active investiga-
tion. Recently, an attempt has been made to increase the technique’s
ability to temporally resolve multiple regions of BOLD activation by
combining EEG-fMRI with multiple source analysis [262].
Finally, EEG-fMRI as implemented and analyzed currently may
be considered purely as a hemodynamic imaging technique, with
EEG simply acting as a time marker for scan classification. However,
we envisage a more symmetrical approach whereby the two forms
of data are used to infer neuronal activity, which must be the ulti-
mate aim of the entire functional imaging enterprise [289].

2.2.11 fMRI of The possibility of recording EEG of good quality during fMRI has
Paroxysmal Activity: created a new instrument, which to date has been mainly used in
Conclusion an exploratory fashion.
EEG-fMRI of ictal and interictal activity in focal epilepsy has
demonstrated the capability to provide new localization information
in a large proportion of cases. In focal epilepsy, varied patterns have
been identified often suggestive of “functional lesions” homologous
to abnormalities seen on structural imaging, but additionally the
involvement of possibly less disease-specific regions. The clinical
value of this information remains uncertain and is the subject of
ongoing investigations. At the very least, EEG-fMRI may provide
complementary data useful for planning of invasive recordings where
conventional localization of the seizure focus is unsuccessful.
However, there are signs that it may be able to offer more.
In generalized epilepsy, EEG-fMRI has revealed activation and
deactivation patterns that are mainly suggestive of less specific
effects linked to generalized spike-wave, rather than reflecting syn-
drome, the spatial distribution of the EEG generators or more spe-
cifically a putative focus responsible for initiating GSW due in part
to the averaging effect of BOLD.

2.2.12 Data Acquisition Some of the brain regions of most interest in epilepsy are subject
and Modeling Challenges to susceptibility artifact, leading to geometric distortions and sig-
in Clinical fMRI nal loss during fMRI acquisition. Ideally in the absence of an
applied gradient, the magnetic field would be homogenous
throughout the bore of an MRI scanner. Unfortunately, the differ-
ent magnetic properties of bone, tissue, and air introduce inhomo-
geneities in the field when a head is introduced into the bore. Brain
regions closest to borders between sinuses and brain or bone and
brain, for example the inferior frontal and MTLs, are most affected,
and therefore especially likely to suffer geometric distortions or
signal loss [290]. This can result in reduced sensitivity and ana-
tomical uncertainties when interpreting the images. Most epilepsy
studies have been performed on 1.5 T clinical MRI scanners.
Scanning at higher field strength improves signal-to-noise ratio but
increases distortions and dropout [291].
786 Rachel C. Thornton et al.

Geometric distortions of the EPI data make it difficult to

directly overlay fMRI activations on coregistered high-resolution
scans. They can be unwarped using techniques that map the local
field in the head [292], though it has been shown that approaches
of this kind can introduce extra noise into the corrected EPI data
[293]. Alternative acquisition sequences that do not experience
geometric distortions are available [294] though these rarely have
the temporal resolution or high signal to noise ratio (SNR) per
unit time of EPI.
The second artifact in EPI data is more serious, as signal loss
leads to sensitivity loss, which is unrecoverable by image processing
techniques. Some of these artifacts can be corrected by shimming,
a process whereby the static magnetic field is made more homog-
enous over the region of interest [290]. Some will remain, how-
ever, leading to distortion and dropout in echo-planar images.
Other approaches to removing dropout often involve acquiring
extra images, leading to a loss of temporal resolution [295], but
more recent work has shown that dropouts and distortions can be
reduced without incurring time penalties if regions of reduced spa-
tial extent are imaged [296]. The use of high-performance gradi-
ents and thin slice acquisitions ameliorate these problems and
improve fMRI quality. fMRI is also extremely sensitive to motion,
although generally epilepsy patients are familiar with MRI scanners
and may move less than control subjects [29].
Motion remains problematic for all applications of fMRI
[234], but the problem may be particularly harmful in patient
studies in view of two factors: first, patients may be more prone to
movement than selected and highly motivated healthy subjects;
second, each patient dataset may have greater value than that from
inter-changeable healthy subjects. Therefore, measures have been
proposed to extract as much information as possible from what are
effectively suboptimal fMRI studies [164, 165, 297].

Acknowledgments

Thanks to Suejen Perani for help with the second edition and to
Philip Allen for his comments on parts of the manuscript and to
Dr. Anna Vaudano and Dr. Serge Vulliemoz for supplying some of
the illustrations. Some of the work reported in this chapter was
funded through a grant from the Medical Research Council (MRC
grant number G0301067) and by the Wellcome Trust. We are
grateful to the Big Lottery Fund, Wolfson Trust, and National
Society for Epilepsy for supporting the NSE MRI scanner. This
work was carried out under the auspices of the UCL/UCLH
Biomedical Comprehensive Research Centre.
 fMRI in Epilepsy 787

References

1. Wagner DD, Sziklas V, Garver KE, Jones- 13. Bonelli SB, Powell R, Thompson PJ,
Gotman M (2009) Material-specific later- Yogarajah M, Focke NK, Stretton J, Vollmar
alization of working memory in the medial C et al (2011) Hippocampal activation corre-
temporal lobe. Neuropsychologia 47:112–122 lates with visual confrontation naming: fMRI
2. Campo P, Garrido MI, Moran RJ, Garcia- findings in controls and patients with tempo-
Morales I, Poch C, Toledano R, Gil-Nagel ral lobe epilepsy. Epilepsy Res 95:246–254
A et al (2013) Network reconfiguration and 14. Gartus A, Foki T, Geissler A, Beisteiner R
working memory impairment in mesial tem- (2009) Improvement of clinical language
poral lobe epilepsy. NeuroImage 72:48–54 localization with an overt semantic and syntac-
3. Stretton J, Winston G, Sidhu M, Centeno tic language functional MR imaging paradigm.
M, Vollmar C, Bonelli S, Symms M et al AJNR Am J Neuroradiol 30:1977–1985
(2012) Neural correlates of working memory 15. Sanjuan A, Bustamante JC, Forn C, Ventura-
in Temporal Lobe Epilepsy--an fMRI study. Campos N, Barros-Loscertales A, Martinez JC,
NeuroImage 60:1696–1703 Villanueva V et al (2010) Comparison of two
4. Hoppe C, Elger CE, Helmstaedter C (2007) fMRI tasks for the evaluation of the expressive
Long-term memory impairment in patients language function. Neuroradiology 52:407–415
with focal epilepsy. Epilepsia 48(Suppl 16. Friedman L, Kenny JT, Wise AL, Wu D, Stuve
9):26–29 TA, Miller DA, Jesberger JA et al (1998)
5. Waites AB, Briellmann RS, Saling MM, Abbott Brain activation during silent word genera-
DF, Jackson GD (2006) Functional connec- tion evaluated with functional MRI. Brain
tivity networks are disrupted in left temporal Lang 64:231–256
lobe epilepsy. Ann Neurol 59:335–343 17. Bonelli SB, Thompson PJ, Yogarajah M,
6. Vlooswijk MC, Jansen JF, Majoie HJ, Vollmar C, Powell RH, Symms MR, McEvoy
Hofman PA, de Krom MC, Aldenkamp AP, AW et al (2012) Imaging language networks
Backes WH (2010) Functional connectiv- before and after anterior temporal lobe resec-
ity and language impairment in cryptogenic tion: results of a longitudinal fMRI study.
localization-related epilepsy. Neurology Epilepsia 53:639–650
75:395–402 18. Fernandez G, Specht K, Weis S, Tendolkar
7. Baxendale S (2002) The role of functional MRI I, Reuber M, Fell J, Klaver P et al (2003)
in the presurgical investigation of temporal Intrasubject reproducibility of presurgical
lobe epilepsy patients: a clinical perspective and language lateralization and mapping using
review. J Clin Exp Neuropsychol 24:664–676 fMRI. Neurology 60:969–975
8. Kirsch HE, Walker JA, Winstanley FS, 19. Weber B, Wellmer J, Schur S, Dinkelacker V,
Hendrickson R, Wong ST, Barbaro NM, Laxer Ruhlmann J, Mormann F, Axmacher N et al
KD et al (2005) Limitations of Wada memory (2006) Presurgical language fMRI in patients
asymmetry as a predictor of outcomes after with drug-resistant epilepsy: effects of task
temporal lobectomy. Neurology 65:676–680 performance. Epilepsia 47:880–886
9. Abbott DF, Waites AB, Lillywhite LM, 20. Duncan J (2009) The current status of neu-
Jackson GD (2010) fMRI assessment of lan- roimaging for epilepsy. Curr Opin Neurol
guage lateralization: an objective approach. 22:179–184
NeuroImage 50:1446–1455 21. Rosazza C, Ghielmetti F, Minati L, Vitali P,
10. Appel S, Duke ES, Martinez AR, Khan OI, Giovagnoli AR, Deleo F, Didato G et al (2013)
Dustin IM, Reeves-Tyer P, Berl MB et al Preoperative language lateralization in tempo-
(2012) Cerebral blood flow and fMRI BOLD ral lobe epilepsy (TLE) predicts peri-ictal, pre-
auditory language activation in temporal lobe and post-operative language performance: an
epilepsy. Epilepsia 53:631–638 fMRI study. NeuroImage Clin 3:73–83
11. Arora J, Pugh K, Westerveld M, Spencer 22. Hamberger MJ, McClelland S 3rd, McKhann
S, Spencer DD, Todd Constable R (2009) GM 2nd, Williams AC, Goodman RR (2007)
Language lateralization in epilepsy patients: Distribution of auditory and visual naming
fMRI validated with the Wada procedure. sites in nonlesional temporal lobe epilepsy
Epilepsia 50:2225–2241 patients and patients with space-occupying
12. Binder JR, Gross WL, Allendorfer JB, Bonilha temporal lobe lesions. Epilepsia 48:531–538
L, Chapin J, Edwards JC, Grabowski TJ et al 23. Hermann BP, Wyler AR (1988) Effects of
(2011) Mapping anterior temporal lobe lan- anterior temporal lobectomy on language
guage areas with fMRI: a multicenter norma- function: a controlled study. Ann Neurol
tive study. NeuroImage 54:1465–1475 23:585–588
788 Rachel C. Thornton et al.

24. Hamberger MJ, Seidel WT (2009) 35. You X, Adjouadi M, Wang J, Guillen MR,
Localization of cortical dysfunction based Bernal B, Sullivan J, Donner E et al (2013) A
on auditory and visual naming performance. decisional space for fMRI pattern separation
J Int Neuropsychol Soc 15:529–535 using the principal component analysis--a com-
25. Specht K, Osnes B, Hugdahl K (2009) parative study of language networks in pediatric
Detection of differential speech-specific pro- epilepsy. Hum Brain Mapp 34:2330–2342
cesses in the temporal lobe using fMRI and a 36. Woermann FG, Jokeit H, Luerding R, Freitag
dynamic “sound morphing” technique. Hum H, Schulz R, Guertler S, Okujava M et al
Brain Mapp 30:3436–3444 (2003) Language lateralization by Wada
26. Schlosser MJ, Aoyagi N, Fulbright RK, Gore test and fMRI in 100 patients with epilepsy.
JC, McCarthy G (1998) Functional MRI Neurology 61:699–701
studies of auditory comprehension. Hum 37. Wang J, You X, Wu W, Guillen MR, Cabrerizo
Brain Mapp 6:1–13 M, Sullivan J, Donner E et al (2014)
27. Lehericy S, Cohen L, Bazin B, Samson S, Classification of fMRI patterns--a study of
Giacomini E, Rougetet R, Hertz-Pannier L the language network segregation in pediat-
et al (2000) Functional MR evaluation of ric localization related epilepsy. Hum Brain
temporal and frontal language dominance Mapp 35:1446–1460
compared with the Wada test. Neurology 38. You X, Adjouadi M, Guillen MR, Ayala M,
54:1625–1633 Barreto A, Rishe N, Sullivan J et al (2011)
28. Gaillard WD, Balsamo L, Xu B, McKinney C, Sub-patterns of language network reorganiza-
Papero PH, Weinstein S, Conry J et al (2004) tion in pediatric localization related epilepsy: a
fMRI language task panel improves determi- multisite study. Hum Brain Mapp 32:784–799
nation of language dominance. Neurology 39. Friston KJ, Price CJ, Fletcher P, Moore C,
63:1403–1408 Frackowiak RS, Dolan RJ (1996) The trou-
29. Gaillard WD, Balsamo L, Xu B, Grandin CB, ble with cognitive subtraction. NeuroImage
Braniecki SH, Papero PH, Weinstein S et al 4:97–104
(2002) Language dominance in partial epi- 40. Price CJ, Friston KJ (1997) Cognitive con-
lepsy patients identified with an fMRI reading junction: a new approach to brain activation
task. Neurology 59:256–265 experiments. NeuroImage 5:261–270
30. Adcock JE, Wise RG, Oxbury JM, Oxbury 41. Risse GL, Gates JR, Fangman MC (1997) A
SM, Matthews PM (2003) Quantitative reconsideration of bilateral language repre-
fMRI assessment of the differences in later- sentation based on the intracarotid amobar-
alization of language-related brain activa- bital procedure. Brain Cogn 33:118–132
tion in patients with temporal lobe epilepsy. 42. Serafetinides EA, Hoare RD, Driver M
NeuroImage 18:423–438 (1965) Intracarotid sodium amylobarbitone
31. Branco DM, Suarez RO, Whalen S, O’Shea and cerebral dominance for speech and con-
JP, Nelson AP, da Costa JC, Golby AJ (2006) sciousness. Brain 88:107–130
Functional MRI of memory in the hip- 43. Rasmussen T, Milner B (1977) The role of
pocampus: Laterality indices may be more early left-brain injury in determining lateral-
meaningful if calculated from whole voxel ization of cerebral speech functions. Ann N Y
distributions. NeuroImage 32:592–602 Acad Sci 299:355–369
32. Liegeois F, Connelly A, Cross JH, Boyd SG, 44. Springer JA, Binder JR, Hammeke TA,
Gadian DG, Vargha-Khadem F, Baldeweg T Swanson SJ, Frost JA, Bellgowan PS, Brewer
(2004) Language reorganization in children CC et al (1999) Language dominance in
with early-onset lesions of the left hemi- neurologically normal and epilepsy sub-
sphere: an fMRI study. Brain 127:1229–1236 jects: a functional MRI study. Brain 122(Pt
33. Karunanayaka P, Kim KK, Holland SK, 11):2033–2046
Szaflarski JP (2011) The effects of left or right 45. Janszky J, Mertens M, Janszky I, Ebner A,
hemispheric epilepsy on language networks Woermann FG (2006) Left-sided interictal
investigated with semantic decision fMRI epileptic activity induces shift of language lat-
task and independent component analysis. eralization in temporal lobe epilepsy: an fMRI
Epilepsy BehavB 20:623–632 study. Epilepsia 47:921–927
34. Mbwana J, Berl MM, Ritzl EK, Rosenberger 46. Janszky J, Jokeit H, Heinemann D, Schulz
L, Mayo J, Weinstein S, Conry JA et al (2009) R, Woermann FG, Ebner A (2003) Epileptic
Limitations to plasticity of language network activity influences the speech organiza-
reorganization in localization related epilepsy. tion in medial temporal lobe epilepsy. Brain
Brain 132:347–356 126:2043–2051
 fMRI in Epilepsy 789

47. Thivard L, Hombrouck J, du Montcel ST, 58. Janecek JK, Swanson SJ, Sabsevitz DS,
Delmaire C, Cohen L, Samson S, Dupont S Hammeke TA, Raghavan M, E Rozman M
et al (2005) Productive and perceptive lan- M, Binder JR (2013) Language lateralization
guage reorganization in temporal lobe epi- by fMRI and Wada testing in 229 patients
lepsy. NeuroImage 24:841–851 with epilepsy: rates and predictors of discor-
48. Berl MM, Balsamo LM, Xu B, Moore EN, dance. Epilepsia 54:314–322
Weinstein SL, Conry JA, Pearl PL et al 59. Benke T, Koylu B, Visani P, Karner E,
(2005) Seizure focus affects regional lan- Brenneis C, Bartha L, Trinka E et al (2006)
guage networks assessed by fMRI. Neurology Language lateralization in temporal lobe epi-
65:1604–1611 lepsy: a comparison between fMRI and the
49. Duke ES, Tesfaye M, Berl MM, Walker Wada Test. Epilepsia 47:1308–1319
JE, Ritzl EK, Fasano RE, Conry JA et al 60. Jayakar P, Bernal B, Santiago Medina L,
(2012) The effect of seizure focus on Altman N (2002) False lateralization of lan-
regional language processing areas. Epilepsia guage cortex on functional MRI after a clus-
53:1044–1050 ter of focal seizures. Neurology 58:490–492
50. Wilke M, Pieper T, Lindner K, Dushe T, 61. Wagner K, Hader C, Metternich B, Buschmann
Staudt M, Grodd W, Holthausen H et al F, Schwarzwald R, Schulze-Bonhage A (2012)
(2011) Clinical functional MRI of the lan- Who needs a Wada test? Present clinical indi-
guage domain in children with epilepsy. Hum cations for amobarbital procedures. J Neurol
Brain Mapp 32:1882–1893 Neurosurg Psychiatry 83:503–509
51. Weber B, Wellmer J, Reuber M, Mormann F, 62. Desmond JE, Sum JM, Wagner AD, Demb
Weis S, Urbach H, Ruhlmann J et al (2006) JB, Shear PK, Glover GH, Gabrieli JD et al
Left hippocampal pathology is associated with (1995) Functional MRI measurement of lan-
atypical language lateralization in patients guage lateralization in Wada-tested patients.
with focal epilepsy. Brain 129:346–351 Brain 118(Pt 6):1411–1419
52. Briellmann RS, Labate A, Harvey AS, Saling 63. Binder JR, Swanson SJ, Hammeke TA, Morris
MM, Sveller C, Lillywhite L, Abbott DF et al GL, Mueller WM, Fischer M, Benbadis S et al
(2006) Is language lateralization in tem- (1996) Determination of language domi-
poral lobe epilepsy patients related to the nance using functional MRI: a comparison
nature of the epileptogenic lesion? Epilepsia with the Wada test. Neurology 46:978–984
47:916–920 64. Hertz-Pannier L, Gaillard WD, Mott SH,
53. Jensen EJ, Hargreaves IS, Pexman PM, Cuenod CA, Bookheimer SY, Weinstein S,
Bass A, Goodyear BG, Federico P (2011) Conry J et al (1997) Noninvasive assessment
Abnormalities of lexical and semantic process- of language dominance in children and ado-
ing in left temporal lobe epilepsy: an fMRI lescents with functional MRI: a preliminary
study. Epilepsia 52:2013–2021 study. Neurology 48:1003–1012
54. Fakhri M, Oghabian MA, Vedaei F, Zandieh 65. Yetkin FZ, Swanson S, Fischer M, Akansel G,
A, Masoom N, Sharifi G, Ghodsi M et al Morris G, Mueller W, Haughton V (1998)
(2013) Atypical language lateralization: an Functional MR of frontal lobe activation:
fMRI study in patients with cerebral lesions. comparison with Wada language results.
Funct Neurol 28:55–61 AJNR Am J Neuroradiol 19:1095–1098
55. Wellmer J, Weber B, Urbach H, Reul J, 66. Benson RR, FitzGerald DB, LeSueur LL,
Fernandez G, Elger CE (2009) Cerebral Kennedy DN, Kwong KK, Buchbinder BR, Davis
lesions can impair fMRI-based language lat- TL et al (1999) Language dominance deter-
eralization. Epilepsia 50:2213–2224 mined by whole brain functional MRI in patients
56. Sanjuan A, Bustamante JC, Garcia-Porcar with brain lesions. Neurology 52:798–809
M, Rodriguez-Pujadas A, Forn C, Martinez 67. Carpentier A, Pugh KR, Westerveld M,
JC, Campos A et al (2013) Bilateral inferior Studholme C, Skrinjar O, Thompson JL,
frontal language-related activation correlates Spencer DD et al (2001) Functional MRI of
with verbal recall in patients with left tempo- language processing: dependence on input
ral lobe epilepsy and typical language distribu- modality and temporal lobe epilepsy. Epilepsia
tion. Epilepsy Res 104:118–124 42:1241–1254
57. Everts R, Harvey AS, Lillywhite L, Wrennall J, 68. Sabbah P, Chassoux F, Leveque C, Landre E,
Abbott DF, Gonzalez L, Kean M et al (2010) Baudoin-Chial S, Devaux B, Mann M et al
Language lateralization correlates with verbal (2003) Functional MR imaging in assessment
memory performance in children with focal of language dominance in epileptic patients.
epilepsy. Epilepsia 51:627–638 NeuroImage 18:460–467
790 Rachel C. Thornton et al.

69. FitzGerald DB, Cosgrove GR, Ronner S, in language processing following left hemi-
Jiang H, Buchbinder BR, Belliveau JW, Rosen sphere injury. Brain 129:754–766
BR et al (1997) Location of language in the 81. Powell HW, Parker GJ, Alexander DC,
cortex: a comparison between functional Symms MR, Boulby PA, Wheeler-Kingshott
MR imaging and electrocortical stimulation. CA, Barker GJ et al (2006) Hemispheric
AJNR Am J Neuroradiol 18:1529–1539 asymmetries in language-related pathways: a
70. Schlosser MJ, Luby M, Spencer DD, Awad combined functional MRI and tractography
IA, McCarthy G (1999) Comparative local- study. NeuroImage 32:388–399
ization of auditory comprehension by using 82. Powell HW, Parker GJ, Alexander DC,
functional magnetic resonance imaging and Symms MR, Boulby PA, Wheeler-Kingshott
cortical stimulation. J Neurosurg 91:626–635 CA, Barker GJ et al (2007) Abnormalities of
71. Pouratian N, Bookheimer SY, Rex DE, Martin language networks in temporal lobe epilepsy.
NA, Toga AW (2002) Utility of preoperative NeuroImage 36:209–221
functional magnetic resonance imaging for 83. Stretton J, Thompson PJ (2012) Frontal lobe
identifying language cortices in patients with function in temporal lobe epilepsy. Epilepsy
vascular malformations. J Neurosurg 97:21–32 Res 98:1–13
72. Rutten GJ, Ramsey NF, van Rijen PC, 84. Centeno M, Thompson PJ, Koepp MJ,
Noordmans HJ, van Veelen CW (2002) Helmstaedter C, Duncan JS (2010) Memory in
Development of a functional magnetic reso- frontal lobe epilepsy. Epilepsy Res 91:123–132
nance imaging protocol for intraoperative 85. Bonelli SB, Thompson PJ, Yogarajah M,
localization of critical temporoparietal lan- Powell RH, Samson RS, McEvoy AW, Symms
guage areas. Ann Neurol 51:350–360 MR et al (2013) Memory reorganization
73. Davies KG, Bell BD, Bush AJ, Hermann following anterior temporal lobe resection:
BP, Dohan FC Jr, Jaap AS (1998) Naming a longitudinal functional MRI study. Brain
decline after left anterior temporal lobectomy 136:1889–1900
correlates with pathological status of resected 86. Scoville WB, Milner B (2000) Loss of recent
hippocampus. Epilepsia 39:407–419 memory after bilateral hippocampal lesions.
74. Saykin AJ, Stafiniak P, Robinson LJ, Flannery 1957. J Neuropsychiatry Clin Neurosci
KA, Gur RC, O’Connor MJ, Sperling MR 12:103–113
(1995) Language before and after temporal 87. Ivnik RJ, Sharbrough FW, Laws ER Jr (1987)
lobectomy: specificity of acute changes and rela- Effects of anterior temporal lobectomy on
tion to early risk factors. Epilepsia 36:1071–1077 cognitive function. J Clin Psychol 43:128–137
75. Hermann BP, Perrine K, Chelune GJ, Barr 88. Spiers HJ, Burgess N, Maguire EA, Baxendale
W, Loring DW, Strauss E, Trenerry MR SA, Hartley T, Thompson PJ, O’Keefe
et al (1999) Visual confrontation nam- J (2001) Unilateral temporal lobectomy
ing following left anterior temporal lobec- patients show lateralized topographical and
tomy: a comparison of surgical approaches. episodic memory deficits in a virtual town.
Neuropsychology 13:3–9 Brain 124:2476–2489
76. Devinsky O, Perrine K, Llinas R, Luciano DJ, 89. Penfield W, Milner B (1958) Memory defi-
Dogali M (1993) Anterior temporal language cit produced by bilateral lesions in the hip-
areas in patients with early onset of temporal pocampal zone. AMA Arch Neurol Psychiatry
lobe epilepsy. Ann Neurol 34:727–732 79:475–497
77. Schwartz TH, Devinsky O, Doyle W, Perrine 90. Warrington EK, Duchen LW (1992) A re-
K (1998) Preoperative predictors of ante- appraisal of a case of persistent global amne-
rior temporal language areas. J Neurosurg sia following right temporal lobectomy: a
89:962–970 clinico-pathological study. Neuropsychologia
78. Sabsevitz DS, Swanson SJ, Hammeke TA, 30:437–450
Spanaki MV, Possing ET, Morris GL 3rd, 91. Loring DW, Hermann BP, Meador KJ, Lee
Mueller WM et al (2003) Use of preoperative GP, Gallagher BB, King DW, Murro AM et al
functional neuroimaging to predict language (1994) Amnesia after unilateral temporal lobec-
deficits from epilepsy surgery. Neurology tomy: a case report. Epilepsia 35:757–763
60:1788–1792
92. Alessio A, Pereira FR, Sercheli MS, Rondina
79. Noppeney U, Price CJ, Duncan JS, Koepp JM, Ozelo HB, Bilevicius E, Pedro T et al
MJ (2005) Reading skills after left anterior (2013) Brain plasticity for verbal and visual
temporal lobe resection: an fMRI study. Brain memories in patients with mesial temporal
128:1377–1385 lobe epilepsy and hippocampal sclerosis: an
80. Voets NL, Adcock JE, Flitney DE, Behrens fMRI study. Hum Brain Mapp 34:186–199
TE, Hart Y, Stacey R, Carpenter K et al 93. Banks SJ, Sziklas V, Sodums DJ, Jones-Gotman
(2006) Distinct right frontal lobe activation M (2012) fMRI of verbal and nonverbal mem-
 fMRI in Epilepsy 791

ory processes in healthy and epileptogenic medial 105. McCormick C, Quraan M, Cohn M, Valiante
temporal lobes. Epilepsy Behav 25:42–49 TA, McAndrews MP (2013) Default mode
94. Chelune GJ (1995) Hippocampal adequacy network connectivity indicates episodic mem-
versus functional reserve: predicting memory ory capacity in mesial temporal lobe epilepsy.
functions following temporal lobectomy. Epilepsia 54:809–818
Arch Clin Neuropsychol 10:413–432 106. Jokeit H, Ebner A, Holthausen H,
95. Chelune GJ, Naugle RI, Luders H, Awad IA Markowitsch HJ, Moch A, Pannek H,
(1991) Prediction of cognitive change as a Schulz R et al (1997) Individual prediction
function of preoperative ability status among of change in delayed recall of prose passages
temporal lobectomy patients seen at 6-month after left-sided anterior temporal lobectomy.
follow-up. Neurology 41:399–404 Neurology 49:481–487
96. Kneebone AC, Chelune GJ, Dinner DS, 107. Helmstaedter C, Elger CE (1996) Cognitive conse-
Naugle RI, Awad IA (1995) Intracarotid quences of two-thirds anterior temporal lobectomy
amobarbital procedure as a predictor of on verbal memory in 144 patients: a three-month
material-specific memory change after anterior follow-up study. Epilepsia 37:171–180
temporal lobectomy. Epilepsia 36:857–865 108. Cheung MC, Chan AS, Lam JM, Chan YL
97. Sass KJ, Spencer DD, Kim JH, Westerveld M, (2009) Pre- and postoperative fMRI and
Novelly RA, Lencz T (1990) Verbal memory clinical memory performance in temporal
impairment correlates with hippocampal pyra- lobe epilepsy. J Neurol Neurosurg Psychiatry
midal cell density. Neurology 40:1694–1697 80:1099–1106
98. Trenerry MR, Jack CR Jr, Ivnik RJ, 109. Corkin S, Amaral DG, Gonzalez RG, Johnson
Sharbrough FW, Cascino GD, Hirschorn KA, Hyman BT (1997) H. M’.s medial tem-
KA, Marsh WR et al (1993) MRI hippocam- poral lobe lesion: findings from magnetic res-
pal volumes and memory function before onance imaging. J Neurosci 17:3964–3979
and after temporal lobectomy. Neurology 110. Fernandez G, Effern A, Grunwald T, Pezer
43:1800–1805 N, Lehnertz K, Dumpelmann M, Van Roost
99. Guedj E, Bettus G, Barbeau EJ, Liegeois- D et al (1999) Real-time tracking of memory
Chauvel C, Confort-Gouny S, Bartolomei formation in the human rhinal cortex and
F, Chauvel P et al (2011) Hyperactivation of hippocampus. Science 285:1582–1585
parahippocampal region and fusiform gyrus 111. Ojemann JG, Akbudak E, Snyder AZ,
associated with successful encoding in medial McKinstry RC, Raichle ME, Conturo TE
temporal lobe epilepsy. Epilepsia 52:1100–1109 (1997) Anatomic localization and quantitative
100. Sidhu MK, Stretton J, Winston GP, Bonelli analysis of gradient refocused echo-planar fMRI
S, Centeno M, Vollmar C, Symms M et al susceptibility artifacts. NeuroImage 6:156–167
(2013) A functional magnetic resonance 112. Greicius MD, Krasnow B, Boyett-Anderson
imaging study mapping the episodic memory JM, Eliez S, Schatzberg AF, Reiss AL,
encoding network in temporal lobe epilepsy. Menon V (2003) Regional analysis of hip-
Brain 136:1868–1888 pocampal activation during memory encod-
101. Bonelli SB, Powell RH, Yogarajah M, Samson ing and retrieval: fMRI study. Hippocampus
RS, Symms MR, Thompson PJ, Koepp MJ 13:164–174
et al (2010) Imaging memory in temporal 113. Lipschutz B, Friston KJ, Ashburner J,
lobe epilepsy: predicting the effects of tempo- Turner R, Price CJ (2001) Assessing study-
ral lobe resection. Brain 133:1186–1199 specific regional variations in fMRI signal.
102. Sidhu MK, Stretton J, Winston GP, Symms NeuroImage 13:392–398
M, Thompson PJ, Koepp MJ, Duncan JS 114. Craik FIM, Lockhart RS (1972) Levels of pro-
(2015) Memory fMRI predicts verbal mem- cessing: A framework for memory research.
ory decline after anterior temporal lobe resec- J Verbal Learn Verbal Behav 11:671–684
tion. Neurology 84:1512–1519 115. Kelley WM, Miezin FM, McDermott KB,
103. Hermann BP, Wyler AR, Somes G, Berry AD Buckner RL, Raichle ME, Cohen NJ, Ollinger
3rd, Dohan FC Jr (1992) Pathological status JM et al (1998) Hemispheric specialization in
of the mesial temporal lobe predicts memory human dorsal frontal cortex and medial tem-
outcome from left anterior temporal lobectomy. poral lobe for verbal and nonverbal memory
Neurosurgery 31:652–656, discussion 656-657 encoding. Neuron 20:927–936
104. Sass KJ, Westerveld M, Buchanan CP, Spencer 116. Demb JB, Desmond JE, Wagner AD, Vaidya
SS, Kim JH, Spencer DD (1994) Degree of CJ, Glover GH, Gabrieli JD (1995) Semantic
hippocampal neuron loss determines sever- encoding and retrieval in the left inferior pre-
ity of verbal memory decrease after left frontal cortex: a functional MRI study of task
anteromesiotemporal lobectomy. Epilepsia difficulty and process specificity. J Neurosci
35:1179–1186 15:5870–5878
792 Rachel C. Thornton et al.

117. Wagner AD, Schacter DL, Rotte M, Koutstaal to unilateral hippocampal sclerosis. Epilepsia
W, Maril A, Dale AM, Rosen BR et al (1998) 48:1512–1525
Building memories: remembering and for- 130. Richardson MP, Strange BA, Thompson PJ,
getting of verbal experiences as predicted by Baxendale SA, Duncan JS, Dolan RJ (2004)
brain activity. Science 281:1188–1191 Pre-operative verbal memory fMRI predicts
118. Buckner RL, Kelley WM, Petersen SE (1999) post-operative memory decline after left tem-
Frontal cortex contributes to human memory poral lobe resection. Brain 127:2419–2426
formation. Nat Neurosci 2:311–314 131. Richardson MP, Strange BA, Duncan JS,
119. Golby AJ, Poldrack RA, Brewer JB, Spencer Dolan RJ (2006) Memory fMRI in left hippo-
D, Desmond JE, Aron AP, Gabrieli JD (2001) campal sclerosis: optimizing the approach to
Material-specific lateralization in the medial predicting postsurgical memory. Neurology
temporal lobe and prefrontal cortex during 66:699–705
memory encoding. Brain 124:1841–1854 132. Powell HW, Richardson MP, Symms MR,
120. Wagner AD, Koutstaal W, Schacter DL (1999) Boulby PA, Thompson PJ, Duncan JS, Koepp
When encoding yields remembering: insights MJ (2008) Preoperative fMRI predicts memory
from event-related neuroimaging. Phil Trans decline following anterior temporal lobe resec-
Roy Soc Lond B Biol Sci 354:1307–1324 tion. J Neurol Neurosurg Psychiatry 79:686–693
121. Powell HW, Koepp MJ, Symms MR, Boulby 133. Rabin ML, Narayan VM, Kimberg DY,
PA, Salek-Haddadi A, Thompson PJ, Duncan Casasanto DJ, Glosser G, Tracy JI, French JA
JS et al (2005) Material-specific lateralization et al (2004) Functional MRI predicts post-
of memory encoding in the medial temporal surgical memory following temporal lobec-
lobe: blocked versus event-related design. tomy. Brain 127:2286–2298
NeuroImage 27:231–239 134. Janszky J, Jokeit H, Kontopoulou K, Mertens
122. Detre JA, Maccotta L, King D, Alsop DC, M, Ebner A, Pohlmann-Eden B, Woermann
Glosser G, D’Esposito M, Zarahn E et al (1998) FG (2005) Functional MRI predicts memory
Functional MRI lateralization of memory in performance after right mesiotemporal epi-
temporal lobe epilepsy. Neurology 50:926–932 lepsy surgery. Epilepsia 46:244–250
123. Doucet G, Osipowicz K, Sharan A, Sperling 135. Price CJ, Friston KJ (1999) Scanning patients
MR, Tracy JI (2013) Hippocampal functional with tasks they can perform. Hum Brain
connectivity patterns during spatial working Mapp 8:102–108
memory differ in right versus left temporal 136. von Helmholtz HLF (2004) Some laws con-
lobe epilepsy. Brain Connect 3:398–406 cerning the distribution of electric currents
124. Stretton J, Winston GP, Sidhu M, Bonelli in volume conductors with applications to
S, Centeno M, Vollmar C, Cleary RA et al experiments on animal electricity. Proc IEEE
(2013) Disrupted segregation of working 92:868–870
memory networks in temporal lobe epilepsy. 137. Geselowitz DB (2004) Introduction to
NeuroImage Clin 2:273–281 “some laws concerning the distribution of
125. Bellgowan PS, Binder JR, Swanson SJ, electric currents in volume conductors with
Hammeke TA, Springer JA, Frost JA, Mueller applications to experiments on animal elec-
WM et al (1998) Side of seizure focus pre- tricity”. Proc IEEE 92:864–867
dicts left medial temporal lobe activation dur- 138. Ives JR, Warach S, Schmitt F, Edelman
ing verbal encoding. Neurology 51:479–484 RR, Schomer DL (1993) Monitoring
126. Jokeit H, Okujava M, Woermann FG (2001) the patient’s EEG during echo planar
Memory fMRI lateralizes temporal lobe epi- MRI. Electroencephalogr Clin Neurophysiol
lepsy. Neurology 57:1786–1793 87:417–420
127. Golby AJ, Poldrack RA, Illes J, Chen D, 139. Detre JA, Alsop DC, Aguirre GK, Sperling
Desmond JE, Gabrieli JD (2002) Memory later- MR (1996) Coupling of cortical and thalamic
alization in medial temporal lobe epilepsy assessed ictal activity in human partial epilepsy: dem-
by functional MRI. Epilepsia 43:855–863 onstration by functional magnetic resonance
128. Richardson MP, Strange BA, Duncan JS, Dolan imaging. Epilepsia 37:657–661
RJ (2003) Preserved verbal memory function in 140. Krings T, Topper R, Reinges MH, Foltys
left medial temporal pathology involves reorgan- H, Spetzger U, Chiappa KH, Gilsbach JM
isation of function to right medial temporal lobe. et al (2000) Hemodynamic changes in sim-
NeuroImage 20(Suppl 1):S112–S119 ple partial epilepsy: a functional MRI study.
129. Powell HW, Richardson MP, Symms MR, Neurology 54:524–527
Boulby PA, Thompson PJ, Duncan JS, Koepp 141. Connelly A (1995) Ictal imaging using func-
MJ (2007) Reorganization of verbal and non- tional magnetic resonance. Magn Reson
verbal memory in temporal lobe epilepsy due Imaging 13:1233–1237
 fMRI in Epilepsy 793

142. Jackson GD, Connelly A, Cross JH, Gordon pulse artifact and a method for its subtraction.
I, Gadian DG (1994) Functional mag- NeuroImage 8:229–239
netic resonance imaging of focal seizures. 155. Wendt RE 3rd, Rokey R, Vick GW 3rd,
Neurology 44:850–856 Johnston DL (1988) Electrocardiographic
143. Jorge J, Grouiller F, Ipek O, Stoermer R, gating and monitoring in NMR imaging.
Michel CM, Figueiredo P, van der Zwaag Magn Reson Imaging 6:89–95
W et al (2015) Simultaneous EEG-fMRI at 156. Poncelet BP, Wedeen VJ, Weisskoff RM,
ultra-high field: artifact prevention and safety Cohen MS (1992) Brain parenchyma motion:
assessment. NeuroImage 105:132–144 measurement with cine echo-planar MR
144. Arrubla J, Neuner I, Dammers J, Breuer L, imaging. Radiology 185:645–651
Warbrick T, Hahn D, Poole MS et al (2014) 157. Tenforde TS, Gaffey CT, Moyer BR, Budinger
Methods for pulse artefact reduction: expe- TF (1983) Cardiovascular alterations in
riences with EEG data recorded at 9.4 T Macaca monkeys exposed to stationary mag-
static magnetic field. J Neurosci Methods netic fields: experimental observations and
232:110–117 theoretical analysis. Bioelectromagnetics 4:1–9
145. Lemieux L, Allen PJ, Franconi F, Symms 158. Goldman RI, Stern JM, Engel J Jr, Cohen
MR, Fish DR (1997) Recording of EEG dur- MS (2000) Acquiring simultaneous EEG
ing fMRI experiments: patient safety. Magn and functional MRI. Clin Neurophysiol
Reson Med 38:943–952 111:1974–1980
146. Mirsattari SM, Lee DH, Jones D, Bihari F, 159. Benar C, Aghakhani Y, Wang Y, Izenberg
Ives JR (2004) MRI compatible EEG elec- A, Al-Asmi A, Dubeau F, Gotman J (2003)
trode system for routine use in the epilepsy Quality of EEG in simultaneous EEG-fMRI
monitoring unit and intensive care unit. Clin for epilepsy. Clin Neurophysiol 114:569–580
Neurophysiol 115:2175–2180 160. Chowdhury ME, Mullinger KJ, Glover
147. Konings MK, Bartels LW, Smits HF, Bakker P, Bowtell R (2014) Reference layer arte-
CJ (2000) Heating around intravascular fact subtraction (RLAS): a novel method of
guidewires by resonating RF waves. J Magn minimizing EEG artefacts during simultane-
Reson Imaging 12:79–85 ous fMRI. NeuroImage 84:307–319
148. Fischer H, Ladebeck R (1998) Echo-planar 161. Salek-Haddadi A, Lemieux L, Merschhemke
imaging image artifacts. In: Schmitt F, M, Diehl B, Allen PJ, Fish DR (2003) EEG
Stehling MK, Turner R (eds) Echo-planar quality during simultaneous functional MRI
imaging: theory, technique, and application. of interictal epileptiform discharges. Magn
Springer, Berlin, pp 179–200 Reson Imaging 21:1159–1166
149. Krakow K, Allen PJ, Symms MR, Lemieux 162. Liston AD, De Munck JC, Hamandi K, Laufs
L, Josephs O, Fish DR (2000) EEG record- H, Ossenblok P, Duncan JS, Lemieux L (2006)
ing during fMRI experiments: image quality. Analysis of EEG-fMRI data in focal epilepsy
Hum Brain Mapp 10:10–15 based on automated spike classification and Signal
150. Bonmassar G, Anami K, Ives J, Belliveau JW Space Projection. NeuroImage 31:1015–1024
(1999) Visual evoked potential (VEP) mea- 163. Allen PJ, Josephs O, Turner R (2000) A
sured by simultaneous 64-channel EEG and method for removing imaging artifact from
3T fMRI. Neuroreport 10:1893–1897 continuous EEG recorded during functional
151. Bonmassar G, Purdon PL, Jaaskelainen IP, MRI. NeuroImage 12:230–239
Chiappa K, Solo V, Brown EN, Belliveau JW 164. Lemieux L, Salek-Haddadi A, Lund TE, Laufs
(2002) Motion and ballistocardiogram artifact H, Carmichael D (2007) Modelling large
removal for interleaved recording of EEG and motion events in fMRI studies of patients with
EPs during MRI. NeuroImage 16:1127–1141 epilepsy. Magn Reson Imaging 25:894–901
152. Scarff CJ, Reynolds A, Goodyear BG, 165. Liston AD, Lund TE, Salek-Haddadi A,
Ponton CW, Dort JC, Eggermont JJ (2004) Hamandi K, Friston KJ, Lemieux L (2006)
Simultaneous 3-T fMRI and high-density Modelling cardiac signal as a confound in
recording of human auditory evoked poten- EEG-fMRI and its application in focal epi-
tials. NeuroImage 23:1129–1142 lepsy studies. NeuroImage 30:827–834
153. Klein C, Hanggi J, Luechinger R, Jancke L 166. Ellingson ML, Liebenthal E, Spanaki MV,
(2015) MRI with and without a high-den- Prieto TE, Binder JR, Ropella KM (2004)
sity EEG cap--what makes the difference? Ballistocardiogram artifact reduction in the
NeuroImage 106:189–197 simultaneous acquisition of auditory ERPS
154. Allen PJ, Polizzi G, Krakow K, Fish DR, and fMRI. NeuroImage 22:1534–1542
Lemieux L (1998) Identification of EEG 167. Kruggel F, Wiggins CJ, Herrmann CS, von
events in the MR scanner: the problem of Cramon DY (2000) Recording of the event-
794 Rachel C. Thornton et al.

related potentials during functional MRI at gram artifact removing. Magn Reson Imaging
3.0 Tesla field strength. Magn Reson Med 24:393–400
44:277–282 179. Mantini D, Perrucci MG, Cugini S, Ferretti A,
168. Sijbersa J, Van Audekerke J, Verhoye M, Van Romani GL, Del Gratta C (2007) Complete
der Linden A, Van Dyck D (2000) Reduction artifact removal for EEG recorded during
of ECG and gradient related artifacts in continuous fMRI using independent compo-
simultaneously recorded human EEG/MRI nent analysis. NeuroImage 34:598–607
data. Magn Reson Imaging 18:881–886 180. Maggioni E, Arrubla J, Warbrick T, Dammers
169. Kim KH, Yoon HW, Park HW (2004) J, Bianchi AM, Reni G, Tosetti M et al (2014)
Improved ballistocardiac artifact removal Removal of pulse artefact from EEG data
from the electroencephalogram recorded in recorded in MR environment at 3T. Setting
fMRI. J Neurosci Methods 135:193–203 of ICA parameters for marking artefactual
170. Wan X, Iwata K, Riera J, Ozaki T, Kitamura components: application to resting-state data.
M, Kawashima R (2006) Artifact reduction PLoS One 9, e112147
for EEG/fMRI recording: nonlinear reduc- 181. Debener S, Strobel A, Sorger B, Peters J,
tion of ballistocardiogram artifacts. Clin Kranczioch C, Engel AK, Goebel R (2007)
Neurophysiol 117:668–680 Improved quality of auditory event-related
171. Niazy RK, Beckmann CF, Iannetti GD, Brady potentials recorded simultaneously with 3-T
JM, Smith SM (2005) Removal of FMRI fMRI: removal of the ballistocardiogram arte-
environment artifacts from EEG data using fact. NeuroImage 34:587–597
optimal basis sets. NeuroImage 28:720–737 182. Harrison AH, Noseworthy MD, Reilly JP,
172. In MH, Lee SY, Park TS, Kim TS, Cho MH, Connolly JF (2014) Ballistocardiogram
Ahn YB (2006) Ballistocardiogram artifact correction in simultaneous EEG/ fMRI
removal from EEG signals using adaptive filter- recordings: a comparison of average artifact
ing of EOG signals. Physiol Meas 27:1227–1240 subtraction and optimal basis set methods
173. Eichele T, Specht K, Moosmann M, Jongsma using two popular software tools. Crit Rev
ML, Quiroga RQ, Nordby H, Hugdahl K Biomed Eng 42:95–107
(2005) Assessing the spatiotemporal evolution 183. Warach S, Ives JR, Schlaug G, Patel MR,
of neuronal activation with single-trial event- Darby DG, Thangaraj V, Edelman RR et al
related potentials and functional MRI. Proc (1996) EEG-triggered echo-planar functional
Natl Acad Sci U S A 102:17798–17803 MRI in epilepsy. Neurology 47:89–93
174. Otzenberger H, Gounot D, Foucher JR 184. Rothlubbers S, Relvas V, Leal A, Murta
(2005) P300 recordings during event-related T, Lemieux L, Figueiredo P (2015)
fMRI: a feasibility study. Brain Res Cogn Characterisation and reduction of the EEG
Brain Res 23:306–315 artefact caused by the helium cooling pump
175. Otzenberger H, Gounot D, Foucher JR in the MR environment: validation in epilepsy
(2007) Optimisation of a post-processing patient data. Brain Topogr 28:208–220
method to remove the pulse artifact from EEG 185. Bonmassar G, Schwartz DP, Liu AK,
data recorded during fMRI: an application to Kwong KK, Dale AM, Belliveau JW (2001)
P300 recordings during e-fMRI. Neurosci Spatiotemporal brain imaging of visual-evoked
Res 57:230–239 activity using interleaved EEG and fMRI
176. Srivastava G, Crottaz-Herbette S, Lau KM, recordings. NeuroImage 13:1035–1043
Glover GH, Menon V (2005) ICA-based 186. Huang-Hellinger FR, Breiter HC,
procedures for removing ballistocardiogram McCormack G, Cohen MS, Kwong KK,
artifacts from EEG data acquired in the MRI Sutton JP, Savoy RL et al (1995) Simultaneous
scanner. NeuroImage 24:50–60 functional magnetic resonance imaging and
177. Nakamura W, Anami K, Mori T, Saitoh O, electrophysiological recording. Hum Brain
Cichocki A, Amari S (2006) Removal of bal- Mapp 3:13–23
listocardiogram artifacts from simultaneously 187. Krakow K, Woermann FG, Symms MR, Allen
recorded EEG and fMRI data using inde- PJ, Lemieux L, Barker GJ, Duncan JS et al
pendent component analysis. IEEE Trans (1999) EEG-triggered functional MRI of
Biomed Eng 53:1294–1308 interictal epileptiform activity in patients with
178. Briselli E, Garreffa G, Bianchi L, Bianciardi partial seizures. Brain 122(Pt 9):1679–1688
M, Macaluso E, Abbafati M, Grazia Marciani 188. Krakow K, Allen PJ, Lemieux L, Symms
M et al (2006) An independent component MR, Fish DR (2000) Methodology: EEG-
analysis-based approach on ballistocardio- correlated fMRI. Adv Neurol 83:187–201
 fMRI in Epilepsy 795

189. Krakow K, Lemieux L, Messina D, Scott CA, Non-invasive epileptic focus localization using
Symms MR, Duncan JS, Fish DR (2001) EEG-triggered functional MRI and electro-
Spatio-temporal imaging of focal interictal magnetic tomography. Electroencephalogr
epileptiform activity using EEG-triggered Clin Neurophysiol 106:508–512
functional MRI. Epileptic Disord 3:67–74 201. Symms MR, Allen PJ, Woermann FG, Polizzi
190. Al-Asmi A, Benar CG, Gross DW, Khani YA, G, Krakow K, Barker GJ, Fish DR et al
Andermann F, Pike B, Dubeau F et al (2003) (1999) Reproducible localization of interictal
fMRI activation in continuous and spike- epileptiform discharges using EEG-triggered
triggered EEG-fMRI studies of epileptic fMRI. Phys Med Biol 44:N161–N168
spikes. Epilepsia 44:1328–1339 202. Patel MR, Blum A, Pearlman JD, Yousuf
191. Anami K, Mori T, Tanaka F, Kawagoe N, Ives JR, Saeteng S, Schomer DL et al
Y, Okamoto J, Yarita M, Ohnishi T et al (1999) Echo-planar functional MR imaging
(2003) Stepping stone sampling for retriev- of epilepsy with concurrent EEG monitoring.
ing artifact-free electroencephalogram dur- AJNR Am J Neuroradiol 20:1916–1919
ing functional magnetic resonance imaging. 203. Krakow K, Wieshmann UC, Woermann FG,
NeuroImage 19:281–295 Symms MR, McLean MA, Lemieux L, Allen
192. Garreffa G, Carni M, Gualniera G, Ricci GB, PJ et al (1999) Multimodal MR imaging:
Bozzao L, De Carli D, Morasso P et al (2003) functional, diffusion tensor, and chemical
Real-time MR artifacts filtering during con- shift imaging in a patient with localization-
tinuous EEG/fMRI acquisition. Magn Reson related epilepsy. Epilepsia 40:1459–1462
Imaging 21:1175–1189 204. Lazeyras F, Blanke O, Perrig S, Zimine I,
193. Mullinger KJ, Yan WX, Bowtell R (2011) Golay X, Delavelle J, Michel CM et al (2000)
Reducing the gradient artefact in simultane- EEG-triggered functional MRI in patients
ous EEG-fMRI by adjusting the subject’s with pharmacoresistant epilepsy. J Magn
axial position. NeuroImage 54:1942–1950 Reson Imaging 12:177–185
194. Hoffmann A, Jager L, Werhahn KJ, 205. Lazeyras F, Blanke O, Zimine I, Delavelle
Jaschke M, Noachtar S, Reiser M (2000) J, Perrig SH, Seeck M (2000) MRI, (1)
Electroencephalography during functional H-MRS, and functional MRI during and
echo-planar imaging: detection of epileptic after prolonged nonconvulsive seizure activ-
spikes using post-processing methods. Magn ity. Neurology 55:1677–1682
Reson Med 44:791–798 206. Jager L, Werhahn KJ, Hoffmann A, Berthold
195. Sijbers J, Michiels I, Verhoye M, Van S, Scholz V, Weber J, Noachtar S et al (2002)
Audekerke J, Van der Linden A, Van Dyck D Focal epileptiform activity in the brain: detec-
(1999) Restoration of MR-induced artifacts tion with spike-related functional MR imaging-
in simultaneously recorded MR/EEG data. -preliminary results. Radiology 223:860–869
Magn Reson Imaging 17:1383–1391 207. Benar CG, Gross DW, Wang Y, Petre V, Pike
196. Hamandi K, Salek-Haddadi A, Laufs H, B, Dubeau F, Gotman J (2002) The BOLD
Liston A, Friston K, Fish DR, Duncan JS response to interictal epileptiform discharges.
et al (2006) EEG-fMRI of idiopathic and sec- NeuroImage 17:1182–1192
ondarily generalized epilepsies. NeuroImage 208. Benar CG, Grova C, Kobayashi E, Bagshaw
31:1700–1710 AP, Aghakhani Y, Dubeau F, Gotman J (2006)
197. Lemieux L, Salek-Haddadi A, Josephs O, Allen EEG-fMRI of epileptic spikes: concordance
P, Toms N, Scott C, Krakow K et al (2001) with EEG source localization and intracranial
Event-related fMRI with simultaneous and con- EEG. NeuroImage 30:1161–1170
tinuous EEG: description of the method and 209. Aghakhani Y, Kobayashi E, Bagshaw AP, Hawco
initial case report. NeuroImage 14:780–787 C, Benar CG, Dubeau F, Gotman J (2006)
198. Salek-Haddadi A, Merschhemke M, Lemieux Cortical and thalamic fMRI responses in partial
L, Fish DR (2002) Simultaneous EEG- epilepsy with focal and bilateral synchronous
correlated Ictal fMRI. NeuroImage 16:32–40 spikes. Clin Neurophysiol 117:177–191
199. Salek-Haddadi A, Diehl B, Hamandi K, 210. Zijlmans M, Huiskamp G, Hersevoort
Merschhemke M, Liston A, Friston K, M, Seppenwoolde JH, van Huffelen AC,
Duncan JS et al (2006) Hemodynamic cor- Leijten FS (2007) EEG-fMRI in the preop-
relates of epileptiform discharges: an EEG- erative work-up for epilepsy surgery. Brain
fMRI study of 63 patients with focal epilepsy. 130:2343–2353
Brain Res 1088:148–166 211. Thornton R, Vulliemoz S, Rodionov R,
200. Seeck M, Lazeyras F, Michel CM, Blanke O, Carmichael DW, Chaudhary UJ, Diehl B,
Gericke CA, Ives J, Delavelle J et al (1998) Laufs H et al (2011) Epileptic networks
796 Rachel C. Thornton et al.

in focal cortical dysplasia revealed using EEG-fMRI in children with pharmacoresis-

electroencephalography-functional magnetic tant focal epilepsy. Epilepsia 48:385–389
resonance imaging. Ann Neurol 70:822–837 223. Jacobs J, Jacobs J, Boor R, Jansen O, Wolff S,
212. Pittau F, Dubeau F, Gotman J (2012) Siniatchkin M, Stephani U (2007) Localization
Contribution of EEG/fMRI to the definition of of epileptic foci in children with focal epilepsies
the epileptic focus. Neurology 78:1479–1487 using 3-Tesla simultaneous EEG-fMRI record-
213. Chaudhary UJ, Carmichael DW, Rodionov ings. Clin Neurophysiol 118:e50–e51
R, Thornton RC, Bartlett P, Vulliemoz S, 224. Jacobs J, Kobayashi E, Boor R, Muhle H,
Micallef C et al (2012) Mapping preictal and Stephan W, Hawco C, Dubeau F et al (2007)
ictal haemodynamic networks using video- Hemodynamic responses to interictal epilep-
electroencephalography and functional imag- tiform discharges in children with symptom-
ing. Brain 135:3645–3663 atic epilepsy. Epilepsia 48:2068–2078
214. Coan AC, Chaudhary UJ, Grouiller F, Campos 225. Mandelkow H, Halder P, Boesiger P, Brandeis
BM, Perani S, De Ciantis A, Vulliemoz S, D (2006) Synchronization facilitates removal
et al. (2015) EEG-fMRI in the pre-surgical of MRI artefacts from concurrent EEG
evaluation of temporal lobe epilepsy. J Neurol recordings and increases usable bandwidth.
Neurosurg Psychiatry (in press) NeuroImage 32:1120–1126
215. Grouiller F, Thornton RC, Groening K, Spinelli 226. Solana AB, Hernandez-Tamames JA,
L, Duncan JS, Schaller K, Siniatchkin M et al Manzanedo E, Garcia-Alvarez R, Zelaya FO,
(2011) With or without spikes: localization of del Pozo F (2014) Gradient induced arti-
focal epileptic activity by simultaneous electro- facts in simultaneous EEG-fMRI: Effect of
encephalography and functional magnetic reso- synchronization on spiral and EPI k-space tra-
nance imaging. Brain 134:2867–2886 jectories. Magn Reson Imaging 32:684–692
216. Tousseyn S, Dupont P, Goffin K, Sunaert S, 227. Negishi M, Abildgaard M, Nixon T, Constable
Van Paesschen W (2015) Correspondence RT (2004) Removal of time-varying gradient
between large-scale ictal and interictal epilep- artifacts from EEG data acquired during continu-
tic networks revealed by single photon emis- ous fMRI. Clin Neurophysiol 115:2181–2192
sion computed tomography (SPECT) and 228. Wan X, Iwata K, Riera J, Kitamura M,
electroencephalography (EEG)-functional Kawashima R (2006) Artifact reduction for
magnetic resonance imaging (fMRI). simultaneous EEG/fMRI recording: adaptive
Epilepsia 56:382–392 FIR reduction of imaging artifacts. Clin
217. Salek-Haddadi A, Lemieux L, Merschhemke Neurophysiol 117:681–692
M, Friston KJ, Duncan JS, Fish DR (2003) 229. Ritter P, Becker R, Graefe C, Villringer A
Functional magnetic resonance imaging (2007) Evaluating gradient artifact correction
of human absence seizures. Ann Neurol of EEG data acquired simultaneously with
53:663–667 fMRI. Magn Reson Imaging 25:923–932
218. Baudewig J, Bittermann HJ, Paulus W, Frahm 230. Masterton RA, Abbott DF, Fleming SW,
J (2001) Simultaneous EEG and functional Jackson GD (2007) Measurement and reduc-
MRI of epileptic activity: a case report. Clin tion of motion and ballistocardiogram arte-
Neurophysiol 112:1196–1200 facts from simultaneous EEG and fMRI
219. Aghakhani Y, Bagshaw AP, Benar CG, recordings. NeuroImage 37:202–211
Hawco C, Andermann F, Dubeau F, Gotman 231. Abbott DF, Masterton RA, Archer JS,
J (2004) fMRI activation during spike and Fleming SW, Warren AE, Jackson GD (2014)
wave discharges in idiopathic generalized epi- Constructing carbon fiber motion-detection
lepsy. Brain 127:1127–1144 loops for simultaneous EEG-fMRI. Front
220. Yang T, Luo C, Li Q, Guo Z, Liu L, Gong Neurol 5:260
Q, Yao D et al (2013) Altered resting-state 232. Friston KJ, Williams S, Howard R, Frackowiak
connectivity during interictal generalized RS, Turner R (1996) Movement-related
spike-wave discharges in drug-naive child- effects in fMRI time-series. Magn Reson Med
hood absence epilepsy. Hum Brain Mapp 35:346–355
34:1761–1767 233. Hajnal JV, Myers R, Oatridge A, Schwieso JE,
221. Vaudano AE, Ruggieri A, Tondelli M, Avanzini Young IR, Bydder GM (1994) Artifacts due
P, Benuzzi F, Gessaroli G, Cantalupo G et al to stimulus correlated motion in functional
(2014) The visual system in eyelid myoclonia imaging of the brain. Magn Reson Med
with absences. Ann Neurol 76:412–427 31:283–291
222. De Tiege X, Laufs H, Boyd SG, Harkness W, 234. Lund TE, Norgaard MD, Rostrup E, Rowe
Allen PJ, Clark CA, Connelly A et al (2007) JB, Paulson OB (2005) Motion or activity:
 fMRI in Epilepsy 797

their role in intra- and inter-subject variation dant with the epileptogenic region deter-
in fMRI. NeuroImage 26:960–964 mined by intracranial EEG. Magn Reson
235. Kobayashi E, Hawco CS, Grova C, Dubeau F, Imaging 24:367–371
Gotman J (2006) Widespread and intense 246. Grouiller F, Thornton R, Groening K, Spinelli
BOLD changes during brief focal electro- L, Duncan JS, Schaller K, Siniatchkin M et al
graphic seizures. Neurology 66:1049–1055 (2011) Localization of focal epileptic activity
236. Baumgartner C, Serles W, Leutmezer F, with EEG-FMRI informed by EEG voltage
Pataraia E, Aull S, Czech T, Pietrzyk U et al maps. Epilepsia 52:169–169
(1998) Preictal SPECT in temporal lobe epi- 247. Morgan VL, Price RR, Arain A, Modur P,
lepsy: regional cerebral blood flow is increased Abou-Khalil B (2004) Resting functional
prior to electroencephalography-seizure MRI with temporal clustering analysis for
onset. J Nucl Med 39:978–982 localization of epileptic activity without
237. Federico P, Abbott DF, Briellmann RS, EEG. NeuroImage 21:473–481
Harvey AS, Jackson GD (2005) Functional 248. Hamandi K, Salek Haddadi A, Liston A,
MRI of the pre-ictal state. Brain Laufs H, Fish DR, Lemieux L (2005) fMRI
128:1811–1817 temporal clustering analysis in patients with
238. Meletti S, Vaudano AE, Tassi L, Caruana F, frequent interictal epileptiform discharges:
Avanzini P (2015) Intracranial time-frequency comparison with EEG-driven analysis.
correlates of seizure-related negative BOLD NeuroImage 26:309–316
response in the sensory-motor network. Clin 249. Aguirre GK, Zarahn E, D’Esposito M (1998)
Neurophysiol 126:847–849 The variability of human, BOLD hemody-
239. Archer JS, Briellman RS, Abbott DF, namic responses. NeuroImage 8:360–369
Syngeniotis A, Wellard RM, Jackson GD 250. Salek-Haddadi A, Friston KJ, Lemieux L,
(2003) Benign epilepsy with centro-temporal Fish DR (2003) Studying spontaneous EEG
spikes: spike triggered fMRI shows somato- activity with fMRI. Brain Res Brain Res Rev
sensory cortex activity. Epilepsia 44:200–204 43:110–133
240. Archer JS, Briellmann RS, Syngeniotis A, 251. Kobayashi E, Bagshaw AP, Grova C, Dubeau
Abbott DF, Jackson GD (2003) Spike- F, Gotman J (2006) Negative BOLD
triggered fMRI in reading epilepsy: involve- responses to epileptic spikes. Hum Brain
ment of left frontal cortex working memory Mapp 27:488–497
area. Neurology 60:415–421 252. Lemieux L, Laufs H, Carmichael D, Paul JS,
241. Tousseyn S, Dupont P, Robben D, Goffin K, Walker MC, Duncan JS (2008) Noncanonical
Sunaert S, Van Paesschen W (2014) A reliable spike-related BOLD responses in focal epi-
and time-saving semiautomatic spike- lepsy. Hum Brain Mapp 29:329–345
template-based analysis of interictal EEG- 253. Lu Y, Bagshaw AP, Grova C, Kobayashi E,
fMRI. Epilepsia 55:2048–2058 Dubeau F, Gotman J (2006) Using voxel-
242. Bagshaw AP, Hawco C, Benar CG, Kobayashi specific hemodynamic response function in
E, Aghakhani Y, Dubeau F, Pike GB et al EEG-fMRI data analysis. NeuroImage
(2005) Analysis of the EEG-fMRI response 32:238–247
to prolonged bursts of interictal epileptiform 254. Watanabe S, An D, Safi-Harb M, Dubeau F,
activity. NeuroImage 24:1099–1112 Gotman J (2014) Hemodynamic response
243. Laufs H, Hamandi K, Salek-Haddadi A, function (HRF) in epilepsy patients with hip-
Kleinschmidt AK, Duncan JS, Lemieux L pocampal sclerosis and focal cortical dysplasia.
(2007) Temporal lobe interictal epileptic dis- Brain Topogr 27:613–619
charges affect cerebral activity in “default 255. Hawco CS, Bagshaw AP, Lu Y, Dubeau F,
mode” brain regions. Hum Brain Mapp Gotman J (2007) BOLD changes occur prior
28:1023–1032 to epileptic spikes seen on scalp
244. Diehl B, Salek-haddadi A, Fish DR, Lemieux EEG. NeuroImage 35:1450–1458
L (2003) Mapping of spikes, slow waves, and 256. Makiranta M, Ruohonen J, Suominen K,
motor tasks in a patient with malformation of Niinimaki J, Sonkajarvi E, Kiviniemi V,
cortical development using simultaneous Seppanen T et al (2005) BOLD signal
EEG and fMRI. Magn Reson Imaging increase preceeds EEG spike activity--a
21:1167–1173 dynamic penicillin induced focal epilepsy in
245. Laufs H, Hamandi K, Walker MC, Scott C, deep anesthesia. NeuroImage 27:715–724
Smith S, Duncan JS, Lemieux L (2006) EEG- 257. Siniatchkin M, Moeller F, Jacobs J, Stephani
fMRI mapping of asymmetrical delta activity U, Boor R, Wolff S, Jansen O et al (2007)
in a patient with refractory epilepsy is concor- Spatial filters and automated spike detection
798 Rachel C. Thornton et al.

based on brain topographies improve sensitiv- 269. Raichle ME, MacLeod AM, Snyder AZ,
ity of EEG-fMRI studies in focal epilepsy. Powers WJ, Gusnard DA, Shulman GL
NeuroImage 37:834–843 (2001) A default mode of brain function.
258. Rosenow F, Luders H (2001) Presurgical Proc Natl Acad Sci U S A 98:676–682
evaluation of epilepsy. Brain 124:1683–1700 270. Lengler U, Kafadar I, Neubauer BA, Krakow
259. Kobayashi E, Bagshaw AP, Benar CG, K (2007) fMRI correlates of interictal epilep-
Aghakhani Y, Andermann F, Dubeau F, tic activity in patients with idiopathic benign
Gotman J (2006) Temporal and extratempo- focal epilepsy of childhood. A simultaneous
ral BOLD responses to temporal lobe interic- EEG-functional MRI study. Epilepsy Res
tal spikes. Epilepsia 47:343–354 75:29–38
260. Lemieux L, Krakow K, Fish DR (2001) 271. Engel J Jr, International League Against E
Comparison of spike-triggered functional MRI (2001) A proposed diagnostic scheme for
BOLD activation and EEG dipole model local- people with epileptic seizures and with epi-
ization. NeuroImage 14:1097–1104 lepsy: report of the ILAE Task Force on
261. Bagshaw AP, Kobayashi E, Dubeau F, Pike Classification and Terminology. Epilepsia
GB, Gotman J (2006) Correspondence 42:796–803
between EEG-fMRI and EEG dipole localisa- 272. Archer JS, Abbott DF, Waites AB, Jackson
tion of interictal discharges in focal epilepsy. GD (2003) fMRI “deactivation” of the poste-
NeuroImage 30:417–425 rior cingulate during generalized spike and
262. Boor R, Jacobs J, Hinzmann A, Bauermann wave. NeuroImage 20:1915–1922
T, Scherg M, Boor S, Vucurevic G et al 273. Gotman J, Grova C, Bagshaw A, Kobayashi
(2007) Combined spike-related functional E, Aghakhani Y, Dubeau F (2005) Generalized
MRI and multiple source analysis in the non- epileptic discharges show thalamocortical
invasive spike localization of benign rolandic activation and suspension of the default state
epilepsy. Clin Neurophysiol 118:901–909 of the brain. Proc Natl Acad Sci U S A
263. Salek-Haddadi A, Lemieux L, Fish DR (2002) 102:15236–15240
Role of functional magnetic resonance imag- 274. Laufs H, Lengler U, Hamandi K, Kleinschmidt
ing in the evaluation of patients with malfor- A, Krakow K (2006) Linking generalized
mations caused by cortical development. spike-and-wave discharges and resting state
Neurosurg Clin N Am 13:63–69, viii brain activity by using EEG/fMRI in a patient
264. Federico P, Archer JS, Abbott DF, Jackson with absence seizures. Epilepsia 47:444–448
GD (2005) Cortical/subcortical BOLD 275. Meeren HK, Pijn JP, Van Luijtelaar EL,
changes associated with epileptic discharges: Coenen AM, Lopes da Silva FH (2002)
an EEG-fMRI study at 3 T. Neurology Cortical focus drives widespread corticotha-
64:1125–1130 lamic networks during spontaneous absence
265. Kobayashi E, Bagshaw AP, Jansen A, seizures in rats. J Neurosci 22:1480–1495
Andermann F, Andermann E, Gotman J, 276. Steriade M, Dossi RC, Nunez A (1991)
Dubeau F (2005) Intrinsic epileptogenicity in Network modulation of a slow intrinsic oscil-
polymicrogyric cortex suggested by EEG- lation of cat thalamocortical neurons impli-
fMRI BOLD responses. Neurology cated in sleep delta waves: cortically induced
64:1263–1266 synchronization and brainstem cholinergic
266. Kobayashi E, Bagshaw AP, Grova C, Gotman suppression. J Neurosci 11:3200–3217
J, Dubeau F (2006) Grey matter heterotopia: 277. Hamandi K, Laufs H, Noth U, Carmichael
what EEG-fMRI can tell us about epileptoge- DW, Duncan JS, Lemieux L (2008) BOLD
nicity of neuronal migration disorders. Brain and perfusion changes during epileptic gener-
129:366–374 alised spike wave activity. NeuroImage
267. Kobayashi E, Bagshaw AP, Gotman J, Dubeau 39:608–618
F (2007) Metabolic correlates of epileptic 278. Moeller F, Siebner HR, Wolff S, Muhle H,
spikes in cerebral cavernous angiomas. Granert O, Jansen O, Stephani U et al (2008)
Epilepsy Res 73:98–103 Simultaneous EEG-fMRI in drug-naive chil-
268. Wiebe S, Blume WT, Girvin JP, Eliasziw M dren with newly diagnosed absence epilepsy.
(2001) Effectiveness and G efficiency of sur- Epilepsia 49:1510–1519
gery for temporal lobe epilepsy study. A ran- 279. Moeller F, Maneshi M, Pittau F, Gholipour T,
domized, controlled trial of surgery for Bellec P, Dubeau F, Grova C et al (2011)
temporal-lobe epilepsy. N Engl J Med Functional connectivity in patients with idiopathic
345:311–318 generalized epilepsy. Epilepsia 52:515–522
 fMRI in Epilepsy 799

280. Vaudano AE, Laufs H, Kiebel SJ, Carmichael during 3-Hz spike-and-wave complexes in
DW, Hamandi K, Guye M, Thornton R et al generalized epilepsy. Magn Reson Imaging
(2009) Causal hierarchy within the thalamo- 22:1441–1444
cortical network in spike and wave discharges. 289. Daunizeau J, Grova C, Marrelec G, Mattout
PLoS One 4, e6475 J, Jbabdi S, Pelegrini-Issac M, Lina JM et al
281. Moeller F, LeVan P, Muhle H, Stephani U, (2007) Symmetrical event-related EEG/
Dubeau F, Siniatchkin M, Gotman J (2010) fMRI information fusion in a variational
Absence seizures: individual patterns revealed Bayesian framework. NeuroImage
by EEG-fMRI. Epilepsia 51:2000–2010 36:69–87
282. Moeller F, Muthuraman M, Stephani U, 290. Jezzard P, Clare S (1999) Sources of distor-
Deuschl G, Raethjen J, Siniatchkin M (2013) tion in functional MRI data. Hum Brain
Representation and propagation of epileptic Mapp 8:80–85
activity in absences and generalized photopar- 291. Bagshaw AP, Torab L, Kobayashi E, Hawco
oxysmal responses. Hum Brain Mapp 34(8): C, Dubeau F, Pike GB, Gotman J (2006)
1896–1909 EEG-fMRI using z-shimming in patients with
283. Shmuel A, Augath M, Oeltermann A, temporal lobe epilepsy. J Magn Reson
Logothetis NK (2006) Negative functional Imaging 24:1025–1032
MRI response correlates with decreases in 292. Jezzard P, Balaban RS (1995) Correction for
neuronal activity in monkey visual area V1. geometric distortion in echo planar images
Nat Neurosci 9:569–577 from B0 field variations. Magn Reson Med
284. Binnie CD (2003) Cognitive impairment 34:65–73
during epileptiform discharges: is it ever justi- 293. Hutton C, Bork A, Josephs O, Deichmann R,
fiable to treat the EEG? Lancet Neurology Ashburner J, Turner R (2002) Image distor-
2:725–730 tion correction in fMRI: A quantitative evalu-
285. Todd N, Josephs O, Callaghan MF, Lutti A, ation. NeuroImage 16:217–240
Weiskopf N (2015) Prospective motion cor- 294. Niendorf T (1999) On the application of
rection of 3D echo-planar imaging data for susceptibility-weighted ultra-fast low-angle
functional MRI using optical tracking. RARE experiments in functional MR imag-
NeuroImage 113:1–12 ing. Magn Reson Med 41:1189–1198
286. Rodionov R, De Martino F, Laufs H, 295. Deichmann R, Josephs O, Hutton C, Corfield
Carmichael DW, Formisano E, Walker M, DR, Turner R (2002) Compensation of
Duncan JS et al (2007) Independent compo- susceptibility-induced BOLD sensitivity losses
nent analysis of interictal fMRI in focal epi- in echo-planar fMRI imaging. NeuroImage
lepsy: comparison with general linear 15:120–135
model-based EEG-correlated 296. Deichmann R, Gottfried JA, Hutton C,
fMRI. NeuroImage 38:488–500 Turner R (2003) Optimized EPI for fMRI
287. Merlet I, Gotman J (2001) Dipole modeling studies of the orbitofrontal cortex.
of scalp electroencephalogram epileptic dis- NeuroImage 19:430–441
charges: correlation with intracerebral fields. 297. Glover GH, Li TQ, Ress D (2000) Image-
Clin Neurophysiol 112:414–430 based method for retrospective correction of
288. Liston AD, Salek-Haddadi A, Kiebel SJ, physiological motion effects in fMRI:
Hamandi K, Turner R, Lemieux L (2004) RETROICOR. Magn Reson Med
The MR detection of neuronal depolarization 44:162–167
 Chapter 25

fMRI in Neurosurgery
Oliver Ganslandt, Christopher Nimsky, Michael Buchfelder,
and Peter Grummich

Abstract
Functional magnetic resonance imaging has evolved from a basic research application to a useful clinical
tool that also has found its place in modern neurosurgery. The localization of functional important brain
areas as language and sensorimotor cortex has been the focus of numerous investigations and can now be
implemented in neurosurgical planning. Since the neurosurgeon must have detailed knowledge about the
individual anatomy and related neurological function to resect a brain tumor with the highest safety, the
need for individualized maps of brain function is essential. Advanced fMRI techniques and modern imag-
ing methods contribute significantly to brain mapping as do already established concepts of electrophysi-
ological monitoring and the Wada test. The implementation of functional maps into neuronavigation
systems enables the surgeon to superimpose anatomy and function to the surgical site. This chapter
describes our experience with the use of fMRI in neurosurgery.

Key words fMRI, Neurosurgery, Functional neuronavigation, Magnetoencephalography, Language,

Somatosensory cortex

1 Introduction

The concept of using information about functionally important

brain areas (also known as “eloquent cortex”) to safely guide neu-
rosurgical procedures has been established in the middle of the
twentieth century by use of electrical stimulation in awake crani-
otomies. Based on the seminal work of Penfield [1], modern neu-
rosurgeons used the technique of electrical stimulation to
meticulously map the cerebral cortex of their patients, for instance,
to delineate the borders of resection in epilepsy and tumor surgery.
Today, electrical stimulation is still regarded as the “gold standard”
for neurosurgical functional brain localization [2, 3]. However,
these invasive direct cortical stimulation methods are not available
for preoperative decision making and surgical planning. They are
also time consuming and demand special resources. Therefore, in
the past decade new efforts have been made to overcome the

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_25, © Springer Science+Business Media New York 2016

801
802 Oliver Ganslandt et al.

limitations of using electrical stimulation in awake craniotomy by

using new techniques of brain imaging and the implementation of
these data into the neurosurgical workspace. In recent years, two
noninvasive techniques have been found especially suitable for pre-
surgical localization of the eloquent cortex: magnetoencephalogra-
phy (MEG) and fMRI. Studies using these techniques successfully
localized functional activity [4, 5].
One of the most interesting applications was the merge of func-
tional brain imaging with frame-based and frameless stereotaxy, also
known as functional neuronavigation [6, 7]. There is strong evi-
dence that the use of functional neuronavigation for lesions adja-
cent to eloquent brain areas may favor clinical outcome [8–10], but
large controlled studies to support this assumption are still needed.
If surgery near eloquent brain areas is planned, a detailed
knowledge about the topographic relation of a lesion to the adja-
cent functional brain area is crucial to avoid postoperative neuro-
logical deficits. In neurosurgery, the primary sensorimotor cortex
and the cortical areas subserving language comprehension and
production are considered to be the main structures of risk. In
temporal lobe surgery memory function is also an important func-
tion to preserve. These structures usually cannot be depicted from
conventional structural imaging techniques. Other reasons that
warrant a detailed evaluation are the individual representation of
these eloquent areas and the phenomenon of cortical reorganiza-
tion of these areas from their original positions [11, 12].
Furthermore, normal sulcal anatomy is not often discernible
because of a space occupying lesion. These situations require meth-
ods for localizing functional areas prior to surgery for decision
making, planning, and avoiding crippling postoperative results.
fMRI has become indispensable in neurosurgery to easily gain
knowledge about the topographic relation of a given lesion to the
functional brain area at risk and thus to plan the surgical approach.
Furthermore, fMRI-derived information about the extent of corti-
cal involvement in function can be used in conjunction with image-
guided surgery during resection of lesions adjacent to eloquent
brain areas under general anesthesia for navigation. The almost
ubiquitous availability of modern MR scanners favors the use of
fMRI over other modalities as MEG or positron emission tomog-
raphy (PET) that demand resources not commonly available. In
addition, its noninvasiveness gives the opportunity to repeat the
examinations and conduct follow-up studies on reorganization of
cortical function. Advances in MRI technology, such as the intro-
duction of higher field strengths, will undoubtedly improve signal
acquisition and processing [13]. Over the last years, a substantial
number of publications have described the usefulness of clinical
fMRI for neurosurgical applications [14, 15]. The use of fMRI for
the presurgical localization of the sensorimotor cortex is now
widely appreciated and has been investigated by several groups
 fMRI in Neurosurgery 803

[14, 16, 17], which also performed comparisons with direct motor
stimulation. Language fMRI has been found to be an alternative to
the invasive Wada test [18–20] for language lateralization.
Furthermore, fMRI has been used to predict memory localization
[21]. Concerning the reliability of fMRI-localization of speech
areas in the frontotemporal cortex, as compared with direct electri-
cal stimulation, the neurosurgical community is still reluctant to
rely on fMRI language alone, since inconsistent agreement has
been found between activation sites by fMRI naming and verb-
generation tasks and cortical stimulation [5]. As language fMRI
and intraoperative electrophysiological monitoring use different
physiological mechanisms the results of language fMRI are not per
se to be considered wrong. The ongoing clinical use of language
fMRI in our department has shown that this modality can be used
with the same results as awake craniotomy.

2 Methods

In our department, all neurosurgical fMRI measurements are

acquired on a 1.5 T MR scanner by echo-planar imaging
(Magnetom Sonata, Siemens, Erlangen, Germany).
Measurements for localization of motor and sensory activity are
performed with 16 slices of 3 mm thickness, a TR = 1580, and a
TE = 60. Stimulation is done in a block paradigm with 120 stimulus
presentations in six blocks. Twenty measurements during rest alter-
nated with 20 measurements during stimulation. During the motor
activation blocks, the patient is asked to perform a motor task: in
particular, we are interested in localizing the cortical representation
of the toes, foot, leg, fingers, hand, arm, tongue, lips, and eye lid.
Our selection of the motor tasks for each patient depends on
the tumor location. Attention is paid that the patient does not move
the opposite limb. With this, the possibility to detect reorganization
of functional areas to the contralateral hemisphere is ensured. Each
patient is also instructed not to touch anything during movement.
For this reason, the motor task is usually not a finger-tapping task.
Only in cases where we are interested in localizing the supplemen-
tary motor area we conduct a finger-tapping task.
For the localization of the sensory cortex, we use a tactile
stimulation of different limbs whose cortical representations we
wish to localize.
Measurements for language are done with 25 slices of 3 mm
thickness, a TR = 2470, and a TE = 60. Stimulation is done in a block
paradigm with 180 measurements in six blocks. We perform 30
measurements in an activation condition during which the patient is
instructed to perform a language task; we alternate these with 30
measurements in a resting condition. We also developed a battery of
several paradigms for mapping of memory function.
804 Oliver Ganslandt et al.

Two tests for factual knowledge consist of a recall of capitals of given

countries. Factual knowledge can also be tested by recall of celebri-
ties. Memorizing and recall of four digits is another test of hippo-
campal memory function (Fig. 1). In other tests the patients are
asked to connect telephone numbers to persons. By means of a mir-
ror that is attached to the head coil, the patient is able to observe
words, numbers, or pictures projected onto a screen.
We developed several stimulation paradigms for localizing the
Broca’s and the Wernicke’s areas. Usually, each patient is asked to
undergo four different paradigms during fMRI measurements. The
paradigms are selected according to tumor location and are adapted
to the abilities of each patient. The length of the interstimulus
interval is also adjusted according to the patient’s abilities, varying
between 900 and 3000 ms. The duration of the stimulus presenta-
tion lies between 600 and 1700 ms (300 ms less than the inter-
stimulus interval). We ask the patients to perform the tasks as
quickly as possible immediately after stimulus presentation and to
perform the task silently to avoid artifacts from mouth movement.
Paradigms are chosen on the basis of: (a) tumor location and
(b) patient cognitive abilities:

Fig. 1 fMRI-guided epilepsy surgery for ganglioglioma in the left hippocampus. We localized activity for factual
knowledge in the parahippocampal gyrus (crosshair). The segmented area is depicted in green in the right
image showing the view through the navigation microscope. Also shown in purple are the fiber tracts of the
visual pathway and occipitofrontal fasciculus (middle)
 fMRI in Neurosurgery 805

(a) In case of tumor location in the inferior parietal area close to the
intraparietal sulcus, we use an arithmetic task so that besides
Wernicke’s area (activated by reading, adding numbers, and for-
mulating the result) the cortex for calculation in the intraparietal
sulcus is also activated and so can be spared during surgery.
In case of tumor location close to Broca’s area, we select
language tasks that are also expressive and demand grammati-
cal abilities, because these may increase activity in Broca’s area.
This happens during the verb-generation task, but also the
verb conjugation task gives suitable Broca’s area activations.
(b) For patients with reduced abilities, simple tasks are selected to
obtain reliable results. Especially in patients who suffer from
word finding disorders, we avoid the picture-naming task. For
patients with better cognitive performance, we select one or
more complex tasks such as verb-generation task, because these
are reported to show a more clear lateralization, whereas in
patients who have difficulties in this complex task the activa-
tion is usually worse than with a simple paradigm.
For motion correction, we apply an image-based prospective
acquisition correction by applying interpolation in the k-space
[22]. We produce activation maps by analyzing the correlation
between signal intensity and a square wave reference function for
each pixel according to the paradigm. Pixels exceeding a signifi-
cance threshold (typical correlations above a threshold of 0.3 with
p < 0.000045) are displayed, if at least six contiguous voxels con-
stitute a cluster, to eliminate isolated voxels. We align the func-
tional slices to magnetization prepared rapid acquisition gradient
echo (MPRAGE) images (160 slices of 1 mm slice thickness).

3 Results

3.1 Localizations Since 2002, we have investigated preoperative fMRI with motor or
sensory stimulation in 515 cases. Of these patients, 205 underwent
tumor resection and 75 had stereotactic brain biopsy. In five addi-
tional patients, invasive electrodes were implanted by fMRI guid-
ance for chronic recording of epileptic discharges.
For language and memory testing, we examined 623 cases and
used additional information from MEG studies. Of them, 465
underwent tumor resection and 53 had stereotactic brain biopsy.
The remaining patients either obtained radiation therapy, endovas-
cular treatment, or were just enrolled in a “wait-and-see” protocol.
It was possible to localize the primary motor and sensory cor-
tex as well as the supplementary motor area (SMA) in all examined
cases (Figs. 2 and 3). Only in one case, the motor activity of the toe
was not detectable by fMRI because of tumor infiltration. However,
in this patient it was possible to obtain motor activation from
nearby muscle representations of the motor homunculus.
806 Oliver Ganslandt et al.

Fig. 2 fMRI activations during movement of left foot in a patient with an oligoastrocytoma (WHO III) in the right
parietal lobe. In front of the activation of the motor cortex (posterior wall of precentral gyrus), activation of the
supplementary motor area (SMA) is also evident

Fig. 3 Comparison of fMRI motor activations during arm movement and sensory
stimulation of the arm (oligoastrocytoma WHO III, same patient as Fig. 2)

We were able to define the motor homunculus along the cen-

tral sulcus in the posterior wall of the precentral gyrus with fMRI
by motor activation of the respective muscle groups (Fig. 4). We
localized toe, foot, leg, arm, hand, finger, thumb, tongue, lip, and
eye movements. Additionally, sometimes we found activity in the
 fMRI in Neurosurgery 807

Fig. 4 fMRI activations during movement of toe, leg, arm, and fingers (note that the lesion, a cavernoma in the
left motor cortex, is located between the cortical representation of the arm and that of the leg in the precentral
gyrus)

ipsilateral homotopic cortex. Especially when the motor task was

more complex there was also activity at the frontal wall of the pre-
central gyrus. Additional activity can be detected in the SMA in the
interhemispheric sulcus.
Activity at the posterior wall of the postcentral gyrus
(Brodman area 2) can be found and sometimes in the gyrus pos-
teriorly, which is likely to represent proprioceptive activation due
to the positions of the limbs.
Sensory activity was seen in the anterior wall (Brodman area 3)
of the postcentral gyrus. Sometimes there was also blood oxygen
level-dependent (BOLD) activation in the homotopic cortex of
the ipsilateral hemisphere that may indicate the presence of mecha-
nisms of cortical plasticity.
With verbal stimulation tasks, we were able to localize lan-
guage, calculation, and memory activity. In the frontal lobe, we
found activity in the following cortical areas: (1) at the bottom of
the opercular part of the inferior frontal gyrus anteriorly to the
precentral gyrus (Brodmann area 44, classical Broca’s area); (2) in
the adjacent part of the frontal cranial edge of the insular cortex;
and (3) close to the upper end of the inferior frontal gyrus, which
sometimes extends into the medial frontal gyrus. Here, there are
usually three cortical areas that extend from the pars triangularis to
pars opercularis (from anterior to posterior).
In the temporo-parietal region, we found language activity in
the superior temporal gyrus at its bottom in the temporal sulcus
and at its lateral side. Activity was also found at the top of the supe-
rior temporal gyrus in the planum temporale. Additional activity
was found in the supramarginal gyrus in the frontal part of the
intraparietal sulcus.
808 Oliver Ganslandt et al.

In 53 % of our patients with high-grade glioma, language

activity was not clearly detectable by fMRI alone, because of
changes in vascular function (see below).

3.2 Laterality Although it is generally accepted that the majority of people has left
hemispheric language dominance, the true number of atypical (right)
dominance is unknown. Studies using the Wada test showed an inci-
dence of left hemispheric dominance in right handers in a range of
63–96 % and a right hemispheric dominance for left handers and ambi-
dextrous patients in 48–75 % [23]. Furthermore, it is thought that
there are varying degrees of language dominance in the population.
In certain circumstances, activity can be located in both hemi-
spheres or reorganization to the other hemisphere could have been
occurred. FMRI is a useful method to clarify this. If activity is only
found in one hemisphere or the activity on one side is much stron-
ger than the activity on the other side, then it is clear that the active
area has to be spared during surgery.
It is important to know that certain stimulation tasks and modali-
ties show more lateralized activations than others. In case of complex
motor tasks, the ipsilateral hemisphere may also show activation.
For the localization of language activity, we found that visual
stimulation shows a more accentuated lateralization than acoustic
stimulation [24]. Stimulation with words, especially in a complex task,
shows a stronger lateralization than a picture-naming task, a finding
that was also described by Herholz et al. [25, 26]. In rare cases, it can
occur that not all language areas are located on the same side.

3.3 Surgery We perform fMRI-guided surgery by coregistering the activation

maps onto a 3D MRI data set that can be used with a navigation
system. Targets and areas at risk determined by fMRI are seg-
mented and made visible for the surgeon through a navigation
microscope (Figs. 5, 6, and 7). Thus functional data are visualized

Fig. 5 Microscopic view with neuronavigation markers showing the sensory activation of the arm area in light
blue and the pyramidal tract in purple (oligoastrocytoma WHO III, same patient as Fig. 2)
 fMRI in Neurosurgery 809

Fig. 6 fMRI activation of Broca’s and Wernicke’s areas and primary motor cortex
after a reading paradigm. The functional mapping was requested to plan surgery
of a cavernoma, which was located between Broca’s area and the motor cortex

Fig. 7 Same patient as Fig. 5. Segmentation lines indicating Broca’s area (left)
and motor cortex of tongue (right). The figure shows the beginning of the corti-
cotomy on a trajectory that spared the eloquent cortices (cross). Postoperatively
the patient was neurologically intact

in the operation field throughout the whole surgery.

Neuronavigation support is provided by the VectorVision Sky
Navigation System (BrainLab AG, Heimstetten, Germany). A fiber
optic connection ensures MR-compatible integration into the
radiofrequency-shielded room of our intraoperative MR suite. A
ceiling-mounted camera is used to monitor the positions of the
operating microscope (Pentero, Zeiss, Oberkochen, Germany),
which is placed outside the 5 G line, and other instruments.
810 Oliver Ganslandt et al.

A 1.0 mm isotropic 3D MPRAGE dataset (TE: 4.38 ms, TR:

2020 ms, slice thickness: 1.0 mm, FOV: 250 × 250 mm, measure-
ment time: 8 min 39 s) is acquired prior to surgery with the head
already fixed in the MR-compatible headholder as navigational refer-
ence dataset. For registration, five adhesive skin fiducial markers are
placed in a scattered pattern on the head surface prior to imaging and
registered with a pointer after their position is defined in the 3D data-
set (Fig. 6). Functional data from MEG and fMRI, which were
acquired preoperatively, are integrated into the 3D dataset.
Furthermore, data from diffusion tensor imaging (DTI) depicting
the course of major white matter tracts are integrated as well as in
selected cases metabolic maps from proton magnetic resonance spec-
troscopy (1H-MRS) are coregistered to the navigational dataset. In
addition, further standard anatomical datasets, such as T2-weighted
images, are coregistered. Repeated landmark checks are performed to
ensure overall accuracy. In case intraoperative imaging depicts some
remaining tumor, which should be further removed, intraoperative
image data are used for updating the navigation system (Fig. 8). After
a rigid registration of pre- and intraoperative images (ImageFusion
software, BrainLAB, Heimstetten, Germany), all data are transferred
to the navigation system and then the initial patient registration file is
restored, so that no repeated patient registration procedure is needed.
In our series with a surgical resection close to the motor cor-
tex, only 15 out of 205 patients (7.3 %) had postoperative neuro-
logic dysfunction. Three of them recovered within 2 days. In the
other cases, the condition improved over several months. One
patient, who had a hemiplegia prior to surgery, was able to move
the affected side after surgery.

Fig. 8 Intraoperative MRI showing the outcome of the fMRI-guided tumor resection. Note that the tumor was
removed sparing the sensory cortex (oligoastrocytoma WHO III right, same patient as Fig. 2)
 fMRI in Neurosurgery 811

No permanent postoperative deterioration of speech was

observed in our patients with surgery close to the language areas
(N = 465 patients). However, in 53 patients, in whom surgery was
conducted very close to the language areas, a transitory deteriora-
tion was observed (11 % of all patients with surgery neighboring
language areas). Thirty-two patients were not able to name part of
the shown objects; this impairment lasted from 1 day to few weeks,
but they all resolved completely.
No patient had suffered from global aphasia after surgery. The
result of having no permanent speech disorder in our patients indi-
cates that our language mapping is reliable. The presence of patients
with transitory disturbances suggests that resection was conducted
close to the boundaries of functional areas. This is in accordance to
other series that evaluated the outcome of glioma surgery in eloquent
areas with direct cortical stimulation. In a recent publication by
Duffau et al., the rate of severe neurological deficits was 6.5 % [27].
The safety margin that should be kept to preserve the func-
tional areas depends from several factors: (1) the kind of functional
center and the situation of reorganization; (2) the situation of blood
supply; and (3) the status of the connectivity fibers. At present, no
recommendations can be given for the exact distance to avoid the
risk of neurologic deficits. Neurosurgeons who use fMRI-guided
neuronavigation have to keep in mind that the fMRI-activation
does not represent the actual extent of the functional brain areas,
but rather a “center of gravity” of the functional units that are mea-
sured. Also one has to take into account that descending pathways
(e.g., the pyramidal tract) have also to be spared. A recent study
that investigated the accuracy between the actual location of the
pyramidal tract and subcortical electric stimulation with stereotactic
navigation found a mean difference in distance of 8.7 ± 3.1 mm
(standard deviation) [28]. Nevertheless, there are functional areas
that can be compensated for, if destroyed. These are the SMA and
the area in the fusiform gyrus for word recognition.
For the language areas, Haglund et al. [29] described in an
electrical stimulation study that above a resection distance of
10 mm from the eloquent areas they observed no permanent lan-
guage deficits. When surgery was 7–10 mm close to the language
area, they found 43 % patients who suffered from permanent lan-
guage deficits (severe or mild aphasia). The 9 % of patients had no
language deficit at all and the remaining 48 % experienced transi-
tory language deficits, which resolved within 4 weeks [29]. Two of
our patients showed an amelioration of language function after
surgery. One patient, who was not able to talk before surgery, was
able to talk afterwards. Another patient, who had severe naming
problems, showed an improvement after surgery.

3.4 Problems Sometimes the BOLD activations are not clearly visible in spite of
with the BOLD Effect the fact that the function is there, as confirmed by MEG
812 Oliver Ganslandt et al.

measurements. In our experience, such a discrepancy between

MEG and fMRI occurred only in the case of large tumors. Previous
reports indicated similar effects of vascular conditions on the
BOLD effect [30–32]. A reduction of the BOLD effect in the
vicinity of a glioma but not in the vicinity of nonglial tumors was
described by Schreiber et al. [33]. These findings are in agreement
with our results. In our series, we found that in 53 % of the patients
with high-grade gliomas the fMRI maps did not give clear indica-
tions of language areas in their vicinity.
Because of the impact of gliomas on the BOLD effect, the
dominant hemisphere sometimes is more easily found by MEG
measurements. This is seen for a patient with an astrocytoma
(WHO Grade II) in Fig. 9. Here MEG localizations of Wernicke’s
and Broca’s area were only on the right side. This was in accor-
dance with the Wada test that showed right hemispheric language
dominance in this left-handed patient. In this patient, fMRI local-
izations of Wernicke’s activity were similar on both sides, in MEG

Fig. 9 Wernicke and Broca activity during reading of fragmentary sentences with mistakes. Comparison
between fMRI (orange) and MEG beamformer localizations at 500 ms (light blue). With fMRI a bilateral activa-
tion in the operculum frontale and in the superior temporal sulcus can be seen. With MEG, activity is only seen
in the right hemisphere. In the first and second image in the lower row, activity of the insula can be seen with
MEG only. Left-handed patient with astrocytoma WHO II
 fMRI in Neurosurgery 813

they were only found in the right hemisphere. The activity detected
by MEG in the right insula was not found by fMRI.
Other reasons that might lead to suboptimal fMRI results are
continuous brain activation during rest or a very short activation of
brain areas. This may be the reason why memory activity in the
hippocampus is difficult to find by fMRI.

4 Conclusions

The use of preoperative fMRI brain mapping provides important

information for: (1) estimating the risk of a surgical procedure; (2)
planning the surgical approach; (3) indicating hemispheric domi-
nance; and (4) revealing whether reorganization of brain function
took place and at what degree. The integration of the functional
markers into the navigation system is a good tool to continuously
track the locations of the functional areas during surgery and enables
a resection close to the eloquent areas to be performed. Thus, fMRI-
guided functional navigation increases the amount of radical surgery
and decreases morbidity. When using fMRI in neurosurgery, it is
important to know that, in certain circumstances, the BOLD effect
can be suppressed, which may lead to wrong conclusions. Beside
integration of fMRI data the additional use of fiber tracking of the
descending pathways as well as other paraclinical investigations
(PET, proton magnetic resonance spectroscopy, MEG, etc.) should
lead to a comprehensive understanding of the options and limita-
tions of glioma surgery adjacent to important functional brain areas.

References

1. Penfield W, Rasmussen T (1950) The cerebral 5. Roux FE, Boulanouar K, Lotterie JA, Mejdoubi
cortex of man. A clinical study of localization M, LeSage JP et al (2003) Language functional
of function. Macmillan, New York magnetic resonance imaging in preoperative
2. Berger MS, Rostomily RC (1997) Low grade assessment of language areas: correlation with
gliomas: functional mapping resection strate- direct cortical stimulation. Neurosurgery
gies, extent of resection, and outcome. 52:1335–1345, discussion 1345–1337
J Neurooncol 34:85–101 6. Nimsky C, Ganslandt O, Kober H, Moller M,
3. Duffau H, Capelle L, Denvil D, Sichez N, Ulmer S et al (1999) Integration of functional
Gatignol P et al (2003) Usefulness of intraop- magnetic resonance imaging supported by
erative electrical subcortical mapping during magnetoencephalography in functional neuro-
surgery for low-grade gliomas located within navigation. Neurosurgery 44:1249–1255, dis-
eloquent brain regions: functional results in a cussion 1255–1246
consecutive series of 103 patients. J Neurosurg 7. Rutten GJ, Ramsey N, Noordmans HJ,
98:764–778 Willems P, van Rijen P et al (2003) Toward
4. Kober H, Moller M, Nimsky C, Vieth J, functional neuronavigation: implementation of
Fahlbusch R et al (2001) New approach to functional magnetic resonance imaging data in
localize speech relevant brain areas and hemi- a surgical guidance system for intraoperative
spheric dominance using spatially filtered mag- identification of motor and language cortices.
netoencephalography. Hum Brain Mapp Technical note and illustrative case. Neurosurg
14:236–250 Focus 15:E6
814 Oliver Ganslandt et al.

8. Rossler K, Sommer B, Grummich P, Hamer uation of temporal and frontal language domi-
HM, Pauli E et al (2015) Risk reduction in nance compared with the Wada test. Neurology
dominant temporal lobe epilepsy surgery com- 54:1625–1633
bining fMRI/DTI maps, neuronavigation and 20. Stippich C, Rapps N, Dreyhaupt J, Durst A, Kress
intraoperative 1.5-Tesla MRI. Stereotact Funct B et al (2007) Localizing and lateralizing lan-
Neurosurg 93:168–177 guage in patients with brain tumors: feasibility of
9. Zhang J, Chen X, Zhao Y, Wang F, Li F et al routine preoperative functional MR imaging in 81
(2015) Impact of intraoperative magnetic reso- consecutive patients. Radiology 243:828–836
nance imaging and functional neuronavigation 21. Branco DM, Suarez RO, Whalen S, O’Shea JP,
on surgical outcome in patients with gliomas Nelson AP et al (2006) Functional MRI of mem-
involving language areas. Neurosurg Rev ory in the hippocampus: laterality indices may be
38:319–330, discussion 330 more meaningful if calculated from whole voxel
10. Sun GC, Chen XL, Yu XG, Zhang M, Liu G distributions. Neuroimage 32:592–602
et al (2015) Functional neuronavigation- 22. Thesen S, Heid O, Mueller E, Schad R (2000)
guided transparieto-occipital cortical resection Prospective acquisition correction for head
of meningiomas in trigone of lateral ventricle. motion with image-base tracking for real-time
World Neurosurg 84(3):756–765 fMRI. Magn Reson Med 44:457–465
11. Duffau H, Denvil D, Capelle L (2002) Long 23. Springer JA, Binder JR, Hammeke TA,
term reshaping of language, sensory, and Swanson SJ, Frost JA et al (1999) Language
motor maps after glioma resection: a new dominance in neurologically normal and epi-
parameter to integrate in the surgical strategy. lepsy subjects: a functional MRI study. Brain
J Neurol Neurosurg Psychiatry 72:511–516 122(Pt 11):2033–2046
12. Grummich P, Nimsky C, Fahlbusch R, 24. Grummich P, Nimsky C, Pauli E, Buchfelder
Ganslandt O (2005) Observation of unaver- M, Ganslandt O (2006) Combining fMRI and
aged giant MEG activity from language areas MEG increases the reliability of presurgical lan-
during speech tasks in patients harboring brain guage localization: a clinical study on the dif-
lesions very close to essential language areas: ference between and congruence of both
expression of brain plasticity in language pro- modalities. Neuroimage 32:1793–1803
cessing networks? Neurosci Lett 380:143–148 25. Herholz K, Reulen HJ, von Stockhausen HM,
13. Tieleman A, Vandemaele P, Seurinck R, Thiel A, Ilmberger J et al (1997) Preoperative acti-
Deblaere K, Achten E (2007) Comparison vation and intraoperative stimulation of language-
between functional magnetic resonance imag- related areas in patients with glioma. Neurosurgery
ing at 1.5 and 3 Tesla: effect of increased field 41:1253–1260, discussion 1260–1262
strength on 4 paradigms used during presurgi- 26. Lazar RM, Marshall RS, Pile-Spellman J, Duong
cal work-up. Invest Radiol 42:130–138 HC, Mohr JP et al (2000) Interhemispheric
14. Matthews PM, Jezzard P (2004) Functional transfer of language in patients with left frontal
magnetic resonance imaging. J Neurol cerebral arteriovenous malformation.
Neurosurg Psychiatry 75:6–12 Neuropsychologia 38:1325–1332
15. Tharin S, Golby A (2007) Functional brain map- 27. Duffau H, Lopes M, Arthuis F, Bitar A, Sichez
ping and its applications to neurosurgery. JP et al (2005) Contribution of intraoperative
Neurosurgery 60:185–201, discussion 201–202 electrical stimulations in surgery of low grade
16. Majos A, Tybor K, Stefanczyk L, Goraj B gliomas: a comparative study between two series
(2005) Cortical mapping by functional mag- without (1985–96) and with (1996–2003) func-
netic resonance imaging in patients with brain tional mapping in the same institution. J Neurol
tumors. Eur Radiol 15:1148–1158 Neurosurg Psychiatry 76:845–851
17. Roux FE, Boulanouar K, Ibarrola D, Tremoulet 28. Berman JI, Berger MS, Chung SW, Nagarajan
M, Chollet F et al (2000) Functional MRI and SS, Henry RG (2007) Accuracy of diffusion
intraoperative brain mapping to evaluate brain plas- tensor magnetic resonance imaging tractogra-
ticity in patients with brain tumours and hemipare- phy assessed using intraoperative subcortical
sis. J Neurol Neurosurg Psychiatry 69:453–463 stimulation mapping and magnetic source
18. Desmond JE, Sum JM, Wagner AD, Demb JB, imaging. J Neurosurg 107:488–494
Shear PK et al (1995) Functional MRI mea- 29. Haglund MM, Berger MS, Shamseldin M,
surement of language lateralization in Wada- Lettich E, Ojemann GA (1994) Cortical local-
tested patients. Brain 118(Pt 6):1411–1419 ization of temporal lobe language sites in
19. Lehericy S, Cohen L, Bazin B, Samson S, patients with gliomas. Neurosurgery 34:567–
Giacomini E et al (2000) Functional MR eval- 576, discussion 576
 fMRI in Neurosurgery 815

30. Hamzei F, Knab R, Weiller C, Roether 32. Holodny AI, Schulder M, Liu WC, Wolko J,
J (2002) Intra- und extrakranielle Maldjian JA et al (2000) The effect of brain
Gefäßstenosen beeinflussen BOLD Antwort. tumors on BOLD functional MR imaging acti-
Aktuelle Neurologie 29:231 vation in the adjacent motor cortex: implica-
31. Holodny AI, Schulder M, Liu WC, Maldjian tions for image-guided neurosurgery. AJNR
JA, Kalnin AJ (1999) Decreased BOLD func- Am J Neuroradiol 21:1415–1422
tional MR activation of the motor and sen- 33. Schreiber A, Hubbe U, Ziyeh S, Hennig
sory cortices adjacent to a glioblastoma J (2000) The influence of gliomas and nonglial
multiforme: implications for image-guided space-occupying lesions on blood-oxygen-
neurosurgery. AJNR Am J Neuroradiol level-dependent contrast enhancement. AJNR
20:609–612 Am J Neuroradiol 21:1055–1063
 Chapter 26

Pharmacological Applications of fMRI

Paul M. Matthews

Abstract
Increasing societal expectations for new drugs, lack of confidence in short-term endpoints related to long-term
outcomes for chronic neurological and psychiatric diseases and rising costs of development in an increasing
cost-constrained market all have created a sense of crisis in CNS drug development. New approaches are
needed. For some time, the potential of clinical functional imaging for more confident progression from pre-
clinical to clinical development stages has been recognized. Pharmacological functional MRI (fMRI), which
refers specifically to applications of fMRI to questions in drug development, provides one set of these tools.
With related structural MRI measures, relatively high resolution data concerning target, disease-relevant
pathophysiology and effects of therapeutic interventions can be related to brain functional anatomy. In this
chapter, current and potential applications of pharmacological fMRI for target validation, patient stratification
and characterization of therapeutic molecule pharmacokinetics and pharmacodynamics are reviewed.
Challenges to better realizing the promise of pharmacological fMRI will be discussed. The review concludes
that there is a strong rationale for greater use of pharmacological fMRI particularly for early phase studies, but
also outlines the need for preclinical and early clinical development to be more seamlessly integrated, for
greater harmonization of clinical imaging methodologies and for sharing of data to facilitate these goals.

Key words Pharmacological fMRI, Target validation, Patient stratification, Pharmacokinetics,

Pharmacodynamics

1 Introduction

Both the pharmaceutical industry and regulators are searching for

better models and for new drug development, particularly for CNS
drugs [1]. Public confidence in the industry has declined in the
face of what is viewed as a lack of commitment to addressing major
diseases with innovative drugs, while new drug costs continue to
escalate. Industry sees the risks of drug development to be high
particularly for chronic CNS diseases, for which there is a notable
lack of consensus regarding underlying causes and mechanisms in
the scientific community. CNS drug development appears uncer-
tain, slow, and expensive.
Pharmacological fMRI provides a relatively direct measure of
CNS functions. Noninvasive imaging methods also allow the same

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_26, © Springer Science+Business Media New York 2016

817
818 Paul M. Matthews

Disease
selection
Gene Taget to PoC to
Lead to FTIH to Phase File and Life cycle
function lead Preclinical commit to
candidate PoC III launch management
Target to target compound Phase III
family
selection

Fig. 1 The “critical path” for drug development. Pharmacological MRI has the potential to enhance the effi-
ciency of early clinical development with better translation of biological concepts from preclinical to clinical
studies, providing a new pharmacodynamic measure and enhancing potential in proof-of-mechanism studies
(FTIH first time in human study, PoC proof of concept study)

endpoint measures to be used in preclinical as in clinical develop-

ment. This facilitates interpretation of clinical imaging outcomes in
terms of molecular and cellular changes found with invasive meth-
ods preclinically [2]. These and related considerations have embed-
ded imaging in drug development already. Almost 30 % of new
molecular entities approved for neuropsychiatric indications by the
Food and Drug Administration between 1995 and 2004 were
developed with contributions from imaging [3]. A 2013 review
identified 70 CNS drug trials registered on the registry website
clinicaltrials.gov, that incorporated imaging endpoints [4]. In
selected areas, such as multiple sclerosis drug development or
recent trials of molecules for Alzheimer’s disease, clinical imaging
measures are used routinely for patient selection, for trials, or for
response and safety monitoring. While most of these applications
have relied on serial structural MRI, they have demonstrated the
feasibility of implementing large scale, regulatory compliant clini-
cal trials with imaging endpoints. They make the case for future use
of pharmacological fMRI plausible.
Another factor that contributes to the plausibility of greater
use of pharmacological fMRI in clinical drug development is the
increasing premium being placed on integration of preclinical
studies and early-phase development in an “experimental medi-
cine” (sitting fluidly on the Phase I/IIa boundary) stage as part of
confidence building and risk mitigation. Experimental medicine
uses human experimentation to address mechanistic questions in
ways that traditionally were reserved for preclinical studies. It is
part of a biologically driven therapeutics development strategy
involving hypothesis-led research that often is performed widely
across levels of biological complexity (e.g., cells to the whole
organism). A fundamental premise is that animals can be used to
model biology, but cannot be expected to model human disease,
which must be studied in the human. With this thinking, the clas-
sically unidirectional “critical path” from drug development
(Fig. 1) is enabled by tools (e.g., from omics and imaging) to
 Pharmacological Applications of fMRI 819

become more powerfully bidirectional (e.g., from preclinical to

clinical data and “back again”).

2 Principles of Functional MRI (fMRI)

FMRI is based on indirect measures of neuronal response mediated

through associated changes in blood flow. Increased neuronal activ-
ity is associated with a local hemodynamic response involving both
increased blood flow and blood volume. This neurovascular cou-
pling is related to the increased local energy consumption associ-
ated with neuronal activity, which generally is believed to reflect
predominantly presynaptic activity [5–7]. The hemodynamic
response has a magnitude and time course that depends on contri-
butions from both inhibitory and excitatory inputs to the local field
potential [8]. It therefore can be considered as a measure of local
information input. While this may be correlated with local multi-
unit activity (neuronal spiking activity) under some conditions,
such a relationship is not necessarily generalizable.
The neurovascular response is regulated by neuronal–glial
interactions mediated by multiple signaling mechanisms.
Pharmacological fMRI applications therefore need to take into
account any potential impact of experimental molecules (or the
disease of interest) on these coupling mechanisms. For example,
the cerebrovascular effects of multiple neurotransmitter systems
that may be the target for therapeutic molecules (e.g., gluta-
mate, dopamine, norepinephrine, serotonin, acetylcholine, and
prostaglandins) are well described [9]. Disorders of cerebrovas-
cular regulation also are recognized in a number of disease states
including not only primary cerebrovascular diseases such as
stroke, but also, e.g., Alzheimer’s disease [10–12].
The most commonly used fMRI methods rely on blood oxy-
gen level-dependent (BOLD) imaging contrast [13, 14]. This
contrast arises because the concentrations of deoxyhemoglobin,
which is paramagnetic and thus locally modulates an applied
static magnetic field, vary with local blood flow and oxygen con-
sumption. In the MRI magnet, where a highly homogeneous (i.e.,
spatially invariant) magnetic field is generated, the paramagnetic
deoxyhemoglobin generates small magnetic field inhomogeneities
around blood vessels. Their magnitude increases with the amount
of paramagnetic deoxyhemoglobin. These inhomogeneities
reduce the MRI signal acquired with a gradient echo MRI acqui-
sition sequence (echo planar imaging or EPI). Transient decreases
in BOLD contrast associated with brain activity reflect neuronal
activation because blood flow increases with greater neuronal
activity to an extent that is larger than is needed simply for
increased oxygen delivery with greater tissue demands. This
reduces the local ratio of deoxy- to oxyhemoglobin in the blood
820 Paul M. Matthews

enough to be associated with an increase in the local EPI MRI

signal. While this effect is small (0.5–5 % typically at 3 T), it can
be measured reliably with signal averaging.
Alternative approaches to brain functional imaging rely on
measures of brain blood flow. The advent of fMRI was heralded by
changes in blood flow measured by tracking a bolus of intravenously
injected exogenous contrast material [15]. Arterial spin labeling
MRI (ASL) has been developed more recently as an alternative,
noninvasive pharmacological fMRI approach that is based on mea-
suring brain activity associated changes in blood flow by means of
noninvasive magnetic “tagging” (with a radiofrequency pulse) of
blood flowing into the brain. Methods have become increasingly
standardized in recent years, are widely available on commercial
clinical imaging systems and can have considerable precision [16].
Both approaches to pharmacological fMRI can be applied in
two general ways. In “task based” pharmacological fMRI, con-
strained shifts in cognitive state are induced to explore the way in
which physiological differences between the states are modulated
by an associated intervention. A typical experiment would involve
acquisition of a series of images over the course of a controlled,
periodic variation in cognitive state (e.g., performing a working
memory task relative to resting) with and without the putative
modulatory intervention of interest. Regions of significant change
in the difference in BOLD signal between the two cognitive states
then are defined by statistical analysis of the time series data.
An alternative design relies on the modulation of brain sponta-
neous activity in the absence of specific stimuli, i.e., in the “resting
state”. This approach is based on the observation that correlated,
local and long-distance temporally varying signals are found with
fMRI just as was previously found in the EEG [17]. This oscilla-
tory activity appears fundamental to brain functional organization.
Far field activity in the gamma range (~30–80 Hz) may be particu-
larly relevant for the BOLD signal responses found in resting-state
fMRI [7, 18]. There are multiple ways of defining the long-dis-
tance oscillatory coupling in fMRI [19], as yet without great stan-
dardization. For both task- and resting-state fMRI applications,
assessment of responses to interventions involves statistical con-
trasts of time-courses before and after the intervention [20].
Both BOLD and ASL-based fMRI signals are low and can be
confounded by other contributions to the temporally varying brain
signal from subject movement (even on the order of mm), cardio-
respiratory variations, image acquisition artifacts, and even differ-
ence in imaging system performance over time [21]. Some artifacts
(e.g., movement) are easier to recognize and can be “edited out”
post hoc [22]. Controlling for potential systematic variation in the
parameters (e.g., increased respiratory rate in anxious subjects with
a brain disease relative to healthy control subjects) as best as is pos-
sible is particularly important [21]. The potential for these factors
 Pharmacological Applications of fMRI 821

to have an impact on outcomes emphasizes the importance of rep-

lications of results across laboratories and study populations,
although this has rarely been achieved to date.

3 Target Validation

The traditional progression of drug development through target

validation in preclinical models that express phenotypes plausibly
related to the human disease is hugely challenged by most of the
major diseases of the brain. Concepts for preclinical analogues of
neuropsychiatric disorders with complex behavioral phenotypes
(e.g., schizophrenia) and the validity of models for other major
diseases including the chronic diseases of late life and those involv-
ing slow, progressive neurodegeneration are limited by differences
in biological context and environment. New strategies for drug
development are needed.
Preclinical models still provide powerful tools for detailed
study of specific biological mechanisms believed to contribute to
disease. With these models, pharmacological fMRI endpoints can
be related to the underlying molecular changes in ways that both
validate interpretation of the imaging endpoints and establish a
framework in which they can be used to infer the dynamics of
molecular pathogenic events. For example, the acute effects of
NMDA receptor antagonism with ketamine were mapped in the
rodent, demonstrating a pattern of cortico-limbic-thalamic activa-
tion and establishing a relationship between specific cognitive sys-
tems and the pharmacology [23]. Similar functional effects also
were seen with other antagonists against the same target [24, 25],
further confirming the specificity of the systems modulated. A
framework for interpretation of these results was able to be pro-
vided by convergent studies using 2-deoxyglucose autoradiogra-
phy [26] as an index of presynaptic activity, along with single unit
electrophysiological recording and immediate early gene expres-
sion [27]. Analogous pharmacological fMRI experiments con-
ducted in human studies provided mapped homologous systems in
humans and to relate the pharmacology to the associated thought
disorder and disturbance of consciousness in turn [28]. While indi-
rect and insufficient alone, these clinical studies together provided
important information supporting target validation of NMDA
receptors for psychotic disorders; the “bi-directional” translational
approach also supported the potential relevance of this preclinical
pharmacology for understanding a form of human psychosis.
An exciting, emerging extension of this approach applies struc-
tural MRI and pharmacological fMRI measures together as endophe-
notypes in testing for heritable quantitative traits [29]. Consider, for
example, a complex genetic disease such as schizophrenia, which
shows a heritable phenotype with variable expression. Both structural
822 Paul M. Matthews

and functional differences can be defined relative to the healthy brain.

The concept of the endophenotype is that their forme fruste are heri-
table and can be identified in people even without clinical expression
of the disease or trait. To the extent that this is true, the imaging
endpoints themselves can be used as outcome measures in searches
for genetic or other factors that may contribute to the disease. An
endophenotype-based target validation approach also may bias detec-
tion towards causative rather than simply (possibly incidental or non-
specific) associated features. Candidate genes DISC1, GRM3, and
COMT, which are associated with altered hippocampal structure and
function [30], glutamatergic fronto-hippocampal function [31], and
prefrontal dopamine responsiveness [32], respectively, all have been
related to imaging endophenotypes for schizophrenia in this way.
The concept of fMRI endophenotypes strengthens the rationale
nosological reclassification of disease in terms of shared neurobio-
logical system dysfunction. Applications of fMRI approaches that
define neurobiological bases for general cognitive processes (such as,
in the context of psychiatric disease, motivation, or reward) facilitate
more holistic views of targets that may be relevant to more than one
disease. For example, fMRI approaches have contributed to the cur-
rent appreciation for neural mechanisms common to addictive
behaviors across a wide range of substances abuse states. Studies of
cue-elicited craving have defined similar activities of the mesolimbic
reward circuit in addictions to nicotine [33], alcohol [34], gambling
[35], amphetamine [36], cocaine [37] and opiates [38].
Combination of pharmacological fMRI with positron emission
tomography (PET) receptor mapping can be used to relate systems-
level dysfunction directly with the molecular targets of drug thera-
pies in ways that enhance target validation for new pharmacological
treatments faster and more cheaply than conventional clinical
designs allow (see, e.g., [39]). In another example, a combined
PET D3 receptor availability and resting-state pharmacological
fMRI study provides a paradigmatic example of the way in which
modulation of both target and system contributes to better defining
fundamental mechanistic relationships between different symptoms
[40]. First, D3 receptor availability was assessed in the ventral teg-
mentum/substantia nigra in healthy subjects using PET with the
D3/D2 selective radioligand, [11C](+)-4-propyl-9-
11
hydroxynaphthoxazine ([ C]PHNO). Differences in receptor
expression and basal dopamine release determine binding of the
[11C]PHNO, which varied across subjects. A resting-state pharma-
cological fMRI study was conducted simultaneously. Parametric
variation of the resting-state pharmacological fMRI functional con-
nectivities with D3 receptor availability measured by PET showed
that low midbrain D3 receptor availability (reflecting dopamine
release) was associated with increased connectivity between orbito-
frontal cortex (OFC) and brain networks implicated in cognitive
control and salience processing. The results together further vali-
dated dopamine D3 receptor signaling as an important modulator
 Pharmacological Applications of fMRI 823

of systems underpinning human goal-directed behavior, while

highlighting differentially modulated interactions between OFC
and networks implicated in cognitive control and reward.
With confidence in the relationship between a pattern of brain
functional network activation and behaviors of interest, the former
can be used as a clinically relevant biomarker for target validation.
One of the first demonstrations of this was with the modulation of
hippocampal activation with a working memory fMRI task based on
allelic differences in a BDNF gene polymorphism [41]. This provided
early evidence in humans supporting target validation of the TrkB
receptor agonism for the treatment of cognitive symptoms associated
with synaptic plasticity [42]. A different example illustrating how
such studies can be used for decision making in drug development
was provided by an imaging experimental medicine study linking to a
PET receptor occupancy of a highly specific μ-opioid antagonist,
GSK1521498, to pharmacological fMRI modulation of brain activa-
tion associated with palatable taste stimuli [43]. This allowed a first
demonstration that salience and reward systems relevant to food
intake were modulated by the target, suggesting the potential of
antagonists as appetite suppressants, an inference supported by a
later, larger Phase IIa study with a direct behavioral endpoint [44].

4 Patient Stratification

A critical issue in early drug development is to establish an appro-

priate level of confidence in the potential of a new molecule to
become a therapy. One way in which this can be done is by better
controlling for the substantial variations in therapeutic responses
between individuals in early-phase studies. As well demonstrated in
oncology [45], stratification of patients based on specific disease
characteristics can enable more powerful trial designs [46].
Consider, hypothetically, the difference in outcome of trials for a
population in which a new molecule has a 50 % treatment effect in
20 % of patients (giving a 10 % net treatment effect, i.e., unlikely to
be detected) relative to that in a stratified population enriched so
that 70 % are responders (a net 35 % treatment effect). By predict-
ing potential responders, imaging also can suggest ways of best
selecting optimal indications for new molecules. To the extent that
the enrichment is successful and any new pharmacological activity
being evaluated is detectable, clinical trials may demonstrate
molecule effects with fewer subjects exposed. This can be of special
value in early Phase II trials when safety data is limited and the
focus is on internal decision making.
An early application of imaging based stratification is expected to
be for enrichment of populations for clinical trials in diseases such as
Alzheimer’s disease for which there is considerable phenotypic over-
lap between different disorders manifesting in the same population
(e.g., dementia and late-life depression). The posterior cingulate and
824 Paul M. Matthews

hippocampus show high functional connectivity in resting-state fMRI

[47] and form the core of a so-called “default mode” network [48].
Decreases in default mode resting-state fMRI connectivity distin-
guish Alzheimer’s patients from healthy subjects and can distinguish
patients with mild cognitive impairment who undergo cognitive
decline and conversion to Alzheimer’s disease from those who remain
stable over a medium term follow-up period [49, 50]. Distinct pat-
terns of resting-state fMRI may distinguish patients with Parkinson’s
disease, for whom reduced resting state functional connectivity from
the basal ganglia was reported [51]. Together, these findings suggest
that resting-state fMRI (conducted in conjunction with other struc-
tural imaging measures), could be used to enrich trials for early dis-
ease modification of Alzheimer’s disease.
Establishing fMRI measures for stratification of patients [52]
also ultimately could aid in establishing prognosis and in patient
management. Where alternative treatment approaches are available
that have potentially significant individual variation in response
across a population, selection of the optimal treatment for an indi-
vidual patient could be assisted by fMRI (personalized medicine).
For example, with depression, treatment responses are highly vari-
able, e.g., only about 70 % of patients respond well to a given anti-
depressant [53]. Higher BOLD signal in the amygdala with a
simple task fMRI may be predictive of subsequent treatment
response [54]. Multivariate fMRI responses that change with treat-
ment in depression also have been proposed as candidate pharma-
cological fMRI markers, e.g., signal change in the ventromedial
prefrontal and anterior cingulate cortices [55] or modulation of
cortico-limbic functional connectivity [56].
In similar ways, there is a potential for integrated structural MRI
and verbal task fMRI to distinguish people with prodromal schizo-
phrenia from phenotypic mimics [57]. Network based analyses pro-
vide evidence for a continuous spectrum of psychosis from healthy
variants to disabling expressions of schizophrenia [58]. Brain func-
tional measures distinguishing abnormal network functions ulti-
mately may provide more meaningful approaches to the classification
of neuropsychiatric diseases for improved prognosis and for target-
ing of treatment [59–62], although establishing the robustness of
classifiers in terms of longer term clinical outcome will demand stan-
dardization of methods and long-term, prospective studies.
Arguably fundamental changes in the understanding of chronic
pain as a disease with individual differences in susceptibility have
developed in recent years in part as a consequence of fMRI studies
[63, 64]. Activity in the posterior insula with nociception provides
a link between the subjectively “painful” experiences of pain empa-
thy [65], hypnotically induced pain [66], and recalled pain experi-
ences [67]. Inspired by studies showing a dopaminergic response
with anticipation of benefit in Parkinson’s disease, nigro-striatal
pathways (as well as those associated with endogenous opioid
 Pharmacological Applications of fMRI 825

release) have been implicated in the placebo response in pain and

depression [68]. Individual variation in pain vulnerability thus is
associated with alterations in wide range brain networks concerned
with reward, motivation/learning, and descending modulatory
control [69]. Greater functional connectivity between the PFC
and nucleus accumbens explains pain persistence, suggesting that
the frontal-striatal connectivity mediates the transition from acute
to chronic pain; cortical-striatal connectivity explains longer term
outcomes of patients with sub-acute back pain [70].
Nonetheless, despite this promise, validation and development
of these concepts as clinical tools or for confident use as an enrich-
ment strategy or as a secondary outcome measure in later-phase
clinical trials appears stalled by lack of standardization of evalua-
tions and methods for quality control and analysis [4]. A focus on
longer term, well powered clinical studies is needed to validate
relationships between fMRI measures and disease pathology or
long-term clinical outcomes. Confident demonstrations are needed
to establish that fMRI or pharmacological fMRI reliably distin-
guish clinically meaningfully changes.

5 Pharmacodynamics

As the previous section highlighted, applications of pharmacologi-

cal fMRI to the direct assessment of drug action are expanding.
Pharmacodynamic data (e.g., testing whether a drug at the chosen
dose has an effect on brain function) can be obtained from analysis
of brain imaging changes induced by the administration of a drug.
The similar intrinsic brain architecture across species can support
translational proof of mechanism studies with comparisons of end-
points from preclinical and imaging-supported Phase I studies using
similar methodologies [71]. Additional information can come from
correlation of brain activity with behavioral effects of drug adminis-
tration [72] (Fig. 2) or with characterization of the way in brain
activity associated with a probe-task is modulated by a drug [73–
75]. This information can inform clinical dose-ranging studies. As
noted earlier, correlations between fMRI measures of brain func-
tional system response and drug receptor or receptor occupancy
measurements by PET are possible [39, 43, 76]. The last, more
recent study [43], demonstrated additionally how integration of
time-receptor occupancy data from PET with fMRI measures can
differentiate the distinct pharmacologies of different antagonists.
In some situations, by providing a measure of endophenotype
responses, pharmacological fMRI can define effects of treatment in
populations too small for behavioral effects to be discerned or
where usual clinical measures are simply insensitive to drug effects
[77–79]. In the simplest application, modulation of brain activa-
tion in functional anatomically plausible regions after dosing with
826 Paul M. Matthews

Fig. 2 Pharmacological fMRI can be performed in both animals and humans to assess correspondences in
tests of mechanisms. (a) Pharmacological fMRI results with metamphetamine challenge of a rodent, identify-
ing major regions in the monamine network as sites of direct or indirect action (Mctx motor cortex, PrL pre-
limbic medial prefrontal cortex, thal thalamus, SSctx somatosensory cortex, AcbSh shell of the nucleus
accumbens, VTA ventral tegmental area) (Images courtesy of Dr. A. Bifone, GSK, Verona). (b) A similar pharma-
cological fMRI experiment with acute amphetamine infusion in human subjects performed using “mind racing”
as a behavioral index of drug effects identified comparable elements of the core response network (OFC
orbitofrontal cortex, ACC anterior cingulate cortex, NAC nucleus accumbens)

candidate molecule simply to provide supportive evidence for rel-

evant direct CNS activity. A retrospective case study of NK-1
receptor antagonists for chronic pain proposed that early decisions
based on fMRI measures could have anticipated the later failure of
clinical trials [80]. However, a potential risk of such entirely phar-
macological fMRI-derived pharmacodynamics markers is that they
may not be specific for (or predictive of) clinically relevant changes.
One way of minimizing this risk is to frame the measures in
terms of important disease symptoms based on the relationship
between fMRI measures and individual symptoms. Mechanistic
plausibility is suggested by the extent to which changes in the asso-
ciated networks have been independently related to clinically
meaningful symptoms. An illustration of this is provided by the
way fMRI has been used to dissect the subjective experience of pain
into anatomically distinct activities of different functional systems
(including arousal and the somatosensory and limbic systems), the
precise pattern for which may vary for an individual depending on
context, mood, and cognitive state [64, 81].
As highlighted in the introduction to this review, imaging has
the potential to bridge directly between preclinical and clinical
studies [2]. While many behaviors cannot be translated across spe-
cies, functional-anatomical correlations allow direct drug responses
 Pharmacological Applications of fMRI 827

elicited in the brain for translation of underlying neurobiology. For

example, pharmacological fMRI experiments in which unstimu-
lated brain responses to acute compound challenges can be used to
define brain regions in which activity is modulated by the same
compound in animals (Fig. 2). Preclinically, these observations can
be linked to results from more invasive studies, e.g., direct measure-
ments of neurotransmitter release that distinguish direct and indi-
rect effects of the compound [82]. Similar observations of drug
modulation of brain activity can be made in human volunteers, pro-
viding a way of confirming mechanism (Fig. 2) [72]. State-
dependent modulation of these regions can further contribute to
this [83]. By relating plasma concentrations to brain responses,
similar approaches could be used to define dose, for example. fMRI
can address the need for evaluation of receptor agonists, partial
agonists, and inverse agonists, as well as antagonists. Even when a
receptor targeted radioligand is available, PET methods generally
will not be informative with the former classes of agents [84].
However, caution is needed in the confidence with which
fMRI endpoints are interpreted. There are two distinct validation
issues that must be addressed. First is the “proof of biology” based
on demonstration that the biological change being measured is
related to the relevant target engagement. Second is the “proof of
concept” that the biological change has relevance for clinical out-
come [9]. Relationships seen with the natural history of the disease
should not be assumed to hold after therapeutic modulations [85].
Testing for any changes in this relationship with pharmacological
modulation is important to ensure that the biomarker remains
plausibly related to a clinically meaningful outcome.
In general, validation of a candidate biomarker’s surrogacy
involves the demonstration that it possesses the properties required
for its use as a substitute for a true endpoint. A surrogate can be
used at the individual subject level when there is a perfect associa-
tion between the surrogate and the final endpoint after adjustment
for treatment. This criterion essentially requires the surrogate
variable to ‘capture’ any relationship between the treatment and the
true endpoint, a notion that can be operationalized by requiring the
true endpoint rate at any follow-up time to be independent of treat-
ment, given the preceding history of the surrogate variable [86].

6 Current Limitations and Some Future Extensions of Pharmacological fMRI

Although there is real promise for pharmacological fMRI, there are

major general challenges to meaningful, quantitative interpretations
of measures that need to be considered in planning applications. A
first challenge is to distinguish disease or pharmacodynamic effects
on hemodynamic coupling from those on neuronal activity and
metabolism [11]. Some limitations to interpretation of the BOLD
response can be addressed with use of complementary forms of MRI
828 Paul M. Matthews

contrast. For example, direct measures of brain blood flow can be

made using noninvasive “arterial spin labeling” MRI methods and
the BOLD signal can be calibrated as a measure of local oxygen
extraction for quantitative MRI [87]. However, even without this
uncertainty, the relationship of blood flow changes to modulation of
presynaptic activity can change with physiological (and, potentially,
pharmacological) context. Even the relative direction of relative acti-
vation in disease states may be difficult to interpret precisely. For
example, reduced activation may reflect brain functional impairment
[88] or improved efficiency [89] in different contexts. Experimental
designs need to recognize this uncertainty and incorporate elements
that allow meaningfully specific interpretations, e.g., by studying
dose–response relations, parametric activity relationships and behav-
ioral correlates [90]. A more direct approach is to link pharmaco-
logical fMRI with electrophysiological measures [91].
General validation of methods to enable their wider use will
depend on standardization across sites, reliability and repeatability,
and the development of validated quantification methods, ideally
largely automated to minimize needs for harmonization of user
training. Practical considerations also need to be address to enable
integrated use of the most accurate and efficient combination of
markers and optimization of costs for the clinical trial environment
[92]. Greater openness and sharing of data would be an important
enabler of this. These steps, while still not yet part of routine prac-
tice in the academic laboratories in which advanced clinical imag-
ing is most often performed, need not stifle innovation, which can
progress in parallel, but is essential of translation of this promising
method as a major tool for drug development is to be achieved.

Acknowledgements

The author gratefully acknowledges support from the Edmond

J. Safra Foundation and from Lily Safra, the Medical Research
Council, the MS Society of Great Britain, the Progressive MS
Alliance and the Imperial College Biomedical Research Council.
He is an NIHR Senior Investigator. He has received additional
research funding from Biogen and GlaxoSmithKline, has received
consultancy funding to his University from Biogen, Novartis,
Adephi Communications and IXICO and honoraria or educational
funding support from Biogen and Novartis.

References
1. Trusheim MR, Berndt ER, Douglas FL (2007) Drug Discov Today Technol
Stratified medicine: strategic and economic 10(3):e343–e350
implications of combining drugs and clinical bio- 3. Uppoor RS et al (2008) The use of imaging in
markers. Nat Rev Drug Discov 6(4):287–293 the early development of neuropharmacologi-
2. Matthews PM et al (2013) Technologies: cal drugs: a survey of approved NDAs. Clin
preclinical imaging for drug development. Pharmacol Ther 84(1):69–74
 Pharmacological Applications of fMRI 829

4. Borsook D, Becerra L, Fava M (2013) Use of 18. Goense JB, Logothetis NK (2008)
functional imaging across clinical phases in Neurophysiology of the BOLD fMRI signal in
CNS drug development. Transl Psychiatry awake monkeys. Curr Biol 18(9):631–640
3:e282 19. Smith SM (2012) The future of FMRI connec-
5. Mathiesen C et al (1998) Modification of tivity. Neuroimage 62(2):1257–1266
activity-dependent increases of cerebral blood 20. FSL—FslWiki (2015) http://fsl.fmrib.ox.ac.
flow by excitatory synaptic activity and spikes in uk/fsl/fslwiki/%5D
rat cerebellar cortex. J Physiol 512(Pt 21. Iannetti GD, Wise RG (2007) BOLD func-
2):555–566 tional MRI in disease and pharmacological
6. Logothetis NK (2003) The underpinnings of studies: room for improvement? Magn Reson
the BOLD functional magnetic resonance Imaging 25(6):978–988
imaging signal. J Neurosci 23(10):3963–3971 22. Beckmann CF, Smith SM (2005) Tensorial
7. Mukamel R et al (2005) Coupling between extensions of independent component analysis
neuronal firing, field potentials, and FMRI in for multisubject FMRI analysis. Neuroimage
human auditory cortex. Science 25(1):294–311
309(5736):951–954 23. Hodkinson DJ et al (2012) Differential effects
8. Caesar K, Thomsen K, Lauritzen M (2003) of anaesthesia on the phMRI response to acute
Dissociation of spikes, synaptic activity, and ketamine challenge. Br J Med Med Res
activity-dependent increments in rat cerebellar 2(3):373–385
blood flow by tonic synaptic inhibition. Proc 24. Littlewood CL et al (2006) Using the BOLD
Natl Acad Sci U S A 100(26):16000–16005 MR signal to differentiate the stereoisomers of
9. Minzenberg MJ (2012) Pharmacological MRI ketamine in the rat. Neuroimage
approaches to understanding mechanisms of 32(4):1733–1746
drug action. Curr Top Behav Neurosci 25. Roberts TJ, Williams SC, Modo M (2008) A
11:365–388 pharmacological MRI assessment of dizocil-
10. Girouard H, Iadecola C (2006) Neurovascular pine (MK-801) in the 3-nitroproprionic acid-
coupling in the normal brain and in hyperten- lesioned rat. Neurosci Lett 444(1):42–47
sion, stroke, and Alzheimer disease. J Appl 26. Miyamoto S et al (2000) Effects of ketamine,
Physiol (1985) 100(1):328–335 MK-801, and amphetamine on regional brain
11. Suri S et al (2015) Reduced cerebrovascular 2-deoxyglucose uptake in freely moving mice.
reactivity in young adults carrying the APOE Neuropsychopharmacology 22(4):400–412
epsilon4 allele. Alzheimers Dement 11(6):648– 27. Homayoun H, Jackson ME, Moghaddam B
657.e1 (2005) Activation of metabotropic glutamate
12. Glodzik L et al (2013) Cerebrovascular reactiv- 2/3 receptors reverses the effects of NMDA
ity to carbon dioxide in Alzheimer’s disease. receptor hypofunction on prefrontal cortex
J Alzheimers Dis 35(3):427–440 unit activity in awake rats. J Neurophysiol
13. Kwong KK et al (1992) Dynamic magnetic 93(4):1989–2001
resonance imaging of human brain activity dur- 28. Deakin JF et al (2008) Glutamate and the neu-
ing primary sensory stimulation. Proc Natl ral basis of the subjective effects of ketamine: a
Acad Sci U S A 89(12):5675–5679 pharmaco-magnetic resonance imaging study.
14. Ogawa S et al (1990) Oxygenation-sensitive Arch Gen Psychiatry 65(2):154–164
contrast in magnetic resonance image of rodent 29. Gottesman II, Gould TD (2003) The endo-
brain at high magnetic fields. Magn Reson phenotype concept in psychiatry: etymology
Med 14(1):68–78 and strategic intentions. Am J Psychiatry
15. Belliveau JW et al (1991) Functional mapping 160(4):636–645
of the human visual cortex by magnetic reso- 30. Callicott JH et al (2005) Variation in DISC1
nance imaging. Science 254(5032):716–719 affects hippocampal structure and function and
16. Mezue M et al (2014) Optimization and reli- increases risk for schizophrenia. Proc Natl Acad
ability of multiple postlabeling delay pseudo- Sci U S A 102(24):8627–8632
continuous arterial spin labeling during rest 31. Egan MF et al (2004) Variation in GRM3
and stimulus-induced functional task activa- affects cognition, prefrontal glutamate, and
tion. J Cereb Blood Flow Metab risk for schizophrenia. Proc Natl Acad Sci U S
34(12):1919–1927 A 101(34):12604–12609
17. Brookes MJ et al (2011) Investigating the elec- 32. Egan MF et al (2001) Effect of COMT
trophysiological basis of resting state networks Val108/158 Met genotype on frontal lobe
using magnetoencephalography. Proc Natl function and risk for schizophrenia. Proc Natl
Acad Sci U S A 108(40):16783–16788 Acad Sci U S A 98(12):6917–6922
830 Paul M. Matthews

33. David SP et al (2005) Ventral striatum/nucleus functional MRI. Proc Natl Acad Sci U S A
accumbens activation to smoking-related pic- 101(13):4637–4642
torial cues in smokers and nonsmokers: a func- 48. Raichle ME, Snyder AZ (2007) A default
tional magnetic resonance imaging study. Biol mode of brain function: a brief history of an
Psychiatry 58(6):488–494 evolving idea. Neuroimage 37(4):1083–1090,
34. Myrick H et al (2004) Differential brain discussion 1097–1099
activity in alcoholics and social drink- 49. Petrella JR et al (2011) Default mode network
ers to alcohol cues: relationship to craving. connectivity in stable vs progressive mild cog-
Neuropsychopharmacology 29(2):393–402 nitive impairment. Neurology 76(6):511–517
35. Reuter J et al (2005) Pathological gambling is 50. Sheline YI, Raichle ME (2013) Resting
linked to reduced activation of the mesolimbic state functional connectivity in preclini-
reward system. Nat Neurosci 8(2):147–148 cal Alzheimer’s disease. Biol Psychiatry
36. Paulus MP, Tapert SF, Schuckit MA 74(5):340–347
(2005) Neural activation patterns of 51. Szewczyk-Krolikowski K et al (2014)
methamphetamine-dependent subjects dur- Functional connectivity in the basal ganglia
ing decision making predict relapse. Arch Gen network differentiates PD patients from con-
Psychiatry 62(7):761–768 trols. Neurology 83(3):208–214
37. Kaufman JN et al (2003) Cingulate hypoactiv- 52. Honey GD et al (2003) The functional neu-
ity in cocaine users during a GO-NOGO task as roanatomy of schizophrenic subsyndromes.
revealed by event-related functional magnetic res- Psychol Med 33(6):1007–1018
onance imaging. J Neurosci 23(21):7839–7843 53. Baghai TC, Moller HJ, Rupprecht R (2006)
38. Forman SD et al (2004) Opiate addicts lack Recent progress in pharmacological and non-
error-dependent activation of rostral anterior pharmacological treatment options of major
cingulate. Biol Psychiatry 55(5):531–537 depression. Curr Pharm Des 12(4):503–515
39. Heinz A et al (2004) Correlation between 54. Canli T et al (2005) Amygdala reactivity to
dopamine D(2) receptors in the ventral stria- emotional faces predicts improvement in major
tum and central processing of alcohol cues and depression. Neuroreport 16(12):1267–1270
craving. Am J Psychiatry 161(10):1783–1789 55. Killgore WD, Yurgelun-Todd DA (2006)
40. Cole DM et al (2012) Orbitofrontal connec- Ventromedial prefrontal activity correlates
tivity with resting-state networks is associated with depressed mood in adolescent children.
with midbrain dopamine D3 receptor availabil- Neuroreport 17(2):167–171
ity. Cereb Cortex 22(12):2784–2793 56. Anand A et al (2005) Antidepressant effect on
41. Egan MF et al (2003) The BDNF val66met connectivity of the mood-regulating circuit:
polymorphism affects activity-dependent secre- an FMRI study. Neuropsychopharmacology
tion of BDNF and human memory and hip- 30(7):1334–1344
pocampal function. Cell 112(2):257–269 57. Allen P et al (2012) Transition to psycho-
42. Lu B, Nagappan G, Lu Y (2014) BDNF and sis associated with prefrontal and subcorti-
synaptic plasticity, cognitive function, and dys- cal dysfunction in ultra high-risk individuals.
function. Handb Exp Pharmacol 220:223–250 Schizophr Bull 38(6):1268–1276
43. Rabiner EA et al (2011) Pharmacological dif- 58. Schmidt A et al (2014) Approaching a network
ferentiation of opioid receptor antagonists by connectivity-driven classification of the psychosis
molecular and functional imaging of target occu- continuum: a selective review and suggestions
pancy and food reward-related brain activation for future research. Front Hum Neurosci 8:1047
in humans. Mol Psychiatry 16(8):826–835, 785 59. del Campo N, Muller U, Sahakian BJ (2012)
44. Ziauddeen H et al (2013) Effects of the mu- Neural and behavioral endophenotypes in
opioid receptor antagonist GSK1521498 on ADHD. Curr Top Behav Neurosci 11:65–91
hedonic and consummatory eating behaviour: a 60. Hasler G, Northoff G (2011) Discovering
proof of mechanism study in binge-eating obese imaging endophenotypes for major depression.
subjects. Mol Psychiatry 18(12):1287–1293 Mol Psychiatry 16(6):604–619
45. Engel RH, Kaklamani VG (2007) HER2- 61. Savitz JB, Drevets WC (2009) Imaging phe-
positive breast cancer: current and future treat- notypes of major depressive disorder: genetic
ment strategies. Drugs 67(9):1329–1341 correlates. Neuroscience 164(1):300–330
46. Matthews PM et al (2014) The emerging 62. Keener MT, Phillips ML (2007) Neuroimaging
agenda of stratified medicine in neurology. Nat in bipolar disorder: a critical review of current
Rev Neurol 10(1):15–26 findings. Curr Psychiatry Rep 9(6):512–520
47. Greicius MD et al (2004) Default-mode 63. Lee MC et al (2013) Amygdala activity con-
network activity distinguishes Alzheimer’s tributes to the dissociative effect of cannabis on
disease from healthy aging: evidence from pain perception. Pain 154(1):124–134
 Pharmacological Applications of fMRI 831

64. Lee MC, Tracey I (2013) Imaging pain: a modulation by rivastigmine. Brain 126(Pt
potent means for investigating pain mecha- 12):2750–2760
nisms in patients. Br J Anaesth 111(1):64–72 79. Matthews PM, Johansen-Berg H, Reddy H
65. Mazzola V et al (2010) Affective response to (2004) Non-invasive mapping of brain func-
a loved one’s pain: insula activity as a func- tions and brain recovery: applying lessons from
tion of individual differences. PLoS One cognitive neuroscience to neurorehabilitation.
5(12):e15268 Restor Neurol Neurosci 22(3–5):245–260
66. Derbyshire SW, Whalley MG, Oakley DA 80. Borsook D et al (2012) Decision-making using
(2009) Fibromyalgia pain and its modulation fMRI in clinical drug development: revisiting
by hypnotic and non-hypnotic suggestion: an NK-1 receptor antagonists for pain. Drug
fMRI analysis. Eur J Pain 13(5):542–550 Discov Today 17(17–18):964–973
67. Fairhurst M et al (2012) An fMRI study 81. Leknes S et al (2013) The importance of con-
exploring the overlap and differences between text: when relative relief renders pain pleasant.
neural representations of physical and recalled Pain 154(3):402–410
pain. PLoS One 7(10):e48711 82. Schwarz AJ et al (2007) In vivo mapping of
68. Murray D, Stoessl AJ (2013) Mechanisms and functional connectivity in neurotransmitter sys-
therapeutic implications of the placebo effect tems using pharmacological MRI. Neuroimage
in neurological and psychiatric conditions. 34(4):1627–1636
Pharmacol Ther 140(3):306–318 83. Batterham RL et al (2007) PYY modulation of
69. Denk F, McMahon SB, Tracey I (2014) Pain cortical and hypothalamic brain areas predicts
vulnerability: a neurobiological perspective. feeding behaviour in humans. Nature
Nat Neurosci 17(2):192–200 450(7166):106–109
70. Baliki MN et al (2012) Corticostriatal func- 84. Borsook D, Becerra L, Hargreaves R (2006) A
tional connectivity predicts transition to chronic role for fMRI in optimizing CNS drug devel-
back pain. Nat Neurosci 15(8):1117–1119 opment. Nat Rev Drug Discov 5(5):411–424
71. Smucny J, Wylie KP, Tregellas JR (2014) 85. Cummings JL (2010) Integrating ADNI
Functional magnetic resonance imag- results into Alzheimer’s disease drug develop-
ing of intrinsic brain networks for transla- ment programs. Neurobiol Aging
tional drug discovery. Trends Pharmacol Sci 31(8):1481–1492
35(8):397–403 86. Prentice RL (1989) Surrogate endpoints in
72. Vollm BA et al (2004) Methamphetamine clinical trials: definition and operational crite-
activates reward circuitry in drug naive ria. Stat Med 8(4):431–440
human subjects. Neuropsychopharmacology 87. Hoge RD et al (1999) Linear coupling between
29(9):1715–1722 cerebral blood flow and oxygen consumption
73. Gerdelat-Mas A et al (2005) Chronic admin- in activated human cortex. Proc Natl Acad Sci
istration of selective serotonin reuptake inhibi- U S A 96(16):9403–9408
tor (SSRI) paroxetine modulates human 88. Rombouts SA et al (2003) Loss of frontal
motor cortex excitability in healthy subjects. fMRI activation in early frontotemporal
Neuroimage 27(2):314–322 dementia compared to early AD. Neurology
74. Pariente J et al (2001) Fluoxetine modulates 60(12):1904–1908
motor performance and cerebral activation of 89. Floyer-Lea A, Matthews PM (2004) Changing
patients recovering from stroke. Ann Neurol brain networks for visuomotor control with
50(6):718–729 increased movement automaticity.
75. Goekoop R et al (2004) Challenging the cho- J Neurophysiol 92(4):2405–2412
linergic system in mild cognitive impairment: 90. Cader S et al (2006) Reduced brain functional
a pharmacological fMRI study. Neuroimage reserve and altered functional connectivity in
23(4):1450–1459 patients with multiple sclerosis. Brain 129(Pt
76. Farahani K et al (1999) Contemporaneous pos- 2):527–537
itron emission tomography and MR imaging at 91. Lachaux JP et al (2007) Relationship between
1.5 T. J Magn Reson Imaging 9(3):497–500 task-related gamma oscillations and BOLD sig-
77. Wilkinson D, Halligan P (2004) The relevance nal: new insights from combined fMRI and
of behavioural measures for functional-imaging intracranial EEG. Hum Brain Mapp
studies of cognition. Nat Rev Neurosci 28(12):1368–1375
5(1):67–73 92. Merlo Pich E et al (2014) Imaging as a bio-
78. Parry AM et al (2003) Potentially adaptive marker in drug discovery for Alzheimer’s dis-
functional changes in cognitive processing for ease: is MRI a suitable technology? Alzheimers
patients with multiple sclerosis and their acute Res Ther 6(4):51
 Chapter 27

Application of fMRI to Monitor Motor Rehabilitation

Steven C. Cramer and Jessica M. Cassidy

Abstract
Motor deficits contribute to disability in a number of neurological conditions. A wide range of emerging
restorative therapies have the potential to reduce this by favorably modifying function. In many medical
contexts, a study of target organ function improves efficacy of a therapeutic intervention. However, the
optimal methods to prescribe a restorative therapy in the setting of central nervous system (CNS) disease
are not clear. Brain mapping studies have the potential to provide useful insights in this regard. Examples of
restorative therapies are provided, and human trials are summarized whereby brain mapping data have
proven useful in promoting motor improvements in subjects with a neurological condition. A number of
forms of brain mapping metrics are under study, including those emphasizing network connectivity obtained
using resting-state fMRI. In some cases, brain mapping findings that correlate with better outcome with
spontaneous behavioral recovery correspond to findings that predict better treatment response in the con-
text of a clinical trial. Similarities across CNS conditions, such as stroke and multiple sclerosis, are discussed.
Further studies are needed to understand which methods have the greatest value to monitor, predict, triage,
and dose restorative therapies in trials that aim to reduce motor, and other neurological, deficits.

Key words Functional neuroimaging, Brain mapping, Stroke, Motor system, Recovery, Repair,
Plasticity, Treatment

1 Motor Deficits and Restorative Therapies

Motor deficits are a major contributor to disability in the setting of

a number of neurological diseases marked by focal central nervous
system (CNS) injury, such as stroke, multiple sclerosis (MS), spinal
cord injury (SCI), and traumatic brain injury [1]. In general,
motor deficits show some degree of spontaneous improvement in
the weeks following the insult. Spontaneous recovery is generally
incomplete, however.
A number of therapies are in development to promote recovery
in patients with motor deficits after a CNS insult. Some target the
acute phase of injury, when the brain is galvanized and produces
growth-related substances at levels reminiscent of development.
Other therapies target patients in the chronic phase. Regardless, the
goal of such therapies is not to salvage injured tissue, rather to pro-
mote repair and restore function.
Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_27, © Springer Science+Business Media New York 2016

833
834 Steven C. Cramer and Jessica M. Cassidy

Many forms of restorative therapy are under study. Examples

include small molecules [2–5], immune approaches such as via
neutralization of the axon growth inhibitor Nogo-A with mono-
clonal antibodies [6], growth factors [7–15], cell-based methods
[16–19], electromagnetic stimulation [20–24], neuroprosthetics
[25, 26], and methods based on various forms of therapy and prac-
tice [27–37]. Issues of inter-individual variability in response to
many of these therapies make it difficult to assess treatment feasi-
bility and/or efficacy. Examination of these restorative therapies
requires judicious analysis of the intended target, the brain.
A key thesis of this chapter is that optimal prescription of such
restorative therapies will be achieved by probing the state of the
brain. Clinical trials often enroll patients based on demographic or
behavioral measures. However, these are only an approximation of
the type of brain state information that is important to promoting
repair and recovery.
There are examples in other medical specialties a measure of tar-
get organ function is obtained in addition to behavior and demo-
graphic data in order to maximize therapeutic gains. For example,
hypothyroidism is optimally treated not by serial behavioral exam,
rather by serial measures of pituitary–thyroid axis via serum TSH.
Treatment of myeloproliferative and related hematological syndromes
is ultimately dosed not by behavioral or demographic measures, but at
least in part on the basis of serial measure of the cell population of
interest. Cardiac arrhythmias and coronary artery disease are often
assessed by evaluating cardiac function, e.g., in the setting of electro-
physiological studies, exercise, or a sympathomimetic challenge. These
practices suggest the general principle that some form of study of the
therapy’s target organ might be useful for optimizing therapy. In the
setting of focal CNS injury, a technique such as fMRI might therefore
be useful for a restorative therapy to assist with study entry criteria, to
define optimal therapy dose or duration, or to serve as a biological
marker of treatment effect. This issue is considered below with respect
to three conditions characterized by an acute focal neurological insult.

2 Stroke

The motor system is among the most frequently affected domains by

stroke [38, 39]. Duncan et al. [40, 41] found that the most dramatic
improvements occurred in the first 30 days post-stroke, though sig-
nificant improvement continued to occur up to 90 days after stroke
in patients with more severe deficits. Nakayama et al. [42] measured
arm disability and found that maximum arm function was achieved
by 80 % of patients within 3 weeks, and by 95 % of patients within 9
weeks. Wade et al. [43] also found that significant improvement was
mainly seen across the first 3 months after stroke. Despite these
improvements, residual motor deficits remain in approximately half
of patients in the chronic phase of stroke [38, 39].
 Application of fMRI to Monitor Motor Rehabilitation 835

A number of studies [44–46] have examined the brain events

underlying the spontaneous recovery of motor behavior that does
arise after stroke. In sum, stroke-related injury is associated with
reduced activation, function, and neurophysiological responsiveness
in injured (or for deep strokes, the overlying/corresponding) pri-
mary cortex. The best spontaneous return of behavior is associated
with resolution of these reductions, i.e., return of activity in primary
cortex, sometimes with particular shifts in the site of activation.
Several compensatory responses may also contribute to spontaneous
behavioral recovery, including increased activation in secondary
areas that are normally connected to the injured zones in a distrib-
uted network, as well as a shift in interhemispheric laterality towards
the contralesional hemisphere. The larger the injury or greater the
deficits, the more these compensatory events are seen. These com-
pensatory responses are tricks of the desperate, but in patients with
injury-related deficits they are better present than absent [47–51].
These events that underlie spontaneous recovery are important
because in some cases they are the same measures used to guide
optimization of therapy-derived recovery.

2.1 Use of Functional One study used functional neuroimaging in a clinical trial of a
Neuroimaging to Guide restorative intervention to extract data from an fMRI scan in order
a Restorative to guide details of decision-making during therapy [24, 52]. An
Intervention in fMRI scan was used to identify the centroid of ipsilesional primary
Patients with Stroke motor cortex activation when patients with stroke moved the
affected hand. This information then guided neurosurgical place-
ment of an investigational epidural cortical stimulation device over
ipsilesional motor cortex. Using this approach, patients receiving
stimulation plus rehabilitation therapy showed significantly greater
arm motor gains than patients receiving rehabilitation therapy
alone. A similar approach was used in studies based on transcranial
magnetic stimulation (TMS) to identify the optimal physiological
representation site for hand motor function. These studies found
repetitive TMS to be useful for improving motor function after
stroke [20, 21].

2.2 Use of Functional An additional application of functional neuroimaging in the setting

Neuroimaging of restorative therapy is to predict behavioral response to treatment.
to Predict Response Several studies have examined this issue [53–60]. For example,
to a Restorative Koski et al. [56], using TMS, and Dong et al. [54], using fMRI,
Intervention have found that changes in brain function early into therapy pre-
in Patients with Stroke dict behavioral gains measured at the end of therapy. Note that in
both cases, the findings that predicted better treatment response
(improved motor evoked response in affected hand with TMS of
ipsilesional hemisphere, and increased laterality of fMRI activation,
i.e., towards the ipsilesional hemisphere, with movement of the
affected hand, respectively) correspond to the findings correlating
better outcome with spontaneous behavioral recovery. This latter
836 Steven C. Cramer and Jessica M. Cassidy

pair of studies also hints at the potential use of human brain map-
ping measures to identify the dose of a restorative therapy in indi-
vidual patients. For example, could a TMS or fMRI measure of
brain function inform a clinician of the likelihood that the brain is
receptive to further change that supports behavioral gains? In this
regard, note that a probe of brain plasticity, such as might be used
to predict treatment response to a restorative intervention, can be
developed even in the setting of severe deficits, such as complete
plegia [61].
Another study found that fMRI had independent value for pre-
dicting treatment response in a restorative therapy trial in patients
with chronic stroke [57]. This study used a multivariate model to
examine the specific ability of a baseline fMRI to predict trial-related
behavioral gains, and compared this fMRI predictive ability directly
to a number of other baseline measures. Patients in this study each
underwent baseline clinical and functional MRI assessments,
received 6 weeks of rehabilitation therapy with or without investi-
gational motor cortex stimulation, then had repeat assessments.
Across all patients, univariate analyses found that several baseline
measures had predictive value for trial-related gains. However, mul-
tiple linear regression modeling found that only two variables
remained significant predictors: degree of motor cortex activation
on fMRI (lower motor cortex activation predicted larger gains) and
arm motor function (greater arm function predicted larger gains).
This study emphasized that an assessment of brain function can be
a unique source of information for clinical decision-making in the
setting of restorative therapy after stroke. Interestingly, clinical
gains during study participation were paralleled by boosts in motor
cortex activity, the latter detected via serial fMRI scanning, suggest-
ing that lower baseline cortical activity in some patients likely repre-
sents under-use of an available cortical resource.
Burke Quinlan and colleagues [60] also utilized a multivariate
model that encompassed various demographic, behavioral, and
neuroimaging measures to determine which metrics best predicted
behavioral gains following a three-week upper-extremity robotic
therapy program in individuals with chronic stroke. Bivariate
screening revealed significant correlations between improvement
in upper-extremity status with baseline MRI and diffusion tensor
imaging measures of brain injury (i.e. infarct volume, cortical
injury, percentage injury to corticospinal tract), task-evoked fMRI
measures of cortical function (i.e. ipsilesional primary motor cor-
tex area (M1) activation), and resting fMRI measures of ipsile-
sional/contralesional M1 connectivity. Subsequent multivariate
linear regression modeling revealed that the percentage injury to
corticospinal tract and ipsilesional/contralesional M1 connectiv-
ity accounted for 44 % of the variance in treatment gains. Brain-
based measures, therefore, depicted better predictive quality than
the more conventional behavioral and demographic measures.
 Application of fMRI to Monitor Motor Rehabilitation 837

A similar study focusing on lower-extremity motor gains after a

course of physical therapy showed that baseline lower-extremity
status and ipsilesional foot primary sensorimotor cortex activation
volume contributed to 63 % of the variance in gait velocity change
[59]. Combined, these studies [59, 60] provide supporting evi-
dence for the use of fMRI measures as potential biomarkers for
rehabilitation gains. There are a number of important variables
that differ across patients, study designs, fMRI acquisition and
analysis methods, and more. As such, further studies are needed to
understand the extent to which the above findings generalize
across other stroke studies.

2.3 Use of Functional Carey et al. [62] found that a population of subjects with chronic
Neuroimaging to stroke, when performing a finger tracking task with the stroke-
Investigate the affected hand, had activation within contralesional brain regions,
Biological i.e., regions that were primarily ipsilateral to movement. After
Mechanisms of training at this task, the normal pattern of laterality of brain activa-
Restorative tion was restored, with activation shifting to ipsilesional brain
Intervention Effects regions, i.e., contralateral to movement, and thereby more closely
in Patients with Stroke resembling findings in healthy control subjects. In this landmark
study, functional neuroimaging provided insights into the mecha-
nistic effects of treatment. Since then, other studies have shown
varying modulatory effects of cortical activation following Botox
[63], constraint-induced movement therapy (CIMT) [64], visuo-
motor tracking task practice [65], implicit motor learning [66],
real-time fMRI feedback training [67], and noninvasive brain stim-
ulation in individuals with chronic stroke [68].
Two meta-analyses [69, 70] extend these results by examining
studies that have employed functional neuroimaging as a biological
marker of treatment effects targeting the motor system after stroke.
A review of 24 studies utilizing sensorimotor tasks in 255 patients
found higher activation in the contralesional M1 (relative to
healthy controls) that decreased over time but was unrelated to
motor outcome. Reorganization consistent with increased
ipsilesional M1 and medial premotor cortex activation was associ-
ated with positive recovery; whereas, increased activation of the
cerebellar vermis was associated with negative recovery. These con-
clusions highlight both beneficial and detrimental examples of
neuroplastic reorganization following stroke as demonstrated by
shifts in premotor and cerebellar vermis activation, respectively. An
earlier meta-analysis that reviewed 13 studies of 121 patients per-
mitted drawing a number of conclusions [70]. Motor deficits have
been most often studied, in part because of their substantial contri-
bution to overall disability after stroke, and in part because of their
relatively high prevalence. Most published studies have focused on
patients with good to excellent outcome at baseline since they were
more able to perform the motor tasks required to probe brain
function. Consequently, less is known about the functional
838 Steven C. Cramer and Jessica M. Cassidy

anatomy of therapy-induced recovery processes in the large

population of patients with more severe deficits after stroke despite
the great need for further study of restorative interventions in this
population. Very few studies have used functional imaging to
examine treatment effects during the first few months after stroke,
when spontaneous behavioral recovery is at its greatest. The effects
that many key variables such as lesion site, recovery level, gender,
and age have on the performance of functional neuroimaging in
this context requires further study. Studies could be improved by
incorporating measures of injury and/or physiology.
Baseline differences in the stroke population under study can
have a significant impact on the informative value of functional
neuroimaging measures in the setting of a clinical trial. One suc-
cessful CIMT study was associated with decreased inter-hemispheric
laterality in a study of weaker patients [71], while a second study
found increased laterality in a study of stronger patients [72] with
chronic stroke. Additionally, Könönen et al. [73] observed
increased sensorimotor area activation after a 2-week CIMT pro-
gram amongst individuals with poorer baseline hand function.
Investigators found no change in hemispheric laterality for premo-
tor and sensorimotor regions of interest.
This divergence in findings emphasizes how differences in a
single variable, such as baseline motor status, might influence the
utility of brain mapping in the setting of a clinical trial, and high-
lights the need for further studies in this regard.

3 Multiple Sclerosis

Motor deficits are common in MS. For example, across a broad

population of subjects with MS, the median time to reach irrevers-
ible limited walking ability for more than 500 m without aid or rest
is 8 years, to walk with unilateral support no more than 100 m
without rest is 20 years, and to walk no more than 10 m without
rest while leaning against a wall or support is 30 years [74]. Upper
extremity motor deficits, such as those related to ataxia and paresis,
are also a common source of disability.
Brain plasticity is an important determinant in MS in at least
two contexts. First, steady destruction of myelin and of axons over
years results in disability. During this period, reorganization of brain
function can reduce the impact of such injury on behavioral status.
Second, approximately 85 % of patients with MS have a relapsing,
remitting course [75], in which a relapse peaks over 1–2 months
and then improves over a similar time period. The resolution of
these MS flares has been attributed to a number of brain events,
such as neurological reserve and resolution of inflammatory insult
[75], and a number of studies suggest that brain plasticity is also
important [76]. Note too that there are numerous asymptomatic
brain lesions for each symptomatic one in most patients with MS, a
 Application of fMRI to Monitor Motor Rehabilitation 839

fact that might further support the importance of brain plasticity in

maintenance of behavioral status in this condition.
Brain plasticity thus is likely important to motor status in MS,
by minimizing the debilitating effects of MS injury accrual over
time, and by promoting recovery from silent or symptomatic MS
flares. A number of studies have provided insights into the brain
events important in this regard, with substantial overlap as com-
pared to findings in patients with stroke. This information gains
importance in the current discussion because events important to
maximizing behavioral status in the natural course of the disease
are likely to be many of the same measures whose measurement
can guide optimization of therapy-derived recovery.
Studies of brain plasticity in MS have found that, early in the
course of the disease, brain activation is larger and more wide-
spread as compared to healthy controls. Later in the disease, later-
ality of activation is reduced (i.e., activation is more bilateral)
[77–79], akin to stroke patients who have larger infarcts or greater
deficits [80, 81]. Bilateral sensorimotor cortical regions are acti-
vated to a greater extent in the setting of MS-related white matter
injury [82, 83]. This increased degree of bilateral organization per-
sists to the greatest extent in subjects with persistent deficits after
an acute MS relapse, and returns to a normal, lateralized (i.e.,
contralateral-predominant) form of organization in subjects with
the least degree of persistent disability [84, 85]. The pattern of
brain activation during performance of a simple motor task in sub-
jects recovered from stroke has been considered similar to the pat-
tern seen in healthy subjects during performance of a complex task
[86]; a similar analogy has been made in subjects with MS [87].
Interestingly, when a subset of subjects with mild MS, depict-
ing no outward clinical deficits or disability despite long-standing
diagnosis, performed a combination of sensory, cognitive, and
motor tasks during an fMRI session, investigators found increased
activation of cognitive-related regions relative to controls [88].
Further, when comparing mild vs. more severe relapsing-remitting
MS phenotypes during an fMRI hand motor task, those subjects
with mild MS showed increased activation throughout sensorimo-
tor regions that correlated with increasing lesion volume and
decreasing cortical volume [89]. These findings reaffirm the notion
that preservation of physical function in MS relies on the deploy-
ment of compensatory brain plasticity mechanisms involving
enhanced recruitment of cognitive and sensorimotor areas.

3.1 Use of Functional The extent to which these spontaneous changes in brain function
Neuroimaging to after MS can be used to monitor therapeutic interventions has
Investigate the been assessed in several small studies. One study tested the effects
Biological Mechanism of increased cholinergic tone on the pattern of fMRI activation
of Restorative during performance of a cognitive task, the Stroop test. At base-
Intervention Effects line, patients with MS and moderate disability had similar behav-
in Patients with MS ioral performance as compared to controls, but on fMRI showed
840 Steven C. Cramer and Jessica M. Cassidy

increased left medial prefrontal, and decreased right frontal, activa-

tion. Treatment with the cholinesterase inhibitor rivastigmine nor-
malized both of these fMRI abnormalities in patients, but had little
effect on a small cohort of healthy control subjects [90].
In another study, administration of 3,4-diaminopyridine to
patients with MS was associated with increased activation in senso-
rimotor cortex and SMA ipsilateral to movement. This pattern is
the reverse of the laterality pattern seen in normals but might cor-
respond to effects of increased injury [80, 81] or task complexity
[91, 92]. TMS measures were also affected by treatment, showing
a drug-induced reduction in intracortical inhibition and increase in
intracortical facilitation [93]. The relationship that these changes
had with behavioral effects of drug administration was not reported.
A few studies have examined experience-dependent plasticity in
MS following motor practice [94, 95]. In one study, subjects with MS
practiced a visuomotor tracking task daily for approximately two
weeks [94]. Investigators examined both short- and long-term (min-
utes and days, respectively) practice effects. They discovered an asso-
ciation between tracking improvement and attenuated blood-oxygen
level dependent (BOLD) signal bilatera in sensorimotor, premotor,
cingulate, temporal, and parahippocampal cortices for short-term
practice improvement and trends of smaller BOLD signal in superior
parietal lobule and occipital cortex for long-term improvement.
Importantly, tracking improvement was independent of MRI-derived
brain pathology metrics; meaning, that improvement was not dictated
by the extent of structural brain damage. Subjects with MS were also
studied with fMRI before and after 30 min of thumb flexion training
[95]. Subjects with MS did not show a training-induced reduction in
contralateral primary sensorimotor and parietal association cortices
that healthy controls illustrated across the training period. Apart from
implications regarding brain function in the setting of MS, these find-
ings demonstrate a limitation of this paradigm for probing short-term,
experience-dependent plasticity in this population [95]. Consideration
of task complexity is also important, and may further explain these
contrasting findings [94, 95] of short-term experience-dependent
plasticity or lack thereof in MS.

4 Spinal Cord Injury

Though SCI can be associated with a range of injury patterns, motor

deficits are generally a prominent feature. At the time of discharge
from initial SCI, the most frequent neurologic category is incom-
plete tetraplegia (34.1 %), followed by complete paraplegia (23.0 %),
complete tetraplegia (18.3 %), and incomplete paraplegia (18.5 %).
Less than 1 % of persons experience complete neurologic recovery by
hospital discharge. By 10 years after SCI, 68 % of persons with para-
plegia, and 76 % of those with tetraplegia, are unemployed [96].
 Application of fMRI to Monitor Motor Rehabilitation 841

Subjects with SCI generally show modest spontaneous sensory

and motor improvement in the first 3–6 months following injury
[97, 98], although significant improvement beyond the first year
post-SCI is uncommon [99]. Motor deficits are thus common and
persistent after SCI, and these impact a number of health, quality
of life, and other issues in subjects with SCI [100–102].

4.1 Use of Functional There has been limited study of the CNS mechanisms underlying
Neuroimaging to spontaneous motor improvement during the months following
Investigate the SCI. Jurkiewicz et al. [103] examined the acute-to-chronic time-
Biological Mechanism course of post-SCI sensorimotor reorganization in four individuals
of Restorative with tetraplegia over a 12-month period. Shortly after injury, sub-
Intervention Effects jects with SCI demonstrated a similar volume of contralateral M1
in Patients with SCI activation as healthy controls when attempting ankle dorsiflexion
movements. However, with increasing time post injury and persist-
ing paralysis, contralateral M1 activation decreased along with pre-
frontal, premotor, supplementary, primary somatosensory, and
posterior parietal cortices and cingulate motor area activation.
These results depict a progressive shift in cortical reorganization
further influenced by lower-extremity disuse. A related study in
individuals with chronic SCI found cortical thinning in the leg area
of the M1 and primary sensory cortex compared to healthy con-
trols [104]. Moreover, subjects with SCI demonstrated increased
activation of the left M1 leg area during right handgrip task com-
pared to controls that was associated with smaller cervical cord area
and impaired upper-extremity function. Lundell et al. [105] also
found associations between spinal cord atrophy, motor function,
and ipsilateral M1, somatosensory, and premotor cortical activa-
tion during ankle dorsiflexion movements in individuals with
chronic SCI. Additional investigation is needed to further substan-
tiate the relationship between neuroplastic reorganization, severity
of SCI, and ensuing motor function.
Studies to date have more been focused on the nature of brain
motor systems function in the chronic state, with some divergence
of results to date. Some studies have found a broad decrease in acti-
vation [106–108], particularly in primary sensorimotor cortex,
whereas others have found supranormal activation [109]. The basis
for these discrepancies remains unclear but could be due to differ-
ences in age or injury pattern of the population studied, years post-
SCI at the time of study, amount of motor function at the time of
study, or the nature of the task used to probe motor system function,
some uncovering deficient processing and others emphasizing supra-
normal efforts to compensate [107, 110]. A commonly described
feature is a change in somatotopic organization within primary sen-
sorimotor cortex contralateral to sensory or motor events, with rep-
resentation of supralesional body regions expanding at the expense
of infralesional body regions [111–115]. Spontaneous changes in
laterality, so prominent in studies of stroke or MS, as above, are
842 Steven C. Cramer and Jessica M. Cassidy

generally not prominent after SCI [116], perhaps due in part to the
fact that injury typically affects the CNS bilaterally or perhaps due in
part to the fact that SCI spares brain commissural fibers whose integ-
rity helps maintain normal hemispheric balance. As such, laterality is
unlikely to be a useful variable to examine in brain mapping studies
of treatment effects in the setting of SCI.
At least two studies have evaluated changes in brain function in
relation to therapy after SCI. Winchester et al. [117] studied body
weight supported treadmill training in four patients with motor
incomplete SCI. These authors compared fMRI during attempted
unilateral foot and toe movement before vs. after training. This
therapy was associated with increased activation within several
bilateral areas, including primary sensorimotor cortex and cerebel-
lum, though to a variable extent. The authors observed that,
although all participants demonstrated a change in the BOLD
signal following training, only those patients who demonstrated a
substantial increase in activation of the cerebellum demonstrated
an improvement in their ability to walk over ground, suggesting
that this measure in this brain region, at least when examined using
this task during fMRI, might be useful as a biological marker of
successful treatment effect.
Another form of intervention that has been evaluated after SCI is
motor imagery. Motor imagery normally activates many of the same
brain regions as motor execution, and has been associated with
improvements in motor performance [118, 119]. The effects of
1-week of motor imagery training to tongue and to foot were evalu-
ated in ten subjects with chronic, complete tetra-/paraplegia plus ten
healthy controls [61]. The behavioral outcome measure was speed of
performance of a complex sequence. Motor imagery training was
associated with a significant improvement in this behavior in non-
paralyzed muscles (tongue for both groups, right foot for healthy
subjects). In both the healthy controls and the subjects with SCI,
serial fMRI scanning (before vs. after training) during attempted
right foot movement was associated with increased fMRI activation
in left putamen, an area associated with motor learning, despite foot
movements being present in controls and absent in subjects with
SCI. Behavioral training can thus result in measurable brain plasticity
that is not accompanied by outward behavioral gains, a finding that
might be important for designing biological markers in trials target-
ing severely disabled patient populations. Note that this fMRI change
was absent in a second healthy control group serially imaged without
training. The main conclusion from this study is that motor imagery
training improves brain function whether or not sensorimotor func-
tion is present in the trained limb. An additional conclusion is that
motor imagery, by virtue of its favorable effects on brain motor sys-
tem organization, might have value as an adjunct motor restorative
therapy. Another key point from this study is that brain plasticity
related to plegic limbs can be studied in subjects with chronic SCI.
 Application of fMRI to Monitor Motor Rehabilitation 843

One hypothesis suggested by this study’s findings is that the

results of a short-term brain plasticity probe, such as this motor
imagery training intervention, will predict response to a longer-
term treatment [17, 18], for example, patients who show the
greatest extent of brain plasticity with such a 1-week motor imag-
ery intervention are those might be those who are most likely to
respond to a more intensive intervention such as stem cell injec-
tions. Thus, at least in chronic SCI, some measure of the capacity
for the brain to adapt in the short-term might predict likelihood of
response to a more intense intervention.

5 Conclusions

Motor deficits are a major source of disability, across a number of

conditions. A number of restorative therapies are under study to
improve motor function in this regard. Optimal prescription of
such therapies might benefit from an assessment of the function of
the target organ, in addition to assessment of behavior or
demographics. This is an approach that has often proven fruitful in
general medical practice, and given the added complexities related
to the CNS, is likely to be particularly important in for application
of CNS restorative therapies.
Towards this goal, establishment of standardized protocols, such
as for measuring motor cortex plasticity [120], to extract measures of
brain function might help maximize the extent to which functional
neuroimaging can be effectively applied. Dynamic protocols that
incite a CNS response, such as over 30 min [121] or days [54, 56] of
activity, might have particular value as compared to a single cross-
sectional behavioral probe. Also, studies that provide a greater under-
standing of the underlying neurobiologic principles related to
spontaneous recovery will also aid application of restorative therapies
given that brain changes important to spontaneous recovery likely
overlap substantially with changes whose measurement can effec-
tively guide trials to maximize treatment-induced gains.
Further studies are needed to better characterize the measures
that have the potential for monitoring, predicting, and dosing in
the setting of a restorative trial of patients with motor deficits.
Also, a minority of studies has examined language, neglect, and
other domains injured in CNS disease, and further studies in these
areas are also needed. Some similarities exist across diseases, such
as those discussed between stroke and MS above, and further
investigation of such points of similarity might prove fruitful in a
broader sense to advancing restorative therapeutics.
The crux of this review centered on task-related fMRI.
Examination of network connectivity using resting-state fMRI may
also serve as a valuable biologic measure of disease status and/or
treatment effectiveness in stroke [122–126], MS [127–129], and
844 Steven C. Cramer and Jessica M. Cassidy

SCI [130, 131]. Finally, this review focused on fMRI as a means of

probing the state of the CNS. Other investigative methods might
also prove useful, including functional, anatomical, and other
forms of probe. Examples include positron emission tomography,
diffusion tensor imaging, proton MR spectroscopy, TMS, electro-
encephalography, and measures of anatomy or perfusion. These
can measure white matter integrity [58], injury in relation to nor-
mal anatomy [132, 133], metabolic state [134, 135], and more
that might prove equally useful in models that aim to inform thera-
peutic approaches to restoring motor function in the setting of
neurological disease.

References

1. Dobkin B (2003) The clinical science of neu- lowing focal stroke. Neuroreport
rologic rehabilitation. Oxford University 9(7):1441–1445
Press, New York 9. Schabitz WR, Berger C, Kollmar R, Seitz M,
2. Chen J, Cui X, Zacharek A, Jiang H, Roberts Tanay E, Kiessling M et al (2004) Effect of
C, Zhang C et al (2007) Niaspan increases brain-derived neurotrophic factor treatment
angiogenesis and improves functional recov- and forced arm use on functional motor
ery after stroke. Ann Neurol 62(1):49–58 recovery after small cortical ischemia. Stroke
3. Li L, Jiang Q, Zhang L, Ding G, Gang Zhang 35(4):992–997
Z, Li Q et al (2007) Angiogenesis and 10. Wang L, Zhang Z, Wang Y, Zhang R, Chopp
improved cerebral blood flow in the ischemic M (2004) Treatment of stroke with erythro-
boundary area detected by MRI after admin- poietin enhances neurogenesis and angiogen-
istration of sildenafil to rats with embolic esis and improves neurological function in
stroke. Brain Res 1132(1):185–192 rats. Stroke 35(7):1732–1737
4. Chen P, Goldberg D, Kolb B, Lanser M, 11. Tsai PT, Ohab JJ, Kertesz N, Groszer M,
Benowitz L (2002) Inosine induces axonal Matter C, Gao J et al (2006) A critical role of
rewiring and improves behavioral outcome erythropoietin receptor in neurogenesis and
after stroke. Proc Natl Acad Sci U S A post-stroke recovery. J Neurosci
99(13):9031–9036 26(4):1269–1274
5. Freret T, Valable S, Chazalviel L, Saulnier R, 12. Schneider UC, Schilling L, Schroeck H, Nebe
Mackenzie ET, Petit E et al (2006) Delayed CT, Vajkoczy P, Woitzik J (2007) Granulocyte-
administration of deferoxamine reduces brain macrophage colony-stimulating factor-
damage and promotes functional recovery induced vessel growth restores cerebral blood
after transient focal cerebral ischemia in the supply after bilateral carotid artery occlusion.
rat. Eur J Neurosci 23(7):1757–1765 Stroke 38(4):1320–1328
6. Papadopoulos CM, Tsai SY, Cheatwood JL, 13. Kolb B, Morshead C, Gonzalez C, Kim M,
Bollnow MR, Kolb BE, Schwab ME et al (2006) Gregg C, Shingo T et al (2007) Growth
Dendritic plasticity in the adult rat following factor-stimulated generation of new cortical
middle cerebral artery occlusion and Nogo-a tissue and functional recovery after stroke
neutralization. Cereb Cortex 16(4):529–536 damage to the motor cortex of rats. J Cereb
7. Kawamata T, Dietrich W, Schallert T, Gotts J, Blood Flow Metab 27(5):983–997
Cocke R, Benowitz L et al (1997) 14. Zhao LR, Berra HH, Duan WM, Singhal S,
Intracisternal basic fibroblast growth factor Mehta J, Apkarian AV et al (2007) Beneficial
(bFGF) enhances functional recovery and effects of hematopoietic growth factor ther-
upregulates the expression of a molecular apy in chronic ischemic stroke in rats. Stroke
marker of neuronal sprouting following focal 38(10):2804–2811
cerebral infarction. Proc Natl Acad Sci U S A 15. Ehrenreich H, Hasselblatt M, Dembowski C,
94:8179–8184 Cepek L, Lewczuk P, Stiefel M et al (2002)
8. Kawamata T, Ren J, Chan T, Charette M, Erythropoietin therapy for acute stroke is
Finklestein S (1998) Intracisternal osteogenic both safe and beneficial. Mol Med
protein-1 enhances functional recovery fol- 8(8):495–505
 Application of fMRI to Monitor Motor Rehabilitation 845

16. Savitz SI, Dinsmore JH, Wechsler LR, 28. Volpe BT, Ferraro M, Lynch D, Christos P,
Rosenbaum DM, Caplan LR (2004) Cell Krol J, Trudell C et al (2005) Robotics and
therapy for stroke. NeuroRx 1(4):406–414 other devices in the treatment of patients
17. Keirstead HS, Nistor G, Bernal G, Totoiu M, recovering from stroke. Curr Neurol Neurosci
Cloutier F, Sharp K et al (2005) Human Rep 5(6):465–470
embryonic stem cell-derived oligodendrocyte 29. Reinkensmeyer D, Emken J, Cramer S (2004)
progenitor cell transplants remyelinate and Robotics, motor learning, and neurologic
restore locomotion after spinal cord injury. recovery. Annu Rev Biomed Eng 6:497–525
J Neurosci 25(19):4694–4705 30. Deutsch JE, Lewis JA, Burdea G (2007)
18. Cummings BJ, Uchida N, Tamaki SJ, Salazar Technical and patient performance using a
DL, Hooshmand M, Summers R et al (2005) virtual reality-integrated telerehabilitation
Human neural stem cells differentiate and system: preliminary finding. IEEE Trans
promote locomotor recovery in spinal cord- Neural Syst Rehabil Eng 15(1):30–35
injured mice. Proc Natl Acad Sci U S A 31. Duncan P, Studenski S, Richards L, Gollub S,
102(39):14069–14074 Lai S, Reker D et al (2003) Randomized clini-
19. Shen LH, Li Y, Chen J, Zacharek A, Gao Q, cal trial of therapeutic exercise in subacute
Kapke A et al (2006) Therapeutic benefit of stroke. Stroke 34(9):2173–2180
bone marrow stromal cells administered 1 32. Woldag H, Hummelsheim H (2002)
month after stroke. J Cereb Blood Flow Evidence-based physiotherapeutic concepts
Metab 27:6–13 for improving arm and hand function in
20. Khedr EM, Ahmed MA, Fathy N, Rothwell stroke patients: a review. J Neurol
JC (2005) Therapeutic trial of repetitive tran- 249(5):518–528
scranial magnetic stimulation after acute isch- 33. French B, Thomas L, Leathley M, Sutton C,
emic stroke. Neurology 65(3):466–468 McAdam J, Forster A et al (2007) Repetitive
21. Kim YH, You SH, Ko MH, Park JW, Lee KH, task training for improving functional ability
Jang SH et al (2006) Repetitive transcranial after stroke. Cochrane Database Syst Rev
magnetic stimulation-induced corticomotor (4):CD006073
excitability and associated motor skill acquisi- 34. Kwakkel G, Wagenaar R, Twisk J, Lankhorst
tion in chronic stroke. Stroke G, Koetsier J (1999) Intensity of leg and arm
37(6):1471–1476 training after primary middle-cerebral-artery
22. Malcolm MP, Triggs WJ, Light KE, Gonzalez stroke: a randomised trial. Lancet
Rothi LJ, Wu S, Reid K et al (2007) Repetitive 354(9174):191–196
transcranial magnetic stimulation as an 35. Van Peppen RP, Kwakkel G, Wood-Dauphinee
adjunct to constraint-induced therapy: an S, Hendriks HJ, Van der Wees PJ, Dekker
exploratory randomized controlled trial. Am J (2004) The impact of physical therapy on
J Phys Med Rehabil 86(9):707–715 functional outcomes after stroke: what’s the
23. Hummel F, Celnik P, Giraux P, Floel A, Wu evidence? Clin Rehabil 18(8):833–862
WH, Gerloff C et al (2005) Effects of non- 36. Luft A, McCombe-Waller S, Whitall J,
invasive cortical stimulation on skilled motor Forrester L, Macko R, Sorkin J et al (2004)
function in chronic stroke. Brain 128(Pt Repetitive bilateral arm training and motor
3):490–499 cortex activation in chronic stroke: a random-
24. Brown JA, Lutsep HL, Weinand M, Cramer ized controlled trial. JAMA
SC (2006) Motor cortex stimulation for the 292(15):1853–1861
enhancement of recovery from stroke: a pro- 37. Wolf SL, Winstein CJ, Miller JP, Taub E,
spective, multicenter safety study. Uswatte G, Morris D et al (2006) Effect of
Neurosurgery 58(3):464–473 constraint-induced movement therapy on
25. Ring H, Rosenthal N (2005) Controlled upper extremity function 3 to 9 months after
study of neuroprosthetic functional electrical stroke: the EXCITE randomized clinical trial.
stimulation in sub-acute post-stroke rehabili- JAMA 296(17):2095–2104
tation. J Rehabil Med 37(1):32–36 38. Rathore S, Hinn A, Cooper L, Tyroler H,
26. Sheffler LR, Chae J (2007) Neuromuscular Rosamond W (2002) Characterization of
electrical stimulation in neurorehabilitation. incident stroke signs and symptoms: findings
Muscle Nerve 35(5):562–590 from the atherosclerosis risk in communities
27. Kwakkel G, Kollen BJ, Krebs HI (2007) study. Stroke 33(11):2718–2721
Effects of robot-assisted therapy on upper 39. Gresham G, Duncan P, Stason W, Adams H,
limb recovery after stroke: a systematic review. Adelman A, Alexander D et al (1995) Post-
Neurorehabil Neural Repair 22:111–121 stroke rehabilitation. U.S. Department of
846 Steven C. Cramer and Jessica M. Cassidy

Health and Human Services. Public Health motor cortex after stroke. Brain 127(Pt
Service, Agency for Health Care Policy and 4):747–758
Research, Rockville, MD 52. Cramer S, Benson R, Himes D, Burra V,
40. Duncan P, Goldstein L, Horner R, Landsman Janowsky J, Weinand M et al (2005) Use of
P, Samsa G, Matchar D (1994) Similar motor functional MRI to guide decisions in a clinical
recovery of upper and lower extremities after stroke trial. Stroke 36(5):e50–e52
stroke. Stroke 25(6):1181–1188 53. Platz T, Kim I, Engel U, Kieselbach A,
41. Duncan P, Goldstein L, Matchar D, Divine Mauritz K (2002) Brain activation pattern as
G, Feussner J (1992) Measurement of motor assessed with multi-modal EEG analysis pre-
recovery after stroke. Stroke 23:1084–1089 dict motor recovery among stroke patients
42. Nakayama H, Jorgensen H, Raaschou H, with mild arm paresis who receive the Arm
Olsen T (1994) Recovery of upper extremity Ability Training. Restor Neurol Neurosci
function in stroke patients: the Copenhagen 20(1–2):21–35
Stroke Study. Arch Phys Med Rehabil 54. Dong Y, Dobkin BH, Cen SY, Wu AD,
75(4):394–398 Winstein CJ (2006) Motor cortex activation
43. Wade D, Langton-Hewer R, Wood V, during treatment may predict therapeutic
Skilbeck C, Ismail H (1983) The hemiplegic gains in paretic hand function after stroke.
arm after stroke: measurement and recovery. Stroke 37(6):1552–1555
J Neurol Neurosurg Psychiatry 55. Fritz SL, Light KE, Patterson TS, Behrman
46(6):521–524 AL, Davis SB (2005) Active finger extension
44. Yozbatiran N, Cramer SC (2006) Imaging predicts outcomes after constraint-induced
motor recovery after stroke. NeuroRx movement therapy for individuals with hemi-
3(4):482–488 paresis after stroke. Stroke 36(6):1172–1177
45. Ward NS, Cohen LG (2004) Mechanisms 56. Koski L, Mernar T, Dobkin B (2004)
underlying recovery of motor function after Immediate and long-term changes in cortico-
stroke. Arch Neurol 61(12):1844–1848 motor output in response to rehabilitation:
46. Baron J, Cohen L, Cramer S, Dobkin B, correlation with functional improvements in
Johansen-Berg H, Loubinoux I et al (2004) chronic stroke. Neurorehabil Neural Repair
Neuroimaging in stroke recovery: a position 18(4):230–249
paper from the First International workshop 57. Cramer S, Parrish T, Levy R, Stebbins G,
on neuroimaging and stroke recovery. Ruland S, Lowry D et al (2007) An assess-
Cerebrovasc Dis (Basel, Switzerland) ment of brain function predicts functional
18(3):260–267 gains in a clinical stroke trial. Stroke 38:520
47. Lotze M, Markert J, Sauseng P, Hoppe J, (abstract)
Plewnia C, Gerloff C (2006) The role of mul- 58. Stinear CM, Barber PA, Smale PR, Coxon JP,
tiple contralesional motor areas for complex Fleming MK, Byblow WD (2007) Functional
hand movements after internal capsular potential in chronic stroke patients depends
lesion. J Neurosci 26(22):6096–6102 on corticospinal tract integrity. Brain 130(Pt
48. Winhuisen L, Thiel A, Schumacher B, Kessler 1):170–180
J, Rudolf J, Haupt WF et al (2005) Role of 59. Burke E, Dobkin BH, Noser EA, Enney LA,
the contralateral inferior frontal gyrus in Cramer SC (2014) Predictors and biomarkers
recovery of language function in poststroke of treatment gains in a clinical stroke trial tar-
aphasia: a combined repetitive transcranial geting the lower extremity. Stroke
magnetic stimulation and positron emission 45(8):2379–2384
tomography study. Stroke 36(8):1759–1763 60. Burke Quinlan E, Dodakian L, See J,
49. Johansen-Berg H, Rushworth M, Bogdanovic McKenzie A, Le V, Wojnowicz M et al (2015)
M, Kischka U, Wimalaratna S, Matthews P Neural function, injury, and stroke subtype
(2002) The role of ipsilateral premotor cortex predict treatment gains after stroke. Ann
in hand movement after stroke. Proc Natl Neurol 77(1):132–145
Acad Sci U S A 99(22):14518–14523 61. Cramer SC, Orr EL, Cohen MJ, Lacourse
50. Werhahn K, Conforto A, Kadom N, Hallett MG (2007) Effects of motor imagery training
M, Cohen L (2003) Contribution of the ipsi- after chronic, complete spinal cord injury.
lateral motor cortex to recovery after chronic Exp Brain Res 177(2):233–242
stroke. Ann Neurol 54(4):464–472 62. Carey J, Kimberley T, Lewis S, Auerbach E,
51. Fridman E, Hanakawa T, Chung M, Hummel Dorsey L, Rundquist P et al (2002) Analysis of
F, Leiguarda R, Cohen L (2004) fMRI and finger tracking training in subjects
Reorganization of the human ipsilesional pre- with chronic stroke. Brain 125(Pt 4):773–788
 Application of fMRI to Monitor Motor Rehabilitation 847

63. Veverka T, Hlustik P, Hok P, Otruba P, Tudos induced movement therapy in chronic stroke.
Z, Zapletalova J et al (2014) Cortical activity Eur J Neurol 19(4):578–586
modulation by botulinum toxin type A in 74. Vukusic S, Confavreux C (2007) Natural his-
patients with post-stroke arm spasticity: real tory of multiple sclerosis: risk factors and
and imagined hand movement. J Neurol Sci prognostic indicators. Curr Opin Neurol
346(1–2):276–283 20(3):269–274
64. Laible M, Grieshammer S, Seidel G, Rijntjes 75. Vollmer T (2007) The natural history of
M, Weiller C, Hamzei F (2012) Association relapses in multiple sclerosis. J Neurol Sci
of activity changes in the primary sensory cor- 256(Suppl 1):S5–S13
tex with successful motor rehabilitation of the 76. Rocca MA, Filippi M (2007) Functional MRI
hand following stroke. Neurorehabil Neural in multiple sclerosis. J Neuroimaging
Repair 26(7):881–888 17(Suppl 1):36S–41S
65. Bosnell RA, Kincses T, Stagg CJ, Tomassini 77. Rocca MA, Colombo B, Falini A, Ghezzi A,
V, Kischka U, Jbabdi S et al (2011) Motor Martinelli V, Scotti G et al (2005) Cortical
practice promotes increased activity in brain adaptation in patients with MS: a cross-
regions structurally disconnected after sub- sectional functional MRI study of disease
cortical stroke. Neurorehabil Neural Repair phenotypes. Lancet Neurol 4(10):618–626
25(7):607–616
78. Wang J, Hier DB (2007) Motor reorganiza-
66. Meehan SK, Randhawa B, Wessel B, Boyd LA tion in multiple sclerosis. Neurol Res
(2011) Implicit sequence-specific motor 29(1):3–8
learning after subcortical stroke is associated
with increased prefrontal brain activations: an 79. Pantano P, Mainero C, Caramia F (2006)
fMRI study. Hum Brain Mapp Functional brain reorganization in multiple
32(2):290–303 sclerosis: evidence from fMRI studies.
J Neuroimaging 16(2):104–114
67. Sitaram R, Veit R, Stevens B, Caria A, Gerloff
C, Birbaumer N et al (2012) Acquired con- 80. Ward N, Brown M, Thompson A, Frackowiak
trol of ventral premotor cortex activity by R (2003) Neural correlates of outcome after
feedback training: an exploratory real-time stroke: a cross-sectional fMRI study. Brain
FMRI and TMS study. Neurorehabil Neural 126(Pt 6):1430–1448
Repair 26(3):256–265 81. Cramer SC, Crafton KR (2006) Somatotopy
68. Stagg CJ, Bachtiar V, O’Shea J, Allman C, and movement representation sites following
Bosnell RA, Kischka U et al (2012) Cortical cortical stroke. Exp Brain Res
activation changes underlying stimulation- 168(1–2):25–32
induced behavioural gains in chronic stroke. 82. Lenzi D, Conte A, Mainero C, Frasca V,
Brain 135(Pt 1):276–284 Fubelli F, Totaro P et al (2007) Effect of cor-
69. Favre I, Zeffiro TA, Detante O, Krainik A, pus callosum damage on ipsilateral motor
Hommel M, Jaillard A (2014) Upper limb activation in patients with multiple sclerosis: a
recovery after stroke is associated with ipsile- functional and anatomical study. Hum Brain
sional primary motor cortical activity: a meta- Mapp 28(7):636–644
analysis. Stroke 45(4):1077–1083 83. Rocca MA, Gallo A, Colombo B, Falini A,
70. Hodics T, Cohen LG, Cramer SC (2006) Scotti G, Comi G et al (2004) Pyramidal tract
Functional imaging of intervention effects in lesions and movement-associated cortical
stroke motor rehabilitation. Arch Phys Med recruitment in patients with MS. Neuroimage
Rehabil 87(12 Suppl):36–42 23(1):141–147
71. Schaechter J, Kraft E, Hilliard T, Dijkhuizen 84. Reddy H, Narayanan S, Matthews P, Hoge R,
R, Benner T, Finklestein S et al (2002) Motor Pike G, Duquette P et al (2000) Relating axo-
recovery and cortical reorganization after nal injury to functional recovery in
constraint-induced movement therapy in MS. Neurology 54(1):236–239
stroke patients: a preliminary study. 85. Mezzapesa DM, Rocca MA, Rodegher M,
Neurorehabil Neural Repair 16(4):326–338 Comi G, Filippi M (2007) Functional cortical
72. Johansen-Berg H, Dawes H, Guy C, Smith S, changes of the sensorimotor network are
Wade D, Matthews P (2002) Correlation associated with clinical recovery in multiple
between motor improvements and altered sclerosis. Hum Brain Mapp 29(5):562–573
fMRI activity after rehabilitative therapy. 86. Cramer S, Nelles G, Benson R, Kaplan J,
Brain 125(Pt 12):2731–2742 Parker R, Kwong K et al (1997) A functional
73. Kononen M, Tarkka IM, Niskanen E, MRI study of subjects recovered from hemi-
Pihlajamaki M, Mervaala E, Pitkanen K et al paretic stroke. Stroke 28(12):2518–2527
(2012) Functional MRI and motor behav- 87. Filippi M, Rocca MA, Mezzapesa DM, Falini A,
ioral changes obtained with constraint- Colombo B, Scotti G et al (2004) A functional
848 Steven C. Cramer and Jessica M. Cassidy

MRI study of cortical activations associated with 100. Jayaraman A, Gregory CM, Bowden M,
object manipulation in patients with Stevens JE, Shah P, Behrman AL et al (2006)
MS. Neuroimage 21(3):1147–1154 Lower extremity skeletal muscle function in
88. Colorado RA, Shukla K, Zhou Y, Wolinsky persons with incomplete spinal cord injury.
JS, Narayana PA (2012) Multi-task functional Spinal Cord 44(11):680–687
MRI in multiple sclerosis patients without 101. DeVivo MJ, Richards JS (1992) Community
clinical disability. Neuroimage reintegration and quality of life following spi-
59(1):573–581 nal cord injury. Paraplegia 30(2):108–112
89. Giorgio A, Portaccio E, Stromillo ML, 102. Frankel HL, Coll JR, Charlifue SW, Whiteneck
Marino S, Zipoli V, Battaglini M et al (2010) GG, Gardner BP, Jamous MA et al (1998)
Cortical functional reorganization and its Long-term survival in spinal cord injury: a fifty
relationship with brain structural damage in year investigation. Spinal Cord 36(4):266–274
patients with benign multiple sclerosis. Mult 103. Jurkiewicz MT, Mikulis DJ, Fehlings MG,
Scler 16(11):1326–1334 Verrier MC (2010) Sensorimotor cortical
90. Parry AM, Scott RB, Palace J, Smith S, activation in patients with cervical spinal cord
Matthews PM (2003) Potentially adaptive injury with persisting paralysis. Neurorehabil
functional changes in cognitive processing for Neural Repair 24(2):136–140
patients with multiple sclerosis and their acute 104. Freund P, Weiskopf N, Ward NS, Hutton C,
modulation by rivastigmine. Brain 126(Pt Gall A, Ciccarelli O et al (2011) Disability, atro-
12):2750–2760 phy and cortical reorganization following spinal
91. Sadato N, Campbell G, Ibanez V, Deiber M, cord injury. Brain 134(Pt 6):1610–1622
Hallett M (1996) Complexity affects regional 105. Lundell H, Christensen MS, Barthelemy D,
cerebral blood flow change during sequential fin- Willerslev-Olsen M, Biering-Sorensen F,
ger movements. J Neurosci 16(8):2691–2700 Nielsen JB (2011) Cerebral activation is cor-
92. Verstynen T, Diedrichsen J, Albert N, related to regional atrophy of the spinal cord
Aparicio P, Ivry RB (2005) Ipsilateral motor and functional motor disability in spinal cord
cortex activity during unimanual hand move- injured individuals. Neuroimage
ments relates to task complexity. 54(2):1254–1261
J Neurophysiol 93(3):1209–1222 106. Cramer SC, Lastra L, Lacourse MG, Cohen
93. Mainero C, Inghilleri M, Pantano P, Conte A, MJ (2005) Brain motor system function after
Lenzi D, Frasca V et al (2004) Enhanced chronic, complete spinal cord injury. Brain
brain motor activity in patients with MS after 128(Pt 12):2941–2950
a single dose of 3,4-diaminopyridine. 107. Jurkiewicz MT, Mikulis DJ, McIlroy WE,
Neurology 62(11):2044–2050 Fehlings MG, Verrier MC (2007)
94. Tomassini V, Johansen-Berg H, Jbabdi S, Sensorimotor cortical plasticity during recov-
Wise RG, Pozzilli C, Palace J et al (2012) ery following spinal cord injury: a longitudi-
Relating brain damage to brain plasticity in nal fMRI study. Neurorehabil Neural Repair
patients with multiple sclerosis. Neurorehabil 21(6):527–538
Neural Repair 26(6):581–593 108. Sabbah P, de Schonen S, Leveque C, Gay S,
95. Morgen K, Kadom N, Sawaki L, Tessitore A, Pfefer F, Nioche C et al (2002) Sensorimotor
Ohayon J, McFarland H et al (2004) Training- cortical activity in patients with complete spinal
dependent plasticity in patients with multiple cord injury: a functional magnetic resonance
sclerosis. Brain 127(Pt 11):2506–2517 imaging study. J Neurotrauma 19(1):53–60
96. Facts and Figures at a Glance—June 2006 109. Alkadhi H, Brugger P, Boendermaker S,
(2007) www.spinalcord.uab.edu Crelier G, Curt A, Hepp-Reymond M et al
97. Geisler F, Dorsey F, Coleman W (1991) (2005) What disconnection tells about motor
Recovery of motor function after spinal-cord imagery: evidence from paraplegic patients.
injury—a randomized, placebo-controlled Cereb Cortex 15(2):131–140
trial with GM-1 ganglioside. N Engl J Med 110. Humphrey D, Mao H, Schaeffer E (eds)
324(26):1829–1838 (2000) Voluntary activation of ineffective cere-
98. Ditunno J, Stover S, Freed M, Ahn J (1992) bral motor areas in short- and long-term para-
Motor recovery of the upper extremities in plegics. Society for Neuroscience (abstract)
traumatic quadriplegia: a multicenter study. 111. Topka H, Cohen L, Cole R, Hallett M (1991)
Arch Phys Med Rehabil 73(5):431–436 Reorganization of corticospinal pathways
99. Kirshblum S, Millis S, McKinley W, Tulsky D following spinal cord injury. Neurology
(2004) Late neurologic recovery after trau- 41(8):1276–1283
matic spinal cord injury. Arch Phys Med 112. Bruehlmeier M, Dietz V, Leenders K, Roelcke
Rehabil 85(11):1811–1817 U, Missimer J, Curt A (1998) How does the
 Application of fMRI to Monitor Motor Rehabilitation 849

human brain deal with a spinal cord injury? 124. Park CH, Chang WH, Ohn SH, Kim ST,
Eur J Neurosci 10(12):3918–3922 Bang OY, Pascual-Leone A et al (2011)
113. Mikulis D, Jurkiewicz M, McIlroy W, Staines W, Longitudinal changes of resting-state func-
Rickards L, Kalsi-Ryan S et al (2002) Adaptation tional connectivity during motor recovery
in the motor cortex following cervical spinal after stroke. Stroke 42(5):1357–1362
cord injury. Neurology 58(5):794–801 125. Carter AR, Patel KR, Astafiev SV, Snyder AZ,
114. Turner J, Lee J, Martinez O, Medlin A, Rengachary J, Strube MJ et al (2012) Upstream
Schandler S, Cohen M (2001) Somatotopy of dysfunction of somatomotor functional connec-
the motor cortex after long-term spinal cord tivity after corticospinal damage in stroke.
injury or amputation. IEEE Trans Neural Syst Neurorehabil Neural Repair 26(1):7–19
Rehabil Eng 9(2):154–160 126. Varkuti B, Guan C, Pan Y, Phua KS, Ang KK,
115. Corbetta M, Burton H, Sinclair R, Conturo Kuah CW et al (2013) Resting state changes
T, Akbudak E, McDonald J (2002) Functional in functional connectivity correlate with
reorganization and stability of somatosensory- movement recovery for BCI and robot-
motor cortical topography in a tetraplegic assisted upper-extremity training after stroke.
subject with late recovery. Proc Natl Acad Sci Neurorehabil Neural Repair 27(1):53–62
U S A 99(26):17066–17071 127. Petsas N, Tomassini V, Filippini N, Sbardella
116. Sabre L, Tomberg T, Korv J, Kepler J, Kepler E, Tona F, Piattella MC et al (2015) Impaired
K, Linnamagi U et al (2013) Brain activation functional connectivity unmasked by simple
in the acute phase of traumatic spinal cord repetitive motor task in early relapsing-
injury. Spinal Cord 51(8):623–629 remitting multiple sclerosis. Neurorehabil
117. Winchester P, McColl R, Querry R, Foreman Neural Repair 29(6):557–565
N, Mosby J, Tansey K et al (2005) Changes in 128. Basile B, Castelli M, Monteleone F, Nocentini
supraspinal activation patterns following U, Caltagirone C, Centonze D et al (2013)
robotic locomotor therapy in motor- Functional connectivity changes within specific
incomplete spinal cord injury. Neurorehabil networks parallel the clinical evolution of mul-
Neural Repair 19(4):313–324 tiple sclerosis. Mult Scler 20(8):1050–1057
118. Lacourse MG, Turner JA, Randolph-Orr E, 129. Filippi M, Agosta F, Spinelli EG, Rocca MA
Schandler SL, Cohen MJ (2004) Cerebral (2013) Imaging resting state brain function in
and cerebellar sensorimotor plasticity follow- multiple sclerosis. J Neurol 260(7):1709–1713
ing motor imagery-based mental practice of a 130. Chisholm AE, Peters S, Borich MR, Boyd
sequential movement. J Rehabil Res Dev LA, Lam T (2015) Short-term cortical plas-
41(4):505–524 ticity associated with feedback-error learning
119. Sharma N, Pomeroy VM, Baron JC (2006) after locomotor training in a patient with
Motor imagery: a backdoor to the motor incomplete spinal cord injury. Phys Ther
system after stroke? Stroke 95(2):257–266
37(7):1941–1952 131. Hou JM, Sun TS, Xiang ZM, Zhang JZ,
120. Kleim J, Kleim E, Cramer S (2007) Systematic Zhang ZC, Zhao M et al (2014) Alterations
assessment of training-induced changes in of resting-state regional and network-level
corticospinal output to hand using frameless neural function after acute spinal cord injury.
stereotaxic transcranial magnetic stimulation. Neuroscience 277:446–454
Nat Protoc 2:1675–1684 132. Newton JM, Ward NS, Parker GJ, Deichmann
121. Kleim JA, Chan S, Pringle E, Schallert K, R, Alexander DC, Friston KJ et al (2006)
Procaccio V, Jimenez R et al (2006) BDNF val- Non-invasive mapping of corticofugal fibres
66met polymorphism is associated with modi- from multiple motor areas—relevance to
fied experience-dependent plasticity in human stroke recovery. Brain 129(Pt 7):1844–1858
motor cortex. Nat Neurosci 9(6):735–737 133. Crafton K, Mark A, Cramer S (2003)
122. Wadden KP, Woodward TS, Metzak PD, Improved understanding of cortical injury by
Lavigne KM, Lakhani B, Auriat AM et al incorporating measures of functional anat-
(2015) Compensatory motor network con- omy. Brain 126(Pt 7):1650–1659
nectivity is associated with motor sequence 134. Heiss W, Emunds H, Herholz K (1993)
learning after subcortical stroke. Behav Brain Cerebral glucose metabolism as a predictor of
Res 286:136–145 rehabilitation after ischemic stroke. Stroke
123. Golestani AM, Tymchuk S, Demchuk A, 24(12):1784–1788
Goodyear BG (2013) Longitudinal evalua- 135. Cappa S, Perani D, Grassi F, Bressi S, Alberoni
tion of resting-state FMRI after acute stroke M, Franceschi M et al (1997) A PET follow-
with hemiparesis. Neurorehabil Neural Repair up study of recovery after stroke in acute
27(2):153–163 aphasics. Brain Lang 56(1):55–67
 Part IV

Future fMRI Development

 Chapter 28

Multimodal Fusion of Structural and Functional

Brain Imaging Data
Jing Sui and Vince D. Calhoun

Abstract
Recent years have witnessed a rapid growth of interest in moving functional magnetic resonance imaging
(fMRI) beyond simple scan-length averages and into approaches that can integrate structural MRI measures
and capture rich multimodal interactions. It is becoming increasingly clear that multimodal fusion is able to
provide more information for individual subjects by exploiting covariation between modalities, rather an
analysis of each modality alone. Multimodal fusion is a more complicated endeavor that must be approached
carefully and efficient methods should be developed to draw generalized and valid conclusions out of high
dimensional data with a limited number of subjects, such as patients with brain disorders. Numerous research
efforts have been reported in the field based on various statistical models, including independent component
analysis (ICA), canonical correlation analysis (CCA), and partial least squares (PLS). In this chapter, we sur-
vey a number of methods previously shown in multimodal fusion reports, performed with or without prior
information, and with their possible strengths and limitations addressed. To examine the function–structure
associations of the brain in a more comprehensive and integrated manner, we also reviewed a number of
multimodal studies that combined fMRI and structural (sMRI and/or diffusion tensor MRI) measures,
which could reveal important brain alterations that may not be fully detected by employing separate analysis
of individual modalities, and also enable us to identify potential brain illness biomarkers.

Key words Multimodal fusion methods, Data driven, Functional magnetic resonance imaging,
Structural MRI, Diffusion MRI, Independent component analysis, Canonical correlation analysis

1 Introduction

There is increasing evidence that instead of focusing on the rela-

tionship between physiological or behavioral features using a single
imaging modality, multimodal brain imaging studies can help pro-
vide a better understanding of inter-subject variability from how
brain structure shapes brain function, to what degree brain func-
tion feeds back to change its structure, and what functional or
structural aspects of physiology ultimately drive cognition and
behavior. Collecting multiple modalities of magnetic resonance
imaging (MRI) data from the same individual has become a com-
mon practice, in order to search for task- or disease-related changes.

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_28, © Springer Science+Business Media New York 2016

853
854 Jing Sui and Vince D. Calhoun

Each imaging technique provides a different view in examining the

brain activity. For example, functional MRI (fMRI) measures the
hemodynamic response related to neural activity in the brain
dynamically; structural MRI (sMRI) provides information about
the tissue type of the brain [gray matter (GM), white matter (WM),
cerebrospinal fluid (CSF)]. Diffusion tensor (DT) MRI can addi-
tionally provide information on structural connectivity among
brain networks. A key motivation for jointly analyzing multimodal
data is to take advantage of the cross-information in the existing
data, thereby potentially revealing important variations that may
only partially be detected by a single modality. The availability of
several modal measurements allows joint analysis via the applica-
tion of a number of statistical approaches, including (but not being
limited to) correlational analyses [1], data integration [2, 3] or
data fusion [4, 5] based on higher-order statistics and/or modern
machine learning algorithms. These methods enable indirect or
direct associations to be inferred on putative structure–function
relationships [6], however, it is not necessary for these modalities
to have been measured simultaneously or have later been processed
in a concurrent fusion model.
Approaches for combining or fusing data in brain imaging can
be conceptualized as having a place on an analytic spectrum with
meta-analysis (highly distilled data) to examine convergent evi-
dence at one end and large-scale computational modeling (highly
detailed theoretical modeling) at the other end [7]. In between are
methods that attempt to perform a direct data fusion [8]. We next
review several multivariate multimodal fusion methods including
their statistical assumption, possible strengths and limitations, and
multimodal neuroimaging applications, especially combining fMRI
with structure measures. Behavioral relevance of the assessed phys-
iological features will be mentioned whenever possible.

2 Summary of Approaches Applied in Joint Analysis of Multimodal MRI Data

Before introduction of the statistical models, we would like to clar-

ify the use of “feature” as input to most of the mentioned models.
We note that this definition of “feature” is somewhat different
than what is used in traditional machine learning algorithms [9].
Basically, a “feature” is a distilled dataset representing the interest-
ing part of each modality [8] and it contributes as an input vector
for each modality and each subject. Usually the brain imaging data
is high dimensional, in order to reduce the redundancy and facili-
tate the identification of relationships between modalities, the raw
data can be preprocessed to generate a second-level output, that is
“feature,” which can be a contrast map calculated from task-related
fMRI by the general linear model (GLM), a component image
such as the “default mode” resulting from a first-level ICA, a
 Multimodal Fusion of Structural and Functional Brain Imaging Data 855

fractional anisotropy (FA) from DT MRI measures, or segmented

gray matter (GM) from sMRI data. The main reason to use fea-
tures is to provide a simpler space in which to link the data. The
trade-off is that some information may be lost, e.g., GM does not
directly measure volume or cortical thickness and FA does not pro-
vide directional information; however, there is considerable evi-
dence that the use of features is quite useful and valid [8, 10].
Figure 1 provides a direct view of the current popular multimodal
data analysis approaches related to fMRI.
“Data integration” is an alternative, but dissimilar approach to
“data fusion.” One characteristic of data integration is that it is
asymmetric, e.g., using the results from one modality to constrain
models of the other—as DT MRI being constrained by fMRI or
sMRI data in [11, 12], or fMRI-informed EEG [13, 14]. While

Fig. 1 Summary of the current multimodal data analysis approaches related to fMRI
856 Jing Sui and Vince D. Calhoun

these are powerful techniques, a limitation is that they may impose

potentially unrealistic assumptions upon the constrained data,
which are fundamentally of a different nature than the known
modality. Another integration example is to analyze each data type
separately and overlay them—thereby not allowing for any interac-
tion between the data types. For example, a data integration
approach would not detect a change in fMRI activation maps that
is related to a change in brain structure remotely.
By contrast, multimodal fusion refers to the use of a common
forward model of neuronal activity that explains different sorts of
data symmetrically [15], which provides more views for individual
subjects and covariation between modalities. This is a more com-
plicated endeavor, especially when studying complex mental ill-
nesses which impact many brain circuits. Also, in the real world,
the conclusions usually need to be drawn out of high dimensional
brain imaging data from a limited number of subjects. Hence effi-
cient methods should be developed carefully.
Multivariate approaches for combing brain imaging data can
be divided into two classes: hypotheses-driven and data-driven.
Hypotheses-driven approaches such as multiple linear regression
[16, 17], dynamic causal modeling (DCM), and confirmatory
structural equation modeling [18], have the advantages of: (1)
allow for testing specific hypotheses about the networks implied in
the experimental paradigm; (2) allow for simultaneous assessment
of several connectivity links, which would have been compromised
by the one-by-one assessment of covariance [19]. However, it is
possible to miss important connectivity links that were not included
in a set of a priori hypotheses and it does not provide information
about inter-voxel relationships [20, 21].
Data-driven approaches include, but are not limited to, princi-
pal component analysis (PCA), ICA, canonical correlation analysis
(CCA), and partial least squares (PLS). These methods are attrac-
tive as they do not require a priori hypotheses about the connec-
tion of interest. Hence, these methods are convenient for the
exploration of the full body of data. However, the some methods
may be more demanding from a computational standpoint.
The multivariate approaches adopted in multimodal MRI
fusion can be further divided into four classes based on the require-
ment of priori and the dimension of the MRI data used:
1. Blind methods that typically use second-level fMRI data (3D
contrast image), including joint ICA (jICA) [22], multimodal
canonical correlation analysis (mCCA) [23], linked ICA [24],
independent vector analysis(IVA) and mCCA + jICA [25, 26].
2. Blind methods that have been developed for use with raw
fMRI data (4D data), including partial least squares (PLS) [27,
28] and multiset CCA [29].
3. Semi-blind methods that use second-level fMRI data (3D con-
trast), e.g., parallel ICA [30], coefficient-constrained ICA
 Multimodal Fusion of Structural and Functional Brain Imaging Data 857

(CC-ICA) [31, 32], PCA with reference (PCA-R) [33, 34]

and informed multimodal PLS [27].
4. Other multimodal fusion applications using 4D fMRI and
EEG data, include multiple linear regression, structural equa-
tion modeling (SEM) [18] or DCM [35].
Here we will emphasize on the data-driven multivariate fusion
methods due to their flexibilities and advantages. The optimization
strategies of seven above mentioned multivariate models are dis-
played in Fig. 2. In the following sections, we will introduce five
blind fusion models and their applications in combining functional
and structural MRI in detail.

Fig. 2 A summary of seven blind and semi-blind data-driven methods for multimodal fusion. Figure modified
and reprinted with permission from Sui et al. [4]
858 Jing Sui and Vince D. Calhoun

3 Review of Multivariate Models Applied in Multimodal Fusion with fMRI

3.1 Joint ICA Joint ICA (jICA) is a second-level fMRI analysis method that
assumes two or more features (modalities) share the same mixing
matrix and maximizes the independence among joint components
[36]. This is a straightforward yet effective method by performing
ICA on the horizontally concatenated features (along voxels). It is
suitable for examining the common connection among modalities
and requires acceptance of the likelihood of changes in one data
type (e.g., GM) being related to another one, such as functional
activation. Joint ICA is feasible to many paired combinations of
features, such as fMRI, sMRI, and DT MRI, or three-way data
fusion [37–41]. In order to control for intensity differences in MR
images based on scanner, template, and population variations, usu-
ally each feature matrix (dimension: number of subject by number
of voxel) is normalized to a study specific template [42, 43].
Figure 3 [36], shows analyzed data collected from groups of
schizophrenia patients and healthy controls using the jICA
approach. The main finding was that group differences in bilateral
parietal and frontal as well as posterior temporal regions in GM
matter distinguished groups. A finding of less patient GM and less
hemodynamic activity for target detection in these bilateral anterior
temporal lobe regions was consistent with previous work. An unex-
pected corollary to this finding was that, in the regions showing the
largest group differences, GM concentrations were larger in patients
vs. controls, suggesting that more GM may be related to less func-
tional connectivity during performance of an auditory oddball task.

Fig. 3 Auditory oddball/gray matter jICA analysis. Only one component demonstrated a significant difference
between patients and controls. The joint source map for the auditory oddball fMRI data (left) and gray matter
(middle) data is presented along with the loading parameters for patients and controls (far right). Reproduced
with permission from Calhoun et al. [36]
 Multimodal Fusion of Structural and Functional Brain Imaging Data 859

3.2 Multimodal CCA Multimodal CCA allows a different mixing matrix for each modality
and is used to find the transformed coordinate system that maxi-
mizes these inter-subject covariations across the two data sets [44].
This method decomposes each dataset into a set of components
and their corresponding mixing profile, which is called canonical
variants (CVs). The CVs have varying levels of activations for dif-
ferent subjects and are linked if they modulate similarly across sub-
jects. After decomposition, the CVs correlate each other only on
the same indices and their corresponding correlation values are
called canonical correlation coefficients. Compared to jICA that
constrains two features to have the same mixing matrix, mCCA is
flexible in that it allows common as well as distinct level of connec-
tion between two features, as shown in Fig. 4, but the associated
source maps may not be spatially sparse, especially when the canon-
ical correlation coefficients are not sufficiently distinct [4].
Multimodal CCA is invariant to differences in the range of the
data types and can be used to jointly analyze very diverse data

Fig. 4 MCCA + jICA enables people to capture components of interest that are either common or distinct across
modalities. For example, when examining group differences across three modalities, joint ICs are significantly
group-discriminative in more than two modalities (green framed), while modality-specific discriminative ICs
(pink framed), i.e., fMRI_IC4, DTI_IC3, and DTI_IC7 only show significant group difference in a single modality.
Reproduced with permission from Sui et al. [5]
860 Jing Sui and Vince D. Calhoun

types. It can also be extended to multiset CCA to incorporate more

than two modalities [45]. Note that multimodal CCA works on
the second-level fMRI feature-contrast maps, while multiset-CCA
can work with 4D raw fMRI data (not shown in Fig. 1), e.g., to
maximize the trial-to-trial covariation between the concurrent
fMRI and EEG data [29].

3.3 Partial Least Partial Least Squares (PLS) as part of a family of multivariate data
Squares analyses, is based on the definition of a linear relationship between
a dependent variable and predictor variables, and hence the goal is
to determine which aspect of a set of observations (e.g., imaging
data) are related directly to another set of data (e.g., experimental
design, behavior) [46]. PLS was first applied to multimodal fusion
by Martinen-Montes et al. [28], where the multiway PLS (N-PLS)
was proposed to find correlations between fMRI time courses
(dependent variables) and the spectral components of the EEG
data (independent variables) from a single subject. Chen et al. [27]
further proposed a multimodal PLS (MMPLS) to simultaneously
characterize the linkage between patterns of PET and GM. The
multimodal PLS can be performed either informed to the variable
of interest such as age or agnostic to this additional information
(with or without priori). Investigators may want to pre-specify
which of these two methods to use in the data analysis. The agnos-
tic MMPLS was used to identify the linkage between PET (depen-
dent data block) and sMRI (independent data block), which is
mainly oriented for extraction of covariance patterns and related
latent variables rather than for classifications.
Even though PLS has some similarity to CCA in that they both
maximize between-set correlations, PLS is based on the definition
of a dependency and works well especially when the dependence
among the constituents of the datasets is explicitly assessed [47];
while CCA does not assign independent/dependent labels to either
of the modalities and treats both equally [48]. Hence PLS is par-
ticularly suited to the analysis of relationships between measures of
brain activity and of behavior or experimental design [49], while
interpretational difficulties in PLS often arise when the effects iden-
tified do not correspond to the a priori expectations of the researcher.

3.4 mCCA + jICA According to many previous findings in brain connectivity studies
which combined function and structure [19, 50], it is plausible to
assume the components decomposed from each modality have some
degree of correlation between their mixing profiles among subjects.
mCCA + jICA is a blind data-driven model that is optimized for this
situation [26, 51] and also to have excellent performance for achiev-
ing both flexible modal association and source separation. It takes
advantage of two complementary approaches: mCCA and jICA,
thus allowing both strong and weak inter-modality connection as
well as the joint independent components. mCCA makes the jICA
 Multimodal Fusion of Structural and Functional Brain Imaging Data 861

job more reliable by providing a closer initial match via correlation;

while jICA further decomposes the remaining mixtures in the asso-
ciated maps and relax the requirement of sufficiently distinction
imposed on the canonical correlation. The code of jICA,
mCCA + jICA can be accessed via the Fusion ICA Toolbox (FIT,
http://mialab.mrn.org/software/fit/). Note that the mCCA + jICA
approach does not increase the computational load appreciably and
is not limited to two-way fusion, but can potentially be extended to
three-way or N-way fusion of multiple data types by replacing the
multimodal CCA with multiset CCA [45]. It enables robust identi-
fication of correspondence among N diverse data types and enables
one to investigate the important question of whether certain disease
risk factors are shared or are distinct across multiple modalities. In
accordance with this notion, this approach has already been used to
fuse fMRI, sMRI, and DT MRI to study schizophrenia [25, 51], as
shown in Fig. 4.

3.5 Linked ICA Linked ICA is a fully probabilistic approach based on a modular
Bayesian framework, which is designed for simultaneously model-
ing and discovering common characteristics across multiple modal-
ities [24]. The combined modalities can potentially have different
units, noise level, spatial smoothness, and intensity distributions.
Each modality group is modeled using a Bayesian tensor ICA
model [52]. Note that the Bayesian ICA differs from standard
methods like FastICA [53] and Infomax [54] in that it incorpo-
rates dimensionality reduction into the ICA method itself by the
use of automatic relevance determination priors on components
[55] and works on the full-dimensionality data directly. Linked
ICA is good at detecting and isolating single-modality noise; how-
ever, it is much more computationally demanding than the above
four methods, and the spatial maps of the decomposed compo-
nents tend to be scattered. In a real application, linked ICA was
applied to high functioning autism by combining DT MRI, voxel-
based morphometry (VBM), and resting-state fMRI connectivity
to assess differences in brain structure and function [56].

4 Other Multimodal Applications Based on Functional and Structural MRI

Measures

All above multivariate, data-driven approaches focus on examining

the inter-modality covariance, which provides a natural way to find
multimodality associations. Whereas, there are many other multi-
modal studies in the context of cross-modal connectivity and classifi-
cations incorporating both functional and structural MRI measures.
Historically, the combination of structural and functional brain
imaging has been used for the purpose of analyzing functional
activity in a priori defined (based on atlas) or data-driven selected
(e.g., based on ICA components) brain regions. A central
862 Jing Sui and Vince D. Calhoun

assumption of systems neuroscience is that the structure of the

brain can predict and/or is related to functional connectivity. The
findings of Segall et al. [57] support this hypothesis, which gener-
ally show that each single structural component derived from ICA
usually corresponds to several resting-state functional components.
Functional information has been shown to be able to improve the
correspondence of functional boundaries across subjects beyond
the standard structural normalization [58]. Studies on psycho-
pathological phenomena could also discover spatial overlaps of
structural and functional alterations in schizophrenia or at risk
mental state using cognitive tasks and GM volume. Salgado-Pineda
et al. [59] found three regions including the thalamus, the anterior
cingulate, and the inferior parietal that showed both structural and
functional impairments associated with attentional processing in
schizophrenia. A follow-up study of the same group [60] also
found both functional alterations (facial emotion task) and GM
volume reductions in the DMN in schizophrenia.
On the other hand, many of the conducted fMRI-DT MRI
studies addressed disruptions of brain connectivity seen with mental
illnesses such as depression, schizophrenia, Alzheimer’s disease, and
bipolar disorder. For example [61], evaluated interactions between
measurements of anatomical and functional connectivity collected in
the same subjects to study global schizophrenia-related alterations in
brain connectivity. Schlosser et al. observed a direct correlation in
schizophrenia between frontal FA reduction and fMRI activation in
regions of the prefrontal and occipital cortices [62]. This finding
highlights a potential relationship between anatomical changes in a
frontal-temporal anatomical circuit and functional alterations in the
prefrontal cortex. Another study [63] also illustrated that DT MRI-
and fMRI-derived topologies are similar, and that the fMRI-DT
MRI combination can provide additional information in order to
choose reasonable seed regions for identifying functionally relevant
networks and to validate reconstructed WM fibers. A recent study of
Koch et al. [64] showed that WM fiber integrity in terms of increased
radial diffusivity of the left superior temporal gyrus is associated with
reduced neuronal activation in lateral frontal and cingulate cortices,
suggesting that intact WM connectivity plays an important role for
the pattern and intensity of functional activations with neuronal net-
works engaged in decision making.
As to the classification, there are additional studies demon-
strating that the potential of the fusion of structural and functional
data may improve the brain disease classification. For example,
mild cognitive impairment (MCI), often an early stage of
Alzheimer’s disease, is difficult to diagnose due to its rather mild
and nearly insignificant symptoms of cognitive impairment. When
trying to identify MCI patients from healthy controls, Kim et al.
[65] showed that the integration of sMRI and fMRI can provide
complementary information to improve the diagnosis of MCI
 Multimodal Fusion of Structural and Functional Brain Imaging Data 863

relative to results from one modality alone (error rate: 6 % using

both versus 15 % using fMRI only and 35 % using sMRI only) [66].
Similarly, Wee et al. [67] integrated information from DT MRI
and resting fMRI by employing multiple-kernel support vector
machines (SVMs), yielding statistically significant improvement
(>7.4 %) in classification accuracy of predicting MCI from healthy
controls by using multimodal data (96.3 %) compared to using
each modality independently. Furthermore, with the ensemble fea-
ture selection strategy and advance support vector machine, Westlet
et al. [68] combined resting-state fMRI, EEG, and sMRI data to
classify schizophrenia from healthy controls and achieved the best
performance with 91 % accuracy compared to each single modali-
ties, confirming the effectiveness and advantages of multimodal
fusion. The above findings suggest the multimodal classification
facilitated by advanced modeling techniques could provide more
accurate and early detection of brain abnormalities, which may not
have been revealed through separate uni-modal analyses as typi-
cally performed in the majority of neuroimaging experiments.

5 Conclusions

It is abundantly clear that there is a diverse and growing collection

of scientific tools available for noninvasively studying human brain
functioning and relating it to cognitive and behavioral measures.
Using these technologies, substantial progress has been made in
characterizing structural/functional brain abnormalities and their
interactions. In addition, a major goal in integrating/fusing
approaches is to capitalize on the relative strengths of each modal-
ity, providing results synergistically. All of the multimodal studies
that are reviewed in this paper are summarized and separated into
different modality categories in Table 1. In general, most studies
we reviewed demonstrate congruent effects across measurement
modalities and combining modalities does provide more differen-
tiating power among multiple diseases.
Although recent multimodal imaging results are promising
[19, 69], much work remains to be done. As the field of multi-
modal imaging is relatively new, most of the studies represent novel
findings; however, replication is needed to draw general conclu-
sions about structure–function relationships. Secondly, multimodal
fusion proves to be fruitful for a more informative understanding of
brain activity and disorders, but fusing as many modalities/features
as possible in the training sample does not guarantee best discrimi-
nation or classification between groups, as reported in [8, 70]; thus
it would be helpful to compare a combination of uni-modal and
multimodal results, as done in [71]. This work can be pursued in
future by using larger data sets and various modalities. Furthermore,
a main challenge in multimodal data fusion comes from
864 Jing Sui and Vince D. Calhoun

Table 1
Summary of studies combining measures of functional and structural MRI

Modality Focus Papers Subject type Methods

fMRI-sMRI Connectivity [6, 57, 59, 60, HC-MDD, HC-SZ, Correlational analysis
65, 72–80] HC-MCI-AD Multiple regression
HC-TBI
Covariance [36, 44, 67, HC-SZ, HC-MDD, jICA, mCCA
81–83] HC-AD
fMRI-DTI Connectivity [61–64, Healthy children, Correlational
84–92] HC-MDD, HC-SZ, analysis, SEM
HC-BP, HC-TBI Multiple Regression
HC-AD-MCI
Covariance [26, 38, 40] HC-SZ, HC-SZ-BP jICA, mCCA + jICA
Three-way fusion connectivity [93–97] HC-ADHD Correlational
Children-Adults analysis
HC-MDD Multiple regression
HC-psychotic
Covariance [8, 24, 25, 48, HC-SZ jICA, mCCA,
51, 70, HC-AD mCCA + jICA
98–101] HC-MCI-AD Linked ICA
SVM
Other fusion fMRI-EEG [16, 27–29, HC-SZ, HC-AD Correlational
applications DTI-sMRI 42, 48, HC-MCI, HC-BP analysis
GM-WM 102–110] Multiple regression
jICA, mCCA
PCA, PLS, parallel
ICA

dissimilarity of the data types being fused and result interpretation.

However, emerging in 2009, N-way multimodal fusion may
become one of the leading directions in future neuroimaging
research given the predominance of multimodal data acquisition
[5]. Finally, introducing multimodal analyses in longitudinal brain
studies has not been done frequently yet, which could be another
new direction, as there are many possibilities for modeling the base-
line and change over multiple time points.
In summary, we are just beginning to unlock the potential of
multimodal imaging, which provides unprecedented opportunities
to further deepen our understanding of the brain disorders [51]
based on various brain imaging measures. The most promising
avenues for the future may rest on developing better models that
can complement and exploit the richness of our data [15]. These
models may well already exist in other disciplines (such as machine
learning, machine vision, computational neuroscience, and behav-
ioral economics) and may enable the broader neurosciences to
access neuroimaging so that key questions can be addressed in a
theoretically grounded fashion.
 Multimodal Fusion of Structural and Functional Brain Imaging Data 865

References

1. Skudlarski P, Jagannathan K, Calhoun VD 15. Friston KJ (2009) Modalities, modes, and

et al (2008) Measuring brain connectivity: models in functional neuroimaging. Science
diffusion tensor imaging validates resting 326(5951):399–403
state temporal correlations. Neuroimage 16. De Martino F, Valente G, de Borst AW et al
43(3):554–561 (2010) Multimodal imaging: an evaluation of
2. Savopol F, Armenakis C (2002) Mergine of univariate and multivariate methods for
heterogeneous data for emergency mapping: simultaneous EEG/fMRI. Magn Reson
data integration or data fusion? Proc. ISPRS Imaging 28(8):1104–1112
3. Ardnt C, Loffeld O (1996) Information 17. Eichele T, Specht K, Moosmann M et al
gained by data fusion, Proc. SPIE, vol. 2784 (2005) Assessing the spatiotemporal evolution
4. Sui J, Adali T, Yu Q et al (2012) A review of of neuronal activation with single-trial event-
multivariate methods for multimodal fusion related potentials and functional MRI. Proc
of brain imaging data. J Neurosci Methods Natl Acad Sci U S A 102(49):17798–17803
204(1):68–81 18. Astolfi L, Cincotti F, Mattia D et al (2004)
5. Sui J, Huster R, Yu Q et al (2014) Function- Estimation of the effective and functional
structure associations of the brain: evidence human cortical connectivity with structural
from multimodal connectivity and covariance equation modeling and directed transfer func-
studies. Neuroimage 102P1:11–23 tion applied to high-resolution EEG. Magn
6. Schultz CC, Fusar-Poli P, Wagner G et al Reson Imaging 22(10):1457–1470
(2012) Multimodal functional and structural 19. Rykhlevskaia E, Gratton G, Fabiani M (2008)
imaging investigations in psychosis research. Combining structural and functional neuro-
Eur Arch Psychiatry Clin Neurosci 262(Suppl imaging data for studying brain connectivity:
2):S97–S106 a review. Psychophysiology 45(2):173–187
7. Horwitz B, Poeppel D (2002) How can 20. Oakes TR, Fox AS, Johnstone T et al (2007)
EEG/MEG and fMRI/PET data be com- Integrating VBM into the General Linear
bined? Hum Brain Mapp 17(1):1–3 Model with voxelwise anatomical covariates.
8. Calhoun VD, Adali T (2009) Feature-based Neuroimage 34(2):500–508
fusion of medical imaging data. IEEE Trans 21. Schlosser R, Gesierich T, Kaufmann B et al
Inf Technol Biomed 13(5):711–720 (2003) Altered effective connectivity during
9. Blum AL, Langley P (1997) Selection of rel- working memory performance in schizophre-
evant features and examples in machine learn- nia: a study with fMRI and structural equa-
ing. Artif Intell 97(1–2):245–271 tion modeling. Neuroimage 19(3):751–763
10. Smith SM, Fox PT, Miller KL et al (2009) 22. Calhoun VD, Adali T, Kiehl KA et al (2006)
Correspondence of the brain’s functional A method for multitask fMRI data fusion
architecture during activation and rest. Proc applied to schizophrenia. Hum Brain Mapp
Natl Acad Sci U S A 106(31):13040–13045 27(7):598–610
11. Goldberg-Zimring D, Mewes AU, Maddah 23. Correa N, Adali T, Calhoun VD (2007)
M et al (2005) Diffusion tensor magnetic Performance of blind source separation algo-
resonance imaging in multiple sclerosis. rithms for fMRI analysis using a group ICA
J Neuroimaging 15(4 Suppl):68S–81S method. Magn Reson Imaging 25(5):684–694
12. Ramnani N, Lee L, Mechelli A et al (2002) 24. Groves AR, Beckmann CF, Smith SM et al
Exploring brain connectivity: a new frontier (2011) Linked independent component anal-
in systems neuroscience. Functional Brain ysis for multimodal data fusion. Neuroimage
Connectivity, 4–6 April 2002, Dusseldorf, 54(3):2198–2217
Germany. Trends Neurosci 25:496–497 25. Sui J, He H, Pearlson GD et al (2012) Three-
13. Henson RN, Flandin G, Friston KJ et al way (N-way) fusion of brain imaging data
(2010) A parametric empirical Bayesian based on mCCA + jICA and its application to
framework for fMRI-constrained MEG/EEG discriminating schizophrenia. Neuroimage
source reconstruction. Hum Brain Mapp 2(66):119–132
31(10):1512–1531 26. Sui J, Pearlson G, Caprihan A et al (2011)
14. Lemieux L (2004) Electroencephalography- Discriminating schizophrenia and bipolar dis-
correlated functional MR imaging studies of order by fusing fMRI and DTI in a multi-
epileptic activity. Neuroimaging Clin N Am modal CCA+ joint ICA model. Neuroimage
14(3):487–506 57(3):839–855
866 Jing Sui and Vince D. Calhoun

27. Chen K, Reiman EM, Huan Z et al (2009) 39. Calhoun VD, Adali T, Liu J (2006) A feature-
Linking functional and structural brain images based approach to combine functional MRI,
with multivariate network analyses: a novel structural MRI and EEG brain imaging data.
application of the partial least square method. Conf Proc IEEE Eng Med Biol Soc 1:3672–3675
Neuroimage 47(2):602–610 40. Teipel SJ, Bokde AL, Meindl T et al (2010)
28. Martinez-Montes E, Valdes-Sosa PA, White matter microstructure underlying
Miwakeichi F et al (2004) Concurrent EEG/ default mode network connectivity in the
fMRI analysis by multiway Partial Least human brain. Neuroimage 49(3):2021–2032
Squares. Neuroimage 22(3):1023–1034 41. Eichele T, Calhoun VD, Debener S (2009) Mining
29. Correa NM, Eichele T, Adali T et al (2010) EEG-fMRI using independent component analy-
Multi-set canonical correlation analysis for the sis. Int J Psychophysiol 73(1):53–61
fusion of concurrent single trial ERP and func- 42. Xu L, Groth KM, Pearlson G et al (2009)
tional MRI. Neuroimage 50(4):1438–1445 Source-based morphometry: the use of
30. Liu J, Pearlson G, Windemuth A et al (2009) independent component analysis to iden-
Combining fMRI and SNP data to investi- tify gray matter differences with applica-
gate connections between brain function and tion to schizophrenia. Hum Brain Mapp
genetics using parallel ICA. Hum Brain Mapp 30(3):711–724
30(1):241–255 43. Shenton ME, Dickey CC, Frumin M et al
31. Sui J, Adali T, Pearlson GD et al (2009) A (2001) A review of MRI findings in schizo-
method for accurate group difference detec- phrenia. Schizophr Res 49(1–2):1–52
tion by constraining the mixing coefficients 44. Correa NM, Li YO, Adali T et al (2008)
in an ICA framework. Hum Brain Mapp Canonical correlation analysis for feature-
30(9):2953–2970 based fusion of biomedical imaging modalities
32. Sui J, Adali T, Pearlson GD et al (2009) An and its application to detection of associative
ICA-based method for the identification networks in schizophrenia. IEEE J Sel Top
of optimal FMRI features and components Signal Process 2(6):998–1007
using combined group-discriminative tech- 45. Li Y-O, Adali T, Wang W et al (2009) Joint
niques. Neuroimage 46(1):73–86 blind source separation by multiset canoni-
33. Caprihan A, Pearlson GD, Calhoun VD (2008) cal correlation analysis. IEEE Trans Signal
Application of principal component analysis to Process 57(10):3918–3929
distinguish patients with schizophrenia from 46. Lin FH, McIntosh AR, Agnew JA et al (2003)
healthy controls based on fractional anisotropy Multivariate analysis of neuronal interac-
measurements. Neuroimage 42(2):675–682 tions in the generalized partial least squares
34. Liu J, Xu L, Calhoun VD (2008). Extracting framework: simulations and empirical studies.
principle components for discriminant analy- Neuroimage 20(2):625–642
sis of FMRI images. ICASSP 449–452 47. Krishnan A, Williams LJ, McIntosh AR et al
35. Ulloa AE, Chen J, Vergara VM et al (2014) (2010) Partial Least Squares (PLS) meth-
Association between copy number varia- ods for neuroimaging: a tutorial and review.
tion losses and alcohol dependence across Neuroimage 56(2):455–475
African American and European American 48. Correa NM, Adali T, Li YO et al (2010)
ethnic groups. Alcohol Clin Exp Res Canonical correlation analysis for data fusion
38(5):1266–1274 and group inferences: examining applications
36. Calhoun VD, Adali T, Giuliani NR et al of medical imaging data. IEEE Signal Process
(2006) Method for multimodal analysis of Mag 27(4):39–50
independent source differences in schizophre- 49. McIntosh AR, Lobaugh NJ (2004) Partial
nia: combining gray matter structural and least squares analysis of neuroimaging data:
auditory oddball functional data. Hum Brain applications and advances. Neuroimage
Mapp 27(1):47–62 23(Suppl 1):S250–S263
37. Xu L, Pearlson G, Calhoun VD (2009) Joint 50. Camara E, Rodriguez-Fornells A, Munte
source based morphometry identifies linked TF (2010) Microstructural brain differences
gray and white matter group differences. predict functional hemodynamic responses
Neuroimage 44(3):777–789 in a reward processing task. J Neurosci
38. Franco AR, Ling J, Caprihan A et al (2008) 30(34):11398–11402
Multimodal and multi-tissue measures of 51. Sui J, He H, Yu Q et al (2013) Combination
connectivity revealed by joint independent of resting state fMRI, DTI, and sMRI data
component analysis. IEEE J Sel Top Signal to discriminate schizophrenia by N-way
Process 2(6):986–997 MCCA + jICA. Front Hum Neurosci 7:235
 Multimodal Fusion of Structural and Functional Brain Imaging Data 867

52. Beckmann CF, Smith SM (2005) Tensorial 65. Kim J, Lee JH (2012) Integration of struc-
extensions of independent component analysis tural and functional magnetic resonance
for multisubject FMRI analysis. Neuroimage imaging improves mild cognitive impairment
25(1):294–311 detection. Magn Reson Imaging
53. Hyvarinen A, Oja E (2000) Independent 31:718–732
component analysis: algorithms and applica- 66. Fan Y, Resnick SM, Wu X et al (2008)
tions. Neural Netw 13(4–5):411–430 Structural and functional biomarkers of pro-
54. Bell AJ, Sejnowski TJ (1995) An information- dromal Alzheimer’s disease: a high-
maximization approach to blind separation dimensional pattern classification study.
and blind deconvolution. Neural Comput Neuroimage 41(2):277–285
7(6):1129–1159 67. Wee CY, Yap PT, Zhang D et al (2011)
55. Bishop CM (1999) Variational principal com- Identification of MCI individuals using
ponents. Artif Neural Netw 7:509, structural and functional connectivity net-
Conference Publication No. 470 works. Neuroimage 59(3):2045–2056
56. Liu J, Chen J, Ehrlich S et al (2014) 68. Westlye LT, Walhovd KB, Bjornerud A et al
Methylation patterns in whole blood corre- (2009) Error-related negativity is mediated
late with symptoms in schizophrenia patients. by fractional anisotropy in the posterior cin-
Schizophr Bull 40(4):769–776 gulate gyrus—a study combining diffusion
57. Segall JM, Allen EA, Jung RE et al (2012) tensor imaging and electrophysiology in
Correspondence between structure and func- healthy adults. Cereb Cortex 19(2):293–304
tion in the human brain at rest. Front 69. Sui J, Yu Q, He H et al (2012) A selective
Neuroinform 6:10 review of multimodal fusion methods in
58. Khullar S, Michael AM, Cahill ND et al schizophrenia. Front Hum Neurosci 6:27
(2011) ICA-fNORM: spatial normalization 70. Zhang H, Liu L, Wu H, Fan Y (2012) Feature
of fMRI data using intrinsic group-ICA net- selection and SVM classification of multiple
works. Front Syst Neurosci 5:93 modality images for predicting MCI, in
59. Salgado-Pineda P, Junque C, Vendrell P et al OHBM, Beijing, China.
(2004) Decreased cerebral activation during 71. Kim DI, Sui J, Rachakonda S et al (2010)
CPT performance: structural and functional Identification of imaging biomarkers in
deficits in schizophrenic patients. Neuroimage schizophrenia: a coefficient-constrained inde-
21(3):840–847 pendent component analysis of the mind
60. Salgado-Pineda P, Fakra E, Delaveau P et al multi-site schizophrenia study.
(2011) Correlated structural and functional Neuroinformatics 8(4):213–229
brain abnormalities in the default mode net- 72. Tian L, Meng C, Yan H et al (2011)
work in schizophrenia patients. Schizophr Convergent evidence from multimodal imag-
Res 125(2–3):101–109 ing reveals amygdala abnormalities in schizo-
61. Skudlarski P, Jagannathan K, Anderson K et al phrenic patients and their first-degree
(2010) Brain connectivity is not only lower relatives. PLoS One 6(12):e28794
but different in schizophrenia: a combined 73. Casey BJ, Tottenham N, Liston C et al (2005)
anatomical and functional approach. Biol Imaging the developing brain: what have we
Psychiatry 68(1):61–69 learned about cognitive development? Trends
62. Schlosser RG, Nenadic I, Wagner G et al Cogn Sci 9(3):104–110
(2007) White matter abnormalities and brain 74. Smieskova R, Allen P, Simon A et al (2012)
activation in schizophrenia: a combined DTI Different duration of at-risk mental state asso-
and fMRI study. Schizophr Res ciated with neurofunctional abnormalities. A
89(1–3):1–11 multimodal imaging study. Hum Brain Mapp
63. Staempfli P, Reischauer C, Jaermann T et al 33(10):2281–2294
(2008) Combining fMRI and DTI: a frame- 75. Rasser PE, Johnston P, Lagopoulos J et al
work for exploring the limits of fMRI-guided (2005) Functional MRI BOLD response to
DTI fiber tracking and for verifying DTI- Tower of London performance of first-
based fiber tractography results. Neuroimage episode schizophrenia patients using cortical
39(1):119–126 pattern matching. Neuroimage
64. Koch K, Wagner G, Schachtzabel C et al 26(3):941–951
(2011) Neural activation and radial diffusivity 76. Fusar-Poli P, Broome MR, Woolley JB et al
in schizophrenia: combined fMRI and diffu- (2011) Altered brain function directly related
sion tensor imaging study. Br J Psychiatry to structural abnormalities in people at ultra
198(3):223–229 high risk of psychosis: longitudinal VBM-
fMRI study. J Psychiatr Res 45(2):190–198
868 Jing Sui and Vince D. Calhoun

77. Michael AM, Baum SA, White T et al (2010) 89. Wang F, Kalmar JH, He Y et al (2009)
Does function follow form?: methods to fuse Functional and structural connectivity
structural and functional brain images show between the perigenual anterior cingulate and
decreased linkage in schizophrenia. amygdala in bipolar disorder. Biol Psychiatry
Neuroimage 49(3):2626–2637 66(5):516–521
78. Michael AM, King MD, Ehrlich S et al (2011) A 90. Schonberg T, Pianka P, Hendler T et al
data-driven investigation of gray matter-function (2006) Characterization of displaced white
correlations in schizophrenia during a working matter by brain tumors using combined DTI
memory task. Front Hum Neurosci 5:71 and fMRI. Neuroimage 30(4):1100–1111
79. Rektorova I, Mikl M, Barrett J et al (2012) 91. Voss HU, Schiff ND (2009) MRI of neuronal
Functional neuroanatomy of vocalization in network structure, function, and plasticity.
patients with Parkinson’s disease. J Neurol Sci Prog Brain Res 175:483–496
313(1–2):7–12 92. Palacios EM, Sala-Llonch R, Junque C et al
80. Harms MP, Wang L, Csernansky JG et al (2012) White matter integrity related to
(2012) Structure-function relationship of functional working memory networks in trau-
working memory activity with hippocampal matic brain injury. Neurology
and prefrontal cortex volumes. Brain Struct 78(12):852–860
Funct 218(1):173–186 93. Jacobson S, Kelleher I, Harley M et al (2009)
81. Choi K, Yang Z, Hu X et al (2008) A com- Structural and functional brain correlates of
bined functional-structural connectivity anal- subclinical psychotic symptoms in 11–13 year
ysis of major depression using joint old schoolchildren. Neuroimage
independent components analysis. Psychiatric 49(2):1875–1885
MRI/MRS 3555 94. Supekar K, Uddin LQ, Prater K et al (2010)
82. Camchong J, MacDonald AW III, Bell C et al Development of functional and structural
(2011) Altered functional and anatomical connectivity within the default mode network
connectivity in schizophrenia. Schizophr Bull in young children. Neuroimage
37(3):640–650 52(1):290–301
83. Kim DJ, Park B, Park HJ (2012) Functional 95. Pomarol-Clotet E, Canales-Rodriguez EJ,
connectivity-based identification of subdivi- Salvador R et al (2010) Medial prefrontal cor-
sions of the basal ganglia and thalamus using tex pathology in schizophrenia as revealed by
multilevel independent component analysis of convergent findings from multimodal imag-
resting state fMRI. Hum Brain Mapp ing. Mol Psychiatry 15(8):823–830
34:1371–1385 96. Sexton CE, Allan CL, Le Masurier M et al
84. Olesen PJ, Nagy Z, Westerberg H et al (2003) (2012) Magnetic resonance imaging in late-
Combined analysis of DTI and fMRI data life depression: multimodal examination of
reveals a joint maturation of white and grey network disruption. Arch Gen Psychiatry
matter in a fronto-parietal network. Brain Res 69(7):680–689
Cogn Brain Res 18(1):48–57 97. Qiu MG, Ye Z, Li QY et al (2011) Changes of
85. Matthews SC, Strigo IA, Simmons AN et al brain structure and function in ADHD chil-
(2011) A multimodal imaging study in U.S. dren. Brain Topogr 24(3–4):243–252
veterans of Operations Iraqi and Enduring 98. Sui J, He H, Liu J et al (2012) Three-way
Freedom with and without major depression FMRI-DTI-methylation data fusion based on
after blast-related concussion. Neuroimage mCCA + jICA and its application to schizo-
54(Suppl 1):S69–S75 phrenia. Conf Proc IEEE Eng Med Biol Soc
86. Zhou Y, Shu N, Liu Y et al (2008) Altered 2012:2692–2695
resting-state functional connectivity and ana- 99. Zhang D, Wang Y, Zhou L et al (2011)
tomical connectivity of hippocampus in Multimodal classification of Alzheimer’s dis-
schizophrenia. Schizophr Res ease and mild cognitive impairment.
100(1–3):120–132 Neuroimage 55(3):856–867
87. Yan H, Tian L, Yan J et al (2012) Functional 100. Groves AR, Smith SM, Fjell AM et al (2012)
and anatomical connectivity abnormalities in Benefits of multi-modal fusion analysis on a
cognitive division of anterior cingulate cortex large-scale dataset: life-span patterns of inter-
in schizophrenia. PLoS One 7(9):e45659 subject variability in cortical morphometry and
88. Soldner J, Meindl T, Koch W et al (2011) white matter microstructure. Neuroimage
Structural and functional neuronal connectiv- 63(1):365–380
ity in Alzheimer’s disease: a combined DTI 101. Sui J, Castro E, Hao H et al (2014) Combination
and fMRI study. Nervenarzt 83(7):878–887 of FMRI-SMRI-EEG data improves discrimina-
 Multimodal Fusion of Structural and Functional Brain Imaging Data 869

tion of schizophrenia patients by ensemble fea- 106. Jagannathan K, Calhoun VD, Gelernter J et al
ture selection. The 36th Annual International (2010) Genetic associations of brain structural
Conference of the IEEE engineering in medi- networks in schizophrenia: a preliminary
cine and biology society (EMBC‘14) Chicago, study. Biol Psychiatry 68(7):657–666
Illinois, USA, no. August 26–30 107. Jamadar S, Powers NR, Meda SA et al (2010)
102. Eichele T, Calhoun VD, Moosmann M et al Genetic influences of cortical gray matter in
(2008) Unmixing concurrent EEG-fMRI language-related regions in healthy controls and
with parallel independent component analy- schizophrenia. Schizophr Res 129:141–148
sis. Int J Psychophysiol 67(3):222–234 108. Hao X, Xu D, Bansal R et al (2011) Multimodal
103. Haller S, Xekardaki A, Delaloye C et al (2011) magnetic resonance imaging: the coordinated
Combined analysis of grey matter voxel-based use of multiple, mutually informative probes to
morphometry and white matter tract-based understand brain structure and function. Hum
spatial statistics in late-life bipolar disorder. Brain Mapp 34(2):253–271
J Psychiatry Neurosci 36(6):391–401 109. Fusar-Poli P, McGuire P, Borgwardt S (2011)
104. Chen Z, Cui L, Li M et al (2011) Voxel based Mapping prodromal psychosis: a critical
morphometric and diffusion tensor imaging review of neuroimaging studies. Eur
analysis in male bipolar patients with first- Psychiatry 27(3):181–191
episode mania. Prog Neuropsychopharmacol 110. Meda SA, Gill A, Stevens MC et al (2012)
Biol Psychiatry 36(2):231–238 Differences in resting-state functional mag-
105. Meda SA, Jagannathan K, Gelernter J et al netic resonance imaging functional network
(2010) A pilot multivariate parallel ICA study connectivity between schizophrenia and psy-
to investigate differential linkage between chotic bipolar probands and their unaffected
neural networks and genetic profiles in schizo- first-degree relatives. Biol Psychiatry
phrenia. Neuroimage 53(3):1007–1015 71(10):881–889
 Chapter 29

Functional MRI of the Spinal Cord

Patrick Stroman and Massimo Filippi

Abstract
Evidence to date shows that fMRI of the spinal cord (spinal fMRI) can reliably demonstrate regions
involved with sensation of tactile, thermal, and painful stimuli, and with motor tasks. Spinal fMRI acquisi-
tion methods based on BOLD contrast have been recently optimized. Results have demonstrated the
ability of spinal fMRI to provide objective assessments of sensory and motor function, and discriminate
responses when modulated by cognitive/emotional factors. Studies have been also carried out with patients
with cord trauma, and in people with multiple sclerosis (MS). The availability of essentially automated
analysis, large extent coverage of the spinal cord, and spatial normalization to permit comparisons with
reference results and labeling of active regions are being implemented with the aim to translate the method
into a practical clinical assessment tool.
The research completed so far indicates that spinal fMRI will be able to demonstrate where the neu-
ronal activity is altered at any level (cervical, thoracic, lumbar, or sacral), whether or not information is
reaching the cord from the periphery, and whether or not there is descending modulation of the response.
It may also be able to provide an objective measure of pain, and to demonstrate the extent and mechanism
of changes over time after an injury.

Key words Spinal fMRI, Blood oxygen level dependent, Multiple sclerosis, Cord trauma, Pain

1 Introduction

Evidence to date shows that fMRI of the spinal cord (spinal fMRI)
can reliably demonstrate regions involved with sensation of tactile,
thermal, and painful stimuli, and with motor tasks. There is also
reliable evidence of the descending modulation of activity in the
spinal cord. While spinal fMRI has not yet been applied or verified
in a clinical setting, its value is expected to be in its ability to pro-
vide objective assessments of sensory and motor function, and dis-
criminate responses when modulated by cognitive/emotional
factors, or even detect responses when a patient cannot feel the
stimulus or is even conscious. Studies have been carried out with
patients with cord trauma, and in people with multiple sclerosis
(MS) to investigate the clinical utility of the results. Robust meth-
ods for analysis, and for displaying the results in an effective m
­ anner

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1_29, © Springer Science+Business Media New York 2016

871
872 Patrick Stroman and Massimo Filippi

to facilitate their interpretation, are also necessary. At present, the

usefulness and reliability of spinal fMRI as a research tool has been
demonstrated, analysis and display methods have been developed,
and further improvements are rapidly developing. The research
completed so far indicates that spinal fMRI will be able to demon-
strate where the neuronal activity is altered at any level (cervical,
thoracic, lumbar, or sacral), whether or not information is reaching
the cord from the periphery, and whether or not there is descend-
ing modulation of the response. It may also be able to provide an
objective measure of pain, and to demonstrate the extent and
mechanism of changes over time after an injury.
In the following paragraphs, we discuss the current evidence
for the most effective spinal fMRI method and the points that are
still under debate, the applications that have been carried out to
date and the degree of reliability and sensitivity these studies dem-
onstrate, and the proposed future developments and applications.

2 Background of Spinal fMRI

As with any fMRI method, spinal fMRI requires alternated

periods of stimulation and a reference condition, while a time-
series of images is acquired over several minutes. Neuronal
activity is detected only in gray matter regions, and is revealed
by the local MR signal intensity having a component of signal
change that corresponds with the stimulation paradigm. Unlike
conventional brain fMRI, unique challenges are encountered in
the heterogeneous tissues of the spine and spinal cord, because
differences in magnetic susceptibilities between tissues produce
spatial variations in the magnetic field. The spinal cord itself
lies within the spinal canal surrounded by cerebrospinal fluid,
and averages 45 cm in length, with cross-sectional dimensions
of roughly 15 mm × 8 mm in the largest regions of the cervical
and lumbar enlargements (Fig. 1). The entire cord is therefore
in close proximity to the heart and lungs, and has been observed
to move with each heart-beat, presumably as a result of the
pulsatile CSF flow around it [1, 2]. An inherent challenge of
fMRI is that it is necessary to monitor the signal intensity
changes in a specific tissue volume, and this tissue may move
between adjacent voxels over the course of the fMRI time-
series, obscuring the measurements that are obtained. Moreover,
other sources of signal intensity change such as random noise
and motion artifacts can interfere with the detection of signal
changes related to neuronal activity. While these challenges
may seem daunting, most have been overcome by adapting
methods for analysis and motion correction that have been
developed for brain fMRI.
 Functional MRI of the Spinal Cord 873

Fig. 1 Anatomy of the spinal cord (reproduced with permission from Blumenfeld H., Neuroanatomy Through
Clinical Cases. Sunderland, MA: Sinauer Associates; 2002, p 22)
874 Patrick Stroman and Massimo Filippi

3 Spinal fMRI Acquisition Methods

The first published example of fMRI in the spinal cord was by

Yoshizawa et al. in 1996 [3]. This work and the earliest attempts by
other groups applied the established brain fMRI methods of the time
to the spinal cord [4, 5]. The consistent features of the studies by
Yoshizawa et al. [3], Stroman et al. [6], Madi et al. [7], and Backes
et al. [8] were that they were carried out with healthy volunteers and
employed a hand motor task with imaging of the cervical spinal cord.
All used gradient-echo methods with echo times (TE) of 40–50 ms
at 1.5 T, and 31 ms at 3 T, as is typical with brain fMRI, and all com-
pared data obtained with transverse and sagittal slices. The areas of
activity in the spinal cord that were demonstrated by these studies
corresponded to areas of neuronal activity that were expected with
the stimuli applied. The conclusions reported from each of these
studies included the point that spinal fMRI is a feasible method for
assessing neuronal activity in the cord. However, the results obtained
also demonstrated variability in the areas of activity, and that it is dif-
ficult to obtain high-quality fMRI data in the spinal cord with gradi-
ent-echo methods and sensitivity to the BOLD effect.
The debate over the choice of imaging method was started
with a study designed to verify that the BOLD effect occurred in
the spinal cord [9], by comparing data obtained with gradient-­
echo and spin-echo methods at the same echo times. This was fol-
lowed by a study to characterize the signal changes detected with
spin-echo methods [10]. The BOLD theory shows that with the
comparison in the first of these studies the gradient-echo method
should produce signal intensity changes between rest and stimula-
tion conditions that are three to four times higher than the spin-­
echo method [11]. The results, however, showed that the two
methods produced signal intensity changes of approximately equal
magnitudes, the image quality obtained with the spin-echo method
was superior, and that the spin-echo method may therefore be
superior for spinal fMRI. However, the apparent departure from
the expected BOLD responses was unexplained.
A number of studies followed which investigated the underly-
ing contrast mechanisms, and it was proposed that changes in tissue
water content related to neuronal activity could augment the BOLD
contrast at short echo time (TE) values [12–16]. However, it has
been shown that the optimal contrast-to-noise ratio with spin-echo
fMRI in the spinal cord is obtained when the TE value is set for
optimal spin-echo BOLD contrast, at approximately 75 ms [17]. As
a result, whether spin-echo or gradient-echo imaging methods are
used for spinal cord fMRI, the contrast provided is BOLD, and the
two methods have roughly equal sensitivity when optimized, as
detailed below. The debate over the choice of imaging method con-
tinues however, between gradient-echo methods which provide
 Functional MRI of the Spinal Cord 875

greater speed, and spin-echo methods which p ­ rovide better image

quality. The real advantage of spin-echo methods was in its lower
sensitivity to the nonuniform magnetic field environment in the
spinal cord, and the ability to acquire images quickly without resort-
ing the echo-planar imaging (EPI) spatial encoding methods.

3.1 Gradient-Echo As mentioned above, the earliest results with BOLD methods [3,
Methods 6–9, 18] showed promise that spinal fMRI is feasible. Yoshizawa
et al. demonstrated areas of activity with a hand motor task (average
signal changes 4.8 %) corresponded with consistent areas of the
spinal cord gray matter; they also showed that the rostral-caudal
distribution of activity corresponded well with the neuroanatomy
[3]. This was the first spinal fMRI study, and it set the standard for
the studies which followed by Stroman et al. [6, 9], Madi et al. [7],
and Backes et al. [8], which had a number of similarities. As in the
study by Yoshizawa et al., each of these employed gradient-echo
imaging (fast gradient-echo or echo-­planar encoding), relatively
thick (5–10 mm) transverse slices, with the echo-time (TE) set for
BOLD sensitivity. All of these studies investigated activity with
motor tasks, two investigated activity with sensory stimuli as well
[6, 9], and three of them [6–8] compared results obtained with
sagittal (4–8 mm) and transverse slices.
The results of these studies showed a number of consistent fea-
tures (Fig. 2). Signal intensity changes with hand motor tasks were
consistently in the range of 4.3–4.8 % by Yoshizawa et al. [3] and
Stroman et al. [6], and 0.5–7.5 % with graded force tasks by Madi
et al. [7], and 8–12 % by Backes et al. [8], although there were

Fig. 2 Example of spinal fMRI results obtained with gradient-recalled echo EPI in transverse slices of the cervi-
cal spinal cord (reproduced from Stroman and Ryner. Magn. Reson. Imaging 19: 27–32, 2001) [9]. (a) The left
side of the body is at the top of this image and the red marks indicate the locations which underwent intensity
changes in response to sensory stimulation of the right hand. The location of the slice is indicated in relation
to the sagittal view of the cervical spine shown in the larger image. (b) Five transverse slices corresponding to
the slices indicated in the sagittal view
876 Patrick Stroman and Massimo Filippi

differences in field strength (1.5 T vs. 3 T) and image resolution

which complicate any direct comparison. Each of these studies
showed a rostral-caudal dependence of the areas with the task being
performed, but only about half observed apparent laterality (left vs.
right) of the active regions [3, 6]. Backes et al. [8] pointed out that
there may be problems with the sensitivity to draining veins and the
small anatomy, and the differences in spatial localization may simply
be attributed to the extent of the draining vein field for the area
being stimulated (ventral vs. dorsal). Another significant feature
introduced by Backes et al., was the use of cardiac gating. In their
results the cardiac gating can be expected to have significantly
reduced the effects of cerebrospinal fluid (CSF) flow and spinal
cord motion, and their results also showed the sensitivity of the
BOLD methods to the draining veins leading from the gray matter
to cord surface, as shown in more recent studies as well [19].
Another consistent feature of many of these studies was that
they used echo-planar imaging (EPI), and made efforts to reduce
the data readout time in order to reduce image distortion, includ-
ing high bandwidth and sampling on the gradient ramps [7] or
multishot acquisitions [8]. Maieron et al. [20] employed methods
of parallel imaging (SENSE encoding) with EPI to reduce the dis-
tortion effects for spinal fMRI. This method showed improve-
ments over previous methods but spatial distortions and variations
in sensitivity in the rostral-caudal direction along the spinal cord
were clearly visible. Even with extensive efforts to reduce distor-
tions in images obtained with EPI encoding the image data suffer
from severe distortions and areas of signal dropout, and they can-
not be used in the presence of implanted devices to stabilize the
spine after an injury.
More recent examples of applications of spinal fMRI with
gradient-­echo methods include one by Summers et al. who investi-
gated details of BOLD responses to innocuous and noxious heat
applied to the hand [21]. Their results demonstrated activity spread
across a small S/I range in ipsilateral dorsal and contralateral ventral
spinal cord, and BOLD time-course responses were shown with
good correspondence with the timing of the thermal stimulation.
This study was carried out an 3 T and employed a segmented gradi-
ent-echo EPI sequence, with an echo time of 23 ms, temporal reso-
lution of 4 s, with images acquired in four segments (segment
TR = 1 s). Fifteen, 4-mm-thick, axial slices were imaged with a field
of view of 186 × 140 mm, acquisition matrix of 170 × 108 and a
reconstructed voxel size of 0.98 × 0.98 × 4 mm. The approach of
dividing the acquisition into multiple segments allowed for a larger
sampling matrix, reduced the distortion, and provided greater spa-
tial precision in the results. Several other studies have also used gra-
dient-echo EPI methods, with details below, and have focused on
pain processing and changes with cognitive state such as attention,
placebo, and nocebo [22–26]. These studies were focused primarily
 Functional MRI of the Spinal Cord 877

on sensory or pain processing, and effects such as placebo, nocebo,

and attention modulation. Four of these examples used almost iden-
tical methods at 3 T, and used gradient-echo EPI acquisitions with
10–12 axial slices which were 5 mm thick, with slice-specific z-shim-
ming, with in-plane resolution of 1 mm × 1 mm with a 128 × 128
matrix, and parallel imaging (GRAPPA) with an acceleration factor
of 2 [23–26]. The repetition times ranged from 1.17 to 1.5 s, and
the TE was 40–42 ms. One recent study was slightly different and
employed spiral encoding instead of EPI [22]. In this study a
128 × 128 matrix was used, with 1.25 mm × 1.25 mm in-plane reso-
lution, with 4 mm thick slices, with a 1.25 s TR and TE of 25 ms. In
each of these examples, BOLD responses were detected in the spinal
cord, ipsilateral to the stimulus, and were highly localized. In many
cases the activity was reported at a single location in the spinal cord.

3.2 Spin-Echo One of the earliest spinal fMRI studies [10] was a comparison of
Methods the signal intensity time-course properties obtained with T2-­
weighted and T2*-weighted acquisitions. The intent was to inves-
tigate whether the BOLD effect occurred in the spinal cord as in
the brain. The T2-weighted data had signal changes that were as
large, or larger, than the T2*-weighted data at approximately the
same echo time, and so the observations were not entirely consis-
tent with the BOLD model. More importantly, the results indi-
cated that spinal fMRI is feasible with both motor and sensory
stimulation at 1.5 T, and can be achieved with good image quality
with spin-echo imaging methods. A series of studies followed, to
investigate the biophysical nature of the underlying contrast mech-
anism, and consistently showed significant BOLD effects, as well as
a contribution from a proton-density change which was greater at
shorter echo times [12–16, 27–30]. However, a key observation
across these studies was that the spatial encoding method—EPI or
fast spin-echo—is a critical factor in the choice of methods. A
detailed analysis of the methods, including characterizations of the
noise, physiological motion, and sensitivity to neuronal activity,
demonstrated several key findings [17]. These included that it is
important to avoid EPI methods for spinal fMRI, and that optimal
sensitivity is obtained with spin-echo fMRI with an echo time (TE)
of 75 ms (at 3 T), which corresponds with the T2 of the spinal cord
tissue. This finding agrees with the established BOLD theory. This
increase in TE, compared to methods used in earlier studies, repre-
sented a small (20 %) but significant increase in sensitivity. These
findings also confirmed that the previous studies with an echo time
of 38 ms were likely dominated by BOLD contrast, with only a
small contribution from the proton-density change [31–37]. This
point is important because it means that earlier studies done with a
shorter echo time, and the more recent studies that are optimized
for BOLD contrast, are dominated by the same contrast mecha-
nism and the results are comparable.
878 Patrick Stroman and Massimo Filippi

The applications of spinal fMRI with spin-echo fMRI methods

have included studies of sensory responses with thermal and vibra-
tion stimuli, pain processing, and effects of traumatic spinal cord
injury and diseases such as MS. Lawrence et al. demonstrated the
ability to localize activity in the spinal cord with vibration stimula-
tion of different sensory dermatomes [38]. The spatial precision was
further demonstrated with a later study of somatotopic mapping of
thermal sensory responses in the cervical spinal cord and brainstem,
with localization of C5 and C8 dermatomes and distinct right/left
responses, as well as corresponding responses in brainstem regions
[39]. A study of response characteristics compared constant thermal
stimuli (i.e. block design) and a stimulus that gradually increased in
intensity in a staircase pattern [40]. The static and dynamic thermal
stimulation paradigms were designed to stimulate different periph-
eral receptors, and the comparison of the fMRI responses revealed
significant differences in the spinal cord and brainstem in terms of
the extent of the activity and the time-courses of the BOLD
responses. These studies serve to demonstrate the spatial precision
and the sensitivity of current spinal fMRI methods.
Several pain studies have also been carried out to date using spi-
nal fMRI with spin-echo methods. One study demonstrated different
responses in the spinal cord and brainstem with innocuous and nox-
ious thermal stimuli [31]. BOLD responses were observed to be cor-
related with individual pain ratings in both the ipsilateral dorsal spinal
cord, corresponding to the segment that was stimulated, and in the
midbrain near the periaqueductal gray matter (PAG). Innocuous and
noxious touch and brush stimuli were also compared in a separate
study, and results demonstrated differences in fMRI responses in the
spinal cord and brainstem [32]. Thermal pain responses in the spinal
cord and brainstem have also been shown to be modulated by manip-
ulating the attention focus of research participants [41], and by sen-
sitizing the skin with capsaicin [35, 36]. Recently, the analgesic effects
of listening to preferred music were investigated, and demonstrated
moderate pain relief and corresponding changes in the cervical spinal
cord and brainstem [42]. These studies consistently demonstrate a
distribution of active regions in the cervical spinal cord with ipsilateral
dorsal activity with a small rostral-caudal spread, within the stimu-
lated segment of the cord and occasionally within adjacent segments,
and contralateral ventral activity. Depending on the stimulus and
study conditions, contralateral dorsal activity has also been observed.
In addition, corresponding brainstem activity is consistently detected
in studies that span the cervical spinal cord and brainstem. The active
regions that have been detected, and their variation with study condi-
tions, consistently demonstrate the role of descending modulation of
sensory and pain responses in the spinal cord from brainstem regions.
These represent important findings for our understanding of human
pain processing, and the methods that have been developed show
promise as tools for characterizing individual pain states.
 Functional MRI of the Spinal Cord 879

A considerable component of the potential value of spinal cord

fMRI methods depends on what they can reveal about pathological
processes in the spinal cord, such as the effects of injury or disease or
aberrant pain conditions. Several spinal fMRI studies have been car-
ried out by the same group to investigate the effects of MS on spinal
cord function [43–49]. These studies used tactile stimuli with spinal
fMRI to probe sensory changes in MS, and demonstrated differences
in responses in the cervical spinal cord depending on the MS sub-
type. Specifically, they observed that cord recruitment was increased
in progressive MS patients compared to healthy control participants
and in SPMS compared to PPMS patients. This finding also builds on
prior studies that showed greater activity in RRMS and SPMS patients
compared to controls, but no difference between patient groups.
The effects of traumatic spinal cord injury on thermal sensory/pain
processing have also been studied with spinal fMRI, even in patients
with implanted devices to stabilize the spine. Cadotte et al. observed
increased fMRI responses to thermal stimulation in the cervical spinal
cord after injury, providing evidence of plasticity [50]. This work was
built upon prior studies that demonstrated spinal cord responses to
sensory stimulation or attempted movement tasks, below the site of
injury [51, 52]. Again in these earlier studies, the fMRI responses
were noted to be larger after spinal cord injury in many participants,
compared to a group of healthy control participants.
The spin-echo methods that are widely used for spinal fMRI
have also been quite similar across a number of recent studies, with
an example shown in Fig. 3 [32, 35–37, 39–42, 45, 48, 50, 52].

Fig. 3 Example of T2-weighted fMRI data acquired with a half-Fourier single-shot

fast spin-echo (HASTE) sequence at 3 T, with 1.5 mm × 1.5 mm in-plane resolu-
tion, 2 mm thick slices, a 192 × 144 acquisition matrix, and echo time of 76 ms
880 Patrick Stroman and Massimo Filippi

These studies were focused primarily on sensory or pain processing,

and effects of attention focus, mood, or stimulation modality on
activity detected in the spinal cord, or the effects of MS or spinal
cord injury on sensory processing. Most of these examples used
very similar methods at 3 T, and used single-shot fast spin-echo
acquisitions, with partial k-space sampling (such as the HASTE
sequence) with nine to ten contiguous sagittal slices which were
2 mm thick, with in-plane resolution of 1.5 mm × 1.5 mm with a
192 × 144 matrix. Parallel imaging has not been used in any of these
studies to date. The repetition times ranged from 6.75 to 9 s. In
earlier studies the TE was typically 32–40 ms. These studies
employed BOLD contrast with a contribution from changes in pro-
ton-density, due to changes in tissue water content [15]. More
recent studies have been carried out with longer TE values of 76 ms
to optimize the BOLD contrast, as described above [40, 42].

4 Comparison of the Currently Most-Used Spin-Echo and Gradient-Echo fMRI

Methods

The spinal fMRI methods that have been used in studies to date
appear to have converged on two widely-used methods, one based
on spin-echo to avoid EPI spatial encoding, and the other based on
gradient-echo EPI. In terms of the magnitude of the BOLD response
that is expected, when the two methods are both optimized, they
can be expected to be approximately equal, as detailed in Table 1.
In Table 1 the terms Δ(1/T2*) and Δ(1/T2) refer to the changes
in transverse relaxation rates between two conditions, such as a
“baseline” state and “active” state. It is well established that, under
the same experimental conditions, Δ(1/T2*) is three to four times
larger than Δ(1/T2) and hence T2*-weighted imaging is most fre-
quently used for fMRI [11, 28]. However, another important fac-
tor is that the echo time, TE, for optimal BOLD contrast is equal
to the transverse relaxation time (i.e., T2* or T2) which is roughly
three times larger for spin-echo than gradient-echo. The net effect
is that the magnitude of the BOLD response detected with spin-
echo methods is nearly equal to that with gradient-echo methods,
when both methods are set for optimal BOLD sensitivity.
The signal-to-noise ratio and image quality are also important
considerations for fMRI methods. The SNR can be compared
between imaging methods with the expression [53]:

total imaging volume (

- TE / T2* or T2 ) 1
SNR µ e
N x N y BW acceleration factor

This equation applies if T1-weighting can be ignored and a roughly

90° flip angle is used for both methods. The “total imaging volume”
is the image field-of-view multiplied by the slice thickness, the values
 Functional MRI of the Spinal Cord 881

Table 1
Comparison of the expected BOLD response magnitude detected with
gradient-echo and spin-echo methods

Gradient-echo Spin-echo

DS æ 1 ö DS æ 1 ö
@ -TE D ç * ÷ @ -TE D ç ÷
S è T2 ø S è T2 ø
DS DS
@ -0.025 s ´ -1.22s -1 @ -0.075 s ´ -0.37 s -1
S S
DS DS
@ 0.031 @ 0.028
S S

Estimates are for data acquired at 3 T, in the spinal cord, with TE values set for optimal
BOLD contrast. The values for the expected changes in relaxation rates, Δ(1/T2*) and
Δ(1/T2), are taken from [28]

of Nx and Ny are the image acquisition matrix dimensions, and the

“acceleration factor” refers to the parallel imaging acceleration fac-
tor. With the acquisition parameters used in the most recent meth-
ods, the spin-echo method is expected to have more than three
times higher SNR than the gradient-echo method (Table 2).
However, another important factor is the acquisition speed, because
the number of volumes that are acquired to detect the BOLD
responses influences the sensitivity. Rearranging the expression
described by Murphy et al. [54] to estimate the number of volumes
needed, we can estimate the effect size (i.e., the % BOLD change)
that can be detected for a given number of volumes (N):

erfc -1 ( p ) 2
eff =
SNR NR (1 - R )

where “eff” is the effect size, “p” is the statistical threshold used,
the function “erfc−1” is the inverse complementary error function,
and “R” is the proportion of time spent in the stimulation condi-
tion, assuming a block design with only two conditions. For the
purposes of this comparison we can set R = 0.5, and p = 10−6, which
corresponds to erfc−1(p) = 3.46. An estimate of the corresponding
t-value is given by √2 erfc−1(p), which is equal to 5.0. Using these
numbers we can compare the relative sensitivities of the methods,
in terms of the % BOLD signal change that can be detected with a
fixed acquisition duration (Table 2).
These estimates show that the faster sampling of the gradient-­
echo EPI method offsets its lower SNR and improves its sensitivity,
but it does not reach the sensitivity of the spin-echo HASTE method.
With either of these methods the duration of the fMRI acquisitions
can be increased to provide greater sensitivity, within practical limits.
882 Patrick Stroman and Massimo Filippi

Table 2
Estimates of SNR for commonly used gradient-echo and spin-echo spinal fMRI acquisitions

Gradient-echo spinal
Typical brain fMRI fMRI Spin-echo spinal fMRI
Imaging 3.3 mm × 3.3 mm 1 mm × 1 mm 1.5 mm × 1.5 mm
parameters 3.3 mm thick slice 5 mm thick slice 2 mm thick slice
200 kHz bandwidth 200 kHz bandwidth 151 kHz bandwidth
64 × 64 matrix 128 × 128 matrix 192 × 144 matrix
TE = T2* TE: 43 ms (~1.7 T2*) TE: 75 ms (T2)
Acceleration factor = 1 (no acceleration factor = 2 acceleration factor = 1
parallel imaging assumed)
Estimated SNR 150 15 56
Acquisition time/ 3s 1.1 s 6.75 s
volume
Estimated effect 0.42 % 2.6 % 1.7 %
size
p = 10−6, 12 min
acquisition
SNR values are estimated compared to a typical brain fMRI method which is assumed to have an SNR of approximately
150 at 3 T

Ultimately, the number of volumes that is acquired influences the

sensitivity for detecting BOLD responses, not the sampling rate [53].
The final factor to be considered when comparing the spin-­echo
and gradient-echo methods is the image quality that is p ­ rovided, as
shown in Fig. 4. Spinal fMRI acquisitions with gradient-echo EPI
methods employ axial slices in all of the examples cited above,
because axial slices appear to provide better image quality. However
sagittal views are extracted from the volume spanned by axial slices,
it can be seen that the images are still severely spatially distorted, and
the distortion depends on the rostral-­caudal position. Slice-specific
shimming has been shown to improve the image quality over that
shown in Fig. 4, but it does not eliminate the distortions [55]. The
net trade-off of using EPI methods is that they introduce more
problems than they solve, and provide higher temporal resolution at
the expense of loss of spatial fidelity, less anatomical coverage,
increased physiological noise, lower SNR, and lower BOLD sensitiv-
ity, compared to single-shot fast spin-­echo methods.

5 Recent Developments

Recent studies have further developed spinal fMRI methods with

regard to improving study designs, our understanding of the noise
characteristics, and analysis methods. One of the greatest technical
 Functional MRI of the Spinal Cord 883

Fig. 4 Comparison of image quality obtained with gradient-echo EPI and spin-echo HASTE sequences for
spinal fMRI. Gradient-echo EPI images were acquired in contiguous axial slices (left panel) and were reformat-
ted into sagittal views (middle frame) for comparison with spin-echo HASTE images acquired in sagittal planes
(right frame). Selected axial slices are shown for comparison, and the slice positions are indicated relative to
the caudal medulla (top slice)

challenges encountered with spinal fMRI is the physiological noise

arising from multiple sources. Cardiac-related motion has been
identified as arising from CSF flow, related movement of the spinal
cord within the spinal canal, and artifacts related to heart move-
ment itself, and the motion has been shown to be a complex inter-
action of several sources [1, 56, 57]. A recent study has identified
an additional important component arising from artifactual move-
ment of the spinal cord in gradient-echo EPI images, in relation to
respiration [58]. Changes in the static magnetic field, B0, due to
changes in lung volume have been shown to cause a shift of the
apparent position of the spine/spinal cord as large as 10 mm in
MR images, when acquired with EPI methods. The shift also varies
with rostral/caudal position along the spine. Understanding this
source of artifactual movement can be expected to improve the
sensitivity and reliability of spinal fMRI methods. The noise char-
acteristics in spin-echo HASTE spinal fMRI data has also been
investigated recently, and have shown that in spite of the multiple
sources of cardiac- and respiratory-related movement, the noise is
884 Patrick Stroman and Massimo Filippi

essentially random with negligible auto-correlation [17]. That is,

the physiological noise is uncorrelated between successive volumes
due to the relatively slow sampling rate. The authors proposed that
the most effective way to reduce the impact of physiological noise
is therefore to simply acquire as much data as possible.
Another recent important development is the ability to accu-
rately coregister 3D spinal cord data. The Medical Image
Registration Toolbox (MIRT) includes a nonrigid 3D registration
tool that has been shown to be effective at reducing noise in spinal
fMRI data acquired in sagittal planes [17, 59, 60]. In addition, an
automated spatial normalization method has been developed
recently, and with it a normalized 3D anatomical template of the
cervical spinal cord and brainstem. A previous user-guided normal-
ization method has been described [17, 61], and this was used to
create a reference template using data from 356 healthy partici-
pants. An iterative process was used to normalize the individual
image data to this template using the automated method, and each
iteration resulted in more precisely coregistered images from indi-
viduals and a more reliable reference template. The resulting tem-
plate consists of 1 mm × 1 mm × 1 mm voxels spanning the entire
cervical spinal cord, brainstem, medial portion of the thalamus,
and includes the corpus collosum.
The automated normalization process consists of first interpo-
lating the data to 1 mm cubic voxels and matching the orientation
to the reference template. Predefined sections of the template are
then matched to sections of the image data using the location at the
maximum cross-correlation, with template sections rotated over a
small range of angles. The first section identified includes the corpus
callosum and thalamus, because of their distinct features. In subse-
quent sections the position and angle are weighted towards pre-
dicted values based on prior segments resulting in a stable mapping
process. The distance along the cord (moving caudally) from the
pontomedullary junction is matched in the template and image data
to ensure that the cord anatomy is not altered in length due to a lack
of rostral/caudal features in the cord tissue [62]. The final step of
the normalization process was to fine-tune the mapping to the nor-
malized template using the MIRT toolbox [60]. The resulting tem-
plate is shown in Fig. 5. The automated normalization process can
be applied to data from individuals acquired in sagittal planes with
spin-echo methods, to enable mapping of data to the normalized
space for group analyses, group contrasts, second-level analyses, etc.
The normalized 3D template has also been used to define an
anatomical region mask for the cervical spinal cord and brainstem.
The region locations and extents were compiled from numerous
anatomical atlases and published papers [63–65]. This mask has
been used to extract fMRI data from specific anatomical regions for
regions-of-interest analyses, and for effective connectivity analyses
using Structural Equation Modeling (SEM) [40, 42]. A SEM
 Functional MRI of the Spinal Cord 885

Fig. 5 A 3D-normalized reference template for the cervical spinal cord, brain-
stem, and medial regions of the thalamus and corpus collosum. Image views
through the center of the volume are shown in axial (upper left), sagittal (upper
right), and coronal (lower right) slices

method has been developed for spinal cord fMRI data, written in
MatLab® (The Mathworks Inc.), based on prior descriptions [66,
67]. The method is based on identifying coordinated BOLD
responses between regions, accounting for the fact that input to one
region may arise simultaneously from multiple other regions [68,
69]. The BOLD signal time-courses in each region are expressed as
a linear combination of the BOLD signal time-courses in other
regions, and the weighting factors for the linear combination (i.e.
the “connectivity strengths”) are determined for a complete net-
work [70]. This analysis requires a predefined anatomical model of
all plausible connections between regions, which is provided by the
normalized temperature and region mask described above. Possible
connectivity relationships between the regions were identified based
on the extensive description of the regions/networks involved in
pain processing provided by Millan [71]. An example of SEM results
from Bosma and Stroman [40] is shown in Fig. 6.
Recent advances in the applications of spinal fMRI serve to fur-
ther demonstrate the reliability and sensitivity of the results, and their
potential value for future clinical applications. The first detailed rest-
ing-state study was carried out by Barry et al. [72] and used a 3D
multishot gradient-echo sequence at 7 T. They demonstrated func-
tional connectivity between right- and left-side gray matter in the spi-
nal cord, in the resting state. This is an important finding for spinal
cord fMRI in general, because of the variations in the baseline state
that may occur, even when a stimulus is not applied. Related studies
have also been carried out using spin-echo methods, by using either
thought directed at a particular region of the body [73] or images
displayed to the participant [74, 75]. In each of these studies, signal
variations were detected in the spinal cord in response to the cogni-
tive/emotional stimuli, demonstrating the influence of descending
input to the spinal cord. These findings may be related to the recent
demonstrations of effects such as placebo [23]. Detailed studies of
886 Patrick Stroman and Massimo Filippi

Fig. 6 SEM results obtained from healthy participants with a block-design stimulus at 45 °C, and a step-wise
increase in temperature to 45 °C. Arrows indicate the direction of the influence, while the line thickness indi-
cates the strength of the SEM connectivity. Solid lines represent positive path coefficients and dotted lines
represent negative coefficients. Abbreviations are as follows: Cord right dorsal region of the C8 spinal cord
segment, PBN parabrachial nucleus, LC locus coeruleus, NRM nucleus raphe magnus, NTS nucleus tractus
solitarius, NGC nucleus gigantocellularis, DRt dorsal reticular nucleus, PAG periaqueductal gray matter, HYP
hypothalamus, Thal thalamus. This figure is reproduced from Bosma and Stroman, J Magn Reson Imaging
2014 (DOI: 10.1002/jmri.24656) [41]

pain processing have also been carried out, showing effects of pain
modulation by changes in attention focus and by listening to music
[23, 24, 42]. Individual differences in pain processing have also been
investigated, and demonstrate correlations between BOLD responses
and individual pain ratings in the PAG, PBN, and spinal cord dorsal
horn [76]. This study provides evidence that spinal fMRI methods are
adequately sensitive to provide characterizations of the pain state in
individuals. Moreover, a study of the effects of spinal cord injury on
thermal sensory processing has demonstrated plasticity, with signifi-
cant differences detected in individuals compared to a group of healthy
control participants [50]. This body of recent work shows that spinal
fMRI methods and applications are rapidly developing and expand-
ing. Their future clinical potential is also further demonstrated.

6 Data Acquisition Details and Analysis Methods

The sensitivity of fMRI studies depends on how well the paradigm

design matches the neural function of interest, and excludes con-
founding effects, possibly more than it depends on the acquisition
 Functional MRI of the Spinal Cord 887

parameters discussed earlier. The spinal cord receives tonic input

from brainstem regions, even when no task is performed or stimu-
lus is applied [71]. Therefore, as has been observed already with
spinal fMRI, the “baseline” state, in the absence of a stimulus, can-
not be considered an “inactive” state [31, 76]. Other important
considerations for spinal fMRI study designs are the properties of
sensory receptors, and the expected input they provide to the spi-
nal cord, communication via interneurons within and between spi-
nal cord segments, and the descending modulation of responses
from brainstem regions [40]. Spinal cord responses cannot be
assumed to be constant across repeated applications of stimuli, or
tasks, because of the emotional/cognitive component of descend-
ing modulation [23, 24, 40–42, 74–77]. An additional conse-
quence of these properties of the spinal cord and brainstem is that
upon application of a stimulus or task, both inhibitory and facili-
tory input signaling to the regions can increase, or decrease. The
net change in metabolic demand can also therefore be an increase,
or a decrease, and negative BOLD responses are possible, and have
been observed consistently across studies [17, 31–33, 35, 37, 39–
42]. These observations across studies show that taking the periph-
eral and supraspinal input signaling into consideration in the study
design can improve the sensitivity of the results and the accuracy of
their interpretation. Moreover, these observations demonstrate the
sensitivity of the results that can currently be achieved.
The key steps in the analysis of spinal fMRI data are essentially
the same as with brain fMRI data. The preprocessing steps typically
include converting the data from DICOM to an easier-to-use for-
mat such as NIfTI, then applying motion-correction by realigning
the data to match one volume in the time series, slice-timing correc-
tion, spatial normalization, and spatial smoothing. The preprocessed
data can then be analyzed with a number of methods such as a
General Linear Model (GLM) to detect predicted BOLD responses,
region-of-interest (ROI) analyses to extract time-course responses
from specific regions, and connectivity analyses. A number of studies
have used brain fMRI analysis software such as SPM (Statistical
Parametric Mapping, Wellcome Trust Centre for Neuroimaging,
London, UK) [23, 24, 26] or AFNI [72, 78] and have adapted its
use for spinal fMRI. Specialized software has also been developed in
MatLab for analysis of spinal fMRI data acquired with spin-echo
methods [17, 40, 42]. A detailed investigation of the noise charac-
teristics and effects of preprocessing steps has shown that efforts to
model physiological noise for inclusion as regressors in GLM analy-
sis provide only a small improvement in the ability to detect BOLD
responses [17, 56, 57]. This limited effect appears to be the result of
the complexity of the physiological noise. A data-driven method of
extracting global sources of variance across the entire spinal cord has
been shown to have a significant effect on improving the sensitivity
of the results [17]. In the same study it was shown that accurate
888 Patrick Stroman and Massimo Filippi

motion-correction (i.e. realignment of the data) provides the greatest

improvement in measurement sensitivity of all of the preprocessing
steps. Temporal filtering, on the other hand, alters the t-value distri-
bution which artificially inflates the significance of the results, lead-
ing to false-­positive results [17, 79]. Ultimately, this study showed
that the best analysis approach (to date) involves only effective
motion-­correction, data-driven estimates of noise regressors in the
GLM. Moreover, the most effective way to deal with either random
or physiological noise is to collect as many volumes as possible to
describe the fMRI time-series. In effect, these results to date have
confirmed principles of fMRI study design and analysis that have
long been known for brain fMRI, but are often over-looked in the
effort to develop more “sophisticated” methods.

7 Conclusions and Future Directions

The evidence to date shows that spinal fMRI has developed rap-
idly over the past several years, and highly sensitive results have
been obtained. Details of responses to sensory and motor para-
digms, pain processing, and descending modulation related to
cognitive and emotional factors have all been demonstrated.
Detailed anatomical mapping of responses has also been demon-
strated. There is no question that spinal fMRI is feasible and effec-
tive for research applications. It has also been shown to provide
valuable information about multiple-sclerosis and traumatic spinal
cord injury. However, it still has many limitations and will require
more development before clinical applications can be considered.
While the technical challenges and limitations of spatial and tem-
poral resolution have been identified, and efforts to overcome
these challenges are proceeding, one key limitation for the prog-
ress of spinal fMRI is the lack of “critical mass” of researchers
working on it. Divisions over the best methods to use have further
limited the pace of development. Consensus over the acquisition
methods would contribute to the development of common soft-
ware methods for analysis. This would reduce the burden of time
and effort for new groups to apply spinal fMRI to new research
questions, and would allow new developments to be shared more
easily between groups. Fortunately, efforts are underway to facili-
tate sharing of ideas such as “Spinal Cord Hack 2014” organized
by Dr. Paul Summers as a satellite meeting of the International
Society for Magnetic Resonance in Medicine (ISMRM) 2014
annual meeting, and the 2015 version being organized by Dr
Julien Cohen-Adad. In addition, the International Spinal Research
Trust and the Wings for Life Foundation have organized interna-
tional imaging workshops [80, 81], and are promoting a multisite
diagnostic trial using spinal cord imaging.
 Functional MRI of the Spinal Cord 889

References
1. Figley CR, Stroman PW (2007) Investigation 13. Figley CR, Stroman PW (2012)
of human cervical and upper thoracic spinal Measurement and characterization of the
cord motion: implications for imaging spinal human spinal cord SEEP response using
cord structure and function. Magn Reson Med event-related spinal fMRI. Magn Reson
58(1):185–189 Imaging 30(4):471–484
2. Figley CR, Yau D, Stroman PW (2008) 14. Stroman PW, Krause V, Malisza KL,
Attenuation of lower-thoracic, lumbar, and Frankenstein UN, Tomanek B (2002)
sacral spinal cord motion: implications for imag- Extravascular proton-density changes as a
ing human spinal cord structure and function. non-­ BOLD component of contrast in fMRI
AJNR Am J Neuroradiol 29(8):1450–1454 of the human spinal cord. Magn Reson Med
3. Yoshizawa T, Nose T, Moore GJ, Sillerud LO 48(1):122–127
(1996) Functional magnetic resonance imag- 15. Stroman PW, Lee AS, Pitchers KK, Andrew
ing of motor activation in the human cervical RD (2008) Magnetic resonance imaging of
spinal cord. Neuroimage 4(3 Pt 1):174–182 neuronal and glial swelling as an indicator of
4. Menon RS, Ogawa S, Kim SG, Ellermann function in cerebral tissue slices. Magn Reson
JM, Merkle H, Tank DW, Ugurbil K (1992) Med 59(4):700–706
Functional brain mapping using magnetic res- 16. Stroman PW, Malisza KL, Onu M (2003)
onance imaging. Signal changes accompany- Functional magnetic resonance imaging at 0.2
ing visual stimulation. Invest Radiol 27(Suppl Tesla. Neuroimage 20(2):1210–1214
2):S47–S53 17. Bosma RL, Stroman PW (2014) Assessment of
5. Ogawa S, Tank DW, Menon R, Ellermann JM, data acquisition parameters, and analysis tech-
Kim SG, Merkle H, Ugurbil K (1992) Intrinsic niques for noise reduction in spinal cord fMRI
signal changes accompanying sensory stimula- data. Magn Reson Imaging 32(5):473–481
tion: functional brain mapping with magnetic 18. Komisaruk BR, Mosier KM, Liu WC,
resonance imaging. Proc Natl Acad Sci U S A Criminale C, Zaborszky L, Whipple B, Kalnin
89(13):5951–5955 A (2002) Functional localization of brainstem
6. Stroman PW, Nance PW, Ryner LN (1999) and cervical spinal cord nuclei in humans with
BOLD MRI of the human cervical spinal cord fMRI. AJNR Am J Neuroradiol 23(4):609–617
at 3 tesla. Magn Reson Med 42(3):571–576 19. Cohen-Adad J, Gauthier CJ, Brooks JC,
7. Madi S, Flanders AE, Vinitski S, Herbison GJ, Slessarev M, Han J, Fisher JA, Rossignol S,
Nissanov J (2001) Functional MR imaging Hoge RD (2010) BOLD signal responses to
of the human cervical spinal cord. AJNR Am controlled hypercapnia in human spinal cord.
J Neuroradiol 22(9):1768–1774 Neuroimage 50(3):1074–1084
8. Backes WH, Mess WH, Wilmink JT (2001) 20. Maieron M, Iannetti GD, Bodurka J, Tracey
Functional MR imaging of the cervical spi- I, Bandettini PA, Porro CA (2007) Functional
nal cord by use of median nerve stimulation responses in the human spinal cord dur-
and fist clenching. AJNR Am J Neuroradiol ing willed motor actions: evidence for side-
22(10):1854–1859 and rate-dependent activity. J Neurosci
9. Stroman PW, Ryner LN (2001) Functional 27(15):4182–4190
MRI of motor and sensory activation in the 21. Summers PE, Ferraro D, Duzzi D, Lui F,
human spinal cord. Magn Reson Imaging Iannetti GD, Porro CA (2010) A quantitative
19(1):27–32 comparison of BOLD fMRI responses to nox-
10. Stroman PW, Krause V, Malisza KL, ious and innocuous stimuli in the human spinal
Frankenstein UN, Tomanek B (2001) cord. Neuroimage 50(4):1408–1415
Characterization of contrast changes in func- 22. Nash P, Wiley K, Brown J, Shinaman R, Ludlow
tional MRI of the human spinal cord at 1.5 D, Sawyer AM, Glover G, Mackey S (2013)
T. Magn Reson Imaging 19(6):833–838 Functional magnetic resonance imaging identi-
11. Bandettini PA, Wong EC, Jesmanowicz A, fies somatotopic organization of nociception in
Hinks RS, Hyde JS (1994) Spin-echo and the human spinal cord. Pain 154(6):776–781
gradient-­echo EPI of human brain activation 23. Eippert F, Finsterbusch J, Bingel U, Buchel
using BOLD contrast: a comparative study at C (2009) Direct evidence for spinal cord
1.5 T. NMR Biomed 7(1–2):12–20 involvement in placebo analgesia. Science
12. Figley CR, Leitch JK, Stroman PW (2010) 326(5951):404
In contrast to BOLD: signal enhancement by 24. Sprenger C, Eippert F, Finsterbusch J, Bingel
extravascular water protons as an alternative U, Rose M, Buchel C (2012) Attention modu-
mechanism of endogenous fMRI signal change. lates spinal cord responses to pain. Curr Biol
Magn Reson Imaging 28(8):1234–1243 22(11):1019–1022
890 Patrick Stroman and Massimo Filippi

25. Geuter S, Buchel C (2013) Facilitation of pain innocuous thermal sensory stimuli and study-­
in the human spinal cord by nocebo treatment. related emotional influences. Magn Reson
J Neurosci 33(34):13784–13790 Imaging 27(10):1333–1346
26. van de Sand MF, Sprenger C, Buchel C (2015) 38. Lawrence JM, Stroman PW, Kollias SS (2008)
BOLD responses to itch in the human spinal Functional magnetic resonance imaging of the
cord. Neuroimage 108:138–143 human spinal cord during vibration stimulation
27. Figley CR, Stroman PW (2011) The role(s) of of different dermatomes. Neuroradiology
astrocytes and astrocyte activity in neurome- 50(3):273–280
tabolism, neurovascular coupling, and the pro- 39. Stroman PW, Bosma RL, Tsyben A (2012)
duction of functional neuroimaging signals. Somatotopic arrangement of thermal sensory
Eur J Neurosci 33(4):577–588 regions in the healthy human spinal cord deter-
28. Stroman PW, Krause V, Frankenstein UN, mined by means of spinal cord functional
Malisza KL, Tomanek B (2001) Spin-echo ver- MRI. Magn Reson Med 68(3):923–931
sus gradient-echo fMRI with short echo times. 40. Bosma RL, Stroman PW (2014) Spinal cord
Magn Reson Imaging 19(6):827–831 response to stepwise and block presentation of
29. Stroman PW, Tomanek B, Krause V, thermal stimuli: a functional MRI study.
Frankenstein UN, Malisza KL (2003) J Magn Reson Imaging 41(5):1318–1325
Functional magnetic resonance imaging of the 41. Stroman PW, Coe BC, Munoz DP (2011)
human brain based on signal enhancement by Influence of attention focus on neural activity
extravascular protons (SEEP fMRI). Magn in the human spinal cord during thermal sen-
Reson Med 49(3):433–439 sory stimulation. Magn Reson Imaging
30. Stroman PW, Kornelsen J, Lawrence J, Malisza 29(1):9–18
KL (2005) Functional magnetic resonance 42. Dobek CE, Beynon ME, Bosma RL, Stroman
imaging based on SEEP contrast: response PW (2014) Music modulation of pain percep-
function and anatomical specificity. Magn tion and pain-related activity in the brain,
Reson Imaging 23(8):843–850 brainstem, and spinal cord: an fMRI study.
31. Cahill CM, Stroman PW (2011) Mapping of J Pain 15(10):1057–1068
neural activity produced by thermal pain in the 43. Agosta F, Valsasina P, Absinta M, Sala S,
healthy human spinal cord and brain stem: a Caputo D, Filippi M (2009) Primary progres-
functional magnetic resonance imaging study. sive multiple sclerosis: tactile-associated func-
Magn Reson Imaging 29(3):342–352 tional MR activity in the cervical spinal cord.
32. Ghazni NF, Cahill CM, Stroman PW (2010) Radiology 253(1):209–215
Tactile sensory and pain networks in the human 44. Agosta F, Valsasina P, Caputo D, Rocca MA,
spinal cord and brain stem mapped by means of Filippi M (2009) Tactile-associated fMRI
functional MR imaging. AJNR Am recruitment of the cervical cord in healthy sub-
J Neuroradiol 31(4):661–667 jects. Hum Brain Mapp 30(1):340–345
33. Kozyrev N, Figley CR, Alexander MS, Richards 45. Agosta F, Valsasina P, Caputo D, Stroman PW,
JS, Bosma RL, Stroman PW (2012) Neural Filippi M (2008) Tactile-associated recruit-
correlates of sexual arousal in the spinal cords ment of the cervical cord is altered in patients
of able-bodied men: a spinal fMRI investiga- with multiple sclerosis. Neuroimage
tion. J Sex Marital Ther 38(5):418–435 39(4):1542–1548
34. Lawrence JM, Kornelsen J, Stroman PW 46. Agosta F, Valsasina P, Rocca MA, Caputo D,
(2011) Noninvasive observation of cervical spi- Sala S, Judica E, Stroman PW, Filippi M (2008)
nal cord activity in children by functional MRI Evidence for enhanced functional activity of
during cold thermal stimulation. Magn Reson cervical cord in relapsing multiple sclerosis.
Imaging 29(6):813–818 Magn Reson Med 59(5):1035–1042
35. Rempe T, Wolff S, Riedel C, Baron R, Stroman 47. Valsasina P, Agosta F, Absinta M, Sala S,
PW, Jansen O, Gierthmuhlen J (2014) Spinal Caputo D, Filippi M (2010) Cervical cord
fMRI reveals decreased descending inhibition functional MRI changes in relapse-onset MS
during secondary mechanical hyperalgesia. patients. J Neurol Neurosurg Psychiatry
PLoS One 9(11):e112325 81(4):405–408
36. Rempe T, Wolff S, Riedel C, Baron R, Stroman 48. Valsasina P, Agosta F, Caputo D, Stroman
PW, Jansen O, Gierthmuhlen J (2014) Spinal PW, Filippi M (2008) Spinal fMRI during
and supraspinal processing of thermal stimuli: proprioceptive and tactile tasks in healthy sub-
an fMRI study. J Magn Reson Imaging jects: activity detected using cross-correlation,
41(4):1046–1055 general linear model and independent compo-
37. Stroman PW (2009) Spinal fMRI investigation nent analysis. Neuroradiology
of human spinal cord function over a range of 50(10):895–902
 Functional MRI of the Spinal Cord 891

49. Valsasina P, Rocca MA, Absinta M, Agosta F, 62. Lang J, Bartram CT (1982) Fila radicularia of
Caputo D, Comi G, Filippi M (2012) Cervical the ventral and dorsal radices of the human spi-
cord FMRI abnormalities differ between the nal cord. Gegenbaurs Morphol Jahrb
progressive forms of multiple sclerosis. Hum 128(4):417–462
Brain Mapp 33(9):2072–2080 63. Gray H (1995) Gray’s anatomy: the anatomical
50. Cadotte DW, Bosma R, Mikulis D, Nugaeva basis of medicine and surgery. In: Williams PL,
N, Smith K, Pokrupa R, Islam O, Stroman PW, Bannister LH, Berry MM, Collins P, Dyson M,
Fehlings MG (2012) Plasticity of the injured Dussek JE, Ferguson MWJ (eds) Gray’s anatomy:
human spinal cord: insights revealed by spinal the anatomical basis of medicine and surgery.
cord functional MRI. PLoS One 7(9):e45560 Churchill-Livingstone, London, pp 975–1011
51. Stroman PW, Kornelsen J, Bergman A, Krause 64. Talairach J, Tournoux P (1988) Co-planar ste-
V, Ethans K, Malisza KL, Tomanek B (2004) rotaxic atlas of the human brain. Thieme
Noninvasive assessment of the injured human Medical Publishers Inc, New York
spinal cord by means of functional magnetic 65. Naidich TP, Duvernoy HM, Delman BN,
resonance imaging. Spinal Cord 42(2):59–66 Sorensen AG, Kollias SS, Haacke EM (2009)
52. Kornelsen J, Stroman PW (2007) Detection of Internal architecture of the brain stem with key
the neuronal activity occurring caudal to the axial sections. Duvernoy’s atlas of the human
site of spinal cord injury that is elicited during brain stem and cerebellum. Springer,
lower limb movement tasks. Spinal Cord New York, pp 79–82
45(7):485–490 66. McArdle JJ, McDonald RP (1984) Some alge-
53. Stroman PW (2011) Essentials of functional braic properties of the Reticular Action Model
MRI. Taylor & Francis Group, LLC, Boca for moment structures. Br J Math Stat Psychol
Raton, FL 37(Pt 2):234–251
54. Murphy K, Bodurka J, Bandettini PA (2007) 67. Craggs JG, Staud R, Robinson ME, Perlstein
How long to scan? The relationship between WM, Price DD (2012) Effective connectivity
fMRI temporal signal to noise ratio and neces- among brain regions associated with slow tem-
sary scan duration. Neuroimage 34(2):565–574 poral summation of C-fiber-evoked pain in
55. Finsterbusch J, Eippert F, Buchel C (2012) fibromyalgia patients and healthy controls.
Single, slice-specific z-shim gradient pulses J Pain 13(4):390–400
improve T2*-weighted imaging of the spinal 68. Buchel C, Friston K (2001) Interactions
cord. Neuroimage 59(3):2307–2315 among neuronal systems assessed with func-
56. Figley CR, Stroman PW (2009) Development tional neuroimaging. Rev Neurol 157(8–9 Pt
and validation of retrospective spinal cord 1):807–815
motion time-course estimates (RESPITE) for 69. Buchel C, Friston KJ (1997) Modulation of
spin-echo spinal fMRI: improved sensitivity connectivity in visual pathways by attention:
and specificity by means of a motion-­ cortical interactions evaluated with structural
compensating general linear model analysis. equation modelling and fMRI. Cereb Cortex
Neuroimage 44(2):421–427 7(8):768–778
57. Brooks JC, Beckmann CF, Miller KL, Wise 70. Bollen KA (1989) Structural equations with
RG, Porro CA, Tracey I, Jenkinson M (2008) latent variables. Wiley, New York
Physiological noise modelling for spinal func- 71. Millan MJ (2002) Descending control of pain.
tional magnetic resonance imaging studies. Prog Neurobiol 66(6):355–474
Neuroimage 39(2):680–692 72. Barry RL, Smith SA, Dula AN, Gore JC (2014)
58. Verma T, Cohen-Adad J (2014) Effect of respi- Resting state functional connectivity in the
ration on the B0 field in the human spinal cord human spinal cord. eLife 3:e02812
at 3T. Magn Reson Med 72(6):1629–1636 73. Kashkouli Nejad K, Sugiura M, Thyreau B,
59. Myronenko A, Song XB (2009) Image regis- Nozawa T, Kotozaki Y, Furusawa Y, Nishino
tration by minimization of residual complexity. K, Nukiwa T, Kawashima R (2014) Spinal
Proc Cvpr IEEE. pp 49–56 fMRI of interoceptive attention/awareness in
60. Myronenko A, Song XB (2010) Intensity-­ experts and novices. Neural Plast 2014:679509
based image registration by minimizing resid- 74. Smith SD, Kornelsen J (2011) Emotion-­
ual complexity. IEEE Trans Med Imag dependent responses in spinal cord neurons: a
29(11):1882–1891 spinal fMRI study. Neuroimage 58(1):269–274
61. Stroman PW, Figley CR, Cahill CM (2008) 75. Kornelsen J, Smith SD, McIver TA (2014) A
Spatial normalization, bulk motion correction neural correlate of visceral emotional responses:
and coregistration for functional magnetic res- evidence from fMRI of the thoracic spinal cord.
onance imaging of the human cervical spinal Soc Cogn Affect Neurosci 10(4):584–588
cord and brainstem. Magn Reson Imaging 76. Khan HS, Bosma RL, Beynon M, Dobek C,
26(6):809–814 McIver T, Stroman PW (2013) Pain processing
892 Patrick Stroman and Massimo Filippi

networks in the brain and spinal cord mapped 80. Stroman PW, Wheeler-Kingshott C, Bacon M,
using Functional Magnetic Resonance Schwab JM, Bosma R, Brooks J, Cadotte D,
Imaging. Program number II6 66.01. 2013 Carlstedt T, Ciccarelli O, Cohen-Adad J, Curt
Meeing Planner San Diego, CA; Society for A, Evangelou N, Fehlings MG, Filippi M, Kelley
Neuroscience BJ, Kollias S, Mackay A, Porro CA, Smith S,
77. McIver TA, Kornelsen J, Smith SD (2013) Strittmatter SM, Summers P, Tracey I (2014)
Limb-specific emotional modulation of cervical The current state-of-the-art of spinal cord imag-
spinal cord neurons. Cogn Affect Behav ing: methods. Neuroimage 84:1070–1081
Neurosci 13(3):464–472 81. Wheeler-Kingshott CA, Stroman PW, Schwab
78. Cox RW (1996) AFNI: software for analysis JM, Bacon M, Bosma R, Brooks J, Cadotte
and visualization of functional magnetic reso- DW, Carlstedt T, Ciccarelli O, Cohen-Adad J,
nance neuroimages. Comput Biomed Res Curt A, Evangelou N, Fehlings MG, Filippi M,
29(3):162–173 Kelley BJ, Kollias S, Mackay A, Porro CA,
79. Friston KJ, Josephs O, Zarahn E, Holmes AP, Smith S, Strittmatter SM, Summers P,
Rouquette S, Poline J (2000) To smooth or not Thompson AJ, Tracey I (2014) The current
to smooth? Bias and efficiency in fMRI time- state-of-the-art of spinal cord imaging: applica-
series analysis. Neuroimage 12(2):196–208 tions. Neuroimage 84:1082–1093
 Chapter 30

Clinical Applications of the Functional Connectome

Massimo Filippi and Maria A. Rocca

Abstract
Network-based analysis of brain functional connections has provided a novel instrument to study the
human brain in healthy and diseased individuals. Graph theory provides a powerful tool to describe quan-
titatively the topological organization of brain connectivity. Using such a framework, the brain can be
depicted as a set of nodes connected by edges. Distinct modifications of brain network topology have been
identified during development and normal aging, whereas disrupted functional connectivity has been asso-
ciated to several neurological and psychiatric conditions, including multiple sclerosis, dementia, amyo-
trophic lateral sclerosis, and schizophrenia. Such an assessment has contributed to explain part of the
clinical manifestations usually observed in these patients, including disability and cognitive impairment.
Future network-based research might reveal different stages of the different diseases, subtypes for cogni-
tive impairments, and connectivity profiles associated with different clinical outcomes.

Key words Brain networks, Structural connectivity, Functional connectivity, Graph theory, Multiple
sclerosis, Dementias, Psychiatric conditions

1 Introduction

Brain function depends on both local information processing

combined with effective global communication and integration of
information within a network of neural interactions. Brain func-
tion is not only based on the properties of single regions, but
rather emerges from interactions of the network as a whole. Brain
areas are interconnected by large-scale bundles of axonal projec-
tions, forming a macroscopic network of white matter pathways
that enable functional communication between distinct, anatomi-
cally separated brain regions. Recent advances in magnetic reso-
nance imaging (MRI) allow the reconstruction of both the
structural and functional connections of this large-scale neural sys-
tem, thus enabling efficient mapping of connectivity across the
entire brain [1]. Network-based analysis of brain structural and
functional connections has provided a novel instrument to study
the human brain in healthy and diseased individuals [2]. Using the
theoretical framework of networks and graphs, the brain can be

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods vol. 119,
DOI 10.1007/978-1-4939-5611-1_30, © Springer Science+Business Media New York 2016

893
894 Massimo Filippi and Maria A. Rocca

represented as a set of nodes (i.e., brain regions) joined by pairs by

lines (i.e., structural or functional connectivity). Graph analysis
has revealed important features of brain organization, such as an
efficient “small-world” architecture (which combines a high level
of segregation with a high level of global efficiency) and distrib-
uted, highly connected network regions, called “hubs.” In a small-
world network, a high clustering coefficient indicates that nodes
tend to form dense regional cliques, implying high efficiency
in local information transfer/processing. Path length and global
efficiency are measures of network integration, which is the ability
to combine specialized information rapidly from distributed brain
regions. Distinct modifications of brain network topology have
been identified during development and normal aging, whereas
disrupted functional network properties have been associated with
several neurological and psychiatric conditions, including neuro-
degenerative diseases, multiple sclerosis (MS), and schizophrenia.
The methodological aspects related to graph analysis (the key
mathematical framework for much of this research) have been
described in details in another chapter of this book (Chapter 10).
This chapter provides a summary of modifications of brain network
topology associated with normal development and aging, and of
how they are perturbed in course of brain diseases.

2 Normal Development

Using graph-theoretical methods, the maturation of the control

network and default-mode network (DMN) have been explored by
analyzing resting state (RS) fMRI data of healthy subjects from 7 to
31 years old [3, 4]. In children, the control network was more inte-
grated and comprised a single system, contrasting with the adult
configuration, which was characterized by a dual-network structure
including cingulo-opercular and fronto-parietal networks. These
data indicated a less between-network segregation and greater
within-network connectivity in children. The DMN, which con-
tains a set of regions usually deactivated during goal-oriented tasks,
was only sparsely connected in children while a cohesive integrated
network emerged in adults. Developmental changes in DMN
regions were characterized by increases in correlation strengths and
occurred in an anterior-posterior orientation. Conversely, the devel-
opmental pattern of the control network involved both increases
and decreases of correlation strengths. Maturation of whole-brain
functional networks from 8 to 25 years old has also been investi-
gated [5]. Such a study showed that although the modular struc-
ture was well built since 8 years old, the modules changed
dramatically from anatomical proximity to a more “distributed”
architecture, grouping regions mainly by their functional roles.
Measures of small-world structure (i.e., clustering coefficient and
 Clinical Applications of the Functional Connectome 895

path length) were preserved through development, indicating that

functional networks in children were efficient for both global and
local information transfer as those in adults. Another RS fMRI
study [6] replicated the previous results and also described a signifi-
cantly decreased subcortical-cortical and increased cortico-cortical
connectivity in the developing brain (Fig. 1). Both these studies
observed the phenomenon of increased long-range connections
and decreased short-range connections, which provides crucial sup-
port for segregation and integration processes at a system level over
brain development. Another study showed that the hub locations
and the core hub–hub network structure was kept consistent from
10 to 20 years of age while the main changes happened to the con-
nectivity linking hubs and nonhub regions [7].
RS functional connectivity (FC) patterns extracted by support
vector machine-based multivariate pattern analysis can make accu-
rate predictions about individuals’ brain maturity across develop-
ment [8]. Gender effects on whole-brain functional networks have
also been explored in healthy children from 6 to 18 years old [9].
Compared to girls, boys had higher global efficiency and shorter
path length, with regional differences mainly located in the DMN,
language, and visual areas. This finding is consistent with the
notion that cognitive and emotional development differs between
girls and boys, particularly in visuospatial, language, and emotion
processing areas of the brain. A study in 12-year-old monozygotic
and dizygotic twins showed that the global network efficiency was
under genetic control [10]. The functional brain network in
infancy was also studied. Two-week-old pediatric subjects only
have primitive and incomplete DMN, whereas, after a marked
increase of connectivity, the DMN in 2 years of age became similar
to that observed in adults. Whole-brain functional networks in the
infant brain already exhibited functional hubs mainly located in
primary sensory and motor areas, which was distinct from the hub
distribution of the adults in the heteromodal association cortex
[11]. Another RS fMRI study showed that the functional brain
networks have the small-world topology immediately after birth,
followed by a remarkable improvement in the network efficiency
and resilience until 2 years of age [12].

3 Normal Aging

The analysis of RS fMRI data has demonstrated a reduced cost-

efficiency in older people with aging with detrimental effects mainly
located to frontal and temporal cortical and subcortical regions
[13]. Another study [14] found the RS FC of both DMN and
dorsal attention network decreased with aging, with long-range
connections showing higher vulnerability to aging effects than
short-range connections. Other investigations confirmed a decrease
896 Massimo Filippi and Maria A. Rocca

Fig. 1 Developmental changes in interregional functional connectivity. (a) Children had significantly greater
subcortical-primary sensory, subcortical-association, subcortical-paralimbic, and lower paralimbic-
association, paralimbic-limbic, association-limbic connectivity than young-adults (p < 0.01, indicated by **).
(b) Graphical representation of developmental changes in functional connectivity along the posterior-anterior
and ventral-dorsal axes, highlighting higher subcortical connectivity (subcortical nodes are shown in green)
and lower paralimbic connectivity (paralimbic nodes are shown in gold) in children, compared to young-adults.
Brain regions are plotted using the y and z coordinates of their centroids (in mm) in the MNI space. 430 pairs
of anatomical regions showed significantly higher correlations in children and 321 pairs showed significantly
higher correlations in young-adults (p < 0.005, FDR corrected). For illustration purposes, the plot shows dif-
ferential connectivity that were most significant, 105 pairs higher in children (indicated in red) and 53 higher
in young-adults (indicated in blue), (p < 0.0001, FDR corrected). From [6] with permission
 Clinical Applications of the Functional Connectome 897

in DMN connectivity (e.g., precuneus and posterior cingulate

regions) with aging. Using supporting vector machine analysis
[15], modifications of connectivity in the sensorimotor and
cingulo-opercular networks were identified as distinguishing char-
acteristics of age-related reorganization (Fig. 2).

4 Multiple Sclerosis

Consistent with the known multifocal distribution of structural

damage to the central nervous system, MS patients experience a
distributed pattern of RS FC abnormalities, which are related to
the extent of structural damage and the severity of clinical disability
and cognitive impairment [16]. Only a few studies have applied
graph analysis methods in these patients.
Several authors have used graph-theoretical analysis of RS fMRI
data to improve the understanding of the mechanisms responsible
for the presence of cognitive deficits and clinical disability in patients
with MS [17–19]. A study from 246 MS patients and 55 matched
healthy controls [18] found that global network properties (includ-
ing network degree, global efficiency, and path length) were abnor-
mal in MS patients compared to controls and contributed to
distinguish cognitively impaired MS patients from controls, but not
the main MS clinical phenotypes. Compared to controls, MS
patients also showed a loss of hubs in the superior frontal gyrus,
precuneus, and anterior cingulum in the left hemisphere; a different
lateralization of basal ganglia hubs (mostly located in the left hemi-
sphere in controls, and in the right hemisphere in MS patients); and
the formation of hubs, not seen in controls, in the left temporal
pole and cerebellar lobule IV-V. Such a modification of regional
network properties was found to contribute to cognitive impair-
ment (Fig. 3) and phenotypic variability of MS.
A RS fMRI study of 16 early MS patients detected increased
network modularity (i.e., diminished functional integration
between separate functional modules) in patients compared to
healthy controls [19]. Such modularity abnormalities in patients
correlated with worse performance at a dual task. Another study
explored the effects of gender on the correlation between network
functional abnormalities and cognitive function [17]. Compared
to male controls, male MS patients had reduced network efficiency,
but normal clustering coefficient. No abnormalities were found in
female patients. Decreased network efficiency in male patients was
correlated with reduced visuospatial memory.
Using a pattern recognition technique, MS patients could be
discriminated from healthy controls based on RS FC with a sensi-
tivity of 82 % and specificity of 86 %. The most discriminative con-
nectivity changes were found in subcortical and temporal regions,
and contralateral connections were more discriminative than ipsi-
lateral ones [20].
Fig. 2 Aging effect on resting-state functional connectivity. Illustration of the consensus features that
decreased positive correlation with age (top) and the consensus features that increased positive correlation
with age and decreased negative correlation with age (bottom). Connections are scaled by their respective
feature weight, with thicker connections representing greater feature weight. From [15] with permission
 Clinical Applications of the Functional Connectome 899

Fig. 3 Functional hub distribution in healthy controls and multiple sclerosis patients. Brain hubs (a left hemi-
sphere, b right hemisphere) of the functional networks of healthy controls (HCs) and patients with MS as a
whole and according to the presence/absence of cognitive impairment. Hubs were identified as brain regions
having either integrated nodal degree or betweenness centrality one standard deviation greater than the net-
work average. Hubs present in HC only are reported in red, hubs present in MS patients only are reported in
blue, and hubs present in all groups are reported in black. CP cognitively preserved, CI cognitively impaired,
ACC anterior cingulate cortex, Caud caudate nucleus, Cereb cerebellum, ITG inferior temporal gyrus, Ling lin-
gual gyrus, MCC middle cingulate cortex, MTG middle temporal gyrus, OFC orbitofrontal cortex, Pall pallidus,
Precun precuneus, Put putamen, SFG superior frontal gyrus, Sup TP superior temporal pole, Thal thalamus.
From [18] with permission

5 Neurodegnerative Diseases

5.1 Alzheimer’s Many studies used graph theoretical analysis in patients with the
Disease and Other most prevalent type of dementia, Alzheimer’s disease (AD). A cor-
Dementias relation between the site of amyloid-β deposition in AD patients
and the location of major hubs as defined by graph theoretical
analysis of RS FC in healthy adults has been demonstrated [21].
These regions include the posterior cingulate cortex/precuneus,
the inferior parietal lobule, and the medial frontal cortex, implying
that the hubs are preferentially affected in the progression of
AD. There is also convergent evidence from methodologically
disparate MRI studies that AD is associated with perturbations of
brain small-world network organization [22–24].
Although studies showed considerable variability in reported
group differences of most graph properties, the average
900 Massimo Filippi and Maria A. Rocca

characteristic path length has been most consistently reported to be

increased in AD, as a result of loss of connectivity, while the cluster-
ing coefficient is likely to be less affected by AD pathology [25].
An fMRI graph analysis study in mild AD [24] suggested that
loss of small-world network properties might provide a clinically
useful diagnostic marker, since clustering was reduced at a global
level and at a local level (in both hippocampi), and global cluster-
ing was able to discriminate AD patients from healthy elderly sub-
jects with relatively high sensitivity (72 %) and specificity (78 %).
Another fMRI study [23] showed that the characteristic path
length of AD brains is closer to the theoretical values of random
networks compared with controls. Decreased RS FC in the parietal
and occipital regions, and increased connectivity in frontal cortices
and corpus striatum were also found [23] (Fig. 4). Decreased
global efficiency and increased local efficiency were also found in
moderate AD cases, in whom the altered brain regions were mainly
located in the DMN, temporal lobe, and subcortical regions [26].
Graph theoretical analysis was recently applied to RS fMRI
data from patients with the behavioral variant of frontotemporal
dementia (bvFTD) [27]. Global and local functional networks
were altered in bvFTD patients, indicated by reduced mean net-
work degree, clustering coefficient, and global efficiency and
increased clustering coefficient relative to normal subjects. Altered
brain regions were located in structures that are closely associated
with neuropathological changes in bvFTD, such as the frontotem-
poral lobes and subcortical regions.

5.2 Amyotrophic Consistent motor and extra-motor brain pathology supports the
Lateral Sclerosis notion of amyotrophic lateral sclerosis (ALS) as a system failure.
Thus, a simplistic (motor-based) approach to ALS is no longer ten-
able. Overall functional organization of the motor network was
unchanged in patients with ALS compared to healthy controls;
however, patients with a stronger and more interconnected motor
network had a more progressive disease course [28].

6 Psychiatric Conditions

Evidence is accumulating that neural network changes are underly-

ing structural and functional brain changes in psychiatric diseases,
and may provide a more sensitive measure to detect brain abnor-
malities than measurements of properties of separate brain areas
alone [29, 30]. The potential of network approaches to psychiatry
can be illustrated by findings in schizophrenia, a severe psychiatric
disease which has for long been hypothesized to reflect a discon-
nection syndrome. Connectomic studies have shown a reduced
whole-brain functional connectivity in patients with schizophrenia
[31, 32]. Functional networks in schizophrenia patients were
 Clinical Applications of the Functional Connectome 901

Fig. 4 Resting-state synchronization in healthy controls and Alzheimer’s disease patients. (a) Matrix of signifi-
cant differences of synchronization between Alzheimer’s disease (AD) and controls (2-tail t-test, p < 0.05
uncorrected). The white and black dots represent brain areas pairs with increased and decreased synchroniza-
tion in AD respectively. (b-d) A subset of connectional differences corresponding to the matrix (a) are plotted
at three superior-to-inferior levels through the anatomical automatic labeled brain template. Lines depict
synchronization between pairs of regions: solid lines = enhanced synchronization; dashed lines = reduced
synchronization. Note the pattern of generalized posterior (parietal and occipital) synchronization reductions
and increased frontal synchronization. From [23] with permission

characterized by reduced clustering and modularity and increased

global efficiency compared with healthy controls, suggesting
increased global integration and decreased local segregation [31,
33, 34]. By assessing dynamic graph properties of time-varying
functional brain connectivity in RS fMRI data, a recent study
902 Massimo Filippi and Maria A. Rocca

demonstrated a decreased variance in the dynamic topological

parameters in patients with schizophrenia compared to healthy
controls, which might contribute to explain the abnormal brain
performance in this mental illness [35]. The disconnectivity
hypothesis in schizophrenia is also supported by studies which
showed an abnormal hub organization. A less hub-dominated con-
figuration has been observed in functional and structural schizo-
phrenia connectomes [31, 36] Specifically, schizophrenia patients
exhibit reduced regional centrality in hubs including the frontal
association, parietal, limbic, and paralimbic brain areas based on
structural studies, and including the frontal, temporal, parietal,
limbic, and occipital areas based on functional studies.

7 Conclusions

The extensive application during the past few years of graph theoreti-
cal approaches to define brain network topology in healthy and dis-
eased people has undoubtedly provided a novel instrument to
characterize functional abnormalities associated with different neuro-
logical and psychiatric conditions and to test hypothesized discon-
nectivity effects in these diseases. Disrupted functional brain
connectivity is present in the major neurological and psychiatric con-
ditions discussed in this chapter and their assessment has contributed
to explain part of the clinical manifestations of these patients.
However, there are also inconsistencies between existing studies,
which might be attributable to the clinical heterogeneity of the
patient groups as well as to differences in imaging modality and ana-
lytic methods. Future network-based research might reveal different
stages of the different diseases, subtypes for cognitive impairments,
and connectivity profiles associated with different clinical outcomes.

References
1. Bullmore E, Sporns O (2009) Complex brain “local to distributed” organization. PLoS
networks: graph theoretical analysis of struc- Comput Biol 5:e1000381
tural and functional systems. Nat Rev Neurosci 6. Supekar K, Musen M, Menon V (2009)
10:186–198 Development of large-scale functional brain
2. Filippi M, van den Heuvel MP, Fornito A et al networks in children. PLoS Biol 7:e1000157
(2013) Assessing brain system dysfunction 7. Hwang K, Hallquist MN, Luna B (2013) The
through MRI-based connectomics. Lancet development of hub architecture in the human
Neurol 12:1189–1199 functional brain network. Cereb Cortex
3. Fair DA, Dosenbach NU, Church JA et al 23:2380–2393
(2007) Development of distinct control net- 8. Dosenbach NU, Nardos B, Cohen AL et al
works through segregation and integration. (2010) Prediction of individual brain maturity
Proc Natl Acad Sci U S A 104:13507–13512 using fMRI. Science 329:1358–1361
4. Fair DA, Cohen AL, Dosenbach NU et al 9. Wu K, Taki Y, Sato K et al (2013) Topological
(2008) The maturing architecture of the organization of functional brain networks in
brain’s default network. Proc Natl Acad Sci U healthy children: differences in relation to age,
S A 105:4028–4032 sex, and intelligence. PLoS One 8:e55347
5. Fair DA, Cohen AL, Power JD et al (2009) 10. van den Heuvel MP, van Soelen IL, Stam CJ,
Functional brain networks develop from a Kahn RS, Boomsma DI, Hulshoff Pol HE
 Clinical Applications of the Functional Connectome 903

(2013) Genetic control of functional brain net- analysis of FMRI resting-state functional con-
work efficiency in children. Eur nectivity. PLoS One 5:e13788
Neuropsychopharmacol 23:19–23 24. Supekar K, Menon V, Rubin D, Musen M,
11. Fransson P, Aden U, Blennow M, Lagercrantz Greicius MD (2008) Network analysis of intrin-
H (2011) The functional architecture of the sic functional brain connectivity in Alzheimer’s
infant brain as revealed by resting-state disease. PLoS Comput Biol 4:e1000100
fMRI. Cereb Cortex 21:145–154 25. Tijms BM, Wink AM, de Haan W et al (2013)
12. Gao W, Gilmore JH, Giovanello KS et al Alzheimer’s disease: connecting findings from
(2011) Temporal and spatial evolution of brain graph theoretical studies of brain networks.
network topology during the first two years of Neurobiol Aging 34:2023–2036
life. PLoS One 6:e25278 26. Zhao X, Liu Y, Wang X et al (2012) Disrupted
13. Achard S, Bullmore E (2007) Efficiency and small-world brain networks in moderate
cost of economical brain functional networks. Alzheimer’s disease: a resting-state FMRI
PLoS Comput Biol 3:e17 study. PLoS One 7:e33540
14. Tomasi D, Volkow ND (2012) Aging and 27. Agosta F, Sala S, Valsasina P et al (2013) Brain
functional brain networks. Mol Psychiatry network connectivity assessed using graph the-
17(471):549–458 ory in frontotemporal dementia. Neurology
15. Meier TB, Desphande AS, Vergun S et al (2012) 9:134–143
Support vector machine classification and charac- 28. Verstraete E, van den Heuvel MP, Veldink JH
terization of age-related reorganization of func- et al (2010) Motor network degeneration in amy-
tional brain networks. Neuroimage 60:601–613 otrophic lateral sclerosis: a structural and func-
16. Rocca M, Valsasina P, Martinelli V et al (2012) tional connectivity study. PLoS One 5:e13664
Large-scle neuronal network dysfunction in 29. Bullmore E, Sporns O (2012) The economy of
relapsing-remitting multiple sclerosis. brain network organization. Nat Rev Neurosci
Neurology 79:1449–1457 13:336–349
17. Schoonheim MM, Hulst HE, Landi D et al 30. van den Heuvel MP, Mandl RC, Stam CJ,
(2012) Gender-related differences in func- Kahn RS, Hulshoff Pol HE (2010) Aberrant
tional connectivity in multiple sclerosis. Mult frontal and temporal complex network struc-
Scler 18:164–173 ture in schizophrenia: a graph theoretical anal-
18. Rocca MA, Valsasina P, Meani A, Falini A, ysis. J Neurosci 30:15915–15926
Comi G, Filippi M (2016) Impaired functional 31. Lynall ME, Bassett DS, Kerwin R et al (2010)
integration in multiple sclerosis: a graph theory Functional connectivity and brain networks in
study. Brain Struct Funct 221:115–131 schizophrenia. J Neurosci 30:9477–9487
19. Gamboa OL, Tagliazucchi E, von Wegner F 32. Bassett DS, Nelson BG, Mueller BA,
et al (2014) Working memory performance of Camchong J, Lim KO (2012) Altered resting
early MS patients correlates inversely with mod- state complexity in schizophrenia. Neuroimage
ularity increases in resting state functional con- 59:2196–2207
nectivity networks. Neuroimage 94:385–395 33. Lo CY, Su TW, Huang CC et al (2015)
20. Richiardi J, Gschwind M, Simioni S et al Randomization and resilience of brain func-
(2012) Classifying minimally disabled multiple tional networks as systems-level endopheno-
sclerosis patients from resting state functional types of schizophrenia. Proc Natl Acad Sci U S
connectivity. Neuroimage 62:2021–2033 A 112:9123–9128
21. Buckner RL, Sepulcre J, Talukdar T et al 34. Alexander-Bloch AF, Gogtay N, Meunier D et al
(2009) Cortical hubs revealed by intrinsic (2010) Disrupted modularity and local connec-
functional connectivity: mapping, assessment tivity of brain functional networks in childhood-
of stability, and relation to Alzheimer’s disease. onset schizophrenia. Front Syst Neurosci 4:147
J Neurosci 29:1860–1873 35. Yu Q, Erhardt EB, Sui J et al (2015) Assessing
22. He Y, Chen Z, Evans A (2008) Structural dynamic brain graphs of time-varying connec-
insights into aberrant topological patterns of tivity in fMRI data: application to healthy con-
large-scale cortical networks in Alzheimer’s trols and patients with schizophrenia.
disease. J Neurosci 28:4756–4766 Neuroimage 107:345–355
23. Sanz-Arigita EJ, Schoonheim MM, 36. Rubinov M, Bullmore E (2013) Schizophrenia
Damoiseaux JS et al (2010) Loss of ‘small- and abnormal brain network hubs. Dialogues
world’ networks in Alzheimer’s disease: graph Clin Neurosci 15:339–349
 INDEX

A B
Acceleration factor............................50, 51, 84–89, 465, 476, Bayesian inference ............................................ 242, 244–247
477, 479, 877, 881, 882 Behaviourally relevant coding ...................................601–602
Activation maps ................................139, 143, 326, 430, 534, Between-group comparison ...................................... 142, 144
651, 748, 805, 808, 856 Biomarkers .......................................345–348, 666, 699–702,
Active noise cancellation (ANC) ........................ 55, 575, 579 711, 717, 721–722, 726–727, 823, 827, 837
Active tasks............................................... 138, 140, 524, 645 Biophysical models ....................209, 211, 241, 242, 250–252
Adjacency matrix ..............................................................297 Bipolar disorder (BD)...............................659, 660, 671, 672,
Alzheimer’s disease (AD) .......................... 121, 266, 271, 437, 674–677, 687, 688, 862
700–705, 707, 709–711, 713–718, 721–727, 818, 819, Block design ............................................................... 70, 758
823, 824, 862, 864, 899–901 Blood oxygen level dependent (BOLD)
Amnesic syndrome ................................................... 742, 753 effect ............................................................. 22, 114, 146
Amyotrophic lateral sclerosis (ALS)........................ 700, 709, initial dip .................................25, 26, 128, 130, 326, 642
712, 716, 717, 719, 720, 900 response ....................................25, 26, 63, 130, 146–147,
ANALYZE .............................................................. 156, 157 323, 460, 469, 470, 483
Anatomical connectivity ........................... 243, 293, 545, 546 Brain activation ........................ 25, 74, 76, 98, 121, 125, 126,
Anterior temporal lobe .................................... 363, 365, 437, 129, 133, 138–140, 142, 331, 332, 342, 347, 368, 375,
687, 741, 858 394, 440, 457–459, 482, 483, 486, 501, 503, 511, 512,
resection (ATLR) ...............................741, 743, 744, 750, 524, 547, 574, 626, 647, 651, 667, 676, 703–705, 709
751, 753, 754, 756, 757, 761 Brain atlases
Anxiety disorders ...............................658, 659, 661, 677–686 deformation atlases .............................................268–270
Arterial spin labeling (ASL) .............................41, 44, 64, 88, density-based ..............................................................268
122, 334, 335, 347, 515, 657–659, 724, 782, 820, 828 disease-specific ............................266, 268, 270–271, 785
Ascending sensory pathways .................................... 389, 390 genetic atlases .............................................................271
Attention label-based ..........................................................268–269
nonselective ................................................................389 Montreal Neurological Institute (MNI) ............ 157, 273,
preparatory .........................................................400–404 289, 463, 719
reflexive.......................................................................388 Talairach & Tournoux (T&T) ...................................... 57
selective...............................................................389–397 Brain-behavior relationships............................ 317, 318, 327,
spatial..................................................343, 388, 392–400, 335, 340, 348, 728
403, 406–411, 548 Brain imaging data
spatial selective ................................... 392, 393, 400, 402 functional and structural MRI .................... 861–862, 864
visual ...................................................336, 387–413, 595, joint ICA (jICA) ................................................ 856, 858
704, 707, 710, 716 multimodal CCA ................................................859–861
voluntary ............................................. 388, 407, 411–413 multimodal MRI data.........................................854–857
voluntary visual ................................... 388, 407, 411–413 multivariate approaches ......................................856–857
Attentional control mechanisms physiological or behavioral features ....................853–854
EEG-fMRI ........................................................407–412 Partial Least Squares (PLS)........................................860
top-down control ........................................................400 Brain networks
Attention control network ........................................ 397, 403 graph projections ........................................................298
Auditory system........................................ 360, 390, 573–602 topological properties .........................................300–308
Automated anatomical labelling (AAL) ...........................290 Brain plasticity.................................... vii, 609, 612, 616, 626,
Automated normalization process ....................................884 836, 838, 839, 842, 843
Average artefact subtraction (AAS).................. 765, 768, 770 Brain systems ............................. 253, 278, 344, 355, 366, 396

Massimo Filippi (ed.), fMRI Techniques and Protocols, Neuromethods, vol. 119,
DOI 10.1007/978-1-4939-5611-1, © Springer Science+Business Media New York 2016

905
 FMRI TECHNIQUES AND PROTOCOLS
906 Index

Button response box ...........................................................34 D

Button response unit (BRU).........................................53, 55
See also Button response box Default mode network (DMN) ....................... 141, 291, 407,
410, 411, 413, 470, 616, 632, 709–713, 718, 719, 721,
C 724, 774, 781, 823, 862, 894, 895, 897, 900
Deformation models......................................... 170–175, 277
Cardiac monitoring ......................................................56–57
Demyelination .................................................. 126, 613, 628
Cerebral blood flow (CBF) ............. 25, 26, 115, 122, 128, 252,
Deoxyhemoglobin ................................25, 26, 116, 250, 251,
333, 334, 468, 514, 515, 640, 658, 659, 678, 686, 782
326, 640, 658, 819
Cerebral blood volume (CBV) ............................. 25, 26, 115
Depression ................. 452, 630, 659, 667–674, 678, 823, 824
Cerebrovascular reactivity ......................................... 641, 642
Descending modulation............................508, 510, 511, 871,
Classical inference ............................................ 242, 244–246
872, 878, 887, 888
Clinical endpoints ....................................................817–818
Diaschisis..........................................................................319
Clinical high risk (CHR) ................................................. 666
Diffeomorphism ............................................... 171–175, 270
Cluster-wise inference ...............223–225, 230, 231, 233, 236
Diffusion anisotropy .........................................................294
CNS vasculitides ..............................................................629
Digital imaging and communication in medicine
Cochlear partition ............................................................590
(DICOM)............................................. 155–158, 887
Cognitive conjunctions ............................................. 745, 746
Dorsal anterior cingulate cortex (dACC) ......... 411, 412, 668
Cognitive deficits...............................318, 319, 617, 662, 897
Drug development ..................... 727, 817, 818, 821, 823, 828
Cognitive subtraction ............................... 327–329, 745, 746
patient stratification............................................823–825
design .........................................................................745
pharmacodynamics ..................................... 818, 825–827
Comparison task....................................................... 138, 745
target validation ..................................................821–823
Compensatory hyperactivation ......................... 705, 714–717
Dry magnet ........................................................................38
Compensatory responses ...................525, 714, 715, 750, 835
3D shape perceptions ...............................................565–566
Computerized tomography (CT) ............... 37, 122, 159, 318
Dynamic causal modelling (DCM) ................. 241–263, 296,
Conceptual combination ..................................................364
337, 406–408, 412, 457, 484, 537, 651, 706, 856, 857
Conjunction analysis ........................................ 141, 144, 530
Dynamic models....................................... 247–250, 252, 253
Connectivity ..................................... 89, 121, 169, 242–244,
252–259, 261, 272, 278, 283–308, 323, 337, 338, E
346, 408, 412, 426, 427, 452, 457, 470, 484, 486,
506, 507, 510, 525, 528, 530, 537, 546, 618–621, Echo-planar imaging (EPI) ....................20–22, 26–28, 79–81,
624, 627, 628, 632, 651–652, 677, 703, 705–707, 83, 86–89, 91–94, 96, 99, 101, 102, 105, 116, 119, 120,
709–713, 717–721, 723–726, 728, 741, 747, 754, 123–124, 127, 128, 131–133, 158, 159, 212, 464–467,
756, 774, 781, 811, 822–825, 836, 843, 854, 856, 470, 473–480, 484, 515, 586, 658, 744, 764, 768–770,
858, 860–862, 864, 884–887, 893–898, 900–902 776, 786, 803, 819, 875–877, 880–883
analysis........................................292, 298–300, 426–427, Echo-time (TE) .............................. 23–24, 26–28, 71–74, 78,
506, 510, 618, 781 80, 83, 84, 87, 93, 97, 99, 101–103, 105, 106, 118,
effective............................... 242, 252–254, 258, 285, 293, 119, 122, 123, 127, 129, 325, 463, 465, 466, 471,
296, 297, 337, 338, 426, 457, 484, 619, 706, 741, 884 473–479, 563–565, 803, 810, 874–877, 879–882
functional ............................ 89, 121, 242, 252–254, 285, Edges
288, 291, 294–297, 299–301, 307, 308, 337, 338, effective connectivity ..........................252–254, 258, 285,
346, 412, 426, 452, 457, 470, 486, 506, 507, 525, 296, 297, 337, 338, 619, 706
530, 537, 618, 620, 621, 624, 627, 628, 632, 707, functional connectivity .......................... 89, 121, 254, 285,
709–714, 718–720, 723–726, 741, 774, 781, 288, 291, 294–295, 299–301, 337, 338, 426, 457, 507,
822–825, 858, 862, 885, 894–896, 898, 900–902 618, 620, 621, 628, 826
Connectomes .................................................... 283, 288, 902 structural connectivity.........................121, 285, 292–295,
Connector hubs ................................................................306 297, 487, 713, 854
Contrast-to-noise ratio (CNR) .................. 99, 116, 476, 874 EEG-fMRI ............................ 60, 62, 65, 407–412, 762–764,
Cortical structure ...................................... 129, 421, 502, 511 768, 770–785
cortical columns .......................................... 120, 129, 130 continuous .......................................... 768, 770, 772, 776
layers ........................................................... 129–130, 260 Effective transverse relaxation time (T2*) ...........................23
Cortical thickness mapping ..............................................271 Electrical stimulation........................242, 243, 497, 502, 505,
Cryo-coolers .......................................................................39 526, 801–803, 811
Cytoarchitecture ...............................275, 276, 289, 290, 292, Electrocortical stimulation (ECS) ............................748–750
506, 507, 554, 557 Electrodermal activity (EDA) ..........................................457
 FMRI TECHNIQUES AND PROTOCOLS
Index
907
Electroencephalography (EEG) ...................... 6, 59–61, 242, 122, 124, 129, 131, 133, 144, 325, 463, 466, 467,
259–262, 278, 288, 319, 322, 323, 408, 409, 412, 496, 470–471, 473, 474, 477, 484, 505, 514, 578, 763, 766,
513, 643, 753, 761–785, 820, 844, 855, 857, 860, 863 785, 802, 876
Electromagnetic noise sources ............................................33 Fixed-effects model .................................................. 217, 218
Electro-oculograms (EOG).............................................. 393 Flip angle............................... 42, 74, 102, 106, 133, 476, 860
Eloquent cortex ........................................ 741, 801, 802, 809 Focal epilepsy.............................744–747, 761–762, 766, 771
Emotion Fractional anisotropy (FA)................................ 294, 855, 862
discrete (primary)........................................................ 454 Frequency representations ........................................593–597
measurement................................453, 454, 457, 483, 484 Frontotemporal dementia (FTD) .............700, 701, 709, 712,
processing...............452, 462, 463, 480, 484, 486, 504, 507, 719–720, 900
508, 512, 616, 665, 666, 669, 675–677, 687, 895 F-statistics ......................... 198, 200–201, 212, 223, 228, 245
secondary ....................................................................454 Functional integration ............................. 242–244, 252–261,
Emotional activation .................451, 453–455, 458, 459, 482 288, 303, 306, 317, 335, 337–339, 754, 898
Epilepsy syndromes ..................................................780–781 Functional localization ............................................. 243, 273
Epileptogenic network......................................................761 Functional magnetic resonance adaptation
Episodic memory (EM) ...........................427–436, 616, 664, (fMR-A) ..................................................................326
665, 689, 713, 756 Functional mapping.................................. 121, 130, 752, 809
encoding ............................................................. 430, 436 Functional near-infrared spectroscopy (fNIRS)............60–61
retrieval ............................................... 428, 430–432, 436 brain stimulation............................................... 30, 62–65
Erdõs-Rényi model .............................................. 5, 286, 287 Functional neuronavigation ..............................................802
Event-related fMRI..............................70, 98–101, 138, 149, Functional performance ....................................................126
150, 249, 252, 329, 397, 401, 422, 423, 425, 426, 429, Functional segregation ...................................... 243–244, 257
431, 439, 501, 528, 714, 756 Functional specialization ..........................121, 242–244, 252,
Event-related potentials (ERP) ................242, 259, 261, 262, 266, 303, 306, 316, 335–337, 529, 546
322, 340, 344, 348, 391–394, 397, 405, 408, 483, 756 Functional variation ..........................................................273
Experimental design ...........70, 76, 98–101, 106, 107, 137–152,
163, 202, 241, 254, 279, 324, 327–330, 335, 348, 412, G
426, 595, 640, 644–647, 745, 755–757, 828, 860 Gadolinium ........................................................ 75, 611, 612
analytic issues......................................................146–151 Galvanic skin response (GSR) ................................ 55, 57–59
block .................70, 98–101, 145, 327, 589, 745, 755, 756 Generalised epilepsy ..........................774, 778, 781–782, 785
event-related ....................................70, 98, 101, 138, 329 Generalized autocalibrating partially parallel acquisitions
participant-response related ........................ 138, 140, 141 (GRAPPA) ..................................49, 84–88, 465, 877
practical issues ....................................................143–146 General linear model (GLM) ............................. 98, 145, 147,
Experimental medicine ............................................. 818, 823 148, 151, 184, 189–206, 210–214, 217, 219, 221–224,
Explanatory variables (EV) ......................191–194, 196, 197, 229, 235, 242, 244–249, 251, 252, 256, 295, 550, 641,
200–208, 212–214, 221, 222, 245, 252, 253 645, 771, 783, 784, 854, 887, 888
Eye tracking .................................................................55, 59 Generative model ............................................. 162, 169, 177
Genetic determinants .......................................................701
F
Geodesic approaches ........................................................268
Factorial design ........................................ 457, 680, 745, 746 Gliomas ............................................................................812
False discovery rate (FDR) .......................226–228, 234, 299, Gradient coils .................................35–36, 45–47, 83, 91, 92,
560, 896 130–132, 470–472, 475, 477, 478, 574, 578, 580
Family-wise error (FWE) ................................. 226–234, 484 Gradient-echo (GE) methods ..................55, 56, 58, 79, 102,
Fast Fourier transform (FFT)................................. 77, 82, 83 119, 123, 128, 325, 326, 465, 473–477, 479, 480,
Field gradients .................................. 7–11, 18, 22, 23, 25, 27, 874–877, 880–883, 885
36, 42, 45–47, 69, 71, 73, 77, 132, 463, 464, 466, 468, Gradient recalled echo (GRE) ........................ 72–74, 77, 78,
469, 472–475, 479 80, 81, 101, 102, 105, 133, 464, 875
Field homogeneity ......................... 25, 42, 44, 87, 91, 93, 127 Graph theory
Field inhomogeneity........................... 40, 41, 71, 79, 89, 116, basic processing pipeline, MRI data ...........................285
123, 126, 158, 213, 325, 465, 470, 509–510, 515, 819 brain graph ..........................284–286, 289, 290, 293, 295
Field of view (FOV) .........................42, 60, 83–89, 102–105, brain networks ............................................ 121, 283–308
167, 467, 474, 673, 763, 764, 810, 876, 880 complex systems ................................................. 288, 525
Field strength ......................... 5, 7–10, 23, 33, 35, 40, 41, 43, connectivity matrix ..................................... 285, 296–299
47, 70, 71, 87, 97, 101–105, 107, 114, 117–119, 121, connectomes ............................................... 283, 288, 902
 FMRI TECHNIQUES AND PROTOCOLS
908 Index

Graph theory (cont.) orthographic decision tasks.........................................368

information-processing and integrating diverse network phonetic decision tasks ...............................................366
elements ................................................................288 phonological decision tasks................................. 368, 373
matrix and graph-based representations, brain retrieval, selection and maintenance ...................365–366
networks ...............................................................284 semantic decision tasks ............................... 368, 372–374
MRI...................................................... 30, 242, 248, 249 sensory discrimination tasks ....................... 368, 372, 374
network growth models ..............................................308 word generation tasks ................................. 362, 368, 372
network topology ................................ 300, 307, 594, 902 Larmor frequency ............................4–12, 18, 19, 22, 71, 115
probabilistic random graphs .......................................286 Long-term memory................... 320, 420, 422, 427, 429, 742
Gray matter (GM)....................................23, 24, 74, 75, 115, Lower motor neuron (LMN) ...........................................709
119, 124, 168, 193, 246, 285, 293, 464, 573, 620, 632, Low-level motion regions.........................................559–562
709, 714, 720, 854, 855, 858, 872, 875, 886
Group analysis ........................... 184, 185, 210, 219, 221, 234 M
Magnetic field gradients ........ 38, 42, 45–47, 69, 71, 462–464
H
Magnetic resonance imaging (MRI). See also Graph theory
Haemodynamic response function (HRF) ................ 26, 146, diffusion tensor (DT) .................................................713
149, 150, 188–190, 206–211, 247, 248, 250–252, magnetization transfer (MT) ...................................... 620
333, 334, 481, 496, 497, 499, 641, 642, 644, 767, 773, Magnetic susceptibility .......................41, 114, 115, 117, 118,
776–778, 784 122, 124–126, 128, 158, 462, 764, 872
Haemoglobin ..............25, 76, 97–98, 114, 640, 642, 657, 658 Magnetoencephalography (MEG) ...................... 6, 288, 319,
Head motion ................................ 37, 55, 59, 70, 95–97, 162, 405, 496, 643, 761, 802
212–215, 467–469, 702, 765, 767, 769, 770, 776, 777 Major depressive disorder .........................660, 667, 669–673,
Hierarchical observation model ................................246–247 676, 677, 687–689
High-level motion regions................................................562 Malformations of cortical development (MCD) ............. 747,
Huntington disease (HD)............ 707, 713, 716, 717, 720–721 780, 782, 783
Matched-trial analysis ......................................................142
I Mean diffusivity (MD) .....................................................294
Medical image registration toolbox (MIRT) .................... 884
Image matrix .................................................... 103–105, 467
MEG-MRI ......................................................................762
Image registration.............................155, 159–161, 164, 165,
Memory encoding .....................705, 710, 719, 723, 753–760
167, 170, 175, 268, 884
Metabolic rate of oxygen consumption (CMR2) ...............25,
Independent component analysis (ICA).................. 212, 213,
26, 115, 333
254, 291, 464, 469, 709, 712, 713
Migraine ...................................................................629–632
Inference ........................... 144, 177, 210, 211, 216, 218–220,
Mild cognitive impairment (MCI) .................. 271, 701, 704,
222–237, 242, 244–247, 252–254, 256, 258, 263,
705, 707, 708, 711–716, 718, 721–723, 725, 862–864
265–279, 305, 318–321, 329, 421, 537, 723, 755, 771
Mirror-neuron system (MNS).......................... 533–536, 622
Initial dip .......................................25, 26, 128, 130, 326, 642
Mixed-effects model.................................................217–220
Interictal activity ............................................... 771, 776, 785
Modularity................................................ 304–306, 897, 901
Interictal epileptiform discharges (IED) ................. 766, 767,
Mood disorders ........................................ 660, 665, 667–677
771–785
Motion artifacts ........... 96, 160, 162, 163, 468–469, 483, 872
Intermodule connectivity ..................................................306
Motor deficits ............................833–834, 837, 838, 840, 843
Intracarotid amytal test (IAT) ......................... 742, 746–751,
Motor evoked potentials (MEP) ........................................63
753, 757, 761, 741742
Motor homunculus ................................................... 805, 806
Intra-modular connectivity ...............................................306
Motor imagery .......................... 525, 532, 533, 716, 842, 843
Isolated spinal cord injury.................................................628
Motor impairment............................................ 629, 639, 651
K Motor skill learning ..................................................532–533
Motor system after stroke
k-space...................................... 13–17, 20–22, 27, 36, 77–88, post-stroke motor networks ................................651–652
91–93, 116, 119, 123, 124, 133, 158, 186, 466, 467, residual functional architecture ...........................647–651
473, 474, 476, 477, 480, 484, 580 Multichannel receive coils ............................................50–51
Multiple linear regression ......................... 147, 836, 856, 857
L
Multiple receive coils ..........................................................83
Language lateralization .............743, 745–749, 751, 753, 803 Multiple sclerosis (MS)
Language mapping paradigms benign ................................................................. 614, 618
naming tasks ...............................................................368 CIS suggestive of ........................................ 613–616, 620
 FMRI TECHNIQUES AND PROTOCOLS
Index
909
primary progressive (PP) ............................................614 Parallel imaging techniques .................................. 49, 79, 103
relapsing-remitting (RR) .................................... 615, 839 Parametric manipulation ........... 139, 141, 142, 565, 596, 659
secondary progressive (SP) .........................................614 Parkinson’s disease (PD) .................... 322, 700, 706, 708, 824
structural damage........................................................897 Paroxysmal activity ...................................................761–786
Multi-subject statistics..............................................216–222 Partial least squares (PLS) ........................................ 856, 860
Multivoxel pattern analysis (MVPA)................ 547, 549, 550 Passive tasks.............................................................. 524, 645
Myeloarchitecture .............................................................292 Penetration panel .................................................... 34, 35, 64
Perceptually relevant coding .....................................598–601
N Performance confounds ............................................644–645
Navigation microscope .....................................................804 Perfusion-based fMRI ......................................................122
n-back test ........................................................................617 Peripheral nerve stimulation (PNS)............................ 46, 123
Negative bold response (NBR) ................. 130, 326, 780, 887 Permutation testing .......................................... 231, 233, 237
Neuroanatomy ..................................139, 141, 358–366, 390, Pharmacological MRI ..............................................817–828
421, 452, 500–510, 678, 683, 873, 875 Pharmacological studies ...........................................344–345
Neurodegenerative diseases .............. 117–118, 699–728, 894 Phased-array coils .........................................................47–50
resting state fMRI ..............................................709–714 Phase encoding ............................. 16, 77, 80, 84, 86, 88, 119,
Neuronal activity .......................... 59, 63, 114–116, 120, 122, 123, 158, 188, 477, 551
127, 128, 130, 183, 250, 252, 255, 258, 296, 410, 550, Phase encoding gradient ............................... 16, 88, 119, 477
616, 641, 658, 762, 778, 782, 785, 819, 827, 856, 872, Phoneme perception ..................356–360, 362, 369, 371, 372
874, 877 Phonological representations ............................ 359, 361, 362
Newman-Girvan Q-statistic .............................................304 Phonology ........................................................ 356, 361, 363
Nociceptive processing ......................495–499, 502, 510, 513 Physiological monitoring.............................. 55, 56, 481, 803
Nociceptors............................................... 498, 500, 503, 510 Positron emission tomography (PET) ..................... 122, 164,
Nodes 273, 321, 391, 421
anatomical atlases ............................... 271, 290–291, 884 Posterior probability maps (PPM) ................... 244–248, 616,
brain network organization .................... 8, 283, 287–289, 620, 621
295, 306, 346 Posttraumatic stress disorder (PTSD) ..................... 468, 659,
cytoarchitectonic regions ............................................289 661, 679, 682–686
data-driven parcellation ......................................291–292 Predictive biomarkers ...............................................721–722
extrinsic heterogeneity ........................................ 289, 290 Pre-surgical assessment ............................ 741, 742, 776, 778
functional parcellation ........................................ 285, 291 Principal component analysis (PCA)................. 60, 209, 253,
intrinsic homogeneity ................................. 289, 290, 292 254, 711, 768–770, 856, 857, 864
quantitative parcellations ............................................292 Principal diffusion direction (PDD) ......................... 712–713
random parcellation ............................................ 285, 291 Projection systems ..............................................................37
spatial embedding ............................................... 289, 290 Proprioceptive stimulation........................................ 616, 807
voxel-based parcellation ..............................................290 Proton magnetic resonance spectroscopy
Nonfluent aphasia.............................................................318 (1H-MRS) .................................................... 810, 813
Normal-appearing brain tissue (NABT) .......................... 620 Provincial hubs ......................................................... 302, 306
Psychiatric disorders ........................................ 657–690, 716,
O 717, 726, 727, 821
Psychophysiological interaction (PPI) .............................. 295
Obsessive-compulsive disorder (OCD)..............659, 661, 667–685
P-values ............................ 195–197, 200, 203, 223, 224, 226,
Orthography .............. 356, 357, 360–361, 363, 366–368, 373
228–234, 237, 244–246
Oxidative metabolism.......................................................128

P Q
Quantitative relaxometry ..............................................70–74
Paced Auditory Serial Addition Test (PASAT) ........ 616, 617
Quenching ..........................................................................39
Paced Visual Serial Addition Test (PVSAT) ....................616
Pain
R
assessment ..........................................................498–500
chronic ..........496–498, 506, 508, 510–512, 514, 515, 826 Radiofrequency (RF)
empathy ...................................................... 512–513, 824 coils................................... 47, 48, 54, 103, 104, 115–117,
modulation .................. 507–508, 510, 511, 515, 630, 886 126, 130, 131, 133
pathways ............................................................. 510, 631 pulse...................................................................... 19, 820
sharp pricking .............................................................500 Random field theory (RFT)..........221, 224, 227–231, 244–246
slow burning ...............................................................500 Random noise................................................... 167, 212, 872
thresholds ........................................... 497, 498, 501, 509 Real time fMRI (rt-fMRI) ............................... 613, 614, 837
 FMRI TECHNIQUES AND PROTOCOLS
910 Index

Receiver bandwidth (RBW) ............................ 102, 103, 107, Sequences

127, 131, 132, 474 conventional ........................................... 77, 79, 107, 133
Regional borders ...........................................................74, 76 pulse..................45, 50, 55, 57, 64, 69–108, 114, 581, 584
Registration algorithms ............................ 161, 164, 171, 268 single shot ....................................................... 20, 79, 101
Regression ........... 138, 147, 191, 192, 194–198, 201–208, 210, Shape sensitive regions .............................................563–565
215, 218–220, 245, 249, 255, 258, 458, 836, 856, 864 Shielding ....................31–34, 47, 50, 53–55, 60, 64, 132, 809
Regressor(s) ........................138, 140, 141, 145, 147–150, 191, Signal loss .............................. 12, 18, 22, 27, 40, 50, 51, 83,
192, 202, 212–215, 219–222, 245, 248, 249, 295, 296, 87, 89, 93–94, 123, 127, 132, 158, 459, 463–466,
468, 481, 499, 887, 888 471–478, 484, 515, 576, 754, 785, 786
Rehabilitation, viii ............. 346, 628, 640, 724, 725, 833–844 Signal-to-noise ratio (SNR) .................... 40, 41, 47–49, 76,
Repetition time (TR) ............................. 71, 72, 90, 106, 123, 79, 83, 85–88, 94, 100–107, 114, 117, 118, 122,
188, 325, 468, 480, 768, 877, 880 124–130, 133, 146, 325, 332, 333, 335, 347, 470,
Respiratory monitoring ......................................................57 471, 473, 474, 477, 478, 514, 578, 581, 582, 584,
Resting state ...............129, 138, 140, 141, 169, 295, 357, 375 586, 593, 643, 785, 786, 880–882
Resting state network (RSN)........................... 464, 469–470, Single-channel coils......................................................47, 48
610, 624, 781, 782 Slice thickness .................................. 19, 20, 89, 93, 102, 107,
Restorative therapy ........................................... 834–836, 842 127, 133, 472, 805, 810, 880
Retinotopic organization ...................545, 546, 550–559, 566 Smoothing .................................. 51, 156, 175–177, 187, 194,
Retrograde trans-synaptic degeneration ...........................319 224, 228, 229, 234, 236, 237, 471, 478, 480–481, 887
Reverse inference ..............................................................320 sMRI. See Structural MRI (sMRI)
Rich-club organization .....................................................301 SNR. See Signal-to-noise ratio (SNR)
Sound presentation system .......................................574–576
S Spatial normalisation .................163–175, 273, 276, 884, 887
Spatial resolution ........................ vii, 3, 4, 6, 12, 14, 16–19, 45,
Sample size ........................ 143, 144, 273, 277, 675, 749, 758 59, 61, 87, 89, 102, 104, 105, 107, 119, 121, 129–132,
SAR. See Specific absorption rate (SAR) 146, 273, 277, 318, 325–327, 340–342, 344, 348, 483,
Scanner synchronization.....................................................55 496, 514, 546, 548, 549, 591, 593, 658
Scan plane .......................................................... 93, 102, 103 Specific absorption rate (SAR) ................................... 49, 131
Schizophrenia ...................... viii, 121, 269, 270, 279, 363, 468, Speech articulation ............................356, 366, 368, 371, 372
659, 660, 662–667, 673, 677, 687–689, 716, 821, 822, Spin ....................... 3, 9, 12, 18, 22, 23, 25, 41, 44, 70, 76, 96,
824, 858, 861–863, 894, 900, 902 106, 116, 122, 123, 127, 131, 132, 162, 465, 468, 724
SCI. See Spinal cord injury (SCI) Spinal cord injury (SCI) ...........................498, 528, 532, 628,
SE. See Spin echo (SE) 833, 840–844, 878–880, 886, 888
SEM. See Structural equation modelling (SEM) Spinal fMRI ..................................... 498, 871–880, 882–888
Semantic decision task............... 368, 372, 373, 743, 749, 758 anatomy ..............................................................872–874
Semantic memory BOLD theory .............................................................874
category specificity..............................................437–440 data acquisition and analysis methods ................886–888
modality specificity .....................................................441 gradient-echo methods ............................... 874, 877–880
taxonomic categories...................................................437 spin-echo methods .............................................875–877
Semantic processing .........................341, 357, 359, 362–364, Spin echo (SE) ........................... 72–73, 78–81, 93, 101, 102,
368, 371–373, 375, 438, 441, 664, 747 119, 122, 127, 128, 325, 612, 874–875, 877–885, 887
Semantics .................................. 340–342, 356–359, 362–375, Spiral imaging .......................70, 79, 81–83, 93–95, 102–103
389, 420, 422, 427, 429, 431, 436–442, 564, 663–666, SPM. See Statistical parametric mapping (SPM)
689, 703, 704, 706, 711, 714, 715, 743, 747, 749, Statistical parametric mapping (SPM) .................... 155, 157,
755–756, 758 161, 171, 244–248, 256, 273, 277, 347, 560, 561, 612,
SENSE. See Sensitivity encoding (SENSE) 615, 625, 887
Sensitivity encoding (SENSE) ........ 49, 84–87, 123, 132, 876 Statistical techniques
Sensorimotor paradigms multivariate .................................................................337
force .................................................................... 524, 525 univariate ....................................................................337
hemispheric dominance ..............................................525 Statistical thresholding ............................................. 760, 881
movement complexity .................................................525 Stereotaxy
movement rate ............................................................524 frame-based ................................................................802
Sensorimotor system ................................................523–537 frameless .....................................................................802
Sentence processing .................................................. 358, 364 Stimulus presentation hardware..........................................53
 FMRI TECHNIQUES AND PROTOCOLS
Index
911
Stroke ........................ viii, 333–335, 614, 639–644, 646–652, Transcranial magnetic stimulation (TMS) ............ 59, 62–65,
706, 747, 819, 833–839, 841, 843 323, 342–343, 348, 405, 484, 512, 523, 528, 529, 643,
Stroop effect .....................................................................357 648, 650, 722, 835, 836, 840, 844
Structural equation modelling (SEM) .......................... 255, Transverse relaxation .................................10, 22, 26, 40, 880
256, 258–259, 337–339, 457, 484, 534, 856, 857, Transverse relaxation time (T2)..................... 22–23, 465, 880
864, 884–886 Traumatic brain injury .............................................. 716, 833
Structural MRI (sMRI) ............................126, 440, 611, 621, T-statistics ................................ 144, 195–198, 200, 214, 222,
669, 704, 727, 747, 761, 766, 818, 821, 824, 854, 855, 223, 231, 234, 236, 480, 643
857, 858, 860–864 Tumor location .........................................................803–805
Structural variation ........................................... 266, 269, 270 Type I error....................................................... 330–332, 760
Subjects at ultra-high risk (UHR) ....................................666 Type II error ........................................99, 100, 330–332, 760
Summary statistics approach ............................ 217, 219, 220
Surrogate markers..................................................... 345, 727 U
Susceptibility artefacts ........................41, 107, 117, 118, 124, UMN. See Upper motor neuron (UMN)
335, 437, 471, 477, 481, 515, 754, 785 Upper motor neuron (UMN) ...........................................709
Susceptibility factors ................................................. 701, 717
Syntax .......................................................356, 358, 364–365, V
367, 373
VBM. See Voxel-based morphometry (VBM)
T Ventral attention network .........................................404–407
Verbal fluency task .................................... 743, 744, 749–751
T1...............................40–42, 70–72, 74, 75, 95, 97, 106, 116, Verbal memory .........................................346, 742, 744, 747,
119, 122, 163, 167, 169, 175, 176, 273, 277, 285, 292, 753–754, 756, 760
476, 526, 534, 554, 587, 615, 620, 625, 626, 650, 658, Verbal working memory ...........................139, 141, 142, 359,
767, 880 362, 368, 666
T2...................................... 22–23, 40, 70–76, 81, 97, 98, 102, Visual analog scale (VAS).................................................499
104, 105, 116, 119, 127, 128, 159, 199, 292, 534, 554, Visual system ....................................130, 339, 372, 392, 410,
619, 620, 624, 810, 877, 879–882 422, 535, 545–566, 611–613, 782
T2* .............. 36, 40–42, 71, 74–76, 81, 97, 104, 105, 640–642, Visual tasks
657–658, 877, 880 discrimination tasks ............................................ 372, 547
Tactile stimulation ............................................ 512, 616, 803 passive viewing ................................................... 336, 373
Task complexity ............................................... 527, 528, 618, Voxel-based morphometry (VBM)................... 231, 276, 861
646, 840 Voxel-wise inference ......................................... 223–225, 230
Task frequency .................................................. 151, 645–646
Task monitoring .......................................................646–647 W
TBM. See Tensor-based morphometry (TBM)
Wada test .................................................. 369, 748, 808, 812
Temporal autocorrelation ................................. 192–194, 231
Warping............................................................45, 51, 87, 93,
Temporal filtering..................................................... 145, 888
102, 107, 155, 156, 159, 163, 164, 166–175,
Temporal lobe epilepsy (TLE) ................ 741–743, 746–752,
267, 269, 273, 276
754, 756–761, 773, 777, 779–783
Wernicke–Lichtheim model of language..........................355
Tensor-based morphometry (TBM)................................. 271
White matter, viii ........................... 22, 74, 75, 119, 129, 271,
Theory of mind (ToM) .................................................... 667
272, 283, 291, 293, 294, 319, 360, 480, 481, 573, 609,
Tissue classification algorithms ........................ 267, 269, 271
610, 751, 810, 839, 844, 854, 893
Tissue segmentation ................................................. 168, 169
White matter disorders (WMD)..............................609–633
3 T MRI scanners...............................................................51
Working memory (WM) network ............ 119, 139, 141, 142,
TMS. See Transcranial magnetic stimulation (TMS)
319–321, 324, 328, 338, 339, 345, 347, 359, 361–363,
Tonotopy ..................................................................591–593
365, 368, 370–373, 375, 401, 420–427, 442, 505, 610,
Topological analysis
616–618, 624, 626, 662, 663, 665, 666, 669, 671, 672,
cost and efficiency...............................................303–304
675, 677, 681, 688, 707, 708, 720, 722, 723, 726, 742,
hub dominance ........................................... 287, 300–301
756, 820, 823
robustness to damage .................................. 301–303, 305
Written word recognition .................................................360
small-worldness ..........................................................303
Tractography seeds ....................285, 293, 294, 620, 751–752 Z
Transcranial direct current stimulation
(TDCS) ...................................................... 30, 64–65 Z-statistics ................. 200–201, 216, 223, 224, 229, 230, 235