Computational Methods in

Neuroimaging
Krishna Prasad Miyapuram
Cognitive Science & Computer Science
Indian Institute of Technology Gandhinagar

[email protected]

Journey of a thousand miles begins with a single step

M.Tech.

M.Sc.

1998

2000
Electronics

2002

2004

Artificial Intelligence

2008

2011

2012

Predictive coding

Neuroeconomics

Cognitive Neuroscience

Outline
Imaging the Human Brain
Physics of Functional MRI
Classical analysis: Statistical Parametric
Mapping
Data Visualization
Beyond Blobs: Functional Connectivity
Machine Learning Methods
Data Mining Techniques

The BIG Question

What is the nature of
human MIND?
Psychology

Artificial
Intelligence

Philosophy

Anthropology

Cognitive
Science
Linguistics

Education

Neuroscience

The small Answer

Study the human
BRAIN!

Neuroimaging Techniques

The Human Brain

Superior
(Dorsal)
Parietal Lobe
Frontal Lobe

Basal ganglia

Anterior

Posterior
Occipital Lobe

Temporal Lobe
Cerebellum

Inferior
(Ventral)

Trees do not have an organ called brain.

Parts of the Brain

Frontal Lobe

Parietal Lobe

Occipital Lobe
Temporal Lobe

Terminology Planes and Sections

Coronal

Axial
Coronal Plane

Saggital

Axial /
Horizontal
Plane

Saggital Plane

3D imaging

Sectional view of an MRI Scanner

Scanner room

Static magnetic field

Gradient coil
RF (Head) coil

Patient Table

The Nobel Prize in Physiology or Medicine for

2003 jointly to

Console room

Paul C. Lauterbur

Sir Peter Mansfield

"for their discoveries concerning

magnetic resonance imaging
http://www.nobel.se/medicine/laureates/2003/press.html

Physics of MRI
(A)

(B) Rest (Normal blood flow)

Brain activity

Oxygen consumption
Cerebral blood flow

Oxyhaemoglobin
Deoxyhaemoglobin

MRI signal intensity

Activation (High blood flow)

(C)

Experimental Design
Task condition

Baseline

Input
Input

Process
Output
Output

A B A B

B A

A A B

Data Analysis
Statistical Analysis
fMRI
time series

General Linear Model

Design matrix
Parameter estimation

Within-subject registration
slice-timing correction
Realignment
Coregistration
(structural to functional)

Statistical Inference
Linear Contrasts
Thresholding
Random Effects Analysis
(Group analysis only)

Between-subject registration
spatial normalization
Spatial smoothing

Preprocessing

Statistical
Parametric Map

Softwares for fMRI Analysis

Statistical Parametric Mapping

SPM is a form of data
reduction, condensing
information (in a
statistically meaningful
way) from a number
of individual scans into
a single image volume
that can be more
easily viewed and
interpreted.
SPM has an extensive web site at:
http://www.fil.ion.ucl.ac.uk/spm

Image Processing

Within-subject registration
slice-timing correction
Realignment
Coregistration
(structural to functional)

Between-subject registration
spatial normalization
Spatial smoothing

Need for motion correction

People move, even if they dont realize!
Same location in the brain

Same location in the grid

Rigid body movement: 3 translation parameters

3 rotation parameters

(A)

(B)

(C)

(D)

z
roll
pitch

yaw

Statistical Analysis
General Linear Model
Design matrix
Parameter estimation

Statistical Inference
Linear Contrasts
Thresholding
Random Effects Analysis
(Group analysis only)

Statistics: How?
fMRI model setup
A General Linear Model
(GLM) is setup modelling
the control and test
conditions as effects of
interest.

y = Xb + e

The GLM is used to specify

the conditions in the form
of a design matrix, which
defines the experimental
design and the nature of
hypothesis testing to be
implemented.

Specifying Contrasts
A contrast can be used to compare different
conditions in the study.
The conditions that are of interest are given a
positive value, such as 1, and conditions that
are subtracted from the conditions of interest
are given a negative value, such as -1.

Thresholding:
During the assessment of
Results, height and extent thresholds are
applied to determine significant activations.

Visualization
Glass Brain for Active-Rest

Brain Slice picture for Active-Rest

Rendering onto Subjects Anatomical Brain

A High resolution
anatomical image
(dimensions: 128x128x160
, resolution 1.95 x 1.95 x 1
mm) is acquired.
This image is Segmented
into Grey, White and CSF
images.
The subjects brain is
extracted from the Grey
and White matter images.
The activations can now
be rendered onto 3D
anatomical image of the
subject

Psychophysiological Interactions
Slides from Roland Benoit, MfD 2007/8
Data from
C. Buchel and K. Friston. Modulation of connectivity in
visual pathways by attention: Cortical interactions
evaluated with structural equation modelling and
fMRI, Cerebral Cortex, 7: 768-778, 1997

Figures from
K.J. Friston, C. Buchel, G.R. Fink, J. Morris, E. Rolls, and
R. Dolan. Psychophysiological and modulatory
interactions in Neuroimaging. NeuroImage, 6:218-229,
1997
Christian Ruffs ppt Experimental Design

Tutorial: http://www.fil.ion.ucl.ac.uk/spm/data/

Functional Connectivity
Functional Segregation

Functional Integration

Functional Connectivity

Effective Connectivity

An Example
Attention

V1
V5

Two Interpretations
Context-sensitive connectivity

Modulation of stimulus-specific responses

Set

source

stimuli

source
target

target

How it works: Interactions

V1 X Attention

How it is done: PPI & SPM5

Estimate GLM
Extract time series at Region of Interest

How it is done: PPI & SPM5

3. Deconvolve, Calculate Interaction, Reconvolve

How it is done: PPI & SPM5

3. Estimate new GLM

How it works: GLM

0

V1

Att V1XAtt

z = -9 mm

Multi Voxel Pattern Analysis

Problem Statement
Over the past decade functional Magnetic Resonance Imaging (fMRI) has
emerged as a powerful technique to locate activity of human brain while
engaged in a particular task or cognitive state.
We consider the inverse problem of detecting the cognitive state of a
human subject based on the fMRI data.

f : fMRI-sequence(t1,tN) CognitiveState

f
fMRI SCAN 1 fMRI SCAN 2

fMRI SCAN N

fMRI time series (N >= 1)

Dominant Cognitive State

Definition and Motivation

What is Cognitive State?
It is the state of a process/operation within the human
brain that affects its mental contents.

Motivation :
Such functions could provide the basis for a new approach
to study human reasoning processes.
Also deeper understanding of the functioning of human
brain could help us build more advanced AI systems.

Visuo Motor Sequence Learning

Visuo-Motor Mapping:
Association of various visual
instructions to appropriate
actions.
-Stopping at red traffic signal
-Driving slowly at speed
breaker
Sequence Learning:
Learning a task that requires
sequencing a number of
actions to achieve a goal.
-Driving a car
-Lacing a shoe

S
e
q
u
e
n
c
e

L
e
a
r
n
i
n
g

Visual
Instruction 1

Motor Action 1

Visual
Instruction 2

Motor Action 2

Visual
Instruction 3

Motor Action 3

Visual
Instruction N

Motor Action N

Visuo-Motor Mappings

Visuo-Motor Tasks
Visual Display

Keypad Response

Visual Display

Keypad Response

2
P
o
s
i
t
i
o
n
S
e
q
u
e
n
c
e

2
1
2
1

1
2

P
o
s
i
t
i
o
n
S
e
q
u
e
n
c
e

1
2

2
1

1
2

1
1
Position-to-Position Mapping

Position-to-Color Mapping

Classification Problem
P2P Vs P2C: Detect the
following cognitive states
subject is paying attention
only towards the position
of the visual stimuli
subject is paying attention
towards the position and
color of the visual stimuli
Early Vs Late Learning: Detect
the following cognitive
states
subject has learnt the V-M
sequence
subject is in the early
process of learning the VM sequence

P2C

Visual
Stimuli

P2P

Visual
Stimuli

Late
Learning

Arbitrary
Mapping
(Position-to-Color)

Response

Trial & Error

Early
Learning
Response

Machine Learning Approach

To estimate the function f: fMRI-sequence(t1, tN) ->
CognitiveState we have explored the following machine
learning techniques:
Gaussian Nave Bayes (GNB) Classifier
k-Nearest Neighbor (kNN) Algorithm
Support Vector Machines (SVM)

Single-Subject Classifier: Classifiers that are trained and

tested with a single subjects fMRI data.
Multiple-Subject Classifier: Classifiers that are trained with
fMRI data of multiple subjects and tested with data of a
new subject,

Major Challenges
1. Very high dimensional data
(184707 voxels/features).
2. Variation in shapes and
sizes of brain across human
subjects.

3. Variation in the level of

fMRI activity across
subjects.

Feature Selection
Select the n most discriminating voxels (Discrim) :
Voxels are selected based on their ability to distinguish
one target class (Cognitive States) from the other.
Select the n most active voxels (Active) : Voxels are
selected based on their ability to distinguish either
target class (Cognitive States) from the baseline
condition.
Select the n feature pairs whose correlation
discriminates the target classes (CorrPair) : Voxel pairs
are selected based on the ability of their correlation to
discriminate the target classes.
We observed Poor performance of Discrim and Active
features and relatively better performance of CorrPair
features for multiple-subject classifiers.

P2P Vs P2C Classification Study

Single Subject

Classification Accuracy (%)

100
96

Discrim

92

Active
Feature
Selection

88
50

100 200

Multiple Subject

GNB

50

100 200

KNN (k=5)

50

100 200

KNN (k=9)

50

100 200
SVM

No. of features

CorrPair Feature Selection

80

No. of
features
200
474

60
40
20
t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72
GNB

KNN (k=5)

KNN (k=9)

SVM

Time Interval

Early Vs Late Learning Study

Classification Accuracy (%)
100

Single Subject

97

Feature
Selection

85
70

Discrim
Active

55
40
50

100
GNB

200

50

100

200

KNN (k=5)

50

100

200

KNN (k=9)

50

100

200

SVM

Multiple Subject

No. of features
CorrPair Feature Selection

80

No. of
features

60

200

40

445

20
t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72
GNB

KNN (k=5)

KNN (k=9)

SVM

Time Interval

Interim Conclusions
The problem of detection of cognitive states in such a high
dimensional feature space is feasible when right choice of
features is made along with suitable methods for
representation of data.
Overall much better performance of single-subject
classifiers over the multiple-subject classifiers.
We were unsuccessful in learning a classifier function for
Four-Way Classification Study", the question that we can
detect all the cognitive states is yet to be answered.
P2C

P2C Early

P2C Late

P2P

P2P Early

P2P Late

Early

Late

Imagery Conditioning
Imagery: Mental States like those that arise
during perception but occur in the absence of
immediate sensory input.
What occurs in your mind when you see the
following word

Imagery Conditioning
Were your mental
contents like this

Has shape (round)

Has colour (brown)
Is type (cake)

OR
Pylyshyn

OR

Neither of the above

Umm! Its yummy!

Kosslyn

Experimental Paradigm

2 sec
1 sec
+

3 sec

What did you See?

Poisson ITI
(mean 4 sec)

Nothing

Perception
Reward

No Reward

Imagination

Scrambled
Picture

Money
Bill

Reward Predicting Responses

Activation in Midbrain is greater
for stimuli predicting reward
than control stimuli irrespective
of the reward being Perceived
or Imagined

Predicting the imagined contents

Midbrain activation (cluster average)

Support Vector Machine classifier trained on midbrain

activation from visually presented trials successfully predicts
whether the participant is imagining a reward or control
picture

Visual presentation
Imagination

CS+ CS CS+ CS-

Participants

Data mining: Meta Analysis

Summary
Computational Tools are indispensible for
neuroimaging
Classical Analysis uses a standard framework
for functional localization
We can ask questions about functional
Integration (a.k.a. effective connectivity)
Machine Learning Methods have made the
reverse inference of cognitive states possible
Further advances in Computational data
mining techniques are to bring in a revolution
in Neuroinformatics

Thank you

http://cogs.iitgn.ac.in

The small Answer

Study the human
BRAIN!

The Nobel Prize in Physiology or Medicine for

2003 jointly to

Paul C. Lauterbur

Sir Peter Mansfield

"for their discoveries concerning

magnetic resonance imaging
http://www.nobel.se/medicine/laureates/2003/press.html