Introduction to cognitive

psychology I.
Sensation and perception
CogPsy modules
• I. Perception
• II. Memory and Attention
• III. Higher order cognitive processes: thinking & reasoning
Requirements
• Written exam will include:
• Literature
• E. B. Goldstein: Sensation and Perception (8th edition!) (bookfi.net)
• M. W. Eysenck & M. T. Keane: Cognitive Psychology (6th edition)
(http://gen.lib.rus.ec)
• Only chapter 1
• Powerpoint slides - will be uploaded
Cognition
= getting information out of the environment, to get
to know the world
What does it mean?

• ., •
• •
•
• • •
•
•
•• • •
••
• •
Information vs. humans
• Information in itself is totally meaningless
• Our mind has to react to the information, it creates the meaning
• Human perception is limited only on information that we can decode

→ Sensation and perception

 What kind of information
is around us?

THE ELECTROMAGNETIC SPECTRUM

+-'
(l)

Micro- Infrared Visible -~

Radio waves waves radiat ion light ~ X-rays Gamma-rays
.±=
:::>

~~I I I I I I I I I I I
I I I I I I I I I I
I

103 1 10-3 1o-s 1o-7 10-9 10-11 10-13

What type of information can we work with?

• Only that we have senses for (input)

 What type of information can we work with?
SMELL

VISION HEARING TASTE TOUCH

ree Nerve
ndlng

Rod Cone

•
Signal Receptor Signaling
Is it easy to decode the information?
Why do we need information?
To detect our environment effectively, so that we can survive in it.

→Recognize danger
→Get food
→Avoiding while obstacles travel from one place to another
→Find a warm place to sleep
→Communicate
→Recognize others
→Distinguish people
…
Information = survival
• Our greatest fears are in connection with lack of information,
insecurity, uncertanity, ambigousity
→ fear from dark
• We can’t predict what will happen
• Seeing bad things coming is better, than seeing nothing…
But:
• To access to information is not enough to adapt to our environment
• We also need a system to process information with → mind

→ compare with previous experience

→ memorize
→ plan according to it
→ communicate about it
…
What do we do with the information?
• Sensation
• Perception
• Reaction
• Memorize
• Recall
• Compute
• Imagine
• If we don’t use it, we forget it
21th century: • Primary aim: to understand the
mysteries of the human mind and brain
• 1990’s Decade of the Brain (next
slides)
• Cognitive Ψ (CP) – became extremly
important within clinical Ψ
• 2 areas: assessment and treatment
• 4 levels of description in Ψ:
cognitive, emotional, behavioral
and physiological
• Usually all of these levels must be
assessed and all of them are
potential interventional points
Number of scientific articles based on
PUBMED
"cognitive" OR "cognition"
45000

40000

35000

30000

25000

20000

15000

10000

5000

0
 0
20000
40000
60000
80000
100000
120000
140000

1944
PUBMED

1946
1948
1950
1952
1954
1956
1958
1960
1962
1964
1966
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
"brain" OR "neuro" OR "neuroscience"

1994
1996
1998
2000
2002
2004
2006
2008
2010
2012
Number of scientific articles based on

2014
2016
2018
What is cognitive Ψ:
• „It is concerned with the internal processes involved in
making sense of the environment and deciding what action
might be appropriate (next slide)”
• These processes include: attention, perception, learning,
memory, language, problem solving, reasoning and thinking.
• Another definition: aiming to understand human cognition by
observing the behavior of people performing various cognitive
tasks.
The system of cognitive processes

Thinking
Imagination
Learning
Memory
Attention
Perception
Sensation
The system of cognitive processes

Thinking
Imagination
Learning
Memory
Attention Input of
information
Perception
Sensation
The system of cognitive processes

Thinking
Imagination Interpreting
Learning information,
giving meaning,
Memory
integrating
Attention features
Perception
Sensation
The system of cognitive processes

Thinking
Imagination Select needed
Learning information
Memory
Attention
Perception
Sensation
The system of cognitive processes

Thinking
Store and recall
Imagination information
Learning
Memory
Attention
Perception
Sensation
The system of cognitive processes

Thinking
ImaginationRecognize, classify
Learning
Memory
Attention
Perception
Sensation
The system of cognitive processes
Manipulate,
Thinking modify
information
Imagination
Learning
Memory
Attention
Perception
Sensation
The system of cognitive processes
Problem solving,
inferences
Thinking
Imagination
Learning
Memory
Attention
Perception
Sensation
 Sen ation V: . Perception
---- -- ------ -- ---- -- ------- ------- - '@- ~ J-" -------- - - - - - - - - -

► 4 Sens t1on and perception

Sensation vs. perception
Soft

Pink
+
+ Smells
Rose
good

Hurts to
touch
+
 Sensation Perception
• Organizes the sensory
Neural response to the stimuli from the environment to
information (features) to
register information
objects, meanings
Needs sensory systems (vision, hearing, olfaction, taste
• Consists of psychological
and touch) to detect physical and chemical stimuli
processes, that are responsible
Constant for recognize, organize and give
meaning to the information
• Turns sensory information
(color, taste, sound…) to
experience
• has way more components than
sensations (expectations…)
„Black patches…”
-

- ~-
„Dalmatian!”
-

- ~-
The need to find meaning
• Detection and recognition activates the reward circles in the brain
• We have been evolved to enjoy recognizing things
• We have been evolved to always search for the meaning
Our perception defines
our reality
• Different information = different
reality
• Different mind processes = different
reality
• Reality is NOT objective.
Sum up and take home

• https://youtu.be/S1jn86eUX0E
Questions, thoughts :)
About the field of cognitive psychology
Sensation
Signals

• Chemical signals:
smell, taste
• Physical tastes:
touch, see, hear
 What type of information can we work with?
SMELL

VISION HEARING TASTE TOUCH

ree Nerve
ndlng

Rod Cone

•
Signal Receptor Signaling
The perceptual process
• Receptor processes
• Sensory receptors are cells specialized to respond to environmental
energy, with each sensory system’s receptors specialized to respond
to a specific type of energy.
• Visual receptors respond to light
• Auditory receptors to pressure changes in the air
• Touch receptors to pressure transmitted through the skin
• Smell and taste receptors to chemicals entering the nose and mouth.
• The receptors transform environmental energy into electrical energy
 RECEPTORS

Vision Touch Smell Hearing Taste

Receptors of the skin
Epidermis

Meissner corpuscles
(touch)

Merkel disks
(touch)

Pacinian corpuscles

f
Krause end bulbs
(touch)
(pressure)

Ruffini endings
Dermis (pressure)

Root hair plexus

(touch)
Transduction
• Receptor processes
1_ ,
Receptor Light is reflected ◄ Environmental
• Example: visual process processes and transformed .....,._ stimulus

• Visual receptors transform e 0

light energy into electrical
energy with the help of Rod and cone reoeptors
(desc ri bed in Chapte r 2)
light-sensitive chemical line the back of the eye.
They chang1e light
called visual pigment, energry into ellectricall
which reacts to light. The energry and infiluence
what we perceirve .
transformation of light
energy to electrical energy Rod Cone 8 Recepto,r Proces,ses
is called transduction
The perceptual process
• Neural processing
• The tree is represented by electrical signals in thousands of visual
receptors, and these signals enter a interconnected network of
neurons, first in the retina, then out the back of the eye, and then in
the brain
• This complex network of neurons transmits signals from the
receptors, through the retina, to the brain, and then within the brain.
0 Perception G Recognition G Action
"I see something" 'Ifs an oak tree ' Let's have a close

Perception Recognition Action

◄

e 0
Figure 1.5 Steps 5- 7: Behavioral responses are perception, recognWon, and action . ©Cengagele in 2014
Neural background of
perception
Bernadett Atndt, 2023.02.13.
Sensation
Signals

• Chemical signals:
smell, taste
• Physical tastes:
touch, see, hear
The perceptual process begins: RECEPTORS
• Sensory receptors are cells specialized to respond to environmental
energy, with each sensory system’s receptors specialized to respond
to a specific type of energy
• Visual receptors respond to light
• Auditory receptors to pressure changes in the air
• Touch receptors to pressure transmitted through the skin
• Smell and taste receptors to chemicals entering the nose and mouth.
• The receptors transform environmental energy into electrical energy
 RECEPTORS

Vision Touch Smell Hearing Taste

Receptors of the skin
Epidermis

Meissner corpuscles
(touch)

Merkel disks
(touch)

Pacinian corpuscles

f
Krause end bulbs
(touch)
(pressure)

Ruffini endings
Dermis (pressure)

Root hair plexus

(touch)
 Environmental energy → Electrical energy
SMELL

VISION HEARING TASTE TOUCH

ree Nerve
ndlng

Rod Cone

•
Signal Receptor Signaling
Transduction
• Receptor processes
1_ ,
Receptor Light is reflected ◄ Environmental
• Example: visual process processes and transformed .....,._ stimulus

• Visual receptors transform e 0

light energy into electrical
energy with the help of Rod and cone reoeptors
(desc ri bed in Chapte r 2)
light-sensitive chemical line the back of the eye.
They chang1e light
called visual pigment, energry into ellectricall
which reacts to light. The energry and infiluence
what we perceirve .
transformation of light
energy to electrical energy Rod Cone 8 Recepto,r Proces,ses
is called transduction
0 Perception G Recognition G Action
"I see something" 'Ifs an oak tree ' Let's have a close

Perception Recognition Action

◄

e 0
Figure 1.5 Steps 5- 7: Behavioral responses are perception, recognWon, and action . ©Cengagele in 2014
The perceptual process
• Neural processing:
• The tree is represented by electrical signals in thousands of visual
receptors, and these signals enter a interconnected network of
neurons, first in the retina, then out the back of the eye, and then in
the brain
• This complex network of neurons transmits signals from the
receptors, through the retina, to the brain, and then within the brain.
 ...
__,.,_.
 so o en o
cort r/
~

Cortical organisation
 Primary motor cortex Primary somatic
sensory cortex
Skeletal
Motor association area
muscle (premotor cortex)
movement Sensory information
from skin,
Sensory musculoskeletal
association system, viscera,
area and taste buds

- - Visual association
area
~ ,...._ Occipital lobe Vision
Prefrontal -r---:""""
association
area

Coordinates Auditory association

information from
area Hearing
other association .., t !Gustatory
areas, controls 'as e 1£ortex Auditory cortex
some behaviors Temporal lobe
Sme//-@~~~~~ory
vision
Light: the stimulus for vision
• Visible light is a type of electromagnetic energy that is produced by
electric charges and is radiated as waves
• It’s wavelength (the distance between the peaks) is important for us:
Visible light has a wavelength between 400 and 700 nanometers
• For humans and for some other animals, the wavelength is associated
with the different colors
ST P P3 STEP2 s
OC SSI Li s:
a el ·n a a e r e
crea e n1 0
ere 0
 Lateral rectus muscle

Iris

1~ .l---r-- Fovea centralis

Pupil Vitreous body (central depression)
(filled with
vitreous humor)

Anterior chamber
(filled with
aqueous humor) Optic nerve and
retinal blood vessels

Suspensory ligaments
Ciliary body and muscle Medial rectus muscle

Right Eye (viewed from above)

The human eye
• Light reflected from objects in the environment enters the eye
through the pupil
• It goes through the cornea (80 percent focusing power) and the lens
(20 percent focusing power), which direct the light on the retina
• Retina = network of neurons on the back of the eye, it contains the
receptors for vision
Quick summary of the eye
• https://youtu.be/unWnZvXJH2o
Cell layers of the retina

Back
of
eyeb II
The main function of the retina
• Reverse retina
• Photoreceptors: rods and cones – they got the names based on the
shape of their outer segments
• The outer segments of the photoreceptors contain visual pigments
that react to light and trigger elecrtical signals
• Fovea: place of sharp vision (when we look directly on an object, its
image falls on the fovea)
• Blind spot: the spot where the optic nerve leaves the eyeball (there
are no photoreceptors)
The blind spot
• The nerve fibers from the whole retina meet here to build up the
optic nerve
• It is hard to detect, because during the cortical processing, our brain
fills up the blind spot’s visual image (using the visual information
coming from the other eye and the surrounding regions)
 Retinal layers Components

9 Nerve fiber layer 10 Inner limiting membrane

Cell layers of the retina: Axons at urface of

retina passing via
8 Ganglion cell layer
optic nerve, chiasm
and traot to la eral
• Choroid – This is made up of a geniculate body

layer of blood vessels that 7 Inner plexiform layer Ganglion cell

supply oxygen and nutrients to

the retina. 6 Inner nuclear layer
MOiier cell
supporting gliat cell)
• Retinal pigment epithelium – Bipolar cell
This is a single layer of cells
that provide essential nutrition 5 Outer plexifo m layer
Amacrine cell

and waste removal for the Horl ontal, cell

photoreceptor cells. Rod

Cone
• Photorecptors – This is where Outer nuclear layer

the rods and cones are located 3 Outer imiting membrane-..;:;:::::::.

that convert light into electrical 2 Photoreceptor layer
Pigmen cells

signals.
1 Pigmen epithelium

Koeppen & S nton: eme and evy Physiology, 6th Editton.

Copyrlgh © 2008 by Mosby, an lmpr nt or Elsevle.r, Inc. All rtghts resef'Ved
 Retinal layers Components

Cell layers of the retina: 9 Nerve fiber layer 10 Inner limiting membrane

Axons at urface of
retina passing via
8 Ganglion cell layer
• Horizontal cells – These cells optic nerve, chiasm
and tract to la eral
are connect to the geniculate ody

photoreceptors and help 7 Inner plexiform layer Ganglion ce I

integrate and regulate the input
from multiple photoreceptor
cells, increasing your visual 6 lnr1er nuclear layer
Muller cell
supporting glia cell)
acuity.
• Bipolar cells – These cells take Amacrine cell
the electrical information from 5 Outer plexiform layer
the photoreceptor cells and pass Hori ontal cell

it along to other retinal cells.

• Ganglion cells – These cells Outer nuclear layer
extend to form an optic nerve 3 Outer limiting membrane-~
that conveys information to the Pigment cells
brain and take the electrical 2 Photorece tor layer

information from the bipolar

cells. 1 Pigment epithelium

Koeppe,, & S ntoo: ema ancl evy Physiology, 6th Edition.

Copyrfght © 2008 b Mosby, an mp nt or El.&e\lier, 1J1c. II rtghts reser;ed
 Fotoreceptors
Outer
• Transduction (concept in Segment
Outer
physics) is the transformation Segment

of one form of energy into Connecting

Cilium
another form of energy. The Inner Inner
process of transduction for Segment Segment

vision—the transformation of
light energy into electrical - - Nucleus
energy—occurs in the
receptors for vision: the rods - - - Axon - - - 1

and cones. Synaptic _r- Synaptic

Ending ~ Ending

Rod Cone
 Rods Memk.raF1e sJ1e lves.
Ii fl e cl itl-i rh ~cl~ pl s.i
~ r c ~ I ~ r pl i; me Fl t -:.

• The rods are not sensitive to color

• They are responsible for our dark-adapted (scotopic)
vision
• The optimum dark-adapted vision is obtained only 1fl fl e r s. e ; m e Fl t
after a considerable period of darkness, say 30
minutes or longer, because the rod adaption process is
much slower than that of the cones.
• The rod sensitivity is shifted toward shorter Out er li m iti n;
me m k. ra Fl e
wavelengths compared to daylight vision.
 Rods MI! m br• rn! sh I! M!! Si
linl!d nith rhadapsi
a r ca I a r pi; m I! nt

• While the visual acuity or visual resolution is

much better with the cones, the rods are better
motion sensors
• Since the rods predominate in the peripheral Mita ch and ri •

vision, that peripheral vision is more light

sensitive, enabling you to see dimmer objects in
your peripheral vision. If you see a dim star in
your peripheral vision, it may disappear when you Dutl!r limitin;
ml!mbr•nl!
look at it directly. Nucll!us

• You can detect motion better with your

peripheral vision, since it is primarily rod vision.
!iyn•ptic bady
 ,,
,,,,,
Cones lnvaginations of
cell membranes
,,
,/,,
that form a stack of/'_,,
,,,,'
Outer
segment
membranous disks ',,, ___ Connecting
where photopigments',, ________ - cilium

• Color sensitives
exist as transmembrane . .- .. , -
proteins , /~ - - --- _ . Inner
. ,,,,' - . I segment
Current understanding is that cones can be divided into: Mitochondria ----------.. •

• "red" cones (64%),

====:.:.:.-:.-:.;~ -,~
• "green" cones (32%), and
• "blue" cones (2%) based on measured response curves.
Nucleus

• The green and red cones are concentrated in the fovea

• The "blue" cones have the highest sensitivity and are Synaptic terminal
mostly found outside the fovea. that forms a synapse l
with a neuron •,,,, ..'
,, tl
h
~ J! ~

Cone cell
 Bl nd spo Co es
( ods
80.000
60.000
0.000
20.000
00.000
80000
60.000
0000
20,000
e e
0
7D° 0 SD°
Angle (d gree

Fi ur 2 4 The dis ri u ion o ro s a d co es in he e i a. T e e e o the le indica es loca ·ons i eg ees

rela i e tot e fo ea . ese locations are epeated alo g e bo o o e cha on he rig t . eve ical
b o n bar ear 20 degrees indica es he ace o e re 1na ee re are no recep o s beca set is is
h re e a lion cells le e he e e to o the op ic ne ve. rA1t:111!1111 1971
Physiological basis of trichromacy: the three cone types

Color sensitivity of rods and cones

Roorda iii s, 1999, a re, 397, 520

Human cone mosaic in vivo. Coloured for illustration only.

Human retina in vitro. M

1
~
·-...,>
·-·-
Cone photoreceptor
spectral sensitivities:
"'
C:
Q) 0.1
"'>
G)

·-co
+"'

Q) 0.01
0:::

400 500 600 700

Wavelengt (nm)
 Threshold curve

"0
0
.r:.

The relation between

U)

threshold and
sensitivity (a)
400 500 600
W velength (nm)
700

Spectral sen itivity curve

400 500 600 700

(b) W velength (nm)

Figur · 2.15 (a) The hres old or seeing a lig t as a functio o

wavelength. {b) Relative sensi ivity as a unc ion of wavelength- he
spec ral sensitivity curve. aid, 19641
 - - - Pure cone c rve Figure 2. 3 Three dar adap a ion
- - - Pure od c rve curves. The red line is e o-s age dar
- - - Bo ods and co es adap a ion cu ve, w· h an ini ·a1 cone
branch and a la er rod bra c , hich
ad lig -adapted se si rvi occurs when e est light is in the
perip eral e ina, as shown in Figure 2.12.
The green line is e co e adapta io
Low
curve, which occurs whe e est ight
alls on the fovea. The p rple curve is the
od adap a ion curve measured i a rod
monochroma. o e at e do nward
movemen o these curves represen s a
increase in se s· ivi y. The curves ac ally
begin at e poin s i dica ing ,, ligh -
adap ed se sitivi ;' but he e is a slig t
dela be een the time he lig ts are
urned o and when measureme of he
curves egins.

en axim m co e se s1 iv,
.s

Dar -adap ed

0 20
Time in dark (min)
Adapting to the dark
• Dark adaptation: the sensibility of the receptors changes with the
time spent in light/dark environment
• Light sensitivity increases in 2 steps: there is an increase at about 3-4
minutes after turning off the light, and 7-10 minutes
• Dark-adapted sensibility is about 100.000 times greater
• Keeping an eye in the dark triggers dark adaptation, which causes the
eye to increase its sensitivity in the dark (pirates’ eye patches)
 Neural Convergence
• Convergence occurs when a number of Rods

neurons synapse onto a single neuron

[ Cones
l
• A great deal of convergence occurs in the
retina because each eye has 126 million Bipolar Ce s [ Bipolar Cells
l
receptors but only 1 million ganglion cells [ Convergence No Convergence

• Thus, on the average, each ganglion cell

receives signals from 126 receptors.
• An important difference between rods and :._! .- --[ Ganglion Cells ]
cones is that the signals from the rods
converge more than do the signals from
the cones
(a) Li t Lig
Neural Convergence
• There are 120 million rods in the retina, but only 6 million cones. Thus, on the
average, about 120 rods send their signals to one ganglion cell, but only about 6
cones send signals to a single ganglion cell.
• Many foveal cones have “private lines” to ganglion cells, so that each ganglion cell
receives signals from only one cone, without convergence
→ that is the cause of the sharp and detailed vision in the fovea
• Convergence causes the rods to be
more sensitive than the cones!
• Lack of convergence causes the cones
to have better acuity than the rods!

lllll
rrr r r
(b)

Figure 2.32 How the wiring o the rods and cones determines
de ail vision. (al Rod neural circui s. On the left, stimulating two
neighboring rods causes he ganglion cell to fire. On the right.
stimulating two separa ed rods causes he same effec . {b) Cone
neural circui s. On the le . stimula ing wo neighboring cones
causes wo neighbo ing ganglio cells o ire . On the right.
stimulating two separated cones causes two separated ganglion
cells to fire. This firing o two neurons, with a space between them.
indica es that wo spo s of light have been presented to the cones .
Focus
• The lens can change its shape (and thus the intensity of bending the
light) to adjust the eye’s focus
• Accomodation: the ciliary muscles at the front of the eye tighten, the
curvature of the lense increases
• Either near or far objects can be in focus, but not both at the same
time
More about anatomy and deficits
• Myopia:
https://www.youtube.com/watch?v=dWqrnsDtmpU
• Hyperopia:
https://www.youtube.com/watch?v=a_cTP1eLJIc
• Laser eye surgery:
https://www.youtube.com/watch?v=dKANhIU7Sxk
Anatomy, convergence, adaptation
Quick summary
• https://youtu.be/o0DYP-u1rNM
 Lateral rectus muscle

Iris

1~ .l---r-- Fovea centralis

Pupil Vitreous body (central depression)
(filled with
vitreous humor)

Anterior chamber
(filled with
aqueous humor) Optic nerve and
retinal blood vessels

Suspensory ligaments
Ciliary body and muscle Medial rectus muscle

Right Eye (viewed from above)

Ophtalmoscope
Figure 1. A view of the retina seen though an ophthalmoscope.
 Fovea centralis

Fotoreceptors

→ cones in the
fovea
The visual field

Foveal vision

Parafoveal vision

Peripherial vision

LAT6MEZ6
The visual field
A receptive field
rround

Surround
cone

Bipol
C I

0 < nt r ----
OFF- urtound o optic n rv
n lio C I
201 1
Processing from retina to visual cortex
• Optic nerve is made up of many individual nerve fibers traveling
together. These fibers are the axons of the retinal ganglion cells.
• Each fiber responds only when a small area of the retina is illuminated.
The area that causes the neuron to fire is called the nerve fiber’s
receptive field.
• A fiber’s receptive field covers a much greater area than a single rod or
cone receptor.

...
The receptive fields (RF)
• RFs are arranged in a center-surround
organization, in which the area in the
“center” of the receptive field responds
differently to light than the area in the
“surround” of the RFs.
• Excitatory area – presenting a spot of
light increases firing Surround
{a) (b}
• Inhibitory area - presenting a spot of light
decreases firing
• (a) excitatory-center, inhibitory-surround
receptive field.
• (b) inhibitory-center, excitatory-surround
receptive field
Receptive fields
Receptive field ~ -:-::-~-.!..;;;
Light surround

A receptive Receptive field

field center

Photoreceptor
cells (cones)
in retina Horizontal ____.-;;
cell

Bipolar -------
cell

(a) On-Center/Off-Surround
Copyriglrt Houglrton Mifflin Company. All riglrts reserved. Receptive Field of a Bipolar Cell
 Receptive fields
• The center-surround receptive fields show that neural processing could result in neurons
that respond best to specific patterns of illumination.
• A small spot of light presented to the excitatory center of the RF causes a small increase
in the rate of nerve firing (a)
• Increasing the light’s size so that it covers the entire center of the RF increases the cell’s
response, as shown in (b).
• When the spot of light becomes large enough that it begins to cover the inhibitory area,
as in (c) and (d). ,., ,, ---
,, --
,., ,,,-
' ., ,, ..... .... -- -' -.._

0 \ ,' 0 \ ((Q)'-:
'
,
/ ;
'
• Thus, because of center-surround antagonism,
I - .- \
I
\
'
0 \
I \
II - 0 - I I 1
r
this neuron responds best to a spot of light that \ + + / I
\ . J \ I \
\. .... - - ,, / ' . ,, I /
--- .,.,,. ✓
'
' ........... _ ,,.,. ,,. /
I -...
........ .__ .,., . .,. ' '- ,.. ,,.
is the size of the excitatory center of the RF.
...
.....

1111111111111 11111111111111111111111111111 1111111 I I I

t
On On
t On
r-
On
t
(a) (b} (c) (d)
 On

r
I
11

I I
What happens after leaving the retina?
The optic nerve and optic tract
Visllal area
of the thalamus

Retina

Visual
cortex
Optic nerve pathway

..
---
~, • ---
Optic nerve damage: consequences
video: https://www.youtube.com/watch?v=cG5ZuK0_qtc
Left E e RightE e Left E e Right E e

temporal fi . Id na. ·sa

· :·'.1 fr ldA·
nos .
n sal fie ld tem ral fi. Id
onna I ision:

00
Optic ero : Ri ht n opsia (=blind)

Opti Chiasm

Optic Trncl
0
Optic chiasm: Bitempo.rnl. hemianot:>sia

Optic trnct: Hamon mous ber i.anopsia

 Frontal lobe
Parietal lobe

Lateral
geniculate
nucleus
Visual cortex (LGN)

V2

Inferior temporal
cortex (ITC)
What type of stimulus is the visual cortex sensitive for?
I I I I
1- - 1+ + 1- - I
1- I 1- I
I - 1+ I - 1
I
1-
-
- ++ -
I
I
I
+ 1
-
I
I
1111111111 11 I
1- -
I -
I -
1- _
1+
I
I+
I
1- _ 1
+ 1 -
I
I
- 1
+ 1_ - I 0
l 0
f t
on
t
off
I I I - 1
Recep ive 1eld-
s1mple cortical cell
(a) (b) (c)

Orientation tuning curve

• https://youtu.be/y_l4kQ5wjiw
~ 25
0
()

.
I
8.
I
:,
0 • https://youtu.be/IOHayh06LJ4
Q.
E

Vertical
Orientation (degree )
(d)

Figure 3.23 (a) The receptive field o a simple co ical cell. (b) This cell responds bes o a vertical bar o ligh
ha covers he e c1 a ory area o the recep ive field . (c} The response decreases as he ba is ii ed so that i
also covers he mhibi ory area . (d) Onenta io uning curve o a simple co ,cal cell for a neuron ha responds
best o a vertical bar (onen a ion = 0).
What does the brain do with visual
information?
• The main functions of visual perception are to…
• Detect - the brain recognizes the object
• Discriminate - diffenerciates its parts
• Localize – determines where it is
• Identify - determines what it is
…objects in our environment.
• Different brain regions are responsible for the different functions
 What and where pathways

https://www.youtube.com/watch?v=nOdXzVL5YKo
 Dorsal proce ing: tream: .. where?"
Stimulus ocaliza · o

Lay rs VS
VI t d
V motor r • pon
We
Motion D pth
/Vb
Wa.
Shape/
Vi ual
, hap ,obj c re ogni ion
IL ll1
Color V ITC
Color

timuJus recognition
I Shape
Ventral proce ing am: "wha ?
VI
■
■

V2

Parvocellula:r
Retina LGN
cs, Orlcmmian Color

o~l.kmof
~ Bi 110t12il,u;-
l':J Dlaparl1y
C'cmc:r ' iii
Rwmund.'
0) Shll[le - I
iii
ill

.........•,' __ _
PluDaN ' !I
!ii
iii
I
., !ii

--------------------------~•
Dorsal pathway („Where?”) Ventral pathway („What?”)

• Movement and distance • Shape, color, depth and

perception texture processing
• Localization • Detailed analization
• This pathway is „color • Identifying things
blind”. It only sees
luminance differences
Higher order visual processes
• Fusiform Face Area (FFA)
→ Lesion: prosopagnosia (face blindness)
The Chemical
Senses
Smell and Taste
• Which sense do you think is the most important one?
• Which sense is the oldest?
• Why do we smell things?
• Why do we taste things?
The olfactory system: functions
Olfaction is extremely important in the
lives of many species
→ Their primary window to the
environment
• Macrosmatic animals: • Microsmatic humans:
it provides cues to orient them Pheromones: chemical signals
in space, to mark territory, to released by an individual that
guide them to specific places, affect the physiology and behavior
animals, and food sources, is of other individuals
also important in sexual
reproduction because it triggers Menstrual synchrony: women
mating behavior who live or work together often
have menstrual periods at about
the same time
Anosmia
The loss of the ability to smell
https://www.youtube.com/watch?v=8qXhh5Rx8NA

„Between two balls of ice cream i’m not able to tell which is vanilla and
which is chocolate.”
Hit by a car, brain tumor, motorcycle accident, inherited
 • Our sense of smell enables us to detect low concentrations of
some odorants.
• the detection threshold: the lowest concentration at which
an odorant can be detected.
1. Detecting
odors • rats : 8 to 50 times more sensitive to odors than humans
• dogs: from 300 to 10,000 times more sensitive - depending
on the odorant
• Why? humans have far fewer receptors than dogs
(10 million vs 1 billion)
2. Identifying odors
• When a person can just detect the presence of
an odor, the person not neccessarily can sense
the quality of the odor—whether it is “floral”
or “pepperminty”.
• The concentration at which quality can be
recognized is called the recognition threshold
• Humans can discriminate between 100,000
different odors
2. Identifying odors
• When we have trouble identifying odors, it sometimes results not
from a deficiency in our olfactory system, but from an inability to
retrieve the odor’s name from our memory….
The puzzles of odors
• How does the olfactory system
know what molecules are
entering the nose?
• The mucosa is located high in the
nasal cavity that contains the
receptors for olfaction
• Odorant molecules are carried
into the nose in an air stream,
which brings these molecules
into contact with the mucosa
 Olfactory receptor neurons (ORN)
Olfactory
tract
_j
• Vision: visual pigments - sensitive to light
lfactory tract
~ ~- - - Cribrifonn plate
of ethmoid bone • Olfaction: olfactory receptors – sensitive
lfactory bulb

,._.._c~~ ~======:~ ~; - - - Filaments of

olfactory nerve
to chemical odorants
asal
- - - - Lamina propria
connective tissue
• each type of olfactory receptor
>nchae Olfactory
Gland
- -- - Axon (~350 different types, >10 000 of each)
.,,,..-- Basal cell
==-- - Olfactory
is sensitive to a narrow range of
Olfactory
sensory neuron odorants.
::::,-- - Supporting epithelia cell
Epithelium • compared to mouses much fewer
=--- Dendrite
~ - - Olfactory cilia (mouse ~1000 types)
ucus

(b) Route of inhaled air

containing odor molecules
 Olfactory receptor neurons (ORN)
• Bettina Malnic and coworkers (1999)
determined the response to a large
number of odorants using calcium
imaging
• The figure indicates how 10 different
ORNs are activated by each odorant
(recognition profile) ~

• Remember that some molecules may

have similar structures but smell
0
different (because they have different BR~
recognition profiles ostanoic acid and
nonanoic acid - figure)
 Activating the
olfactory bulb
• These ORNs send signals to rants .u.::e

structures called glomeruli

in the olfactory bulb
• Each glomerulus collects
information from a
particular type of ORN
(all of the ~10,000 ORNs
of a paticular type send
their signals to just one
glomerulus)
Higher-order olfactory processing
• Signals are transmitted from the olfactory bulb to the
piriform cortex
• primary olfactory cortex (PC)
• and the amygdala,
• and then to the orbitofrontal cortex (OFC)
(secondary olfactory cortex).
• The amygdala is associated with
emotions and so plays a role
in the emotional reactions
that odors can elicit
The location of the olfactory cortex
Sensory
association
area
J
----,.;ii..- Visual 2
area
Occipl
Visual

Auditory associati
area
Taste !Gustatory
l ortex Auditory cortex
Temporal lobe
Smell JO Ifactory
~ ortex
 Higher-order olfactory processing
• Three main principles:
• (1) many molecules are creating a single
perception like “coffee” or “bacon,” (> 100 Fu I I111111---
different molecules)
• (2) the ability to separate odors from one
amz n n r ......
I I
another in the environment (coffee and
bacon in the same time)
• (3) the effect of past experience and
learning on odor perception (positive
PL■■-1--
experiences – positive rating of the odor)
• These indicate that odor perception must
involve more than just a hardwired
“readout” of the pattern of ORN firing!
• More research is needed…
Henning’s smell prism
Flowery Fruity

Putrid

Spicy Resinous

Burnt
The gustatory system
• Function: choose which foods to eat and
which to avoid
→ Get nutrition rich food, avoid harmful, non-
edible things, detect immature fruit…
→ Evolution: preference for useful things
 The taste system
• Function: choose which foods to eat and
which to avoid
• sweetness is often associated with
compounds that have nutritive or caloric
value (important for life). It also trigger
anticipatory metabolic responses that
prepares the gastrointestinal system for
processing these substances
• Bitter compounds have the opposite
effect—they trigger automatic rejection
responses to help the organism avoid
harmful substances
 The taste system
• Function: choose which foods to eat and
which to avoid
• sweetness is often associated with
compounds that have nutritive or caloric
value (important for life). It also trigger
anticipatory metabolic responses that
prepares the gastrointestinal system for
processing these substances
• Bitter compounds have the opposite
effect—they trigger automatic rejection
responses to help the organism avoid
harmful substances
• When people are deprived of sodium (salt)
through sweating, they will often seek out
foods that taste salty in order to refuel the
salt their body needs
 Basic taste qualities and preferences
• Basic taste sensations:
salty, sour, sweet, bitter and
umami (meaty, savory,
associated with the flavor-
enhancing properties of MSG,
monosodium glutamate)
• Universal – in every culture
• People can describe most of
their taste experiences on the
basis of these sensations

Control Sweet Sour Bitter

(normal)
 Basic taste qualities and preferences
• Basic taste sensations:
salty, sour, sweet, bitter and
umami (meaty, savory,
associated with the flavor-
enhancing properties of MSG,
monosodium glutamate)
• Universal – in every culture
• Inherted
• People can describe most of
their taste experiences on the
basis of these sensations
Control Sweet Sour Bitter
(normal)
Henning’s taste tetrahedron
Saline

Sour

Sweet

Bitter
 Structure of the taste system
The process of tasting begins with the tongue
Papillae of Tongue (receptors are stimulated by taste molecules)
The surface of the tongue contains ridges and
valleys caused by papillae (4 types):
• (1) filiform papillae (found over the entire surface
of the tongue)
Palatine tonsil
• (2) fungiform papillae - shaped like mushrooms
(found at the tip and sides of the tongue)
Lingual tonsil
Circumvallat
• (3) foliate papillae (found along the back of the
papillae tongue on the sides)
• (4) circumvilliate papillae - shaped like flat
mounds (found at the back of the tongue)
Fungiform Filiform papillae
papillae ~ ...,_..;..
Surface of the tounge
Structure of the
taste system Taste buds fl
Circumvallate papilla

• All of the papillae

contain taste buds
(except the filiform
papillae – there is no
sense of taste), the
whole tongue constains
~10 000 buds

Fungiform papilla Filiform papilla Foliate papilla

Transitional cell
 Structure of the taste system
• Each taste bud contains 50–100 taste cells, which have tips
(chemicals contact receptor sites located on the tips)

Epllhellui at tongue
Epiglottis

~-P~ - Palatf ne tonsil

Connective
Lingual tonsil tl-ssu
Surface of
th@ to11gue

Circumvallate papilla

Basal
cell

ustatory hairs _/
(mlcrovllll) emerging
from taste pore
 Nerves from tongue
• Muscles:
Vagus nerve [X]
→ muscle movements in the
mouth and throat,
including speech
Hypoglossal nerve [XII]
• Taste:
Superficial petronasal neve [IX]
Trigeminus nerve [V]
Facial nerve [VII]
 Cortical processing
• The fibers from the tongue, mouth,
and throat make connections in the
brain stem (nucleus of the solitary
tract)
• From there signals travel to the
thalamus
• Then to two areas in the frontal lobe—
rw VII
the insula and frontal operculum
IN ,v

• https://www.youtube.com/watch?v=w
GXoYippog8
 The perception of flavor
• Most people look forward to
eating not because it is necessary
for survival but because of the
pleasure it brings
• What these people refer to as
“taste” when describing their
experience of food is usually a
combination of
• taste and smell (+vision?)
• This combination is called
flavor
• We are usually dealing with
not just one or two different
flavors but many
The perception of flavor
• Most of the basic research on flavor
has focused on showing how taste
and smell interact and on factors
that influence our perception of
flavor
• Your experiences?
Ageusea
• Ageusia is the loss of taste functions
Flavour
Try it out!
• “Tasting” With and Without the Nose
• Notice both the quality and the intensity of the taste as you are
drinking coffee, orange juice and wine with and without opened
nostril.
• First, close your nostrils, and notice whether you perceive a flavor.
Then drink the beverage normally with nostrils open, and notice the
flavor.
• Do you feel the difference?
 The physiology of flavor perception
• The mouth and the Orthonasal Retro nasal
nose are connected
olfactory olfactory
and both are bulb
bulb
important to sense odor odor
olfactory perception
falvour olfactory perception
receptors ~ receptors
• A number of cortical
areas that serve
both taste and
olfaction are • ••••••••
•
probably involved in
the perception of the
flavor of food 'out-there' 'in-the-mouth'
 The physiology of flavor perception
• The OFC receives inputs from the primary
cortical areas for taste and olfaction, as
well as from the primary somatosensory
cortex and from the inferotemporal cortex
in the visual „what” pathway
• The OFC contains many bimodal neurons,
those that respond to more than one
sense
• For example, some bimodal neurons
respond to both taste and smell, and
other neurons respond to taste and vision
→ Multisensory integration
 The physiology of flavor perception
12

• Because of these properties, and the fact that

the OFC is the first place where taste and smell
information is combined, it has been suggested
that the OFC is a cortical center for detecting
flavor and for the perceptual representation
of foods
• The OFC is affected by hunger:
The firing of neurons in the OFC reflects the
extent to which an animal will consume a
particular food (figure) .
• -1

l-2
.....
Thanks for your attention!
• Sum-up:
• https://www.youtube.com/watch?v=mFm3yA1nslE

• Five senses of attraction:

• https://psychologybehindlove.weebly.com/5-senses-of-attraction.html

• The science of wine tasting:

• https://www.youtube.com/watch?v=rymYS76U1eM

• Super tasters:
• https://www.youtube.com/watch?v=W7Pzhvypg9A
Thanks for your attention!
• Sum-up:
• https://www.youtube.com/watch?v=mFm3yA1nslE

• Five senses of attraction:

• https://psychologybehindlove.weebly.com/5-senses-of-attraction.html

• The science of wine tasting:

• https://www.youtube.com/watch?v=rymYS76U1eM

• Super tasters:
• https://www.youtube.com/watch?v=W7Pzhvypg9A
 The cutaneous senses,
proprioception, kinesthesis
debate
• Cutaneous senses: touch and pain perceptions, usually caused by the
stimulation of the skin
• Proprioception: the ability to sense the position of the body and limbs
• Kinesthesis: the ability to sense the movement of the body and limbs
The importance of touch
Losing of touch?
• When asked which sense they would choose to lose, if they had to
lose either vision, hearing, or touch, some people pick touch
• This is understandable given the high value we place on seeing and
hearing, but making a decision to lose the sense of touch would be a
serious mistake
Losing of touch?
• Because although people who are blind or deaf
can get along quite well, people with a rare
condition that results in losing the ability to feel
sensations though the skin often suffer constant
bruises, burns, and broken bones in the absence
of the warnings provided by touch and pain

\''
....
• Besides, many everyday actions would become extremely hard (eg.
Typing, eating, dressing up, sexual activity)
• Experiments with temporarily anesthetized hands —> People tend to
apply much more force than necessary
Video
• Congenital insensitivity to pain (CIP)
• https://www.youtube.com/watch?v=1vLsZ_dXFAg
The skin
• The heaviest organ in human body
• Warning function (against burning, aching,
dangerous stimuli)
• Prevents body fluids from escaping
• Protects us by keeping bacteria, chemical
agents, and dirt from penetrating our bodies
• Provides us with information about the
various stimuli that contact it (sun’s rays heat
our skin, pinprick is painful, touch of our
beloved)
 The skin Epidermis

• Epidermis – layer of dead cells is

part of the external layer of skin Dermis

• Dermis – layer below the epidermis

• Mechanoreceptors – receptors that Subcutaneous
respond to mechanical stimulation tissue
such as pressure, stretching
or vibration.
Sensory receptors in human skin
!db1 rod 0t .

epidermis
free
nelVe endi gs
(pa·n hea Meissner
cold) corpuscles
(touch)

Merkel
disks Pac nia
(touch) corpuscles
(pressure)

Krause
end bulbs
(touch) Ruffini
endings
(pressure)
---=-a...___. root hair plexus (touch)
 Mechanoreceptors close to the surface
1. Merkel receptor
• close to the surface of the
skin, near the epidermis.
• the Merkel receptor fires
continuously, as long as the
stimulus is on
nnnnnrnun
2. Meissner corpuscle
____r-i_
• close to the surface of the
skin, near the epidermis.
• the Meissner corpuscle
fires only when the
stimulus is first applied and •
when it is removed.
 Mechanoreceptors deeper in the skin
3. Ruffini cylinder
• located deeper in the skin
• the Ruffini cylinder responds
continuously
• it is associated with
perceiving stretching of the
skin
4. Pacinian corpuscle
. ,, II 11 ..
_r-----i_
• the Pacinian corpuscle FIRIIIID
Willlwww v--111,.,,
responds when the stimulus
is applied and removed.
• it is associated with sensing
rapid vibrations and fine
texture
Coutaneous receptive field
• The area of skin which, when stimulated, influences the firing of the
neuron
 Pathways from skin to cortex
• Nerve fibers from receptors in the
peripheral nerves that enter the
spinal cord through the dorsal root
• The nerve fibers then go up along
the medial lemniscal and the
spinothalamic pathway
• Lemniscal pathway: carries signals
related to sensing the positions of
the limbs (proprioception) and
perceiving touch
• Spinothalamic pathway transmits 11,1, ......____
signals related to temperature and
pain
Pathways from skin to cortex
• Fibers from both pathways cross
over to the other side of the brain
(signals from the left side of the
body reach the thalamus in the
right hemisphere)
• Most of these fibers synapse in the
ventrolateral nucleus in the
thalamus (thalamus is involved in
vision and hearing as well)
• From the thalamus, signals travel to Mr11-.,■ 1ilclla--
the somatosensory receiving area
(S1) in the parietal lobe of the cortex
and possibly to the secondary
somatosensory cortex (S2)
Somatosensory cortex
• The somatosensory cortex is
organized into maps that
correspond to locations on
the body
• This body map is called the
homunculus (“little man”).
• The homunculus shows that
some areas on the skin are
represented by large area of
the brain (thumb, lips,
tongue) others by smaler
(back, forearm)
• There are a number of
homonculi mapped in S1 and
(b)
S2
 Homunculus
• https://www.youtube. a,

com/watch?v=fxZWtc /0- ...

C
0-
"'
3 -.
0mYpQ&t=275s
,9 ~

0':/ /4e.., G
«('

s 'P

s,and j w

Lateral Medial

Homunculus Somatosensory Map

 llallallinl fin
dlngill'2
Neuroplasticity
I. •tmrn J
• One of the most fantastic feature of the brain
• If a computer could change its software
and HARDWARE!
• Experience-dependent plasticity - particular
brain area can become larger if that function is
used often
• E.g. Jenkins and Merzenich (1987) showed that
increasing stimulation of a specific area of the
skin causes an expansion of the cortical area
receiving signals from that area of skin in
monkeys (measured before and after 3
months of training)
Neuroplasticity
• For example:
• Violinists use their fingers (on
their left hand) for tiny
movements, they practice
houndreds of hours in their
lives
• The sensory representation of
their fingertips will have a
larger area in their
somatosensory cortex
compared to non-violinists
(Elbert et al, 1995)
Different types of perception:
• (1) perceiving details
• (2) perceiving vibration,
• (3) perceiving texture
• (4) perceiving (shapes of) objects
1. Perceiving details
• Example: Braille - the system of raised dots that enables blind people to read
with their fingertips
• A Braille character consists of a cell made up of from one to six dots
• Experienced Braille readers can read at a rate of about 100 words per minute
• •
a b C d e f g h I J k
•· •·
•· •• ••
·• •••• ••
•· ••
•• ••
•• •·• ·•
•• ••
•
• •

I m n 0 p q r s t u V
•• •• ••• •·• •• •• •· ·• •• •• ••
•• • •• •· •• ••••
•

• •
• • •
•
•· ••
• •·
• • • •· •· •• ••

W X y z
•••• •• ••• ••
•

• •• ••
• ·•
••
 Measuring tactile acuity
• The classic method is the two-point threshold
• the minimum separation between two points on the skin that
when stimulated is perceived as two points
• The two-point threshold is measured by gently touching the
skin with two points, and having the person indicate whether
he or she feels one point or two
 Cortical mechanisms for tactile acuity
There is a strong correlation between
• A) the density of receptors in the skin
• B) the representation of the body in
the brain and the
• C) acuity at different locations on the
body
Two-point thresholds for humans:
Two-point threshold on the hand
• There is a high density of Merkel receptors
in the fingertips, because the fingertips are 2n1m
the parts of the body that are most
sensitive to details
 36 to 75 mm

1.1 mm 3 to 8 mm
2. Perceiving vibration
• When your phone starts ringing in your
hands you can sense these vibrations
with your fingers and hands
• The mechanoreceptor that is responsible
for this is the Pacinian corpuscle (PC)
• The PC consists of layers (like an onion),
with fluid between each layer.
• It transmits only rapidly applied pressure
(like vibration), but does not transmit
continuous pressure Ill Ill
3. Perceiving texture
• We can sense textures ranging from coarse (teeth of a
comb) to fine (the surface of a paper sheet)
• Two important cues:
• Spatial cues - result in feeling different shapes,
sizes, and distributions of these surface elements
Works also without moving
• Temporal cues - provide information in the form of
vibrations that occur as a result of the movement over
the surface —> fine details (try it out)
➢When it is allowed to move the fingers across the
surface, we are able to detect the difference between
the fine textures, but if it is not allowed, it is impossible
4. Perceiving objects
• Identifying objects by intent (active touch) by haptic exploration
• Haptic perception is an extremely complex process because as we will see, the
sensory, motor, and cognitive systems must all work together.
• Researches have shown that people can accurately identify most common objects
within 1 or 2 seconds
 Lateral Motion Unsupported Holding
(Texture) (Weight)
,..,._.,,~
. 1- ---

Haptic exploration C'\,

'(1
l
~. } ~

-
...

---:r-- _,
t
'~ /- , /

strategies / \\(

--~
\ -\ '---,·;•'
-;~:

Enclosure
Pressure (Global Shape)
(Hardness) (Volume)
iC~~ -

Static Contad
(Temperature)
 To identify objects three distinct systems are used:
Detecting cutaneous sensations
such as temperature, texture,
softness, vibration and the
Sensory movements and positions of the
system fingers and hands
Thinking about the information
provided by the sensory and
motor systems
Compare to previous
experiences, if there is
a match Moving the fingers
→ recognized object Motory and hands
Using different haptic
system strategies to get more
information about the
shape and the texture
 Physiology of object perception
• As we move from mechanoreceptors toward the brain,
we see that neurons become more specialized
• This is similar to what occurs in the visual system
• Neurons in the ventral posterior nucleus (in the
thalamus) have center-surround receptive fields that
.....,
·IIURllllld
are similar to the center-surround receptive fields in
the lateral geniculate nucleus (the visual area of the
thalamus)
Monkey arm:
Physiology of object perception
• In the cortex there are neurons that respond to specific orientations and
neurons that respond to movement across the skin in a specified direction
• There are also neurons in the monkey’s somatosensory cortex that respond
when the monkey grasps a specific object

I I
· Ill JIIII
. . .

· ----- f illllllll 11 H
11 111

C•J
r (bJ
Pain
Pain
• Pain functions to warn us of potentially damaging situations
A and helps us avoid or deal with cuts, burns, and broken bones.
• People born without the ability to feel pain might be unaware
A of broken bones, infections, or internal injuries—situations
that could easily be life-threatening
• Definition: “Pain is an unpleasant sensory and emotional
A experience associated with actual or potential tissue damage”
Pain

• Three types:
• Nociceptive pain - is caused by
activation of receptors called nociceptors
• There are a number of different kinds of nociceptors,
which respond to different stimuli—heat, chemical,
severe pressure, cold
• Inflammatory pain - is caused by damage to tissues and
inflammations to joints or by tumor cells
• Neuropathic pain - is caused by lesions or other damage
to the nervous system
 Models of pain
• According to the „direct
pathway model”, pain
occurs when nociceptor
receptors in the skin are
stimulated

Node ptors
Skin
• But we already know that pain can be
affected by other factors as well
(e.g. by a person’s mental state
– mood, anxiety)
• Pain can occur when there is no
stimulation of the skin
• Phantom limb - people who have had a
limb amputated continue to experience
the limb after the surgery (somethimes
they feel pain)
• Pain can be affected by a person’s
attention
• The perception of pain can increase if
perception is focused on it, or
decreased if it is ignored
 Video
• Mirror therapy – phantom limb
• https://www.youtube.com/watch
?v=hrqi1B9Xbt0
 Proprioception
• The ability to sense stimuli arising within the body
regarding
• position,
• motion, and
• equilibrium

• Even if a person is blindfolded, he or she knows

through proprioception if an arm is above the
head or hanging by the side of the body
• Sensory information from muscles, tendons
and ligaments
Proprioception is used constantly • We can experience
the effects of
proprioception in
our everyday
Sl.ddenwelght
actions
change causes
musde to·extand.
•retching the
muscle splnclea·

•
Muscle splnclea trigger
I raflat to maintain
" ·, "" w "

tanslon In 1taa ll'IUld9

Video summary
• https://www.dailymotion.com/video/x73wjh2
(4:50-10:40; 18:30-26:50; 51:15-)
Proprioception
The Brain receives and interprets
information from multiple inputs:

r Vestibula;r organs in the

/ inner ear send information about
rotation, acceleration, and
position.

J Eyes send visual informtion.

Stretch receptors in skin,

muscles & joints send informatio
about the position of body parts.
Perceiving Color
0
 • • - • ·••* ··-·-.•-•
' • • • •.-
... --. ,-•
* ♦ .•-•..,*·•·
. ..
• • * • • ••-•·· • •- •-· • - - .- -, •• - • -.
... • • - • • • ... * • * ... • ... •... • • • ............
• - + I • - • * • I A. • A• •I A *..l •• -
- f • - .I. . I. •.+. -.•. . A - • •
·--·-■ - • • , .... - ■ *- • .. • - · •
. . . .•·
. . . . . . ••-.. . . .-. ...... ···-
. . . . ...... ,. . ...*

•. . .. *. . ... •. .... •. .
'
.....
_ .4 ...... ..
-•♦ . ,• • _ ♦ ♦- . . . . . . . . . . . .
•• •· . ••
...... * ♦ *•"' · · · • · - ,- ... ,-
.......... -

.
........ _ .. , ........ - ............. ... *
. +.
-··
·•.-·•·

....
··~·-•····- ·· ....
...
.•
•
-•• ......
♦ +

.
......... .
■
•
-♦
• ...
■
..........
• ....
• • • ... ..
.............
• ••
- • -*- • ..._.... -··

•-·
*
... ••
•
••
■
....
• ■ •
-•
•

·*•'
i
1. ,

. . -··· . . . ... .. ....... . .

- .......
*
■ I• - I
.. • · - ·•-'
• • t - I -
... • •
*-
.........
• .. • ...
I: *• - I•• I .. ':'
....

. . .. .... . . . ... .
11 • .& • A• A

• I ...- • * • • A • • • j,, ... - • - • ' • • - • • ••

• - ... ... • • * ... . - * • • lf. • , .... • ... - •
•• •• - • ■ • • -. - ... ..._ • • .... • • • ......... _ ... ... . - ... . *
- •... . 1
... .... ...
■
.... ...
■
...
• +
...... If-
- •• t •
. . ... .... ...
• - *• - +-• · •• •• , ..A ..■ ,,,1, ,■ ,- I
+

• ..._.. • • • I • • - • ,• ..._ "'- * ■ ■- I ■ • • • t • ■ .._ I • • & • •

■ ■ ■ ■ ■ ■ + .t. *
... • -*- •*•_... -··
I • - ...,

"'•• ... -,
. . .+ . . • •
-·A ·•-•··· ■•

I: A
... ....
-,· . ,... •
■_ .. I •
*•
• ,
••
•
•
+
,•••••
-
••
* •
♦
I
•
•
-•·■ -
.... •
1#-
-

.. ...
• • • I•
•
* •
• ■
..... ··-·
I • •
••
♦
...

A
•• I

e
■
A.
 Functions of Color
• Important signaling function
– Identify and classify things
– Helps facilitate perceptual organization
• Tell one object from another
• Pick out objects within scenes
• Tanaka & Presnell (1999)
• We interact with color all the time
– We pick favourites (blue is the most preferred)
– Associate it with emotions
• Red = embarrassment
• Green = envy
• Purple = rage
 Mr I.
• Became color-blind at 65
• Car accindent, concussion
• He became more depressed vs. Partially color-blined are not
disturbed by their decreased color perception
 What is color?
• Light = electromagnetic radiation
• It has a quantity — luminance
• And a quality — color
 Newton:
• Separated white light with a prism

TR ~TI ,E
01' TH -

LI G
 Visible light

White
light
Prism

Visible light
400 500 600 700

Ultra- Broadcast AC
Gamma violet Infrared
X-rays Radar bands circuits
rays rays rays

1 3 5 7 9 11 13 15 17
10 -5
10 -3
10 -1
10 10 10 10 10 10 10 10 10
Wavelength in nanometers (billionths of a meter)
• Pure/uniqe colors: blue, green, yellow, red

• Short wavelength – blue

• Middle – green
• Long wavelength – yellow, red
• People can discriminate between about 200
different colors
– Creating new colors: saturation, wavelength,
intensity
• We can create about a million or more different colors
Sunlight
• Color of light vs. Color of
Refiection
objects
– Color of light: related to
its wavelength
– Color of object:
determined by the Absorption
wavelength of light that
is reflected from the
object into our eyes

Refraction
• Achromatic colors occur
when light is reflected
equally across the
spectrum
• Transparent things ->
selective transmission:
only some wavelength
pass through them
• Selective reflection:
Chromatic colors occur when
some wavelenghts are
reflected more than others

)
 90

.....
I 40

I
Wllvelengll1 (llnl)
• Mixing paints vs. Mixing lights
– Mixing lights: additive color mixture
– Mixing paints: substractive color mixture

S M L S M L S M L

Medium +
long wave lengths

Short + medium + Blue paint Yellow paint Blue paint

long wavelengths + Yellow paint
We need different amount of light to see
different colors
'1800
'1600
'j' 1400
·E
- -

~
Q,
1200
cb - - - -
!E 1000
a),
Mesopic
~
0 800
- - -
c::
'E 600
::)

400
200
0 L..........,~-

380 420 4l6,0

1 soo 540 s.ao, 520
wavelength (nm),
5,5.0 100
 Trichromatic Theory of Color Vision
• = color vision depends on the activity of three different
receptor mechanisms (Young and Helmholtz)
• People with normal color vision need at least three
wavelengths to match any other wavelength
• Young-Helmholtz theory of color vision
– Color vision depends on three receptor mechanisms, each with
different spectral sensitivities
 Physiological evidence
• 3 different cone pigments
– Short-wavelength pigment (S)
– Middle-wavelength pigment (M)
– Long-wavelength pigment (L)
• Visual pigment = opsin + retinal
S L
M
 Roorda & Wiillinms, i1 999, Nature, 39 7 , 52[1
1

HL1ma111 cone mosaii1c in vivo. Co.loured for illus-tration on~y.

Da.ce1, 1993, J Neurosci, 13, 5334

Human retina in vitro. M

Cone photoreceptor
spectral sensitivities:

400 500 600 700

Wavelength (nm)
The eye ha5, 3 types of photorecep,tors:
s,ensitive to r,ed , green o,r b 11Lie light.

The bra.in t1ransforms RGIB into s.ep·arate

birig1ht11ess andl co la r cham1e s (e,. g.., LHS).
 Color Deficiency
• Partial loss of color perception, associated
with the receptors in the retina that are
present at birth
• Dalton → Daltonism
• Ishihara plates
– 3 types:
• Monochromatism
• Dichromatism
• trichromatism
Mutation or non-express·on o,f p,ho,t o,pig1ment prote·n
g,e nes c,a uses colour vision deficiency

1
(I)
/
rn- 00100
I
I
(1) I
0 I

-
:c
,m
0.
I,,.., g I
,0
(I.I
.c
,m I
'
I ~
\
(1) I

-c::
\
.2= I
,m I ' 12
(1) I \~
.; \ ,( P
\
\

0
''
4 00 500 600 700
Wavelength (nm)
ml• '
 Mlutation or los,s of photopigment proteins, 1

causes colour vis,ion deficiency

1 1 1 1 1

Normal tricbromacy Anomalous. trichrom acy Dichromacy Monochromacy

• Iack of disoli mi 11ation in • reduced colour visio11 •total co lour blind
1

oertain colour ranges • 1% of population (d' 2%) •vmy rare

•3% of popullation (d' 5.6%)

Protanomaly Protanopii. a
•L shiftedl towardls M

Deuteranomaly Deuteranopia S-. M-•. L-cone-

•M sh:i.fted towar&,. L monoc·hromacy

Tritanomaly Tritanopia Rod-monochromacy

• S shifted towards M ·J~ g • only scotopic?§>ion
·a ~~
P's ,e udoisochrom,a tic lshihara-,p,llates for th,e dia.gnosis o,f
r,e d-,gr,e en colour deficiency

Triichromat Protanope

S M IL SM

j]
S M IL SM

5f>
• Monochromatism
– Rare form of color-blindness (10 out of a million)
– Hereditary
– No functioning cones
– See everything in shades of lightness
– Sensitive to bright lights -> often wear dark glasses as protection
• Dichromatism
– Experience some colors
– 3 major form: protanopia, deuteranopia, tritanopia
– protanopia & deuteranopia are inherited through a gene located on
the X chromosome (sex-linked)
Trichromat IDichromat
(protanope: no L-cone)
• Protanopia
– Affects 1% males, 0,02% females
– Perceives short-wavelength light (blue); as the wavelength
is increased, the blue becomes less saturated, and finally
grey (=neutral point)
• Deuteranopia
– Affects 1% males, 0,01% females
– Perceives blue at short wavelength, sees yellow at long
wavelength
• Tritanopia
– Rare, affects 0,002% males, 0,001% females
– Perceives blue at short wavelength, red at long wavelength

(a) (b) (c ) (d )
 Protanope

400 t 700
492
(a)

Deutera11110,pe

700
498
(b)

Trltanope

400 t· 700
570
(c)
Trikt,omat Dikromat l\lonokromat

111 1 1
 Sensitivity
• Human: about 400-700 nm
• Birds, fish, insects: UVA
• Phintella vittata (spider): UVB too!

100 280 315 400 700

WAVELENGTH (nm)
Crab: 16 type of cones
 Opponent-Process Theory of Color Vision
• Ewald Hering

– color vision is caused by opposing responses generated by blue and

yellow and by red and green
• Hering:

– Viewing a green field generates a red afterimage

– Yellow field creates a blue afterimage

– The opposite is true as well

• Hering:
– People who are color-blind to red are also color-
blind to green
– People who can’t see blue also can’t see yellow

Red and green are paired

Blue and yellow are paired
• Physiological evidence
– Opponent neurons: retina, lateral genigulate
nucleus
• Respond with an excitatory response to light from one
part of the spectrum and with inhibitory response to
light from another part
 Color in the Cortex
• Semir Zeki: the idea of an area specialized for color
– V4 responses
• achromatopsia
↕
– Color perception results from activity in many different visual areas
that respond not only to color but to other qualities (such as form)
 Opponent Neurons
• Two types:
– Single-opponent neuron
• Important for perceiving the color within regions
– Double-opponent neuron
• Important for perceiving boundaries between colors
Single-opponent Double-opponent
receptive field receptive field

(a) (b)
 The Relation Between Color and Form
• The visual system determines an object’s form, and then color
fills in the form
• ↕
• Close connection between processing of form and color
• Color may play a role in determining form
Contract-effects
 Color constancy
• We perceive the colors of objects as being relatively constant
even under different illumination
– Chromatic adaptation: prolonged exposure to chrimatic colors
– Surroundings
– Memory color: the effect on perception of prior knowledge of the
typical colors of objects
 Lightness constancy
• The amount of light reaching the eye from an
object depends on:
– Illumination – the total amount of light that is
striking the object’s surface
– Object’s reflectance – the proportion of this light
that the object reflects into our eyes
• Lightness constancy: the perception of
lightness is determined by the object’s
reflectance (not the intensity of illumination!)
– Remains the same no matter what the illumination
F nw:::irn 1-1 ArtP1<:nn
 Look at blue Look at yellow
squares squares
• Content © 2008 R.Beau Lotto
• http://www.lottolab.org/articles/illusionsoflight.asp
• Content © 2008 R.Beau Lotto
• http://www.lottolab.org/articles/illusionsoflight.asp
• Content © 2008 R.Beau Lotto
• http://www.lottolab.org/articles/illusionsoflight.asp
• Content © 2008 R.Beau Lotto
• http://www.lottolab.org/articles/illusionsoflight.asp
• https://www.illusionsindex.org/i/grey-strawberries
„The dress”
Depth perception
Three sources of space information
1. Body
• Body surface localization
– Identifying the own body as spatial stimulus
• Proprioception
– Position of body parts compared to each other
• Kinesthesis
– Percieving the body in movement
2. EGOCENTRIC space
Percieving the space around us – the referential center is the own body

3. ALLOCENTRIC space
Percieving the space around us – in an independent spatial system
g
 Goodale & Milner (1992, TINS)

 Dorsal: Postertor Parietal

 Movement, localization
 egocentric
 Ventral:
 Object-recognition
- ~
Pnmary .
 allocentric visual
Ii..
Cortex 1

lnfeJtote ·- poral
Cortex
 „The inner GPS"

The first component of the brain’s localization system:

1971, John O'Keefe, animal experiments

If the rat was in a particular part of the labirynth, the same neuron group of it’s
hyppocampus has been active

Being in another part of the labirynth activated other type of neurons – „space cells”
for mapping our environment
The different type of visual stimuli used for
depth perception
= spatial cues
Oculomotor cues
Relying on muscles responsible for eye-movements
Oculomotor Oculomotor nerve ( II}

cues .I nferior aspect ,o f brain

1, Convergence

IJT----.!L Oc lomotor
2, Accommodation ( II) nerve

division
Clliaty
gangron

POSl'ERl'0R
Accomodation
• Increasing and decreasing
the lens’ refractive power
• Ciliary muscles
• Try it with your finger! Clo e "foe

·······• i••·
------
.. ■
.. ■ ■ ■ I ■ I ■
 ACCOMMODATION
• Definition: Accommodation is the mechanism by which the

eye changes its refractive power by altering the shape of the

lens in order to focus objects at variable distances.

Unaccommodated I Accommodated
lens ____,.___ lens

Ciliary muscle Ciliary muscle

relaxed ;:~_:,::_ r...!:Jc:::::~~::_ contracted
Zonular fibers _-:;:;:- ---~---; Zonular fibers
undertension.:. ~~~~§ ~~~~ relaxed

RELAXED ; ACCOMMODATED 4
Convergence
Convergence

(a) (b)
 1""11..._.,,....._ _ Panu:,~n!s
Convergence F ll:$ trJfr At·~a

• Crossing the axis of

the eyeballs to
maintain the focus
• Is usually connected
to accomodation
• Try it with your
finger!
Pictoral cues
Binocular and monocular information
Binocular cue
Binocular depth
Retinal disparity
• Difference between the two eye’s retinal image
Far finger Far ting:er
I
'\ I
I

'
\
\ J
I
\
\
\ J
I '
\
\
\ ear ting'e
I '
\
\
\

'

,ear finge

Far finge and

n.ear finger
For left eye, near Righ eye Left eye F righ ey,e ,
ting:er rov,e rs far closed closed bolh near and fa
finger fingers are visib le
LE RE
Binoculard Depth
2-D image o fla screen 3-0 infoomation on flat s,c een
Information – 3D
• differences in the images on
the left and right retinas
• the basis of the stereoscopic
vision

Left eye Right eye Left eye Right eye

image image image im age
Pemeplio:n = 2-0 Peroe ·o:n = 3-0
The longer the distance between two objects, the larger the difference:

.Dafrd
.Tulp

~
Horopter
• Imaginary line, where there is
no disparity
Monocular static cues
Occlusion
Occlusion
• The object that at least paritally
covers the other is closer than the
one that is covered (a} Eye and scene

• E.g. the tree and the house

• What if they are further apart and

we cannot rely on occlusion?
• No information on the actual
distance!

(b) Image ot scene- o:n r,etfna

Size
 Size
• Holway and Boring 1941
• Two circles with different
sizes
• Change distance from test
circle to match the size of
the other one
• What’s so intresting?
They coind the term
visual angle
Size constancy
• How tall is the first tree?
• How tall is the last one?
/jj 1
H~VE. NO ENSE: OF VEPiH PERC£Pi10N.
II COUU? '(OU TELL ME- IS 11-\AT SOMEONE..
ljl $TANV\NG WA'( UP 11-\~RE: ON THE C012NER>
I: MAN tN '(OUR 1--\Al ?

h'I
' ~
I

\?II ·ii
•I '
Perspective
convergence
Texture
gradient
Texture
gradient
• Perceived density
Atmospheric
perspective
Atmospheric
perspective
• Sharpness of the objects
• Light blue tint
Shadows
Shadows
false shadows…
 Monocular
non-static cues
Motion parallax
Motion-produced
cues
• Motion parallax
• Furhter objects seem to
„move” slower
• Deletion and accretion
• Occlusion due to
movement

►
Move

Posi .io Pos1tion 2 Pos ition 1 Position 2

Figure # 9.28 / book page 264
Flow field out car window

•
Fixation point

Your movement
https://www.youtube.com/watch?v=QWMFpxkGO_s
 Adelbert Ames, Jr. (August 19,
1880-1955) was an American
scientist who made contributions
to physics, physiology,
ophthalmology, psychology, and
philosophy. He pioneered the
study of physiological optics at
Dartmouth College, serving as a
research professor, then as
director of research in the
Dartmouth Eye Institute.
He conducted important research into aspects of binocular
vision, including cyclophoria and aniseikonia. Ames is
perhaps best known for constructing some illusions of visual
perception: the Ames room, the Ames window, and the
Ames chair.
Kinetic depth – Ames window

Ware, C., Chapter 8 of Information Visualization: Perception for Design. 2000, San Fancisco: Morgan Kaufmann.

The Ames-room
• https://www.youtube.com/watch?v=hCV2Ba5wrcs
• Az Ames szoba valós és
észlelt alaprajza
I

'' ''
I
\ ' l
\

Ap pa ren t po siti on ---------

of pe rso n A \ Actua l and
,, ap pa ren t pos itio n
\
\

'' , of pe rso n B
'' '' \
' \
' \ l

'' ' l
I
I
\ I
\ \
\ \
I

''I ,
\

'

Ap pa ren t
sha pe of' roo m
Impossible objects
Aniseikonia is an ocular condition where there is a
significant difference in the perceived size of images. It can
occur as an overall difference between the two eyes.
Gr. "an" = "not" , + "is(o)" = "equal," + "eikōn" = "image"
Strabismus (axis of the 2 eyes not paralel) too much difference
between the retinal images
Diplopia – the 2 retinal images are not merged
Taking action
Cognitive Psychology I.
Ecological validity

 = the experiment matches its stimuli, conditions and procedures to those

present in the natural world
 Ecological approach to perception (Gibson, 1950, 1962, 1972)
 Focusing on perception in its natural context
 Major goal:
 Determine how movement creates perceptual information that guides further movement and
helps observers perceive the environment
Optic flow
 Movement creates perceptual information
 Let’s imagine that as we moving forward everything around us is stationary
 Even though the objects aren’t moving, our movement will create the impression where the
objects around us are moving past as we look out the window
 = OPTIC FLOW
 (Motion parallax – depth and relative distance)
 Gradient of flow
 Optic flow is more rapid near to the observer
 The observer uses this information to determine his/her/their speed of movement
 Focus of expansion (FOE)
 There is no flow at the destination
 Invariant information
 Information that remains constant (Gradient of flow, FOE)
Optic flow
FOE

Gradient of flow
Self-produced information

 When the movement itself creates information

 And the information is used to guide further information
 Bardy & Laurent (1998)
 Gymnasts performed somersaults more poorly with their eyes closed
 BUT this wasn’t true for the novices
 Coordinate movements in line with their perception is a skill that novices have not yet learned
The senses do not work in isolation

 DEMONSTRATION
 Stand up. Raise one foot from the ground and stay balanced on the other. Then close your
eyes and notice what happens.
 Keeping balance
 Vestibular canals – inner ears + receptors in the joints and muscles
 + VISION provides a frame of reference that helps the muscles making adjustments
 Conflicting informations?
 Lee & Aronson (1974)
 13- to 16-month-old toddlers
 „swinging room”
  26% swayed
 23% staggered
 33% fell down
(a) Roofl'l s Floor re ns s·a iooary

 Adults?
 The vortex tunnel
 (Museum of Illusions,
Budapest)

(c) room s n s awa , pe,so sways cxw d to compensa e

Wayfinding
 How to find a destination that isn’t visible from our starting point?
 Percieving objects
 Remembering objects…
 …and their relation to the overall scene
 Landmarks = cues to indicate where to turn
 Hamid et al., 2010
 Decision-point landmarks
 Non-decision-point landmarks
 Description of the route (written/drawing) (Miller & Carlson, 2011)
 Exhibits located at decision-points were included more often
 Greater level of brain activity – parahippocampal gyrus (Janzen & van Turennout, 2004)
 Activity in the parahippocampal gyrus automatically highlights decision-points landmarks

 Topographical agnosia (inability to recognize landmarks) – loss of brain tissue in the parahippocampal gyrus
Individual differences in wayfinding

 The role of experience

 Maguire et al. (2006)
 London bus drivers – using specific routes
 London taxi drivers – many different places
 Taxi drivers recognized more landmarks in London
 Also their hippocampus was larger 2
 Experience dependent plasticity
~ .8

(a)
Cognitive map
 Edward Tolman
 Experiments with rats
 Latent learning
 The rat did not learn a sequence of moves, but created a cognitive map
of the spatial layout of the maze
 It was able to use this map to locate the food

C C C

,----------

A A
( ) plor maz (c) Turi" I for food
Cognitive map
 John O’Keefe
 Recorded the activity of individual neurons in the The Nobel Prize in
hippocampus Physiology or Medicine
 + noted the rat’s location 2014
 Individual neurons fired when the rat was in a specific
place within the box, different cells preferred different
locations
 PLACE CELLS = only fire when an animal is in a certain
place in the environment
 May-Britt and Edvard Moser Photo: David Bis op, Photo: Wikimedia Photo: Wikimedia

 GRID CELLS (entorhinal cortex)

UCL Commons CC Commons CC
John O'Keefe May-Britt Moser Edvard I. Moser

 Code distance and direction information Prize share: 1/ 2 Prize share: 1/4 Prize share: 1/4

 Humans? The Nobel Prize in Physiology or Medicine 2014 was divided,

one ha tf awarded to John O'Keefe, the other half jointly to
 Neurons similar to grid cells May~Britt Moser and Edvard I. Moser "for their discoveries of

 Noisier pattern cells that constitute a positioning system in the brain 0.

Affordances

= information that indicates how an object can be used

(Gibson, 1979)

 Perception includes not only physical properties (shape,

size, color, orientation) but also information about how the
object could be used
 Affordances go beyond simple recognition
 Temporal lobe demage – impaired ability of naming objects
 Indentifying objects faster based on functional info
 Using their knowledge of affordances (Humpreys & Riddoch, 2001)
 Affordances provide information on that how to interact with an
object, how to execute actions with a particular object
Mirror neurons

 „watching others act”

 Giacomo Rizzolatti
 Experimenters pick up food -> firing neurons in premotor cortex
 Mirror neurons = neurons that respond both when a monkey observes someone
elso grasping an object and when the monkey grasps the food
 Mostly independent from the grabbed object

s
l!
a,
C
"C
u:
(a) (b) {c)
Mirror neurons

 What about intention?

 Iacoboni et al., 2005
 Intention films caused greater activity in areas of the brain known to have mirror neuron
properties
 Mirror neuron area is involved with understanding intentions

0.7
,- Control film: Context Control film : Action Intention film
06 -
,....
__... Intention: Drinking
0.5
._, -
GI
en 0 -
i
s:.(,) ,...
'i! 0.3
c; -
SIi -
Cleaning up
(/)
,_
02

0
- I

....
0
Ac on Or nklng Cl nlng Cleaning up
Perceiving motion
Functions of motion perception

 Camuflage
 Color, shape, patterning are
similar to those found in the
sorrounding
 Movement organises all the
elements into a single figure
that is separated from the
background
 Motion attracts attention
 Attention is automatically drawn
to salient object
 Motion is salient
Understanding events
https://www.youtube.com/watc
h?v=VTNmLt7QX8E
 Heider & Simmel (1944)
 2 and 1/2 –minute
 „house” + three „characters”
 Subjects created stories to explain the
objects actions
Life without motion perception

 Akinetopsia = „motion blindness”

 Motion is either very difficult or impossible to perceive

 Case study
 the execution of daily activities is imapired
 E.g. pouring a cup of tea
 Following dialogues
 Crossing the street
Photo receptors

Akinetopsl

Edf\h ♦
n, Ell Facts.oro

https://www.youtube.com/watch?v=QV59aAtznSg
 Studying motion perception
•••
• •
 Real motion = actual motion of an object
 Illusory motion = perception of motion without Phi Phenomenon

o ".' w1eur •§§@f-

actual motion https://www.youtube.com/watch?v=-
 Apparent motion zbzt7Cb2e4

 Max Wertheimer https://www.youtube.com/watch?v=M5e0QO

 Two stimuli in slightly different location YI0j0&ab_channel=TseLab
 Alternated with the correct timing
 Induced motion
 when motion of one object causes a nearby stationary
object to appear to move
 Motion afteraffect https://www.youtube.com/watch?v=OAV
 Viewing a moving stimulus causes a stationary stimulus XHzAWS60
appear to move (waterfall – move upward) https://www.youtube.com/watch?v=oNh
cpOIQCNs
Real and apparent motion

 Larsen et al (2006)
 Control condition (simultaneous)
 Real motion display (back and forth movement)
 Apparent motion display (flashed one after another)

 Neural responses are similar for

real and apparent movement
__•II•- • I • ----► • I
(visual cortex)

(a) Contro
The movement area of the brain

 Striate cortex (occipital lobe – info from the retina)

 Hubel & Wiesel (1959, 1965) – complex cells
 Responds to bars that move in a specific direction
 Directinally sensitive cells
 Middle temporal (MT) area
 Newsome at al (1995)
 COHERENCE No correla ·on
Cohel'ence • o
50% correlation
Coherence a 50%
100% correlation
Co rence • l 00%

 As the coherence increased

 A, the judgements became more accurate
:.,......0

/' r /'~ °' 11\ ,I'

·1 11
 B, MT neurons fired more rapidly
0-
1 t 1 11
,✓ !'" ~11~ -1 y 11, 1 11
 MST – medial superior temporal area (a)
~ ~
(b)
l (c)

 Involved in eye movements

The movement area of the brain

 Human subjects
 Transcranial magnetic stimulation (TMS)
 Temporarily distrupts the normal functioning of
the neurons
 If a particular behavior is distrupted, the distrupted
area of the brain is involved in that behavior
 Applying TMS to the MT cortex
 Difficulty determining the direction in which the
dots was moving
 Temporary akinetopsia
Microstimulation
Percep Ion
.•-

 Stimulating neurons in the MT cortex

 Britten et al. 1992
 Stimulating neurons that are activated by
downward motion
 Instead of rightward motion the monkey (a) Nos lat1on

responds as though the dots were moving

•
downward and to the right Pc ccp Ion
••
 Microstimulation or leisoning of the MST
cortex
 Affects a monkey’s ability to reach for a moving
object
(t>) S1Jm.Jla1.JO
Motion of point-Light Walkers

 Motor cortex only activated by humanly possible movements

(a) ( (
Motion of point-Light Walkers

https://www.youtube.com/watch?v=
rEVB6kW9p6k

https://www.youtube.com/watch?v=
KT89CQ2nRpo

https://www.youtube.com/watch?v=
1F5ICP9SYLU

Johansson: Motion Perception part 1

" BioMotionlab Feliratkozas
258 eliratkoz6
Motion of Point-Light Walkers

 Motor cortex only activated by humanly possible movements

 If the person wears the lights stationary the lights look like a meaningless
pattern
 As the person starts moving the motion of the lights is perceived as a
walking person
 Self-produced motion of a person/living organism = biological motion
 Grossman & Blake, 2001
 Measuring brain activity while participants observing point-light walkers and scrambled
dots resembling point-light walkers
 Superior temporal sulcus (more active for biological motion)
 + FFS (fusiform face area), prefrontal cortex (mirror neurons)
 -> network of areas that together are specalized for the perception of biological motion
Motion of point-Light Walkers
• • •
 Grossman et al, 2005 •
• •
•
•
 Using TMS to distrupt the operation of the •• • • •
STS • • • •
 Biological motion + scrambled display a
•
• (b)
•
 Extra dots to create noise • • • • • •
•• ••
 Decide if a display was biological or • • •
scrambled motion • •••
•• •
• •• •
 Decreases the observers’ ability to perceive • •
biological motion • • • • • •
 Stimulating other motion sensitive areas (e.g.
MT) had no effect
(c •
• • •
• ) .
•• • • • •
(a) B
Perceiving objects and scenes
Textbook: pp. 93-116.
Perceptual organization
= the process by which elements in the
environment become perceptually grouped to
create our perception of an object
 Incoming stimulation is organized into
coherent units
 Involves two components
 GROUPING
 Process by which visual events are „put together” into
units/objects
 https://www.youtube.com/watch?v=VKTj2jsGG3k
 https://michaelbach.de/ot/cog-Dalmatian/
 SEGREGATION
 Separating one area or object from another
 .

-- . ,- -
.
,
~

~ . ~,~,,' -~ <

· -1 •
•• •
-
- - - ;J
-.:- -~ •
•

-. -
~

~ ,•,,
.. .,,,,. --
:lit~
-. ,-.
,
.... ' ~

.
.... -.

,,
,
_,,....
-~
~
• - _, ~
.
 .

--

_,,....
-~
~
• -
..
_, ~
.
Structuralism
 Wilhelm Wundt
 Sensation vs perception
 Sensations
 Response to stimulation
 Analogues to the atoms of chemistry Structuralists:
 Perceptions Number of sensations add up to create
 More complex conscious experiences such as our our perception of the face
awerness of objects
 Combine sensations to create complex
perceptions
Structuralism
 Wilhelm Wundt
 Sensation vs perception •: I

 Sensations
 Response to stimulation Structuralists:
Number of sensations add up to create
 Analogues to the atoms of chemistry
our perception of the face
 Perceptions
 More complex conscious experiences such as our
awerness of objects
 Combine sensations to create complex
perceptions

Gestalt psychologists
 Gestalt
 Wertheimer (a) One light flashes
 Toy stroboscope
 Creating an illusion of movement by rapidly alternating
two slightly different pictures
 This cannot be explained by the structuralist point of (b) Darkness
view
 Apparent movement
 Illusory movement
 Can’t be explained by sensations (c) The second light
 Perceptual system creates the perception of flashes
movement where there actually is none
 „The whole is different than the sum of its parts”
 Illusory contour
 No physical edges are present (d) Flash-dark-flash
 It cannot be explained by sensation since there aren’t
any sensations along the contours
 Georges Seurat
Pointillism
Pointillism is a technique of painting in which small,
distinct dots of color are applied in patterns to form an
image.
 Principles of perceptual organization
 Good continuation How continuation affects us?

 Straight or smoothly curving lines w nd no lik this:

are seen as belonging together, and

the lines tend to be seen in such a
way as to follow the smoothest path
Principles of perceptual organization

•••••
• • • ••
• • • ••
• • • ••
• • • ••
• • • ••
How do you perceive this?
Principles of perceptual organization
 Similarity
• • • ••
 Similar things appear to be grouped
together • • • ••
 Grouping can also occur because of
similarity of shape, size, or orientation • • • ••
• • • ••
• • • ••
• • • ••
How do you perceive this?
-■ ♦ ■ -
-■ --♦
Principles of perceptual organization
 Proximity (nearness)
•••••
 Things that are near each other
appear to be grouped together • • • ••
• • • ••
• • • ••
• • • ••
• • • ••
How do you perceive this?
Principles of perceptual organization
 Proximity (nearness)
•• • ••
 Things that are near each other
appear to be grouped together •• • ••
•• • ••
•• • ••
•• • ••
•• • ••
How do you perceive this?
Principles of perceptual organization
 Proximity (nearness)
• •• ••
 Things that are near each other
appear to be grouped together • •• ••
• •• ••
• •• ••
- -- - - • •• ••
- -- -- -- - ♦
-
• •• ••
- -- -- How do you perceive this?
Principles of perceptual organization
 Common fate
 Things that are moving in the same direction appear
to be grouped together
 E.g., flock of hundred of birds (1 group/unit) -> some
of them start flying in another direction (2
groups/unit)
 Works with dissimilar objects too
 https://www.youtube.com/watch?v=LZMaTtPHBMk
\
~~,

'\~,~
\' '~

),.;!~~
,~
~
...,
••••••••••
• • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
•••••••••• • • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
• • • •• • • • ••
Perceptual segregation
 Figure-ground segregation
 Edgar Rubin, 1915
 Figures are
 More thing like and more memorable
 Seen as being in front of the ground
 If the vase is seen as a figure it appears to be in
front the white background
 Border separating the figure from the ground
appears to belong to the figure
Image based factors that determine which
area is the figure
 Areas lower in the fiels of view are more The typical scene that we percieve regularly
likely to be perceived as figure (Ehrenstein, 1930;
Koffka, 1935)
 Vecera et al., 2002
 Upper-lower preference but no left-right
preference
 Figures are more likely to be
perceived on the convex side of
borders (bulge outward) (Kanizsa &
Gerbino, 1976)

 Peterson & Salvagio, 2008

 Whether the red square was „on” or
„off” a perceived figure
 Black area - 89%
 Two component display – 58%
 Segregation is determined by the
wider scene
 Gestalt psychologist: Segregation of the figure
from the ground precedes the recognition of an
object
 Gibson & Peterson (1994)
 Meaningfulness can influence the assignment of an area
as figure
 Recognition must be occurring either before or at the
same time as the figure is being separated from the
ground
 Perceiving the gist of a scene
 Objects are compact and are acted upon
 Scenes are extended in space and are acted within
 Large and complex
 Gist of a scene = general description of the type of scene
 We can identify important properties by viewing a scene only for a fraction of a second
 Oliva & Torralba (2001, 2006) – GLOBAL IMAGE FEATURES
 Holistic, rapidly perceived and they are associated with specific types of scenes
 Information about the structure and spatial layout of a scene
 Degree of naturalness
 Degree of openness
 Degree of roughness
 Degree of expansion
 Color
Scene schema A

 Palmer, 1975 B

 Context scene (e.g., kitchen)

 Briefly flashed target picture C

 Oliva & Torralba, 2007

 The multiple personalities of a blob

blob

(a) (b)

(c) (d)
Neural background
 FFA – fusiform face area
 PPA – parahippocampal place area
 EBA – extrastriate body area
 Binocular rivalry
 Tong et al., 1998
 Colored glasses (house or face)
 Binocular rivalry occured
 Images remained the same on the retina
 The perception of the images alternated back and forth every few seconds
 fMRI – brain activity changed depending on what the person was experiencing
PPA – Parahippocampal place area
 Epstein and Kanwisher (1998)
 „place area”
 It responded to pictures of buildings, furnished and empty rooms
 Parahippocampal cortex (PHC)
 Identifies the location without any statements regarding its function
 Spatial layout hypothesis
 Responds to the surface geometry/geometric layout of a scene
 Scenes ccause larger responses than buildings
 But buildings are partial scenes
 PHC responds to to qualities that are relevant to navigation through a scene or locating
a place (Epstein, 2008)
 Mullay & Maguire, 2011 – (sense of three dimensional space)
 Space defining (large oak bed)
 Space ambigous objects (small white fan heater)
Visual agnosia
 Impairment of visual object recognition with otherwise intact vision
 Two types of visual agnosia
 Apperceptive agnosia
 Individuals’ knowledge of the objects is intact but they are unable to organize/integrate
the different pieces of information into a coherent whole/object.
 Associative agnosia
 Characterised by fairly normal perceptual processing but an impaired ability to derive
the meaning of objects
 Prospagnosia
Mental Imagery
The Oxford Handbook of Cognitive Neuroscience
▪ When we think of
▪ ….the best way to load a luggage in the trunk of a car
▪ …the fastest route to go from point A to point B
▪ …easiest way to assemble a bookshelf
we visualize actions/use mental images instead of performing them

▪ Mental images are representations that are similar to perceptions but they
do not require an external stimulation to be created
▪ Representations preserve the perceptible properties of the stimuli they
represent (Kosslyn et al., 2006)
▪ The stimulus itself is not present physically
▪ Relies on top-down processing
Early studies on mental imagery

▪ Plato – memories are like stamp images created from wax

▪ Private and introspective nature
▪ Mental representations cannot be observed directly
▪ Behaviorist (Watson, 1913) deny the existence of mental images (black box)
▪ This had a long-lasting negative impact on the research of mental images
▪ 70s’ – mental imagery is a key part of memory (Paivio, 1971), problem solving, and
creativity
▪ Shepard and Metzler (1971) – mental rotation paradigm
▪ 2 3D objects with several arms
▪ Decide whether the two objects had the same shape or not
▪ Response time increased linearly with increasing angular disparity between the two objects
▪ The linear increase in RTs associated with the increasing rotation angles reflects the cognitive manipulation
Demonstration
(Kosslyn, 1975)
Rabbit beside an elephant Rabbit beside a fly
„Imagery debate”
▪ Analog/propositional debate

Steven Kosslyn Zenon Pylyshyn

 „Imagery debate”
▪ Analog/propositional debate
t
▪ Mental images are deciptive representations
▪ Information about a mental image is stored in an
analog code = pictorial representation
▪ (1) each part of the representation corresponds to a
part of the represented object
▪ (2) distances between representations of the parts (in a
Steven Kosslyn representational space) correspond to the to the
distances between the parts on the object itself
▪ Analogy between the real object and the mental image
▪ Mental imagery is a relative of perception
▪ Eg. Mental image of a triangle is registered in a similar
fashion, preserving the same relationship among the lines
Kosslyn, Ball and Reiser
"
"
... .
11

"

(1978)
13

l 'I 6 ~ IC 12 1" 16 lti

0 1a11 mc~((m)

▪ A drawn map of an island

▪ 7 different objects (such as tree, rock,
bungalow etc.)
▪ 5 cm distance between the objects

▪ Participants memorized the map

▪ Walk around the island in their imagination and
hit a button when they reach the target
▪ The imaginary scanning time increases linearly
with the distance
▪ „Image scanning paradigm”
Kosslyn, Ball and Reiser
(1978)
▪ A drawn map of an island
▪ 7 different objects (such as tree, rock,
bungalow etc.)
▪ 5 cm distance between the objects

▪ Participants memorized the map

▪ Walk around the island in their imagination and
hit a button when they reach the target
▪ The imaginary scanning time increases linearly
with the distance
▪ „Image scanning paradigm”
„Imagery debate”
▪ Analog/propositional debate

Steven Kosslyn Zenon Pylyshyn

„Imagery debate”
▪ Analog/propositional debate

Zenon Pylyshyn

▪ We store images in terms of a propositional code

▪ We do not rely on visuo-spatial mental images, but instead on descriptive
representations
▪ Propositional code = descriptive representation: is an abstract, language-
like representation
▪ Storage is neither visual nor spatial, does not resemble the original
stimulus
▪ Mental imagery is a close relative of language (not perception)
▪ Propositional Representation
▪ Rocking chair left of couch
▪ Candy in the candy dish
▪ Rug below coffee table
▪ Window behind couch

▪ Analog Representation
How to resolve the imagery debate?
▪ Neuroimaging data
▪ If visual imagery relies on representations and cognitive processes
similar to those involved in during visual perception, then visual mental
imagery should rely on the same brain areas that support visual
perception (Kosslyn, 1994)
▪ fMRI or PET -> prove correlation
▪ TMS or lesions -> information about causation
▪ Neural substrates of visual imagery are similar to those of visual
perception
▪ Visual cortex is widely activated during imagery
▪ Positive correlation between visual cortex activity and vividness of
imagery (Amedi et al., 2005)
Neural background of visual mental
imagery (Pearson, 2019)
▪ Voluntary mental imagery is based on combinations of
information retrieved from stored memory
1

▪ It is difficult to imagine content you have never been

exposed to
)
▪ Frontal areas play more of a general organizational role in
coordinating spatial and sensory areas, rather than holding
---
imagery representations or content
▪ Hippocampus probably involved in spatial and memory elements
of imagery
▪ Visual cortex (V1, V2) fig. I A top down general model of voluntary mental
imagery: a reverse hierarchy. The · · ial m ntal ac ·onto
▪ Main body of research c ate a mental ima e' ins'h• in he cortical proces.sin
hierard,y. inthefronta cortex (step ~ This ·ggersa
▪ Controversial
cascade of neura e n s. run ·ng 'ba wanfs'. that ne
▪ There are patterns of activity common to perception and imagery that emerge re rieves stored informa ion or- memori s from rn<>f"e
as early as V1 pos erior re tons such as m a temporal areas (step 2) •
and sensory and spatial representations o the · ry
▪ Ventral (what) and dorsal (where) streams content ar thenformedt eps 3). f the 'reque ed'
represen a ion involves mo men and spatial locations
▪ VP damage – object perception and the ability to visualize shapes are impaired hen other areas like the mfdd e temporal area and parie al
lobes would also be involved.
Graphical depiction of the cognitive
processes related to mental imagery
Prospective Eye witness
memory memory

Arithmetic
False memory

avigation

Moral decisions

Sports

Creativity
Motor contro l

Reading Mind wandering

comprehension
VIVIDNESS OF VISUAL IMAGERY
QUESTIONNAIRE (VVIQ)
▪ For each item on this questionnaire, try to form a visual image, and consider your
experience carefully. For any image that you do experience, rate how vivid it is using the
five-point scale described below. If you do not have a visual image, rate vividness as ‘1’.
Only use ‘5’ for images that are truly as lively and vivid as real seeing. Please note that
there are no right or wrong answers to the questions, and that it is not necessarily
desirable to experience imagery or, if you do, to have more vivid imagery.

▪ Perfecting clear and lively as real seeing 5

▪ Clear and lively 4
▪ Moderately clear and lively 3
▪ Dim and vague; flat 2
▪ No image at all, you only “know” that you are thinking of the object 1
▪ For items 1-4, think of some relative or friend whom you
frequently see (but who is not with you at present) and consider
carefully the picture that comes before your mind’s eye.

▪ 1. The exact contour of face, head, shoulders and body

▪ 2. Characteristic poses of head, attitudes of body etc.
▪ 3. The precise carriage, length of step etc., in walking
▪ 4. The different colours worn in some familiar clothes
▪ Visualise a rising sun. Consider carefully the picture that comes
before your mind’s eye.

▪ 5. The sun rising above the horizon into a hazy sky

▪ 6. The sky clears and surrounds the sun with blueness
▪ 7. Clouds. A storm blows up with flashes of lightning
▪ 8. A rainbow appears
▪ Think of the front of a shop which you often go to. Consider the
picture that comes before your mind’s eye.

▪ 9. The overall appearance of the shop from the opposite side of the road
▪ 10. A window display including colours, shapes and details Of individual items for
sale
▪ 11. You are near the entrance. The colour, shape and details of the door.
▪ 12. You enter the shop and go to the counter. The counter Assistant serves you.
Money changes hands.
▪ Finally think of a country scene which involves trees, mountains
and a lake. Consider the picture that comes before your mind’s
eye.

▪ 13. The contours of the landscape

▪ 14. The colour and shape of the trees
▪ 15. the colour and shape of the lake
▪ 16. A strong wind blows on the trees and on the lake causing reflections in the water.
Aphantasia
https://www.youtube.com/watch?v=Xa84hA3OsHU
▪ Inability to experience voluntary object imagery (Zeman et al., 2015)
▪ Cannot form depictive objects representations
▪ Case of M.X.
▪ MRT – He could solve the task 30
▪ But showed a non-linear pattern of Rts :,,.

▪ Different strategy?
20 'O
:::r
A>
a
A>

▪ match individual cubes and angles perceptually before 10

!!?.
A>

responding >,
<.>
C:
~
▪ More than poor metacognition of visual imagery
Cl>
~ 0 0
g' 30 C:
... ,:s

▪ Scored lower than controls on all imagery vividness II.

20 0

▪ BUT: spatial imagery is intact 0

...0
J

▪ Dorsal pathway 10

▪ Individuals with aphantasia can still perform many daily tasks by 0

utilizing different strategies (e.g. semantic encoding) 16 24 32 40 48 56 64 72 80

▪ Still unclear WIQ

Hearing and Speech
„If a tree falls in the forest and no one
is there to hear it, is there a sound?”
„If a tree falls in the forest and no one is there to
hear it, is there a sound?”
▪ Sound
▪ Physical stimulus
▪ Sound is pressure changes in the air or other medium (such as
water)
▪ Perceptual response
▪ Sound is the experience we have when we hear

YES NO
• Physical definition • Perceptual definition
• The falling tree causes pressure • If no one is in the forest there will
changes be no experience
• Sound • Sound perception
• „the sound had a frequency of • „the piercing sound of the trumpet
1000 Hz” filled the room”
 The sound 1gh Perception:
Hig Pitch

▪ Frequency: the number of times per second that

a sound pressure wave repeats itself
▪ Measured in units called Hertz (hz) Perception:

▪ 1 Hz = 1 cycle per second

Low LowP1tcn

- - - - - 1/100 second - - - - --i

▪ Most sensitive at frequencies between 2000-4000Hz Figure 11.3 Three d1 erent frequencies o a pure tone. Higher
frequencies are assoc ted w1 h the perception of high r pitches

(speech!)
▪ https://www.youtube.com/watch?v=XLfQpv2ZRPU
▪ Human frequency range: 20Hz to 20000Hz
▪ Changes with age
▪ https://www.youtube.com/watch?v=VxcbppCX6Rk
How Old Are You r Ears? {Hearing Test)

Pitch = perceptual quality •

AsapSCIENCE &
10,4Mfeliratkoz0
4$1!
"'1111•-
jfo$
1 ■ IP
a'.) 126 E i;:Jl A> Megosztas

we describe as high or low

-
Understanding Sound Waves I MED- EL

- ~~~
 Pitch
https://virtualpiano.net/

▪ Psychological and not a physiological property of a sound

▪ Based on that how we perceive a sound
▪ Tone height

- Tones separated by octaves

have the same tone chroma
- Notes with the same tone
chroma have fundamental 2 3 4 5 6 7 8 9 0 q w e y u 0 p a s d g h k z X C V b n m

frequency separated by a
multiple of two
- E.g., A1 = 27.5Hz, A2 = 55Hz,
A3 = 110Hz t
Low pitch
C2 – 65.4 Hz
High pitch
C7 –
t
- Similar perceptual experiences approx. 2000 Hz
Amusia
▪ „tone deafness”
▪ Inability to determine the direction of the pitch change
▪ Impaired identification of pitch direction but not basic pitch
discrimination
▪ Right temporal lobe
▪ Decreased volume of the superior temporal arcuate fasciculus in the
right hemisphere
▪ Connects the temporal and frontal lobe
 R I iv Arnpli u nd D
fo E ironmental Sounds

SO D ITlJO LSIDB

Barely aud ible 0

Oh s old)
▪ Amplitude - relative strength of sound Le s rus , g 10 20

waves, which we perceive as loudness Qu i

or volume. Amplitude is measured in

communi 100 40
kin oic ,000 0

decibels (dB), which refer to the sound E pre s u wa r in 00,000 00

pressure level or intensity Prop II r pl n

t eoff
t
1,000,000 20

10,000,000 0

P rcep 10n:
Louder

rce 10n:
Quieter Louder Low So

Time
't J
Figure 11.4 Three d. r nt amphrudes of a pu e one Lar r
amph ud 1s associated w, the percep on of greater loudness.
Timbre --
m
'V
20
Guitar

https://www.youtube.com/watch?v=VRAXK4QKJ1Q I
C o
&.
•
a:• 0 200

▪ The quality that distinguishes between Frequency (Hz)

two tones that have the same

Bassoo
30
m
loudness, pitch and duration, but still -
'V

C
20

sounds different
&_ 10
c!

▪ Flute (clear) vs Oboe (reedy)

0 200 00
Frequency (Hz)

▪ G3 – 196Hz Alto saxophone

20
C
&. 0
i
o 200 0,000
Frequ ncy (Hz)

Figure 11. 10 Freq ency spec ra or a ui ar, a bassoon, and an

alto sa ophone playi g a one wi h a f undamen al frequency o
196 Hz The position o the Imes on he ho tZOntal axis mdicates
the frequencies of the harmonics and eir heig t ,nd,cates heir
n ens, es.
Standard: 440Hz (A)
Produces a very pure tone
Mainly the fundamental
frequency
 Organ of Corti
Tectorial

Hearing loss
membrane

Cells of

▪ Hearing loss is usually associated

with damage of the hair cells
I
▪ Presbycusis
Nerve
claudius fibres
boettcher Deiters membrane
Tunnel of
▪ Caused by hair cell damage resulting Corti

from the cumulative effects over time -::

of noise exposure and age-related
•,•

.......
1::
,
..
,....
•!.~

degeneration ~::
,:! Malle us lncus

▪ The loss of sensitivity is greatest at ·.·.,, .

!:!•,
.
\' Scala vestibuli

high frequencies
\

' .'.,
,,. ,,,
. ': .'. ,',''_.:'-·--~◄"/

Organ of Corti
Basilar Scala
Sound waves Auditory canal Tympanic Round membrane tympani
membrane window
 Noise-Induced hearing loss
▪ Loud noises cause degeneration of the hair cells
▪ United States Occupational Safety and Health
Agency
▪ Workers cannot be exposed to sound levels greater
then 85 dBs for an 8-hour work shift
▪ Leisure noises
▪ Listening to music (headphones), riding motorcycles,
playing musical instruments (in a band), concerts,
sport events
▪ Depends on the intensity and the length of the
exposure
▪ Ear protection/turn down the volume
Hidden hearing loss (Plack et al., 2014)
▪ Normal performance on a standard hearing test
▪ Measuring tresholds for hearing tones across the frequency
spectrum
▪ Quiet room
▪ Indicate when you hear a tone
▪ Difficulties in a noisy environment
Speech
▪ Basic unit of speech = PHONEME
▪ = shortest segment of speech that, if changed, would change
the meaning of a word
▪ E.g., BIT/PIT/BAT
▪ Refers not to letters but to speech sounds that determine the meaning
of what people say
Multimodal perception
https://www.youtube.com/watch?v=2k8fHR9jKVM
Multimodal perception
https://www.youtube.com/watch?v=2k8fHR9jKVM

▪ Perception of speech can be influenced

by information from a number of
Perception

8
different senses
▪ McGurk effect (McGurk & MacDonald, 1976)
▪ Audiovisual speech perception
▪ In a noisy environment we use information
provided by the speaker’s lip movements
Figure 13.9 Th McGu k ect. wo e 1s saym /ba-oa/.
u her lip ov m n s co respand a/ a ta/, so h liste r r po s
arin /fa-fa/.
The role of meaning
▪ Easier to perceive phonemes that appear in a meaningful
context (Rubin et al., 1976)
▪ Leg, bat, sin (580ms)
▪ Jum, baf, teg (631ms)
▪ Respond by pressing a key as rapidly as they can when you hear a
sound that begin with ‚b’
▪ Phonemic restoration effect
▪ https://www.youtube.com/watch?v=pF1IfcfHKD4
▪ https://www.youtube.com/watch?v=fLPQZgYMJ7Q
▪ „The state governors met with their respective legislatures convening in the
capital city”
▪ Bottom-up (nature of the acoustic signal)
▪ Top-down (context that produces expectations in the listener)
Speech segmentation
▪ Foreign language ~ „unbroken string”
▪ Perception of words is not based only on the energy stimulating
the receptors
▪ Knowledge of the meaning of words
▪ Transitional probabilities
...... t&te Sound ...,for . .
\Wfdl·······•'•..... -Nob
that lt II c11Jiajt 1D tel fRlffl 11118
........... Mid ...... . .
Oll•llagia Spa■• .... ......
of u.. Sandin.

s p BG T A IO N
▪Read the following words: „Anna Mary
Candy Lights Since Imp Pulp Lay Things”

▪ Change the perceptual organization of the sounds

▪ The change is achieved by our knowledge of the meaning
of the sounds
▪ „Be a big girl and eat your vegetables”
▪ „The thing Big Earl loved most in the world was his car”
▪ „I scream, you scream, we all scream, for ice cream”

▪ The different organizations are achieved by the meaning of

the sentence in which these words appear
Transitional probabilities
▪ The chances one sound will follow another sound
▪ pretty baby – pre ttyba by
▪ Learned by statistical learning
▪ Highlighting patterns/structure
▪ By observation
▪ Infants as younf as 8 months of age are capable of statistical learning
(Saffran et al., 1996)
 Perceiving degraded speech
nowl d I
meaning

▪ Top-down vs bottom-up processing op-do

▪ I scream vs Ice cream

▪ The context specify the meaning S ec
percepoon

▪ We use top-down processing to adapt adverse

conditions Bot om-up

▪ E.g., background noise, poor room acoustic, cell AcoustJc si

phones… Figure 3. 14 Soeedl perce

roe ssin f s do nowl

▪ Davis et al., processing f se o

▪ Degraded sentence -> clear sentence -> degraded sentence

▪ Improved perfromance at the second time (knowledge)
▪ This effect was observed for sentences heard at the first time
▪ Both listening and reading -> KONOWING THE CONTENT
Neural background
▪ Kriegstein et al., 2005
▪ fMRI
▪ Sentences spoken by unfamiliar and familiar speakers
▪ Just listening to speech
▪ STS (Superior temporal sulcus), area associated with speech perception (Belin et
al., 2000)
▪ Carry out tasks that involved paying attention to the sounds of familiar
voices
▪ FFA
▪ For unfamiliar faces no FFA activation was observed
Broca’s aphasia
▪ Damage in the frontal lobe, Broca’s area ->
▪ Speech is slow, labored, jumble sentence structure,
ungrammatical speech
Figur 13. 15 Broca·s a Werni e's areas Broca·s area is in e
▪ More than the deficit in speech production ron al lobe, and Nem,c e's 1s 1n e t mporal lobe.

lll
▪ „The apple was eaten by the girl”
▪ „The boy was pushed by the girl”
▪ Difficulty with the second sentence –
wether the girl pushed the boy or the boy
▪ Problems in processing the structure pushed the girl?
of sentences
▪ Cognitive control deficit during the use of
language
Wernicke’s aphasia
▪ Werncike’s area, temporal lobe
▪ The produced speech is grammatically correct and fluent but
incoherent
▪ Produce meaningless speech +
unable to understand
11111 t:'DIII wtb,oul lmOII'. sentences and writing
IIWtll!ClaadllOW
▪ More widespread difficulties in
myamLUlll-
understanding compared to
DOW
p. )
Broca
▪ Broca – hardships with sentences where the
meaning depends on the context

▪ „word deafness”
 TO TE T - hort rion
am : - - - - - - - - - - - - , - - - - - - - ; - - - - - . - : - . - - - - - - - - -
rlirth: _ _ _ _ _ _ _ _ _ _ ,ar~ ncl months.
Schooling: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Oatc: _ _ _ _ __
E1t1mia n _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __

Part I ( LI tokens)
I • Touch a ircl .
2- Touch a. quare .
. Touch a yel l , 1 en
4 - Touch ,, r.:d C)nc .
. Touch u black one.
o- I ouch a gre n one.
7-Touch a whit· ond .
l'art 2 (Only I• rg t ken )
. Touch the yellow quarc.
9. Touch the blzck circle.
I 1he n 1rcle.
I 1he "hiI quarc.
Part 3 ( II
I • • le.

□□
I quarc.
I uarc re n.
I le.
Part 4(0n ly
lo
1
e .-.xi :rnd 1he gre,•11
qwre ::ind the black
uar • and the gl'Ct'fl
re.
•
0
I
Part - ( 11 lokens)
rck and red ci re le.

2 • Touch 1he lar e whi1e circle :md the ~m:il n . qu:i re

o e eeo
■ ■ □□ ■
21- Touch 1he. ircle and 1he larg , square.
h 1h.: lu qu re aEd the la qudre.
qu re and the ma II green irel e.
Part 6 0
nn Ih m: ·n • 1m1n:,
ire le with lhc red quarc.
e e e oo
· i1 ·]~ tJ • .
27-To irde
l'lll re a, are.
1ft rc le. • h are.
Put arc 11
T u h 1h • :;qua res :;lo, , n ui.,; I
- ~ ~ ~ cik ~~d~m u~
~'-Tu h 11th •cird l .
- Touch 1hc red ci rc qwuc .
• In t ·ad of die white u..rc, Touch the yd lo circ le.
• In atldi 1io11 to 1ovchi11" 1l1 yello," dn.:11.:, o ·h b · ·k 1:ir ·I .
• mre:
Emotion regulation and music
Mood-congruency principle in music selection behaviour in
everyday-life situations
Brief Music in Mood Regulation
Scale (B-MMR) (Saarakiallio, 2012)
The two emotion dimensions of ‘‘valence’’ and ‘‘arousal’’
explain a high amount of variance in selecting music in
emotional situations

This situation-specific selection of different emotionally

laden pieces of music is a clear indicator of the attempt of
the listener seeking out individually fitting music for the
support, control, or change of the current state of
experienced emotions

Selection of positively evaluated music in situations

characterised by negative valence and high arousal is
associated with emotional moderating regulation (Thoma et
al., 2012)
Reviews

The human imagination: the cognitive

neuroscience of visual mental imagery
Joel Pearson
Abstract | Mental imagery can be advantageous, unnecessary and even clinically disruptive.
With methodological constraints now overcome, research has shown that visual imagery
involves a network of brain areas from the frontal cortex to sensory areas, overlapping with
the default mode network , and can function much like a weak version of afferent perception.
Imagery vividness and strength range from completely absent (aphantasia) to photo-​like
(hyperphantasia). Both the anatomy and function of the primary visual cortex are related
to visual imagery. The use of imagery as a tool has been linked to many compound cognitive
processes and imagery plays both symptomatic and mechanistic roles in neurological and
mental disorders and treatments.

What is the human imagination? What is this amazing individuals can experience vivid colour without colour
ability, which most of us have, that allows us to travel information stimulating the retina2. In post-​traumatic
through space and time, testing out different virtual stress disorder (PTSD), individuals experience flash­
worlds, objects, foods, fears and pleasures? When we backs or vivid, intrusive memories of trauma expe­
think about the sensory characteristics of something rienced as involuntary imagery 3. Such conscious,
like an apple or a sunset, most of us have a conscious involuntary sensory experiences, without a direct cor­
visual experience of those things. We literally become responding sensory input, have elsewhere been dubbed
conscious of some version of the apple or sunset, albeit ‘phantom perceptions’2. One proposed overarching
a degraded, fuzzy or weak experience. framework is that internal experiences like imagery
Mental imagery has played a central role in discus­ can be divided into two types of imagery-​like experi­
sions of mental function for thousands of years, first by ences, where one is voluntary and the other involuntary.
philosophers, then by psychologists, and now by neuro­ These two categories are analogous to the subtypes of
scientists. Almost any behaviour or cognitive process attention — endogenous and exogenous4 — and, as in
that might gain from sensory simulation tends to utilize the early days of research into attention, current work
mental images. is just beginning to shed light on the commonalities
However, to a minority of individuals, the idea that and differences between these two forms of phantom
people can have a voluntary conscious sensory experi­ perception2.
ence in their mind’s eye comes as a complete surprise; Importantly, mental imagery covers all five senses;
these individuals lack the ability to voluntarily form however, visual mental imagery research has tended to
mental images: aphantasia1. On the other side of the dominate, as is also the case with perception research.
spectrum, strong imagery plays a core role in many Focusing research on one faculty can make the endeav­
anxiety disorders, depression, schizophrenia and our more tractable and studying vision has several
Parkinson disease, and is increasingly being harnessed advantages over studying the other senses. Humans
as a uniquely powerful tool for psychological treatment. are visual creatures; this is clear when we look at the
Mapping out imagery’s seemingly contradictory contri­ proportion of cortical tissue assigned to processing
butions to human cognition, whereby imagery can be visual information as compared to the other senses.
both advantageous and clinically disruptive, or even Additionally, visual perception is vivid and full of
possibly unnecessary (aphantasia), offers exciting novel detailed information, often making visual stimuli
insights into the human mind. informative tools, as is the case with many illusions that
School of Psychology, When people talk about the mind’s eye, they typi­ induce forms of involuntary phantom vision. However,
The University of New South
Wales, Sydney, Australia.
cally refer to the voluntary experience of creating a it is worth noting the limitations this focus on vision
conscious sensory experience at will. However, there are has on our overall understanding of mental imagery
e-​mail: jpearson@
unsw.edu.au many examples of involuntary sensory experiences that and it is perhaps too often assumed that findings
https://doi.org/10.1038/ are equally decoupled from direct sensory input. For from visual imagery will naturally generalize to the
s41583-019-0202-9 example, in synaesthesia and in many visual illusions, other senses.

624 | OCTOBER 2019 | volume 20 www.nature.com/nrn

 Reviews

Reverse directionality
Neural mechanisms of visual imagery Due largely to the imagery debate (Box 1), the major­
The reverse direction of neural Mental imagery involves activity across a large neural ity of neural imaging work on mental imagery has
information flow, for example, network spanning frontal areas right back to primary focused on early visual areas and the overlap or similar­
from the top-​down, as opposed sensory areas. The neuroscience of imagery can be ities between visual imagery and visual perception. As
to the bottom-​up.
divided into work looking at the mechanisms involved such, not much brain imaging work has investigated this
in triggering the imagery process, mechanisms of gener­ systemic model of voluntary imagery in operation as a
ation or manipulation, as well as mechanisms underlying whole, though evidence from various different studies
the strength and vividness of visual imagery content and does appear to support it.
its overlap with sensory perception. However, imagining
an apple, doing a mental rotation task or imagining per­ Frontal areas. Forming or manipulating a mental image
forming a sporting activity are three very different tasks does appear to involve activity in the frontal areas7–10.
that too often get referred to simply as imagery. Much However, the activity in frontal areas seems largely inde­
work has been done investigating visual imagery in pendent of the precise content of the imagery9–11. This
particular, and the role of the primary visual cortex suggests that frontal areas play more of a general organ­
(V1), for reasons relating to the imagery debate (Box 1). izational or executive role in coordinating spatial and
Further, more recently, the large individual differences sensory areas, rather than holding imagery representa­
in imagery strength and vividness have been harnessed tions or content per se. The frontal cortex shows up as
to obtain information about the neural mechanisms of an active area in many functional magnetic resonance
imagery. Here, a brief overview is given of the evidence imaging (fMRI) studies7,9–12. However, despite some
for the involvement of different neural areas and how evidence of reverse directionality through the processing
imagery is processed across the network of these areas. hierarchy5,6, it remains unclear how the different areas
throughout the brain cooperate dynamically to achieve
Imagery creation: vision in reverse the goal of an imagery experience. Part of the difficulty
It is hypothesized that voluntary mental imagery is based in parsing out the isolated role of frontal areas is due
on combinations of information retrieved from stored to the diversity of types of imagery tasks performed in
memory. Accordingly, a very simple yet intuitive model fMRI experiments. For example, imagining an apple,
of functional voluntary imagery has been proposed as doing a mental rotation task or imagining a physical
depicted in Fig. 1: a reverse visual hierarchy5,6. In addi­ activity are three very different tasks that logically should
tion to the empirical evidence, this model also makes involve different brain areas, yet all get referred to under
intuitive sense as it is difficult to imagine content you the umbrella term ‘imagery’13,14.
have never been exposed to and therefore have no mem­
ories of. Typically, in imagery generation, we are com­ Hippocampus. Some evidence suggests that the hippo­
bining different content that our senses have previously campus might be required to form complex or spatially
been exposed to and is stored in our memory. distributed images. When individuals form such images,
a blood oxygen level-​dependent (BOLD) response has
been documented in the hippocampus15,16. Likewise,
Box 1 | Imagery debate
individuals with hippocampal damage show spatially
Starting in the 1970s and running until the 2000s, a debate over the nature of mental related impairments in constructing imagined experi­
representation dominated mental imagery research21,134. The debate hinged on the ences and their narrative descriptions lack richness and
question of what formats the brain could represent information in. One side of the spatial coherence17. When single neurons are measured
debate argued that information about visual objects was stored in a symbolic, and even directly in humans, a small percentage of hippocampal
language-​like, format135, whereas the other side argued that such information can be
cells do respond to imagery18; however, other work has
stored in a number of different ways, including depictive formats. Accordingly,
the debate largely focused on the two representational options: propositional and
failed to show a role for the hippocampus in imagery cre­
depictive. A propositional format would be something similar to a descriptive format ation19,20. In sum, although probably involved in spatial
used in a language. Depictive representations are often referred to as pictorial for and memory elements of imagery, the exact role of the
shorthand, as depictive representations require a functional space like a two-​dimensional hippocampus in imagery remains unclear.
XY coordinate plane or picture (see ref.21 for examples).
Throughout the 1970s and 1980s, various behavioural paradigms produced evidence Visual cortex. Questions about the role of the early visual
that imagery could be represented in a depictive format, only for the other side of the cortex, in particular V1, in mental imagery have domi­
debate to point out alternative interpretations of the data136,137. When neuroimaging nated brain imaging work on mental imagery for the last
became available in the 1990s, the debate focused on showing a blood oxygenation 25 years. From the early 1990s to late 2000s, over 20 studies
level-​dependent or positron emission tomography response to imagery in the primary
investigated the role of area V1 in imagery3. Most of
visual cortex, following the logic that the primary visual cortex represents visual
information depictively.
these studies were motivated by what is referred to as the
In 2015, Pearson and Kosslyn made the strong claim that the so-​called imagery debate imagery debate, now generally considered to be over21
was over21 and the abundance of evidence that imagery can be depictive, while still (Box 1). However, those studies were not without con­
involving semantic information, is now overwhelming. Evidence spans from behavioural troversy; although many did not find significant BOLD
evidence that imagery is represented locally in retinotopic visual and orientation space responses above baseline in the early visual cortex10,22–27,
to functional magnetic resonance imaging voxel-​wise Gabor wavelet models trained on many others did13,28–38. In hindsight, these discrepancies
depictive visual features (perceptual retinotopic location, spatial frequency and can be explained by inter-​study differences in imagery
orientation); see ref.21 for a full discussion. Few public debates have lasted so many tasks, imagery content (for example, a single visual fea­
decades; this unique dialogue adds to the rich and often controversial history of ture versus a complex image), as well as by individual
imagery research.
differences in imagery strength or vividness3,35,39.

NATuRE REvIEWS | NeuroscIence volume 20 | OCTOBER 2019 | 625

Reviews

Voxel-​wise models of
With the advent of multivariate decoding or pre­ Default mode network and imagery
perception dictive data analysis techniques in fMRI, there is now The default mode network (DMN) is a networked group
Magnetic resonance imaging clear evidence that the content of mental imagery can of brain areas that regularly show up during periods of
and functional magnetic be decoded from early visual areas, including V1 and V2 non-​task rest, it is largely defined by the functional con­
resonance imaging decoding
(refs38,40–42). It is interesting that this appears to be the nectivity or temporal relationships between activity in
methods that are constrained
by or based on individual voxel case despite relatively low BOLD amplitude responses. spatially remote areas such as the posterior cingulate
responses to perception, which The decoding algorithms used by these approaches can cortex, dorsal medial prefrontal cortex and hippocam­
are then used to decode be trained on visual perception, visual working mem­ pus43. However, the DMN also shows similarities with
imagery.
ory content and, of course, imagery, and in all cases can networks identified as being active during cognitively
Spatial transformations
accurately decode the content of mental imagery40,42. demanding tasks44. Interestingly, the networks known
Transformations in a spatial Further, when these algorithms are trained on perceptual to be involved in reexperiencing the past or creating
domain. images and constrained by voxel-​wise models of perception possible future experiences overlap with the DMN45–48
(the decoding model is based on perceptual information and has become known for its association with such
Qualia
such as contrast and spatial frequency), imagery can still processes. One of the few studies to look at conscious
The conscious sense or feeling
of something, different from be accurately decoded, suggesting imagery in the early manipulation, construction and destruction of mental
detection. visual cortex is composed of the same visual features as images showed that multiple areas, including the dorso­
afferent sensory perception42. lateral prefrontal cortex, posterior parietal cortex, pos­
Taken together, the abovementioned brain imaging terior precuneus and occipital cortex act together as a
work involving the visual cortex suggests there are pat­ network to construct and deconstruct abstract mental
terns of activity common to perception and imagery that images, that is, images that are not representations from
emerge as early as V1, but become increasingly similar specific episodic memories7. In sum, evidence suggests
with ascension up the visual processing hierarchy3. This that the DMN is involved in imagery generation or con­
visual hierarchy trend fits well into the overall model of struction; however, because most of these studies have
imagery dynamics (Fig. 1). If voluntary imagery is a top-​ investigated complex scenes from episodic memory or
down, triggered instance of sensory–memory recall that future scenarios, it is hard to separate the relationships
uses the visual cortex (and depending on imagery content, between network elements and temporal and memory
hippocampal and/or parietal areas), then it naturally fol­ components from the isolated act of forming the pure
lows that higher-​level visual areas (for example, visual area sensory elements of a mental image. Although consider­
3 and fusiform face area) that are physically and/or synap­ ing that many individuals are likely engaging in imagery
tically closer to the trigger source (frontal and medial tem­ during a resting state scan, when they are typically asked
poral lobe), would have stronger or more perception-​like to relax and think about whatever they like, it should not
representations than more distant areas like V1 (Fig. 2). be surprising that the DMN relates to imagery. Future
work should compare the DMN with measures of pure
visual imagery of abstract features or objects and probe
mind-​wandering imagery content during these resting
3 scans to divulge the role of the DMN in imagery.

Ventral and dorsal streams

1 A common way to functionally divide the visual sys­
tem is division into the two processing streams that run
ventrally and dorsally. The ventral pathway is associated
3 with properties of objects and, when areas in this stream
2
are damaged, object perception and the ability to vis­
ualize shape are both disrupted. Damage to the dorsal
pathway, by contrast, disrupts the ability to visualize
locations or spatial transformations14,49. In sum, it seems
that the data support parallels between perceptual and
imagery processing regarding these two streams of visual
processing. Indeed, recent work on aphantasia (see later
section), suggests that spatial manipulations are normal
in individuals who have no object-​based imagery50.
Fig. 1 | A top-​down general model of voluntary mental
imagery: a reverse hierarchy. The initial mental action to Involuntary imagery
create a mental image ‘begins’ high in the cortical processing As mentioned above, there are other types of imagery
hierarchy, in the frontal cortex (step 1). This triggers a that are considered involuntary, for example, those trig­
cascade of neural events, running ‘backwards’, that next
gered by an association. These images are often referred
retrieves stored information or memories from more
to as phantom perception2 as they involve sensory qualia
posterior regions such as medial temporal areas (step 2)20,
and sensory and spatial representations of the imagery without concurrent or direct sensory stimulation.
content are then formed (steps 3). If the ‘requested’ Animal work has demonstrated that, as learning
representation involves movement and spatial locations, occurs between two stimuli, sensory neurons in areas
then other areas like the middle temporal area and parietal like the middle temporal or inferior temporal cortex
lobes would also be involved. begin to respond to an absent stimulus as if it were

626 | OCTOBER 2019 | volume 20 www.nature.com/nrn

 Reviews

In sum, much like voluntary imagery, involuntary

Top-down images seem to involve the appropriate, correspon­

&
imagery signal ding sensory cortices. For example, involuntary motion
Imagery imagery will involve the middle temporal area, whereas
(High-level brain areas) involuntary colour imagery will involve V1 and V4.
More overlap What currently remains unknown, is the involvement,
between if any, of higher-​level, non-​sensory areas like the frontal
imagery and and parietal cortices (Fig. 3). Although it seems logical
perception
areas that these areas would not be involved in generating
involuntary imagery, we lack good datasets to show this.

Strong representations
Neural mechanisms of imagery strength
At least since the 1800s, people have wondered why
imagery differs so much from person to person. Now
Less overlap that the imagery debate is resolved21, the mechanisms
between of imagery, how an image is formed and why some
imagery and
perception are stronger than others (both inter-​individually and
areas intra-​individually) remains one of the primary research
Perception questions.

Strong representations
(Low-level brain areas) Recent work has begun to investigate the relation­
Bottom-up ship between V1 surface size and imagery character­
perception

v
istics59. V1 surface size, like imagery, can vary between
individuals substantially60. Research using objective
Fig. 2 | Graphical depiction showing the two streams — bottom-​up perception and perceptual measures of imagery (the binocular rivalry
top-​down voluntary imagery. Voluntary imagery and perception have a greater overlap technique) (Box 2) has shown that the surface size of
in high-​level areas (dark blue) than in lower-​level areas (light blue). This graphical depiction
both V1 and V2, but not V3, negatively predicts the
may well not hold for involuntary imagery.
sensory strength of mental imagery59. In other words,
those with stronger imagery had smaller primary visual
actually pre­sent51–53. Nevertheless, despite these data sug­ cortices. Further, the binocular rivalry technique allows
gesting that the animal might have a conscious imagery-​ imagery precision to be tested across a number of visual
like experience of the absent stimulus, we simply do features. In this study, the precision of visual imagery
not know. was measured by the spread, or generalization, of
Related work with humans has demonstrated that imagery across both spatial orientation and retinotopic
colour memory of colour-​specific fruit (for example, an visual space. Both these measures of imagery precision
orange is typically orange; a banana is typically yellow) showed a positive predictive relationship with V1 sur­
can be decoded from the BOLD response in V1 when face size59, much like perception does61. Interestingly,
humans view grey-​scale images54. These brain-​imaging those with the strongest imagery and smallest V1s did
studies are supported by behavioural research suggest­ not have the most precise imagery, suggesting that, as
ing we have a colour experience when viewing such imagery becomes stronger, it also becomes less precise.
grey-​scale colour-​specific images55 (Fig. 3). Other work This relationship between more vivid and stronger
has demonstrated that the perceptual ‘filling in’ of low-​ imagery and a smaller visual cortex is also an emerg­
contrast moving texture patterns leads to activity in V1 ing trend in the clinical literature. Human brain his­
and secondary V2 (ref.56) and, likewise, colour filling or tology shows that the size of V1 and its total number
‘neon colour spreading’ is associated with greater activity of neurons are reduced in schizophrenia compared
in V1 for the corresponding regions of the visual field to normal healthy control brains by around 25%62.
where the illusion is experienced57. This is compared to the overall brain size reduction
Prior expectations about an upcoming visual stim­ in patients with schizophrenia of only around 2%.
ulus can have a strong effect on subsequent sensory Individuals with schizophrenia show enhanced self-​
perception; indeed, fMRI work has demonstrated that reported vividness of mental imagery63 and individuals
a sensory template of the upcoming stimulus is for­ with schizotypal personality disorder show stronger sen­
med in V1 even when the perceptual stimulus is never sory imagery in the objective binocular rivalry imagery
presented58. Interestingly, it remains unknown how paradigm compared to controls64. Likewise, in PTSD,
voluntary or involuntary such a representation is; are patients who report more vivid voluntary self-​initiated
individuals taking up a strategy of using voluntary imagery65 also show signs of reduced visual cortex
imagery to aid the upcoming perceptual task? Or are size66,67. Furthermore, stimulant-​dependent individuals
these representations purely reflexive and involuntary? who demonstrate strong involuntary imagery that plays
Schizotypal personality Are these representations conscious? New work suggests an inductive role in substance dependencies68 also show
disorder that images can form prior to conscious voluntary effort, a reduction in grey matter around the visual cortex69.
A mental disorder suggesting the existance of non-​conscious involuntary It is worth noting, however, that the link between
characterized by social anxiety, 41
thought disorder, paranoid
images . Fig. 3 shows a proposed overview of some of imagery strength and V1 size is correlational, and so,
ideation, derealization and the types of imagery and their possible overlap in the despite this relationship showing up across both clinical
transient psychosis. visual cortex. and non-​clinical populations, it does not clearly specify a

NATuRE REvIEWS | NeuroscIence volume 20 | OCTOBER 2019 | 627

Reviews

Association-induced image One possible way to conceptualize the imagery net­

work and characteristics of V1, is that, at least in terms
of visual imagery, the visual cortex is something akin to
Involuntary a ‘representational blackboard’ that can form representa­
Voluntary CS Imagery
Image of US
(associative) tions from either the bottom-​up or top-​down inputs77.
The brain is not in a silent state waiting for sensory
= stimulation, but always active, showing intrinsic activ­
ity dynamics, which emerge as spontaneous or ongoing
Top-down Top-down activity80. The degree of this ongoing activity noise in this
‘representational blackboard’ will impact imagery, and so
too will the strength of the top-​down signal (analogous
Lateral to the strength or contrast of ‘chalk’ in this blackboard
Overlap?
Involuntary metaphor). A given individual might have low levels of
(perceptual)
noise activity in V1, but a weak top-​down signal, and
therefore the overall degree of imagery strength, might
Sensory cortex be weak. Another individual, by contrast, might have a
representations
very strong top-​down signal, but their visual cortex is
Fig. 3 | Mapping out the different types of visual imagery — voluntary, involuntary much noisier, hence the imagery representation becomes
(associative) and involuntary (local perceptual). The involuntary associative imagery corrupted (Fig. 4). In this scenario, an individual with
would be the product of associative learning. For example, after thousands of co-​occurrences both a strong top-​down signal and low noise, or quieter
seeing tomatoes as red, a black and white image of a tomato is seen as slightly red due to sensory cortex, should have the strongest imagery (bot­
red involuntary imagery; your brain literally fills in the colour. Likewise, the classic case of tom left Fig. 4). Presently, this is only a theory, and there
conditioning with Pavlov’s dogs, does the sound of the ringing bell induce an image of the are many other clues suggesting imagery might have a
food in the dog’s mind? Here, the speaker symbol, the conditioned stimulus (CS) illustrates
complex multi-​component mechanism, as it is clear that
this sound-​induced involuntary image, the unconditioned stimulus (US). Perceptual
involuntary imagery, or phantom vision, is when the structure of low-​level perceptual brain imagery can be affected by many other factors such as
areas produces involuntary vision like the colour spreading in neon-​colour spreading illusion. memory, hormones81, multiple neurological and mental
Adapted with permission from ref.2, Elsevier. disorders, and even psychedelics82.

Imagery, like a weak form of perception

functional mechanism. Interestingly, the data suggest a The conscious experience of imagining and perceiving
reciprocal relationship between V1 and frontal cortex something are clearly very different for almost everyone.
size, with a smaller V1 predicting larger frontal areas70. However, evidence from the past 100 years of behavioural
If frontal areas are responsible for the top-​down imagery research suggests that visual imagery can have a func­
signal, then a scenario with larger frontal areas and tional effect on sensory processing akin to a weak form of
smaller V1 might enable more control over the visual visual perception3. For example, both imagining and per­
cortex (Fig. 4). Moreover, V1 surface area is influenced, ceiving oriented lines can have a similar effect on subse­
in part, by genetic differences71–73, suggesting a possible quent perception83,84. Imagery content can also undergo
indirect genetic contribution to imagery strength. different types of learning. Imagining visual content, just
In addition to the anatomy of early visual cortex, evi­ like perceptual content, can induce visual per­ceptual
dence suggests that cortical activity metrics also predict learning, which improves subsequent visual sensitivity to
imagery vividness35,38. Further, recent behavioural work those specific types of stimuli85. Likewise, the content of
has shown that imagery vividness is not a static trait-​ visual imagery can undergo associative learning with an
like characteristic of imagery, but dynamically changes emotional stimulus that transfers to perceptual stimuli in
from moment to moment within an individual59,74–76, and a manner specific to early visual processing86. In addition,
these changes are predicted by a network of activity over the brightness of imagined stimuli corresponds to pre­
the whole brain, including the degree of overlap with dictable changes in pupil diameter, without concurrent
perception in the visual system38. sensory stimulation87.
Some recent data suggest that the excitability of the There is now substantive evidence that visual imagery
visual cortex might play a role in imagery strength77. can have a facilitative effect on visual perception of a
This work shows correlational evidence that uses trans­ relevant visual stimulus74,76,83,84. Indeed, even when
cranial magnetic stimulation-​induced phosphenes and imagery and perception are separated in time by another
fMRI resting activity as proxies of cortical excitability. demanding cognitive task, these facilitatory effects
This study shows that causative transcranial direct cur­ remain83. Weak perceptual stimuli (low contrast or lumi­
rent stimulation, which alters resting excitability, had a nance) also show this same facilitatory priming effect
weak effect on imagery strength by either up or down­ on subsequent perception83,88,89. The energy (strength or
regulating it77. These multiple data sources suggest that duration) of a prior stimulus seems to dictate whether
stronger visual imagery is associated with lower resting that prior stimulus has a facilitatory (priming for weak
activity in the visual cortex. These data, although still images) or suppressive (adaptation for strong images)
preliminary, suggest that imagery, along with other expe­ effect on subsequent perception. Likewise, longer
riences of phantom vision, like synaesthesia78 (which durations, or epochs, of imagery have a stronger prim­
itself is linked to stronger imagery79), might depend on ing effect on subsequent perception41,74,83. Hence, the
visual cortex excitability. degree to which imagery affects subsequent perception

628 | OCTOBER 2019 | volume 20 www.nature.com/nrn

 Reviews

has been taken as a measure of the sensory strength or imagery93, and takes researchers one step closer to
visual energy of imagery76 (see Box 2 on measurement being able to recreate the sensory contents of the mind.
techniques). Just like visual perception, visual imagery However, despite the evidence suggesting commonali­
seems to be largely local in retinotopic and orientation ties between weak perception and imagery, the two are
space42,59,79,83,90,91, lending further support to the similarities clearly very different conscious experiences and future
between weak visual perception and visual imagery. work should aim to map out why non-​clinical imagery
A criticism of some imagery research was the lack of is not experienced with the full conscious experience of
methodology to differentiate visual imagery and visual afferent perception.
attention. For example, if an individual imagines a par­
ticular colour while they are performing a perceptual Imagery in normal cognitive function
task involving colour, they will be differentially attending Recent years have produced many publications linking
to the colour they imagine. Hence, any effects of imagery mental imagery to a range of cognitive processes from
and attention will be difficult to separate. Importantly, episodic and visual working memory, spatial navigation,
in some of this research, the effect of prior imagery and reading comprehension to creativity and moral decision-​
prior visual attention can be dissociated in a number of making (Fig. 5). If mental imagery is a sensory simula­
ways such as through the timing of these effects and their tion of events, objects or scenarios in isolation from the
susceptibility to sensory disruption83; this provides fur­ afferent sense organ-​induced versions of these things,
ther evidence that these sensory-​like effects are driven by then how and when do we use imagery in everyday life?
imagery and not attention. In addition, these perceptual
priming effects are linked to the reported imagery vivid­ Memory and imagery
ness on a trial-​by-trial basis74, suggesting the effects are Despite the fact that the fields of mental imagery, visual
coupled to the imagery generation process. working and autobiographical episodic memory have
Brain imaging work also provides evidence that developed independently, evidence suggests that many
imagery can be thought of as a form of top-​down weak people utilize imagery to perform most forms of visual
perception. Multiple studies have now used BOLD acti­ memory tasks94–97. Indeed, in the episodic memory liter­
vation patterns during afferent visual perception to later ature, thinking about the future is referred to as imagin­
decode the contents of mental images42,91,92, suggesting ing the future. Depending on who you are and probably
an overlap in neural representation. Further, deep learn­ how strong your imagery is, the role of imagery across
ing algorithms trained on a limited set of perceptual fea­ different types of memory might seem obvious and self-​
tures can be used to decode untrained features in mental evident. However, the fields of imagery and working
memory have diverged and until recently have remained
largely separate98.
Box 2 | New measurement techniques In regard to visual working memory, when people are
Like dreaming or hallucinations, research into mental imagery has been constrained by asked how they complete such tasks, they typically report
a lack of reliable objective measurement methods. From the early use of questionnaires using two primary strategies. One strategy is to pick
by Galton in the 1800s110, self-​report questionnaires have been the gold standard used out salient features in a test array that the person is
to measure imagery. However, recent years have seen a flurry of new and more objective required to remember and encode them in a pro­positio­
measurement techniques. nal or phonological form, which is then com­pared to
Behavioural techniques the test99. The other strategy commonly des­cribed is the
In 2008, we published the first work showing how you could use a visual illusion, called creation of detailed mental images during the retention
binocular rivalry, to measure imagery. This technique involves randomly cuing an interval, which are then compared to the sub­sequent
individual to imagine a coloured object before a brief rivalry presentation. The content test arrays94,95,99,100. These descriptions are synonymous
of the imagery primes dominance in the rivalry illusion in a manner that reflects the with descriptions of mental imagery and it is likely that
imagery vividness both on a trial-​by-trial and individual differences basis74,83. Importantly, these individuals use mental imagery as a mnemonic
this technique works even when a cognitively demanding task is added between the tool to retain visual information. Both behavioural and
imagery period and the rivalry presentation. This technique can be made even more brain imaging work now suggests that sensory imagery
objective, in terms of removing direct subjective reports, by measuring rivalry
representations are used by people who have imagery to
dominance with an embedded probe task138. In this case, rivalry dominance is assessed
using an embedded probe task, where the probe is easier to detect when it is complete some forms of visual working memory
embedded in the dominant rivalry pattern, and then the influence of imagery on rivalry tasks40,94,95,101. Additionally, both imagery and visual
can be determined by the level of probe detection. Using such a performance-​based working memory can change perception83,102, and they
dependent variable instead of rivalry dominance reports means that imagery can be show similar capacity functions103,104. However, research
measured without any subjective reports. In addition, such methods can be used to also suggests that not all people use visual imagery
measure imagery of different visual features such as pure colour or motion imagery, or to solve visual working memory tasks, with irrele­
even complex object imagery, using the one dependent measure90,133,138, or even vant visual information impairing only some partic­
compare imagery and synaesthesia using the one common measure79. ipant’s performance on visual working memory tasks.
Imaging techniques Specifically, only participants with good imagery appear
Recent functional magnetic resonance imaging research has clearly shown that to be affected by irrelevant visual information94,95,105, sug­
imagery content can be objectively investigated using brain imaging40,42,139. What gesting that only they use visual imagery and low-​level
someone imagines can be decoded or predicted based only on the brain’s response to visual regions to perform these tasks. The idea that only
the afferent perceptual versions of that same content. In fact, imagery content can now good imagers use visual imagery, and hence the visual
be decoded with perceptual training data from different object classes93, and other
cortex, to perform visual working memory tasks may
methods like electroencephalography and pupil dynamics can also be used.
help explain much of the current controversy in both the

NATuRE REvIEWS | NeuroscIence volume 20 | OCTOBER 2019 | 629

 Reviews

Low
Noise level in visual cortex
High Why does it matter that imagery is utilized across
Low such a large range of cognitive processes? We have
Weakest imagery known since Francis Galton’s seminal paper on mental
imagery in 1880 that the spectrum of imagery vivid­
ness spans from aphantasia to hyperphantasia (highly
vivid, almost realistic imagery)110. Accordingly, if many
cognitive processes use imagery, but imagery can vary
so much across the population, then it follows that the
Top-down signal strength

strength of imagery will have a strong knock-​on effect for

how someone performs any dependent tasks. Anything
that might affect imagery strength might also have an
Strongest imagery effect on visual working and episodic memory, reading
comprehension, the veracity of eye witness memory,
moral decisions and maybe even creativity, not to men­
tion dimensions of mental health (Fig. 5). Therefore, one
cannot fully understand any of these dependent com­
pound cognitive processes without taking into account
or understanding the variations in mental imagery
strength. Accordingly, understanding imagery and meas­
uring it accurately holds the potential key to unlocking
High the mysteries of multiple cognitive processes105.
Fig. 4 | Theoretical representation of visual imagery of a square, showing possible
Mental and neurological disorders
interaction between the strength of the top-​down imagery signal and noise in the
visual cortex. A strong top-​down signal and low noise (bottom left) gives the strongest Mental imagery plays a major role in much of psycho­
mental image (square), whereas a high level of neural noise and a weak top-​down pathology, perhaps unsurprisingly given experimental
imagery signal would produce the weakest imagery experience (top right). findings that visual thought produces more emotion than
verbal thought111. Intrusive, affect-​laden mental imagery
causes functional distress across a range of mood dis­
visual working memory modelling and neuroimaging orders, for example, by subserving ruminative and worry
literature105. states112. In depression, individuals have difficulty form­
Likewise, to anyone who has good imagery it is ing positive future imagery113 and imagining suicidal acts
immediately obvious that imagery would be an impor­ may increase the risk of suicide114. Patients with bipolar
tant experiential tool when referencing autobiographi­ disorder show high spontaneous use of imagery and
cal memories or future projections. When I think of my intrusive imagery115,116, and future-​oriented imagery
birthday last year, visual flashes of people sitting around ‘flash forwards’ to suicidal acts115. Involuntary intru­
a table sharing food and chatting involuntarily flash sive visual images are a key component of cravings in
through my mind. Imagery has long been discussed as a addiction117–119 and of flashbacks in PTSD. The realness,
crucial element to autobiographical thinking. However, or ‘nowness’, of intrusive images predicts the severity
the inherent difficulties in measuring imagery are per­ of PTSD symptoms and outcomes120 and the ease with
haps dwarfed by the very personal, bespoke and complex which these intrusions can be triggered through viewing
phenomenology involved in autobiographical memory related images predicts PTSD severity120. An individual’s
measurement. Needless to say, investigating the role of non-​traumatic visual imagery predicts PTSD sympto­
imagery in episodic memory brings many methodological mology, with more vivid imagery being associated with
challenges. more frequent flashbacks65. Intervention research has
Despite a primary reliance on self-​report techniques, also suggested that disrupting the visuospatial sketch­
research suggests that individuals with vivid imagery do pad by playing the game Tetris after viewing traumatic
re-​experience episodes from their life via memory dif­ footage reduces subsequent intrusive imagery121. It has
ferently to those with weak imagery106–109. Data suggest been theorized that playing Tetris occupies the visual
that individuals with vivid imagery report more sensory areas of the brain (or the visuospatial sketchpad or rep­
details both in episodic and future event construction108, resentational blackboard of the mind), and this impairs
and also rate their experience of reliving the event as an individual’s ability to replay the event during memory
more real than those with poor imagery106, while more consolidation.
recent work has shown that object and spatial imagery Both schizophrenia and Parkinson disease have been
predict different dimensions of episodic memory96,109. associated with stronger sensory imagery, measured
Interestingly, people without any imagery (aphantasics), using the objective binocular rivalry technique64,122. In
can still perform visual working memory tasks and seem fact, vivid imagery has been proposed as a trait maker
to have episodic memories of their lives (see section for schizophrenia63. As mentioned above, it is interesting
below on aphantasia). Hence, it would seem that using that both in the normal population and in schizophre­
imagery for such compound cognitive tasks is just one of nia, a smaller V1 is associated with stronger imagery.
many available strategies, underpinning the importance In patients with Parkinson disease, the strength of an
of probing the particular strategy an individual might individual’s visual imagery predicts the severity of their
use for memory tasks. visual hallucinations122.

630 | OCTOBER 2019 | volume 20 www.nature.com/nrn

 Reviews

In addition to featuring alongside or as a symptom of small proportion of otherwise healthy people report
the disorder, visual imagery increasingly plays a role in having no visual experience at all when they attempt
evidence-​based psychological treatment. For example, to imagine something. In other words, their minds are
cognitive behavioural therapy often includes ‘imaginal completely blind — no matter how hard they try, they
exposure’, which involves the patient repeatedly imag­ do not have a conscious visual experience of mental
ining a feared object or context until their anxiety level imagery; this phenomenon has only recently been given
subsides123. Imagery rescripting has been somewhat the name congenital aphantasia1. Initially, it was unclear
successful in treating PTSD, social and snake phobias, if aphantasia was more of an issue of imagery reporting,
and a variety of other psychological disorders124. It typi­ in that people with aphantasia might still have func­
cally involves the practice of rescripting the imagery in a tional imagery, but not be aware of it or have a differ­
sequence to a more positive or adaptive outcome. ent criterion for reporting it. However, recent research
As the evidence grows that visual imagery can trigger goes beyond subjective self-​reports of aphantasia and
a larger emotional response than propositional or sym­ has shown that, along with floor scores on imagery
bolic thoughts125, the understanding of imagery both as a questionnaires, people with aphantasia score signifi­
symptom and as a strategic therapeutic treatment grows. cantly lower on the sensory binocular rivalry measure
Clinical research should aim to leverage data and theory of visual imagery50 (Box 2); in other words, the sensory
from fundamental brain research on imagery as soon as strength of their imagery is also lacking, suggesting that
possible, as there are many promising avenues for uti­ aphantasia is more than poor metacognition of visual
lizing or manipulating imagery as a novel therapeutic imagery.
intervention. Interestingly, although people with aphantasia scored
lower than controls on all imagery vividness or content
Extreme differences in visual imagery measures, they actually scored slightly higher than con­
Aphantasia. Since Galton’s seminal paper in the 1800s110, trols on spatial imagery questionnaires50. This suggests
the idea of a category of people whose mind is com­ that, although the ‘what’ or content of imagery typically
pletely blind was largely forgotten until recently1,126. associated with the ventral stream of visual processing
Initially based just on self-​report, it seemed that a is lost, the ‘where’, or spatial properties, associated with
dorsal areas might still be intact. It also suggests that
aphantasia could dissociate along the classic ‘what and
where’ processing pathways in the brain127. Accordingly,
Prospective Eye witness
areas of the brain that process spatial properties, includ­
memory memory ing the hippocampus, may not be the underlying cause
of aphantasia. Other work provides evidence that peo­
Arithmetic ple with aphantasia can perform easy and medium, but
False memory not hard visual working memory tasks128. Likewise,
a case report of two people with aphantasia showed
Navigation that they scored low on ratings of ‘reliving’ their epi­
sodic memories, although they still reported believing
Episodic Mental Visual working
memory imagery memory Moral decisions these memories106. Many individuals with aphantasia
describe visual imagery during dreams1, suggesting that
Sports
involuntary forms of imagery might still be intact.
Although the popular media has shown a fascination
Creativity with aphantasia, it is still early days in terms of scientific
Motor control research. Perhaps the most poignant question surrounds
Reading
the idea that aphantasia could either be just one end of a
Mind wandering
comprehension spectrum of imagery abilities or it could be in a category
all of its own, different to people with very weak imagery.
Further research is needed to shed light on these two
Fig. 5 | Graphical depiction of the cognitive processes related to mental imagery. possibilities.
The cognitive processes shown in the dark blue zone — visual working and episodic The fact that individuals with aphantasia can still per­
memory — are there because reasonable evidence exists that, in people with mental form many daily tasks that non-​aphantasics use men­
imagery (excluding people with aphantasia), they are largely reliant on mental imagery tal imagery for, for example, visual working memory,
for normal functioning and there is a ‘good’ amount of evidence to support this suggests they use a different strategy and brain mech­
relationship94,95. Evidence has shown that those with strong imagery will utilize it as a anisms to do so. Presently, it is unknown what mem­
mnemonic tool to perform visual working memory tasks94,95. This evidence is beyond ory aids they use, perhaps semantic encoding or some
correlational and has used sensory perturbation methodologies (see section on non-​representational geometry or symbols. It is worth
memory)94,95. Likewise, episodic memory research has specifically looked at the rich thinking about the possible implications of this, if one
sensory nature of remembered past experiences as well as the vividness of projecting
was to simply go by performance on a compound task
oneself into possible future episodic experiences96,109. Across a range of different
methodologies, effect sizes and degree of evidence, the cognitive processes shown in like visual working memory, it would be very difficult to
the lighter blue zone have been hypothesized to involve mental imagery, although the differentiate a person with aphantasia and a person with
degree of evidence is less than that for the dark blue zone. The outer lighter blue zone normal mental imagery. However, it is safe to say that
shows moral decisions140, eye witness memory141, false memories142, arithmetic, the underlying neural mechanism used to perform the
navigation143, sports144, reading comprehension145, mind wandering146 and creativity147. task by these two individuals would be quite different.

NATuRE REvIEWS | NeuroscIence volume 20 | OCTOBER 2019 | 631

Reviews

Such a scenario underlines the importance of testing as imagery research gains popularity, researchers will
imagery ability when investigating cognitive processes reignite eidetic and hyperphantasic imagery research.
like visual working or episodic memory. Further, this
suggests that using imagery as a cognitive tool is not The future of imagery research
funda­mental to performing other cognitive functions Now that the imagery debate is over21, and we have mul­
and has interesting implications for the way we should tiple new objective methods to investigate imagery, what
theorize about imagery as a cognitive tool and its possible should imagery researchers focus on?
role in evolution. Visual perception is a constructive affair; it involves
both feedforward and feedback signals working closely
Hyperphantasia or eidetic imagery. Hyperphantasia, and interactively together. Hence, isolating the effects of
or what has previously been called eidetic imagery, is bottom-​up feedforward or feedback signals during per­
at the other end of the imagery spectrum — consisting ception is very difficult. Voluntary imagery is perhaps
of strong and vivid imagery. Early in the 20th century, the only pure form of sensory representation that can
the topic of eidetic imagery attracted much attention, be solely due to feedback signals and can occur in com­
commonly described as a highly detailed, almost photo-​ plete sensory isolation (although not always). Hence, it
realistic image experienced directly after seeing the represents a unique window to study the dynamics of
object, but also days or weeks later129–132. Eidetic imagery, feedback signals in the brain and understand the con­
like aphantasia, is only observed in a small percentage structive nature of visual perception — an important
of the population, mainly in children, with frequency step of reductionism to understand the mechanisms
estimates from 0–11% in children129. Unlike afterim­ behind sensory perception. As perception is interactive,
ages, eidetic images remain still while the eye moves, and hence cannot be understood without parsing out the
are experienced in positive colours, projected out into mechanisms of feedback, it can be argued that imagery
space (much like a form of synaesthesia) and individuals is the key to understanding normal visual perception. It
use present, not past tense language, when referring to will be interesting to see how imagery research can be
the image. harnessed to understand visual perception.
Eidetic imagery has been assessed in a number of Several pertinent questions regarding imagery
ways, from verbal descriptions to more objective meth­ remain, including the issue of why imagery rarely feels
ods such as showing individuals random-​dot stereo­ as strong and vivid as afferent sensory perception; the
grams, pictures of hundreds of randomly placed dots, causes underlying aphantasia and the large range of
with those in the centre shifted to give the perception individual differences in the experience of imagery;
of depth when the two images are shown one to each whether there exists a training protocol that might
eye simultaneously; however, for eidetic imagers the improve imagery strength; whether imagery indeed
depth patch could be seen clearly, even when the two induces more activity in visual cortical areas, as opposed
images were shown up to 24 hours apart, by combin­ to being just modulatory; and whether voluntary or
ing the first or eidetic image with the second perceptual involuntary imagery could ever exist unconsciously133.
image130. Because modern day psychophysics, fMRI As many of the new methods to investigate imagery are
and transcranial magnetic stimulation methods have only newly available, it is exciting to think over all the
not been used to investigate eidetic imagery, we do not future imagery research waiting to be performed.
have a lot of data to suggest a possible mechanism or
why it seems more prevalent in children. Hopefully, Published online 5 August 2019

1. Zeman, A., Dewar, M. & Della Sala, S. Lives without mechanisms for visual working memory. Curr. Opin. 17. Hassabis, D., Kumaran, D., Vann, S. D. & Maguire, E. A.
imagery — congenital aphantasia. Cortex 73, Neurobiol. 15, 175–182 (2005). Patients with hippocampal amnesia cannot imagine
378–380 (2015). 9. Yomogida, Y. Mental visual synthesis is originated in new experiences. Proc. Natl Acad. Sci. USA 104,
This article documents and coins the term the fronto-​temporal network of the left hemisphere. 1726–1731 (2007).
aphantasia, described as the complete lack Cereb. Cortex 14, 1376–1383 (2004). 18. Kreiman, G., Koch, C. & Fried, I. Imagery neurons
of visual imagery ability. 10. Ishai, A., Ungerleider, L. G. & Haxby, J. V. Distributed in the human brain. Nature 408, 357–361 (2000).
2. Pearson, J. & Westbrook, F. Phantom perception: neural systems for the generation of visual images. 19. Maguire, E. A., Vargha-​Khadem, F. & Hassabis, D.
voluntary and involuntary non-​retinal vision. Trends Neuron 28, 979–990 (2000). Imagining fictitious and future experiences: evidence
Cogn. Sci. 19, 278–284 (2015). 11. Goebel, R., Khorram-​Sefat, D., Muckli, L., Hacker, H. from developmental amnesia. Neuropsychologia 48,
This opinion paper proposes a unifying framework & Singer, W. The constructive nature of vision: direct 3187–3192 (2010).
for both voluntary and involuntary imagery. evidence from functional magnetic resonance imaging 20. Kim, S. et al. Sparing of spatial mental imagery in
3. Pearson, J., Naselaris, T., Holmes, E. A. & Kosslyn, S. M. studies of apparent motion and motion imagery. patients with hippocampal lesions. Learn. Mem. 20,
Mental imagery: functional mechanisms and clinical Eur. J. Neurosci. 10, 1563–1573 (1998). 657–663 (2013).
applications. Trends Cogn. Sci. 19, 590–602 12. Mellet, E. et al. Functional anatomy of spatial mental 21. Pearson, J. & Kosslyn, S. M. The heterogeneity of
(2015). imagery generated from verbal instructions. mental representation: ending the imagery debate.
4. Egeth, H. E. & Yantis, S. Visual attention: control, J. Neurosci. 16, 6504–6512 (1996). Proc. Natl Acad. Sci. USA 112, 10089–10092 (2015).
representation, and time course. Annu. Rev. Psychol. 13. O’Craven, K. M. & Kanwisher, N. Mental imagery This paper proposes an end to the ‘imagery debate’
48, 269–297 (1997). of faces and places activates corresponding stimulus- based on the discussed evidence that imagery can
5. Dijkstra, N., Zeidman, P., Ondobaka, S., Gerven, M. A. J. specific brain regions. J. Cogn. Neurosci. 12, be represented in the brain in a depictive manner.
& Friston, K. Distinct top-​down and bottom-​up brain 1013–1023 (2000). 22. D’Esposito, M. et al. A functional MRI study of mental
connectivity during visual perception and imagery. 14. Kosslyn, S. M., Ganis, G. & Thompson, W. L. Neural image generation. Neuropsychologia 35, 725–730
Sci. Rep. 7, 5677 (2017). foundations of imagery. Nat. Rev. Neurosci. 2, (1997).
6. Dentico, D. et al. Reversal of cortical information flow 635–642 (2001). 23. Knauff, M., Kassubek, J., Mulack, T. & Greenlee, M. W.
during visual imagery as compared to visual 15. Hassabis, D., Kumaran, D. & Maguire, E. A. Using Cortical activation evoked by visual mental imagery
perception. Neuroimage 100, 237–243 (2014). imagination to understand the neural basis of episodic as measured by fMRI. Neuroreport 11, 3957–3962
7. Schlegel, A. et al. Network structure and dynamics of memory. J. Neurosci. 27, 14365–14374 (2007). (2000).
the mental workspace. Proc. Natl Acad. Sci. USA 110, 16. Bird, C. M., Capponi, C., King, J. A., Doeller, C. F. 24. Trojano, L. et al. Matching two imagined clocks:
16277–16282 (2013). & Burgess, N. Establishing the boundaries: the the functional anatomy of spatial analysis in the
8. Ranganath, C. & D’Esposito, M. Directing the mind’s hippocampal contribution to imagining scenes. absence of visual stimulation. Cereb. Cortex 10,
eye: prefrontal, inferior and medial temporal J. Neurosci. 30, 11688–11695 (2010). 473–481 (2000).

632 | OCTOBER 2019 | volume 20 www.nature.com/nrn

 Reviews

25. Wheeler, M. E., Petersen, S. E. & Buckner, R. L. 47. Hassabis, D. & Maguire, E. A. Deconstructing episodic 72. Winkler, A. M. et al. Cortical thickness or grey matter
Memory’s echo: vivid remembering reactivates memory with construction. Trends Cogn. Sci. 11, volume? The importance of selecting the phenotype
sensory-​specific cortex. Proc. Natl Acad. Sci. USA 97, 299–306 (2007). for imaging genetics studies. Neuroimage 53,
11125 (2000). 48. Gerlach, K. D., Spreng, R. N., Gilmore, A. W. & 1135–1146 (2010).
26. Formisano, E. et al. Tracking the mind’s image in Schacter, D. L. Solving future problems: default 73. Bakken, T. E. et al. Association of common genetic
the brain I: time-​resolved fMRI during visuospatial network and executive activity associated with variants in GPCPD1 with scaling of visual cortical
mental imagery. Neuron 35, 185–194 (2002). goal-​directed mental simulations. Neuroimage 55, surface area in humans. Proc. Natl Acad. Sci. USA
27. Sack, A. T. et al. Tracking the mind’s image in 1816–1824 (2011). 109, 3985–3990 (2012).
the brain II: transcranial magnetic stimulation reveals 49. Levine, D. N., Warach, J. & Farah, M. Two visual 74. Pearson, J., Rademaker, R. L. & Tong, F. Evaluating
parietal asymmetry in visuospatial imagery. Neuron systems in mental imagery. Neurology 35, 1010 the mind’s eye: the metacognition of visual imagery.
35, 195–204 (2002). (1985). Psychol. Sci. 22, 1535–1542 (2011).
28. Le Bihan, D. et al. Activation of human primary visual 50. Keogh, R. & Pearson, J. The blind mind: no sensory 75. Rademaker, R. L. & Pearson, J. Training visual
cortex during visual recall: a magnetic resonance visual imagery in aphantasia. Cortex 105, 53–60 imagery: improvements of metacognition, but not
imaging study. Proc. Natl Acad. Sci. USA 90, (2017). imagery strength. Front. Psychol. 3, 224 (2012).
11802–11805 (1993). 51. Sakai, K. & Miyashita, Y. Neural organization for the 76. Pearson, J. New directions in mental-​imagery research:
29. Sabbah, P. et al. Functional magnetic resonance long-​term memory of paired associates. Nature 354, the binocular-​rivalry technique and decoding fMRI
imaging at 1.5T during sensorimotor and cognitive 152–155 (1991). patterns. Curr. Dir. Psychol. Sci. 23, 178–183 (2014).
task. Eur. Neurol. 35, 131–136 (1995). 52. Messinger, A., Squire, L. R., Zola, S. M. & Albright, T. D. 77. Keogh, R., Bergmann, J. & Pearson, J. Cortical
30. Chen, W. et al. Human primary visual cortex and Neuronal representations of stimulus associations excitability controls the strength of mental imagery.
lateral geniculate nucleus activation during visual develop in the temporal lobe during learning. Preprint at bioRxiv https://www.biorxiv.org/content/
imagery. Neuroreport 9, 3669–3674 (1998). Proc. Natl Acad. Sci. USA 98, 12239–12244 (2001). 10.1101/093690v1 (2016).
31. Ishai, A. Visual imagery of famous faces: effects of 53. Schlack, A. & Albright, T. D. Remembering visual 78. Terhune, D. B., Tai, S., Cowey, A., Popescu, T. &
memory and attention revealed by fMRI. Neuroimage motion: neural correlates of associative plasticity Kadosh, R. C. Enhanced cortical excitability in
17, 1729–1741 (2002). and motion recall in cortical area MT. Neuron 53, grapheme-​color synesthesia and its modulation. Curr.
32. Ganis, G., Thompson, W. L. & Kosslyn, S. M. Brain 881–890 (2007). Biol. 21, 2006–2009 (2011).
areas underlying visual mental imagery and visual 54. Bannert, M. M. & Bartels, A. Decoding the yellow of 79. Chiou, R., Rich, A. N., Rogers, S. & Pearson, J.
perception: an fMRI study. Cogn. Brain Res. 20, a gray banana. Curr. Biol. 23, 2268–2272 (2013). Exploring the functional nature of synaesthetic colour:
226–241 (2004). 55. Hansen, T., Olkkonen, M., Walter, S. & Gegenfurtner, dissociations from colour perception and imagery.
33. Klein, I., Paradis, A. L., Poline, J. B., Kossly, S. M. K. R. Memory modulates color appearance. Cognition 177, 107–121 (2018).
& Le Bihan, D. Transient activity in the human calcarine Nat. Neurosci. 9, 1367–1368 (2006). 80. Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A.
cortex during visual-​mental imagery: an event-​related 56. Meng, M., Remus, D. A. & Tong, F. Filling-​in of visual Dynamics of ongoing activity: explanation of the large
fMRI study. J. Cogn. Neurosci. 12 (Suppl. 2), 15–23 phantoms in the human brain. Nat. Neurosci. 8, variability in evoked cortical responses. Science 273,
(2000). 1248–1254 (2005). 1868–1871 (1996).
34. Lambert, S., Sampaio, E., Scheiber, C. & Mauss, Y. 57. Sasaki, Y. & Watanabe, T. The primary visual cortex 81. Wassell, J., Rogers, S. L., Felmingam, K. L., Bryant, R. A.
Neural substrates of animal mental imagery: calcarine fills in color. Proc. Natl Acad. Sci. USA 101, & Pearson, J. Biological psychology. Biol. Psychol.
sulcus and dorsal pathway involvement — an fMRI 18251–18256 (2004). 107, 61–68 (2015).
study. Brain Res. 924, 176–183 (2002). 58. Kok, P., Failing, M. F. & de Lange, F. P. Prior 82. Kraehenmann, R. et al. LSD increases primary process
35. Cui, X., Jeter, C. B., Yang, D., Montague, P. R. expectations evoke stimulus templates in the primary thinking via serotonin 2A receptor activation. Front.
& Eagleman, D. M. Vividness of mental imagery: visual cortex. J. Cogn. Neurosci. 26, 1546–1554 Pharmacol. 8, 418–419 (2017).
individual variability can be measured objectively. (2014). 83. Pearson, J., Clifford, C. W. G. & Tong, F. The functional
Vision Res. 47, 474–478 (2007). 59. Bergmann, J., Genc, E., Kohler, A., Singer, W. impact of mental imagery on conscious perception.
36. Amedi, A., Malach, R. & Pascual-​Leone, A. Negative & Pearson, J. Smaller primary visual cortex is Curr. Biol. 18, 982–986 (2008).
BOLD differentiates visual imagery and perception. associated with stronger, but less precise mental This study shows that the content of visual imagery
Neuron 48, 859–872 (2005). imagery. Cereb. Cortex 26, 3838–3850 (2016). can bias or prime subsequent binocular rivalry; this
37. Reddy, L., Tsuchiya, N. & Serre, T. Reading the mind’s This study shows that stronger but less precise paper was the basis for using binocular rivalry as
eye: decoding category information during mental imagery is associated with a smaller primary and a measurement tool for imagery.
imagery. Neuroimage 50, 818–825 (2010). secondary visual cortex. 84. Ishai, A. & Sagi, D. Common mechanisms of visual
38. Dijkstra, N., Bosch, S. E. & van Gerven, M. A. J. 60. Stensaas, S. S., Eddington, D. K. & Dobelle, W. H. The imagery and perception. Science 268, 1772–1774
Vividness of visual imagery depends on the neural topography and variability of the primary visual cortex (1995).
overlap with perception in visual areas. J. Neurosci. in man. J. Neurosurg. 40, 747–755 (1974). 85. Tartaglia, E. M., Bamert, L., Mast, F. W. & Herzog, M. H.
37, 1367–1373 (2017). 61. Song, C., Schwarzkopf, D. S. & Rees, G. Variability Human perceptual learning by mental imagery.
39. Kosslyn, S. M. & Thompson, W. L. When is early in visual cortex size reflects tradeoff between local Curr. Biol. 19, 2081–2085 (2009).
visual cortex activated during visual mental imagery? orientation sensitivity and global orientation This study shows that training with a purely imaged
Psychol. Bull. 129, 723–746 (2003). modulation. Nat. Commun. 4, 1–10 (2013). visual stimulus transfers to improve performance in
40. Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C. 62. Dorph-​Petersen, K.-A., Pierri, J. N., Wu, Q., perceptual tasks.
& de Lange, F. P. Shared representations for working Sampson, A. R. & Lewis, D. A. Primary visual cortex 86. Lewis, D. E., O’Reilly, M. J. & Khuu, S. K. Conditioning
memory and mental imagery in early visual cortex. volume and total neuron number are reduced in the mind’s eye associative learning with voluntary
Curr. Biol. 23, 1427–1431 (2013). schizophrenia. J. Comp. Neurol. 501, 290–301 mental imagery. Clin. Psychol. Sci. 1, 390–400 (2013).
This paper shows that both imagery and visual (2007). 87. Laeng, B. & Sulutvedt, U. The eye pupil adjusts to
working memory can be decoded in the brain 63. Sack, A. T., van de Ven, V. G., Etschenberg, S., Schatz, D. imaginary light. Psychol. Sci. 25, 188–197 (2014).
based on training on either, showing evidence & Linden, D. E. J. Enhanced vividness of mental 88. Brascamp, J. W., Knapen, T. H. J., Kanai, R., van Ee, R.
of a common brain representation. imagery as a trait marker of schizophrenia? Schizophr. & van den Berg, A. V. Flash suppression and flash
41. Koenig-​Robert, R. & Pearson, J. Decoding the Bull. 31, 97–104 (2005). facilitation in binocular rivalry. J. Vis. 7, 12 (2007).
contents and strength of imagery before volitional 64. Maróthi, R. & Kéri, S. Enhanced mental imagery and 89. Tanaka, Y. & Sagi, D. A perceptual memory for low-​
engagement. Sci. Rep. 9, 3504 (2019). intact perceptual organization in schizotypal contrast visual signals. Proc. Natl Acad. Sci. USA 95,
This paper shows that the content and vividness personality disorder. Psychiatry Res. 259, 433–438 12729–12733 (1998).
of a mental image can be decoded in the brain up (2018). 90. Chang, S., Lewis, D. E. & Pearson, J. The functional
to 11 seconds before an individual decides which 65. Morina, N., Leibold, E. & Ehring, T. Vividness effects of color perception and color imagery. J. Vis.
pattern to imagine. of general mental imagery is associated with the 13, 4 (2013).
42. Naselaris, T., Olman, C. A., Stansbury, D. E., Ugurbil, K. occurrence of intrusive memories. J. Behav. Ther. Exp. 91. Slotnick, S. D., Thompson, W. L. & Kosslyn, S. M.
& Gallant, J. L. A voxel-​wise encoding model for early Psychiatry 44, 221–226 (2013). Visual mental imagery induces retinotopically
visual areas decodes mental images of remembered 66. Chao, L. L., Lenoci, M. & Neylan, T. C. Effects of post-​ organized activation of early visual areas. Cereb. Cortex
scenes. Neuroimage 105, 215–228 (2015). traumatic stress disorder on occipital lobe function 15, 1570–1583 (2005).
This study shows that mental imagery content can and structure. Neuroreport 23, 412–419 (2012). 92. Thirion, B. et al. Inverse retinotopy: inferring the visual
be decoded in the early visual cortex when the 67. Tavanti, M. et al. Evidence of diffuse damage in content of images from brain activation patterns.
decoding model is based on depictive perceptual frontal and occipital cortex in the brain of patients Neuroimage 33, 1104–1116 (2006).
features. with post-​traumatic stress disorder. Neurol. Sci. 33, 93. Horikawa, T. & Kamitani, Y. Generic decoding of seen
43. Fox, M. D. et al. The human brain is intrinsically 59–68 (2011). and imagined objects using hierarchical visual
organized into dynamic, anticorrelated functional 68. Kavanagh, D. J., Andrade, J. & May, J. Imaginary features. Nat. Commun. 8, 15037 (2017).
networks. Proc. Natl Acad. Sci. USA 102, 9673–9678 relish and exquisite torture: the elaborated intrusion 94. Keogh, R. & Pearson, J. Mental imagery and visual
(2005). theory of desire. Psychol. Rev. 112, 446–467 working memory. PLOS ONE 6, e29221 (2011).
44. Smith, S. M. et al. Correspondence of the brain’s (2005). 95. Keogh, R. & Pearson, J. The sensory strength of
functional architecture during activation and rest. Proc. 69. Ersche, K. D. et al. Abnormal brain structure voluntary visual imagery predicts visual working
Natl Acad. Sci. USA 106, 13040–13045 (2009). implicated in stimulant drug addiction. Science 335, memory capacity. J. Vis. 14, 7 (2014).
45. Østby, Y. et al. Mental time travel and default-​mode 601–604 (2012). 96. Aydin, C. The differential contributions of visual
network functional connectivity in the developing 70. Song, C., Schwarzkopf, D. S., Kanai, R. & Rees, G. imagery constructs on autobiographical thinking.
brain. Proc. Natl Acad. Sci. USA 109, 16800–16804 Reciprocal anatomical relationship between primary Memory 26, 189–200 (2017).
(2012). sensory and prefrontal cortices in the human brain. 97. Schacter, D. L. et al. The future of memory:
46. Andrews-​Hanna, J. R., Reidler, J. S., Sepulcre, J., J. Neurosci. 31, 9472–9480 (2011). remembering, imagining, and the brain. Neuron 76,
Poulin, R. & Buckner, R. L. Functional-​anatomic 71. Panizzon, M. S. et al. Distinct genetic influences 677–694 (2012).
fractionation of the brain’s default network. Neuron on cortical surface area and cortical thickness. 98. Tong, F. Imagery and visual working memory: one and
65, 550–562 (2010). Cereb. Cortex 19, 2728–2735 (2009). the same? Trends Cogn. Sci. 17, 489–490 (2013).

NATuRE REvIEWS | NeuroscIence volume 20 | OCTOBER 2019 | 633

Reviews

99. Berger, G. H. & Gaunitz, S. C. Self-​rated imagery and 116. Holmes, E. A. et al. Mood stability versus mood 135. Kosslyn, S. M. Mental images and the brain.
encoding strategies in visual memory. Br. J. Psychol. instability in bipolar disorder: a possible role for Cogn. Neuropsychol. 22, 333–347 (2005).
70, 21–24 (1979). emotional mental imagery. Behav. Res. Ther. 49, 136. Pylyshyn, Z. W. What the mind’s eye tells the mind’s
100. Harrison, S. A. & Tong, F. Decoding reveals the 707–713 (2011). brain: a critique of mental imagery. Psychol. Bull. 80,
contents of visual working memory in early visual 117. Tiggemann, M. & Kemps, E. The phenomenology of 1–24 (1973).
areas. Nature 458, 632–635 (2009). food cravings: the role of mental imagery. Appetite 45, 137. Pylyshyn, Z. Return of the mental image: are there
101. Borst, G., Ganis, G., Thompson, W. L. & Kosslyn, S. M. 305–313 (2005). really pictures in the brain? Trends Cogn. Sci. 7,
Representations in mental imagery and working 118. Connor, J. P. et al. Addictive behaviors. Addict. Behav. 113–118 (2003).
memory: evidence from different types of visual 39, 721–724 (2014). This review provides an updated summary of the
masks. Mem. Cognit. 40, 204–217 (2011). 119. May, J., Andrade, J., Panabokke, N. & Kavanagh, D. imagery debate.
102. Kang, M.-S., Hong, S. W., Blake, R. & Woodman, G. F. Visuospatial tasks suppress craving for cigarettes. 138. Chang, S. & Pearson, J. The functional effects of prior
Visual working memory contaminates perception. Behav. Res. Ther. 48, 476–485 (2010). motion imagery and motion perception. Cortex 105,
Psychon Bull. Rev. 18, 860–869 (2011). 120. Michael, T., Ehlers, A., Halligan, S. L. & Clark, D. M. 83–96 (2017).
103. Keogh, R. & Pearson, J. The perceptual and Unwanted memories of assault: what intrusion 139. Stokes, M., Thompson, R., Cusack, R. & Duncan, J.
phenomenal capacity of mental imagery. Cognition characteristics are associated with PTSD? Behav. Res. Top-​down activation of shape-​specific population codes
162, 124–132 (2017). Ther. 43, 613–628 (2005). in visual cortex during mental imagery. J. Neurosci.
This study shows a new method to measure the 121. Holmes, E. A., James, E. L., Kilford, E. J. & Deeprose, C. 29, 1565–1572 (2009).
capacity function of visual imagery and shows that Key steps in developing a cognitive vaccine against 140. Amit, E. & Greene, J. D. You see, the ends don’t justify
it is quite limited. traumatic flashbacks: visuospatial tetris versus verbal the means: visual imagery and moral judgment.
104. Luck, S. J. & Vogel, E. K. Visual working memory pub quiz. PLOS ONE 5, e13706 (2010). Psychol. Sci. 23, 861–868 (2012).
capacity: from psychophysics and neurobiology to 122. Shine, J. M. et al. Imagine that: elevated sensory 141. Dobson, M. & Markham, R. Imagery ability and
individual differences. Trends Cogn. Sci. 17, 391–400 strength of mental imagery in individuals with source monitoring: implications for eyewitness
(2013). Parkinson’s disease and visual hallucinations. Proc. R. memory. Br. J. Psychol. 84, 111–118 (1993).
105. Pearson, J. & Keogh, R. Redefining visual working Soc. B 282, 20142047 (2014). 142. Gonsalves, B. et al. Neural evidence that vivid
memory: a cognitive-​strategy, brain-​region approach. 123. Foa, E. B., Steketee, G., Turner, R. M. & Fischer, S. C. imagining can lead to false remembering. Psychol. Sci.
Curr. Dir. Psychol. Sci. 28, 266–273 (2019). Effects of imaginal exposure to feared disasters in 15, 655–660 (2004).
106. Greenberg, D. L. & Knowlton, B. J. The role of visual obsessive-​compulsive checkers. Behav. Res. Ther. 18, 143. Bird, C. M., Bisby, J. A. & Burgess, N. The
imagery in autobiographical memory. Mem. Cognit. 449–455 (1980). hippocampus and spatial constraints on mental
42, 922–934 (2014). 124. Hunt, M. & Fenton, M. Imagery rescripting versus imagery. Front. Hum. Neurosci. 6, 142 (2012).
107. Sheldon, S., Amaral, R. & Levine, B. Individual in vivo exposure in the treatment of snake fear. 144. Jones, L. & Stuth, G. The uses of mental imagery
differences in visual imagery determine how event J. Behav. Ther. Exp. Psychiatry 38, 329–344 (2007). in athletics: an overview. Appl. Prev. Psychol. 6,
information is remembered. Memory 25, 360–369 125. Holmes, E. A. & Mathews, A. Mental imagery and 101–115 (1997).
(2017). emotion: a special relationship? Emotion 5, 489–497 145. Dils, A. T. & Boroditsky, L. Visual motion aftereffect
108. D’Argembeau, A. & Van der Linden, M. Individual (2005). from understanding motion language. Proc. Natl Acad.
differences in the phenomenology of mental time 126. Zeman, A. Z. J. et al. Loss of imagery phenomenology Sci. USA 107, 16396–16400 (2010).
travel: the effect of vivid visual imagery and emotion with intact visuo-​spatial task performance: a case of 146. Christian, B. M., Miles, L. K., Parkinson, C. &
regulation strategies. Conscious Cogn. 15, 342–350 ‘blind imagination’. Neuropsychologia 48, 145–155 Macrae, C. N. Visual perspective and the characteristics
(2006). (2010). of mind wandering. Front. Psychol. 4, 699 (2013).
109. Vannucci, M., Pelagatti, C., Chiorri, C. & Mazzoni, G. 127. Ungerleider, L. G. & Haxby, J. V. ‘What’ and ‘where’ in 147. Palmiero, M., Cardi, V. & Belardinelli, M. O. The role
Visual object imagery and autobiographical memory: the human brain. Curr. Opin. Neurobiol. 4, 157–165 of vividness of visual mental imagery on different
object Imagers are better at remembering their (1994). dimensions of creativity. Creat. Res. J. 23, 372–375
personal past. Memory 24, 455–470 (2015). 128. Jacobs, C., Schwarzkopf, D. S. & Silvanto, J. Visual (2011).
110. Galton, F. Statistics of mental imagery. Mind 5, working memory performance in aphantasia. Cortex
301–318 (1880). 105, 61–73 (2017). Acknowledgements
This paper was the first formal empirical paper 129. Gray, C. R. & Gummerman, K. The enigmatic eidetic The author thanks R. Keogh, R. Koenig-​Robert and A. Dawes
investing imagery vividness, including the first image: a critical examination of methods, data, and for helpful feedback and discussion on this paper. This paper,
report of what is now called aphantasia. theories. Psychol. Bull. 82, 383–407 (1975). and some of the work discussed in it, was supported by
111. Holmes, E. A. & Mathews, A. Mental imagery in 130. Stromeyer, C. F. & Psotka, J. The detailed texture Australian National Health and Medical Research Council
emotion and emotional disorders. Clin. Psychol. Rev. of eidetic images. Nature 225, 346–349 (1970). grants APP1024800, APP1046198 and APP1085404,
30, 349–362 (2010). 131. Haber, R. N. Twenty years of haunting eidetic imagery: a Career Development Fellowship APP1049596 and an
112. Hackmann, A., Bennett-​Levy, J. & Holmes, E. A. where’s the ghost? Behav. Brain Sci. 2, 616–617 Australian Research Council discovery project grant
Oxford Guide to Imagery in Cognitive Therapy (Oxford (1979). DP140101560.
Univ. Press, 2011). 132. Allport, G. W. Eidetic imagery. Br. J. Psychol. 15,
113. Blackwell, S. E. et al. Positive imagery-​based cognitive 99–120 (1924). Competing interests
bias modification as a web-​based treatment tool for 133. Kwok, E. L., Leys, G., Koenig-​Robert, R. & Pearson, J. The author declares no competing interests.
depressed adults: a randomized controlled trial. Clin. Measuring thought-​control failure: sensory
Psychol. Sci. 3, 91–111 (2015). mechanisms and individual differences. Psychol. Sci. Peer review information
114. Crane, C., Shah, D., Barnhofer, T. & Holmes, E. A. 57, 811–821 (2019). Nature Reviews Neuroscience thanks D. Kavanagh, J. Hohwy
Suicidal imagery in a previously depressed community This study shows that, even when people think they and the other, anonymous, reviewer(s) for their contribution
sample. Clin. Psychol. Psychother. 19, 57–69 (2011). have successfully suppressed a mental image, it is to the peer review of this work.
115. Hales, S. A., Deeprose, C., Goodwin, G. M. still actually there and biases subsequent perception
& Holmes, E. A. Cognitions in bipolar affective disorder (a possible candidate for unconscious imagery). Publisher’s note
and unipolar depression: imagining suicide. Bipolar 134. Kosslyn, S. M. Image and Mind (Harvard Univ. Springer Nature remains neutral with regard to jurisdictional
Disord. 13, 651–661 (2011). Press, 1980). claims in published maps and institutional affiliations.

634 | OCTOBER 2019 | volume 20 www.nature.com/nrn