Training Spatial Skills:

A Meta-analysis

Linda Liu Hand, David H. Uttal & Loren Marulis

Northwestern University

Nora S. Newcombe
Temple University

1
Importance of training?
• Potential to improve skills relevant to STEM (Hedges &
Chung, in prep; Shea, Lubinski & Benbow, 2001)
– High spatial ability: More likely to have STEM major and STEM job
– Can also reduce disparities in STEM achievement
• How and how much?
• Goal: To aggregate systematically past research on
spatial training to determine consensus in literature.

2
http://www.spatialintelligence.org
Overview
 What is training?
 How can we compare training effectiveness across
studies?
 Research questions:
1. How much do (vs. can) spatial skills improve?
• Might vary by task – Embedded Figures vs. Water-level Task?
2. What works?
• Impact of grouping variables
3. Are training effects durable?
4. Does training generalize (transfer) to untrained tests?

3
http://www.spatialintelligence.org
Examples: Video games
• Effect of playing videogames (Tetris) on mental rotation and
Paper Folding Test (Wright, Thompson, Ganis, Newcombe &
Kosslyn, 2008).

MRT (g = 1.09) PFT (g = .87)

4
http://www.spatialintelligence.org
Examples: Spatial experience
• Effects of various life experiences on spatial skills:
– Fashion designers: Effect of experience with pattern-
making on spatial skills (Workman, Caldwell & Kallal, 1999)

Dress pattern making Differential Aptitude Test-Spatial Relations

(g = .32)

5
http://www.spatialintelligence.org
Examples: Spatial coursework
Engineering course using Improved Purdue Spatial
multi-media software and Visualization Test
workbook performance (Sorby, 2008)
•Isometric pictorials from coded plans
•Multi-view drawings
•Paper folding/2-D to 3-D transformations
•Object rotations about one axis
•Object rotations about two or more axes
•Cutting planes and cross sections
•Surfaces and solids of revolution
•Combining solids

g = 2.02
6
http://www.spatialintelligence.org
Examples: Repeated practice
Repeated practice on different Group Embedded Figures
(Chance & Goldstein, 1971; Schaeffer & Thomas, 1999)

Pretest GEFT Training Posttest GEFT

g = 1.12

http://www.spatialintelligence.org
Methods
• Searched for both published and unpublished work:
– Dissertations, conference posters, technical reports.
– Electronic searches, references lists, direct contacts
• Coded on several grouping variables, including:
– Age, sex, ability level (i.e., prescreened for low
performers?)
– Outcome measure, type of training
– Publication status, random assignment, location of study
(classroom?), feedback provided (yes/no), training
frequency

8
http://www.spatialintelligence.org
Effect Sizes
• Standard measure of efficacy across studies
– Does not depend on individual measurement (raw score)
– Expresses mean change, as a result of training or
experience, in standard deviation units.
• Final “sample”
– 101 published (76) and unpublished (25) studies

9
http://www.spatialintelligence.org
Analysis Plan
• How do we make sense of various training methods and
dependent variables?
• Created 5 conceptual categories of dependent
variables and 3 categories of training.
• Describe each category then compare size of training
effects in each category.

10
http://www.spatialintelligence.org
• Categories of dependent variables

Spatial Perceive objects Example: Mazes,

perception amidst distracting Embedded Figures Task
background
Perspective taking Visualize a scene from a
different location
Assembly/ Put together objects into
transformation larger config. or
transform (3-D to 2-D)
Mental rotation Rotation of 2-D or 3-D
pictures or objects
Spatial principles Understand abstract
principles (e.g.,
11 horizontality)
http://www.spatialintelligence.org
• Categories of dependent variables

Spatial perception Perceive objects amidst Example:

distracting background Three mountains task
Perspective Visualize a scene from
taking a different location
Assembly/ Put together objects into
transformation larger config. or
transform (3-D to 2-D)
Mental rotation Rotation of 2-D or 3-D
pictures or objects
Spatial principles Understand abstract
principles (e.g.,
horizontality)
12
http://www.spatialintelligence.org
• Categories of dependent variables

Spatial perception Perceive objects amidst Example:

distracting background Form Board Test
Perspective taking Visualize a scene from a
different location
Assembly/ Put together objects
transformation into larger config. or Yes
transform (3-D to 2-D) or
Mental rotation Rotation of 2-D or 3-D No?
pictures or objects
Spatial principles Understand abstract
principles (e.g.,
horizontality)
13
http://www.spatialintelligence.org
• Categories of dependent variables

Spatial perception Perceive objects amidst Example:

distracting background MRT, Card rotation
Perspective taking Visualize a scene from a
different location
Assembly/ Put together objects into
transformation larger config. or Same or
transform (3-D to 2-D) different?
Mental rotation Rotation of 2-D or 3-D
pictures or objects
Spatial principles Understand abstract
principles (e.g.,
horizontality)
14
http://www.spatialintelligence.org
• Categories of dependent variables

Spatial perception Perceive objects amidst Example:

distracting background Water level task
Perspective taking Visualize a scene from a
different location
Assembly/ Put together objects into
transformation larger config. or
transform (3-D to 2-D)
Mental rotation Rotation of 2-D or 3-D A B
pictures or objects
Draw a line in B
Spatial principles Understand abstract for the water.
principles (e.g.,
horizontality)
15
http://www.spatialintelligence.org
Outcome Description Examples of Linn and Carroll
category measures Petersen (1993)
(1985)
Spatial Perceiving objects, Embedded Figures Spatial Visuospatial
perception paths, or spatial Task, Visualization Perceptual
configurations amidst Mazes Speed
distracting background
information.
Assembly/ Piecing together objects Form Board, Spatial Spatial
Transformation into more complex Block Design, Visualization Visualization
configurations or Paper Folding
visualizing and mentally
transforming objects,
often from 2D to 3D or
vice versa.
Rotation Rotate 2D or 3D objects. Vandenberg Mental Spatial
Mental Rotation, Rotation Relations/
Cube Comparison, Speeded
Card Rotation Rotation
Spatial Understanding abstract Water-level Task, Spatial Not included
principles spatial principles, such Plumb-line Task Perception
as horizontal invariance
or verticality.
Perspective Visualizing an Piaget’s Three Not included Not included
taking environment in its Mountains Task
entirety from a different
position.
Types of Training

Video games Designed for Example: Tetris, Zaxxon

recreation and
entertainment.
Courses Full-length or short-term
enhanced courses.
Spatial task Direct rehearsal or
training - Specific practice on outcome
measure of interest.
Spatial task Transfer of training to
training- Transfer reference tests.

17
http://www.spatialintelligence.org
 Video games Designed for recreation Example: Dress-making,
and entertainment. spatial modules, Drafting
(vs. water purification)
Courses Full-length or short-
term enhancements.
Spatial task Direct rehearsal or
training - Specific practice on outcome
measure of interest.
Spatial task Transfer of training to
training- Transfer reference tests.

18
http://www.spatialintelligence.org
 Video games Designed for recreation Example: Repeated practice
and entertainment. on the GEFT, VMRT, WLT

Courses Full-length or short-term

enhancements.
Spatial task Direct rehearsal or
training - Specific practice on outcome
measure of interest.
Same or
Spatial task Transfer of training to different?
training- Transfer reference tests.

19
http://www.spatialintelligence.org A B
 Video games Designed for recreation Example: Regular WLT 
and entertainment. test on irregular WLT;
Tetris  test on PFT
Courses Full-length or short-term
enhancements.
Spatial task Direct rehearsal or
A B
training - Specific practice on outcome
measure of interest.
Spatial task Transfer of training to
training -Transfer reference tests.

20
http://www.spatialintelligence.org
 Results
• Overall effectiveness of training
• Control group effects
• Age and Sex
• Are some kinds of training better than
others? Are some outcome measures more
malleable than others?
• Duration
• Transfer
Overall Effectiveness

• 101 studies
– Mean effect size = .65 (i.e., 2/3 a SD of improvement)
– “Moderate” improvement (Cohen, 1988)

For IQ (SD = 15), .65 SD

would be an increase of
9.75 points.
Control group effects
Experimental groups do significantly exceed
control groups
• Overall, treatment groups improve significantly more from
training than control groups do.

E
xpe
rim
enta
lgro
up

Control groups Treatment groups

g = .56 g = .75

0
.00
0 1
.00
0 2
.00
0

24 g
g
http://www.spatialintelligence.org
Experimental groups do significantly exceed
control groups
• Treatment groups improve more in nearly all cases:

Control Treatment C vs. T

sig.
Outcome category g (SE) N g (SE) N
Spatial perception .65 (.10) 11 .96 (.11) 11 p < .05
Perspective taking .46 (.12) 5 .89 (.10) 5 p < .01
Assembly/transform. .71 (.05) 25 .78 (.05) 25 n. s.
Spatial principles .18 (.09) 7 .75 (.07) 7 p < .001
Mental rotation .51 (.04) 31 .67 (.04) 31 p < .01
AVERAGE .56 (.03) 55 .75 (.03) 55 p < .001

† Homogeneity achieved
ab
Groups labeled with different superscripts are significantly different.
* Age x Control group type χ2 significant, p < .05
But,….

• Type of control group and improvement in control group

really matters for understanding
– Overall effectiveness
• What works depends on what did or did not happen to the
control group
– Different effects of different types of training

26
http://www.spatialintelligence.org
Why control groups matter

• Crestor = New anti-cholesterol

drug.
• Similar drug Vytorin halted =
Clinical trial failed to show it was
any better than an already-
available medication.
• Difference?
– Vytorin study: Head-to-head
comparison with another
drug.
– Crestor study based on
comparing its effectiveness
to placebo = nothing

27
http://www.spatialintelligence.org
Why control groups matter
• Important to separate control and treatment groups
– Spatial principles highest Ec effect size, lowest Control group g
– Spatial perception lowest Ec effect size, highest g for Control.
1.2

1 0.95

0.76
0.8
A B 0.64
Mean effect
0.6
size (g)
0.4
0.18
0.2

0
Spatial principles Spatial perception

Control Treatment
28
http://www.spatialintelligence.org
Why control groups improve so much

• Classic sources of test-retest effects

– Understanding the task (e.g, which key to press when)
– Test-specific strategies (e.g., eliminating foils quickly;
keeping fingers on the correct keys; looking for similar
problems)
• But also, the possibility of more interesting learning from
the tests
– Fluency in finding correct structures
– Better allocation of attention and working memory
– Multiple tests provide a form of indirect training
• Alignment and comparison (Gentner & Markman, 1994, 1997)
29
http://www.spatialintelligence.org
Our specific claim

• Some of the learning in the control groups is not just of the

boring type
• Some people learn something from taking the tests
• Points to malleability of spatial skill
– Even without instruction, just a chance to practice,
people improve, often rather dramatically

30
http://www.spatialintelligence.org
What’s the evidence to support this claim?

• Acknowledge: Post-hoc—NEED EXPERIMENTAL

RESEARCH
• But….
• Magnitude of control group improvement is about twice as
great as for other tests
• Control groups show transfer!
– Hard to explain on basis of “boring” effects alone
• Variation helps

31
http://www.spatialintelligence.org
Test variety is effective training
• Test-retest effect: Not just number of repetitions
• Number of separate tests given during pretest-posttest:
0.9 0.78
0.8
Mean effect size (g)

0.7 0.59
0.6 0.49
0.5
0.4
0.3
0.2
0.1
0
One measure 2 - 4 measures 5 or more
measures
Number of test-retest measures per study
32
http://www.spatialintelligence.org
Age effects: It’s all in the control group
• Does malleability vary by age?
– On average, effect size significantly higher for children
than for adults, p < .05
– Initially, appears that children are more malleable…

33
http://www.spatialintelligence.org
Age effects: Control and Experimental

• When looking at control groups, adult control

groups improve significantly more.
• Thus, children may appear more malleable because
their control groups improve less than adults’ do.

Age Treatment Group Control Group

Effect Size Effect Size
Children .70 (.05) .42 (.05)
Adults .77 (.03) .62 (.03)
(ns) p < .01

34
http://www.spatialintelligence.org
Sex differences
• Does malleability vary by sex? No difference in mean
effect size, both sexes respond to training (same g)
• Overall, results from prior work are most consistent
with last scenario.
• Male advantage is similar in magnitude at pre and post

M M
F
F

Training  Training 
35
http://www.spatialintelligence.org
Training works – What works?
• Focus on treatment effect sizes:
Grouping variable Results
Outcome measure Outcomes largely similar in malleability: Only
significant difference: Spatial perception (.96) >
mental rotation (.67).
Training frequency More frequent training  larger g for mental
rotation only (.81 for multi-session vs. .38 for single).
Feedback during FB led to larger effect sizes for most outcome
training measures except spatial perception (opposite is true)
Random assignment Led to lower effect sizes (more rigorous).

Variables involved in Age, Publication status, Location (classroom or not)

NO significant effects
36
http://www.spatialintelligence.org
Training works – What works?
• Focus on treatment effect sizes:
Type of training

Courses Full-length superior to short-term enhancement

• Due to training being distributed (over longer period
of time, in more sessions)
• Even though total number of hours of training did
not differ for the two.
Video games Results similar for all outcome measures, but video
games involving mental rotation produced
especially strong training effects.

37
http://www.spatialintelligence.org
Duration: Training lasts
• Majority of studies (85%) tested only immediate impact of training.
• Among treatment groups: No significant decline in effect size
measured immediately, 2 weeks after, or more than 2 weeks after
end of training (which includes up to 3 months later).
1
0.9
0.8 0.76
0.7
Effect size (g)

0.59 0.57
0.6
0.5
0.4
0.3
0.2
0.1
0
Immediate test Up to 2 weeks More than 2
38
http://www.spatialintelligence.org
weeks
Training transfers
• Why does this matter?
– Suggests training is NOT just a practice effect
– If spatial training has effects that extend beyond mere
practice, training should transfer to untrained tasks.
• Near vs. Far transfer:
– Near g = 1.01
– Far g = .56 A B
– But Far is more durable.

39
http://www.spatialintelligence.org
Training transfers
• Studies expecting to obtain far transfer might use training
that produces especially durable effects:
1.2
1.05
1
Effect size (g)

0.8 0.72
0.58
0.6
0.37
0.4

0.2

0
Near Far
Transfer type

Immediate test Delayed test

40
http://www.spatialintelligence.org
Real impact of training?

Real value of a .65 SD increase?

Marginal improvements on raw scores may lead to important
gains in other areas:
• Increase of .65 SD in height = 1.63 inches in height
(among females18-24 yrs)?
• Increase of .65 SD on the LSAT = 3 points (Average
score is 156/180 among current law school students)?

• Value of 1 inch increase above average? $789 per year 1

• Value of 1 point gain on LSAT? $2,600 in starting salary 2

1 Berkowitz, Ruth. "One Point on the LSAT: How Much Is It Worth?" American
Economist 42 (2) 1998.
2 Judge, T. A., & Cable, D. M. “The Effect of Physical Height on Workplace

Success and Income” Journal of Applied Psychology, 89(3) 2004.

 M
F

Training 

42
http://www.spatialintelligence.org
Conclusions
• Training leads to improvements in spatial skills that are:
– Durable - No significant losses in pretest-posttest
improvement, even when retested 3 months later.
– Generalizable to other tasks – Training leads to
improvements on untrained tasks.

43
http://www.spatialintelligence.org
Conclusions
• How much can spatial skills improve?
– Use longer periods of training
• 47% of studies performed only one single session of training
• 85% conducted only an immediate posttest
• When long periods of training are used, durable effects AND far
transfer are observed.
– Test a larger range of outcome measures
• 48% of outcome measures are mental rotation
• Vs. 9% perspective taking, 11% spatial principles, etc.
– Include a variety of methods of training
• Allows for alignment and comparison across problems (Gentner
& Markman, 1994, 1997)

44
http://www.spatialintelligence.org
Future directions
• To develop best-practice guidelines for spatial
interventions at elementary and high school levels.
• Investigate transfer to STEM in more detail.
• Understand thresholds for success

45
http://www.spatialintelligence.org
Acknowledgements
• Larry Hedges (NU)
• Spyros Konstantopoulos (NU)
• David B. Wilson (George Mason University)
• Chris Warren and Alison Lewis
• Research assistance:
– Kate O’Doherty
– Bridget O’Brien
– Eleanor Tushman
– Maggie Carlin
– Laura Mesa, Bonnie Vu, Melissa Sifuentes

46
http://www.spatialintelligence.org