10

Introduction to the Ecological

Approach to Visual Perception

The approaches to understanding visual percep- is a perfectly reasonable assumption to make.

tion that we discussed in the second part of this Psychophysical methods would be an adequate
book rest on a common assumption that the func- means of studying perception, using verbal or
tion of perception is to provide us with awareness nonverbal reports of people’s perceptual experi-
of the surrounding world. With our eyes open, we ences to probe a common set of processes that are
have an immediate impression of a solid world of involved in all uses of vision.
surfaces, objects, and people, and of their shapes, In Chapter 3 (p. 54) we saw that physiological
movements, or identities. From the origins of evidence has challenged this assumption, by
thinking about visual perception through to showing that visual information is processed in at
modern vision science, the goal has been to least two partially independent neural pathways.
understand how such impressions are obtained One pathway provides a fast link between the
from a fluctuating pattern of light falling on the early stages of visual processing and the control
retina. of actions such as reaching and grasping, and in
Obviously, perception must do more than cre- some circumstances this pathway can operate
ate a flow of inner, private impressions of the independently of conscious visual awareness. The
world. Much of the time, we are engaged in activ- implication of these discoveries is that the role of
ities such as walking, handling objects, or com- visual perception in the control of action is a
municating with other people. These actions all problem for research in its own right.
require body movements to be controlled by Some years before these physiological
information obtained through perception. Trad- advances were made, theoretical arguments were
itionally, this role of perception has been seen as a being developed by James Gibson that would lead
secondary one, which can be addressed once we to a stronger conclusion still, that perception and
understand how the representation of the world action are intimately bound up with each other
underlying our conscious awareness of it is cre- and cannot be treated as separate problems. In
ated. If all the processes involved in controlling this chapter, we will review these developments
our actions draw on this representation, then this from a historical viewpoint. We will first describe

301
302 VISUAL PERCEPTION

the theory that Gibson developed over a 35-year his subsequent books. Gibson’s theory
period in three major books (Gibson, 1950a, emphasises the ground on which an animal lives
1966, 1979; see also Reed & Jones, 1982), and and moves around, or above which an insect,
then outline the further developments made by bird, or pilot flies. The ground consists of surfaces
his followers in the “ecological” school of psych- at different distances and slants. The surfaces are
ology. In the following chapters of Part III, we composed of texture elements. Pebbles, grains of
will describe theory and research that address sand, or blades of grass are all elements of texture
specific aspects of the links between visual per- that, while not identical, possess statistical regu-
ception and the control of action. While early larity—the average size and spacing of elements
studies of these problems often started from a of the same kind of texture will remain roughly
strongly ecological theoretical position, interest in constant for different samples. Some surfaces sur-
the visual control of action has recently spread round objects, and these objects may be attached
more widely. We will therefore draw on a wider to the ground (e.g., rocks and trees), or detached
range of theoretical approaches to these problems and independently mobile (e.g., animals). Object
in later chapters than the specifically Gibsonian surfaces, like ground surfaces, have texture. The
one that we sketch in this chapter. environment thus consists of textured surfaces
that are themselves immersed in a medium (air).
Gibson argues that we need an appropriate geom-
etry to describe the environment, which will not
J.J. GIBSON’S THEORY OF PERCEPTION
necessarily be one based on abstractions such as
“points” and “planes”, as conventional geom-
During World War II Gibson addressed himself etries are. An ecological geometry must take
to the problem of how to train pilots quickly, or surfaces and texture elements as its starting point.
how to discriminate potentially successful from
unsuccessful pilots prior to training. The most A surface is substantial; a plane is not. A
difficult, and hence dangerous, aspects of flying surface is textured; a plane is not. A surface
are landing and take-off. To land a plane success- is never perfectly transparent; a plane is. A
fully you must know where you are located rela- surface can be seen; a plane can only be
tive to the air strip, your angle of approach, and visualized (Gibson, 1979, p. 35).
how to modify your approach so that you are aim-
ing for the right position at the right speed. Gib- The structure that exists in the surfaces of the
son therefore felt that good depth perception was environment in turn structures the light that
likely to be a prerequisite of good flying. He dis- reaches an observer, as we saw in Chapter 1. Gib-
covered, however, that tests based on the pictorial son argues that it is the structure in the light, rather
cues to depth, and training measures devised to than stimulation by light, that furnishes informa-
make people capitalise on depth information, had tion for visual perception. Stimulation per se does
little success when applied to the problem of not lead to perception, as evidenced by perceptual
training pilots. Here was a clear practical example experience in a Ganzfeld (Metzger, 1930; Gibson
of the perception of relative distance, and yet & Dibble, 1952; Gibson & Waddell, 1952). Dif-
attempts to improve “depth perception” were fuse unstructured light, as might be obtained by
fruitless. placing halves of table-tennis balls over the eyes
Such observations led Gibson to reformulate and sitting in a bright room, produces perception
his views of visual perception radically. In his of nothingness. To perceive things, rather than
1950 book he began by suggesting that the clas- no-thing, the light must be structured. In order to
sical approach to “depth” or “space” perception describe the structure in light we need an “eco-
should be replaced by an approach that logical” optics (Gibson, 1961), rather than a
emphasised the perception of surfaces in the description at the level of the physics of photons,
environment. This emphasis remained throughout waves, and so on. The physics of photons coupled
 10. THE ECOLOGICAL APPROACH 303

with the biochemistry of photoreceptor action

can be used to explain how light is emitted and
propagated and how receptors are stimulated by
it, but not how the world is perceived. An eco-
logical optics must cut across the boundaries
between physical and physiological optics and the
psychology of perception.
Gibson rejected the claim that the retinal
image is the starting point for visual processing.
He argued that the whole array of light rays
reaching an observer, after structuring by surfaces
and objects in the world, provides direct informa-
tion about the layout of those surfaces and
objects, and about movement within the world
and by the observer. In Chapter 1 we described
how light is structured in the optic array, and here
we remind you briefly of the important points.
The ambient optic array at any point above the
ground consists of an innumerable collection of
light rays of different wavelengths and intensities. Examples of texture density gradients.
Some have been emitted from light sources such
as the sun, while others have been reflected from
surrounding surfaces. These rays form a hier-
archical and overlapping set of solid angles. The
solid angles corresponding to the tiniest texture
elements are nested within those that correspond
to the boundaries of larger regions or objects.
Changes in the pattern or properties of the light
from one solid angle to another signal boundaries
in the world, where for example one object par-
tially conceals or occludes another object, or the
ground.
In his early work, Gibson considered in par-
ticular how textured surfaces structure the light
reflected from them, and how the resulting struc-
ture in the optic array could inform an observer
about the shapes and orientations of those sur- Surface shape and slant can be revealed by texture.
faces. The ground surfaces on which we move
usually have statistically regular texture, made up In Chapter 7 (p. 187) we saw that simulations
of elements of similar size and shape. Light of textured surfaces can give strong impressions
reflected from such surfaces will form a structure of solid shape and enable people to make judge-
in the optic array known as a texture gradient. ments of the relative depths of the objects that
Figure 10.1 shows examples of how texture gradi- they suggest. For Gibson, however, texture gra-
ents (of artificially regular proportions) can give dients are not just a cue to depth that can be
impressions of surfaces receding into the dis- combined with others, but a more fundamental
tance. Figure 10.2 shows how the local shape of a property of ecological optics that specifies
surface may be given by the change in texture unambiguously the layout of a perceiver’s sur-
gradient. roundings. For example, changing the slant of a
304 VISUAL PERCEPTION

textured surface relative to a perceiver will pro- tance of an object is given by its point of attach-
duce changes in the texture gradients in the optic ment to the ground relative to this scale.
array that could specify the slope of the ground Recently, Sinai, Ooi, and He (1998) demon-
leading away from a perceiver. Gibson and co- strated that people’s judgements of the distances
workers (Beck & Gibson, 1955; Gibson 1950b; of objects are influenced by the structure of the
Gibson & Cornsweet, 1952) showed that the per- ground surface on which they rest. Sinai et al.
ceived slant of a simulated surface is influenced asked participants to judge the remembered dis-
by its texture gradient. The later research that we tance of an object several metres away by match-
described in Chapter 7 has identified more pre- ing the distance of another object or by walking
cisely the properties of texture gradients that are the same distance blindfolded. Judgements were
responsible for the perception of surface orienta- accurate when the object lay on the same flat, uni-
tion, using sophisticated computer graphics form ground surface as the participant, but
techniques to build on Gibson’s original insights. showed systematic errors if there was a hole in the
A second way in which Gibson argued that ground or a boundary between two textures (e.g.,
texture gradients are important for perception is concrete and grass) lying between the participant
in providing information about the distances and and the object.
sizes of objects. A traditional analysis of the The problem of perceiving the size of an
problem of perceiving size and distance would object can also be solved, Gibson argued, because
resemble that shown in Figure 10.3, which illus- an object will cover the same amount of back-
trates how the same image could potentially be ground texture whatever its distance from the
cast by an infinite number of objects of different observer (strictly, this is true only for the base of
sizes, inclinations, and distances from the obser- an object that is standing on the ground). This
ver. Gibson argued that this kind of analysis is theory provides an explanation of the phenom-
mistaken because it treats objects as geometric enon of size constancy; the fact that we see an
forms floating in empty air rather than as solid, object as the same size whatever its distance from
textured objects lying on a textured ground sur- us. Traditional theories sought to explain size
face. Because he stressed the importance of con- constancy in terms of a scaling of retinal image
sidering objects in the context of a background size using cues to distance. In the laboratory, the
surface, Gibson (1950a) termed his own theory a perceived size of distant objects tends to decrease
“ground” theory of perception in contrast with with increasing distance, in the same way as image
traditional “air” theories. An “air” theory implies size does, suggesting that a scaling mechanism
that the size or distance of an object can only be produces a distortion in these conditions. How-
perceived if the other quantity is known in ever, Gibson (1947) showed that in an open,
advance, but Gibson argued that this problem ploughed (and hence textured) field, estimates of
disappears as soon as a textured ground surface is the height of a distant stake merely became more
introduced. Gradients of texture in a receding variable at great distance rather than decreasing.
surface provide a continuous scale, and the dis- Size constancy also fails if we view objects from a

The kind of drawing typically used

by students of visual perception to
illustrate the ambiguity of the retinal
image.
 10. THE ECOLOGICAL APPROACH 305

height, rather than at a horizontal distance. Thus, structure, is a myth, or at least a limiting
from the top of a high building people on the case. Invariants of structure do not exist
pavement below us appear insect-like in their except in relation to variants (Gibson, 1979,
proportions. Traditional theory would explain p. 87).
this in terms of the absence of cues to distance.
Gibson would say that when viewing from a Variants in information are produced by
height, the absence of the ground removes the movement of the observer and the motion of
continuous scale of texture necessary for accurate objects in the world. The fact that observers
size perception. actively explore their world allows powerful
These examples of the role of texture gradients information from motion perspective to tell them
in perception illustrate an important concept in both about their position relative to structures in
Gibson’s theory, that of invariants in the structure the world and about their own movements. When
of the optic array. An invariant is a property of an observer moves (as in Figure 10.4) the entire
the array that is determined by some specific optic array is transformed. Such transformations
property of the world and remains constant as contain information about both the layout and
other conditions vary. For example, gradients in shapes of objects and surfaces in the world, and
several properties of optic texture depend in regu- about the observer’s movement relative to the
lar and predictable ways on the slant of a surface, world.
but are invariant with changes in the nature of the
texture elements themselves, for example whether Perception of the world and of the self go
they are pebbles, cobbles, or tiles. Similarly, the together and only occur over time (Gibson,
amount of background texture covered by an 1975, p. 49).
object increases with its size but is invariant with
its distance. The cornerstone of Gibson’s theory Figure 10.5 shows an example of motion per-
was the argument that lawful relationships such spective. As an observer walks past a collection of
as these imply that properties of the optic array objects at different distances the relative motions
can unambiguously specify properties of the present in the changing optic array will be specif-
world. Traditional theories, Gibson argued, ically correlated with the layout of such objects.
ignored the structure of a perceiver’s natural sur- Indeed as an observer moves in any way in the
roundings, and erroneously concluded that the world this locomotion will always be accom-
structure of light is inadequate to specify proper- panied by flow in the optic array. The nature of
ties of the world and has to be supplemented by optic flow patterns is specific to certain types of
inference and stored knowledge. A full discussion movement, (see Figures 10.6–10.8). If a pilot is
of the concept of invariants and its significance trying to land an aeroplane (Figure 10.6) there
for visual perception can be found in Cutting will be streaming in the optic array radiating out
(1986). from the point at which he or she is aiming. This
So far, we have described Gibson’s analysis of point is known as the centre of expansion, or pole,
how the optic array sampled by an observer at of the optic flow field. The array of optical tex-
any instant of time can provide information ture elements (produced by light reflected from
about the surrounding world. In doing so, we the texture elements in the world) expands centri-
have omitted entirely another important aspect of fugally, with elements successively passing out of
his theory, namely his argument that transform- the bounded visual field of the observer and new
ations in the optic array produced by the active elements emerging at the centre of expansion. If
movement of an observer are essential for visual one was sitting on the roof of a train facing
perception. backwards there would be a continuous inward
streaming of optical texture elements towards the
What is clear to me now that was not clear point from which one was travelling (Figure 10.7).
before is that structure as such, frozen If you chose the softer option of remaining seated
306 VISUAL PERCEPTION

When an observer moves

the entire optic array is
transformed. From Gibson
(1966). Copyright © 1966 by
Houghton Mifflin Company.
Reprinted with permission.

Successive views of a row of

fence posts as an observer
moves past them. The
observer travels from right to
left between each of the frames
from left to right. From Gibson
(1950a). Copyright © 1950 by
Houghton Mifflin Company.
Reprinted with permission.

at a train window the flow pattern would be as in 4. A shift of the centre of outflow from one visual
Figure 10.8. solid angle to another specifies a change in the
Gibson (1979) described the relationship direction of locomotion, a turn, and a remain-
between optic flow and locomotion more form- ing of the centre within the same solid angle
ally in the following way (abridged from Gibson, specifies no change in direction.
1979, pp. 227–229):
These arguments for the power of optic flow to
1. Flow of the ambient array specifies locomotion provide animals or people with information about
and nonflow specifies stasis. their own motion through the world have been
2. Outflow specifies approach and inflow specifies very influential, and have inspired a large body of
retreat from. research on the visual control of movement that
3. The focus or centre of outflow specifies the dir- we will consider in Chapters 11 and 12. However,
ection of locomotion in the environment. Gibson saw the role of perceiver movement as
 10. THE ECOLOGICAL APPROACH 307

The optic flow field for a pilot

landing an aeroplane. From
Gibson (1950a). Copyright ©
1950 by Houghton Mifflin
Company. Reprinted with
permission.

The optic flow field for a person sitting

on the roof of a train, facing
backwards.

more fundamental, relevant not just to the con- Receptors are stimulated whereas an organ
trol of locomotion but to all aspects of visual is activated . . . the eye is part of a dual
perception. This is expressed in his notion of organ, one of a pair of eyes, and they are
perceptual systems to contrast with the traditional set in a head that can turn, attached to a
“senses”. Gibson (1966, 1979) claimed that it was body that can move from place to place.
an entire perceptual system whose job it is to These organs make a hierarchy and consti-
“see” (1979, p. 53): tute what I have called a perceptual system.
308 VISUAL PERCEPTION

The optic flow field for a person sitting

on a train and looking out of the
window as they travel from right to left
through this terrain.

Movement by the observer, whether of body, If the object changes its distance from the obser-
head, or eyes, is one way in which variant infor- ver this change will be accompanied by magnifica-
mation is obtained. The other way is through tion (if approaching) or minification (if receding)
motion or change in objects in the world—i.e., of the texture elements of its own surface, and the
through events. Events include objects or animals covering up or uncovering of texture elements of
translating, rotating, colliding, or growing, chan- the background. Texture elements that are
ging colour, or disappearing. All such events are covered up by object motion in one direction are
accompanied by disturbances in the structure of uncovered by motion in the reverse direction. The
the optic array. Rigid translation of an object same is true of observer movement. Texture elem-
across the field of view involves the progressive ents that pass out of the observer’s view when
accretion, deletion, and shearing of texture elem- movement is in one direction will reappear if the
ents. An object will progressively cover up (or movement is reversed. Gibson claims that this
“wipe out”) texture elements in the direction of principle of reversible occlusion underlies the
its movement, uncover (or “unwipe”) them from observer’s impression of a constant and stable
behind and shear the elements crossed by the visual world where even those surfaces moment-
edges parallel to its movement (see Figure 10.9). arily hidden are still “perceived”.

As an object moves, elements

of background texture are
progressively wiped out
(covered up) by its leading
edge, unwiped (revealed) by
its trailing edge, and sheared
by edges parallel to its
direction of movement.
 10. THE ECOLOGICAL APPROACH 309

Once one considers the total array of light To primitive man each thing says what it is
there is no ambiguity about whether it is oneself and what he ought to do with it . . . a fruit
or objects in the world that are moving. Eye says ‘Eat me’; water says ‘Drink me’; thun-
movements do not change the structure of the der says ‘Fear me’.
ambient optic array, but only allow a different
portion of the array to be sampled. Movement of A sawn-off tree trunk of the right flatness
the head and body is always accompanied by a and size affords “sitting-on” by a human, or
systematic flow pattern in the total array. Move- “hopping-on” for a frog; if a surface is flat, ex-
ment of an object within the world produces local tended, and substantial its property of affording
disturbances in the structure of the array. Thus support to terrestrial animals is implicitly given.
the major distinction between movement within Gibson makes the strong claim that there is
the world or by the observer can be specified information in the light to specify the affordances
unambiguously by different flow patterns in the of the environment (Gibson, 1979, p. 127):
optic array. On the basis of this logical argument,
Gibson concluded that eye movements are not This is a radical hypothesis, for it implies
taken into account when we perceive motion. As that the ‘values’ and ‘meanings’ of things in
we saw in Chapter 8, however, empirical evidence the environment can be directly perceived.
tells us that they are (see p. 263).
Gibson’s approach to the psychology of per- An illustration of the concept of affordance is
ception became progressively more radical. provided by Warren (1984), who studied people’s
Although his 1950 book was a major departure judgements of whether staircases with differently
from earlier approaches, it did not directly chal- proportioned steps, depicted in pictures, could be
lenge the assumption that the function of percep- climbed in the normal way or not. Participants
tion is to create an awareness of the world that taller and shorter than average differed in their
can be tapped by psychophysical methods. In his judgements, which proved to be determined by
later work, however, Gibson (1966, 1979) the ratio of step height to the individual partici-
developed more and more strongly the argument pant’s leg length. Their judgements were therefore
that perception is “direct”. If there are properties influenced both by the geometry of the steps that
of the transforming optic array that can they viewed and by the mechanical properties of
unambiguously specify properties of the sur- their own limbs; in Gibson’s terms, they were sen-
rounding world, Gibson argued, then there is no sitive to the affordance of “climbability”. While
need for cognitive processes to interpret visual these are important findings, they cannot test
input. In time, he extended this argument from Gibson’s wider theory directly. They may imply
specific contexts such as the perception of dis- that people pick up directly an invariant property
tance, size, or slant to a general rejection of of the pattern of light reflected from a staircase
cognitive processes and representations in the that specifies “climbability”, but it could equally
explanation of perception. He claimed that the well be that inferential processes are involved in
end product of perception is not an internal rep- relating the perceived dimensions of the steps to
resentation of the visual world—a “percept” of information about stride length held in memory.
which a person is consciously aware—but the While it is relatively easy to appreciate that
detection of affordances. The affordance of some affordances like “climbable” or “graspable”
surface or object in the environment is what it might be specified in the optic array, it is much
offers the animal—whether it can be grasped or less easy to appreciate how qualities such as “eat-
eaten, trodden on or sat upon. The notion of an able” or “writable-with” could be contained
affordance can be traced back to the Gestalt within the light. At the point where Gibson (1979)
psychologists, and particularly Koffka’s idea of claims that a letter-box affords the posting of let-
the “demand character” of an object (Koffka, ters by humans of western culture, his theory is
1935, p. 7): the most controversial. Nevertheless, Gibson
310 VISUAL PERCEPTION

claimed that the concept of affordances can perception of what it is that the picture repre-
bridge the gap that exists in more cognitive theor- sents. For example, a picture of an apple, as a flat
ies between perception and action. Within the surface, affords little apart from inspection. The
theory of affordances, perception is an invitation affordances of the object depicted, that it can be
to act, and action is an essential component of grasped, thrown, or eaten, are perceived indirectly
perception. However, Gibson’s claim that all and without ever fooling an adult observer into
aspects of perception can be understood with- actually trying to reach for and eat the picture.
out appeal to the concept of representation is Everyday perception is of the direct, not the
problematic, and we will return to this issue in indirect kind.
Chapter 14. Gibson’s final theoretical position is radical
Gibson thus asserts that optical information indeed. It stands apart from the mainstream of
specifies surfaces of support, falling-off places, perceptual theory. Some have likened Gibson’s
impending collision, and so on. And, he claims, ideas to those of the Gestaltists, who took a simi-
affordances are perceived directly, without the lar phenomenological approach to seeing. How-
need for mediation by cognitive processes. The ever, the Gestaltists were nativist in philosophy,
major task for the ecological psychologist is to while Gibson sees learning as important; and the
discover the invariant information that animals Gestaltists sought to explain perceptual phenom-
have evolved to detect, and to discover the mech- ena in terms of the physiology of the brain, unlike
anisms by which they become attuned to this Gibson. It would be as legitimate to compare
information. Gibson denies the need for memory Gibson to the behaviourists, who looked at stim-
in explaining perception. Incoming percepts are uli and responses but did not care to speculate on
not matched against previously laid-down traces, intervening stages of processing. On the other
rather the perceptual system has evolved to hand, the behaviourists saw animals as prodded
“resonate” to certain invariant information. The into action by discrete stimuli or sensations—
concept of “resonance” is left rather vague by while for Gibson, perception and action are
Gibson, but presumably it implies that there intimately interlinked. Thus Gibson’s approach is
should be neurons or neural networks sensitive unique and for many years was ignored by the
to variables of higher order than features such as vast majority of perceptual psychologists. This is
lines and edges. largely because the difference between Gibsonian
Gibson decries traditional laboratory experi- and traditional accounts of perception is more
ments in perception in which observers are pre- profound than might be appreciated from the
sented with “stimuli”, devoid of context. In such preceding pages. The differences between the two
situations the optical information is indeed approaches are not just psychological but verge
impoverished but this will only tell us how a on the philosophical. Traditional perceptual the-
human observer copes with artificially impover- ory holds that perception is indirect and mediated
ished inputs, and may tell us nothing of percep- by higher cognitive processes. We do not “just
tion in the optically rich real environment. He see” the world but actively construct it from
denies that ambiguous figures and illusions fragmentary perceptual data. Gibson is a “direct
should be the starting point for a psychology of realist”. He holds that perception is direct and
perception. While these may be interesting, and unmediated by inference and problem solving.
may be analysed in terms of the invariant infor- We will discuss some of these theoretical con-
mation that they contain, they are not character- troversies further in Chapter 14, but for now our
istic of normal perception. In the real world such aim is to describe how the relationship between
perceptual distortions are rare. Additionally, Gib- perception and action continued to be explored
son regards the perception of pictures (the focus within the approach that Gibson founded. The
of much research in perception) as involving two goal of this school of “Ecological Psychology”
components—the direct perception of the picture has been to explain how perception and action
as a picture, i.e., as a flat surface, and the indirect are linked without invoking the idea of a
 10. THE ECOLOGICAL APPROACH 311

representation of the surrounding world that This first problem is possibly compounded by the
stands between them. In order to give an outline second—that of context-conditioned variability
of the kind of alternative that it proposes, we (Turvey et al., 1982). The context in which any
need first to consider some basic questions about particular muscle contraction occurs affects the
the control of movements of the body and limbs. actual limb movement achieved. The movement
produced by a given contraction depends on the
current configuration of the parts of the limb, the
current motions of the adjoining limb segments,
THE CONTROL OF HUMAN ACTION
and the external forces against which each muscle
must work. An executive pressing the buttons, in
A theory of the control of action must explain its such a model, would have to have moment to
flexibility; the fact that animals and people are moment information available about the external
able to achieve equivalent ends in a variety of forces and the dynamic and static aspects of the
ways. Turvey (1977) gives as an example the current configurations of the limb segments.
observation that we can draw a letter A with a pen Turvey et al. (1978) liken the problem of the
on paper, with a finger on someone’s back, or push-button executive to that which an air pilot
with a toe in the sand. At least some of these would face if he or she had to control individually
activities may be novel, but are not difficult for us. each of the mechanical segments used to guide
In the same way we can recognise a letter A drawn the flight of an aeroplane. At a minimum, an
in different ways by different people (see Chapter aeroplane has two ailerons at the back of the
9). Some abstract representation of the letter wings, which can be moved up or down to control
seems to allow for the way in which we can gener- roll; two elevators on the tail, which if moved up
alise both when perceiving and when acting. or down control pitch; and a rudder at the back
Suppose then we wished to draw an A in the which can be moved left or right to control yaw
sand with a toe. Somehow the abstract represen- (cf. Figure 11.3, p. 317). There is thus one degree
tation that we wish to particularise must be trans- of freedom for each of these five hinged parts. If
lated into a specific pattern of motor (i.e., muscle) each of these parts had to be altered individually
activity. It might be possible, in principle, to con- the pilot would be faced with an impossible
ceive of each muscle involved in this action being informational load. Even if the mechanical parts
independently instructed by commands issued could only be moved to one of eight positions the
from a high level in the nervous system. Thus our control system would still have to keep track of 85
abstract conception of the letter A might be trans- (32,768) independent states.
lated into a series of independent commands to a Of course no air pilot actually has to cope with
variety of muscles in the leg, foot, and toes. This this task because the mechanical components of
kind of “push-button” metaphor for the control the guidance system are in fact linked. The ail-
of action has been criticised by Bernstein (1967) erons are yoked so that when one moves up the
whose arguments have been summarised and other moves down. The rudder is linked to the
extended by Turvey and his colleagues (Fitch, ailerons so that it moves left when the right ail-
Tuller, & Turvey, 1982; Tuller, Turvey, & Fitch, eron goes down, and the elevators on the tail sec-
1982; Turvey 1977; Turvey, Fitch, & Tuller, 1982; tion move together—both up or both down. This
Turvey, Shaw, & Mace 1978). linkage reduces the degrees of freedom to two,
There are two different, but closely associated and the guidance of the aircraft can be achieved
problems with the push-button metaphor. The with a joystick which also has two degrees of
first is known as the degrees of freedom problem. freedom (it can be moved forward or backward
An “executive” issuing independent commands to for ascent or descent and from side to side to
all those muscles involved in even the simplest of bank or turn.)
movements would have a very great deal of Turvey (1977) and Turvey et al. (1978) suggest
moment to moment computation to perform. that combinations of muscles in animals are
312 VISUAL PERCEPTION

similarly linked and constrained to act together as test models of this kind by measuring changes in
co-ordinative structures. To some extent, these can rhythmic actions, such as finger tapping, after a
function autonomously, without control from perturbation.
higher levels in the nervous system. Spinal reflexes An example of this approach can be found in
can work in this way, even though they may Kay, Saltzman, and Kelso (1991), who provide
involve quite complicated actions. For example, evidence that finger tapping is driven both by a
an animal with the upper part of the spinal cord peripheral oscillator arising from a co-ordinative
completely sectioned will still repeatedly scratch structure of arm and hand muscles and by a cen-
an itch on its body with whichever foot can most tral neural oscillator. The interaction between the
easily reach it. two oscillators appears to work in both direc-
The concept of co-ordinative structures goes tions; the peripheral system is not driven passively
beyond simple reflex acts, however, to include pat- by the central one, but can also influence its activ-
terns of interlimb co-ordination in voluntary acts. ity. This two-way interaction underlying finger
An everyday example is given by the difficulty we tapping provides one example of a more general
experience if we try to beat out two quite different argument made by Turvey et al. (1978) that motor
rhythms simultaneously with different hands. The control is not organised hierarchically, with the
hands seem constrained to act together in this executive issuing commands that pass uni-
situation. Kelso, Putnam, and Goodman (1983) directionally and without modification to lower
have demonstrated this more formally. If two levels. Rather they suggest that the system must
hands are required to make movements of differ- be organised as a heterarchy, or, more radically, as
ent difficulties and directions, the movement of a coalition. In a heterarchical organisation no one
each hand is influenced by that of the other. Such part of the system should be seen as dominating
patterns of mutual constraint and interaction the others. All levels in a heterarchy contribute
would not be expected if an “executive” equally to hypothesis testing and decision
independently commanded each muscle. Instead, making.
Turvey (1977) suggests, the executive commands An important structure that contributes to the
groups of muscles that function co-operatively organisation and control of action is the seg-
together, and so the number of degrees of free- mental apparatus of the mammalian spinal cord.
dom can be greatly reduced. The spinal cord is made up of a series of segments
Co-ordinative structures can also solve some (marked out by the vertebrae), within each of
of the problems of context-conditioned vari- which there are neuronal loops that control sim-
ability. For example, a co-ordinative structure can ple reflexes (like the knee jerk), without involving
take care of the local context in which an action any communication with the brain. Complex vol-
takes place if it behaves like a mass–spring sys- untary activities may involve the recruitment,
tem. The equilibrium point of a spring to which a modification, and elaboration of these simple
mass is attached is not affected however the mass reflexes, which form the bases for co-ordinative
is pushed or pulled. The spring always returns to structures. This may be achieved in part by tuning
rest at the same length, without any executive of the segmental apparatus prior to a movement
monitoring its movements over time. Further occurring. Turvey (1977) cites evidence from
models with similar properties have been Gurfinkel et al. (1971) in support of the notion of
developed to describe the behaviour of co- tuning. If a person is asked to flex one leg, it typ-
ordinative structures, using concepts from ically takes about 170 ms between the command
dynamical systems theory (for an introduction, and the flexion occurring. If, during this latency
see Abraham, Abraham, & Shaw, 1991). For period, the knee-jerk reflex is elicited, its ampli-
example, some systems of muscles show the prop- tude is enhanced relative to a control condition
erties of limit-cycle oscillators, tending to return where no command is present. It therefore
to a particular pattern of oscillation, rather than appears that an instruction issued from the brain
a set point, after being disturbed. It is possible to to the leg involves the preparation or tuning of
 10. THE ECOLOGICAL APPROACH 313

the segmental apparatus prior to the actual Consider someone catching a ball: as the action
movement of the leg occurring. proceeds, the ball is tracked by head and eyes, the
If we continue to consider leg movements, in arms move, and the fingers open and then close
the more complex activities of walking or run- on the ball. At each stage, the activity of co-
ning, there is evidence that the organisation of the ordinative structures is modulated by particular
segmental apparatus of the spinal cord allows the kinds of optical information. In the ecological
initiation and maintenance of stepping move- view, we should not assume that perception
ments of the limbs without sensory input. How- delivers a representation of the trajectory, speed,
ever the form of the stepping pattern must be and size of the ball, and then leaves the rest to a
tailored to the external forces. This can be motor system.
achieved by using afferent information obtained Evidence in support of the interdependence of
from reflex structures and also by tuning the seg- perception and motor tasks comes from experi-
mental apparatus on the detection of relevant ments showing that people use visual information
information obtained primarily through vision. differently when they are performing different
In this way a basic pattern of activity can be actions. For example, Bootsma (1989) compared
attuned to the current contextual demands. the behaviour of people hitting a falling ball with
a bat, closing a switch to release an artificial arm
Implications for the role of perception holding a bat, and closing a switch to signal when
The traditional metaphor of a central executive the ball passed a marker. From a traditional point
controlling action in a push-button fashion com- of view, all these tasks seem equivalent: in each
plements the view that perception acts to provide one, the subject must detect when the ball reaches
a representation of the surrounding world for the a particular point and then execute some action.
central executive to use in making decisions. Hav- From an ecological perspective, however, the
ing discussed the objections to such metaphors, different actions involved may use visual infor-
what does the ecological approach outlined above mation in different ways. The results gave some
imply about the role of perception? Turvey and support to the ecological view, as the timing of
his colleagues argue that visual information of the natural batting movements was less variable
particular kinds must be injected into unfolding than that of the switch-closing movements. The
activities at appropriate points, after which the implication is that the control of the natural
co-ordinative structures that have been activated perception–action coupling is more accurate than
and tuned can take care of themselves to a large that of an apparently “simpler” movement.
extent. They devolve the responsibility for these The most progress towards a detailed under-
“injections” of visual information to the co- standing of the links between perception and co-
ordinative structures themselves (Fitch et al., ordinative structures has been made in cases of
1982, p. 272): rhythmic movements. Schmidt, Carello, and Tur-
vey (1990) studied the synchronisation of limb
We do not want a model in which the brain movements between two people, by asking pairs
interprets the perceptual information, of participants to sit with their legs crossed, in
decides what portion of the information to view of each other. They were then both asked to
supply a given coordinative structure, and swing one lower leg up and down, keeping its
when to supply it. Instead, the organisation movement in time with the other person’s leg
of the coordinative structure should be movement. The interesting result from these
such as to accept only certain information experiments was that the coupled oscillations
at certain times. of the two legs of different people behaved in
the same way as those of two fingers of the same
An implication of the ecological approach is person (Kelso et al., 1983; see earlier). Visual
that the information obtained through vision is information specifying the movement of the other
not independent of the motor activity it controls. person’s leg therefore seems to become coupled to
314 VISUAL PERCEPTION

the observer’s leg rhythm in the same way that control a multi-jointed model person or limb
different motor systems become coupled. without any explicit coding of movement param-
eters. A successful example is Hinton’s (1984)
model of reaching forwards without losing one’s
balance, using a simple two-dimensional model
CONCLUSIONS
person. The problem is to avoid swinging the arm
out in a way that shifts the centre of gravity of the
The analysis of the control of human action that person as a whole to an unstable position—to
we have discussed in this chapter is an important avoid this, other limb and trunk movements must
extension of Gibson’s theory, and represents the occur to compensate. Hinton found that a con-
achievements of ecological theorists in develop- nectionist model could satisfy these two con-
ing tools for understanding the mutual coupling straints (touch the object, while maintaining
of perception and action. The approach aims to centre of gravity above the foot) simultaneously,
understand the structure of action, and its modu- and that the solution was much more elegant
lation by environmental information, in terms of when combinations of joint angles were adjusted
the behaviour of dynamical systems governed by synergistically (cf. our earlier discussions of co-
physical laws, rather than the behaviour of a ordinative structures).
“central executive” controlling the muscles (see An important difference between the ecologi-
Kelso, 1995). The aim is clearly an ambitious one, cal approach and other approaches to under-
and the progress made has so far been on a few standing the control of action is that it treats
specific problems such as the control of rhythmic perception and action as interlocked processes
limb movements and the coupling of the under- and tries to avoid separating out motor control
lying dynamics to optical information. as a distinct problem. Although it is possible to
The problems of the degrees of freedom in imagine how this principle might be carried
motor control, and of context-conditioned vari- through into more detailed models of those
ability, have been tackled from other theoretical aspects of perception involved in the moment-by-
perspectives, and the reader may be interested in moment control of movements such as reaching,
comparing these with the ecological approach walking, or jumping, it does not seem directly
sketched here. These include the analysis of the relevant to understanding how perception yields
kinematics (trajectory and speed) of human awareness. In particular, it is difficult to see how
movements to discover underlying invariants the principle is relevant when we acquire know-
(e.g., Lacquaniti, 1989), and use of single-cell ledge of the world through perception that is used
recording methods to determine how parameters to guide action at a much later time—or indeed
of movement are coded in the activity of single might never be used in this way at all. We will
cells in the spinal cord and brain (e.g., Bizzi, return to these problems in Chapter 14, but in the
Mussa-Ivaldi, & Giszter, 1991; Fetz, 1992; Geor- following three chapters we will describe research
gopoulos, 1991). Connectionist modelling tech- on the visual control of movement that has been
niques (see Chapter 4) have also been applied to carried out both inside and outside the ecological
the problem, attempting to train a network to school that Gibson founded.